Sound reproduction method, computer program, and sound reproduction device

Abstract

This acoustic reproduction method includes: an acquisition step for acquiring first region information that indicates a first region (A1) in which an acoustic image of a first sound is positioned, and direction information that indicates a direction (D) in which the head of a listener (L) is oriented; a determination step for acquiring second region information that indicates a second region (A2) in which an acoustic image of a second sound is positioned within a sound reproduction space and determining, on the basis of the direction information, whether a first direction (D1) in which the first sound reaches the listener (L) and a second direction (D2) in which the second sound reaches the listener (L) have plane symmetry with respect to a prescribed plane (S) serving as a plane of symmetry, where the prescribed plane (S) is a plane passing through both ears of the listener (L); a processing step for implementing, on sound information, a modification process in which the second direction (D2) is modified when it is determined that the first direction (D1) and the second direction (D2) have plane symmetry; and an output step for outputting the sound information on which the modification process was implemented.

Description

Technical Field

[0007] , The method includes: an acquisition step of acquiring information indicating multiple sounds reaching a listener in a sound reproduction space, and direction information indicating the direction the listener's head is facing; a determination step of determining whether the first direction in which a first sound included in the multiple sounds indicated by the acquired information reaches the listener, and the second direction in which a second sound included in the multiple sounds indicated by the acquired information reaches the listener, are symmetrical with respect to the predetermined plane, assuming that the predetermined plane is a plane perpendicular to the direction the listener's head is facing and passing through both of the listener's ears; a processing step of applying a modification process to the sound information indicating the second sound to change the second direction if it is determined that the first direction and the second direction are symmetrical with respect to the predetermined plane; and an output step of outputting the sound information to which the modification process has been applied. ,

[0006] , , , , ,

[0008] , , ,

[0009] , , The system includes: an acquisition unit that acquires information indicating multiple sounds reaching a listener in a sound reproduction space, and direction information indicating the direction the listener's head is facing; a determination unit that determines whether the first direction in which a first sound included in the multiple sounds indicated by the acquired information reaches the listener, and the second direction in which a second sound included in the multiple sounds indicated by the acquired information reaches the listener, are symmetrical with respect to the predetermined plane, assuming that the predetermined plane is a plane perpendicular to the direction the listener's head is facing and passing through both of the listener's ears; a processing unit that applies a modification process to the sound information indicating the second sound to change the second direction if it is determined that the first direction and the second direction are symmetrical with respect to the predetermined plane; and an output unit that outputs the sound information to which the modification process has been applied. ,

[0001] The present disclosure relates to an acoustic reproduction method and the like.

Background Art

[0002] In Patent Document 1, an acoustic reproduction method is disclosed in which one or more first sounds (acoustic objects) and one or more other second sounds (acoustic objects) are respectively subjected to processing (rendering) according to a first rule and a second rule.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, it may be difficult for a listener to accurately perceive two sounds reaching the listener.

[0005] Therefore, an object of the present disclosure is to provide an acoustic reproduction method and the like that make it easier for a listener to accurately perceive two sounds reaching the listener.

Means for Solving the Problems

[0006] An acoustic reproduction method according to an aspect of the present disclosure is The method includes: an acquisition step of acquiring information indicating multiple sounds reaching a listener in a sound reproduction space, and direction information indicating the direction the listener's head is facing; a determination step of determining whether the first direction in which a first sound included in the multiple sounds indicated by the acquired information reaches the listener, and the second direction in which a second sound included in the multiple sounds indicated by the acquired information reaches the listener, are symmetrical with respect to the predetermined plane, assuming that the predetermined plane is a plane perpendicular to the direction the listener's head is facing and passing through both of the listener's ears; a processing step of applying a modification process to the sound information indicating the second sound to change the second direction if it is determined that the first direction and the second direction are symmetrical with respect to the predetermined plane; and an output step of outputting the sound information to which the modification process has been applied.

[0007] A program according to an aspect of the present disclosure causes a computer to execute the above acoustic reproduction method.

[0008] An acoustic reproduction device according to an aspect of the present disclosure is The system includes: an acquisition unit that acquires information indicating multiple sounds reaching a listener in a sound reproduction space, and direction information indicating the direction the listener's head is facing; a determination unit that determines whether the first direction in which a first sound included in the multiple sounds indicated by the acquired information reaches the listener, and the second direction in which a second sound included in the multiple sounds indicated by the acquired information reaches the listener, are symmetrical with respect to the predetermined plane, assuming that the predetermined plane is a plane perpendicular to the direction the listener's head is facing and passing through both of the listener's ears; a processing unit that applies a modification process to the sound information indicating the second sound to change the second direction if it is determined that the first direction and the second direction are symmetrical with respect to the predetermined plane; and an output unit that outputs the sound information to which the modification process has been applied.

[0009] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0010] According to one aspect of the present disclosure, the sound reproduction method makes it easier for the listener to accurately perceive the two sounds that reach them. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing the functional configuration of an audio playback device according to Embodiment 1. [Figure 2] Figure 2 is a schematic diagram showing the sound reproduction space according to Embodiment 1. [Figure 3] Figure 3 is a flowchart showing an example of the operation of the sound reproduction device according to Embodiment 1. [Figure 4] Figure 4 is a schematic diagram illustrating the second sound in the sound reproduction space according to Embodiment 1. [Figure 5] Figure 5 is a schematic diagram showing the sound reproduction space after the second sound information has been modified. [Figure 6] Figure 6 is a schematic diagram showing an example of the sound reproduction space when the direction the listener's head is facing changes, according to Embodiment 1. [Figure 7] Figure 7 is a schematic diagram showing another example of the sound reproduction space when the direction the listener's head is facing changes, according to Embodiment 1. [Figure 8] Figure 8 is a block diagram showing the functional configuration of the sound reproduction device according to Embodiment 2. [Figure 9] Figure 9 is a flowchart showing an example of the operation of the sound reproduction device according to Embodiment 2. [Figure 10] Figure 10 is a schematic diagram illustrating the second sound in the sound reproduction space according to Embodiment 2. [Figure 11] FIG. 11 is a schematic diagram showing a sound reproduction space after the second sound information has been subjected to change processing.

Embodiments for Carrying Out the Invention

[0012] (Findings on which the present disclosure is based) Conventionally, there has been known an acoustic reproduction method in which, by performing processing according to different rules on each of a plurality of sounds, a listener can hear, that is, perceive, the plurality of sounds.

[0013] For example, Patent Document 1 discloses the following acoustic reproduction method. In this acoustic reproduction method, a plurality of sounds are classified so as to belong to a first group or a second group according to the actions of the listener. Further, processing according to a first rule is performed on one or more first sounds belonging to the first group, and processing according to a second rule is performed on one or more second sounds belonging to the second group.

[0014] For example, the first rule defines changing the intensity of the first sound and changing the distance between the first sound and the listener, and the second rule defines changing the intensity of the second sound and changing the distance between the second sound and the listener.

[0015] According to such an acoustic reproduction method, a listener can perceive two sounds (the first sound and the second sound).

[0016] However, in the acoustic reproduction method disclosed in Patent Document 1, there are cases where it is difficult for the listener to accurately perceive the two sounds reaching the listener. This will be described below.

[0017] The predetermined plane is defined as a plane perpendicular to the direction the listener's head is facing and passing through the listener. When this predetermined plane is the plane of symmetry, if the first direction in which the first sound reaches the listener and the second direction in which the second sound reaches the listener are symmetrical, it becomes difficult for the listener to accurately perceive the two sounds reaching them. In other words, the case where the first direction and the second direction are symmetrical is also the case where the angle between the direction in which the first sound reaches the listener (first direction) and the predetermined plane is equal to the angle between the direction in which the second sound reaches the listener (second direction) and the predetermined plane.

[0018] In such cases, when a listener hears the first and second tones, problems arise such as the first and second tones sounding like they are coming from the same direction, meaning the listener cannot accurately perceive the first and second tones. This problem arises because humans have perceptual characteristics (more specifically, auditory characteristics) that stem from the shape of the auricle and the discrimination limit.

[0019] In such cases, even if the intensity of the first and second sounds is modified as shown in the sound reproduction method disclosed in Patent Document 1, the listener will still perceive the first and second sounds as coming from the same direction. Therefore, there is a need for sound reproduction methods that make it easier for the listener to accurately perceive the two sounds reaching them.

[0020] Therefore, an acoustic reproduction method according to one aspect of the present disclosure includes: an acquisition step of acquiring first region information indicating a first region in which the sound image of a first sound, which is an object sound reaching a listener in a sound reproduction space, is localized, and direction information indicating the direction in which the listener's head is facing; a determination step of acquiring second region information indicating a second region in which the sound image of a second sound reaching the listener in the sound reproduction space is localized, assuming that a predetermined plane is a plane perpendicular to the direction in which the listener's head is facing and passing through both of the listener's ears, and determining whether the first direction in which the first sound reaches the listener and the second direction in which the second sound reaches the listener are symmetrical in plane with respect to the predetermined plane, based on the acquired direction information; a processing step of acquiring sound information indicating the second sound, and applying a modification process to the acquired sound information such that the second direction is changed so that the first direction and the second direction are not symmetrical in plane, if it is determined that the first direction and the second direction are symmetrical in plane; and an output step of outputting the sound information to which the modification process has been applied.

[0021] As a result, the first and second directions are not symmetrical. Furthermore, the angle between the direction in which the first sound reaches the listener (first direction) and the predetermined plane is different from the angle between the direction in which the second sound reaches the listener (second direction) and the predetermined plane. Therefore, even if the listener hears both the first and second sounds, the above-mentioned problems, such as the first and second sounds sounding like they are coming from the same direction, are suppressed, allowing the listener to accurately perceive both the first and second sounds. In other words, an acoustic reproduction method is realized that makes it easier for the listener to accurately perceive the two sounds reaching them.

[0022] Furthermore, for example, the second sound may be an object sound different from the first sound, and the sound reproduction method may include an extraction step of acquiring audio content information and extracting the first region information, second region information, and sound information contained in the acquired audio content information, wherein the acquisition step may acquire the extracted first region information, the determination step may acquire the extracted second region information, and the processing step may acquire the extracted sound information.

[0023] This enables an acoustic reproduction method in which, even when the second sound is a different object sound from the first sound, the listener can more easily perceive the two sounds that reach them accurately.

[0024] Furthermore, for example, the acquisition step may involve acquiring spatial information indicating the shape of the sound reproduction space, and the sound reproduction method may include a determination step in which, based on the acquired first region information and the acquired spatial information, the sound image of the second sound, which is a reflected sound of the first sound, is localized in the second region, the determination step may involve acquiring second region information indicating the determined second region, and the processing step may involve acquiring sound information indicating the first sound as sound information indicating the second sound.

[0025] This enables an acoustic reproduction method in which, even when the second sound is a reflection of the first sound, the listener can more easily perceive the two sounds that reach them accurately.

[0026] Furthermore, for example, in the processing step, the modification process may be performed such that the second direction is shifted so that at least one of the interaural intensity difference of the second sound and the interaural time difference of the second sound becomes larger.

[0027] Thus, by shifting the direction in which the second sound reaches the listener (second direction), the angle between the direction in which the first sound reaches the listener (first direction) and a predetermined plane is different from the angle between the direction in which the second sound reaches the listener (second direction) and the predetermined plane. Furthermore, as the interaural intensity difference of the second sound increases, the listener becomes more perceptible to the direction in which the second sound reaches the listener (second direction). Similarly, as the interaural time difference of the second sound increases, the listener becomes more perceptible to the direction in which the second sound reaches the listener (second direction). Therefore, by shifting the direction in which the second sound reaches the listener (second direction) such that at least one of the interaural intensity difference and the interaural time difference increases, an acoustic reproduction method is realized in which the listener becomes more able to perceive the two sounds reaching the listener more accurately.

[0028] Furthermore, a computer program relating to one aspect of this disclosure is a computer program that causes a computer to execute the above-described sound reproduction method.

[0029] This allows the computer to perform the above sound playback method according to the program.

[0030] Furthermore, an acoustic reproduction device according to one aspect of the present disclosure includes: an acquisition unit that acquires first region information indicating a first region in which the sound image of a first sound, which is an object sound reaching a listener in a sound reproduction space, is localized, and direction information indicating the direction in which the listener's head is facing; a determination unit that, when a predetermined plane is defined as a plane perpendicular to the direction in which the listener's head is facing and passing through both of the listener's ears, acquires second region information indicating a second region in which the sound image of a second sound reaching the listener in the sound reproduction space is localized, and determines, based on the acquired direction information, whether the first direction in which the first sound reaches the listener and the second direction in which the second sound reaches the listener are symmetrical with respect to the predetermined plane; a processing unit that, when it is determined that the first direction and the second direction are symmetrical with respect to

[0031] As a result, the first and second directions are not symmetrical. Furthermore, the angle between the direction in which the first sound reaches the listener (first direction) and the predetermined plane is different from the angle between the direction in which the second sound reaches the listener (second direction) and the predetermined plane. Therefore, even if the listener hears both the first and second sounds, the occurrence of the above-mentioned problems, such as the first and second sounds sounding like they are coming from the same direction, is suppressed, allowing the listener to accurately perceive both the first and second sounds. In other words, an acoustic reproduction device is realized that makes it easier for the listener to accurately perceive the two sounds reaching them.

[0032] Furthermore, these comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0033] The embodiments will be described in detail below with reference to the drawings.

[0034] The embodiments described below are all general or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the scope of the claims.

[0035] Furthermore, in the following explanation, elements may be assigned ordinal numbers such as the first and second. These ordinal numbers are assigned to identify the elements and do not necessarily correspond to a meaningful order. These ordinal numbers may be rearranged, newly assigned, or removed as appropriate.

[0036] Furthermore, each figure is a schematic diagram and not necessarily a strictly accurate representation. Therefore, the scale and other aspects may not necessarily be consistent across all figures. In each figure, substantially identical components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0037] In this specification, terms and numerical ranges indicating relationships between elements, such as parallel or perpendicular, do not represent only strict meanings, but also include substantially equivalent ranges, such as differences of a few percent.

[0038] (Embodiment 1) [composition] First, the configuration of the sound reproduction device 100 according to Embodiment 1 will be described. Figure 1 is a block diagram showing the functional configuration of the sound reproduction device 100 according to this embodiment. Figure 2 is a schematic diagram showing the sound reproduction space according to this embodiment.

[0039] The sound reproduction device 100 according to this embodiment processes sound information indicating a first sound and sound information indicating a second sound, and outputs them to headphones 200 worn by the listener L, thereby allowing the listener L to hear the first sound and the second sound. More specifically, the sound reproduction device 100 is a stereophonic sound reproduction device that allows the listener L to hear stereophonic sound. The sound reproduction device 100 according to this embodiment is a device that can be applied to various applications such as virtual reality or augmented reality (VR / AR) as an example.

[0040] Figure 2 shows the first sound, which is an object sound reaching the listener L in the sound reproduction space. More specifically, Figure 2 is a view of the sound reproduction space from above the listener L toward the listener L, that is, from above the listener L's head toward the listener L along a vertical downward direction. In this embodiment, the sound reproduction space refers to the virtual reality space or augmented reality space used in various applications such as virtual reality or augmented reality (VR / AR).

[0041] In the sound reproduction space shown in Figure 2, 0 o'clock, 3 o'clock, and 9 o'clock are indicated to explain direction, corresponding to the time shown on the clock face. The white arrows indicate the direction D that the listener L's head is pointing. In Figure 2, the direction D that the listener L's head is pointing towards, located at the center (also called the origin) of the clock face, is the direction of 0 o'clock. Hereafter, the direction connecting the listener L and 0 o'clock may be referred to as the "direction of 0 o'clock," and the same applies to the other times indicated on the clock face.

[0042] Now, the first sound according to this embodiment will be described.

[0043] As shown in Figure 2, the sound image of the first sound, which is an object sound, is localized in the first region A1. In other words, the first sound is the sound that reaches the listener L from the first region A1 in the sound reproduction space. Also, as shown in Figure 2, the first sound is the sound that reaches the listener L from the first region A1 when viewed vertically downwards from above the listener L's head towards the listener L. The first region A1 is shown as a black dot in Figure 2. The direction in which the first sound reaches the listener L is the first direction D1.

[0044] Furthermore, the predetermined plane S is defined as a plane perpendicular to the direction D in which the listener L's head is facing, and which passes through both of the listener L's ears. In this embodiment, the predetermined plane S is a plane perpendicular to the above-mentioned direction D (more specifically, a plane parallel to the vertical direction) and which passes through both of the listener L's ears. The predetermined plane S can also be said to be the coronal plane of the listener L. In Figure 2, since the direction D in which the listener L's head is facing is the 12 o'clock direction, the predetermined plane S is shown by dashed lines extending in the 3 o'clock and 9 o'clock directions.

[0045] The second note, which is not shown in Figure 2, will be discussed later.

[0046] Next, I will explain the headphones 200.

[0047] As shown in Figure 1, the headphones 200 are an audio output device comprising a head sensor unit 201 and a second output unit 202.

[0048] The head sensor unit 201 senses the direction D in which the listener L's head is facing and outputs directional information indicating the direction D in which the listener L's head is facing to the sound reproduction device 100. Note that the direction D in which the listener L's head is facing is also the direction in which the listener L's face is facing.

[0049] The head sensor unit 201 may sense 6DoF (Degrees of Freedom) information of the listener L's head. For example, the head sensor unit 201 may be an inertial measurement unit (IMU), an accelerometer, a gyroscope, a magnetic sensor, or a combination thereof.

[0050] Furthermore, if we define the direction D in which the listener L's head is facing as the front of the listener L, and the direction opposite to the front as the rear of the listener L, then, as shown in Figure 2, in this embodiment, the first sound is a sound that reaches the listener L from the front of the listener L.

[0051] The second output unit 202 is a device for reproducing the first and second tones. More specifically, the second output unit 202 reproduces the first and second tones based on the sound information indicating the first tone and the sound information indicating the second tone, which have been processed by and output from the sound reproduction device 100. In the following, the sound information indicating the first tone may be referred to as the first tone information, and the sound information indicating the second tone may be referred to as the second tone information.

[0052] Next, we will explain the sound reproduction device 100 shown in Figure 1.

[0053] As shown in Figure 1, the sound reproduction device 100 includes an extraction unit 110, an information processing unit 120, a convolution processing unit 130, and a first output unit 140.

[0054] The extraction unit 110 acquires audio content information and extracts predetermined information contained in the acquired audio content information. The extraction unit 110 acquires audio content information from a storage device (not shown) outside the sound playback device 100, for example. Alternatively, the extraction unit 110 may acquire audio content information stored in a storage device (not shown) provided by the sound playback device 100 itself. The extraction unit 110 includes a region information extraction unit 111, a spatial information extraction unit 112, and a sound information extraction unit 113.

[0055] The region information extraction unit 111 extracts first region information contained in the acquired audio content information. The first region information is information indicating the first region A1 in which the sound image of the first sound is localized. More specifically, the first region information is information indicating the position of the first region A1 in the sound playback space.

[0056] The spatial information extraction unit 112 extracts spatial information contained in the acquired audio content information. Spatial information refers to information that indicates the shape of the sound playback space, and more specifically, it refers to information that indicates the installation position and shape of objects (walls, doors, floors, ceilings, furniture, etc.) in the sound playback space. The spatial information also includes information that indicates which frequencies and to what extent the objects reflect sound.

[0057] The sound information extraction unit 113 extracts the first sound information contained in the acquired audio content information. The first sound information is information that indicates the first sound, which is an object sound, and is digital data expressed in formats such as WAVE, MP3, or WMA.

[0058] Thus, in this embodiment, the audio content information includes first domain information, first sound information, and spatial information.

[0059] Furthermore, the audio content information should preferably be encoded using a format such as MPEG-H 3D Audio (ISO / IEC 23008-3) (hereinafter referred to as MPEG-H 3D Audio). In other words, the extraction unit 110 acquires the audio content information, which is an encoded bitstream. The extraction unit 110 acquires the audio content information and decodes it. The extraction unit 110 performs the decoding process based on the above-mentioned MPEG-H 3D Audio, etc. In other words, for example, the extraction unit 110 functions as a decoder.

[0060] The information processing unit 120 determines the positional relationship between the first region A1 where the sound image of the first sound is localized and the second region where the sound image of the second sound is localized, based on the first region information, spatial information, and directional information. The information processing unit 120 includes an acquisition unit 121, a determination unit 122, and a judgment unit 123.

[0061] The acquisition unit 121 acquires the first region information and spatial information extracted by the extraction unit 110. More specifically, it acquires the first region information extracted by the region information extraction unit 111 and the spatial information extracted by the spatial information extraction unit 112. The acquisition unit 121 also acquires directional information sensed by the headphones 200 (more specifically, the head sensor unit 201).

[0062] The determination unit 122 determines the second region in which the sound image of the second sound is localized, based on the acquired first region information and spatial information.

[0063] Now, let's explain the second sound. For example, if there are objects installed in the sound reproduction space, the first sound reaches the listener L directly and also reaches the listener L after being reflected by the installed object. In this embodiment, the second sound is the first sound that reaches the listener L after being reflected by the installed object; in other words, it is the reflected sound of the first sound. The direction in which the second sound reaches the listener L is the second direction.

[0064] The determination unit 122 determines whether or not there is a reflected sound of the first sound (second sound) based on the first region A1 indicated by the first region information and the installation position and shape of the installed object indicated by the spatial information. If there is a second sound, it determines the second region in which the sound image of the second sound is localized. Furthermore, the determination unit 122 outputs second region information indicating the determined second region to the judgment unit 123. The second region information is information indicating the position of the second region in the sound reproduction space.

[0065] The determination unit 123 acquires second region information indicating the second region in the sound reproduction space where the sound image of the second sound reaching the listener L is localized. In this embodiment, the determination unit 123 acquires second region information indicating the second region determined by the decision unit 122. Furthermore, based on the direction information acquired by the acquisition unit 121, the determination unit 123 determines whether the first direction D1 in which the first sound reaches the listener L and the second direction in which the second sound reaches the listener L are symmetrical with respect to a predetermined plane S. Furthermore, the determination unit 123 outputs the result of the determination to the convolution processing unit 130.

[0066] The convolution processing unit 130 processes the sound information indicating the first tone (first tone information) and the sound information indicating the second tone (second tone information) based on the result determined by the judgment unit 123. The convolution processing unit 130 includes a first tone processing unit 131, a second tone processing unit 132, and an HRTF (Head-Related Transfer Function) storage unit 133.

[0067] The first sound processing unit 131 processes the first sound information by referring to the head-related transfer function stored in the HRTF memory unit 133. More specifically, the first sound processing unit 131 convolves the head-related transfer function into the first sound information so that the first sound reaches the listener L from the first region A1 indicated by the first region information acquired by the acquisition unit 121. The first sound processing unit 131 acquires the first sound information extracted from the audio content information by the sound information extraction unit 113 of the extraction unit 110 and applies the above processing to the acquired first sound information.

[0068] The second sound processing unit 132 is an example of a processing unit that processes the second sound information by referring to the head-related transfer function stored in the HRTF memory unit 133. More specifically, the second sound processing unit 132 performs a process of convolving the head-related transfer function into the second sound information so that the second sound reaches the listener L from the second region determined by the determination unit 122. As described above, the second sound is a reflection of the first sound. Therefore, the second sound processing unit 132 acquires the first sound information extracted from the audio content information by the sound information extraction unit 113 of the extraction unit 110 as the second sound information, and performs the above processing on the acquired second sound information.

[0069] If the determination unit 123 determines that the first direction D1 and the second direction are symmetrical in plane, the second sound processing unit 132 performs the following processing. The second sound processing unit 132 acquires sound information indicating the second sound (second sound information) and applies a modification process to the acquired second sound information so that the second direction in which the second sound reaches the listener L is changed so that the first direction D1 and the second direction are not symmetrical in plane. In other words, in this case, the second sound processing unit 132 performs a process to convolve the head-related transfer function into the second sound information so that the second region is changed and the second direction in which the second sound reaches the listener L is changed.

[0070] The HRTF memory unit 133 is a memory device used by the first sound processing unit 131 and the second sound processing unit 132, in which head-level transfer functions are stored.

[0071] The first sound information processed by the first sound processing unit 131 is output to the first output unit 140. Similarly, the second sound information processed by the second sound processing unit 132 is output to the first output unit 140.

[0072] The first output unit 140 is an example of an output unit, and acquires the outputted first sound information and second sound information, and outputs the acquired first sound information and second sound information to the headphones 200. In this embodiment, the first output unit 140 mixes the acquired first sound information and acquired second sound information, and outputs the mixed first sound information and second sound information to the headphones 200.

[0073] Furthermore, if the determination unit 123 determines that the first direction D1 and the second direction are not symmetrical in plane, the first output unit 140 acquires the processed first sound information and the processed second sound information. Also, if the determination unit 123 determines that the first direction D1 and the second direction are symmetrical in plane, the first output unit 140 acquires the processed first sound information and the second sound information to which the above modification processing has been applied.

[0074] Furthermore, the second output unit 202 of the headphones 200 reproduces the first and second tones based on the first and second tones output by the first output unit 140.

[0075] In this way, the information processing unit 120, the convolution processing unit 130, and the first output unit 140 output first and second sound information that can be played back by the headphones 200, based on the information extracted by the extraction unit 110. In other words, for example, the information processing unit 120, the convolution processing unit 130, and the first output unit 140 function as a renderer.

[0076] [Example of operation] The following describes an example of the operation of the sound reproduction method performed by the sound reproduction device 100. Figure 3 is a flowchart of an example of the operation of the sound reproduction device 100 according to this embodiment.

[0077] First, the extraction unit 110 acquires audio content information (S10).

[0078] The extraction unit 110 extracts first region information and first sound information related to the first sound, as well as spatial information, from the acquired audio content information (S20). More specifically, the region information extraction unit 111 extracts the first region information contained in the audio content information. The spatial information extraction unit 112 extracts the spatial information contained in the audio content information. The sound information extraction unit 113 extracts the first sound information contained in the audio content information. The extraction unit 110 outputs the extracted first region information, first sound information, and spatial information.

[0079] Furthermore, the information processing unit 120 acquires first region information, direction information, and spatial information indicating the first region A1 (S30). More specifically, the acquisition unit 121 of the information processing unit 120 acquires the first region information and spatial information output from the extraction unit 110, and the direction information output from the head sensor unit 201 of the headphones 200. This step S30 corresponds to the acquisition step.

[0080] Next, the determination unit 122 determines the second region where the sound image of the second sound, which is a reflected sound, is localized, based on the acquired first region information and spatial information (S40). This step S40 corresponds to the determination step.

[0081] Here, the process in step S40 will be explained in more detail using Figure 4.

[0082] Figure 4 is a schematic diagram illustrating the second sound in the sound reproduction space according to this embodiment. Note that, like Figure 2, Figure 4 is a view taken from above the listener L's head, vertically downwards, toward the listener L, and the same applies to Figures 5-7, 10, and 11, which will be described later.

[0083] As described above, the second sound in this embodiment is a reflected sound of the first sound. The determination unit 122 determines whether or not there is a reflected sound of the first sound (second sound) based on the first region A1 indicated by the first region information and the installation position and shape of the installed object indicated by the spatial information. Here, it is preferable that the first region A1 and the installation position and shape of the installed object are indicated by coordinate positions such as the x axis, y axis and z axis.

[0084] Figure 4 shows a wall W, which is an example of an installed object. In this case, the first sound reflects off the wall W and reaches the listener L, so it is determined that there is a second sound, which is a reflected sound of the first sound. Furthermore, if there is a second sound, the determination unit 122 determines the second region A2 in which the sound image of the second sound is localized, based on the acquired first region information and spatial information.

[0085] As described above, in this embodiment, the first sound is a sound that reaches the listener L from the front. Furthermore, the second sound is a reflected sound that reaches the listener L from the rear.

[0086] As shown in Figure 4, the second sound, which is a reflection of the first sound, is the sound that reaches the listener L from the second region A2 in the sound reproduction space. Furthermore, as shown in Figure 4, the second sound is the sound that reaches the listener L from the second region A2 when viewed vertically downwards from above the listener L's head towards the listener L. In Figure 4, the second region A2 is shown as a black dot, indicating the direction in which the second sound reaches the listener L (second direction D2).

[0087] Furthermore, the determination unit 122 outputs second region information indicating the determined second region A2 to the judgment unit 123.

[0088] The judgment unit 123 acquires second region information indicating the second region A2 in which the sound image of the second sound reaching the listener L is localized in the sound reproduction space (S50). More specifically, the judgment unit 123 acquires second region information indicating the second region A2 determined by the decision unit 122.

[0089] Furthermore, the determination unit 123 determines, based on the direction information acquired by the acquisition unit 121, whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to a predetermined plane S (S60). This step S60 corresponds to the determination step.

[0090] In this embodiment, the distance between the listener L and the first region A1 is the same as the distance between the listener L and the second region A2. Therefore, the case where the first direction D1 and the second direction D2 are symmetrical in plane corresponds to the case where the second region A2 is located at a plane-symmetric position of the first region A1.

[0091] Here, the process of step S60 will be explained in more detail using Figure 4.

[0092] In step S60, since directional information has already been acquired, it is clear how the predetermined plane S, which is a plane perpendicular to the direction D in which the listener L's head is facing and passes through the listener L, is located in the sound reproduction space. For example, in Figure 4, since the direction D in which the listener L's head is facing is the 12 o'clock direction, the predetermined plane S extends in the 3 o'clock and 9 o'clock directions, and it is desirable that the coordinate positions of the x, y, and z axes of the predetermined plane S are clear.

[0093] Furthermore, it is preferable that the already acquired first-region information and second-region information each indicate the position of the first-region A1 using coordinates such as the x, y, and z axes, and the position of the second-region A2 using coordinates such as the x, y, and z axes.

[0094] Based on this information, the determination unit 123 determines whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to a predetermined plane S. Furthermore, the determination unit 123 outputs the result of its determination to the convolution processing unit 130. The convolution processing unit 130 acquires the result determined by the determination unit 123.

[0095] In Figure 4, θ1 is shown as the angle between the first direction D1, from which the first sound reaches the listener L, and the predetermined plane S (hereinafter sometimes referred to as the first angle). Also, θ2 is shown as the angle between the second direction D2, from which the second sound reaches the listener L, and the predetermined plane S (hereinafter sometimes referred to as the second angle). If the first direction D1 and the second direction D2 are symmetrical, the first angle (θ1) is equal to the second angle (θ2).

[0096] As explained in (Knowledge on which this disclosure is based), when the first direction D1 and the second direction D2 are symmetrical, it becomes difficult for the listener L to accurately perceive the two sounds (here, the first sound and the second sound) that reach the listener L. More specifically, when the listener L hears the first sound and the second sound, the first sound and the second sound appear to come from the same direction. For example, the first sound and the second sound are heard in such a way that both the sound image of the first sound and the sound image of the second sound reach the listener L from the first direction D1. As a result, to the listener L, it sounds as if the reflected sound of the second sound has disappeared. In other words, in such a case, the listener L cannot accurately perceive the first sound and the second sound.

[0097] In this embodiment, the distance between the listener L and the first region A1 is the same as the distance between the listener L and the second region A2, but this is not limited to this. In other words, even if the distance between the listener L and the first region A1 is different from the distance between the listener L and the second region A2, the problem described in (the knowledge on which this disclosure is based) will occur if the first direction D1 and the second direction D2 are symmetrical in plane.

[0098] Let's explain the operation example again using Figure 3.

[0099] First, we will explain the case where the determination unit 123 determines that the first direction D1 and the second direction D2 are symmetrical in plane (Yes in S60). Here, as an example, we will explain the case where the first angle and the second angle satisfy θ1 = θ2 = 80°.

[0100] In this case, the convolution processing unit 130 (second tone processing unit 132) acquires second tone information indicating the second tone and performs the following processing. The convolution processing unit 130 (second tone processing unit 132) applies a modification process (modification process) to the acquired second tone information so that the second direction D2, to which the second tone reaches the listener L, is changed so that the first direction D1 and the second direction D2 are not symmetrical (S70). At this time, the convolution processing unit 130 (first tone processing unit 131) also processes the first tone information. More specifically, the first tone processing unit 131 performs a process to convolve the head-related transfer function into the first tone information so that the first tone reaches the listener L from the first region A1. The convolution processing unit 130 outputs the processed first tone information and the modified second tone information to the first output unit 140. This step S70 corresponds to the processing step.

[0101] Furthermore, the first output unit 140 outputs the second sound information, which has been modified and output by the convolution processing unit 130, to the headphones 200 (S80). More specifically, the first output unit 140 mixes the first sound information and the second sound information output by the convolution processing unit 130 and outputs the mixed first sound information and second sound information to the headphones 200. This step S80 corresponds to the output step.

[0102] Then, the second output unit 202 of the headphones 200 reproduces the first and second tones based on the first and second tones output by the first output unit 140.

[0103] Here, the sound reaching the listener L in the sound reproduction space as a result of the operations in steps S70 and S80 will be explained in more detail using Figure 5.

[0104] Figure 5 is a schematic diagram showing the sound reproduction space after the second sound information has been modified.

[0105] The modification process changes the region in which the sound image of the second sound is localized, from the second region A2 shown in Figure 4 to the second region A21 shown in Figure 5. In other words, the second direction in which the second sound reaches the listener L changes from the second direction D2 shown in Figure 4 to the second direction D21 shown in Figure 5. Figure 5 shows a dotted arrow, which indicates the change from the second region A2 shown in Figure 4 to the second region A21 shown in Figure 5.

[0106] Furthermore, the first sound information is processed by the first sound processing unit 131 so that the first sound reaches the listener L from the first region A1. Therefore, as shown in Figure 5, the first sound reaches the listener L from the first region A1.

[0107] Furthermore, when the second sound information is modified, the second angle, which is the angle between the second direction from which the second sound reaches the listener L and the predetermined plane S, is changed from θ2 shown in Figure 4 to θ21 shown in Figure 5. As a result, when the second sound information is modified, the first angle (θ1) and the second angle (θ21) become different values. Therefore, even if the listener L hears both the first and second sounds, the occurrence of the above-mentioned problems, such as the first and second sounds sounding like they are coming from the same direction, is suppressed.

[0108] Furthermore, it is desirable that the absolute value of the difference between θ2 and θ21 (i.e., |θ2-θ21|) be between 4° and 20°, even better if it is between 6° and 15°, and even better if it is between 8° and 12°. For example, as mentioned above, θ1=θ2=80°, but θ21=70°. By having the absolute value of the difference between θ2 and θ21 fall within the above range, the occurrence of the above problem is sufficiently suppressed.

[0109] In summary, the sound reproduction method according to this embodiment includes an acquisition step, a determination step, a processing step, and an output step.

[0110] In the acquisition step, first region information is acquired, which indicates the first region A1 in which the sound image of the first sound, an object sound reaching the listener L in the sound reproduction space, is localized, and direction information is acquired, which indicates the direction D in which the listener L's head is facing. A predetermined plane S is defined as a plane perpendicular to the direction D in which the listener L's head is facing and passing through both of the listener L's ears. In the judgment step, second region information is acquired, which indicates the second region A2 in which the sound image of the second sound reaching the listener L in the sound reproduction space is localized. Furthermore, in the judgment step, based on the acquired direction information, it is determined whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to the predetermined plane S as the plane of symmetry. In the processing step, if it is determined that the first direction D1 and the second direction D2 are symmetrical in plane, sound information indicating the second tone (second tone information) is acquired, and a modification process is applied to the acquired second tone information so that the second direction D2 is changed so that the first direction D1 and the second direction D2 are no longer symmetrical in plane. In the output step, the modified second tone information is output.

[0111] As a result, the first direction D1 and the second direction D21 are not symmetrical. Furthermore, the angle (first angle) θ1 between the first direction D1, where the first sound reaches the listener L, and the predetermined plane S is different from the angle (second angle) θ21 between the second direction D21, where the second sound reaches the listener L, and the predetermined plane S. Therefore, even if the listener L hears both the first and second sounds, the occurrence of the above-mentioned problems, such as the first and second sounds sounding like they are coming from the same direction, is suppressed, allowing the listener L to accurately perceive both the first and second sounds. In other words, an acoustic reproduction method is realized that makes it easier for the listener L to accurately perceive the two sounds reaching them.

[0112] Furthermore, in the modification process according to this embodiment, the distance between the second sound and the listener L is kept constant. In addition, in the modification process according to this embodiment, the intensity of the second sound is kept constant.

[0113] Thus, even if the distance between the second sound and the listener L, and the intensity of the second sound are kept constant, the occurrence of the above-mentioned problem can be sufficiently suppressed by applying the modification process according to this embodiment.

[0114] In this embodiment, the acquisition step acquires spatial information indicating the shape of the sound reproduction space. The sound reproduction method includes a determination step in which a second region A2 is determined based on the acquired first region information and the acquired spatial information, in which the sound image of the second sound, which is a reflected sound of the first sound, is localized. In the determination step, second region information indicating the determined second region A2 is acquired. In the processing step, sound information indicating the first sound (first sound information) is acquired as sound information indicating the second sound (second sound information).

[0115] This enables an acoustic reproduction method in which, even when the second sound is a reflection of the first sound, the listener L can more easily perceive the two sounds that reach the listener L accurately.

[0116] Furthermore, in this embodiment, a modification process is performed so that the second angle θ2 > θ21. In other words, in the processing step, a modification process is performed to shift the second direction D2 so that at least one of the interaural intensity difference of the second sound and the interaural time difference of the second sound becomes larger. The interaural intensity difference of the second sound refers to the difference in intensity of the second sound in both ears of the listener L, and the interaural time difference of the second sound refers to the difference in arrival time of the second sound in both ears of the listener L.

[0117] As a result, the second direction D2 in which the second sound reaches the listener L is shifted, and the relationship between θ1 and θ21 becomes as follows. That is, θ1, which is the angle (first angle) between the first direction D1 in which the first sound reaches the listener L and the predetermined plane S, is different from θ21, which is the angle (second angle) between the second direction D21 in which the second sound reaches the listener L and the predetermined plane S. Furthermore, as the interaural intensity difference of the second sound increases, the listener L becomes more perceptible to the second direction D21 in which the second sound reaches the listener L. Similarly, as the interaural time difference of the second sound increases, the listener L becomes more perceptible to the second direction D21 in which the second sound reaches the listener L. Therefore, by increasing at least one of the interaural intensity difference and the interaural time difference, an acoustic reproduction method is realized in which the listener L becomes more accurately able to perceive the two sounds reaching the listener L.

[0118] In contrast to this embodiment, a modification process may be applied so that the second angle θ2 < θ21. In other words, a modification process may be applied so that the second direction D2 in which the second sound reaches the listener L is shifted so that both the interaural intensity difference of the second sound and the interaural time difference of the second sound are small. Even in this case, the listener L can accurately perceive the first and second sounds.

[0119] Furthermore, the program according to this embodiment may be a program that causes a computer to execute the above-described sound reproduction method.

[0120] This allows the computer to perform the above sound playback method according to the program.

[0121] Furthermore, the sound reproduction device 100 according to this embodiment includes an acquisition unit 121, a determination unit 123, a processing unit (second sound processing unit 132), and an output unit (first output unit 140). The acquisition unit 121 acquires first region information indicating a first region A1 in which the sound image of the first sound, which is an object sound reaching the listener L in the sound reproduction space, is localized, and direction information indicating the direction in which the listener L's head is facing. The predetermined plane S is defined as a plane perpendicular to the direction D in which the listener L's head is facing and passing through both of the listener L's ears. The determination unit 123 acquires second region information indicating a second region A2 in which the sound image of the second sound reaching the listener L in the sound reproduction space is localized. Furthermore, based on the acquired direction information, the determination unit 123 determines whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to the predetermined plane S. The second sound processing unit 132, when it determines that the first direction D1 and the second direction D2 are symmetrical in plane, acquires sound information indicating the second sound (second sound information) and applies a modification process to the acquired second sound information so that the second direction D2 is changed so that the first direction D1 and the second direction D2 are no longer symmetrical in plane. The first output unit 140 outputs the modified second sound information.

[0122] As a result, the first direction D1 and the second direction D21 are not symmetrical. Furthermore, the angle (first angle) θ1 between the first direction D1, where the first sound reaches the listener L, and the predetermined plane S is different from the angle (second angle) θ21 between the second direction D21, where the second sound reaches the listener L, and the predetermined plane S. Therefore, even if the listener L hears both the first and second sounds, the occurrence of the above-mentioned problems, such as the first and second sounds sounding from the same direction, is suppressed, and the listener L can accurately perceive both the first and second sounds. In other words, an acoustic reproduction device 100 is realized that makes it easier for the listener L to accurately perceive the two sounds reaching the listener L.

[0123] Next, we will explain the case where the determination unit 123 determines that the first direction D1 and the second direction D2 are not symmetrical (No in S60). In other words, this is the case where θ1 ≠ θ2.

[0124] In this case, the convolution processing unit 130 (second tone processing unit 132) acquires second tone information indicating the second tone and applies a process to the acquired second tone information so that the second direction D2 in which the second tone reaches the listener L is not changed (S90). More specifically, the second tone processing unit 132 performs a process to convolve the head-related transfer function into the second tone information so that the second tone reaches the listener L from the second region A2. In other words, unlike step S70, the second tone processing unit 132 applies a process to the second tone information that is different from the modification process described above. At this time, similar to step S70, the convolution processing unit 130 (first tone processing unit 131) also processes the first tone information. More specifically, the first tone processing unit 131 performs a process to convolve the head-related transfer function into the first tone information so that the first tone reaches the listener L from the first region A1. The convolution processing unit 130 outputs the processed first sound information and the processed second sound information to the first output unit 140.

[0125] Furthermore, the first output unit 140 outputs the second sound information, which has been processed and output by the convolution processing unit 130, to the headphones 200 (S100). More specifically, the first output unit 140 mixes the first sound information and the second sound information output by the convolution processing unit 130 and outputs the mixed first sound information and second sound information to the headphones 200.

[0126] Then, the second output unit 202 of the headphones 200 reproduces the first and second tones based on the first and second tones output by the first output unit 140.

[0127] If S60 is No, that is, if θ1 ≠ θ2, then even if the listener L hears the first and second tones, problems such as the first and second tones sounding like they are coming from the same direction do not occur. In other words, even in this case, an acoustic reproduction method is realized that makes it easier for the listener L to accurately perceive the two tones that reach them.

[0128] Furthermore, Figure 6 will be used to explain the case where the direction D of the listener L's head changes slightly while the listener L is listening to the first and second tones shown in Figure 5.

[0129] Figure 6 is a schematic diagram showing an example of the sound reproduction space when the direction D towards which the listener L's head is facing changes according to this embodiment.

[0130] In Figure 5, the direction D that listener L's head is pointing is the 12 o'clock direction. Here, as shown in Figure 6, the angle between the direction D that listener L's head is pointing and the 12 o'clock direction is α. In other words, compared to the state in Figure 5, the direction D that listener L's head is pointing is rotated clockwise by α in the state in Figure 6. Note that α is, for example, between 0° and 10°, and in this case, it is a very small value such as 2°.

[0131] Furthermore, because the direction D in which the listener L's head is facing has changed clockwise, the predetermined plane S has also changed clockwise.

[0132] Thus, when α is a very small value such as 2°, the second sound processing unit 132 further processes the acquired second sound information. More specifically, the second sound processing unit 132 applies a maintenance process to the second sound information, which is a process that maintains a constant angle (second angle) between the second direction D22 in which the second sound reaches the listener L and the predetermined plane S.

[0133] In other words, in Figure 6, which shows the case where the maintenance treatment is applied, the second angle is θ22, and since the second angle is maintained at a constant value, θ22 = θ21. Thus, when α is a very small value, the second angle is maintained before and after the maintenance treatment is applied.

[0134] Furthermore, since the predetermined plane S is rotating, when the maintenance process is applied, the sound image of the second sound is localized to a second region A22, which is a different region from the second region A21 shown in Figure 5. Also, a dotted arrow is shown in Figure 6, and this dotted arrow indicates the change from the second region A21 shown in Figure 5 to the second region A22 shown in Figure 6.

[0135] In this case, the first angle, which is the angle between the first direction D1 from which the first sound reaches the listener L and the predetermined plane S, is θ12, satisfying θ12 = θ1 - α. Note that α is a very small value and can be ignored, so θ12 = θ1. On the other hand, the second angle is θ22 = θ21. As explained above using Figure 5, the first angle θ1 and the second angle θ21 are different values, and therefore in Figure 6 as well, the first angle θ12 (i.e., θ1) and the second angle θ22 (i.e., θ21) are different values. In this case, the first direction D1 and the second direction D22 are not symmetrical. Therefore, even if the listener L hears the first and second sounds, the occurrence of the above-mentioned problem, such as the first and second sounds sounding from the same direction, is suppressed.

[0136] Let's consider the case where α is a very small value, as shown in Figure 6, and no modification processing is applied to the second sound information, resulting in the second sound reaching the listener L from the second region A2, as shown in Figure 4. In this case, the angle (first angle) between the first direction D1, from which the first sound reaches the listener L, and the predetermined plane S is θ1-α. Similarly, the angle (second angle) between the second direction D2, from which the second sound reaches the listener L, and the predetermined plane S is θ2+α. Since α is a very small value, it can be ignored, so the first angle becomes θ1 and the second angle becomes θ2. When θ1=θ2 is satisfied, the first direction D1 and the second direction D2 become symmetrical. In other words, if α is a very small value and no maintenance processing is applied to the second sound information, problems will occur such as the first and second sounds appearing to come from the same direction when the listener L hears both the first and second sounds.

[0137] Furthermore, Figure 7 will be used to explain the case where the direction D of the listener L's head changes significantly while the listener L is listening to the first and second tones shown in Figure 5.

[0138] Figure 7 is a schematic diagram showing another example of the sound reproduction space when the direction D towards which the listener L's head is facing changes according to this embodiment.

[0139] In Figure 5, the direction D that listener L's head is pointing is the 12 o'clock direction. Here, as shown in Figure 7, the angle between the direction D that listener L's head is pointing and the 12 o'clock direction is β. In other words, compared to the state in Figure 5, the direction D that listener L's head is pointing is rotated clockwise by β in the state in Figure 7. Note that β is, for example, between 10° and 90°, and in this case, it is a large value such as 30°.

[0140] Furthermore, because the direction D in which the listener L's head is facing has changed clockwise, the predetermined plane S has also changed clockwise.

[0141] Thus, when β is a large value, the second sound processing unit 132 performs the same process as in step S90, convolving the head transfer function into the second sound information so that the second sound reaches the listener L from the second region A2.

[0142] In this case, the second angle is θ23, more specifically, θ23 = θ2 + β. At this time, the first angle, which is the angle between the first direction D1 in which the first sound reaches the listener L and the predetermined plane S, is θ13, and θ13 = θ1 - β.

[0143] The first angle θ13 (i.e., θ1-β) and the second angle θ23 (i.e., θ2+β) are different values, and the first direction D1 and the second direction D2 are not symmetrical. Therefore, even if the listener L hears the first and second tones, the occurrence of the above-mentioned problem, such as the first and second tones sounding as coming from the same direction, is suppressed. In other words, if the direction D that the listener L's head is facing changes significantly, as shown in Figure 7, the second direction D2 from which the second tone reaches the listener L does not need to change.

[0144] (Embodiment 2) In Embodiment 1, the second sound is a reflection of the first sound, and sound information indicating the first sound (first sound information) is obtained as sound information indicating the second sound (second sound information), but this is not limited to this. In Embodiment 2, the second sound is an object sound different from the first sound, and the second sound information indicating the second sound is extracted and obtained from audio content information.

[0145] [composition] Next, the configuration of the sound reproduction device 100a according to Embodiment 2 will be described.

[0146] Figure 8 is a block diagram showing the functional configuration of the sound reproduction device 100a according to this embodiment.

[0147] The sound reproduction device 100a according to this embodiment differs from the sound reproduction device 100 mainly in that it includes an extraction unit 110a instead of an extraction unit 110, an information processing unit 120a instead of an information processing unit 120, and a convolution processing unit 130a instead of a convolution processing unit 130.

[0148] In other words, the sound reproduction device 100a comprises an extraction unit 110a, an information processing unit 120a, a convolution processing unit 130a, and a first output unit 140.

[0149] As described above, the second sound in this embodiment is an object sound different from the first sound. Both the first and second sounds are object sounds and are not particularly limited, but may be sounds caused by people, such as a person singing, a person speaking, or a person clapping their hands, or sounds caused by objects other than people, such as the sound of a car driving. In this embodiment, the first sound is a woman singing, and the second sound is a man speaking. In this embodiment, information related to such first and second sounds is included in the audio content information.

[0150] In the following, the sound reproduction device 100a according to this embodiment will be described, focusing on the differences from the sound reproduction device 100 according to Embodiment 1.

[0151] The extraction unit 110a includes a region information extraction unit 111a, a spatial information extraction unit 112, and a sound information extraction unit 113a.

[0152] The region information extraction unit 111a extracts the first region information and the second region information contained in the acquired audio content information. The second region information is information indicating the second region A2 in which the sound image of the second sound is localized. More specifically, the second region information is information indicating the position of the second region A2 in the sound playback space.

[0153] The sound information extraction unit 113a extracts the first sound information and the second sound information contained in the acquired audio content information. The second sound information is information indicating the second sound, which is an object sound, and is digital data expressed in formats such as WAVE, MP3, or WMA.

[0154] The information processing unit 120a determines the positional relationship between the first region A1 where the sound image of the first sound is localized and the second region A2 where the sound image of the second sound is localized, based on the first region information, the second sound information, spatial information, and directional information. The information processing unit 120a includes an acquisition unit 121a and a determination unit 123a. In other words, unlike the information processing unit 120 according to Embodiment 1, the information processing unit 120a does not need to have a determination unit 122.

[0155] The acquisition unit 121a acquires the first region information, second region information, and spatial information extracted by the extraction unit 110a. More specifically, it acquires the first region information and second region information extracted by the region information extraction unit 111a, and the spatial information extracted by the spatial information extraction unit 112. The acquisition unit 121a also acquires directional information sensed by the headphones 200 (more specifically, the head sensor unit 201).

[0156] The determination unit 123a acquires second region information indicating the second region A2 in which the sound image of the second sound reaching the listener L is localized in the sound reproduction space. In this embodiment, the determination unit 123a acquires the second region information extracted by the extraction unit 110a and acquired by the acquisition unit 121a. Furthermore, based on the direction information acquired by the acquisition unit 121a, the determination unit 123a determines whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to a predetermined plane S. Furthermore, the determination unit 123a outputs the result of the determination to the convolution processing unit 130a.

[0157] The convolution processing unit 130a processes the sound information indicating the first tone (first tone information) and the sound information indicating the second tone (second tone information) based on the result determined by the determination unit 123a. The convolution processing unit 130a includes a first tone processing unit 131a, a second tone processing unit 132a, and an HRTF storage unit 133.

[0158] The first sound processing unit 131a processes the first sound information by referring to the head-related transfer function stored in the HRTF memory unit 133. More specifically, the first sound processing unit 131a convolves the head-related transfer function into the first sound information so that the first sound reaches the listener L from the first region A1 indicated by the first region information acquired by the acquisition unit 121a. The first sound processing unit 131a acquires the first sound information extracted from the audio content information by the sound information extraction unit 113a of the extraction unit 110a, and then processes the acquired first sound information as described above.

[0159] The second sound processing unit 132a processes the second sound information by referring to the head-related transfer function stored in the HRTF memory unit 133. More specifically, the second sound processing unit 132a convolves the head-related transfer function into the second sound information so that the second sound reaches the listener L from the second region A2 indicated by the second region information extracted by the extraction unit 110a. The second sound processing unit 132a obtains the second sound information extracted from the audio content information by the sound information extraction unit 113a of the extraction unit 110a and applies the above processing to the obtained second sound information.

[0160] Furthermore, if the determination unit 123a determines that the first direction D1 and the second direction D2 are symmetrical in plane, the second sound processing unit 132a performs the following processing: The second sound processing unit 132a acquires the second sound information and applies a modification process (modification process) to the acquired second sound information so that the second direction D2 to which the second sound reaches the listener L is changed so that the first direction D1 and the second direction D2 are not symmetrical in plane.

[0161] The first sound information processed by the first sound processing unit 131a is output to the first output unit 140. Similarly, the second sound information processed by the second sound processing unit 132a is output to the first output unit 140.

[0162] [Example of operation] The following describes an example of the operation of the sound reproduction method performed by the sound reproduction device 100a. Figure 9 is a flowchart of an example of the operation of the sound reproduction device 100a according to this embodiment.

[0163] First, the extraction unit 110a acquires audio content information (S10).

[0164] The extraction unit 110a extracts first region information and first tone information related to the first tone, second region information and second tone information related to the second tone, and spatial information from the acquired audio content information (S20a). More specifically, the region information extraction unit 111a extracts the first region information and second region information contained in the audio content information. The spatial information extraction unit 112 extracts the spatial information contained in the audio content information. The tone information extraction unit 113a extracts the first tone information and second tone information contained in the audio content information. The extraction unit 110a outputs the extracted first region information, first tone information, second region information, second tone information, and spatial information. This step S20a corresponds to the extraction step.

[0165] Furthermore, the information processing unit 120a acquires first region information indicating the first region A1, second region information indicating the second region A2, direction information, and spatial information (S30a). More specifically, the acquisition unit 121a of the information processing unit 120a acquires the first region information, second region information, and spatial information output from the extraction unit 110a, and the direction information output from the head sensor unit 201 of the headphones 200. The acquisition unit 121a outputs the first region information indicating the first region A1, second region information indicating the second region A2, direction information, and spatial information to the judgment unit 123a.

[0166] Furthermore, the determination unit 123a determines, based on the direction information acquired by the acquisition unit 121a, whether the first direction D1 in which the first sound reaches the listener L and the second direction D2 in which the second sound reaches the listener L are symmetrical with respect to a predetermined plane S (S60).

[0167] Here, the processing of step S60 in the operation example according to this embodiment will be explained in more detail with reference to Figure 10. Figure 10 is a schematic diagram for explaining the second sound in the sound reproduction space according to this embodiment.

[0168] In step S60 according to this embodiment, the same processing as in step S60 according to Embodiment 1 is preferable. That is, the determination unit 123a may make the above determination based on the coordinate positions of the x, y, and z axes of the predetermined plane S, the coordinate positions of the x, y, and z axes of the first region A1, and the coordinate positions of the x, y, and z axes of the second region A2.

[0169] The decision unit 123a outputs the result of its decision to the convolution processing unit 130a. The convolution processing unit 130a acquires the result of the decision made by the decision unit 123a.

[0170] In Figure 10, the first angle is shown as θ1 and the second angle as θ2. When the first direction D1 and the second direction D2 are symmetrical, the first angle (θ1) is equal to the second angle (θ2). In this case, it becomes difficult for the listener L to accurately perceive the two sounds (here, the first and second sounds) that reach the listener L. More specifically, when the listener L hears the first and second sounds, the first and second sounds appear to come from the same direction, and the listener L cannot accurately perceive the first and second sounds.

[0171] Let's explain the operation example again using Figure 9.

[0172] First, let's explain the case where the determination unit 123a determines that the first direction D1 and the second direction D2 are symmetrical in plane (Yes in S60).

[0173] In this case, the second sound processing unit 132a acquires second sound information indicating the second sound and applies a modification process (modification process) to the acquired second sound information so that the second direction D2 in which the second sound reaches the listener L is changed so that the first direction D1 and the second direction D2 are not plane-symmetric (S70). At this time, the first sound processing unit 131a also processes the first sound information. More specifically, the first sound processing unit 131a performs a process to convolve the head-related transfer function into the first sound information so that the first sound reaches the listener L from the first region A1. The convolution processing unit 130a outputs the processed first sound information and the modified second sound information to the first output unit 140.

[0174] Furthermore, the first output unit 140 outputs the second sound information, which has been modified and output by the convolution processing unit 130a, to the headphones 200 (S80).

[0175] Then, the second output unit 202 of the headphones 200 reproduces the first and second tones based on the first and second tones output by the first output unit 140.

[0176] Here, the sound reaching the listener L in the sound reproduction space as a result of the operations in steps S70 and S80 in the operation example of this embodiment will be explained in more detail with reference to Figure 11.

[0177] Figure 11 is a schematic diagram showing the sound reproduction space after the second sound information has been modified.

[0178] As a result of the modification process, the region in which the sound image of the second sound is localized changes from the second region A2 shown in Figure 10 to the second region A21 shown in Figure 11. In other words, the second direction in which the second sound reaches the listener L changes from the second direction D2 shown in Figure 10 to the second direction D21 shown in Figure 11.

[0179] Furthermore, the first sound information is processed by the first sound processing unit 131a so that the first sound reaches the listener L from the first region A1. Therefore, as shown in Figure 11, the first sound reaches the listener L from the first region A1.

[0180] Furthermore, when the second sound information is modified, the second angle, which is the angle between the second direction from which the second sound reaches the listener L and the predetermined plane S, is changed from θ2 shown in Figure 10 to θ21 shown in Figure 11. As a result, when the second sound information is modified, the first angle (θ1) and the second angle (θ21) become different values. Therefore, even if the listener L hears both the first and second sounds, the occurrence of the above-mentioned problems, such as the first and second sounds sounding like they are coming from the same direction, is suppressed.

[0181] In other words, in this embodiment, the second sound is an object sound different from the first sound. The sound reproduction method includes an extraction step of acquiring audio content information and extracting the first region information, second region information, and sound information (second sound information) contained in the acquired audio content information. In the acquisition step, the extracted first region information is acquired. In the judgment step, the extracted second region information is acquired. In the processing step, the extracted sound information (second sound information) is acquired.

[0182] This enables an acoustic reproduction method in which, even when the second sound is a different object sound from the first sound, the listener L can more easily perceive the two sounds that reach the listener L accurately.

[0183] Next, we will explain the case where the determination unit 123a determines that the first direction D1 and the second direction D2 are not symmetrical (No in S60). In other words, this is the case where θ1 ≠ θ2.

[0184] In this case, the second sound processing unit 132a acquires second sound information indicating the second sound and applies a process to the acquired second sound information so that the second direction D2 in which the second sound reaches the listener L is not changed (S90). More specifically, the second sound processing unit 132a applies a process to convolve the head-related transfer function into the second sound information so that the second sound reaches the listener L from the second region A2. In other words, unlike step S70, the second sound processing unit 132a applies a process to the second sound information that is different from the modification process described above. At this time, similar to step S70, the first sound processing unit 131a also processes the first sound information. More specifically, the first sound processing unit 131a applies a process to convolve the head-related transfer function into the first sound information so that the first sound reaches the listener L from the first region A1. The convolution processing unit 130a outputs the processed first tone information and the processed second tone information to the first output unit 140.

[0185] Furthermore, the first output unit 140 outputs the second sound information, which has been processed and output by the convolution processing unit 130a, to the headphones 200 (S100).

[0186] Then, the second output unit 202 of the headphones 200 reproduces the first and second tones based on the first and second tones output by the first output unit 140.

[0187] If S60 is No, that is, if θ1 ≠ θ2, then even if the listener L hears the first and second tones, problems such as the first and second tones sounding like they are coming from the same direction do not occur. In other words, even in this case, an acoustic reproduction method is realized that makes it easier for the listener L to accurately perceive the two tones that reach them.

[0188] (Other embodiments) The sound reproduction apparatus and sound reproduction method according to the embodiments of this disclosure have been described above based on embodiments, but this disclosure is not limited to these embodiments. For example, other embodiments realized by arbitrarily combining the components described herein, or by excluding some of the components, may also be considered embodiments of this disclosure. Furthermore, modifications obtained by applying various modifications to the above embodiments that a person skilled in the art could conceive of without departing from the spirit of this disclosure, that is, the meaning indicated by the wording in the claims, are also included in this disclosure.

[0189] Furthermore, the following forms may also be included within the scope of one or more aspects of this disclosure.

[0190] (1) Some of the components constituting the above-described sound reproduction device may be a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor achieves its function by operating in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer in order to achieve a predetermined function.

[0191] (2) Some of the components constituting the above-described sound reproduction device and sound reproduction method may be composed of a single system LSI (Large Scale Integration). The system LSI is a highly functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system comprising a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its function by operating the microprocessor in accordance with the computer program.

[0192] (3) Some of the components constituting the above-described sound reproduction device may consist of detachable IC cards or standalone modules attached to each device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-described multi-functional LSI. The microprocessor operates according to a computer program, thereby enabling the IC card or module to perform its function. The IC card or module may be tamper-resistant.

[0193] (4) Furthermore, some of the components constituting the above-described sound reproduction device may be the computer program or the digital signal recorded on a recording medium that can be read by a computer, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, it may be the digital signal recorded on these recording media.

[0194] Furthermore, some of the components constituting the above-mentioned sound reproduction device may transmit the computer program or the digital signal via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0195] (5) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of the computer program.

[0196] (6) The Disclosure may also provide a computer system comprising a microprocessor and memory, wherein the memory stores the computer program, and the microprocessor operates in accordance with the computer program.

[0197] (7) Alternatively, the program or the digital signal may be carried out by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like.

[0198] (8) The above embodiments and the above modified examples may be combined.

[0199] Furthermore, although not shown in Figure 2, etc., an image synchronized with the sound output from the headphones 200 may be presented to the listener L. In this case, for example, a display device such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel may be provided around the listener L, and the image may be presented on such a display device. Alternatively, the image may be presented to the listener L by wearing a head-mounted display or the like. [Industrial applicability]

[0200] This disclosure is applicable to sound reproduction methods and sound reproduction devices, and is particularly applicable to stereophonic sound reproduction systems and the like. [Explanation of symbols]

[0201] 100, 100a Sound reproduction device 110, 110a Extraction part 111, 111a Area information extraction part 112 Spatial information extraction part 113, 113a Sound information extraction section 120, 120a Information Processing Unit 121, 121a acquisition part 122 Decision Section 123, 123a Judgment section 130, 130a Convolutional Processing Unit 131, 131a First sound processing unit 132, 132a Second sound processing unit 133 HRTF storage section 140 First Output Section 200 headphones 201 Head sensor unit 202 Second Output Section A1 1st area A2, A21, A22 2nd area D direction D1 1st direction D2, D21, D22 2nd direction L listener S Predetermined plane

Claims

1. An acquisition step of acquiring information indicating multiple sounds reaching the listener in the sound reproduction space, and directional information indicating the direction the listener's head is facing, When a predetermined plane is defined as a plane perpendicular to the direction in which the listener's head is facing and passing through both of the listener's ears, A determination step of determining whether the first direction in which a first sound, included in the plurality of sounds indicated by the acquired information, reaches the listener, and the second direction in which a second sound, included in the plurality of sounds indicated by the acquired information, reaches the listener, are symmetrical with respect to the predetermined plane, If it is determined that the first direction and the second direction are symmetrical in plane, the process includes applying a modification process to the sound information indicating the second sound to change the second direction, The output step includes outputting the sound information that has undergone the modification process. Sound reproduction method.

2. The modification process is a process of changing the second direction such that the first direction and the second direction are not symmetrical in plane. The sound reproduction method according to claim 1.

3. The aforementioned second sound is a different object sound from the aforementioned first sound. The sound reproduction method includes an extraction step of acquiring audio content information, first region information indicating a first region where the sound image of the first sound contained in the acquired audio content information is localized, second region information indicating a second region where the sound image of the second sound is localized, and the sound information, In the acquisition step, the extracted first region information is acquired, In the aforementioned determination step, the extracted second region information is obtained, In the processing step described above, the extracted sound information is acquired. The sound reproduction method according to claim 1.

4. In the acquisition step, spatial information indicating the shape of the sound reproduction space is acquired. The sound reproduction method includes a determination step of determining the second region in which the sound image of the second sound, which is a reflected sound of the first sound, is localized, based on the acquired first region information and the acquired spatial information. In the determination step, the second region information indicating the determined second region is obtained, In the processing step described above, sound information indicating the first sound is acquired as sound information indicating the second sound. The sound reproduction method according to claim 3.

5. In the processing step, the modification process is performed such that the second direction is shifted so that at least one of the interaural intensity difference of the second sound and the interaural time difference of the second sound becomes larger. The sound reproduction method according to claim 2.

6. A computer program for causing a computer to execute the sound reproduction method described in any one of claims 1 to 5.

7. An acquisition unit that acquires information indicating multiple sounds reaching the listener in the sound reproduction space, and directional information indicating the direction the listener's head is facing, When a predetermined plane is defined as a plane perpendicular to the direction in which the listener's head is facing and passing through both of the listener's ears, A determination unit that determines whether the first direction in which a first sound, included in the plurality of sounds indicated by the acquired information, reaches the listener, and the second direction in which a second sound, included in the plurality of sounds indicated by the acquired information, reaches the listener, are symmetrical with respect to the predetermined plane, A processing unit that, when it is determined that the first direction and the second direction are symmetrical in plane, applies a modification process to the sound information indicating the second sound to change the second direction, The system includes an output unit that outputs the sound information after the modification process has been performed. Sound reproduction device.

Images