Sound reproduction method, program product, and sound reproduction device

Abstract

The present application provides a sound reproduction method, a program product, and a sound reproduction apparatus. The sound reproduction method includes: a signal acquisition step of acquiring a plurality of first audio signals and classification information, the first audio signals representing first sounds reaching a listener from a first range as a prescribed angular range, and the classification information being information in which the plurality of first audio signals are classified; and a correction processing step of, in a case where one of the plurality of first sounds is determined to be a trigger sound, implementing correction processing on a first audio signal representing the determined trigger sound and one or more first audio signals representing one or more first sounds classified into the same group as the trigger sound, among the plurality of acquired first audio signals, based on the acquired classification information.

Description

[0001] This application is a divisional application of the invention patent application filed on March 18, 2021, with application number 202180020825.8 and entitled "Sound Reproduction Method, Recording Medium and Sound Reproduction Apparatus". Technical Field

[0002] This disclosure relates to methods for sound reproduction, etc. Background Technology

[0003] Patent document 1 discloses a technology for a stereo sound reproduction system that achieves a sense of presence by outputting sound from multiple speakers arranged around the listener.

[0004] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2005-287002 Summary of the Invention

[0005] The problem that the invention aims to solve Furthermore, people (here referring to the listeners of sounds) perceive sounds arriving from behind them less readily than they perceive sounds arriving from in front of them.

[0006] Therefore, the purpose of this disclosure is to provide a sound reproduction method, etc., that improves the perception level of sound arriving from behind the listener.

[0007] Methods used to solve problems A sound reproduction method according to a technical solution of this disclosure includes: a signal acquisition step, acquiring a plurality of first audio signals and classification information, wherein the first audio signals represent the sound arriving at the listener from a first range defined as an angular range, i.e., a first sound, and the classification information is information on the classification of the plurality of first audio signals; and a correction processing step, wherein, in the case that one of the plurality of first sounds is determined to be a trigger tone, based on the acquired classification information, correction processing is performed on the first audio signal representing the determined trigger tone among the acquired plurality of first audio signals, and on one or more first audio signals representing one or more first sounds classified into the same group as the trigger tone.

[0008] A program product relating to the present disclosure is a program product containing a computer program for causing a computer to perform the above-described sound reproduction method.

[0009] An audio reproduction apparatus according to a technical solution of this disclosure includes: a signal acquisition unit that acquires a plurality of first audio signals and classification information, wherein the first audio signals represent a sound, i.e., a first sound, arriving at a listener from a first range defined as an angular range, and the classification information is information on the classification of the plurality of first audio signals; and a correction processing unit that, when one of the plurality of first sounds is determined to be a trigger tone, performs correction processing on the first audio signal representing the determined trigger tone and one or more first audio signals representing one or more first sounds classified into the same group as the trigger tone, based on the acquired classification information.

[0010] In addition, these inclusive or specific technical solutions can also be implemented by non-transitory recording media such as systems, devices, methods, integrated circuits, computer programs, or computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0011] Invention Effects The sound reproduction method of the present disclosure can improve the perception level of sound arriving from behind the listener. Attached Figure Description

[0012] Figure 1 This is a block diagram showing the functional structure of the sound reproduction device according to Embodiment 1.

[0013] Figure 2 This is a schematic diagram illustrating an example of the use of sound output from multiple speakers according to Embodiment 1.

[0014] Figure 3 This is a flowchart illustrating an example of the operation of the sound reproduction device according to Embodiment 1.

[0015] Figure 4 This is a schematic diagram illustrating an example of the judgment made by the correction processing unit in Implementation Method 1.

[0016] Figure 5 This is a schematic diagram illustrating another example of the judgment made by the correction processing unit in Implementation 1.

[0017] Figure 6 This is a schematic diagram illustrating another example of the judgment made by the correction processing unit in Implementation 1.

[0018] Figure 7 This is a diagram illustrating an example of the correction processing performed by the correction processing unit in Implementation Method 1.

[0019] Figure 8This is a diagram illustrating another example of the correction processing performed by the correction processing unit in Implementation Method 1.

[0020] Figure 9 This is a diagram illustrating another example of the correction processing performed by the correction processing unit in Implementation Method 1.

[0021] Figure 10 This is a schematic diagram illustrating an example of the correction processing performed on the first audio signal in accordance with Embodiment 1.

[0022] Figure 11 This is a schematic diagram illustrating another example of the correction processing performed on the first audio signal in accordance with Implementation 1.

[0023] Figure 12 This is a block diagram showing the functional structure of the sound reproduction device and the sound acquisition device according to Embodiment 2.

[0024] Figure 13 This is a schematic diagram illustrating sound collection performed by the sound collection device according to Embodiment 2.

[0025] Figure 14 This is a schematic diagram illustrating an example of the correction processing performed on a plurality of first audio signals in accordance with Embodiment 2. Detailed Implementation

[0026] (The understanding that forms the basis of this disclosure) Previously, there were known techniques for sound reproduction that achieved a sense of presence by outputting multiple audio signals, each representing a different sound, from multiple speakers arranged around the listener.

[0027] For example, the stereo sound reproduction system disclosed in Patent Document 1 includes a main speaker, surround speakers, and a stereo sound reproduction device.

[0028] The main loudspeaker amplifies the sound represented by the main audio signal when the listener is positioned within the pointing angle, the surround loudspeakers amplify the sound represented by the surround audio signal when facing the walls of the sound field space, and the stereo sound reproduction device amplifies the sound of each loudspeaker individually.

[0029] Furthermore, this stereo sound reproduction device includes a signal adjustment unit, a delay time addition unit, and an output unit. The signal adjustment unit adjusts the frequency characteristics of the surround audio signal based on the propagation environment during amplification. The delay time addition unit adds a delay time corresponding to the surround signal to the main audio signal. The output unit outputs the main audio signal with the added delay time to the main speaker and the adjusted surround audio signal to the surround speakers.

[0030] Such a stereo sound reproduction system can create a sound field space that provides a high degree of immersion.

[0031] Furthermore, humans (here, the listener of sound) perceive sounds arriving from behind them less readily than sounds arriving from in front. For example, humans have a perceptual characteristic (more specifically, an auditory characteristic) that makes it difficult to perceive the location or direction of sounds arriving from behind them. This perceptual characteristic stems from the shape of the human ear and the limits of its discrimination.

[0032] Furthermore, when two sounds (e.g., the target sound and ambient sound) arrive from behind the listener, one sound (e.g., the target sound) may be masked by the other sound (e.g., the ambient sound). In this situation, the listener has difficulty listening to the target sound and therefore cannot perceive the location or direction of the target sound arriving from behind the listener.

[0033] As an example, in the stereo sound reproduction system disclosed in Patent Document 1, even when the sound represented by the main audio signal and the sound represented by the surround audio signal arrive from behind the listener, the listener has difficulty perceiving the sound represented by the main audio signal. Therefore, there is a need for sound reproduction methods that improve the perception level of sound arriving from behind the listener.

[0034] Therefore, the sound reproduction method of one technical solution of this disclosure includes: a signal acquisition step, acquiring a first audio signal representing a first sound and a second audio signal representing a second sound, wherein the first sound is a sound arriving at the listener from a predetermined angular range, i.e., a first range, and the second sound is a sound arriving at the listener from a predetermined direction; an information acquisition step, acquiring information about the direction in which the listener's head is facing, i.e., directional information; a correction processing step, when the range behind the listener's head direction is taken as the front is defined as the second range, and when it is determined based on the acquired directional information that the first range and the predetermined direction are included in the second range, performing correction processing on at least one of the acquired first audio signal and the acquired second audio signal, wherein the correction processing is a process that increases the intensity of the second audio signal relative to the intensity of the first audio signal; and a mixing processing step, mixing the first audio signal and the second audio signal after the correction processing and outputting them to an output channel.

[0035] Therefore, when the first range and the defined orientation are included in the second range, the intensity of the second audio signal representing the second sound becomes stronger. Thus, the listener can easily hear the second sound arriving from behind (i.e., behind the listener) when the direction in which the listener's head is facing is considered forward. In other words, a sound reproduction method is implemented that can improve the perception level of the second sound arriving from behind the listener.

[0036] As an example, when the first sound is an ambient sound and the second sound is the target sound, it is possible to suppress the target sound from being masked by the ambient sound. That is, to achieve an audio reproduction method that can improve the perception level of the target sound arriving from behind the listener.

[0037] For example, the first range mentioned above is the range behind the reference orientation determined by the position of the output channel mentioned above.

[0038] Therefore, even if the first sound arrives at the listener from a range behind the reference direction, the listener can easily hear the second sound arriving from behind the listener.

[0039] For example, the above-described correction process is a process of correcting at least one of the gain of the first audio signal and the gain of the second audio signal.

[0040] Therefore, the gain of at least one of the first audio signal representing the first sound and the second audio signal representing the second sound can be corrected, so that the listener can more easily hear the second sound arriving from behind the listener.

[0041] For example, the above-described correction process is at least one of the process of reducing the gain of the acquired first audio signal and the process of increasing the gain of the acquired second audio signal.

[0042] Therefore, by performing at least one of the processes of reducing the gain of the first audio signal representing the first sound and increasing the gain of the second audio signal representing the second sound, the listener can more easily hear the second sound arriving from behind the listener.

[0043] For example, the above-described correction process is a process of correcting based on at least one of the frequency components of the first audio signal and the frequency components of the second audio signal.

[0044] Therefore, at least one of the frequency components of the first audio signal representing the first sound and the frequency components of the second audio signal representing the second sound can be modified, making it easier for the listener to hear the second sound arriving from behind the listener.

[0045] For example, the above correction process is based on reducing the spectrum of the frequency components of the first audio signal to be smaller than that based on the spectrum of the frequency components of the second audio signal.

[0046] As a result, the intensity of the spectrum of the frequency components of the first audio signal representing the first sound decreases, making it easier for the listener to hear the second sound arriving from behind the listener.

[0047] For example, the above-mentioned correction processing step is performed based on the positional relationship between the second range and the specified orientation; the above-mentioned correction processing is a process of correcting at least one of the gain of the first audio signal and the gain of the second audio signal, or a process of correcting based on at least one of the frequency characteristics of the first audio signal and the frequency characteristics of the second audio signal.

[0048] Therefore, correction processing can be performed based on the positional relationship between the second range D2 and the specified orientation, making it easier for the listener to hear the second sound arriving from behind the listener.

[0049] For example, when the second range is divided into the range to the right rear of the listener (i.e., the right rear range), the range to the left rear (i.e., the left rear range), and the range between the right rear range and the left rear range (i.e., the central rear range), in the correction processing step, if it is determined that the specified orientation is included in the right rear range or the left rear range, the correction processing is performed to either reduce the gain of the acquired first audio signal or increase the gain of the acquired second audio signal. If it is determined that the specified orientation is included in the central rear range, the correction processing is performed to either reduce the gain of the acquired first audio signal or increase the gain of the acquired second audio signal.

[0050] Therefore, when the specified direction is contained within the central rear area, a correction process is performed to make the intensity of the second audio signal representing the second sound stronger than the intensity of the first audio signal representing the first sound, compared to when the specified direction is contained within the right rear area or left rear area. Consequently, the listener can more easily hear the second sound arriving from behind the listener.

[0051] For example, the signal acquisition step acquires: multiple first audio signals and second audio signals representing multiple first sounds; and classification information, which is information that classifies the multiple first audio signals based on the frequency characteristics of each of the multiple first audio signals. The correction processing step performs the correction processing based on the acquired orientation information and classification information. The multiple first sounds are sounds collected from multiple first ranges respectively.

[0052] Therefore, the correction processing step can be performed on each group that has been classified into multiple first audio signals. Thus, the processing load of the correction processing step can be reduced.

[0053] For example, a sound reproduction method according to a technical solution of this disclosure includes: a signal acquisition step, acquiring a plurality of first audio signals representing a plurality of first sounds and a second audio signal representing a second sound, wherein the plurality of first sounds are multiple sounds arriving at the listener from a plurality of predetermined angle ranges, i.e., a plurality of first ranges, and the second sound is a sound arriving at the listener from a predetermined direction; an information acquisition step, acquiring information about the direction in which the listener's head is facing, i.e., direction information; and a correction processing step, when setting the range behind when the direction in which the listener's head is facing is taken as the front as the second range, based on the acquired... If the obtained directional information determines that the plurality of first ranges and the specified directional information are included in the second range, a correction process is performed on at least one of the obtained plurality of first audio signals and the obtained second audio signal. The correction process is to increase the intensity of the second audio signal relative to the intensity of the plurality of first audio signals. A mixing process is then performed to mix the corrected plurality of first audio signals and at least one of the second audio signals and output them to the output channel. The plurality of first sounds are sounds collected from the plurality of first ranges respectively.

[0054] Therefore, when the first range and the defined orientation are included in the second range, the intensity of the second audio signal representing the second sound becomes stronger. Thus, the listener can easily hear the second sound arriving from behind (i.e., behind the listener) when the direction in which the listener's head is facing is considered forward. In other words, a sound reproduction method is implemented that can improve the perception level of the second sound arriving from behind the listener.

[0055] Furthermore, the correction processing step can be performed on each group that has been classified into multiple first audio signals. Therefore, the processing load of the correction processing step can be reduced.

[0056] For example, a recording medium for a technical solution of this disclosure may also be a recording medium containing a computer program for causing a computer to execute the above-described sound reproduction method.

[0057] Therefore, the computer can execute the above-mentioned sound reproduction method according to the program.

[0058] For example, an audio reproduction apparatus according to a technical solution of this disclosure includes: a signal acquisition unit that acquires a first audio signal representing a first sound and a second audio signal representing a second sound, wherein the first sound is a sound that reaches the listener from a predetermined angular range, i.e., a first range, and the second sound is a sound that reaches the listener from a predetermined direction; an information acquisition unit that acquires information about the direction in which the listener's head is facing, i.e., directional information; a correction processing unit that, when defining the range behind the listener's head direction as the second range, performs correction processing on at least one of the acquired first audio signal and the acquired second audio signal, wherein the correction processing is a process that increases the intensity of the second audio signal relative to the intensity of the first audio signal; and a mixing processing unit that mixes at least one of the first audio signal and the second audio signal after the correction processing and outputs it to an output channel.

[0059] Therefore, when the first range and the defined direction are included in the second range, the intensity of the second audio signal representing the second sound becomes stronger. Thus, the listener can easily hear the second sound arriving from behind (i.e., behind the listener) when the direction in which the listener's head is facing is considered forward. In other words, an audio reproduction device is realized that can improve the perception level of the second sound arriving from behind the listener.

[0060] As an example, when the first sound is an ambient sound and the second sound is the target sound, it is possible to suppress the target sound from being masked by the ambient sound. That is, to realize an audio reproduction device that can improve the perception level of the target sound arriving from behind the listener.

[0061] Furthermore, these inclusive or specific technical solutions can also be implemented by non-transitory recording media such as systems, devices, methods, integrated circuits, computer programs, or computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0062] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0063] Furthermore, the embodiments described below are inclusive or specific examples. The numerical values, shapes, materials, constituent elements, the arrangement and connection of constituent elements, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit this disclosure.

[0064] Furthermore, in the following description, elements are sometimes assigned ordinal numbers such as 1, 2, and 3. These ordinal numbers are assigned to elements for identification purposes and do not necessarily correspond to a meaningful order. These ordinal numbers can also be appropriately replaced, newly assigned, or removed.

[0065] Furthermore, the figures are schematic diagrams and not necessarily rigorous representations. Additionally, substantially identical structures are assigned the same labels across the figures, and repetitive descriptions are omitted or simplified.

[0066] (Implementation Method 1) [structure] First, the structure of the sound reproduction device 100 according to Embodiment 1 will be described. Figure 1 This is a block diagram illustrating the functional structure of the sound reproduction device 100 according to this embodiment. Figure 2 This is a schematic diagram illustrating an example of the use of sound output from multiple speakers 1, 2, 3, 4 and 5 in relation to this embodiment.

[0067] The audio reproduction device 100 of this embodiment is used to process and transmit multiple acquired audio signals to... Figure 1 and Figure 2 The device 100 is a device that uses multiple loudspeakers 1, 2, 3, 4, and 5 to output sound, allowing the listener L to hear sounds represented by multiple audio signals. More specifically, the sound reproduction device 100 is a stereo sound reproduction device used to allow the listener L to hear stereo sound.

[0068] Furthermore, the sound reproduction device 100 processes multiple acquired audio signals based on the orientation information output by the head sensor 300. The orientation information refers to the direction in which the listener L's head is facing. The direction in which the listener L's head is facing is also the direction in which the listener L's face is facing. Additionally, orientation, for example, means direction.

[0069] The head sensor 300 is a device for sensing the direction in which the listener L's head is facing. The head sensor 300 can be a device that senses the 6DOF (Degrees of Freedom) information of the listener L's head. For example, the head sensor 300 is a device worn on the listener L's head and can be an inertial measurement unit (IMU), an accelerometer, a gyroscope, a magnetometer, or a combination thereof.

[0070] In addition, such as Figure 2 As shown, in this embodiment, multiple (here, five) speakers 1, 2, 3, 4, and 5 are arranged to surround the listener L. Figure 2In this clock face, to indicate direction, 0 o'clock, 3 o'clock, 6 o'clock, and 9 o'clock are shown to correspond to the time displayed on the clock face. Furthermore, the hollow arrow indicates the direction in which the listener L's head is facing; the direction in which the listener L's head is facing, located at the center of the clock face (also called the origin), is the 0 o'clock direction. Hereinafter, the connection between the listener L and the 0 o'clock direction will be referred to as "0 o'clock direction," and the same applies to other times displayed on the clock face.

[0071] In this embodiment, the five speakers 1, 2, 3, 4, and 5 consist of a central speaker, a front right speaker, a rear right speaker, a rear left speaker, and a front left speaker. Furthermore, speaker 1, which serves as the central speaker, is positioned at the 0-position.

[0072] The five speakers 1, 2, 3, 4 and 5 are amplification devices that output multiple audio signals representing the sound from the audio reproduction device 100.

[0073] like Figure 1 As shown, the audio reproduction device 100 includes a first signal processing unit 110, a first decoding unit 121, a second decoding unit 122, a first correction processing unit 131, a second correction processing unit 132, an information acquisition unit 140, and a mixing processing unit 150.

[0074] The first signal processing unit 110 is a processing unit that acquires multiple audio signals. The first signal processing unit 110 can acquire multiple audio signals by receiving signals from... Figure 2 Multiple audio signals can be obtained by sending multiple audio signals from other constituent elements not represented in the text. Figure 2 The storage device does not represent multiple audio signals stored in the image. The multiple audio signals acquired by the first signal processing unit 110 include a first audio signal and a second audio signal.

[0075] Here, the first audio signal and the second audio signal will be explained.

[0076] The first audio signal is a signal representing the first sound, which is the sound that reaches the listener L from a predetermined angular range, namely the first range D1. For example, the first range D1 is the range behind a reference orientation determined by the positions of the five speakers 1, 2, 3, 4, and 5, which serve as output channels. In this embodiment, the reference orientation is the orientation from the listener L toward the speaker 1, which serves as the central speaker, for example, the 0 o'clock orientation, but it is not limited to this. The 6 o'clock orientation is behind the 0 o'clock orientation, as long as the 6 o'clock orientation behind the reference orientation is included in the first range D1. Furthermore, the first range D1 is the range from the 3 o'clock orientation to the 9 o'clock orientation (i.e., a range with an angle of 180°), but it is not limited to this. In addition, the reference orientation is constant regardless of the orientation of the listener L's head, so the first range D1 is constant regardless of the orientation of the listener L's head.

[0077] The first sound is the sound that reaches the listener L from all or part of the extended first range D1, and is called ambient sound or noise. Furthermore, the first sound may also be referred to as surrounding sound. In this embodiment, the first sound is ambient sound that reaches the listener L from the entire area of ​​the first range D1. Here, the first sound is from... Figure 2 The area with the dotted pattern in the image reaches the listener L's sound as a whole.

[0078] The second audio signal is a signal representing the second sound, which is the sound that arrives at the listener L from a specified location.

[0079] The second sound, for example, is a sound image located in... Figure 2 The sound at the black dot shown. Furthermore, the second sound can also be a sound that reaches the listener L from a narrower range than the first sound. As an example, the second sound is the so-called target sound, which refers to the sound that the listener L primarily hears. Furthermore, the target sound can also be a sound other than ambient sound.

[0080] In addition, such as Figure 2 As shown, in this embodiment, the designated direction is 5 o'clock, and the arrow indicates that the second sound arrives at the listener L from the designated direction. Furthermore, the designated direction remains constant regardless of the direction in which the listener L's head is facing.

[0081] The first signal processing unit 110 will be described again.

[0082] Furthermore, the first signal processing unit 110 performs the process of separating multiple audio signals into a first audio signal and a second audio signal. The first signal processing unit 110 outputs the separated first audio signal to the first decoding unit 121 and outputs the separated second audio signal to the second decoding unit 122. In this embodiment, as an example, the first signal processing unit 110 is a multiplexer, but it is not limited to this.

[0083] Furthermore, in this embodiment, the multiple audio signals acquired by the first signal processing unit 110 are encoded using MPEG-H 3D audio (ISO / IEC 23008-3) (hereinafter referred to as MPEG-H 3D audio). That is, the first signal processing unit 110 acquires multiple audio signals as a bitstream to be encoded.

[0084] As an example of a signal acquisition unit, the first decoding unit 121 and the second decoding unit 122 acquire multiple audio signals. Specifically, the first decoding unit 121 acquires and decodes a first audio signal separated by the first signal processing unit 110. The second decoding unit 122 acquires and decodes a second audio signal separated by the first signal processing unit 110. The first decoding unit 121 and the second decoding unit 122 perform decoding processing based on the aforementioned MPEG-H 3D audio, etc.

[0085] The first decoding unit 121 outputs the decoded first audio signal to the first correction processing unit 131, and the second decoding unit 122 outputs the decoded second audio signal to the second correction processing unit 132.

[0086] Furthermore, the first decoding unit 121 outputs first information to the information acquisition unit 140, the first information being information indicating the first range D1 contained in the first audio signal. The second decoding unit 122 outputs second information to the information acquisition unit 140, the second information being information indicating the location, i.e., the predetermined location, of the second sound contained in the second audio signal reaching the listener L.

[0087] The information acquisition unit 140 is a processing unit that acquires the orientation information output from the head sensor 300. Furthermore, the information acquisition unit 140 acquires first information output from the first decoding unit 121 and second information output from the second decoding unit 122. The information acquisition unit 140 outputs the acquired orientation information, first information, and second information to the first correction processing unit 131 and the second correction processing unit 132.

[0088] The first correction processing unit 131 and the second correction processing unit 132 are examples of correction processing units. The correction processing unit is a processing unit that performs correction processing on at least one of the first audio signal and the second audio signal.

[0089] The first correction processing unit 131 acquires the first audio signal acquired by the first decoding unit 121, the location information acquired by the information acquisition unit 140, the first information, and the second information. The second correction processing unit 132 acquires the second audio signal acquired by the second decoding unit 122, and the location information, the first information, and the second information acquired by the information acquisition unit 140.

[0090] The correction processing units (first correction processing unit 131 and second correction processing unit 132) based on the acquired orientation information, under specified conditions (in... Figures 3-6 (As described later), at least one of the first audio signal and the second audio signal is corrected. More specifically, the first correction processing unit 131 performs correction processing on the first audio signal, and the second correction processing unit 132 performs correction processing on the second audio signal.

[0091] Here, when the first audio signal and the second audio signal have been corrected, the first correction processing unit 131 outputs the corrected first audio signal to the mixing processing unit 150, and the second correction processing unit 132 outputs the corrected second audio signal to the mixing processing unit 150.

[0092] Furthermore, when the first audio signal has been corrected, the first correction processing unit 131 outputs the corrected first audio signal to the mixing processing unit 150, and the second correction processing unit 132 outputs the second audio signal that has not been corrected to the mixing processing unit 150.

[0093] Furthermore, when the second audio signal has been corrected, the first correction processing unit 131 outputs the first audio signal without correction processing to the mixing processing unit 150, and the second correction processing unit 132 outputs the second audio signal after correction processing to the mixing processing unit 150.

[0094] The mixing processing unit 150 is a processing unit that mixes at least one of the first audio signal and the second audio signal after the correction processing unit has performed correction processing and outputs it to a plurality of speakers 1, 2, 3, 4 and 5 as output channels.

[0095] More specifically, when correction processing is applied to the first audio signal and the second audio signal, the mixing processing unit 150 mixes the corrected first audio signal and the second audio signal and outputs them. When correction processing is applied to the first audio signal, the mixing processing unit 150 mixes the corrected first audio signal and the uncorrected second audio signal and outputs them. When correction processing is applied to the second audio signal, the mixing processing unit 150 mixes the uncorrected first audio signal and the corrected second audio signal and outputs them.

[0096] In another example, instead of using multiple speakers 1, 2, 3, 4, and 5 arranged around the listener L, an earphone arranged next to the listener L's ear is used as the output channel. In this case, when mixing the first audio signal and the second audio signal, the mixing processing unit 150 performs convolution processing on the head-related transfer function and outputs it.

[0097] [Action Example] The following describes an example of the operation of the sound reproduction method performed by the sound reproduction device 100. Figure 3 This is a flowchart illustrating an example of the operation of the sound reproduction device 100 according to this embodiment.

[0098] The first signal processing unit 110 acquires multiple audio signals (S10).

[0099] The first signal processing unit 110 separates the multiple audio signals obtained by the first signal processing unit 110 into a first audio signal and a second audio signal (S20).

[0100] The first decoding unit 121 and the second decoding unit 122 acquire the separated first audio signal and second audio signal, respectively (S30). Step S30 is a signal acquisition step. More specifically, the first decoding unit 121 acquires the first audio signal, and the second decoding unit 122 acquires the second audio signal. Furthermore, the first decoding unit 121 decodes the first audio signal, and the second decoding unit 122 decodes the second audio signal.

[0101] Here, the information acquisition unit 140 acquires the orientation information output by the head sensor 300 (S40). Step S40 is the information acquisition step. In addition, the information acquisition unit 140 acquires first information and second information. The first information indicates a first range D1, which is the range contained by the first audio signal of the first sound. The second information indicates the orientation of the second sound reaching the listener L, i.e., the specified orientation.

[0102] Then, the information acquisition unit 140 outputs the acquired location information, first information and second information to the first correction processing unit 131 and the second correction processing unit 132 (i.e., correction processing units).

[0103] The correction processing unit acquires the first audio signal, the second audio signal, azimuth information, first information, and second information. Then, based on the azimuth information, the correction processing unit determines whether the first range D1 and the specified azimuth are included in the second range D2 (S50). More specifically, the correction processing unit performs the above determination based on the acquired azimuth information, first information, and second information.

[0104] Here, using Figures 4-6 The judgment made by the correction processing unit and the second scope D2 are explained.

[0105] Figures 4-6 This is a schematic diagram illustrating an example of the judgment made by the correction processing unit in this embodiment. More specifically, in Figure 4 and Figure 5 In the middle, the correction processing unit determines that the first range D1 and the specified orientation are included in the second range D2. Figure 6 In the middle, the correction processing unit determined that the first range D1 and the specified orientation are not included in the second range D2. Furthermore, according to... Figure 4 , Figure 5 and Figure 6 The order indicates the direction in which the listener L's head is facing, changing clockwise.

[0106] like Figures 4-6 As shown, the second range D2 is the range behind the listener L when the direction the listener L's head is facing is taken as the front. In other words, the second range D2 is the range behind the listener L. Furthermore, the second range D2 is the range centered on the direction directly opposite the direction the listener L's head is facing. As an example, such as... Figure 4 As shown, when the listener L's head is facing at 0 o'clock, the second range D2 is a range from 4 o'clock to 8 o'clock centered on the opposite direction to 0 o'clock, i.e., 6 o'clock (i.e., a range with an angle of 120°). However, the second range D2 is not limited to this. Furthermore, the second range D2 is set based on the orientation information acquired by the information acquisition unit 140. Additionally, as... Figures 4-6 As shown, if the orientation of the listener L's head changes, the second range D2 changes accordingly, but the first range D1 and the specified orientation remain unchanged as described above.

[0107] That is, the correction processing unit determines whether the first range D1 and the specified orientation are included in the second range D2, which is the range behind the listener L, determined based on the orientation information. The specific positional relationship between the first range D1, the specified orientation, and the second range D2 will be explained below.

[0108] First, such as Figure 4 and Figure 5As shown, the case where the correction processing unit determines that the first range D1 and the specified orientation are both included in the second range D2 ("Yes" in step S50) will be explained.

[0109] In such Figure 4 When the head orientation of the listener L is shown as 0 o'clock, the second range D2 is the range from 4 o'clock to 8 o'clock. Furthermore, the first range D1, which relates to the first sound (ambient sound), is the range from 3 o'clock to 9 o'clock, and the prescribed orientation related to the second sound (target sound) is 5 o'clock. That is, the prescribed orientation is included in a portion of the first range D1, and this portion of the first range D1 is included in the second range D2. At this time, the correction processing unit determines that both the first range D1 and the prescribed orientation are included in the second range D2. Therefore, the first sound and the second sound are sounds arriving at the listener L from the second range D2 (behind the listener L).

[0110] Furthermore, in such Figure 5 The orientation of the listener L's head shown is relative to the direction of the head. Figure 4 The same applies when the movement is clockwise.

[0111] exist Figure 4 and Figure 5 In the indicated case, the correction processing unit performs correction processing on at least one of the first audio signal and the second audio signal. Here, as an example, the correction processing unit performs correction processing on both the first audio signal and the second audio signal (S60). More specifically, the first correction processing unit 131 performs correction processing on the first audio signal, and the second correction processing unit 132 performs correction processing on the second audio signal. Step S60 is a correction processing step.

[0112] Furthermore, the correction processing performed by the correction processing unit is a process that increases the intensity of the second audio signal relative to the intensity of the first audio signal. "Increased intensity of the audio signal" refers, for example, to an increase in the volume or sound pressure level of the sound represented by the audio signal. Details regarding the correction processing will be explained in Examples 1 through 3 below.

[0113] The first correction processing unit 131 outputs the first audio signal after correction processing to the mixing processing unit 150, and the second correction processing unit 132 outputs the second audio signal after correction processing to the mixing processing unit 150.

[0114] The mixing processing unit 150 mixes the first audio signal and the second audio signal after the correction processing unit has performed correction processing, and outputs them to multiple speakers 1, 2, 3, 4 and 5 as output channels (S70). Step S70 is the mixing processing step.

[0115] Next, as Figure 6 As shown, the case where the correction processing unit determines that the first range D1 and the specified orientation are not included in the second range D2 ("No" in step S50) will be explained.

[0116] In such Figure 6 If the listener L's head is facing at 2 o'clock, then the second range D2 is the range from 6 o'clock to 10 o'clock. Furthermore, the first range D1 and the specified orientation are relative to... Figure 4 and Figure 5 There is no change. At this point, the correction processing unit determines that the specified orientation is not included in the second range D2. More specifically, the correction processing unit determines that at least one of the first range D1 and the specified orientation is not included in the second range D2.

[0117] exist Figure 6 In the case shown, the correction processing unit does not perform correction processing on the first audio signal and the second audio signal (S80). The first correction processing unit 131 outputs the first audio signal without correction processing to the mixing processing unit 150, and the second correction processing unit 132 outputs the second audio signal without correction processing to the mixing processing unit 150.

[0118] The mixing processing unit 150 mixes the first audio signal and the second audio signal that have not been corrected by the correction processing unit and outputs them to the multiple speakers 1, 2, 3, 4 and 5, which serve as output channels (S90).

[0119] Thus, in this embodiment, when the correction processing unit determines that the first range D1 and the predetermined orientation are included in the second range D2, the correction processing unit performs correction processing on at least one of the first audio signal and the second audio signal. This correction processing increases the intensity of the second audio signal relative to the intensity of the first audio signal.

[0120] Therefore, when the first range D1 and the specified orientation are included in the second range D2, the intensity of the second audio signal representing the second sound becomes stronger. Thus, the listener L can easily hear the second sound arriving from behind (i.e., behind the listener L) when the direction in which the listener L's head is facing is considered forward. In other words, an audio reproduction device 100 and an audio reproduction method are realized that can improve the perception level of the second sound arriving from behind the listener L.

[0121] As an example, when the first sound is an ambient sound and the second sound is the target sound, it is possible to suppress the target sound from being masked by the ambient sound. That is, an audio reproduction device 100 is realized that can improve the perception level of the target sound arriving from behind the listener L.

[0122] Furthermore, the first range D1 is the range behind the reference orientation determined by the positions of the five speakers 1, 2, 3, 4 and 5.

[0123] Therefore, when the first sound reaches the listener L from behind the reference direction, the listener L is also more likely to hear the second sound that reaches the listener L from behind.

[0124] Here, we will describe the first to third examples of the correction processing performed by the correction processing department.

[0125] <Example 1> In the first example, the correction process is a process of correcting at least one of the gain of the first audio signal obtained by the first decoding unit 121 and the gain of the second audio signal obtained by the second decoding unit 122. More specifically, the correction process is at least one of a process of decreasing the gain of the first audio signal and a process of increasing the gain of the second audio signal.

[0126] Figure 7 This diagram illustrates an example of the correction processing performed by the correction processing unit in this embodiment. More specifically, Figure 7 (a) is a graph showing the relationship between the time and amplitude of the first and second audio signals before the correction processing was implemented. Additionally, in Figure 7 The first range D1 and multiple speakers 1, 2, 3, 4 and 5 are omitted from the text and will be discussed later. Figure 8 and Figure 9 The same applies to China.

[0127] exist Figure 7 (b) illustrates an example where no correction processing was applied to the first and second audio signals. Figure 7 The positional relationship between the first range D1, the specified orientation, and the second range D2, as shown in (b), is equivalent to... Figure 6 That is, in Figure 7 (b) indicates that in Figure 3 The case where step S50 is "No" indicates that the correction processing unit does not perform correction processing on the first audio signal and the second audio signal.

[0128] exist Figure 7 (c) shows an example of corrective processing applied to the first and second audio signals. Figure 7 The positional relationship between the first range D1, the specified orientation, and the second range D2 shown in (c) is equivalent to Figure 4 That is, in Figure 7 (c) indicates that in Figure 3 The case where "Yes" is shown in step S50.

[0129] In this case, the correction processing unit performs at least one of the following correction processes: reducing the gain of the first audio signal and increasing the gain of the second audio signal. Specifically, the correction processing unit performs correction processes that reduce the gain of the first audio signal and increase the gain of the second audio signal. Thus, by correcting the gains of the first and second audio signals, as... Figure 7 As shown, the amplitudes of the first and second audio signals are corrected. Specifically, the correction processing unit performs both processes: reducing the amplitude of the first audio signal representing the first sound and increasing the amplitude of the second audio signal representing the second sound. Therefore, listener L can more easily hear the second sound.

[0130] In the first example, the correction process is a process of correcting the gain of at least one of the first audio signal and the second audio signal. As a result, the amplitude of at least one of the first audio signal representing the first sound and the second audio signal representing the second sound is corrected, making it easier for the listener L to hear the second sound.

[0131] More specifically, the correction process is at least one of the processes of reducing the gain of the first audio signal representing the first sound and increasing the gain of the second audio signal representing the second sound. As a result, the listener L can hear the second sound more easily.

[0132] <Example 2> In the second example, the correction process is a process of correcting at least one of the frequency components of the first audio signal obtained by the first decoding unit 121 and the frequency components of the second audio signal obtained by the second decoding unit 122. More specifically, the correction process is a process of reducing the spectrum of the frequency components of the first audio signal to be smaller than the spectrum of the frequency components of the second audio signal. Here, as an example, the correction process is a process of subtracting the spectrum of the frequency components of the second audio signal from the spectrum of the frequency components of the first audio signal.

[0133] Figure 8 This diagram illustrates another example of the correction processing performed by the correction processing unit in this embodiment. More specifically, Figure 8 (a) is a graph showing the spectrum of the frequency components of the first and second audio signals before the correction processing was performed. The spectrum of the frequency components is obtained, for example, by performing a Fourier transform on the first and second audio signals.

[0134] exist Figure 8 (b) shows an example where no correction processing was performed on the first and second audio signals. Figure 8 The positional relationship between the first range D1, the specified orientation, and the second range D2 shown in (b) is equivalent to Figure 6 That is, in Figure 8 (b) indicates Figure 3 The case where step S50 is "No" indicates that the correction processing unit does not perform correction processing on the first audio signal and the second audio signal.

[0135] exist Figure 8 (c) represents an example where a correction process was applied to the first audio signal. Figure 8 The positional relationship between the first range D1, the specified orientation, and the second range D2 shown in (c) is equivalent to Figure 4 That is, in Figure 8 (c) indicates Figure 3 The case where "Yes" is shown in step S50.

[0136] In this case, the correction processing unit (more specifically, the first correction processing unit 131) performs the process of subtracting the spectrum based on the frequency components of the second audio signal from the spectrum based on the frequency components of the first audio signal. As a result, as... Figure 8 As shown in (c), the intensity of the spectrum based on the frequency components of the first audio signal representing the first sound decreases. On the other hand, since no correction processing is applied to the second audio signal, the intensity of the spectrum based on the frequency components of the second audio signal representing the second sound remains constant. That is, the intensity of the spectrum based on a portion of the frequency components of the first audio signal decreases, while the intensity of the second audio signal remains constant. Therefore, listener L finds it easier to hear the second sound.

[0137] In the second example, the correction process involves modifying at least one of the frequency components of the first audio signal representing the first sound and the frequency components of the second audio signal representing the second sound. As a result, the listener L can more easily hear the second sound.

[0138] Furthermore, the correction process involves reducing the spectrum of the frequency components of the first audio signal to be smaller than the spectrum of the frequency components of the second audio signal. Here, the correction process is the subtraction of the spectrum of the frequency components of the second audio signal from the spectrum of the frequency components of the first audio signal. As a result, the intensity of the spectrum based on a portion of the frequency components of the first audio signal representing the first sound decreases, making it easier for the listener L to hear the second sound.

[0139] Furthermore, the correction process can also be based on reducing the spectrum of the frequency components of the first audio signal to a predetermined ratio smaller than the spectrum of the frequency components of the second audio signal. For example, the correction process can also be implemented so that the peak intensity of the spectrum of the frequency components of the second audio signal is below a predetermined ratio relative to the peak intensity of the spectrum of the frequency components of the first audio signal.

[0140] <Example 3> In the third example, the correction processing unit performs correction processing based on the positional relationship between the second range D2 and a predetermined orientation. In this case, the correction processing either corrects the gain of at least one of the first audio signal and the second audio signal, or it corrects the gain based on the frequency characteristics of the first audio signal and the frequency characteristics of the second audio signal. Here, the correction processing corrects the gain of at least one of the first audio signal and the second audio signal.

[0141] Figure 9 This diagram illustrates another example of the correction processing performed by the correction processing unit in this embodiment. More specifically, Figure 9 (a) is a graph showing the relationship between the time and amplitude of the first and second audio signals before the correction processing was implemented. Furthermore, in Figure 9 Examples (b) and (c) represent cases where at least one of the gains of the first and second audio signals has been corrected. Additionally, in Figure 9 (c) represents an example of the second sound arriving at the listener L from the 7 o'clock position.

[0142] Furthermore, in the third example, the second range D2 is divided as follows. Figure 9 As shown in (b) and (c), the second range D2 is divided into the area to the right rear of the listener L, namely the right rear range D21; the area to the left rear, namely the left rear range D23; and the area between the right rear range D21 and the left rear range D23, namely the central rear range D22. Furthermore, the central rear range D22 includes the area directly behind the listener L.

[0143] exist Figure 9 In (b), an example is shown where the correction processing unit determines that the specified orientation (here, the 5 o'clock orientation) is included in the right rear range D21. In this case, the correction processing unit performs correction processing as either reducing the gain of the first audio signal or increasing the gain of the second audio signal. Here, the correction processing unit (more specifically, the second correction processing unit 132) performs correction processing as increasing the gain of the second audio signal.

[0144] Therefore, listener L can easily hear the second sound.

[0145] In addition, although there is no illustration, the correction processing unit also performs the same correction processing in cases where the specified orientation is determined to be included in the left rear range D23.

[0146] In addition, Figure 9(c) illustrates an example where the correction processing unit determines that the specified orientation (here, the 7 o'clock orientation) is included within the central rear range D22. In this case, the correction processing unit performs correction processing that decreases the gain of the first audio signal and increases the gain of the second audio signal. Here, the first correction processing unit 131 performs correction processing that decreases the gain of the first audio signal, and the second correction processing unit 132 performs correction processing that increases the gain of the second audio signal. As a result, the correction is that the amplitude of the first audio signal decreases and the amplitude of the second audio signal increases.

[0147] Therefore, listener L and Figure 9 Compared to the example represented by (b), the second voice is easier to hear.

[0148] As mentioned above, people have a low level of perception for sounds arriving from behind them. Furthermore, the closer the sound is to directly behind the person, the lower the level of perception for that sound.

[0149] Therefore, the correction process shown in Example 3 is performed. That is, the correction process is performed based on the positional relationship between the second range D2 and the specified orientation. More specifically, when the specified orientation is contained within the central rear range D22, which includes the orientation directly behind the listener L, the following correction process is performed. In this case, compared to the case where the specified orientation is contained within the right rear range D21, a correction process is performed in which the intensity of the second audio signal representing the second sound becomes stronger than the intensity of the first audio signal representing the first sound. Therefore, the listener L can hear the second sound more easily.

[0150] [Details of the correction process] Furthermore, using Figure 10 and Figure 11 This section details how the correction processing unit performs correction processing on the first audio signal representing the first sound.

[0151] Figure 10 This is a schematic diagram illustrating an example of the correction processing performed on the first audio signal in this embodiment. Figure 11 This is a schematic diagram illustrating another example of the correction processing performed on the first audio signal in this embodiment. Additionally, in Figure 10 and Figure 11 In, with Figure 2 Similarly, the direction in which the listener L's head is facing is the 0 position.

[0152] In the first to third examples described above, the correction processing unit may also perform correction processing on the first audio signal representing a portion of the first sound, as shown below.

[0153] For example, such as Figure 10 As shown, the correction processing unit performs correction processing on the first audio signal representing the sound from the entire range of the second range D2 to the listener L in the first sound. The sound from the entire range of the second range D2 to the listener L in the first sound is... Figure 10 The overall sound of the area with light dots in the first sound reaches the listener L. Additionally, the other sounds in the first sound originate from... Figure 10 The overall sound of the area with concentrated highlights reaches the listener L.

[0154] In this case, the correction processing unit may, for example, implement a correction processing that reduces the gain of the first audio signal representing the sound from the entire range of the second range D2 to the listener L in the first sound.

[0155] In addition, for example, Figure 11 As shown, the correction processing unit performs correction processing on the first audio signal representing the sounds arriving at listener L from a predetermined direction from the second sound in the first sound. For example... Figure 11 As shown, as an example, the area around the specified orientation is a range D11 of approximately 30° centered on the specified orientation, but it is not limited to this.

[0156] Furthermore, the sound in the first sound that reaches the listener L from the surrounding area of ​​the specified direction is from... Figure 11 The overall sound of the area with light dots in the first sound reaches the listener L. Additionally, the other sounds in the first sound originate from... Figure 11 The overall sound of the area with concentrated highlights reaches the listener L.

[0157] In this case, the correction processing unit may, for example, perform a correction processing that reduces the gain of the first audio signal representing the sound arriving at the listener L from a predetermined direction from the second sound in the first sound.

[0158] In this way, correction processing can also be performed on the first audio signal representing a portion of the first sound. As a result, it is no longer necessary to perform correction processing on the entire first audio signal, thus reducing the processing load of the first correction processing unit 131 that corrects the first audio signal.

[0159] Alternatively, the same processing can be applied to the first audio signal representing all the sounds in the first sound.

[0160] (Implementation Method 2) Next, the sound reproduction device 100a of Embodiment 2 will be described.

[0161] Figure 12This is a block diagram showing the functional structure of the sound reproduction device 100a and the sound acquisition device 200 in this embodiment.

[0162] In this embodiment, the sound collected by the sound collection device 500 is output from multiple speakers 1, 2, 3, 4, and 5 via the sound acquisition device 200 and the sound reproduction device 100a. More specifically, the sound acquisition device 200 acquires multiple audio signals based on the sound collected by the sound collection device 500 and outputs them to the sound reproduction device 100a. The sound reproduction device 100a acquires the multiple audio signals output by the sound acquisition device 200 and outputs them to the multiple speakers 1, 2, 3, 4, and 5.

[0163] The sound collecting device 500 is a device that collects sound arriving at it; for example, it is a microphone. The sound collecting device 500 can also be directional. Therefore, the sound collecting device 500 can collect sound from a specific direction. The sound collecting device 500 uses an A / D converter to convert the collected sound and outputs it as an audio signal to the sound acquisition device 200. Alternatively, multiple sound collecting devices 500 can be provided.

[0164] use Figure 13 The sound collection device 500 will be described in more detail.

[0165] Figure 13 This is a schematic diagram illustrating sound collection performed by the sound collection device 500 according to this embodiment.

[0166] exist Figure 13 In, with Figure 2 Similarly, 0 o'clock, 3 o'clock, 6 o'clock, and 9 o'clock are indicated to show the direction, corresponding to the time displayed on the clock face. The sound collecting device 500 is located at the center (also called the origin) of the clock face and collects the sound reaching the sound collecting device 500. Hereinafter, the connection between the sound collecting device 500 and the 0 o'clock direction will be referred to as "0 o'clock direction", and the same applies to the other times displayed on the clock face.

[0167] The sound collection device 500 collects multiple first sounds and second sounds.

[0168] Here, the sound collection device 500 collects four first sounds as multiple first sounds. Additionally, for identification, such as... Figure 13 As shown, they are denoted as the first sound A, the first sound B-1, the first sound B-2, and the first sound B-3.

[0169] The sound collecting device 500 is capable of collecting sound from a specific direction, so as an example, such as Figure 13As shown, the area surrounding the sound collection device 500 is divided into four parts, and sound is collected in each of these parts. Here, the area surrounding the sound collection device 500 is divided into four parts: the area from 0 o'clock to 3 o'clock, the area from 3 o'clock to 6 o'clock, the area from 6 o'clock to 9 o'clock, and the area from 9 o'clock to 0 o'clock.

[0170] In this embodiment, the multiple first sounds are sounds that arrive at the sound collection device 500 from a predetermined angular range, i.e., a first range D1, that is, sounds that are respectively picked up by the sound collection device 500 from multiple first ranges D1. In addition, the first range D1 corresponds to one of the four ranges.

[0171] Specifically, such as Figure 13 As shown, the first sound A is the sound that arrives at the sound collection device 500 from the first range D1, which is the range from 0 o'clock to 3 o'clock. That is, the first sound A is the sound collected from this first range D1. Similarly, the first sounds B-1, B-2, and B-3 are the sounds that arrive at the sound collection device 500 from the first range D1, which is the range from 3 o'clock to 6 o'clock, from 6 o'clock to 9 o'clock, and from 9 o'clock to 0 o'clock, respectively. That is, the first sounds B-1, B-2, and B-3 are the sounds collected from these three first ranges D1 respectively. In addition, there is a case where the first sounds B-1, B-2, and B-3 are collectively referred to as the first sound B.

[0172] Furthermore, the first sound A here is from Figure 13 The entire area marked with a diagonal line in the text reaches the listener L. Similarly, the first sound B-1, the first sound B-2, and the first sound B-3 are from... Figure 13 The entire area marked with dots in the text reaches the listener L's sound. Regarding... Figure 14 The same applies.

[0173] The second sound is the sound that arrives at the sound collection device 500 from a predetermined location (here, the 5 o'clock position). The second sound is also collected in the same manner as the multiple first sounds, according to each segmented area.

[0174] Furthermore, the relationship between the sound collected by the sound collection device 500 and the sound output from the multiple speakers 1, 2, 3, 4, and 5 will be explained. The multiple speakers 1, 2, 3, 4, and 5 output sound to reproduce the sound collected by the sound collection device 500. That is, in this embodiment, since both the listener L and the sound collection device 500 are positioned at the origin, the second sound arriving at the sound collection device 500 from a predetermined direction is heard by the listener L as the sound arriving at the listener L from the predetermined direction. Similarly, the first sound A arriving at the sound collection device 500 from the first range D1 (the range from the 0 o'clock position to the 3 o'clock position) is heard by the listener L as the sound arriving at the listener L from the first range D1.

[0175] The sound collection device 500 outputs multiple audio signals to the sound acquisition device 200. These multiple audio signals include multiple first audio signals representing multiple first sounds and second audio signals representing second sounds. Furthermore, the multiple first audio signals include a first audio signal representing first sound A and a first audio signal representing first sound B. More specifically, the first audio signal representing first sound B includes three first audio signals representing first sound B-1, first sound B-2, and first sound B-3, respectively.

[0176] The sound acquisition device 200 acquires multiple audio signals output by the sound collection device 500. Additionally, the sound acquisition device 200 can also acquire classification information at this time.

[0177] The classification information is information that classifies multiple first audio signals based on their respective frequency characteristics. That is, in the classification information, multiple first audio signals are classified into different groups according to their respective frequency characteristics.

[0178] In this embodiment, the first sound A and the first sound B are sounds of different types and have different frequency characteristics. Therefore, the first audio signal representing the first sound A and the first audio signal representing the first sound B are classified into different groups.

[0179] That is, the first audio signal representing the first sound A is classified into one group, and the three first audio signals representing the first sound B-1, the first sound B-2, and the first sound B-3 are classified into another group.

[0180] Alternatively, instead of the audio acquisition device 200 acquiring the classification information, the audio acquisition device 200 may generate classification information based on the acquired multiple audio signals. That is, the classification information can also be generated by... Figure 13 The processing unit of the audio acquisition device 200, which is not shown in the figure, generates the audio.

[0181] Next, the constituent elements of the sound acquisition device 200 will be described. For example... Figure 12 As shown, the audio acquisition device 200 is a device that includes an encoding unit (a plurality of first encoding units 221 and second encoding units 222) and a second signal processing unit 210.

[0182] The encoding unit (multiple first encoding units 221 and second encoding units 222) acquires multiple audio signals and classification information output by the sound collection device 500. The encoding unit encodes the multiple audio signals after acquisition. More specifically, the multiple first encoding units 221 acquire and encode multiple first audio signals, and the second encoding unit 222 acquires and encodes second audio signals. The multiple first encoding units 221 and second encoding units 222 perform encoding processing based on the aforementioned MPEG-H 3D audio, etc.

[0183] Here, each of the multiple first encoding units 221 can be established in a one-to-one correspondence with each of the multiple first audio signals classified into different groups as indicated by the classification information. Each of the multiple first encoding units 221 encodes the multiple first audio signals into which the corresponding units are established. For example, the classification information indicates two groups (the group to which the first audio signal of the first sound A is classified and the group to which the first audio signal of the first sound B is classified). Therefore, two first encoding units 221 are provided here, one of which encodes the first audio signal representing the first sound A, and the other of which encodes the first audio signal representing the first sound B. In addition, when the audio acquisition device 200 is equipped with one first encoding unit 221, this one first encoding unit 221 acquires and encodes multiple first audio signals.

[0184] The encoding unit outputs the encoded first audio signals and the encoded second audio signals, along with classification information, to the second signal processing unit 210.

[0185] The second signal processing unit 210 acquires the encoded plurality of first audio signals and the encoded second audio signal, as well as classification information. The second signal processing unit 210 combines the encoded plurality of first audio signals and the encoded second audio signal into a single encoded plurality of audio signals. The encoded plurality of audio signals are so-called multiplexed plurality of audio signals. Furthermore, in this embodiment, the second signal processing unit 210 is, for example, a multiplexer, but it is not limited to this.

[0186] The second signal processing unit 210 outputs multiple audio signals and classification information as encoded bit streams to the sound reproduction device 100a (more specifically, the first signal processing unit 110).

[0187] Regarding the processing performed by the audio reproduction device 100a below, the main differences from Embodiment 1 will be explained. Furthermore, in this embodiment, the audio reproduction device 100a includes a plurality of first decoding units 121, unlike Embodiment 1.

[0188] The first signal processing unit 110 acquires multiple output audio signals and classification information, and performs processing to separate the multiple audio signals into multiple first audio signals and second audio signals. The first signal processing unit 110 outputs the separated multiple first audio signals and classification information to multiple first decoding units 121, and outputs the separated second audio signals and classification information to second decoding units 122.

[0189] Multiple first decoding units 121 acquire multiple first audio signals separated by the first signal processing unit 110 and decode them.

[0190] Here, each of the multiple first decoding units 121 can be established in a one-to-one correspondence with each of the multiple first audio signals classified into different groups represented by classification information. Each of the multiple first decoding units 121 decodes the multiple first audio signals for which a correspondence has been established. Similar to the first encoding unit 221 described above, two first decoding units 121 are provided here. One of the two first decoding units 121 decodes the first audio signal representing the first sound A, and the other of the two first decoding units 121 decodes the first audio signal representing the first sound B. In addition, when the sound reproduction device 100a is equipped with one first decoding unit 121, this one first decoding unit 121 acquires and decodes multiple first audio signals.

[0191] Multiple first decoding units 121 output the decoded multiple first audio signals and classification information to the first correction processing unit 131. In addition, the second decoding unit 122 outputs the decoded second audio signals and classification information to the second correction processing unit 132.

[0192] Furthermore, the first correction processing unit 131 acquires multiple first audio signals and classification information acquired by multiple first decoding units 121, as well as location information, first information and second information acquired by the information acquisition unit 140.

[0193] Similarly, the second correction processing unit 132 acquires the second audio signal and classification information acquired by the second decoding unit 122, as well as the location information, first information and second information acquired by the information acquisition unit 140.

[0194] In addition, the first information related to this embodiment includes information indicating one first range D1 related to the first sound A and three first ranges D1 related to the first sound B contained in the plurality of first audio signals.

[0195] Next, use Figure 14 The correction process implemented by the correction processing department will be explained. Figure 14 This is a schematic diagram illustrating an example of the correction processing performed on a plurality of first audio signals in accordance with this embodiment. Figure 14 (a) represents the example before the corrective action was taken. Figure 14 (b) represents an example after the corrective processing has been implemented.

[0196] In this embodiment, the correction processing unit performs correction processing based on location information and classification information. Here, we will explain the case where the correction processing unit determines that one of the multiple first ranges D1 and the specified location is included in a second range D2. In this case, the correction processing unit performs correction processing on at least one of a first audio signal and a second audio signal representing a first sound arriving at the listener L from that one first range D1. More specifically, the correction processing unit performs correction processing on at least one of all first audio signals and second audio signals classified into the same group as that one first audio signal, based on the classification information.

[0197] For example, in Figure 14 In the process, the correction processing unit determines that the first range D1 (the range from 3 o'clock to 6 o'clock) and the specified direction (5 o'clock) are included in the second range D2 (the range from 4 o'clock to 8 o'clock). The sound reaching the listener L from the first range D1 is the first sound B-1. All the first audio signals classified into the same group as the first audio signal representing the first sound B-1 are the three first audio signals representing the first sound B-1, the first sound B-2, and the first sound B-3, respectively.

[0198] That is, the correction processing unit performs correction processing on at least one of the three first audio signals (in other words, the first audio signals representing the first sound B) and the second audio signal representing the first sound B-1, the first sound B-2 and the first sound B-3 respectively.

[0199] Therefore, the correction processing unit can perform correction processing for each group to which the multiple first audio signals are classified. Here, the correction processing unit can perform correction processing on the three first audio signals representing the first sound B-1, the first sound B-2, and the first sound B-3 together. Therefore, the processing load of the correction processing unit can be reduced.

[0200] (Other implementation methods) The above description, based on embodiments, outlines the sound reproduction apparatus and method relating to the technical solutions of this disclosure. However, this disclosure is not limited to these embodiments. For example, other embodiments implemented by arbitrarily combining the constituent elements described in this specification or by removing certain constituent elements may also be considered embodiments of this disclosure. Furthermore, variations derived from the above embodiments by applying various modifications conceived by those skilled in the art without departing from the spirit of this disclosure, i.e., the meaning of the language expressed in the claims, are also included in this disclosure.

[0201] Furthermore, the forms shown below may also be included within the scope of one or more technical solutions disclosed herein.

[0202] (1) A component of the aforementioned sound reproduction device may also be a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the aforementioned RAM or hard disk unit. Each device performs its function by having the microprocessor operate according to the aforementioned computer program. Here, the computer program is composed of multiple command codes representing instructions to the computer in order to achieve the specified function.

[0203] (2) A portion of the constituent elements of the aforementioned sound reproduction device and sound reproduction method may also be constituted by a single system LSI (Large Scale Integration). A system LSI is a multifunctional LSI manufactured by integrating multiple components onto a single chip; specifically, it is a computer system comprising a microprocessor, ROM, RAM, etc. The computer program is stored in the RAM. The system LSI performs its function by having the microprocessor operate according to the computer program.

[0204] (3) A component of the aforementioned audio reproduction device may also be a module consisting of an IC card or a unit that is detachable from each device. The aforementioned IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The aforementioned IC card or module may also include the aforementioned multi-functional LSI. The aforementioned IC card or module achieves its function by operating according to a computer program via a microprocessor. The IC card or module may also be tamper-resistant.

[0205] (4) Furthermore, a component of the aforementioned sound reproduction device may also be a recording medium capable of being read by a computer, such as a floppy disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc), semiconductor memory, etc. Alternatively, it may be a digital signal recorded on these recording media.

[0206] Furthermore, the computer program or digital signal that constitutes part of the aforementioned sound reproduction device can also be transmitted via electrical communication lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0207] (5) This disclosure may also be the method shown above. In addition, it may be a computer program that implements these methods by means of a computer, or a digital signal composed of the above computer program.

[0208] (6) In addition, the present disclosure may also be a computer system having a microprocessor and a memory, wherein the memory stores the computer program and the microprocessor operates according to the computer program.

[0209] (7) Alternatively, the above-mentioned program or digital signal may be recorded in the above-mentioned recording medium and transferred, or the above-mentioned program or digital signal may be transferred via the above-mentioned network, etc., by an independent computer system.

[0210] (8) The above embodiments and the above variations can also be combined separately.

[0211] In addition, although Figure 2 Although not explicitly stated, an image linked to the sound output from multiple speakers 1, 2, 3, 4, and 5 can be presented to the listener L. In this case, for example, a display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel can be placed around the listener L, and the image can be displayed on that device. Alternatively, the image can be presented by the listener L wearing a head-mounted display or similar device.

[0212] Furthermore, in the above embodiments, such as Figure 2 As shown, five speakers 1, 2, 3, 4, and 5 are provided, but the system is not limited to this. For example, a 5.1ch surround system with speakers 1, 2, 3, 4, and 5 corresponding to a subwoofer can also be used. Furthermore, a multi-channel surround system with two speakers can also be used, but the system is not limited to these.

[0213] Industrial availability This disclosure can be used in sound reproduction devices and sound reproduction methods, and is particularly applicable to stereo sound reproduction systems, etc.

[0214] Label Explanation 1, 2, 3, 4, 5 speakers 100, 100a Sound Reproduction Device 110 First Signal Processing Unit 121 First Decoding Department 122 Second Decoding Department 131 First Correction Processing Department 132 Second Correction Processing Department 140 Information Acquisition Department 150 Mixing Processing Section 200 Sound Acquisition Device 210 Second Signal Processing Unit 221 First Coding Section 222 Second Coding Section 300 head sensors 500 sound collection devices D1, first range D2 Second Range D11 range D21 Right rear area D22 Central Rear Area D23 Left rear area L Listener

Claims

1. A method for sound reproduction, wherein, include: The signal acquisition step involves acquiring multiple first audio signals and classification information. The first audio signal represents the sound that arrives at the listener from a first range defined as an angular range, i.e., the first sound. The classification information is information on the classification of the multiple first audio signals. as well as The correction processing step involves, in the case where one of the plurality of first sounds is determined to be a trigger sound, performing correction processing on the first audio signal representing the determined trigger sound and one or more first audio signals representing one or more first sounds classified into the same group as the trigger sound, based on the obtained classification information.

2. The sound reproduction method according to claim 1, wherein, This includes an information acquisition step, in which information about the direction in which the listener's head is facing is obtained, i.e., direction information. When the range behind is defined as the direction in which the listener's head is facing, and this range is then defined as the second range, In the correction process step, if, based on the obtained direction information, it is determined that the first range of one of the multiple first sounds is included in the second range, then one of the first sounds is determined as the trigger tone.

3. The sound reproduction method according to claim 2, wherein, In the signal acquisition step, a second audio signal is acquired, which represents the sound arriving at the listener from a predetermined direction, i.e., the second sound. In the correction process step, if, based on the obtained direction information, it is determined that the first range of one of the multiple first sounds and the specified direction are included in the second range, then one of the first sounds is determined as the trigger tone.

4. The sound reproduction method according to claim 1, wherein, The first sound is ambient sound.

5. The sound reproduction method according to claim 1, wherein, The classification information is information that classifies multiple first audio signals based on their respective frequency characteristics.

6. The sound reproduction method according to claim 1, wherein, In the signal acquisition step, a second audio signal is acquired, which represents the sound arriving at the listener from a predetermined direction, i.e., the second sound. In the correction processing step, the correction processing is performed to increase the intensity of the acquired second audio signal relative to the intensity of the first audio signal representing the trigger tone and the intensity of one or more of the first audio signals.

7. The sound reproduction method according to claim 2, wherein, The range shown in the second range varies depending on the direction in which the listener's head is facing.

8. The sound reproduction method according to claim 6, wherein, In the correction process step, the correction process is performed, which is to reduce the intensity of the first audio signal representing the trigger tone and the intensity of one or more of the first audio signals.

9. The sound reproduction method according to claim 6, wherein, The correction process is a process of correcting the gain of the first audio signal representing the trigger tone and the gain of one or more of the first audio signals.

10. The sound reproduction method according to claim 6, wherein, The correction process is a process of reducing the gain of the first audio signal representing the trigger tone and the gain of more than one of the first audio signals.

11. The sound reproduction method according to claim 6, wherein, The correction process is a process of correcting based on the frequency components of the first audio signal representing the trigger tone and based on the frequency components of one or more of the first audio signals.

12. The sound reproduction method according to claim 6, wherein, The correction process is to reduce the spectrum of the frequency components of the first audio signal representing the trigger tone and the spectrum of the frequency components of one or more of the first audio signals to be smaller than the spectrum of the frequency components of the obtained second audio signal.

13. A program product containing a computer program for causing a computer to perform the sound reproduction method according to any one of claims 1 to 12.

14. An audio reproduction device, wherein, have: The signal acquisition unit acquires multiple first audio signals and classification information, wherein the first audio signal represents the sound that arrives at the listener from a first range defined as an angular range, i.e., the first sound, and the classification information is information on the classification of the multiple first audio signals; as well as The correction processing unit, when determining one of the plurality of first sounds as a trigger tone, performs correction processing on the first audio signal representing the determined trigger tone and one or more first audio signals representing one or more first sounds classified into the same group as the trigger tone, based on the acquired classification information.

Images