Multichannel audio equipment and programs

Abstract

We provide a multi-channel audio system that can create an environment where family, friends, and other acquaintances who live geographically far away feel as if they live nearby. [Solution] Multi-channel audio devices 1a and 1b generate multi-channel audio data to make audible the sound as if the sound collected from the sounds of an acquaintance's daily life, received from another multi-channel audio device 1 via the network 5, were emitted from this virtual sound source in the virtual acoustic space, based on the sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position determined by the relative position with respect to the listening point P. Then, the generated multi-channel audio data is reproduced as a multi-channel audio signal and output from speakers 2a-1 to 2a-5 and 2b-1 to 2b-5 connected to themselves.

Description

Technical Field

[0004] ,

[0006] , , ,

[0005] , , ,

[0001] The present invention relates to a multi-channel audio device.

Background Art

[0002] Conventionally, there is known a multi-channel audio device that reproduces multi-channel audio data as audio signals of a plurality of channels and outputs the reproduced audio signals from speakers corresponding to the channels of the audio signals (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Here, the virtual sound source is set, for example, outside the room where the listening point is located (e.g., a location in a room adjacent to the room where the listening point is set, or a location beyond the window of the room where the listening point is set). Furthermore, the audio data received from the predetermined device may include, for example, everyday sounds collected in a room where a family member, friend, or other acquaintance lives.

[0008] For example, the present invention is A multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, A storage means that stores the sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position identified by its relative position to the listening point, Receiving means for receiving audio data from a predetermined device via a network, The system includes a generation means that generates multi-channel audio data for making audible the sound when audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results stored in the storage means. [Effects of the Invention]

[0009] In this invention, based on the sound field analysis results in a virtual acoustic space where a listening point and a virtual sound source having a predetermined positional relationship with the listening point are arranged, multi-channel audio data is generated to make the sound audible as if audio data received from a predetermined device via a network were emitted from this virtual sound source in the virtual acoustic space, and the generated multi-channel audio data is played back and output. For example, by setting a virtual sound source outside the room, the resident of that room will perceive the sound of the audio data acquired from the predetermined device as if it were coming from nearby. Therefore, according to this invention, by receiving sounds of daily life collected in the room where family, friends, or other acquaintances live from a predetermined device via a network, it is possible to realize an environment in which the user feels as if family, friends, or other acquaintances living geographically far away are living nearby in their daily life. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic diagram of a multi-channel audio system according to one embodiment of the present invention. [Figure 2] Figure 2 is a sequence diagram illustrating the operation of a multi-channel audio system according to one embodiment of the present invention. [Figure 3] Figure 3 is a sequence diagram illustrating the operation of a multi-channel audio system according to one embodiment of the present invention, and is a continuation of Figure 2. [Figure 4] Figure 4 is a sequence diagram illustrating the operation of a multi-channel audio system according to one embodiment of the present invention, and is a continuation of Figure 3. [Figure 5] Figure 5 is a schematic diagram of the functional configuration of the multi-channel audio device 1. [Figure 6] Figure 6 is a flowchart illustrating the virtual playback process of the multi-channel audio device 1. [Figure 7]Figure 7 is a flowchart illustrating the sound collection process of the multi-channel audio device 1. [Figure 8] Figure 8 is a flowchart illustrating the content playback process of the multi-channel audio device 1. [Figure 9] Figure 9 is a schematic diagram of the functional configuration of the wireless terminal 3. [Figure 10] Figure 10 is a flowchart illustrating the operation of the wireless terminal 3. [Figure 11] Figure 11(A) shows an example of a virtual living room selection screen, and Figure 11(B) shows an example of a wall type selection screen. [Modes for carrying out the invention]

[0011] An embodiment of the present invention will be described below with reference to the drawings.

[0012] Figure 1 is a schematic diagram of the multi-channel audio system according to this embodiment.

[0013] As shown in the figure, the multi-channel audio system according to this embodiment is installed in two geographically separated rooms, A and B. Room A is the actual living room of user A, and the multi-channel audio system installed in room A comprises a multi-channel audio device 1a connected to a network 5 such as a WAN (Wide Area Network) or LAN (Local Area Network) via an access point 4a, a plurality of speakers 2a-1 to 2a-5, and a wireless terminal 3a that functions as a remote controller for the multi-channel audio device 1a. Room B is the actual living room of user B, and the multi-channel audio system installed in room B comprises a multi-channel audio device 1b connected to the network 5 via an access point 4b, a plurality of speakers 2b-1 to 2b-5, and a wireless terminal 3b that functions as a remote controller for the multi-channel audio device 1b.

[0014] Hereinafter, the multi-channel audio devices 1a and 1b are simply referred to as the multi-channel audio device 1, the speakers 2a-1 to 2a-5 and 2b-1 to 2b-5 are simply referred to as the speakers 2-1 to 2-5 or the speaker 2, the wireless terminals 3a and 3b are simply referred to as the wireless terminal 3, and the access points 4a and 4b are simply referred to as the access point 4.

[0015] The multi-channel audio device 1 downloads multi-channel audio data of music from the media server 6 via the access point 4 and the network 5. Then, the downloaded multi-channel audio data is reproduced as audio signals of the center channel (hereinafter referred to as Cch), the front left channel (hereinafter referred to as FLch), the front right channel (hereinafter referred to as FRch), the surround left channel (hereinafter referred to as SLch), and the surround right channel (hereinafter referred to as SRch), and the audio signals of each reproduced channel are output from the corresponding channel speaker 2.

[0016] Also, the multi-channel audio device 1 receives sound collection data of ambient sounds from other multi-channel audio devices 1 via the access point 4 and the network 5. Then, for the listening point P located within the area surrounded by the speakers 2-1 to 2-5 arranged in the room, in a virtual acoustic space where a virtual sound source is set at a predetermined position outside the area surrounded by the speakers 2-1 to 2-5, multi-channel audio data for visualizing the sound when the received audio data is emitted from the virtual sound source is generated. Then, the generated multi-channel audio data is reproduced as audio signals of Cch, FLch, FRch, SLch, and SRch, and the audio signals of each reproduced channel are output to the corresponding channel speaker 2.

[0017] Furthermore, the multi-channel audio device 1 acquires sound data of ambient sounds collected by a wireless terminal 3 that functions as its own remote controller. It then transmits the ambient sound data to other multi-channel audio devices 1 via the access point 4 and network 5.

[0018] Speaker 2 is provided to correspond to each of the multiple channels of audio signals reproduced by the multi-channel audio device 1, and emits sound according to the audio signal of its corresponding channel output from the multi-channel audio device 1.

[0019] Here, speaker 2-1 is provided for channel C, speaker 2-2 for channel FL, speaker 2-3 for channel FR, speaker 2-4 for channel SL, and speaker 2-5 for channel SR.

[0020] Wireless terminal 3 is wirelessly connected to multi-channel audio device 1 and functions as a remote controller for multi-channel audio device 1 as described above. Wireless terminal 3 also collects ambient sounds and transmits the collected sound data to multi-channel audio device 1.

[0021] Figures 2 to 4 are sequence diagrams illustrating the operation of the multi-channel audio system according to this embodiment.

[0022] In User A's actual living room A, when the wireless terminal 3a receives a virtual playback instruction from User A to create a sound field in which the sounds of User B's actual living room B appear as if they are coming from the room next to Room A (S100), it displays a selection screen (virtual living room selection screen) for User A to select a virtual adjacent room from among a plurality of virtual adjacent rooms defined as rooms virtually adjacent to Room A to be set as User B's virtual living room, and a selection screen (wall type selection screen) for User A to select the type of virtual wall (wall type) separating Room A and the virtual living room, and accepts the selection of the virtual living room and wall type from User A (S101). Then, the wireless terminal 3a transmits a virtual playback request to the multi-channel audio device 1a, along with the specification of the virtual living room and wall type (S102).

[0023] Here, the multi-channel audio device 1a has pre-stored sound field data, which is the result of sound field analysis using a sound source (virtual sound source) virtually set up in each virtual adjacent room, and a receiving point on a virtual boundary line assumed to be around the listening point P of room A (set to correspond to the placement positions of speakers 2a-1 to 2a-5). In addition, the multi-channel audio device 1a has pre-stored sound attenuation rates for each type of virtual wall.

[0024] When the multi-channel audio device 1a receives a virtual playback request from the wireless terminal 3a, it identifies the sound field data stored in association with the virtual living room specified in the virtual playback request, and the attenuation rate stored in association with the wall type specified in the virtual playback request (S103). Then, the multi-channel audio device 1a transmits a sound collection request to the multi-channel audio device 1b (S104).

[0025] Next, when the multi-channel audio device 1b receives a sound collection request from the multi-channel audio device 1a, it transmits this sound collection request to the wireless terminal 3b (S105). In response, the wireless terminal 3b begins to collect sounds of daily life in user B's actual living room B (S106) and begins to transmit the collected sound data to the multi-channel audio device 1b (S107). The multi-channel audio device 1b transmits the collected sound data of daily life in room B transmitted from the wireless terminal 3b to the multi-channel audio device 1b (S108).

[0026] Next, in order to recreate a three-dimensional sound field that gives user A in room A the sensation that sounds of daily life in room B are being emitted from beyond a virtual wall, the multi-channel audio device 1a starts generating multi-channel audio data to form a three-dimensional sound field (in the virtual acoustic space, the sound is made three-dimensionally audible when sound collected data of daily life in room B is emitted into room A from a virtual sound source set in a virtual living room adjacent to room A, separated by a virtual wall) within room A, based on the sound field data and attenuation rate identified in S103, using speakers 2a-1 to 2a-5 (S109), and also starts playback and output of the generated multi-channel audio data (S110). Then, speakers 2a-1 to 2a-5 emit sound according to the audio signal of their corresponding channel output from the multi-channel audio device 1a (S111).

[0027] As a result, user A can perceive the sounds of user B's daily life, who lives in a geographically distant room B from user A, as if they were sounds of user B's daily life coming from next door to user A through the wall.

[0028] Next, in User B's actual living room B, when the wireless terminal 3b receives a virtual playback instruction from User B to create a sound field that makes it seem as if the sounds of User A's actual living room A are coming from the room next to Room B (S112), it displays a virtual living room selection screen for User B to select a virtual adjacent room from among several virtual adjacent rooms that have been virtually defined as rooms adjacent to Room B, and a wall type selection screen for User B to select the type of wall for the virtual wall separating Room B and the virtual living room, and accepts the selection of the virtual living room and wall type from User B (S113). Then, the wireless terminal 3b transmits a virtual playback request to the multi-channel audio device 1b, along with the specification of the virtual living room and wall type (S114).

[0029] Here, the multi-channel audio device 1b has pre-stored sound field data, which is the result of sound field analysis using a sound source (virtual sound source) virtually set up in each virtual adjacent room, and a receiving point on a virtual boundary line assumed to be around the listening point P in room B (set to correspond to the placement positions of speakers 2b-1 to 2b-5). In addition, the multi-channel audio device 1b has pre-stored sound attenuation rates for each type of virtual wall.

[0030] When the multi-channel audio device 1b receives a virtual playback request from the wireless terminal 3b, it identifies the sound field data stored in association with the virtual living room specified in the virtual playback request, and the attenuation rate stored in association with the wall type specified in the virtual playback request (S115). Then, the multi-channel audio device 1b sends a sound collection request to the multi-channel audio device 1a (S116). It also starts echo cancellation processing to remove the sound collection data of living sounds from room A received from the multi-channel audio device 1a from the sound collection data of living sounds from room B that is sent to the multi-channel audio device 1a (S117).

[0031] Next, when the multi-channel audio device 1a receives a sound collection request from the multi-channel audio device 1b, it transmits this request to the wireless terminal 3a (S118). In response, the wireless terminal 3a begins collecting ambient sounds from user A's actual living room A (S119) and begins transmitting the collected data to the multi-channel audio device 1a (S120). In response, the multi-channel audio device 1a begins echo cancellation processing to remove the ambient sound data from room B received from the multi-channel audio device 1b from the ambient sound data from room A transmitted from the wireless terminal 3a (S121). Then, it transmits the echo-cancelled ambient sound data from room A to the multi-channel audio device 1b (S122).

[0032] Next, in order to recreate a three-dimensional sound field that gives user B in room B the sensation that the sounds of daily life in room A are being emitted from beyond a virtual wall, the multi-channel audio device 1b, based on the sound field data and attenuation rate identified in S115, starts generating multi-channel audio data to form a three-dimensional sound field (in the virtual acoustic space, the sound is made three-dimensionally audible when the sound collected data of daily life in room A is emitted into room B from a virtual sound source set in a virtual living room adjacent to room B, separated by a virtual wall) within room B using speakers 2b-1 to 2b-5 (S123), and also starts playback and output of the generated multi-channel audio data (S124). Then, speakers 2b-1 to 2b-5 emit sound according to the audio signal of their corresponding channel output from the multi-channel audio device 1b (S125).

[0033] As a result, user B can perceive the sounds of user A's daily life, who lives in room A geographically distant from user B's actual living room B, as if they were sounds of user A's daily life coming from next door to user B through the wall.

[0034] Subsequently, in user A's actual living room A, when wireless terminal 3a receives a content playback instruction with a specified song from user A (S126), it transmits this content playback request with the specified song to multi-channel audio device 1a (S127). In response, multi-channel audio device 1a accesses media server 6 and downloads the multi-channel audio data of the song specified in the content playback request (S128), and plays this multi-channel audio data (S129). For each channel, it superimposes the audio signal played from the song's multi-channel audio data with the audio signal generated based on the sound collection data of the living sounds in room B received from multi-channel audio device 1b, and outputs it to the corresponding speakers 2a-1 to 2a-5 (S130). Speakers 2a-1 to 2a-5 then emit sound according to the audio signal of their corresponding channel output from multi-channel audio device 1a (S131).

[0035] This allows user A to listen to their desired music in an environment where the sounds of user B's daily life, who resides in a geographically distant room B, are perceived as if they were sounds coming through a wall.

[0036] Next, the details of the multi-channel audio device 1 and wireless terminal 3 that constitute the multi-channel audio system according to this embodiment will be described.

[0037] Note that an existing speaker can be used for speaker 2, so a detailed explanation of that is omitted.

[0038] First, let's explain the details of the multi-channel audio device 1.

[0039] Figure 5 is a schematic diagram of the functional configuration of the multi-channel audio device 1.

[0040] As shown in the figure, the multi-channel audio device 1 includes a wireless interface unit 100, a speaker interface unit 101, a music acquisition unit 102, a content storage unit 103, a sound field data storage unit 104, an attenuation rate storage unit 105, a request processing unit 106, a sound collection data transmission unit 107, a sound collection data reception unit 108, an echo cancellation processing unit 109, a multi-channel audio generation unit 110, a multi-channel audio playback unit 111, a superposition unit 112, and a main control unit 113.

[0041] The wireless interface unit 100 is an interface for wireless communication with other multi-channel audio devices 1, a wireless terminal 3 that functions as a remote controller for the device 1, and a media server 6 via the access point 4.

[0042] The speaker interface unit 101 is an interface for connecting speaker 2.

[0043] The music acquisition unit 102 accesses the media server 6 via the wireless interface unit 100 and downloads the multi-channel audio data of the music from the media server 6.

[0044] The content storage unit 103 stores multi-channel audio data of songs downloaded from the media server 6.

[0045] The sound field data storage unit 104 stores sound field data, which is the result of sound field analysis, for each virtual room (virtual adjacent room) defined as a room virtually adjacent to the room in which the device 1 is installed. This data includes a virtual sound source set within that virtual adjacent room (a virtual sound source located at a position determined by its relative position to the listening point P in the room in which the device 1 is installed) and a receiving point on a virtual boundary line assumed to surround the listening point P (set to correspond to the placement positions of speakers 2-1 to 2-5 installed in the room in which the device 1 is installed). For example, the sound field analysis of the virtual sound source (3D sound field calculation) and the generation of multi-channel audio data to be emitted from speaker 2 are described in "Review Paper: Sound Field Simulation and Sound Space Rendering," by Takao Tsuchiya, [online], IEICE Fundamentals Review Vol.10 No.3, [Retrieved December 16, 2024].<URL:https: / / www.jstage.jst.go.jp / article / essfr / 10 / 3 / 10_206 / _pdf> The technology described can be used.

[0046] The attenuation rate storage unit 105 stores the attenuation rate of sound passing through each type of virtual wall (for example, reinforced concrete, wood, sliding doors) that separates the room where the device 1 is installed (hereinafter referred to as the room where the device is installed) from the virtual adjacent room.

[0047] The request processing unit 106 processes various requests via the wireless interface unit 100. For example, in accordance with a virtual playback request received from the wireless terminal 3, it sends a sound collection request to another pre-configured multi-channel audio device 1 (a multi-channel audio device 1 installed in the actual room of a user to whom a virtual living room has been assigned). Also, in accordance with a sound collection request received from the other multi-channel audio device 1, it sends a sound collection request to the wireless terminal 3, which functions as a remote controller for its own device 1, and obtains sound collection data of ambient noise in the room where its own device is installed from this wireless terminal 3.

[0048] The sound collection data transmission unit 107 transmits the sound collection data acquired from the wireless terminal 3 to another multi-channel audio device 1.

[0049] The sound collection data receiving unit 108 receives sound collection data from another multi-channel audio device 1.

[0050] The echo cancellation processing unit 109 performs echo cancellation processing when, while the sound collection data of ambient sounds in the room where the device is installed, acquired from the wireless terminal 3, is being transmitted to another multi-channel audio device 1 by the sound collection data transmission unit 107, the sound collection data receiving unit 108 receives ambient sounds from another room from the other multi-channel audio device 1. This process removes the sound collection data received by the sound collection data receiving unit 108 from the sound collection data transmitted by the sound collection data transmission unit 107.

[0051] The multi-channel audio generation unit 110 calculates a sound field (in a virtual acoustic space, the sound is made three-dimensionally audible when sound collection data of living sounds from a virtual sound source set in the virtual living room is emitted into the room where the device is installed) that makes the sound image of the reproduced sound relative to the sound collection data of living sounds from another room received by the sound collection data reception unit 108 from another multi-channel audio device 1 perceive outside the room where the device is installed, and generates multi-channel audio data to form this sound field within the room where the device is installed. Note that the 3D sound field calculation (sound field analysis) of the virtual sound source and the generation of multi-channel audio data to be radiated from speaker 2 are performed as described above, for example, "Review Paper: Sound Field Simulation and Sound Space Rendering," Takao Tsuchiya, [online], IEICE Fundamentals Review Vol.10 No.3, [Retrieved December 16, 2024].<URL:https: / / www.jstage.jst.go.jp / article / essfr / 10 / 3 / 10_206 / _pdf> The technology described can be used.

[0052] Furthermore, the multi-channel audio generation unit 110 adjusts the volume level of the multi-channel audio data generated as described above based on the attenuation rate stored in the attenuation rate storage unit 105, which is linked to the wall type specified in the virtual playback request received by the request processing unit 106 from the wireless terminal 3.

[0053] The multi-channel audio playback unit 111 reproduces and outputs the multi-channel audio data generated by the multi-channel audio generation unit 110 from the collected ambient sound data as Cch, FLch, FRch, SLch, and SRch audio signals. Similarly, it also reproduces and outputs the multi-channel audio data of the music stored in the content storage unit 103 as Cch, FLch, FRch, SLch, and SRch audio signals.

[0054] The superposition unit 112, when multi-channel audio data generated from ambient sound data (multi-channel audio data output from the multi-channel audio generation unit 110) and multi-channel audio data of a musical piece (multi-channel audio data read from the content storage unit 103) are being played in parallel by the multi-channel audio playback unit 111, superimposes the audio signal played from the multi-channel audio data of the musical piece onto the audio signal played from the multi-channel audio data of the musical piece for each channel of Cch, FLch, FRch, SLch, and SRch, and mixes the two.

[0055] The main control unit 113 then comprehensively controls each of the parts 100 to 112 of the multi-channel audio device 1.

[0056] The functional configuration of the multi-channel audio device 1 shown in Figure 5 can be implemented in hardware using integrated logic ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays), or in software using a computer such as a DSP (Digital Signal Processor). Alternatively, it can be implemented as a process in a general-purpose computer such as a PC, which is equipped with a CPU, memory, auxiliary storage devices such as flash memory and hard disk drives, and wireless communication devices such as wireless LAN adapters, by having the CPU load a predetermined program from the auxiliary storage device into memory and execute it.

[0057] Figure 6 is a flowchart illustrating the virtual playback process of the multi-channel audio device 1.

[0058] This flow is executed according to the instructions of the main control unit 113 when the request processing unit 106 receives a virtual playback request from a wireless terminal 3, which functions as a remote controller for the device 1, via the wireless interface unit 100.

[0059] First, the main control unit 113 identifies the sound field data stored in the sound field data storage unit 104 that is associated with the virtual living room specified in the virtual playback request received by the request processing unit 106, and also identifies the attenuation rate stored in the attenuation rate storage unit 105 that is associated with the wall type specified in this virtual playback request (S200).

[0060] Furthermore, the request processing unit 106 transmits a sound collection request to another pre-configured multi-channel audio device 1 via the wireless interface unit 100. As a result, the sound collection data receiving unit 108 begins receiving sound collection data (the actual sounds of daily life in the user's room to which a virtual living room has been assigned) sent from the other multi-channel audio device 1 via the wireless interface unit 100 (S201).

[0061] Next, the multi-channel audio generation unit 110 starts multi-channel audio data generation and adjustment processing based on the sound field data and attenuation rate identified by the main control unit 113 (S202). Subsequently, in the virtual acoustic space, the sound field data is used to calculate the acoustics that would occur if the sound collection data received by the sound collection data receiving unit 108 were emitted from the virtual sound source into the room where the device is installed. Multi-channel audio data is then generated to make the calculated acoustics three-dimensionally audible, and the volume level of the generated multi-channel audio data is adjusted based on the attenuation rate.

[0062] Next, the multi-channel audio playback unit 111 starts playing the multi-channel audio data generated by the multi-channel audio generation unit 110 (S203). If the content playback process described later (see Figure 8) is not currently being performed (NO in S204), the audio signals of the Cch, FLch, FRch, SLch, and SRch channels played back by the multi-channel audio playback unit 111 are output from the speaker interface unit 101 (S205).

[0063] On the other hand, if the content playback process described later is in progress (YES in S204), that is, if the multi-channel audio playback unit 111 is playing multi-channel audio data generated by the multi-channel audio generation unit 110 (multi-channel audio data generated from sound collection data of everyday sounds) in parallel with playing multi-channel audio data of a song read from the content storage unit 103, the superposition unit 112 superimposes the playback audio signal of the multi-channel audio data generated by the multi-channel audio generation unit 110 onto the playback audio signal of the multi-channel audio data of the song read from the content storage unit 103 for each of the Cch, FLch, FRch, SLch, and SRch channels, mixes the two, and outputs it from the speaker interface unit 101 (S206).

[0064] Figure 7 is a flowchart illustrating the sound collection process of the multi-channel audio device 1.

[0065] This flow is executed according to the instructions of the main control unit 113 when the request processing unit 106 receives a sound collection request from another multi-channel audio device 1 via the wireless interface unit 100.

[0066] First, the request processing unit 106 sends a sound collection request via the wireless interface unit 100 to the wireless terminal 3, which functions as a remote controller for the device 1, and starts acquiring sound collection data of the living sounds in the room where the device is installed from the wireless terminal 3 (S210).

[0067] Then, if the virtual playback process shown in Figure 6 is not in progress (NO in S211), the sound collection data transmission unit 107 transmits the sound collection data of the ambient sounds in the room where the device is installed, acquired by the request processing unit 106, to another pre-configured multi-channel audio device 1 via the wireless interface unit 100 (S213).

[0068] On the other hand, if the virtual playback process shown in Figure 6 is being performed (YES in S211), the echo cancellation processing unit 109 performs echo cancellation processing on the sound collection data acquired by the request processing unit 106 to remove sound collection data of ambient noise from other rooms received by the sound collection data receiving unit 108 (S212). Then, the sound collection data transmission unit 107 transmits the sound collection data, which has undergone echo cancellation processing, to another pre-configured multi-channel audio device 1 via the wireless interface unit 100 (S213).

[0069] Figure 8 is a flowchart illustrating the content playback process of the multi-channel audio device 1.

[0070] This flow is executed according to the instructions of the main control unit 113 when the music acquisition unit 102 receives a content playback request from the wireless terminal 3, which functions as a remote controller for the device 1, via the wireless interface unit 100.

[0071] First, the music acquisition unit 102 accesses the media server 6 via the wireless interface unit 100 and downloads the multi-channel audio data of the music specified in the content playback request from the media server 6 (S220). Then, it stores the downloaded multi-channel audio data in the content storage unit 103.

[0072] Then, the multi-channel audio playback unit 111 starts playing the multi-channel audio data of the music stored in the content storage unit 103 (S221). If the virtual playback process shown in Figure 6 is not currently being performed (NO in S222), the multi-channel audio playback unit 111 outputs playback audio signals for the Cch, FLch, FRch, SLch, and SRch channels generated from the multi-channel audio data of the music from the speaker interface unit 101 (S223).

[0073] On the other hand, if the virtual playback process shown in Figure 6 is being performed (YES in S222), that is, if the multi-channel audio playback unit 111 is playing back multi-channel audio data generated from ambient sound data by the multi-channel audio generation unit 110 in parallel with the playback of multi-channel audio data of the music, the superposition unit 112 superimposes the playback audio signal of the multi-channel audio data generated from ambient sound data onto the playback audio signal of the multi-channel audio data of the music for each channel of Cch, FLch, FRch, SLch, and SRch, mixes the two, and outputs it from the speaker interface unit 101 (S224).

[0074] Next, we will explain the details of wireless terminal 3.

[0075] Figure 9 is a schematic diagram of the functional configuration of the wireless terminal 3.

[0076] As shown in the figure, the wireless terminal 3 includes a wireless interface unit 300, a human-machine interface unit 301, a sound-collecting microphone 302, a request transmission unit 303, a sound collection request reception unit 304, a sound collection data transmission unit 305, and a main control unit 306.

[0077] The wireless interface unit 300 is an interface for wireless communication with a multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the access point 4.

[0078] The human-machine interface unit 301 is an interface for displaying information to the user and receiving various instructions from the user, and has input / output devices such as a touch panel.

[0079] The sound-collecting microphone 302 collects everyday sounds from the user's actual living room.

[0080] The request transmission unit 303 transmits a virtual playback request, which includes the setting of a virtual living room and wall type, and a content playback request, which includes the specification of a song, to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300.

[0081] The sound collection request receiving unit 304 receives a sound collection request from the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300.

[0082] The sound collection data transmission unit 305 transmits sound collection data of everyday sounds collected from the user's actual living room by the sound collection microphone 302 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300, in accordance with the sound collection request received by the sound collection request reception unit 304.

[0083] The main control unit 306 then comprehensively controls each of the wireless terminal 300-305.

[0084] Furthermore, the functional configuration of the wireless terminal 3 shown in Figure 9 is realized as a process in a portable computer such as a smartphone or tablet PC, which is equipped with a CPU, memory, auxiliary storage device such as flash memory, wireless communication device such as a wireless LAN adapter, microphone, and input / output device such as a touch panel, by the CPU executing a predetermined program.

[0085] Figure 10 is a flowchart illustrating the operation of the wireless terminal 3.

[0086] When the main control unit 306 receives a virtual playback instruction from the user via the man-machine interface unit 301 (YES in S300), it displays a virtual living room selection screen on the display screen of the man-machine interface unit 301, as shown in Figure 11(A), and allows the user to select a virtual adjacent room from among several virtual adjacent rooms to be set as the virtual living room of an acquaintance user, such as a family member or friend (S301). In the example shown in Figure 11(A), the user can select the virtual living room of an acquaintance user by selecting one of the four virtual adjacent rooms 32-1 to 32-4 that are assumed to be positioned in front, behind, to the left and right of the user's actual living room, and then touching the selection button 33. The main control unit 306 also displays a wall type selection screen on the display screen of the man-machine interface unit 301, as shown in Figure 11(B), and allows the user to select a wall type for the virtual wall separating the user's actual living room from among several wall types (S302). In the example shown in Figure 11(B), the wall type can be selected by choosing one of the following wall types: "reinforced concrete" 34-1, "wooden" 34-2, and "sliding door" 34-3, and then touching the selection button 35. Then, the request transmission unit 303 transmits a virtual playback request, along with the selected virtual living room and wall type, to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S303).

[0087] Furthermore, when the sound collection request receiving unit 304 receives a sound collection request from the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (YES in S304), the main control unit 306 causes the sound collection microphone 302 to start collecting ambient sounds (S305). Then, the sound collection data transmission unit 305 starts transmitting the sound collection data of ambient sounds collected by the sound collection microphone 302 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S306).

[0088] Furthermore, when the main control unit 306 receives a content playback instruction from the user via the human-machine interface unit 301, including the specification of a song (YES in S307), it notifies the request transmission unit 303 of the song specified in this content playback instruction. In response, the request transmission unit 303 transmits a content playback request including the song specification notified by the main control unit 306 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S308).

[0089] One embodiment of the present invention has been described above.

[0090] In this embodiment, the multi-channel audio device 1 calculates the acoustics when sound data collected from everyday sounds received from another multi-channel audio device 1 is emitted from the virtual sound source into the user's living room, based on sound field data of a virtual acoustic space where a virtual sound source is set up in the user's living room and an adjacent virtual living room. It then generates and plays back multi-channel audio data to make the calculated acoustics three-dimensionally audible. For example, the user will perceive the everyday sounds of an acquaintance living in a geographically distant room as if they were everyday sounds coming from a neighboring room through the wall. Therefore, according to this embodiment, it is possible to create an environment in which the user can feel as if family, friends, or other acquaintances living geographically far away are living nearby.

[0091] Furthermore, in this embodiment, the multi-channel audio device 1 pre-stores sound field data, which is the result of sound field analysis in a virtual acoustic space where a virtual sound source set up in the virtual adjacent room and the user's living room are placed, for each virtual adjacent room. Based on the sound field data of the virtual adjacent room selected by the user as the virtual living room of an acquaintance user from among these multiple virtual adjacent rooms, the device calculates the acoustics when sound data collected from everyday sounds received from other multi-channel audio devices 1 is emitted from this virtual sound source into the user's actual living room, and generates multi-channel audio data to make the calculated acoustics three-dimensionally audible. Therefore, according to this embodiment, it is possible to hear the everyday sounds of distant users such as family and friends coming from a direction desired by the user, as if they were the everyday sounds of a neighbor.

[0092] Furthermore, in this embodiment, the multi-channel audio device 1 pre-stores the sound attenuation rate for each type of virtual wall, and adjusts the volume level of the multi-channel audio data generated from the sound collection data of ambient noise received from other multi-channel audio devices 1 based on the attenuation rate of the wall type selected by the user. Therefore, according to this embodiment, the volume of ambient noise can be adjusted so that the ambient noise of acquaintances such as family members and friends living in geographically distant rooms is perceived as more natural ambient noise heard through the virtual wall of the wall type selected by the user.

[0093] Furthermore, in this embodiment, when the multi-channel audio device 1 is playing back multi-channel audio data of a song, if multi-channel audio data is generated from sound collection data of ambient sounds, it superimposes the audio signals of each channel played back from the multi-channel audio data of the song onto the audio signals of each channel played back from the multi-channel audio data of the song, and outputs them. Therefore, according to this embodiment, the user can enjoy the song in an environment where they can feel as if they are hearing the sounds of family, friends, etc., who live geographically far away, coming through a wall.

[0094] It should be noted that the present invention is not limited to the embodiments described above, and numerous modifications are possible within the scope of its essence.

[0095] For example, in the above embodiment, the multi-channel audio device 1 assumes that four virtual adjacent rooms are arranged in front of, behind, to the left and right of the user's living room as virtual adjacent rooms (see Figure 11(A)). However, the present invention is not limited to this. For example, in addition to the four virtual adjacent rooms adjacent to the user's living room in front of, behind, to the left and right, it may also be assumed that virtual adjacent rooms are arranged above and below the user's living room, and the arrangement is not limited to simply adjacent rooms, but any positional relationship of distance, azimuth angle, and elevation angle that can occur within the same house may be assumed. Furthermore, the multi-channel audio device 1 allows the user to select a virtual living room from among the multiple virtual adjacent rooms, thereby allowing the user to select a virtual sound source from which the sounds of the daily lives of geographically distant family members, friends, or other acquaintances are emitted. However, the present invention is not limited to this. The user may be allowed to directly select the relative positional relationship of the virtual sound source with respect to the listening point P in the user's living room, and then select a virtual sound source set to a position determined by this relative positional relationship.

[0096] Furthermore, in the above embodiment, the multi-channel audio device 1 assumes three types of walls as virtual walls: "reinforced concrete," "wooden," and "sliding doors" (see Figure 11(B)). However, the present invention is not limited to these. For example, in addition to these three types of walls, other wall types that are frequently used in actual living spaces, such as "paper screens" and "glass windows," may also be assumed. The multi-channel audio device 1 also allows the user to select the type of virtual wall, thereby allowing the user to select the attenuation rate of the multi-channel audio data generated from sound collection data of the actual living room of the acquaintance user. However, the present invention is not limited to these. The user may be allowed to directly select the attenuation rate. The attenuation rate may be given as a single value, or it may be a table of different values ​​depending on the frequency. These values ​​may be amplitude only, or they may include phase change information.

[0097] Furthermore, in the above embodiment, the multi-channel audio device 1 acquires sound collection data of ambient sounds in the room where the device is installed from a wireless terminal 3 that functions as a remote controller for the device 1. However, the present invention is not limited to this. The multi-channel audio device 1 may have a built-in or connected microphone to collect ambient sounds in the room where the device is installed.

[0098] Furthermore, in the above embodiment, the method for creating sound field data may utilize not only sound field analysis but also actual measured data of sound propagation from outside the user's room to inside the room. In addition, in the above embodiment, structural information such as walls and pillars of each user's room may be stored in an external database, and an external server may perform sound field analysis and save sound field data by performing calculations using this structural information.

[0099] Furthermore, in the above embodiment, the multi-channel audio device 1 receives sound collection data of everyday sounds from another multi-channel audio device 1 that has been set in advance, and generates and plays / outputs multi-channel audio data to make the sound of this collected sound collection data audible in three dimensions as if it were emitted from a virtual sound source set in a virtual living room to the room where the device is installed. However, the present invention is not limited to this. The multi-channel audio device 1 (referred to as the first multi-channel audio device 1) may receive audio data of a song being played / output by another multi-channel audio device 1 (referred to as the second multi-channel audio device 1), and generate and plays / outputs multi-channel audio data to make the sound of this audio data audible in three dimensions as if it were emitted from a virtual sound source set in a virtual living room. Even in this case, the user can perceive in real time as if the song being listened to by family, friends, etc. who live geographically far away were coming from nearby, and can feel closer to the lives of family, friends, etc. In this case, the second multi-channel audio device 1 may reduce the music playback output sound of the second multi-channel audio device 1, which is included in the sound collection data received from the first multi-channel audio device 1, in the same manner as the echo canceller described above, based on the audio data of the music that the second multi-channel audio device 1 is playing and outputting.

[0100] Furthermore, in each of the embodiments described above, a multi-channel audio device 1 that reproduces multi-channel audio data into audio signals for Cch, FLch, FRch, SLch, and SRch and outputs them was used as an example. However, the present invention is not limited to this. The present invention is broadly applicable to multi-channel audio devices that reproduce multi-channel audio data into audio signals for multiple channels and output them. [Explanation of Symbols]

[0101] 1, 1a, 1b: Multichannel audio equipment 2, 2a-1~2a-2-5, 2b-1~2b-5: Speaker 3, 3a, 3b: Wireless terminals; 4a, 4b: Access points 5: Network 6: Media Server 100: Wireless interface unit 101: Speaker interface unit 102: Music acquisition unit 103: Content storage unit 104: Sound field data storage unit 105: Attenuation rate storage unit 106: Request Processing Unit 107: Sound Collection Data Transmission Unit 108: Sound collection data receiving unit 109: Echo cancellation processing unit 110: Multichannel audio generation unit 111: Multi-channel audio playback unit 112: Superimposed section 113: Main control section 300: Wireless Interface Unit 301: Human-machine interface unit 302: Sound collection microphone 303: Request transmission unit 304: Sound collection request reception unit 305: Sound collection data transmission unit 306: Main control unit

Claims

1. A multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, A storage means that stores the sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position identified by its relative position to the listening point, Receiving means for receiving audio data from a predetermined device via a network, The system includes a generation means that generates multi-channel audio data for making the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results stored in the storage means. A multi-channel audio device characterized by the following features.

2. A multi-channel audio device according to claim 1, The aforementioned storage means is For each of the multiple virtual sound sources having different relative positional relationships with respect to the listening point, the sound field analysis results in the virtual acoustic space are stored. The aforementioned multi-channel audio device is The system further includes a sound source selection means that, in accordance with the user's instructions, selects one of the multiple virtual sound sources. The generating means is Based on the sound field analysis results of the virtual sound source selected by the sound source selection means from among the sound field analysis results stored in the storage means, the receiving means generates multi-channel audio data to make the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space. A multi-channel audio device characterized by the following features.

3. A multi-channel audio device according to claim 1, The system further includes a setting means for setting the type of virtual wall that virtually separates the listening point and the virtual sound source, in accordance with the user's instructions. The generating means is The volume level of the multi-channel audio data generated by the generation means is adjusted according to the type of virtual wall set by the setting means. A multi-channel audio device characterized by the following features.

4. A multi-channel audio device according to claim 1, The system further includes a superposition means that, when multi-channel audio data other than the multi-channel audio data generated by the generation means is being played back and output, and multi-channel audio data is generated by the generation means, superimposes and outputs the audio signals of each channel played back from the other multi-channel audio data and the audio signals of each channel played back from the multi-channel audio data generated by the generation means. A multi-channel audio device characterized by the following features.

5. A program that causes a computer to function as a multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, A storage means that stores the sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position identified by its relative position to the listening point. Receiving means for receiving audio data from a predetermined device via a network, and Based on the sound field analysis results stored in the storage means, the computer functions as a generation means for generating multi-channel audio data to make the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space. A program characterized by the following features.

Images