Multi-channel audio device and program

Abstract

[Problem] To provide a multi-channel audio device with which it is possible to achieve an environment where it seems as if acquaintances such as family and friends who live geographically apart therefrom are living nearby.　[Solution] On the basis of a sound field analysis result in a virtual acoustic space in which a virtual sound source is set at a position specified by a relative positional relationship with a listening point P, a multi-channel audio device 1a, 1b generates multi-channel audio data for making sound audible when collected sound data of an acquaintance's daily-life sounds, which is received from another multi-channel audio device 1 via a network 5, is emitted from the virtual sound source in the virtual acoustic space. The multi-channel audio device then reproduces the generated multi-channel audio data as audio signals for a plurality of channels and outputs the same from speakers 2a-1 to 2a-5, 2b-1 to 2b-5 connected thereto.

Description

Multi-channel audio device and program

[0001] The present invention relates to a multi-channel audio device.

[0002] Conventionally, there is known a multi-channel audio device that reproduces multi-channel audio data as audio signals of a plurality of channels and outputs the reproduced audio signals from speakers corresponding to the channels of the audio signals (for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2002-367290

[0004] By the way, for example, students and working people who start living alone independently from their parents' homes have a big change in their living environment from the environment where they have been surrounded by their families to the environment where they live alone, and they may feel lonely in their daily lives. In recent years, with the progress of the aging of society, the number of single elderly households and households consisting only of the elderly has been increasing, and family members and friends often worry about the situation of the elderly living separately in their daily lives.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide a multi-channel audio device capable of realizing an environment in which a user can feel as if acquaintances such as family members and friends living geographically apart are living in the neighborhood in their daily lives.

[0006] In order to solve the above problems, in the present invention, based on the sound field analysis result of a virtual acoustic space in which a virtual sound source is set at a position specified by the relative positional relationship with respect to the listening point, the reproduced sound of audio data received from a predetermined device via a network is played from this virtual sound source into the virtual acoustic space. Multi-channel audio data is generated for stereoscopically audible the sound. Then, the generated multi-channel audio data is reproduced and output as audio signals of a plurality of channels.

[0007] Here, the virtual sound source is set, for example, outside the room where the listening point is located (e.g., a location in a room adjacent to the room where the listening point is set, or a location beyond the window of the room where the listening point is set). Furthermore, the audio data received from the predetermined device may include, for example, everyday sounds collected in a room where a family member, friend, or other acquaintance lives.

[0008] For example, the present invention is a multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, comprising: a storage means that stores sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position specified by the relative positional relationship with respect to a listening point; a receiving means that receives audio data from a predetermined device via a network; and a generating means that generates multi-channel audio data for making the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results stored in the storage means.

[0009] In this invention, based on the sound field analysis results in a virtual acoustic space where a listening point and a virtual sound source having a predetermined positional relationship with the listening point are arranged, multi-channel audio data is generated to make the sound audible as if audio data received from a predetermined device via a network were emitted from this virtual sound source in the virtual acoustic space, and the generated multi-channel audio data is played back and output. For example, by setting a virtual sound source outside the room, the resident of that room will perceive the sound of the audio data acquired from the predetermined device as if it were coming from nearby. Therefore, according to this invention, by receiving sounds of daily life collected in the room where family, friends, or other acquaintances live from a predetermined device via a network, it is possible to realize an environment in which the user feels as if family, friends, or other acquaintances living geographically far away are living nearby in their daily life.

[0010] Figure 1 is a schematic diagram of a multi-channel audio system according to one embodiment of the present invention. Figure 2 is a sequence diagram for explaining the operation of the multi-channel audio system according to one embodiment of the present invention. Figure 3 is a sequence diagram for explaining the operation of the multi-channel audio system according to one embodiment of the present invention, and is a continuation of Figure 2. Figure 4 is a sequence diagram for explaining the operation of the multi-channel audio system according to one embodiment of the present invention, and is a continuation of Figure 3. Figure 5 is a schematic functional configuration diagram of the multi-channel audio device 1. Figure 6 is a flowchart for explaining the virtual playback processing of the multi-channel audio device 1. Figure 7 is a flowchart for explaining the sound collection processing of the multi-channel audio device 1. Figure 8 is a flowchart for explaining the content playback processing of the multi-channel audio device 1. Figure 9 is a schematic functional configuration diagram of the wireless terminal 3. Figure 10 is a flowchart for explaining the operation of the wireless terminal 3. Figure 11(A) is a diagram showing an example of a virtual living room selection screen, and Figure 11(B) is a diagram showing an example of a wall type selection screen.

[0011] An embodiment of the present invention will be described below with reference to the drawings.

[0012] Figure 1 is a schematic diagram of the multi-channel audio system according to this embodiment.

[0013] As shown in the figure, the multi-channel audio system according to this embodiment is installed in two geographically separated rooms, A and B. Room A is the actual living room of user A, and the multi-channel audio system installed in room A includes a multi-channel audio device 1a connected to a network 5 such as a WAN (Wide Area Network) or LAN (Local Area Network) via an access point 4a, a plurality of speakers 2a-1 to 2a-5, and a wireless terminal 3a that functions as a remote controller for the multi-channel audio device 1a. Room B is the actual living room of user B, and the multi-channel audio system installed in room B includes a multi-channel audio device 1b connected to the network 5 via an access point 4b, a plurality of speakers 2b-1 to 2b-5, and a wireless terminal 3b that functions as a remote controller for the multi-channel audio device 1b.

[0014] Hereinafter, multi-channel audio devices 1a and 1b will also be referred to simply as multi-channel audio device 1, speakers 2a-1 to 2a-5 and 2b-1 to 2b-5 will also be referred to simply as speakers 2-1 to 2-5 or speaker 2, wireless terminals 3a and 3b will also be referred to simply as wireless terminal 3, and access points 4a and 4b will also be referred to simply as access point 4.

[0015] The multi-channel audio device 1 downloads multi-channel audio data of a song from the media server 6 via the access point 4 and the network 5. The downloaded multi-channel audio data is then reproduced as audio signals for the center channel (hereinafter referred to as Cch), front left channel (hereinafter referred to as FLch), front right channel (hereinafter referred to as FRch), surround left channel (hereinafter referred to as SLch), and surround right channel (hereinafter referred to as SRch), and the reproduced audio signals for each channel are output from the speaker 2 of the corresponding channel.

[0016] Furthermore, the multi-channel audio device 1 receives ambient sound data from other multi-channel audio devices 1 via the access point 4 and the network 5. Then, for a listening point P located within the area enclosed by speakers 2-1 to 2-5 placed in the room, the device generates multi-channel audio data from a virtual sound source set at a predetermined location outside the area enclosed by speakers 2-1 to 2-5 to make the sound audible when the received audio data is emitted.The generated multi-channel audio data is then reproduced as audio signals for Cch, FLch, FRch, SLch, and SRch, and the audio signals of each reproduced channel are output to the corresponding channel speaker 2.

[0017] Furthermore, the multi-channel audio device 1 acquires sound data of ambient sounds collected by a wireless terminal 3 that functions as its own remote controller. It then transmits the ambient sound data to other multi-channel audio devices 1 via the access point 4 and the network 5.

[0018] Speaker 2 is provided to correspond to each of the multiple channels of audio signals reproduced by the multi-channel audio device 1, and emits sound according to the audio signal of its corresponding channel output from the multi-channel audio device 1.

[0019] Here, speaker 2-1 is provided corresponding to channel C, speaker 2-2 is provided corresponding to channel FL, speaker 2-3 is provided corresponding to channel FR, speaker 2-4 is provided corresponding to channel SL, and speaker 2-5 is provided corresponding to channel SR.

[0020] The wireless terminal 3 is wirelessly connected to the multi-channel audio device 1 and functions as a remote controller for the multi-channel audio device 1 as described above. The wireless terminal 3 also collects ambient sounds and transmits the collected sound data to the multi-channel audio device 1.

[0021] Figures 2 to 4 are sequence diagrams illustrating the operation of the multi-channel audio system according to this embodiment.

[0022] In User A's actual living room A, when the wireless terminal 3a receives a virtual playback instruction from User A to create a sound field that makes it seem as if the sounds of User B's actual living room B are coming from the room next to Room A (S100), it displays a selection screen (virtual living room selection screen) for User A to select a virtual adjacent room from among a plurality of virtual adjacent rooms defined as rooms virtually adjacent to Room A to be set as User B's virtual living room, and a selection screen (wall type selection screen) for User A to select the type of virtual wall (wall type) separating Room A and the virtual living room, and accepts the selection of the virtual living room and wall type from User A (S101). Then, the wireless terminal 3a transmits a virtual playback request with the specified virtual living room and wall type to the multi-channel audio device 1a (S102).

[0023] Here, the multi-channel audio device 1a has pre-stored sound field data, which is the result of sound field analysis using a sound source (virtual sound source) virtually set up in each virtual adjacent room, and a receiving point on a virtual boundary line assumed to be around the listening point P of room A (set to correspond to the placement positions of speakers 2a-1 to 2a-5). In addition, the multi-channel audio device 1a has pre-stored sound attenuation rates for each type of virtual wall.

[0024] When the multi-channel audio device 1a receives a virtual playback request from the wireless terminal 3a, it identifies the sound field data stored in association with the virtual living room specified in the virtual playback request, and the attenuation rate stored in association with the wall type specified in the virtual playback request (S103). Then, the multi-channel audio device 1a transmits a sound collection request to the multi-channel audio device 1b (S104).

[0025] Next, when the multi-channel audio device 1b receives a sound collection request from the multi-channel audio device 1a, it transmits this sound collection request to the wireless terminal 3b (S105). In response, the wireless terminal 3b begins to collect sounds of daily life in user B's actual living room B (S106) and begins to transmit the collected sound data to the multi-channel audio device 1b (S107). The multi-channel audio device 1b transmits the collected sound data of daily life in room B transmitted from the wireless terminal 3b to the multi-channel audio device 1b (S108).

[0026] Next, in order to recreate a three-dimensional sound field that gives user A in room A the sensation that sounds of daily life in room B are being emitted from beyond a virtual wall, the multi-channel audio device 1a starts generating multi-channel audio data to form a three-dimensional sound field (in a virtual acoustic space, the sound is made three-dimensionally audible when sound collected data of daily life in room B is emitted into room A from a virtual sound source set in a virtual living room adjacent to room A, separated by a virtual wall) within room A, based on the sound field data and attenuation rate identified in S103, using speakers 2a-1 to 2a-5 (S109), and starts playing back and outputting the generated multi-channel audio data (S110). Then, speakers 2a-1 to 2a-5 emit sound according to the audio signal of their corresponding channel output from the multi-channel audio device 1a (S111).

[0027] As a result, user A can perceive the sounds of user B's daily life, who lives in a geographically distant room B from user A, as if they were sounds of user B's daily life coming from next door to user A through the wall.

[0028] Next, in User B's actual living room B, when the wireless terminal 3b receives a virtual playback instruction from User B to create a sound field that makes it seem as if the sounds of User A's actual living room A are coming from the room next to Room B (S112), it displays a virtual living room selection screen for User B to select a virtual adjacent room from among several virtual adjacent rooms that are virtually defined as rooms adjacent to Room B, and a wall type selection screen for User B to select the type of wall for the virtual wall separating Room B and the virtual living room, and accepts the selection of the virtual living room and wall type from User B (S113). Then, the wireless terminal 3b transmits a virtual playback request with the specified virtual living room and wall type to the multi-channel audio device 1b (S114).

[0029] Here, the multi-channel audio device 1b has pre-stored sound field data, which is the result of sound field analysis using a sound source (virtual sound source) virtually set up in each virtual adjacent room, and a receiving point on a virtual boundary line assumed to be around the listening point P in room B (set to correspond to the placement positions of speakers 2b-1 to 2b-5). In addition, the multi-channel audio device 1b has pre-stored sound attenuation rates for each type of virtual wall.

[0030] When the multi-channel audio device 1b receives a virtual playback request from the wireless terminal 3b, it identifies the sound field data stored in association with the virtual living room specified in the virtual playback request, and the attenuation rate stored in association with the wall type specified in the virtual playback request (S115). Then, the multi-channel audio device 1b sends a sound collection request to the multi-channel audio device 1a (S116). It also starts an echo cancellation process to remove the sound collection data of living sounds from room A received from the multi-channel audio device 1a from the sound collection data of living sounds from room B that is sent to the multi-channel audio device 1a (S117).

[0031] Next, when the multi-channel audio device 1a receives a sound collection request from the multi-channel audio device 1b, it transmits this request to the wireless terminal 3a (S118). In response, the wireless terminal 3a begins collecting the sounds of daily life in user A's actual living room A (S119) and begins transmitting the collected sound data to the multi-channel audio device 1a (S120). In response, the multi-channel audio device 1a begins echo cancellation processing to remove the sound collection data of daily life in room B received from the multi-channel audio device 1b from the sound collection data of daily life in room A transmitted from the wireless terminal 3a (S121). Then, it transmits the echo-cancelled sound collection data of daily life in room A to the multi-channel audio device 1b (S122).

[0032] Next, in order to recreate a three-dimensional sound field that gives user B in room B the sensation that the sounds of daily life in room A are being emitted from beyond a virtual wall, the multi-channel audio device 1b starts generating multi-channel audio data to form a three-dimensional sound field (in a virtual acoustic space, the sound is made three-dimensionally audible when the sound collected data of daily life in room A, received from the multi-channel audio device 1a, is emitted into room B from a virtual sound source set in a virtual living room adjacent to room B, separated by a virtual wall) within room B using speakers 2b-1 to 2b-5, based on the sound field data and attenuation rate identified in S115 (S123), and starts playing back and outputting the generated multi-channel audio data (S124). Then, speakers 2b-1 to 2b-5 emit sound according to the audio signal of their corresponding channel output from the multi-channel audio device 1b (S125).

[0033] This allows user B to perceive the sounds of user A's daily life, who resides in room A geographically distant from their own living room B, as if they were sounds of daily life coming from next door to their own living room B through the wall.

[0034] Subsequently, in user A's actual living room A, when wireless terminal 3a receives a content playback instruction from user A with a specified song (S126), it transmits this content playback request with the specified song to multi-channel audio device 1a (S127). In response, multi-channel audio device 1a accesses media server 6 and downloads the multi-channel audio data of the song specified in the content playback request (S128), and plays this multi-channel audio data (S129). For each channel, it superimposes the audio signal played from the song's multi-channel audio data with the audio signal generated based on the sound collection data of the living sounds in room B received from multi-channel audio device 1b, and outputs it to the corresponding speakers 2a-1 to 2a-5 (S130). Speakers 2a-1 to 2a-5 then emit sound according to the audio signal of their corresponding channel output from multi-channel audio device 1a (S131).

[0035] This allows user A to listen to their desired music in an environment where the sounds of user B's daily life, who resides in a geographically distant room B, are perceived as if they were sounds coming through a wall.

[0036] Next, the details of the multi-channel audio device 1 and wireless terminal 3 that constitute the multi-channel audio system according to this embodiment will be described.

[0037] Note that an existing speaker can be used for speaker 2, so a detailed explanation of that is omitted.

[0038] First, I will explain the details of the multi-channel audio device 1.

[0039] Figure 5 is a schematic diagram of the functional configuration of the multi-channel audio device 1.

[0040] As shown in the figure, the multi-channel audio device 1 includes a wireless interface unit 100, a speaker interface unit 101, a music acquisition unit 102, a content storage unit 103, a sound field data storage unit 104, an attenuation rate storage unit 105, a request processing unit 106, a sound collection data transmission unit 107, a sound collection data reception unit 108, an echo cancellation processing unit 109, a multi-channel audio generation unit 110, a multi-channel audio playback unit 111, an overlapping unit 112, and a main control unit 113.

[0041] The wireless interface unit 100 is an interface for wireless communication with other multi-channel audio devices 1, a wireless terminal 3 that functions as a remote controller of the self-device 1, and a media server 6 via an access point 4.

[0042] The speaker interface unit 101 is an interface for connecting the speaker 2.

[0043] The music acquisition unit 102 accesses the media server 6 via the wireless interface unit 100 and downloads multi-channel audio data of music from the media server 6.

[0044] The multi-channel audio data of the music downloaded from the media server 6 is stored in the content storage unit 103.

[0045] In the sound field data storage unit 104, for each virtual room (virtual adjacent room) defined as a room that is virtually adjacent to the installation room of the own device 1, a virtual sound source (a virtual sound source at a position specified by the relative positional relationship with the listening point P in the installation room of the own device 1) set in this virtual adjacent room, and a sound reception point on a virtual boundary line assumed around the listening point P (set so as to correspond to the arrangement positions of the speakers 2-1 to 2-5 installed in the installation room of the own device 1), the sound field data, which is the result of sound field analysis using these, is stored. Note that for the sound field analysis (third-order sound field calculation) of the virtual sound source and the generation of multi-channel audio data to be radiated from the speaker 2, for example, the technology described in "Explanatory Paper Sound Field Simulation and Sound Space Rendering" by Takao Tsuchiya, [online], IEICE Fundamental Review Vol. 10 No. 3, [searched on December 16, 2024], <URL: https: / / www.jstage.jst.go.jp / article / essfr / 10 / 3 / 10_206 / _pdf> can be used.

[0046] In the attenuation rate storage unit 105, for each wall type (for example, reinforced concrete structure, wooden structure, sliding door) of the virtual wall that separates the installation room of the own device 1 (hereinafter, the own device installation room) and the virtual adjacent room, the attenuation rate of the sound passing through this virtual wall is stored.

[0047] The request processing unit 106 processes various requests via the wireless interface unit 100. For example, in accordance with a virtual playback request received from the wireless terminal 3, a sound collection request is transmitted to another multi-channel audio device 1 (a multi-channel audio device 1 installed in the actual room of the user to whom the virtual living room is assigned) that is set in advance. Also, in accordance with a sound collection request received from another multi-channel audio device 1, a sound collection request is transmitted to the wireless terminal 3 that functions as a remote controller of the own device 1, and the sound collection data of the ambient sound in the own device installation room is acquired from this wireless terminal 3.

[0048] The sound collection data transmission unit 107 transmits the sound collection data acquired from the wireless terminal 3 to another multi-channel audio device 1.

[0049] The sound collection data receiving unit 108 receives sound collection data from another multi-channel audio device 1.

[0050] The echo cancellation processing unit 109 performs echo cancellation processing when, while the sound collection data of ambient sounds in the room where the device is installed, acquired from the wireless terminal 3, is being transmitted to another multi-channel audio device 1 by the sound collection data transmission unit 107, the sound collection data receiving unit 108 receives ambient sounds from another multi-channel audio device 1, and the echo cancellation processing unit 109 removes the sound collection data received by the sound collection data receiving unit 108 from the sound collection data transmitted by the sound collection data transmission unit 107.

[0051] The multichannel audio generation unit 110 calculates a sound field (in a virtual acoustic space, the sound when sound collection data of living sounds from a virtual sound source set in the virtual living room is emitted into the room where the device is installed) that makes the sound image of the reproduced sound relative to the sound collection data of living sounds from another room received by the sound collection data reception unit 108 from another multichannel audio device 1 perceive outside the room where the device is installed, and generates multichannel audio data to form this sound field within the room where the device is installed. Note that the 3D sound field calculation (sound field analysis) of the virtual sound source and the generation of multichannel audio data to be emitted from speaker 2 are performed as described above, for example, "Review Paper: Sound Field Simulation and Sound Space Rendering" by Takao Tsuchiya, [online], IEICE Fundamentals Review Vol. 10 No. 3. [Retrieved December 16, 2024] The technology described in <URL: https: / / www.jstage.jst.go.jp / article / essfr / 10 / 3 / 10_206 / _pdf> can be used.

[0052] Furthermore, the multi-channel audio generation unit 110 adjusts the volume level of the multi-channel audio data generated as described above based on the attenuation rate stored in the attenuation rate storage unit 105, which is linked to the wall type specified in the virtual playback request received by the request processing unit 106 from the wireless terminal 3.

[0053] The multi-channel audio playback unit 111 reproduces and outputs the multi-channel audio data generated by the multi-channel audio generation unit 110 from the sound collection data of ambient noise as audio signals for Cch, FLch, FRch, SLch, and SRch. Similarly, it reproduces and outputs the multi-channel audio data of music stored in the content storage unit 103 as audio signals for Cch, FLch, FRch, SLch, and SRch.

[0054] The superposition unit 112, when multi-channel audio data generated from ambient sound data (multi-channel audio data output from the multi-channel audio generation unit 110) and multi-channel audio data of a musical piece (multi-channel audio data read from the content storage unit 103) are being played in parallel by the multi-channel audio playback unit 111, superimposes the audio signal played from the multi-channel audio data of the musical piece onto the audio signal played from the multi-channel audio data of the musical piece for each channel of Cch, FLch, FRch, SLch, and SRch, and mixes the two.

[0055] The main control unit 113 then comprehensively controls each of the parts 100 to 112 of the multi-channel audio device 1.

[0056] The functional configuration of the multi-channel audio device 1 shown in Figure 5 can be implemented in hardware using integrated logic ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays), or in software using a computer such as a DSP (Digital Signal Processor). Alternatively, it can be implemented as a process in a general-purpose computer such as a PC, which is equipped with a CPU, memory, auxiliary storage devices such as flash memory and hard disk drives, and wireless communication devices such as wireless LAN adapters, by the CPU loading a predetermined program from the auxiliary storage device into memory and executing it.

[0057] Figure 6 is a flowchart illustrating the virtual playback process of the multi-channel audio device 1.

[0058] This flow is executed according to the instructions of the main control unit 113 when the request processing unit 106 receives a virtual playback request from a wireless terminal 3, which functions as a remote controller for the device 1, via the wireless interface unit 100.

[0059] First, the main control unit 113 identifies the sound field data stored in the sound field data storage unit 104 that is associated with the virtual living room specified in the virtual playback request received by the request processing unit 106, and also identifies the attenuation rate stored in the attenuation rate storage unit 105 that is associated with the wall type specified in this virtual playback request (S200).

[0060] Furthermore, the request processing unit 106 transmits a sound collection request to another pre-configured multi-channel audio device 1 via the wireless interface unit 100. As a result, the sound collection data receiving unit 108 begins receiving sound collection data (the actual sounds of daily life in the user's room to which a virtual living room has been assigned) sent from the other multi-channel audio device 1 via the wireless interface unit 100 (S201).

[0061] Next, the multi-channel audio generation unit 110 starts multi-channel audio data generation and adjustment processing based on the sound field data and attenuation rate specified by the main control unit 113 (S202). Subsequently, in the virtual acoustic space, the sound field data is used to calculate the acoustics that would occur if the sound collection data received by the sound collection data receiving unit 108 were emitted from the virtual sound source into the room where the device is installed. Multi-channel audio data is then generated to make the calculated acoustics three-dimensionally audible, and the volume level of the generated multi-channel audio data is adjusted based on the attenuation rate.

[0062] Next, the multi-channel audio playback unit 111 starts playing the multi-channel audio data generated by the multi-channel audio generation unit 110 (S203). If the content playback process described later (see Figure 8) is not currently being performed (NO in S204), the audio signals of the Cch, FLch, FRch, SLch, and SRch channels played back by the multi-channel audio playback unit 111 are output from the speaker interface unit 101 (S205).

[0063] On the other hand, if the content playback process described later is in progress (YES in S204), that is, if the multi-channel audio playback unit 111 is playing multi-channel audio data generated by the multi-channel audio generation unit 110 (multi-channel audio data generated from sound collection data of everyday sounds) in parallel with playing multi-channel audio data of a song read from the content storage unit 103, the superposition unit 112 superimposes the playback audio signal of the multi-channel audio data generated by the multi-channel audio generation unit 110 onto the playback audio signal of the multi-channel audio data of the song read from the content storage unit 103 for each channel of Cch, FLch, FRch, SLch, and SRch, mixes the two, and outputs it from the speaker interface unit 101 (S206).

[0064] Figure 7 is a flowchart illustrating the sound collection process of the multi-channel audio device 1.

[0065] This flow is executed according to the instructions of the main control unit 113 when the request processing unit 106 receives a sound collection request from another multi-channel audio device 1 via the wireless interface unit 100.

[0066] First, the request processing unit 106 transmits a sound collection request to the wireless terminal 3, which functions as a remote controller for the device 1, via the wireless interface unit 100, and begins acquiring sound collection data of the living sounds in the room where the device is installed from the wireless terminal 3 (S210).

[0067] Then, if the virtual playback process shown in Figure 6 is not in progress (NO in S211), the sound collection data transmission unit 107 transmits the sound collection data of the ambient sounds in the room where the device is installed, acquired by the request processing unit 106, to another pre-configured multi-channel audio device 1 via the wireless interface unit 100 (S213).

[0068] On the other hand, if the virtual playback process shown in Figure 6 is being performed (YES in S211), the echo cancellation processing unit 109 performs echo cancellation processing on the sound collection data acquired by the request processing unit 106 to remove sound collection data of ambient noise from other rooms received by the sound collection data receiving unit 108 (S212). Then, the sound collection data transmission unit 107 transmits the sound collection data, which has undergone echo cancellation processing, to another pre-configured multi-channel audio device 1 via the wireless interface unit 100 (S213).

[0069] Figure 8 is a flowchart illustrating the content playback process of the multi-channel audio device 1.

[0070] This flow is executed according to the instructions of the main control unit 113 when the music acquisition unit 102 receives a content playback request from the wireless terminal 3, which functions as a remote controller for the device 1, via the wireless interface unit 100.

[0071] First, the music acquisition unit 102 accesses the media server 6 via the wireless interface unit 100 and downloads the multi-channel audio data of the music specified in the content playback request from the media server 6 (S220). Then, it stores the downloaded multi-channel audio data in the content storage unit 103.

[0072] Then, the multi-channel audio playback unit 111 starts playing the multi-channel audio data of the music stored in the content storage unit 103 (S221). If the virtual playback process shown in Figure 6 is not currently being performed (NO in S222), the multi-channel audio playback unit 111 outputs the playback audio signals for the Cch, FLch, FRch, SLch, and SRch channels generated from the multi-channel audio data of the music from the speaker interface unit 101 (S223).

[0073] On the other hand, if the virtual playback process shown in Figure 6 is being performed (YES in S222), that is, if the multi-channel audio playback unit 111 is playing back multi-channel audio data generated from ambient sound data by the multi-channel audio generation unit 110 in parallel with the playback of multi-channel audio data of the music, the superposition unit 112 superimposes the playback audio signal of the multi-channel audio data generated from ambient sound data onto the playback audio signal of the multi-channel audio data of the music for each channel of Cch, FLch, FRch, SLch, and SRch, mixes the two, and outputs it from the speaker interface unit 101 (S224).

[0074] Next, we will explain the details of the wireless terminal 3.

[0075] Figure 9 is a schematic diagram of the functional configuration of the wireless terminal 3.

[0076] As shown in the figure, the wireless terminal 3 includes a wireless interface unit 300, a human-machine interface unit 301, a sound-collecting microphone 302, a request transmission unit 303, a sound collection request reception unit 304, a sound collection data transmission unit 305, and a main control unit 306.

[0077] The wireless interface unit 300 is an interface for wireless communication with a multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the access point 4.

[0078] The human-machine interface unit 301 is an interface for displaying information to the user and receiving various instructions from the user, and has input / output devices such as a touch panel.

[0079] The sound-collecting microphone 302 collects everyday sounds from the user's actual living room.

[0080] The request transmission unit 303 transmits a virtual playback request, which includes the setting of a virtual living room and wall type, and a content playback request, which includes the specification of a song, to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300.

[0081] The sound collection request receiving unit 304 receives a sound collection request from the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300.

[0082] The sound collection data transmission unit 305 transmits sound collection data of everyday sounds collected from the user's actual living room by the sound collection microphone 302 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 300, in accordance with the sound collection request received by the sound collection request reception unit 304.

[0083] The main control unit 306 then comprehensively controls each of the wireless terminal 300 to 305.

[0084] Furthermore, the functional configuration of the wireless terminal 3 shown in Figure 9 is realized as a process in a portable computer such as a smartphone or tablet PC, which is equipped with a CPU, memory, auxiliary storage device such as flash memory, wireless communication device such as a wireless LAN adapter, microphone, and input / output device such as a touch panel, by the CPU executing a predetermined program.

[0085] Figure 10 is a flowchart illustrating the operation of the wireless terminal 3.

[0086] When the main control unit 306 receives a virtual playback instruction from the user via the human-machine interface unit 301 (YES in S300), it displays a virtual living room selection screen on the display screen of the human-machine interface unit 301, as shown in Figure 11(A), and allows the user to select a virtual adjacent room from among several virtual adjacent rooms to be set as the virtual living room of an acquaintance user, such as a family member or friend (S301). In the example shown in Figure 11(A), the user can select the virtual living room of an acquaintance user by selecting one of the four virtual adjacent rooms 32-1 to 32-4 that are assumed to be positioned in front, behind, to the left and right of the user's actual living room, and then touching the selection button 33. The main control unit 306 also displays a wall type selection screen on the display screen of the human-machine interface unit 301, as shown in Figure 11(B), and allows the user to select a wall type for the virtual wall separating the user's actual living room from among several wall types (S302). In the example shown in Figure 11(B), the wall type can be selected by choosing one of the following wall types: "reinforced concrete" 34-1, "wooden" 34-2, and "sliding door" 34-3, and then touching the selection button 35. Then, the request transmission unit 303 transmits a virtual playback request, along with the selected virtual living room and wall type, to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S303).

[0087] Furthermore, when the sound collection request receiving unit 304 receives a sound collection request from the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (YES in S304), the main control unit 306 causes the sound collection microphone 302 to start collecting ambient sounds (S305). Then, the sound collection data transmission unit 305 starts transmitting the sound collection data of ambient sounds collected by the sound collection microphone 302 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S306).

[0088] Furthermore, when the main control unit 306 receives a content playback instruction from the user via the human-machine interface unit 301, including the specification of a song (YES in S307), it notifies the request transmission unit 303 of the song specified in this content playback instruction. In response, the request transmission unit 303 transmits a content playback request including the song specification notified by the main control unit 306 to the multi-channel audio device 1, which uses its own terminal 3 as a remote controller, via the wireless interface unit 100 (S308).

[0089] One embodiment of the present invention has been described above.

[0090] In this embodiment, the multi-channel audio device 1 calculates the acoustics when sound data collected from everyday sounds received from another multi-channel audio device 1 is emitted from the virtual sound source into the user's living room, based on sound field data of a virtual acoustic space where a virtual sound source is set up in the user's living room and an adjacent virtual living room. It then generates, plays, and outputs multi-channel audio data to make the calculated acoustics three-dimensionally audible. For example, the user will perceive the everyday sounds of an acquaintance living in a geographically distant room as if they were everyday sounds coming from a neighboring room through the wall. Therefore, according to this embodiment, it is possible to create an environment in which the user can feel as if family, friends, or other acquaintances living geographically far away are living nearby.

[0091] Furthermore, in this embodiment, the multi-channel audio device 1 pre-stores sound field data, which is the result of sound field analysis in a virtual acoustic space where a virtual sound source set up in the virtual adjacent room and the user's living room are placed, for each virtual adjacent room. Based on the sound field data of the virtual adjacent room selected by the user as the virtual living room of an acquaintance user from among these multiple virtual adjacent rooms, the device calculates the acoustics when sound data collected from everyday sounds received from another multi-channel audio device 1 is emitted from this virtual sound source into the user's actual living room, and generates multi-channel audio data to make the calculated acoustics audible in three dimensions. Therefore, according to this embodiment, it is possible to hear the everyday sounds of distant users such as family and friends coming from a direction desired by the user, as if they were the everyday sounds of a neighbor.

[0092] Furthermore, in this embodiment, the multi-channel audio device 1 pre-stores the sound attenuation rate for each type of virtual wall, and adjusts the volume level of the multi-channel audio data generated from the sound collection data of everyday sounds received from other multi-channel audio devices 1 based on the attenuation rate of the wall type selected by the user. Therefore, according to this embodiment, the volume of everyday sounds from acquaintances such as family members and friends living in geographically distant rooms can be adjusted so that they are perceived as more natural everyday sounds heard through the virtual wall of the wall type selected by the user.

[0093] Furthermore, in this embodiment, when the multi-channel audio device 1 is playing back the multi-channel audio data of a song, if multi-channel audio data is generated from sound collection data of ambient sounds, it superimposes the audio signals of each channel played back from the multi-channel audio data of the song onto the audio signals of each channel played back from the multi-channel audio data of the song, and outputs them. Therefore, according to this embodiment, the user can enjoy the song in an environment where they can feel as if they are hearing the sounds of family, friends, etc., who live geographically far away, as if they were coming through a wall.

[0094] It should be noted that the present invention is not limited to the embodiments described above, and numerous modifications are possible within the scope of its essence.

[0095] For example, in the above embodiment, the multi-channel audio device 1 assumes that four virtual adjacent rooms are arranged in front of, behind, to the left and right of the user's living room as virtual adjacent rooms (see Figure 11(A)). However, the present invention is not limited to this. For example, in addition to the four virtual adjacent rooms adjacent to the user's living room in front of, behind, to the left and right, it may also be assumed that virtual adjacent rooms are arranged above and below the user's living room, and the arrangement is not limited to simply adjacent rooms, but any positional relationship of distance, azimuth angle, and elevation angle that can occur within the same house may be assumed. Furthermore, the multi-channel audio device 1 allows the user to select a virtual living room from among the multiple virtual adjacent rooms, thereby allowing the user to select a virtual sound source from which the sounds of the daily lives of geographically distant family members, friends, or other acquaintances are emitted. However, the present invention is not limited to this. The user may be allowed to directly select the relative positional relationship of the virtual sound source with respect to the listening point P in the user's living room, and then select a virtual sound source set to a position determined by this relative positional relationship.

[0096] Furthermore, in the above embodiment, the multi-channel audio device 1 assumes three types of walls as virtual walls: "reinforced concrete," "wooden," and "sliding doors" (see Figure 11(B)). However, the present invention is not limited to these. For example, in addition to these three types of walls, other wall types that are frequently used in actual living spaces, such as "paper screens" and "glass windows," may also be assumed. The multi-channel audio device 1 also allows the user to select the type of virtual wall, thereby allowing the user to select the attenuation rate of the multi-channel audio data generated from sound collection data of the living sounds in the actual living room of the acquaintance user. However, the present invention is not limited to these. The user may be allowed to directly select the attenuation rate. The attenuation rate may be given as a single value, or it may be a table of different values ​​depending on the frequency. Furthermore, these values ​​may be amplitude only, or they may include phase change information.

[0097] Furthermore, in the above embodiment, the multi-channel audio device 1 acquires sound collection data of ambient sounds in the room where the device is installed from a wireless terminal 3 that functions as a remote controller for the device 1. However, the present invention is not limited to this. The multi-channel audio device 1 may have a built-in or connected microphone to collect ambient sounds in the room where the device is installed.

[0098] Furthermore, in the above embodiment, the method for creating sound field data may utilize not only sound field analysis but also actual measured data of sound propagation from outside the user's room to inside the room. In addition, in the above embodiment, structural information such as walls and pillars of each user's room may be stored in an external database, and an external server may perform sound field analysis and save sound field data by performing calculations using this structural information.

[0099] Furthermore, in the above embodiment, the multi-channel audio device 1 receives sound collection data of everyday sounds from another multi-channel audio device 1 that has been set in advance, and generates, plays, and outputs multi-channel audio data to make the sound of this collected sound collection data audible in three dimensions as if it were emitted from a virtual sound source set in a virtual living room to the room where the device is installed. However, the present invention is not limited to this. The multi-channel audio device 1 (referred to as the first multi-channel audio device 1) may receive audio data of a song being played and output by another multi-channel audio device 1 (referred to as the second multi-channel audio device 1), and generate, plays, and outputs multi-channel audio data to make the sound of this audio data audible in three dimensions as if it were emitted from a virtual sound source set in a virtual living room. Even in this case, the user can perceive in real time as if the song being listened to by family, friends, etc. who live geographically far away were coming from nearby, and can feel closer to the lives of family, friends, etc. In this case, the second multi-channel audio device 1 may reduce the music playback output sound of the second multi-channel audio device 1, which is included in the sound collection data received from the first multi-channel audio device 1, using the same method as the echo canceller described above, based on the audio data of the music that the second multi-channel audio device 1 is playing and outputting.

[0100] Furthermore, in each of the embodiments described above, a multi-channel audio device 1 that reproduces multi-channel audio data into audio signals for Cch, FLch, FRch, SLch, and SRch and outputs them was used as an example. However, the present invention is not limited to this. The present invention is broadly applicable to multi-channel audio devices that reproduce multi-channel audio data into audio signals for multiple channels and output them.

[0101] 1, 1a, 1b: Multi-channel audio device 2, 2a-1 to 2a-2-5, 2b-1 to 2b-5: Speakers 3, 3a, 3b: Wireless terminals 4a, 4b: Access points 5: Network 6: Media server 100: Wireless interface unit 101: Speaker interface unit 102: Music acquisition unit 103: Content storage unit 104: Sound field data storage unit 105: Attenuation rate storage unit 106: Request processing unit 107: Sound collection data transmission unit 108: Sound collection data reception unit 109: Echo cancellation processing unit 110: Multi-channel audio generation unit 111: Multi-channel audio playback unit 112: Superposition unit 113: Main control unit 300: Wireless interface unit 301: Man-machine interface unit 302: Sound collection microphone 303: Request transmission unit 304: Sound collection request reception unit 305: Sound collection data transmission unit 306: Main control unit

Claims

1. A multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, comprising: a storage means that stores sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position specified by the relative positional relationship with respect to a listening point; a receiving means that receives audio data from a predetermined device via a network; and a generating means that generates multi-channel audio data for making the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results stored in the storage means.

2. A multichannel audio device according to claim 1, wherein the storage means stores sound field analysis results in the virtual acoustic space for each of the plurality of virtual sound sources having different relative positional relationships with respect to the listening point, the multichannel audio device further includes a sound source selection means that selects one of the plurality of virtual sound sources according to the user's instructions, and the generation means generates multichannel audio data for making audible the sound when audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results of the virtual sound source selected by the sound source selection means from among the sound field analysis results stored in the storage means.

3. A multichannel audio device according to claim 1, further comprising setting means for setting the type of virtual wall that virtually separates the listening point and the virtual sound source in accordance with the user's instructions, wherein the generating means adjusts the volume level of the multichannel audio data generated by the generating means according to the type of virtual wall set by the setting means.

4. A multichannel audio device according to claim 1, further comprising a superposition means for superimposing and outputting, when multichannel audio data other than multichannel audio data generated by the generation means is generated by the generation means, the audio signals of each channel reproduced from the other multichannel audio data and the audio signals of each channel reproduced from the multichannel audio data generated by the generation means.

5. A program that causes a computer to function as a multi-channel audio device that reproduces and outputs multi-channel audio data as multiple-channel audio signals, the program comprising: a storage means that stores sound field analysis results in a virtual acoustic space where a virtual sound source is set at a position specified by the relative positional relationship with respect to a listening point; a receiving means that receives audio data from a predetermined device via a network; and a generating means that generates multi-channel audio data for making the sound audible when the audio data received by the receiving means is emitted from the virtual sound source in the virtual acoustic space, based on the sound field analysis results stored in the storage means.

Images