Reproduction device, reproduction system, reproduction method, and reproduction program

Abstract

To provide a reproduction device, a reproduction system, and a reproduction method capable of enhancing a sense of presence.SOLUTION: A reproduction device according to an embodiment includes an acquisition unit, a determination unit, and a reproduction unit. The acquisition unit acquires sound source information related to a sound source. The determination unit determines the output characteristics of the sound source in the virtual speaker on the basis of the positional relationship between a virtual speaker arranged in a virtual space and a virtual listener arranged in the virtual space. The reproduction unit reproduces the sound source through a real speaker arranged in a real space on the basis of the output characteristics determined by the determination unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a playback device, a playback system 、 and a playback method. and playback program

Background Art

[0002] Conventionally, for example, there has been proposed a technique that allows a user to experience a sense of presence as if they were at a concert venue even from a remote location by expressing audio and video recorded in a real space such as a concert venue in a virtual space such as VR or AR.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when expressing audio in a virtual space, the sound source output is from the speakers in the space where the user wearing the VR device or AR device is located (for example, a room), so there is a risk of lacking a sense of presence.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a playback device, a playback system 、 and a playback method and playback program that can enhance the sense of presence.

Means for Solving the Problems

[0006] ​In order to solve the above-described problems and achieve the object, a playback device according to the present invention includes an acquisition unit, a determination unit, and a playback unit. The acquisition unit acquires sound source information regarding a sound source. The determination unit determines output characteristics of the sound source at the virtual speaker based on a positional relationship between a virtual speaker arranged in a virtual space and a virtual listener arranged in the virtual space. The playback unit plays back the sound source via a real speaker arranged in a real space based on the output characteristics determined by the determination unit.

Advantages of the Invention

[0007] According to the present invention, the sense of presence can be enhanced.

Brief Description of the Drawings

[0008]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of a playback device, a playback system, and a playback method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by the embodiments shown below.

[0010] First, the outline of the playback method according to the embodiment will be described with reference to FIGS. 1A and 1B. FIGS. 1A and 1B are diagrams showing the outline of the playback method according to the embodiment. The playback method according to the embodiment is executed by the playback device 1 shown in FIGS. 1A and 1B.

[0011] As shown in FIG. 1A, in the present disclosure, in an acoustic space SS which is a real space such as a concert hall or a live venue, an actual listener RL (hereinafter, the actual listener RL) actually records a sound source and video, and the recorded sound source and video are reproduced by a playback device 1 in a virtual space VS such as VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality) (see FIG. 1B). In this way, a user U different from the actual listener RL can experience a sense of presence as if being in the acoustic space SS while being in a real space RS such as at home.

[0012] Specifically, as shown in FIG. 1A, the playback device 1 is communicatively connected to an actual speaker 200 disposed in the real space RS, and reproduces the recorded sound source via the actual speaker 200. Further, the playback device 1 has a display unit 5 (see FIG. 3) capable of displaying the virtual space VS, and reproduces the recorded video via the display unit 5. In FIG. 1A, the arrangement of the actual speaker 200 shows the case where it is fixed at a predetermined position in the real space RS, but the actual speaker 200 may be, for example, something worn by the user U such as earphones.

[0013] In the present disclosure, by the playback device 1 according to the embodiment executing the playback method shown in FIG. 1B, the sense of presence in the virtual space VS can be enhanced by the sound source reproduced via the actual speaker 200.

[0014] Hereinafter, the playback method according to the embodiment will be described with reference to FIG. 1B. In FIG. 1B, for convenience of explanation, the virtual space VS is shown in a top view, but actually, the user U can view videos in various line-of-sight directions with the virtual listener VL as the viewpoint via the playback device 1.

[0015] As shown in FIG. 1B, in the playback method according to the embodiment, the playback device 1 first acquires sound source information regarding the sound source recorded by the recording device 100 (step S1). Note that the playback device 1 also acquires video information recorded by the recording device 100 together with the sound source information. The sound source information and the video information may be acquired directly from the recording device 100, or may be acquired from a cloud server (not shown) that stores the sound source information and the video information. Further, the sound source information and the video information may be acquired via a storage medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a flash memory.

[0016] Subsequently, in the playback method according to the embodiment, based on the positional relationship between the virtual speaker 300 arranged in the virtual space VS and the virtual listener VL arranged in the virtual space VS, the output characteristics of the sound source at the virtual speaker 300 are determined (step S2). The output characteristics include, for example, the frequency characteristics, phase characteristics, and gain characteristics (volume characteristics) of the sound source.

[0017] Specifically, in step S2, first, the playback device 1 arranges the virtual speaker 300 and the virtual listener VL in the virtual space VS. The virtual speaker 300 and the virtual listener VL may be arranged at predetermined positions, or may be arranged at positions specified by the user U. Then, the playback device 1 determines the output characteristics of the sound source at the virtual speaker 300 based on the positional relationship between the arranged virtual speaker 300 and virtual listener VL.

[0018] Specifically, the playback device 1 determines output characteristics such that the user U actually hears the sound source in the acoustic space SS, which is the real space, based on the direction in which the virtual speaker 300 exists with respect to the virtual listener VL and the distance from the virtual listener VL to the virtual speaker 300.

[0019] That is, when the virtual listener VL is arranged at the central position of the four virtual speakers 300, the playback device 1 determines output characteristics in which a sound source with an equal volume (gain) is output from the four virtual speakers 300 toward the virtual listener VL.

[0020] Also, as shown in FIG. 1B, when the virtual listener VL moves from the central position to the left side in the plane of the paper, for example, the playback device 1 increases the volume of the two virtual speakers 300 on the left side in the plane of the paper and decreases the volume of the two virtual speakers 300 on the right side in the plane of the paper. That is, when the virtual listener VL moves to the left side, output characteristics are determined such that the sound of the performer on the left side of the stage arranged in the virtual space VS can be heard loudly and the sound of the performer on the right side of the stage can be heard softly. Details of the determination process of the output characteristics will be described later.

[0021] Subsequently, in the playback method according to the embodiment, based on the determined output characteristics, a sound source is played back through the actual speakers 200 arranged in the real space RS (see FIG. 1A) (step S3). Specifically, the playback device 1 determines the actual output characteristics of the actual speakers 200 based on the positional relationship between the virtual speakers 300 and the actual speakers 200, and plays back the sound source based on the determined actual output characteristics. That is, the playback device 1 makes the actual output characteristics of the sound source in the actual speakers 200 such that the output characteristics of the sound source in the virtual speakers 300 are obtained.

[0022] Thereby, the sound source played back from the actual speakers 200 has the output characteristics of the sound source in the virtual speakers 300. In other words, the sound source played back from the actual speakers 200 can give the user U a feeling of actually listening to the sound source in the acoustic space SS which is the real space. That is, according to the playback method according to the embodiment, the sense of presence in the virtual space VS can be enhanced.

[0023] Next, with reference to FIG. 2, a configuration example of the playback system according to the embodiment will be described. FIG. 2 is a block diagram showing a configuration example of the playback system S according to the embodiment.

[0024] As shown in FIG. 2, the playback system S includes a playback device 1 and a recording device 100, and these are communicably connected via a communication network N such as the Internet.

[0025] The playback device 1 is a device that executes the playback method according to the embodiment, and as described above, is a device capable of displaying a three-dimensional virtual space VS such as VR or AR. The playback device 1 is, for example, of the goggle type as shown in FIG. 1B. Further, the playback device 1 may be, for example, a smartphone, a tablet terminal, a notebook PC, a desktop PC, or the like. Further, the virtual space VS displayed on the playback device 1 may be two-dimensional rather than limited to three-dimensional.

[0026] The recording device 100 is a device that records sound sources and images, and as shown in FIG. 2, includes a microphone 110 for recording sound that is a sound source and a camera 120 for recording images. The recording device 100 transmits sound source information regarding the sound source recorded by the microphone 110 and video information regarding the video recorded by the camera 120 to the playback device 1.

[0027] In FIG. 2, an example is shown in which the sound source information and the video information are directly transmitted from the recording device 100 to the playback device 1. However, the recording device 100 may transmit the sound source information and the video information to a cloud server (not shown). In such a case, the playback device 1 may acquire the sound source information and the video information stored in the cloud server.

[0028] Next, with reference to FIG. 3, a configuration example of the playback device 1 according to the embodiment will be described. FIG. 3 is a functional block diagram showing a configuration example of the playback device 1 according to the embodiment. In the block diagram of FIG. 3, only the components necessary for explaining the features of the present embodiment are represented by functional blocks, and the description of general components is omitted.

[0029] In other words, each component illustrated in the block diagram of FIG. 3 is conceptually functional, and does not necessarily have to be physically configured as shown in the figure. For example, the specific form of the distribution and integration of each functional block is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0030] As shown in FIG. 3, the playback device 1 includes a communication unit 2, a control unit 3, a storage unit 4, and a display unit 5. Further, the playback device 1 is connected to an actual speaker 200. The playback device 1 and the actual speaker 200 are connected by short-range wireless communication such as Bluetooth (registered trademark), for example. Note that the playback device 1 and the actual speaker 200 may be connected by wire.

[0031] In FIG. 3, the playback device 1 is configured to include the display unit 5 (integrated configuration), but the display unit 5 may be configured separately. Also, the playback device 1 may be configured with the actual speaker 200 integrated.

[0032] The communication unit 2 is a communication interface that is bidirectionally communicably connected to the communication network N, and transmits and receives information to and from the recording device 100.

[0033] The control unit 3 includes an acquisition unit 31, a reception unit 32, a determination unit 33, and a playback unit 34, and includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, input / output ports, etc., and various circuits.

[0034] The CPU of the computer functions as the acquisition unit 31, the reception unit 32, the determination unit 33, and the playback unit 34 of the control unit 3, for example, by reading and executing a program stored in the ROM.

[0035] Also, at least a part or all of the acquisition unit 31, the reception unit 32, the determination unit 33, and the playback unit 34 of the control unit 3 can be configured by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0036] Further, the storage unit 4 is a storage unit composed of a storage device such as a non-volatile memory, a data flash, or a hard disk drive. Such storage unit 4 stores arrangement information 41 and various programs.

[0037] The arrangement information 41 is information including the position information of the actual speaker 200. For example, the position information of the actual speaker 200 is information on the relative position between the user U and the actual speaker 200. Also, the position information of the actual speaker 200 may be coordinate information indicating the absolute position in the real space RS. Note that the position information of the actual speaker 200 may be registered in advance by the user U, or the playback device 1 may be provided with a camera (not shown), and the position information of the actual speaker 200 may be detected from the image of the camera.

[0038] Also, when the playback device 1 and the actual speaker 200 are wirelessly connected, the position information of the actual speaker 200 may be detected based on the arrival direction and signal strength of the communication signal.

[0039] The display unit 5 is a display capable of displaying the virtual space VS.

[0040] Next, each function (acquisition unit 31, reception unit 32, determination unit 33, and playback unit 34) of the control unit 3 will be described.

[0041] The acquisition unit 31 acquires various information. The acquisition unit 31 acquires, for example, sound source information regarding a sound source from the recording device 100. The sound source may include any type of sound such as voice, musical instrument sound, digital sound, etc.

[0042] Also, the acquisition unit 31 acquires video information regarding the video recorded by the recording device 100 together with the sound source information. Note that the acquisition unit 31 may acquire the sound source information and the video information separately, or may acquire information in which the sound source information and the video information are integrated, such as a video with audio.

[0043] Further, the acquisition unit 31 acquires the position information of the actual speaker 200 in the real space RS. The position of the actual speaker 200 is expressed as the relative position (relative direction and relative distance) with respect to the user U. The position of the actual speaker 200 may be input (specified) by the user U, or the playback device 1 may include a camera (not shown), and the position of the actual speaker 200 recognized from the image of the camera may be acquired.

[0044] Also, when a sound source is recorded in the acoustic space SS which is the real space, the acquisition unit 31 acquires acoustic information regarding the acoustic characteristics in such acoustic space SS. The acoustic information includes, for example, reflection characteristic information regarding the reflection characteristics of sound in reflectors (such as walls, etc.) existing in the acoustic space SS.

[0045] For example, the acquisition unit 31 estimates the material of the reflector based on the captured image of the reflector such as video information, and acquires (estimates) the reflection characteristic information corresponding to the estimated material. The reflection characteristic information is, for example, information such as the sound reflectivity. Note that the reflectivity may be the reflectivity of the entire sound, or may be the reflectivity for each frequency band of the sound source.

[0046] Also, the acoustic information includes information regarding people (number of people and information on their existing positions) existing in the acoustic space SS which is the real space. This is because the acoustic characteristics change according to the number of people existing in the acoustic space SS, such as it becoming more difficult for the sound source to be reflected as the number of people increases. Note that when other users exist as avatars in the virtual space VS, the sound source information may include information regarding the avatars (number of avatars and information on their existing positions).

[0047] The reception unit 32 receives various information from the user U. For example, the reception unit 32 receives the designation of the listening direction starting from the virtual listener VL in the virtual space VS. Details of the listening direction will be described later with reference to FIG. 4.

[0048] Also, the reception unit 32 receives the position change of the virtual listener VL. Further, the reception unit 32 receives the playback instructions for the sound source and video.

[0049] The determination unit 33 determines the output characteristics of the sound source in the virtual speaker 300 based on the positional relationship between the virtual speaker 300 arranged in the virtual space VS and the virtual listener VL arranged in the virtual space VS.

[0050] Specifically, the determination unit 33 first arranges the virtual speaker 300 and the virtual listener VL in the virtual space VS. The virtual speaker 300 may be arranged at a predetermined position or at a position specified by the user U. Further, the determination unit 33 may recognize the sound source (such as a speaker or an audience) from the video information acquired by the acquisition unit 31 and arrange the virtual speaker 300 at a position corresponding to such a sound source. Alternatively, the determination unit 33 may use the position corresponding to the position of the recording device 100 (see FIG. 1A) as the position of the virtual speaker 300.

[0051] Also, the virtual listener VL is arranged at a predetermined initial position after the user U enters the virtual space VS (after logging in). After being arranged at the initial position, the virtual space VS can be moved by the movement operation (such as mouse or keyboard operation) of the user U.

[0052] The initial position of the virtual listener VL may be a predetermined position, or an arbitrary position may be selected by the user U. Also, when a plurality of virtual listeners VL can enter the virtual space VS (a plurality of users U can log in), for example, they may be sequentially arranged at predetermined positions in the order of entry. Specifically, when seats are arranged in the virtual space VS such as a concert hall, the virtual listener VL may be arranged at each seat position in the order of entry, or at the seat position specified (ticket purchase) by the user U in advance.

[0053] Then, the determination unit 33 determines the output characteristics of the sound source in the virtual speaker 300 based on the positional relationship between the position of the virtual listener VL (the initial position or the position after movement) and the position of the virtual speaker 300. The output characteristics are, for example, the frequency characteristics, phase characteristics, gain characteristics, directivity characteristics, etc. of the sound source. Note that the final position of the virtual listener VL is the position where it is arranged when a reproduction instruction is received by the reception unit 32.

[0054] Specifically, the determination unit 33 sets a predetermined position in the real space such as a concert hall as the position of the virtual listener VL, and determines the output characteristics of the sound source in the virtual speaker 300 so that the same sound source that can actually be heard at the predetermined position in the real space can be heard at the position of the virtual listener VL.

[0055] Specifically, the determination unit 33 determines the output characteristics such that the user U actually hears the sound source in the acoustic space SS which is the real space, based on the direction in which the virtual speaker 300 exists with respect to the virtual listener VL and the distance from the virtual listener VL to the virtual speaker 300.

[0056] More specifically, the determination unit 33 determines the direction from the virtual speaker 300 to the virtual listener VL as the sound direction (directivity characteristic), and determines that the volume (gain characteristic) becomes smaller as the distance from the virtual speaker 300 to the virtual listener VL becomes longer.

[0057] For example, when the sound source is an orchestra, if the position of the virtual listener VL is arranged to the left of the performer group, the determination unit 33 increases the volume (gain) of the virtual speaker 300 arranged on the left side of the performer group and decreases the volume (gain) of the virtual speaker 300 arranged on the right side. As a result, the user U who is not participating in the concert hall can hear a sound source as if actually being in the concert hall in the virtual space VS. That is, the sense of presence can be enhanced.

[0058] Also, in the concert hall, when the performers are performing in the front and the audience exists in the rear, if the position of the virtual listener VL is in the rear, the determination unit 33 decreases the volume of the virtual speaker 300 arranged in the front and increases the volume of the virtual speaker 300 in the rear. That is, the sound of the performers is small and the sound of the audience (background noise) is large for the sound source heard at the position of the virtual listener VL.

[0059] In addition, when the acquisition unit 31 acquires the acoustic information of the acoustic space SS, the determination unit 33 determines the output characteristics in consideration of such acoustic information. Specifically, the determination unit 33 estimates the reverberant sound in which the sound output from the virtual speaker 300 is reflected by a reflector and reaches the virtual listener VL based on the distance from a reflector such as a wall in the acoustic space SS to the virtual listener VL, the distance from the virtual speaker 300 to the reflector, and the sound reflectance (reflection characteristic information) at the reflector. Therefore, since the position and apparent shape of the reflector or the like differ according to the position of the virtual listener VL, the output characteristics vary depending on the position of the virtual listener VL. Then, the determination unit 33 determines the output characteristics such that the sound source directly reaching the virtual listener VL from the virtual speaker 300 becomes an acoustic sound source to which the estimated reverberant sound is added.

[0060] Specifically, the determination unit 33 determines the output characteristics of the acoustic sound source by combining the output characteristics of the sound source and the output characteristics of the reverberant sound. Note that the output characteristics of the reverberant sound are output characteristics in which high-frequency components (frequency components with large attenuation) are reduced, the phase is delayed, or the gain (volume) is decreased with respect to the output characteristics of the sound source. In this way, by determining the output characteristics in consideration of the acoustic information, the determination unit 33 can add a reverberant sound component to the sound source reproduced by the subsequent reproduction unit 34, so that the user U can hear the sound source as if listening in the acoustic space SS.

[0061] In addition, when the acoustic information includes information about people existing in the acoustic space SS which is the real space (information about the number of people and their existing positions) or information about the avatars of other users existing in the virtual space VS, the determination unit 33 may determine the output characteristics of the sound source based on such information.

[0062] Specifically, the determination unit 33 determines the output characteristics such that the attenuation amount of the sound source increases as the number of listeners existing in the acoustic space SS which is the real space increases, or as the number of avatars existing in the virtual space VS increases.

[0063] The playback unit 34 plays back a sound source via the actual speaker 200 arranged in the real space RS based on the output characteristics determined by the determination unit 33. Specifically, the playback unit 34 first sets the actual speaker 200 in the virtual space VS.

[0064] Specifically, the playback unit 34 sets the relative position of the actual speaker 200 with respect to the virtual listener VL in the virtual space VS so as to be the same as the relative position of the actual speaker with respect to the user U. When the virtual listener VL moves, the actual speaker 200 also moves accordingly. That is, the relative position of the actual speaker 200 with respect to the virtual listener VL is always kept constant.

[0065] Then, the playback unit 34 determines the actual output characteristics of the sound source output from the actual speaker 200 based on the positional relationship between the virtual speaker 300 and the actual speaker 200, and plays back the sound source based on the determined actual output characteristics. The actual output characteristics include frequency characteristics, phase characteristics, gain characteristics, directivity characteristics, etc. That is, acoustic signal processing is performed using an acoustic transfer function or the like so that the characteristics of the sound reaching the virtual listener VL when the sound is output from the virtual speaker 300 are the same as the characteristics of the sound reaching the actual listener (user U) when the sound is output from the actual speaker 200.

[0066] Specifically, the playback unit 34 determines the actual output characteristics by correcting the output characteristics of the sound source output from the actual speaker 200 so as to be the output characteristics determined by the determination unit 33. Then, the playback unit 34 plays back the sound source with the actual output characteristics determined from the actual speaker 200. In this way, based on the positional relationship between the virtual speaker 300 and the actual speaker 200, by determining the actual output characteristics of the actual speaker 200, a sound source with a higher sense of presence can be played back.

[0067] In addition, the playback unit 34 plays back the video information acquired by the acquisition unit 31 together with the sound source. Specifically, the playback unit 34 detects the orientation of the face of the user U wearing the playback device 1 which is a VR device, and displays the video in the line-of-sight direction (the line-of-sight direction with the virtual listener VL as the viewpoint) corresponding to the orientation of the face. Note that the line-of-sight direction may be received as an operation by an operation member such as a button operation or a joystick of the user U.

[0068] Note that when the playback device 1 plays back a sound source, the user U may be able to hear the sound source from a specific direction (the listening direction) in the virtual space VS. This point will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of emphasizing and playing back the sound source in the listening direction.

[0069] In the example shown in FIG. 4, an example is shown in which the line-of-sight direction VF (line-of-sight range) of the virtual listener VL faces the front of the stage, while the listening direction received by the reception unit 32 is the right direction of the stage.

[0070] In such a case, the playback device 1 plays back the sound source so that the sound source on the right side of the stage becomes louder while displaying the entire stage based on the line-of-sight direction VF. Specifically, the determination unit 33 increases the gain of the virtual speaker 300 corresponding to the received listening direction, and decreases (or sets to zero) the gain of the virtual speaker 300 away from the listening direction. Thereby, the user U can emphasize and hear the sound source on the right side of the stage, that is, in the listening direction.

[0071] Note that in FIG. 4, an example of emphasizing the sound source in the listening direction is given, but for example, only the sound source in the listening direction may be erased.

[0072] Next, an example of realizing a pseudo surround system will be described with reference to FIG. 5. FIG. 5 is a diagram showing the processing of pseudo surround. In FIG. 5, an example is shown in which five (for example, 5.1ch) virtual speakers 300 are arranged in the virtual space VS and there are four actual speakers 200. That is, the playback device 1 plays back a pseudo 5.1ch surround sound source with the sound source output from the four actual speakers 200.

[0073] Specifically, the playback unit 34 determines the actual output characteristics and plays back the sound source by correcting the attenuation amount and phase of the sound source output from the actual speaker 200 according to the distance and direction (angle) from the actual speaker 200 to each virtual speaker 300. Thereby, even when the number of channels between the actual speaker 200 and the virtual speaker 300 is different (especially when the number of channels of the virtual speaker 300 is larger), a sound source that is pseudo-adjusted to the number of channels of the virtual speaker 300 can be played back from the actual speaker 200, so that the sense of presence in the virtual space VS can be enhanced.

[0074] Next, with reference to FIG. 6, the procedure of the process executed in the playback apparatus 1 according to the embodiment will be described. FIG. 6 is a flowchart showing the process procedure of the process executed by the playback apparatus 1 according to the embodiment.

[0075] As shown in FIG. 6, first, the acquisition unit 31 acquires, for example, sound source information regarding the sound source recorded by the recording apparatus 100 and video information recorded by the recording apparatus 100 (step S101).

[0076] Subsequently, the acquisition unit 31 acquires acoustic information regarding the acoustic characteristics in the acoustic space SS (step S102). For example, based on the video information recorded by the recording apparatus 100, the acquisition unit 31 estimates reflection characteristic information regarding the sound reflection characteristics on the wall surrounding the acoustic space SS, and acquires the estimated reflection characteristic information as acoustic information.

[0077] Subsequently, the acquisition unit 31 acquires the position information of the virtual speaker 300 in the virtual space VS (step S103).

[0078] Subsequently, the acquisition unit 31 acquires the position information of the virtual listener VL (step S104).

[0079] Subsequently, the determination unit 33 determines the output characteristics of the sound source in the virtual speaker 300 based on the positional relationship between the virtual speaker 300 and the virtual listener VL (step S105).

[0080] Subsequently, the acquisition unit 31 acquires the position information of the actual speaker 200 (step S106).

[0081] Subsequently, the playback unit 34 determines (corrects) the actual output characteristics of the sound source in the actual speaker 200 based on the output characteristics determined by the determination unit 33 (step S107).

[0082] Subsequently, the playback unit 34 plays back the sound source via the actual speaker 200 and displays video information via the display unit 5 based on the determined actual output characteristics (step S108), and ends the process.

[0083] As described above, the playback device 1 according to the embodiment includes an acquisition unit 31, a determination unit 33, and a playback unit 34. The acquisition unit 31 acquires sound source information regarding the sound source. The determination unit 33 determines the output characteristics of the sound source in the virtual speaker 300 based on the positional relationship between the virtual speaker 300 arranged in the virtual space VS and the virtual listener VL arranged in the virtual space VS. The playback unit 34 plays back the sound source via the actual speaker 200 arranged in the real space RS based on the output characteristics determined by the determination unit 33. Thereby, the sense of presence can be enhanced.

[0084] Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the specific details and representative embodiments represented and described as above. Accordingly, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0085] 1 Playback device 2 Communication unit 3 Control unit 4 Memory unit 5 Display unit 31 Acquisition unit 32 Reception unit 33 Decision unit 34 Reproduction unit 41 Configuration information 100 Recording device 110 Microphone 120 Camera 200 Real speaker 300 Virtual speaker N Communication network RL Real listener RS Real space S Reproduction system SS Acoustic space U User VF Line-of-sight direction VL Virtual listener VS Virtual space

Claims

1. A playback device that plays back a sound source based on sound source information regarding the sound source, comprising a control unit, wherein the control unit, determines output characteristics of the sound source at the virtual speaker based on a positional relationship between a virtual speaker arranged in a virtual space and a virtual listener arranged in the virtual space, and a listening direction other than the front direction of the virtual listener specified starting from the virtual listener, and plays back the sound source through a real speaker arranged in the real space based on the output characteristics. Playback device.

2. The sound source is a sound source recorded in a predetermined acoustic space, wherein the control unit, acquires acoustic information regarding acoustic characteristics in the acoustic space, and determines the output characteristics based on the acoustic information. The playback device according to claim 1.

3. The acoustic information, includes reflection characteristic information regarding sound reflection characteristics of reflectors existing in the acoustic space. The playback device according to claim 2.

4. The control unit, acquires the reflection characteristic information estimated based on an imaging image obtained by imaging the reflector. The playback device according to claim 3.

5. The control unit, determines actual output characteristics of the sound source output from the real speaker based on a positional relationship between the virtual speaker and the real speaker, and plays back the sound source based on the determined actual output characteristics. The playback device according to any one of claims 1 to 4.

6. A playback system comprising the playback device according to any one of claims 1 to 5, and a recording device that transmits sound source information regarding the sound source recorded in the real space where the sound source flows to the playback device. Playback system.

7. A playback method executed by a computer, wherein output characteristics of a sound source at a virtual speaker are determined based on a positional relationship between a virtual speaker arranged in a virtual space and a virtual listener arranged in the virtual space, and a listening direction other than the front direction of the virtual listener specified starting from the virtual listener, and the sound source is played back through a real speaker arranged in the real space based on the output characteristics. Playback method.

8. A playback program executed by a computer, wherein output characteristics of a sound source at a virtual speaker are determined based on a positional relationship between a virtual speaker arranged in a virtual space and a virtual listener arranged in the virtual space, and a listening direction other than the front direction of the virtual listener specified starting from the virtual listener, Based on the output characteristics, reproduce the sound source through actual speakers arranged in the real space. Reproduction program.

Images