Information processing apparatus, information processing method, program, recording and reproducing system

Abstract

The present invention reduces the feeling of discomfort associated with how sensory stimulation information is transmitted resulting from the difference between the recording environment and the reproduction environment. An information processing device according to the present art comprises: a positional relationship information acquisition unit for acquiring information regarding a first positional relationship that is defined as the positional relationship between a recording target and a recording device for recording in a recording environment where sensory stimulation information transmitted from the recording target is recorded, and information regarding a second positional relationship that is defined as the positional relationship between a user and the recording target in a reproduction environment of the recorded sensory stimulation information; and a correction unit for correcting, on the basis of the information regarding the first positional relationship and the information regarding the second positional relationship, an output signal of a reproduction device for reproducing the sensory stimulation information.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, an information processing method, a program, and a recording and reproducing system.

Background Art

[0002] Conventionally, various techniques have been proposed for presenting sensory stimulus information such as recorded tactile stimuli to a user, for example, as in Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recorded contents such as sports and movies, by reproducing sensory stimulus information such as video, sound, and vibration with a playback device, a user can enjoy the content while being in a place different from the recording location. However, such sensory stimulus information has different propagation methods depending on the environment, such as propagation speed and attenuation rate. Therefore, when the recording environment and the playback environment of the content are different, the way sound and vibration are transmitted with respect to the video may differ from the user's perception, giving a sense of discomfort to the user who appreciates the content and possibly impairing the sense of presence of the content itself.

[0005] The present technology has been made in view of the above circumstances, and an object thereof is to alleviate the discomfort in the way of transmission of sensory stimulus information caused by the difference between the recording environment and the playback environment.

Means for Solving the Problems

[0006] The information processing apparatus according to the present technology includes a recording device that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target, information on a first positional relationship that is the positional relationship between the recording device and the recording target, and information on a second positional relationship that is the positional relationship between the recording target and the user in the playback environment of the recorded sensory stimulus information. The information processing apparatus further includes a positional relationship information acquisition unit that acquires the information, and a correction unit that corrects an output signal of a playback device that plays back the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. The sensory stimulus information is, for example, various types of stimulus information that can be perceived by a user, such as video, sound, tactile stimuli, and odors. The recording target is, for example, an object such as a vibration source or a sound source in the recording environment. The recording device is, for example, an imaging device that captures an image, a microphone that collects acoustic information, an IMU (Inertial Measurement Unit) that collects vibration information, or the like. The position of the recording target in the playback environment is position information of an object that a user who has received the sensory stimulus information perceives to exist. For example, for a user who is viewing a display unit (e.g., a screen or a display) on which a video of a basketball as a recording target is projected, it is the position of the basketball that is virtually located on the other side of the display unit. Therefore, the positional relationship between the recording target and the user in the playback environment is the relationship between the virtual position of the basketball and the actual position of the user. Based on the information on the first positional relationship in the recording environment and the information on the second positional relationship in the playback environment, the audio signal of a speaker as a playback device and the output signal of a tactile signal presentation device are corrected, so that the user can experience a sensory stimulus corresponding to the viewing position. Specifically, a user closer to the display unit can experience a louder sound and stronger vibration.

[0007] In the information processing apparatus according to the present technology described above, when the playback device includes a plurality of types of playback devices corresponding to a plurality of types of the sensory stimulus information, the information on the second positional relationship may be the information on the relationship between the virtual position of the recording target recognized by the user via the display unit on which the image is played back and the position of the user. That is, the output signals of the auditory information and the tactile information are corrected according to the visual information received by the user.

[0008] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit performs the correction according to the change in the first positional relationship in the recording environment. Thereby, the output signal is corrected according to the case where the recording device has moved in the recording environment or the case where the recording target has moved.

[0009] In the information processing apparatus according to the present technology described above, it is conceivable that the change in the first positional relationship is in accordance with the change in the position of the recording target in the recording environment. Thereby, the output signal of the playback device is corrected according to the change in the position of the recording target.

[0010] In the information processing apparatus according to the present technology described above, it is conceivable that the change in the first positional relationship is in accordance with the change in the position of the recording device in the recording environment. Thereby, the output signal of the playback device is corrected according to the change in the position of the recording device.

[0011] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit performs the correction according to the change in the second positional relationship in the playback environment. Thereby, the output signal is corrected according to the case where the playback device has moved in the playback environment or the case where the user has moved.

[0012] In the information processing apparatus according to the present technology described above, it is conceivable that the change in the second positional relationship is in accordance with the change in the position of the user in the playback environment. As a result, the output signal of the playback device is corrected according to the change in the viewing position of the user.

[0013] In the information processing apparatus according to the present technology described above, it is conceivable that the change in the second positional relationship is in accordance with the change in the position of the playback device in the playback environment. As a result, the output signal of the playback device is corrected according to the change in the playback device.

[0014] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit corrects the output timing of the output signal. As a result, for example, when the position of the playback device (display unit such as a display or a screen) that outputs video is far from the user, it becomes possible to delay the output timing of the output signal of the speaker as the playback device.

[0015] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit corrects the signal strength of the output signal. As a result, for example, when the position of the playback device (display unit such as a display or a screen) that outputs video is far from the user, it becomes possible to make the signal strength of the output signal of the speaker as the playback device smaller.

[0016] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit performs the correction according to the environmental information in the recording environment. As a result, the output signal is corrected so as to provide a sensory stimulus equivalent to the case where the user is actually present in the recording environment.

[0017] In the information processing apparatus according to the present technology described above, it is conceivable that the environmental information is temperature information. As a result, even if the temperature of the environment where the user actually is is different from the temperature of the recording environment, the output signal is corrected so that the user can experience the sensory stimulus that the user would experience if the user were in the recording environment in the playback environment.

[0018] In the information processing apparatus according to the present technology described above, it is conceivable that the environmental information is information on a medium. Thereby, even if the medium in the reproduction environment (such as in water or air) is different from the medium in the recording environment, or even if the temperature of the environment is different from the temperature of the recording environment in the air where the user actually is, the output signal is corrected so that the user can experience in the reproduction environment the sensory stimulus that the user would experience when in the recording environment.

[0019] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit corrects the output signal of the playback device that plays back acoustic information as the sensory stimulus information. Thereby, the output signal is corrected for acoustic information that is likely to have a time-of-arrival shift due to the difference in the positions of the recording position and the playback position.

[0020] In the information processing apparatus according to the present technology described above, it is conceivable that the correction unit corrects the output signal of the playback device that plays back vibration information as the sensory stimulus information. Thereby, the output signal is corrected for vibration information that is likely to have a time-of-arrival shift due to the difference in the positions of the recording position and the playback position.

[0021] The information processing method according to the present technology is an information processing method in which an information processing apparatus executes a positional relationship information acquisition process of acquiring information on a first positional relationship that is the positional relationship between a recording device that performs the recording in a recording environment in which sensory stimulus information transmitted from a recording target is recorded and the recording target, and information on a second positional relationship that is the positional relationship between the recording target and the user in a playback environment of the recorded sensory stimulus information, and a correction process of correcting an output signal of a playback device that plays back the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. Even with such an information processing method, the same operation as that of the information processing apparatus according to the present technology described above can be obtained.

[0022] The program according to the present technology is a program that causes an information processing device to execute a position relationship information acquisition function for acquiring information on a first position relationship that is the positional relationship between a recording device that performs the recording and the recording target in a recording environment for recording sensory stimulus information transmitted from the recording target, and information on a second position relationship that is the positional relationship between the recording target and the user in the playback environment of the recorded sensory stimulus information, and a correction function for correcting an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship. Such a program according to the present technology realizes the information processing device according to the present technology described above.

[0023] A recording / reproducing system including the recording device and the reproduction control device according to the present technology, wherein the recording device includes a sensory stimulus information acquisition unit that acquires sensory stimulus information transmitted from a recording target, and a detection unit that detects information on a first position relationship that is the positional relationship between the recording device and the recording target, and the reproduction control device includes a position relationship information acquisition unit that acquires the information on the first position relationship and information on a second position relationship that is the positional relationship between the recording target and the user in the playback environment of the acquired sensory stimulus information, and a correction unit that corrects an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship. With such a recording / reproducing system according to the present technology, based on the information on the first position relationship in the recording environment and the information on the second position relationship in the playback environment, the audio signal of the speaker as the playback device and the output signal of the tactile signal presentation device are corrected, so that the user can experience a sensory stimulus corresponding to the viewing position.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0025] Hereinafter, embodiments according to the present technology will be described in the following order. <1. Configuration Example of Recording / Playback System> <2. Differences in Transmission by Sensory Stimulus Information> <3. Configuration Example of Recording Apparatus> <4. Configuration Example of Playback Control Apparatus> <5. Correction Process of Sensory Stimulus Information> <6. Summary> <7. The Present Technology>

[0026] In addition, each configuration described in the drawings referred to in the description is only an extraction of the configuration of the main part related to the present technology, and the configuration described in the drawings can be variously changed according to the design and the like as long as it does not deviate from the technical idea of the present technology. Also, the configurations once described below may be denoted by the same reference numerals and the description may be omitted thereafter. Furthermore, the present technology is not limited to the present embodiment, and modifications, improvements, etc. within the range capable of achieving the object of the present technology are included in the present technology.

[0027] <1. Configuration Example of Recording / Playback System> A configuration example of the recording and playback system 1 in this embodiment will be described with reference to FIG. 1. The recording and playback system 1 includes a recording device 2, a playback control device 3, and a playback device 4. The environment for realizing the recording and playback system 1 is divided into a recording environment and a playback environment.

[0028] The recording device 2 records the sensory stimulus information transmitted from the recording target in the recording environment. Here, the recording target is, for example, an object or a person that becomes a vibration source or a sound source in the recording environment. The recording device 2 includes, for example, an imaging device that captures images, a microphone that collects acoustic information, and an IMU (Inertial Measurement Unit) that collects vibration information. The sensory stimulus information is, for example, various stimulus information that can be perceived by the user, such as video, sound, tactile stimuli, and smells. The recording device 2 generates encoded data Ec by encoding the recorded sensory stimulus information.

[0029] The playback control device 3 performs playback control processing for allowing the user to experience various sensory stimulus information recorded in the recording environment. For this purpose, the playback control device 3 acquires the encoded data Ec encoded in the recording environment.

[0030] Note that the playback control device 3 may appropriately mount a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and acquire the encoded data Ec from the recording medium.

[0031] The playback control device 3 outputs the sensory stimulus information obtained by decoding the acquired encoded data Ec to a plurality of playback devices 4. By each playback device 4 playing back the sensory stimulus information, the user can experience the sensory stimulus information recorded in the recording environment while being in the playback environment.

[0032] Here, a configuration example of the playback device 4 will be described with reference to FIG. 2. Figure 2 shows an example in which user Hm sitting on chair 5 in a playback environment experiences the sensory stimulation information output from playback device 4.

[0033] A plurality of types of playback devices 4 are provided according to a plurality of types of sensory stimulation information. For example, a video playback device 4a, an audio playback device 4b, a vibration playback device 4c, etc. are used as the playback device 4.

[0034] The video playback device 4a is a device that outputs a video that stimulates the vision of user Hm, and is, for example, a TV (Television) device having a display unit as shown in the figure. In addition, various other application examples are conceivable for the video playback device 4a, such as a projector device and a screen installed in a movie theater, and a head-mounted display (HMD) worn by user Hm.

[0035] The audio playback device 4b is a device that outputs a sound that stimulates the hearing of user Hm, and is, for example, headphones or earphones as shown in the figure. In addition to being worn by user Hm, the audio playback device 4b has various other application examples, such as speakers arranged in a movie theater or indoors.

[0036] The vibration playback device 4c is a device that outputs vibrations that stimulate the sense of touch of user Hm, and is, for example, a vibration vest equipped with a vibration device such as a vibrator or an actuator as shown in the figure. In addition, the vibration playback device 4c can be applied to various other devices equipped with a vibration device. For example, it may be a smart device such as a smartphone, a controller used in a game or an attraction, or a chair on which user Hm sits. It may also be the wall or ceiling of an attraction vehicle, or a floor with a built-in vibration device.

[0037] Note that the above configuration described with reference to Figure 2 is merely an example, and the configuration example of the playback device 4 is not limited to the above. For example, in the above description, the video playback device 4a, the audio playback device 4b, and the vibration playback device 4c have been described as a plurality of playback devices 4. However, each playback device 4 may be partially or entirely integrated. For example, the chair 5 on which the user Hm sits may function as the audio playback device 4b and the vibration playback device 4c having an audio playback unit and a vibration playback unit. Also, as shown in FIG. 3, the head-mounted display HMD may function as the video playback device 4a and the audio playback device 4b.

[0038] In addition, a plurality of playback devices 4 of the same type may be provided. For example, as a plurality of vibration playback devices 4c, a vibration vest worn on the upper body of the user Hm and a vibration supporter worn on the knee of the user Hm may be provided. With the above configuration, the recording and playback system 1 in the present embodiment is realized.

[0039] <2. Differences in Transmission by Sensory Stimulation Information> The problems to be solved in the present embodiment will be described with reference to FIGS. 4 and 5. FIG. 4 schematically shows the positional relationship between the recording target and the recording device 2 in the recording environment, and the positional relationship between the recording target and the user Hm in the playback environment.

[0040] In FIG. 4, the position of the recording target is shown as the target position T, the position of the recording device 2 in the recording environment is shown as the recording position R, the position of a certain user Hm in the playback environment is shown as the somatic sensation position A, and the position of another user Hm is shown as the somatic sensation position B. Also, the somatic sensation positions A and B, which are the positions of the user Hm, are the playback positions of the audio playback device 4b and the vibration playback device 4c.

[0041] Here, the target position T in the playback environment is the position where an object to be recognized by the user Hm who has received the sensory stimulation information should exist. For example, it is the virtual position of the recording target that the user Hm recognizes through the display unit of the video playback device 4a on which the video (image) is played back.

[0042] Specifically, for user Hm who is viewing a display (the display unit of video playback device 4a) on which a video of, for example, basketball is projected as the recording target, it is the position of the basketball that is virtually located on the other side of the display. Therefore, the positional relationship between the recording target and user Hm in the playback environment is the relationship between the virtual position of the basketball and the actual position (viewing position) of user Hm.

[0043] In this embodiment, as an example, a scenario will be described assuming that user Hm is enjoying the video displayed on the display (video playback device 4a) while wearing headphones (audio playback device 4b) and a vibration vest (vibration playback device 4c) as shown in FIG. 2.

[0044] In order for user Hm to feel the sensory stimulation information recorded by recording device 2 at the somatosensory position A without losing the sense of presence, it suffices that the target position T as seen from the recording position R in the recording environment is the same as the target position T as seen from the somatosensory position A in the playback environment (that is, the virtual position of the recording target).

[0045] However, in reality, the positional relationship between the target position T and the recording position R in the recording environment and the positional relationship between the target position T and the somatosensory position A in the playback environment are almost always different.

[0046] Specifically, in the example shown in FIG. 4, the somatosensory positions A and B in the playback environment are farther from the target position T than the recording position R in the recording environment. Also, the somatosensory position B is farther from the target position T than the somatosensory position A.

[0047] In this case, Table 1 in FIG. 4 shows the arrival times of each piece of sensory stimulation information emitted from the target position T to each position. Here, the time when the target position T emits the sensory stimulation information is set as time T0, and it is assumed that time elapses in the right direction on the horizontal axis.

[0048] Also, in Table 1 of FIG. 4, the time when the sensory stimulus information from the target position T reaches the recording position R is shown as "Tr", the time when it reaches the somatosensory position A is shown as "Ta", and the time when it reaches the somatosensory position B is shown as "Tb".

[0049] When considered in this way, among the sensory stimulus information, since the transmission speed of video information is based on the speed of light, it is hardly affected by the positional relationship from the target position T, and the transmission times to each position (recording position R, somatosensory position A, somatosensory position B) are almost the same.

[0050] On the other hand, since the transmission speeds of acoustic information and vibration information are significantly slower than that of video information, they are easily affected by the distance from the target position T as shown in the figure, and the transmission times to each position increase as the distance from the target position T increases.

[0051] Also, as shown in FIG. 5, for acoustic information and vibration information, the transmission speed (Table 1), the transmission distance per unit time (Table 2), and the arrival time at a predetermined distance (Table 3) are significantly different. Specifically, the transmission speed of vibration information is faster than that of acoustic information, and the arrival time at a predetermined distance is earlier.

[0052] This difference is due to differences in the media used for transmission, etc. Tables 1 to 3 in FIG. 5 show, as an example, a case where air is used for the transmission of acoustic information (sound), and solids such as iron and wood are used for the transmission of signal information.

[0053] Therefore, for example, if the sensory stimulus information recorded by the recording device 2 at a timing corresponding to the positional relationship (direction and distance) between the target position T and the recording position R is directly output as it is to the somatosensory position A or the somatosensory position B with a different positional relationship from the target position T, there will be a deviation from what the user Hm expects and the transmission of video information, acoustic information, and vibration information, which may give the user Hm a sense of discomfort.

[0054] Specifically, the acoustic information recorded in the recording environment is recorded with a delay compared to the video information due to the difference in transmission speed. And the amount of the delay is assumed to correspond to the distance between the recording target and the recording position R.

[0055] On the other hand, it is desirable that the delay amount of the acoustic information in the playback environment depends on the positional relationship between the virtual position of the recording target (target position T) perceived by the user Hm and the physical position A of the user Hm.

[0056] Therefore, when the distance from the target position T to the recording position R in the recording environment is different from the distance from the target position T as the virtual position in the playback environment to the physical position A, if the acoustic information recorded at the recording position R is played back without correction, the user Hm will perceive the acoustic information that should originally be perceived at the recording position R at the physical position A. As a result, the user Hm may feel a sense of discomfort.

[0057] For example, when the distance between the position of the user Hm and the virtual position of the recording target in the playback environment is greater (farther) than the distance between the recording target and the recording position R in the recording environment, the acoustic information will reach the user Hm earlier than expected, thus impairing the sense of immersion.

[0058] Therefore, in the present embodiment, when the positional relationship between the target position T and the recording position R in the recording environment is different from the positional relationship between the target position T (the aforementioned virtual position) and the physical position (physical position A or physical position B) in the playback environment, the acoustic information output from the acoustic playback device 4b and the vibration information output from the vibration playback device 4c are corrected. By correcting the acoustic information and the vibration information in this way, it is possible to suppress the situation where the sound and vibration felt by the user Hm deviate from the assumption, and prevent the sense of immersion from being impaired.

[0059] <3. Configuration Example of Recording Device> A configuration example of the recording device 2 in the present embodiment will be described with reference to FIG. 6. The recording device 2 is configured to include a computer device such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). Then, by the CPU, DSP, etc. executing various programs, the recording device 2 is provided with various functions.

[0060] The recording device 2 includes a camera unit 10, an image signal processing unit 11, an acoustic input unit 12, an acoustic signal processing unit 13, a vibration detection unit 14, a vibration signal processing unit 15, a sensor unit 16, an encoding unit 17, a memory unit 18, and a communication unit 19. Also, the image signal processing unit 11, the acoustic signal processing unit 13, the vibration signal processing unit 15, the sensor unit 16, the encoding unit 17, the memory unit 18, and the communication unit 19 are interconnected via a bus 20, and various data and control signals are exchanged therebetween.

[0061] The camera unit 10 includes an imaging unit having an imaging element such as a CCD (Charged-coupled devices) image sensor or a CMOS (Complementary metal-oxide-semiconductor) image sensor and an imaging optical system. The imaging unit receives light based on a subject (recording target) imaged by the lens system, performs photoelectric conversion, and converts it into an electrical signal (imaging image signal). The camera unit 10 supplies the converted imaging image signal to the image signal processing unit 11.

[0062] The image signal processing unit 11 is configured as an image processing processor such as a DSP (Digital Signal Processor), and obtains image data (imaging image data) by performing various signal processes such as A / D (Analog / Digital) conversion process on the imaging image signal from the camera unit 10. The imaging image data obtained by the image signal processing unit 11 is supplied to the encoding unit 17. Note that the imaging image data may be temporarily stored (recorded) in the memory unit 18.

[0063] The acoustic input unit 12 is configured by, for example, a microphone, etc., picks up sound emitted from the recording target, and converts it into an electrical signal (acoustic signal). The acoustic input unit 12 supplies the converted electrical signal to the acoustic signal processing unit 13.

[0064] The acoustic signal processing unit 13 obtains acoustic data by performing various signal processes such as amplification process and A / D conversion process of the acoustic signal. The acoustic data obtained by the acoustic signal processing unit 13 is supplied to the encoding unit 17. Note that the acoustic data may be temporarily stored in the storage unit 18.

[0065] The vibration detection unit 14 is a vibration detection gyro sensor such as an IMU, for example, and detects vibrations generated by the recording target as a sound source and converts them into electrical signals (vibration signals). The vibration detection unit 14 supplies the converted electrical signals to the vibration signal processing unit 15.

[0066] The vibration signal processing unit 15 obtains vibration data (tactile data) by performing various signal processes such as amplification processing and A / D conversion processing on the vibration signals. The vibration data obtained by the vibration signal processing unit 15 is supplied to the encoding unit 17. Note that the vibration data may be temporarily stored in the storage unit 18. Alternatively, the vibration data may be generated by passing the acoustic data through a filter, or the produced and edited vibration data may be recorded in the storage unit 18 in advance.

[0067] The sensor unit 16 includes a position sensor that detects the position of the recording device 2, a distance measuring sensor that detects the distance from the recording device 2 to the recording target, a temperature sensor that detects the temperature of the recording environment, and various sensors for detecting the medium of the recording environment, etc., and detects various information.

[0068] Note that the above-described camera unit 10 can also function as a distance measuring sensor. For example, the camera unit 10 includes two imaging units, and each imaging unit is arranged to be separated in the left-right direction. By arranging each imaging unit to be separated in the left-right direction, a parallax for obtaining distance information can be generated between the imaging images of each other. That is, based on the imaging images obtained by these imaging units, information on the distance to the recording target can be obtained by the stereo method. In other words, three-dimensional data representing the positions in the X-axis, Y-axis, and Z-axis directions to the recording target in the real space can be obtained. Alternatively, the camera unit 10 may function as a distance measuring sensor by including a TOF (Time-of-Flight) sensor, an image plane phase difference sensor, or the like.

[0069] The encoding unit 17 encodes the sensory stimulus information supplied from the image signal processing unit 11, the audio signal processing unit 13, and the vibration signal processing unit 15 according to a predetermined data format, and stores the encoded data Ec obtained by the encoding in the storage unit 18.

[0070] The encoded data Ec obtained by the encoding unit 17 is temporarily stored in the storage unit 18. Also, the storage unit 18 may temporarily store signal data from the various signal processing units of the image signal processing unit 11, the audio signal processing unit 13, and the vibration signal processing unit 15, and various detection information from the sensor unit 16. The storage unit 18 may be a removable recording medium such as a memory card, an optical disk, or a magnetic tape, or may be a fixed type HDD (Hard Disk Drive) or a semiconductor memory module.

[0071] The communication unit 19 communicates with external devices. Specifically, the communication unit 19 is composed of modules that perform wireless communication such as a mobile communication system for mobile phones called 3G, 4G communication, etc., a wireless LAN (Local Area Network) according to the IEEE802.11 series, or wired communication such as a wired LAN or USB. Thereby, it is possible to transfer various data with an external device such as the playback control device 3. Thus, the recording device 2 in the present embodiment is realized.

[0072] In the present embodiment, an example in which all of the camera unit 10, the audio input unit 12, and the vibration detection unit 14 are provided in the recording device 2 has been described. However, a part or all of the above units may be provided in an external device that can communicate with the recording device 2. In this case, a position sensor is provided in the external device in order to acquire the position information of the above units. The external device supplies its own position information to the recording device 2 in addition to the acquired sensory stimulus information.

[0073] Each external device may be provided with various sensors and the like for acquiring environmental information. Here, the environmental information includes temperature information, humidity information, medium information, etc., and is information that is a factor in the change in the transmission speed of sensory stimulus information.

[0074] Further, an external device having each of the camera unit 10, the acoustic input unit 12, and the vibration detection unit 14 may be provided as a separate recording device 2. In this case, for each external device, various signal processes such as amplification processing and A / D conversion processing are performed on the recorded sensory stimulus information, and encoded data Ec is generated by performing encoding processing.

[0075] <4. Configuration Example of the Reproduction Control Device> A configuration example of the reproduction control device 3 in the present embodiment will be described with reference to FIG. 7. The reproduction control device 3 is configured to include a computer device such as a CPU or a DSP. Then, by the CPU, DSP, etc. executing various programs, the reproduction control device 3 has various functions.

[0076] The reproduction control device 3 has a decoding unit 30, a signal processing unit 31, a reproduction data generation unit 32, a storage unit 33, and a communication unit 34. Further, the decoding unit 30, the signal processing unit 31, the reproduction data generation unit 32, the storage unit 33, and the communication unit 34 are mutually connected via a bus 40, and various data, control signals, etc. are exchanged.

[0077] The decoding unit 30 decodes the encoded data Ec encoded by the recording device 2 according to a predetermined data format. By decoding the encoded data Ec, the decoding unit 30 acquires image data, acoustic data, vibration data, etc. obtained by recording sensory stimulus information.

[0078] The signal processing unit 31 is configured by a microcomputer including a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The signal processing unit 31 has functions as a position relationship information acquisition unit 35 and a correction unit 36.

[0079] The position relationship information acquisition unit 35 acquires information on a first position relationship that is the position relationship between the recording device 2 and the recording target in the recording environment, and information on a second position relationship that is the position relationship between the recording target (virtual position) and the user Hm in the playback environment of the recorded sensory stimulus information.

[0080] Here, the first position relationship and the second position relationship will be described with reference to FIG. 8. FIG. 8 conceptually shows the position relationship among the recording target, the recording device 2, and the user Hm. Here, similar to FIG. 4, the position of the recording target is shown as the target position T, the position of the recording device 2 is shown as the recording position R, and the position of the user Hm is shown as the somatosensory position A. Also, an example in which the video playback device 4a includes a screen as a display unit is shown.

[0081] On the recording environment side, the recording device 2 records sensory stimulus information propagated from recording targets such as basketballs and players in the basketball court BC. Also, on the playback environment side, as shown in FIG. 2, the user Hm wearing the acoustic playback device 4b and the vibration playback device 4c is enjoying the output video of the video playback device 4a on the screen.

[0082] In this case, the first position relationship is the position relationship between the target position T and the recording position R on the recording environment side. For example, the first position relationship can be represented by the distance Lx between the target position T and the recording position R.

[0083] Also, the second position relationship is the position relationship between the virtual position (target position T) of the recording target perceived by the user Hm via the screen and the somatosensory position A on the playback environment side. For example, the second position relationship can be represented by the distance Ly between the virtual target position T and the somatosensory position A. And the distance Ly between the virtual position of the recording target and the somatosensory position A is the distance obtained by adding the distance Lx between the target position T and the recording position R and the distance Lz from the screen (display unit of the video playback device 4a) to the somatosensory position A.

[0084] When the user Hm is wearing the playback device 4, the position of the worn playback device 4 is the body-sensed position A of the user Hm. Therefore, the second positional relationship can be considered as the positional relationship between the recording target at the virtual position and the position of the playback device 4 worn by the user Hm. Also, the positional relationship referred to here includes various positional relationships from one-dimensional to three-dimensional.

[0085] The positional relationship information acquisition unit 35 shown in FIG. 7 acquires information on the positional relationship (first positional relationship) between the target position T and the recording position R based on the acquired position information of the recording device 2. Also, the positional relationship information acquisition unit 35 specifies the positional relationship between the body-sensed position A and the recording position R by acquiring the position information of various playback devices 4, and acquires (calculates) the information on the second positional relationship based on the positional relationship between the body-sensed position A and the recording position R and the positional relationship (first positional relationship) between the recording position R and the target position T.

[0086] The correction unit 36 corrects the output signal to the playback device 4 that plays each sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship acquired by the positional relationship information acquisition unit 35. For example, the correction unit 36 adjusts the time until the sensory stimulus information reaches the user Hm (body-sensed position A) by correcting the output timing of the output signal. Also, the correction unit 36 adjusts the intensity of the sensory stimulus information sensed by the user Hm at the body-sensed position A by correcting the signal intensity of the output signal.

[0087] The correction unit 36 corrects the output signal of the playback device 4 according to changes in the first positional relationship and changes in the second positional relationship. The first positional relationship changes, for example, according to the movement of the recording target in the recording environment, the movement of the recording device 2, etc. Also, the second positional relationship changes, for example, due to the movement of the user Hm in the playback environment, the movement of various playback devices 4, etc. Details of the process of correcting the output signal of the playback device 4 will be described later. As described above, various functions of the signal processing unit 31 in the present embodiment are realized.

[0088] The reproduction data generation unit 32 performs predetermined processes such as codec processing and packetization processing according to the output destination reproduction device 4 on the output signal corrected by the signal processing unit 31.

[0089] The storage unit 33 temporarily stores the reproduction data generated by the reproduction data generation unit 32. The storage unit 33 may be a removable recording medium such as a memory card, an optical disk, a magnetic tape, etc., or may be a fixed type HDD (Hard Disk Drive), a semiconductor memory module, etc. When the storage unit 33 is a removable recording medium, it is also possible to supply the encoded data Ec to the decoding unit 30 by connecting the recording medium on which the encoded data Ec is recorded to the reproduction control device 3.

[0090] The communication unit 34 communicates with external devices in the same manner as the communication unit 19 in FIG. 6. As a result, the communication unit 34 receives the encoded data Ec from the recording device 2. In addition, the communication unit 34 transmits the reproduction data generated by the reproduction data generation unit 32 to various reproduction devices 4. Note that the communication unit 34 may acquire position information and the like from various reproduction devices 4. As described above, the reproduction control device 3 in the present embodiment is realized.

[0091] <5. Correction Process of Sensory Stimulation Information> The correction process of the sensory stimulation information executed by the signal processing unit 31 of the reproduction control device 3 in the present embodiment will be described with reference to FIG. 9. Here, as an example, as shown in FIG. 2, an example will be described in which the user Hm wears the acoustic reproduction device 4b and the vibration reproduction device 4c and views the video displayed on the display unit of the video reproduction device 4a.

[0092] First, in step S101, the signal processing unit 31 calculates the distance between the recording target and the recording device 2. For example, the signal processing unit 31 may acquire the information of the target position T and the recording position R and calculate the distance Lx, or may acquire the distance information from the recording device 2 to the target position T acquired by the recording device 2 as the distance Lx.

[0093] Subsequently, in step S102, the signal processing unit 31 calculates the distance Ly between the recording target and the user Hm. Here, as shown in FIG. 8, the distance Ly can be calculated by adding the distance Lx and the distance Lz. Since the distance Lx has been calculated by the process of step S101, the signal processing unit 31 can obtain the distance Ly by calculating the distance Lz.

[0094] The distance Lz can be calculated based on the positional relationship between the user Hm and the recording position R. Here, since the user Hm wears the acoustic playback device 4b, the position of the user Hm can be detected by detecting the position of the acoustic playback device 4b. According to the example of FIG. 2, it may also be possible to detect the position of the vibration playback device 4c worn by the user Hm.

[0095] Also, as shown in FIG. 8, the display unit of the video playback device 4a can be regarded as the recording position R. Therefore, the distance Lz between the recording position R and the user Hm can be regarded as the distance between the display unit of the video playback device 4a and the acoustic playback device 4b.

[0096] Several methods for calculating the distance between the display unit of the video playback device 4a and the acoustic playback device 4b can be considered. For example, a distance measuring unit may be provided in the video playback device 4a, and the distance measuring unit may calculate the distance to the acoustic playback device 4b (or the user Hm). Alternatively, the distance may be calculated from the difference between the position information of the display unit of the video playback device 4a and the position information of the acoustic playback device 4b in the same coordinate system. Also, when the seat position is determined in advance, such as in a movie theater, the distance between the seat used by the user Hm and the screen may be obtained from a database storing the distance information between the seat position and the screen.

[0097] In this way, the signal processing unit 31 calculates (obtains) the distance between the target position T (virtual position) and the somatosensory position A. The distance Ly indicates a second positional relationship showing the virtual positional relationship between the recording target and the user Hm.

[0098] In step S103, the signal processing unit 31 corrects the output signal of the sensory stimulus information to each playback device 4. Based on the first positional relationship and the second positional relationship, the signal processing unit 31 corrects the output timing (delay amount) and the signal intensity (attenuation amount) of the output signal.

[0099] For example, assuming that the distance from the target position T to the recording position R shown in FIG. 8 is "Lx [m]", the virtual distance from the target position T to the somatic sensation position A is "Ly [m]", and the sound propagation speed is "Vaudio [m / s]", the correction amount "Taudio [mS]" of the delay amount of the acoustic information output timing is calculated by, for example, the following [Equation 1]. Taudio = (Ly - Lx) / Vaudio ··· [Equation 1] The value of Taudio calculated here indicates the time required for the sound to propagate from the recording position R to the somatic sensation position A.

[0100] Also, assuming that the vibration propagation speed is "Vhaptics [m / s]", the correction amount "Thaptics [mS]" of the delay amount of the vibration information output timing to be corrected is calculated by, for example, the following [Equation 2]. Thaptics = (Ly - Lx) / Vhaptics ··· [Equation 2] The value of Thaptics calculated here indicates the time required for the sound to propagate from the recording position R to the somatic sensation position A.

[0101] Also, the correction amount "Gaudio [dB]" of the attenuation amount of the acoustic information is calculated by, for example, the following [Equation 3]. In [Equation 3], "10" is the base of the logarithm, and (Ly / Lx) is the true number of the logarithm. Gaudio = 20log(10, (Ly / Lx)) ··· [Equation 3] The value of Gaudio calculated here indicates the attenuation amount while the sound reaches the somatic sensation position A from the recording position R.

[0102] The correction amount "Ghaptics [dB]" of the attenuation amount of the vibration information is calculated, for example, by the following [Equation 4]. Note that "10" in [Equation 4] is the base of the logarithm, and (Ly / Lx) is the true number of the logarithm. Ghaptics = 20 log(10,(Ly / Lx)) ··· [Equation 4] The value of Ghaptics calculated here indicates the attenuation amount while the sound reaches the somatosensory position A from the recording position R.

[0103] The signal processing unit 31 calculates various correction amounts of the acoustic information and the vibration information based on the above [Equation 1] to [Equation 4]. In addition, correction amounts of the delay amount and the attenuation amount of the output timing may be calculated also for the video information. However, since the delay amount and the attenuation amount of the video information are much smaller than the transmission system delay, the calculation of the correction amounts of the delay amount and the attenuation amount of the output timing regarding the video information is not performed here.

[0104] Also, the signal processing unit 31 performs correction according to the environmental information in the recording environment in the correction process of step S103. Specifically, the signal processing unit 31 corrects the delay amount and the attenuation amount of the output timing in the sensory stimulus information output to the playback device 4 according to the environmental information.

[0105] For example, when performing correction based on the air temperature, the signal processing unit 31 acquires the air temperature information of the recording environment from the recording device 2. The signal processing unit 31 calculates the correction amounts of the delay amount and the attenuation amount of the output timing of the acoustic information and the vibration information based on the acquired air temperature information.

[0106] The signal processing unit 31 may perform correction based on the air temperature information of the playback environment in the correction process of step S103. In this case, the signal processing unit 31 acquires the air temperature information of the playback environment from the playback device 4 provided with the temperature sensor, and calculates the correction amounts of the delay amount and the attenuation amount of the output timing of the acoustic information and the vibration information based on the acquired air temperature information.

[0107] Thus, even if the temperature of the playback environment where the user Hm actually is is different from the temperature of the recording environment, various sensory stimulus information is corrected so that the user Hm can experience in the playback environment the sensory stimuli that the user Hm would experience if the user Hm were in the recording environment.

[0108] Also, in the correction process of step S103, the signal processing unit 31 can perform correction according to the medium used for transmitting the sensory stimulus in the recording environment. For this purpose, the signal processing unit 31 acquires information on the medium of the recording environment from the recording device 2. The medium information is information for specifying, for example, by what the acoustic information or vibration information is transmitted. Specifically, it is information such as the information on water, air, or the material of the floor.

[0109] The signal processing unit 31 calculates correction amounts for the delay amount and attenuation amount of the output timing of the acoustic information and vibration information based on the medium information at the time of recording acquired from the recording device 2.

[0110] Also, the signal processing unit 31 may perform correction based on information on the medium of the playback environment from the playback device 4, for example. At this time, the signal processing unit 31 calculates correction amounts for the delay amount and attenuation amount of the output timing of the acoustic information and vibration information based on the medium information of the playback environment acquired from the playback device 4.

[0111] Thus, even if the medium of the playback environment (such as in water or in the air) is different from the medium of the recording environment, various sensory stimulus information is corrected so that the user Hm can experience in the playback environment the sensory stimuli that the user Hm would experience if the user Hm were in the recording environment.

[0112] Subsequently, in step S104, the signal processing unit 31 sets filter coefficients for correcting the delay amount and attenuation amount of the output timing of the sensory stimulus information based on the various correction amounts calculated in step S103. The signal processing unit 31 corrects various sensory stimulus information based on the filter coefficients. As a result, an output signal with an appropriate delay amount and attenuation amount based on the filter coefficient is output to the reproduction data generation unit 32.

[0113] In step S105, the reproduction control device 3 executes reproduction control processing based on the corrected sensory stimulus information. In the reproduction control processing, for example, the communication unit 34 transmits the reproduction data for the video reproduction device 4a generated by the reproduction data generation unit 32 from the video information obtained from the encoded data Ec to the video reproduction device 4a, causing the video reproduction device 4a to execute an output based on the video information.

[0114] Also, the communication unit 34 transmits the reproduction data for the audio reproduction device 4b generated by the reproduction data generation unit 32 from the corrected acoustic information to the audio reproduction device 4b, causing the audio reproduction device 4b to execute an output based on the corrected acoustic information.

[0115] Furthermore, the communication unit 34 transmits the reproduction data for the vibration reproduction device 4c generated by the reproduction data generation unit 32 from the corrected vibration information to the vibration reproduction device 4c, causing the vibration reproduction device 4c to execute an output based on the corrected vibration information.

[0116] When there is no change in the first positional relationship and the second positional relationship, the signal processing unit 31 proceeds with the processing in the order of steps S106, S107, and S108, and then until the information related to various sensory stimulus information such as the end of the content screening in step S111 and the information is no longer output to the reproduction device 4, the corrected acoustic information and vibration information are used for reproduction based on the filter coefficient set in step S104.

[0117] When a change occurs in the first positional relationship or the second positional relationship, steps S109 and S110 are executed based on the determination processing in steps S106, S107, and S108, and the filter coefficient is updated.

[0118] Specifically, in step S106, it is confirmed whether there is a change in the target position T of the recording target. If a change is recognized, the signal processing unit 31 proceeds to step S109. When the target position T changes, the positional relationship (distance Lx) between the recording target (target position T) and the recording device 2 (recording position R) and the positional relationship (distance Ly) between the recording target (target position T) and the user Hm (sensed position A) change. That is, the first positional relationship and the second positional relationship are in a changed state.

[0119] When the target position T of the recording target changes, for example, when the recording target moves during recording in the recording environment, or when the video of the recording target that is far away switches to the video of the recording target that is approaching during playback by the video playback device 4a in the playback environment, or when the scene (recording viewpoint) of the video switches. Also, as an example of the change in the target position T of the recording target, a case where the video switches when the user Hm turns around while wearing the head-mounted display HMD can be considered.

[0120] In this case, in step S109, the signal processing unit 31 calculates the distance Lx and the distance Ly again in the same manner as in step S103, and substitutes the calculated distance Lx and distance Ly into the above [Equation 1] to [Equation 4] to calculate the correction amounts for the delay amount and the attenuation amount of the input timing of the sensory stimulus information again.

[0121] Then, in step S110, the signal processing unit 31 updates (re-sets) the filter coefficients for correcting the delay amount and the attenuation amount of the output timing of the sensory stimulus information based on the correction amounts calculated in step S109. And the signal processing unit 31 corrects various sensory stimulus information based on the filter coefficients.

[0122] If the playback control process has not ended, the signal processing unit 31 proceeds to steps S111 and S105 in order. In step S105, the signal processing unit 31 executes the playback control process of the playback device 4 based on the sensory stimulus information corrected based on the updated filter coefficients. Thereafter, the signal processing unit 31 performs the same process as above.

[0123] Also, when the target position T to be recorded does not change in step S106, the signal processing unit 31 checks, in step S107, whether the recording position R of the recording device 2 changes. If a change is recognized, the process proceeds to step S109. As a result, the positional relationship (distance Lx) between the recording target (target position T) and the recording device 2 (recording position R) changes. That is, it becomes a state where the first positional relationship has changed.

[0124] When the recording position R changes, for example, it may be considered that the recording device 2 has moved in the recording environment, or the position of the display unit of the video playback device 4a that can be regarded as the recording position R has moved in the playback environment.

[0125] In this case, in step S109, the signal processing unit 31 calculates the distance Lx and the distance Ly again in the same manner as in step S103, and calculates again the correction amounts for the delay amount and the attenuation amount of the input timing of the sensory stimulus information based on the above [Equation 1] to [Equation 4].

[0126] Then, in step S110, the signal processing unit 31 updates (re-sets) the filter coefficients for correcting the delay amount and the attenuation amount of the output timing of the sensory stimulus information based on the correction amounts calculated in step S109. Then, the signal processing unit 31 corrects various sensory stimulus information based on the filter coefficients.

[0127] If the playback control process has not ended, the signal processing unit 31 proceeds with the processes in steps S111 and S105 in this order. In step S105, the signal processing unit 31 executes the playback control process of the playback device 4 based on the sensory stimulus information corrected based on the updated filter coefficients. Thereafter, the signal processing unit 31 performs the same processing as described above.

[0128] Also, when the recording position R of the recording device 2 does not change in step S107, the signal processing unit 31 checks, in step S108, whether the body sensation position A of the user Hm changes. If a change is recognized, the process proceeds to step S109.

[0129] At this time, as the perceived position A changes, the positional relationship (distance Ly) between the recording target (target position T) and the user Hm (perceived position) changes. That is, the second positional relationship is in a changed state.

[0130] When there is a change in the perceived position A of the user Hm, it is conceivable that the vehicle on which the user Hm is riding moves, or the user Hm himself / herself moves.

[0131] Note that the signal processing unit 31 can also assume that the perceived position A of the user Hm has changed when there is a change in the position of the playback device 4 worn by the user Hm. This is because the position of the playback device 4 can be regarded as the position of the user Hm. That is, it can be said that the positional relationship (distance Ly) between the recording target and the user Hm changes due to the change in the position of the playback device 4 being worn by the user Hm.

[0132] In this case, when the perceived position A of the user Hm changes in step S108, the signal processing unit 31 calculates the distance Lx and the distance Ly again in step S109 in the same manner as in step S103, and calculates the correction amounts of the delay amount and the attenuation amount of the input timing of the sensory stimulus information again based on the above [Equation 1] to [Equation 4].

[0133] Then, in step S110, the signal processing unit 31 updates (re-sets) the filter coefficients for correcting the delay amount and the attenuation amount of the output timing of the sensory stimulus information based on the correction amounts calculated in step S109. Then, the signal processing unit 31 corrects various sensory stimulus information based on the filter coefficients.

[0134] If the playback control process has not ended, the signal processing unit 31 proceeds with the processing in the order of steps S111 and S105. In step S105, the signal processing unit 31 executes the playback control process of the playback device 4 based on the sensory stimulus information corrected based on the updated filter coefficients. Thereafter, the signal processing unit 31 performs the same processing as above.

[0135] During the execution of the above processing, when there is no reproduction data to be transmitted to the playback device 4 in step S111, the signal processing unit 31 ends the processing shown in FIG. 9. Note that the signal processing unit 31 may end the processing shown in FIG. 9 by detecting the end of reproduction by the playback device 4. As described above, the correction processing of the sensory stimulation information by the signal processing unit 31 in the present embodiment is realized.

[0136] In the present embodiment, after the setting of the filter coefficient is completed in step S104, the signal processing unit 31 may recalculate the correction amounts of the delay amount and the attenuation amount of the output timing of the sensory stimulation information according to changes in the temperature information of the recording environment or the playback environment.

[0137] When the temperature information of the recording environment changes, it is conceivable that the video scene changes, such as when switching from an underwater video to an above-water video during playback by the video playback device 4a in the playback environment. When the temperature information of the playback environment changes, for example, it is conceivable that the temperature inside a movie theater changes due to an increase in the number of visitors or a change in air conditioning.

[0138] In this case, when the signal processing unit 31 detects a change in the temperature information of the recording environment or the playback environment, in step S109, it recalculates the correction amounts of the delay amount and the attenuation amount of the output timing of the sensory stimulation information according to the changed temperature information.

[0139] Then, based on the correction amount in step S110, the signal processing unit 31 updates (re-sets) the filter coefficient for correcting the delay amount and the attenuation amount of the output timing of the sensory stimulation information.

[0140] In the present embodiment, after the setting of the filter coefficient is completed in step S104, the signal processing unit 31 may recalculate the correction amounts of the delay amount and the attenuation amount of the output timing of the sensory stimulation information according to changes in the medium information of the recording environment or the playback environment. The case where the medium information of the recording environment changes includes cases where the video scene changes, such as when switching from an underwater video to an above-water video during playback by the video playback device 4a in the playback environment. The case where the medium information of the playback environment changes includes, for example, the vehicle of the user Hm experiencing an attraction entering the water.

[0141] In this case, when the signal processing unit 31 detects a change in the medium information of the recording environment or the playback environment, in step S109, it recalculates the correction amounts for the delay amount and the attenuation amount of the output timing of the sensory stimulation information according to the changed medium information.

[0142] Then, in step S110, the signal processing unit 31 updates (re-sets) the filter coefficients for correcting the delay amount and the attenuation amount of the output timing of the sensory stimulation information based on the correction amounts.

[0143] In the present embodiment, an example in which the signal processing unit 31, as the playback device 4, executes the correction process of the sensory stimulation information shown in FIG. 9 for the video playback device 4a, the audio playback device 4b, and the vibration playback device 4c has been described. However, the present embodiment can also be applied to examples in which the process is executed for, for example, the video playback device 4a and the audio playback device 4b, or examples in which the process is executed for the video playback device 4a and the vibration playback device 4c. That is, the correction process of the sensory stimulation information in the present embodiment can be realized by the cooperation of two or more playback devices 4.

[0144] <6. Summary> The information processing apparatus (signal processing unit 31 in the playback control apparatus 3) in the above-described embodiment of the present invention includes a recording apparatus 2 that performs recording in a recording environment for recording sensory stimulus information transmitted from a recording target, information on a first positional relationship that is the positional relationship between the recording apparatus 2 and the recording target, and information on a second positional relationship that is the positional relationship between the recording target and the user Hm in the playback environment of the recorded sensory stimulus information. The apparatus further includes a positional relationship information acquisition unit 35 that acquires the information, and a correction unit 36 that corrects an output signal of a playback apparatus 4 that plays back the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship (see FIGS. 7 and 9). Based on the information on the first positional relationship in the recording environment and the information on the second positional relationship in the playback environment, the acoustic signal of the acoustic playback apparatus 4b as the playback apparatus 4 and the output signal of the vibration playback apparatus 4c are corrected, so that the user Hm can experience sensory stimuli corresponding to the viewing position. Specifically, the closer the user Hm is to the screen or display (display unit of the video playback apparatus 4a), the greater the sound and vibration the user Hm can experience. As a result, the user Hm can receive highly immersive sensory stimulus information as if the user were in the recording environment.

[0145] In the present embodiment, when the playback apparatus 4 includes a plurality of types of playback apparatuses 4 corresponding to a plurality of types of the sensory stimulus information, the information on the second positional relationship is information on the relationship between the virtual position (target position T) of the recording target recognized by the user Hm via the display unit on which the image is played back and the position (sensory position A) of the user Hm (see FIGS. 4 and 8, etc.). That is, the output signals of the acoustic information and the vibration information (tactile information) are corrected according to the video information received by the user Hm. Therefore, since it is possible to provide the user Hm with stimuli such as sound and vibration according to the direction and distance from the recording target (for example, a basketball or a player in the case of recording a basketball game) that the user Hm virtually visually recognizes via the screen or display (display unit of the video playback apparatus 4a), it is possible to provide an immersive playback environment without giving the user Hm a sense of discomfort or uneasiness.

[0146] In the signal processing unit 31 of the present embodiment, the correction unit 36 performs the above correction in accordance with changes in the first positional relationship in the recording environment (see S106, S107, etc. in FIG. 9). As a result, correction of the sensory stimulus information is performed according to the case where the recording device 2 moves in the recording environment or the case where the recording target moves. Therefore, it is possible to provide sensory stimulus information with a signal intensity that does not cause discomfort to the user Hm. That is, it becomes possible to let the user Hm experience various stimuli without giving a sense of discomfort or unpleasantness.

[0147] In the present embodiment, the change in the first positional relationship is made in accordance with the change in the position of the recording target in the recording environment (see S106, etc. in FIG. 9). As a result, the output signal of the playback device 4 is corrected according to the change in the position of the recording target. That is, it becomes possible to make the user Hm perceive the change in the position of the recording target by changing various sensory stimulus information. Therefore, since the user Hm can recognize the change in the position of the recording target in a pluralistic manner, it is possible to enhance the sense of presence.

[0148] In the present embodiment, the change in the first positional relationship is made in accordance with the change in the position of the recording device 2 in the recording environment (see S107, etc. in FIG. 9). As a result, the output signal of the playback device 4 is corrected according to the change in the position of the recording device 2. That is, it becomes possible to make the user Hm perceive the change in the position of the recording device 2, in other words, the change in the viewpoint (or viewing angle) of the user Hm, by changing various sensory stimulus information. Therefore, since the user Hm can recognize the change in the position of the recording device 2 in a pluralistic manner, it is possible to enhance the sense of presence.

[0149] In the signal processing unit 31 of the present embodiment, the correction unit 36 performs the above correction in accordance with changes in the second positional relationship in the playback environment (see S108, etc. in FIG. 9). As a result, correction of the output signal of the playback device 4 is performed according to the case where the playback device 4 moves in the playback environment or the case where the user Hm moves. Therefore, it is possible to provide the sensory stimulation information to the user Hm with a signal strength that does not cause discomfort. That is, it is possible to allow the user Hm to experience various stimuli without giving a sense of discomfort or unpleasantness.

[0150] In the present embodiment, the change in the second positional relationship is made according to the change in the position of the user Hm in the playback environment (see S108 etc. in FIG. 9). Thereby, the output signal of the playback device 4 is corrected according to the change in the body-sensed position A of the user Hm. That is, it is possible to make the user Hm perceive the change in the position by changing various sensory stimulation information. For example, when the user Hm approaches the screen or display, the playback volume increases, or when the user Hm moves away from the screen or the like, the playback volume decreases. Therefore, it is possible to provide sensory stimulation information with a sense of presence as if the user Hm were in the recording environment.

[0151] In the present embodiment, the signal processing unit 31 is such that the change in the second positional relationship is made according to the change in the position of the playback device 4 in the playback environment (see S108 etc. in FIG. 9). Thereby, the output signal of the playback device 4 is corrected according to the change in the playback device 4. That is, it is possible to make the user Hm perceive the change in the position of the playback device 4 by changing various sensory stimulation information. Therefore, since the user Hm can recognize the change in the position of the recording target in a pluralistic manner, it is possible to enhance the sense of presence.

[0152] In the signal processing unit 31 of the present embodiment, the correction unit 36 corrects the output timing of the output signal (see S103, S105, S109 etc. in FIG. 9). Thereby, for example, when the position of the video playback device 4a (display unit such as a screen) that outputs video is far from the user Hm, it is possible to delay the output timing of the output signal of the audio playback device 4b as the playback device 4. Therefore, it is possible to provide the user Hm with the sensory stimulation information at an appropriate timing according to the positional relationship between the user Hm and the virtual recording target, and it is possible to enhance the sense of presence.

[0153] In the signal processing unit 31 of the present embodiment, the correction unit 36 corrects the signal intensity of the output signal (see S103, S109, etc. in FIG. 9). Thereby, for example, when the position of the video playback device 4a (display unit such as a screen) where the video is output is far from the user Hm, it is possible to make the signal intensity of the output signal of the audio playback device 4b as the playback device 4 smaller. Therefore, since the playback sound adjusted (corrected) to an appropriate playback volume according to the positional relationship between the user Hm and the virtual recording target is provided to the user Hm, it is possible to enhance the sense of presence.

[0154] In the signal processing unit 31 of the present embodiment, the correction unit 36 performs the above correction according to the environmental information in the recording environment (see S103, etc. in FIG. 9). Thereby, the output signal to the playback device 4 is corrected so as to provide a sensory stimulation equivalent to the case where the user Hm is actually in the recording environment. Therefore, it is possible to provide the user Hm with the sensory stimulation information as if the user Hm were actually in the recording environment, and it is possible to enhance the sense of presence.

[0155] In the present embodiment, the environmental information is temperature information (see S103, etc. in FIG. 9). Thereby, even if the temperature of the environment where the user Hm actually is is different from the temperature of the recording environment, the output signal to the playback device 4 is corrected so that the user Hm can experience the sensory stimulation that the user Hm would experience if the user Hm were in the recording environment in the playback environment. Therefore, it is possible to provide a playback environment with a high sense of presence as if the user Hm were actually in the recording environment.

[0156] In the present embodiment, the environmental information is medium information. (See S103, etc. in FIG. 9) As a result, even if the medium of the playback environment (such as in water or air) is different from the medium of the recording environment, or even if the temperature of the playback environment is different from the temperature of the recording environment in the atmosphere where the user Hm is actually located, the output signal to the playback device 4 is corrected so that the user Hm can experience in the playback environment the sensory stimuli that the user Hm would experience when in the recording environment. Therefore, it is possible to provide a playback environment with a high sense of presence as if the user Hm is actually in the recording environment.

[0157] In the signal processing unit 31 of the present embodiment, the correction unit 36 corrects the output signal of the playback device 4 that plays back acoustic information as sensory stimulus information (see S103, S109, etc. in FIG. 9). As a result, the output signal to the playback device 4 is corrected for acoustic information that is likely to have a shift in arrival time due to the difference in the positions of the recording position and the playback position. Therefore, it is possible to output from the playback device 4 acoustic information that does not give the user Hm a sense of discomfort.

[0158] In the signal processing unit 31 of the present embodiment, the correction unit 36 corrects the output signal of the playback device 4 that plays back vibration information as sensory stimulus information (see S103, S109, etc. in FIG. 9). As a result, the output signal to the playback device 4 is corrected for vibration information that is likely to have a shift in arrival time due to the difference in the positions of the recording position and the playback position. Therefore, it is possible to output from the playback device 4 acoustic information that does not give the user Hm a sense of discomfort.

[0159] Also, the recording and playback system in this embodiment includes a sensory stimulus information acquisition unit that acquires sensory stimulus information transmitted from a recording target, and a detection unit that detects information on a first positional relationship between the recording device 2 (recording position R) and the recording target (target position T). The recording device 2 includes a positional relationship information acquisition unit 35 that acquires the information on the first positional relationship and information on a second positional relationship between the recording target (target position T) and the user Hm (somatosensory position) in the playback environment of the recorded sensory stimulus information, and a correction unit 36 that corrects the output signal of a playback device that plays back the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. (See FIGS. 1, 9, etc.) With such a recording and playback system as an embodiment, the same operations and effects as those of the above-described embodiment can be obtained.

[0160] Also, the information processing method in this embodiment is an information processing method in which an information processing device executes a positional relationship information acquisition process of acquiring information on a first positional relationship between the recording device 2 (recording position R) and the recording target (target position T), which is the positional relationship in the recording environment where the sensory stimulus information transmitted from the recording target is recorded, and information on a second positional relationship between the recording target (target position T) and the user Hm (somatosensory position) in the playback environment of the recorded sensory stimulus information, and a correction process of correcting the output signal of the playback device 4 that plays back the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. With such an information processing method as an embodiment, the same operations and effects as those of the information processing device as the above-described embodiment can be obtained.

[0161] Here, the functions of the information processing device described so far can be realized as software processing by, for example, a CPU, a DSP, etc. The software processing is executed based on a program.

[0162] As an embodiment, the program causes an information processing device to execute a position relationship information acquisition function that acquires information on a first position relationship between a recording device that performs recording in a recording environment for recording sensory stimulus information transmitted from a recording target and the recording target, and information on a second position relationship between the recording target and a user in a playback environment of the recorded sensory stimulus information. The program is a program that causes the information processing device to execute a correction function that corrects an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship. By such a program, the information processing device as described in the above embodiment can be realized as a playback control device 3. In the present embodiment, the present technology is realized by the playback control device 3 performing the processing of FIG. 9 based on the above program. However, by recording the above program in the recording device 2 or the playback device 4, the present technology can also be realized by the processing executed by the recording device 2 or the playback device 4. Further, the processing may be realized by an external server executing the above program.

[0163] The above program can be pre-recorded in a recording medium built in a device such as a computer device, or in a ROM or the like in a microcomputer having a CPU. Alternatively, it can be temporarily or permanently stored (recorded) in a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD, a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such a removable recording medium can be provided as so-called packaged software. In addition to being installed from a removable recording medium to a personal computer or the like, the above program can also be downloaded from a download site via a network such as the Internet or a LAN (Local Area Network).

[0164] Also, according to this program, it is suitable for wide provision of the playback control device 3 of the embodiment. For example, by downloading the program to a personal computer, a portable information processing device, a mobile phone, a game device, an AV (Audio Visual) device, etc., the personal computer, etc. can be made to function as the playback control device 3.

[0165] <7. The present technology> Also, the present technology can also adopt the following configuration. (1) A position relationship information acquisition unit that acquires information on a first position relationship that is the positional relationship between a recording device that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target, and the recording target, and information on a second position relationship that is the positional relationship between the recording target and a user in the playback environment of the recorded sensory stimulus information; A correction unit that corrects an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship; An information processing device. (2) When the playback device includes a plurality of types of playback devices corresponding to a plurality of types of the sensory stimulus information, the information on the second position relationship is information on the relationship between the virtual position of the recording target recognized by the user via a display unit on which an image is played back and the position of the user. The information processing device according to (1) above. (3) The correction unit performs the correction in response to a change in the first position relationship in the recording environment. The information processing device according to (1) or (2) above. (4) The change in the first position relationship is in response to a change in the position of the recording target in the recording environment. The information processing device according to (3) above. (5) The change in the first position relationship is in response to a change in the position of the recording device in the recording environment. The information processing apparatus according to (3) or (4) above. (6) The correction unit performs the correction according to a change in the second positional relationship in the playback environment. The information processing apparatus according to any one of (1) to (5) above. (7) The change in the second positional relationship is assumed to be according to a change in the position of the user in the playback environment. The information processing apparatus according to (6) above. (8) The change in the second positional relationship is assumed to be according to a change in the position of the playback apparatus in the playback environment. The information processing apparatus according to (6) or (7) above. (9) The correction unit corrects the output timing of the output signal. The information processing apparatus according to any one of (1) to (8) above. (10) The correction unit corrects the signal strength of the output signal. The information processing apparatus according to any one of (1) to (9) above. (11) The correction unit performs the correction based on a difference between the environmental information in the recording environment and the environmental information in the playback environment. The information processing apparatus according to any one of (1) to (10) above. (12) The difference in the environmental information is assumed to be a difference in temperature. The information processing apparatus according to (11) above. (13) The difference in the environmental information is assumed to be a difference in medium. The information processing apparatus according to (11) above. (14) The correction unit corrects the output signal of the playback apparatus that plays back acoustic information as the sensory stimulus information. The information processing apparatus according to any one of (1) to (13) above. (15) The correction unit corrects the output signal of the playback apparatus that plays back vibration information as the sensory stimulus information. The information processing apparatus according to any one of (1) to (14) above. (16) Position relationship information acquisition processing for acquiring information on a first position relationship that is the positional relationship between a recording device that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target and the recording target, and information on a second position relationship that is the positional relationship between the recording target and a user in the playback environment of the recorded sensory stimulus information; Correction processing for correcting an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship; An information processing method executed by an information processing apparatus. (17) A position relationship information acquisition function for acquiring information on a first position relationship that is the positional relationship between a recording device that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target and the recording target, and information on a second position relationship that is the positional relationship between the recording target and a user in the playback environment of the recorded sensory stimulus information; A correction function for correcting an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship; A program for causing an information processing apparatus to execute. (18) In a recording / playback system including a recording device and a playback control device, The recording device includes: A sensory stimulus information acquisition unit that acquires sensory stimulus information transmitted from a recording target; A detection unit that detects information on a first position relationship that is the positional relationship between the recording device and the recording target; The playback control device includes: A position relationship information acquisition unit that acquires the information on the first position relationship and information on a second position relationship that is the positional relationship between the recording target and a user in the playback environment of the recorded sensory stimulus information; A correction unit that corrects an output signal of a playback device that plays back the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship; A recording and playback system comprising

[0166] Finally, the effects described in this disclosure are illustrative and not limiting, and there may be other effects, or some of the effects described in this disclosure may be achieved. Also, the embodiments described in this disclosure are merely examples, and the technology is not limited to the above-described embodiments. Therefore, various modifications can be made according to the design and the like as long as they do not deviate from the technical idea of the technology even if they are outside the above-described embodiments. Note that not all combinations of the configurations described in the embodiments are essential for solving the problems.

Explanation of Reference Numerals

[0167] 1 Recording and playback system 2 Recording device 3 Reproduction control device 4 Reproduction device 4a Video reproduction device 4b Audio reproduction device 4c Vibration reproduction device 10 Camera unit 11 Image signal processing unit 12 Audio input unit 13 Audio signal processing unit 14 Vibration detection unit 15 Vibration signal processing unit 16 Sensor unit 17 Encoding unit 30 Decoding unit 31 Signal processing unit 35 Position relationship information acquisition unit 36 Correction unit T Target position R Recording position A, B Somatosensory positions Hm User

Claims

1. A position relationship information acquisition unit that acquires information on a first position relationship that is the positional relationship between a recording device that performs recording in a recording environment for recording sensory stimulus information transmitted from a recording target, and the recording target, and information on a second position relationship that is the positional relationship between the recording target and a user in a reproduction environment of the recorded sensory stimulus information; A correction unit that corrects an output signal of a reproduction device that reproduces the sensory stimulus information based on the information on the first position relationship and the information on the second position relationship; and When the reproduction device includes a plurality of types of reproduction devices corresponding to a plurality of types of the sensory stimulus information, the information on the second position relationship is information on the relationship between the virtual position of the recording target recognized by the user via a display unit on which an image is reproduced and the position of the user An information processing apparatus.

2. The correction unit performs the correction in response to a change in the first position relationship in the recording environment The information processing apparatus according to Claim 1.

3. The change in the first position relationship is made in response to a change in the position of the recording target in the recording environment The information processing apparatus according to Claim 2.

4. The change in the first position relationship is made in response to a change in the position of the recording device in the recording environment The information processing apparatus according to Claim 2.

5. The correction unit performs the correction in response to a change in the second position relationship in the reproduction environment The information processing apparatus according to Claim 1.

6. The change in the second position relationship is made in response to a change in the position of the user in the reproduction environment The information processing apparatus according to Claim 5.

7. The change in the second position relationship is made in response to a change in the position of the reproduction device in the reproduction environment The information processing apparatus according to Claim 5.

8. The correction unit corrects the output timing of the output signal The information processing apparatus according to Claim 1.

9. The correction unit corrects the signal strength of the output signal The information processing apparatus according to Claim 1.

10. The correction unit performs the correction according to environmental information in the recording environment The information processing apparatus according to Claim 1.

11. The environmental information is temperature information The information processing apparatus according to Claim 10.

12. The environmental information is medium information The information processing apparatus according to Claim 10.

13. The correction unit corrects an output signal of a reproduction device that reproduces acoustic information as the sensory stimulus information The information processing apparatus according to claim 1.

14. A recording apparatus that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target, information on a first positional relationship that is the positional relationship between the recording apparatus and the recording target, and the recorded sensory stimulus information. A positional relationship information acquisition unit that acquires information on a second positional relationship that is the positional relationship between the recording target and the user in the reproduction environment; A correction unit that corrects an output signal of a reproduction apparatus that reproduces the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. The correction unit corrects an output signal of the reproduction apparatus that reproduces vibration information as the sensory stimulus information. Information processing apparatus.

15. A positional relationship information acquisition process for acquiring information on a first positional relationship that is the positional relationship between a recording apparatus that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target and the recording target, and the recorded sensory stimulus information. Information on a second positional relationship that is the positional relationship between the recording target and the user in the reproduction environment; A correction process for correcting an output signal of a reproduction apparatus that reproduces vibration information as the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. An information processing method executed by an information processing apparatus.

16. A positional relationship information acquisition function for acquiring information on a first positional relationship that is the positional relationship between a recording apparatus that performs the recording in a recording environment for recording sensory stimulus information transmitted from a recording target and the recording target, and the recorded sensory stimulus information. Information on a second positional relationship that is the positional relationship between the recording target and the user in the reproduction environment; A correction function for correcting an output signal of a reproduction apparatus that reproduces vibration information as the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship. A program for causing an information processing apparatus to execute.

17. In a recording / reproduction system including a recording apparatus and a reproduction control apparatus, The recording apparatus includes: A sensory stimulus information acquisition unit that acquires sensory stimulus information transmitted from a recording target; A detection unit that detects information on a first positional relationship that is the positional relationship between the recording apparatus and the recording target; The reproduction control apparatus includes: A positional relationship information acquisition unit that acquires the information on the first positional relationship and information on a second positional relationship that is the positional relationship between the recording target and the user in the reproduction environment of the acquired sensory stimulus information; A correction unit that corrects an output signal of a playback device that plays back vibration information as the sensory stimulus information based on the information on the first positional relationship and the information on the second positional relationship; A recording and playback system comprising the same.

Images