Terminal device, method, and program

Abstract

To record memories of activities on a VR space.SOLUTION: A terminal device includes: a calculation unit that calculates a relative position between a display device that displays virtual reality used by a user and a terminal device actually held by the user; a display control unit that performs display of a virtual terminal device according to a control signal from the terminal device on a space of virtual reality and changes an imaging range displayed on the virtual terminal device on the basis of the relative position on the space of virtual reality displayed on the display device; an image data acquisition unit that acquires image data about an image on the space of the imaging range changed by the display control unit on the space of virtual reality; a storage unit that stores information; and a storage control unit that stores, in the storage unit, the imaging data of the imaging range acquired by the image data acquisition unit on the basis of saving operation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a terminal device, a method, and a program. [Background technology]

[0002] 2. Description of the Related Art There is known a technology that enables multiple users to experience a virtual reality (VR) space.

[0003] For example, Patent Document 1 describes a technique that enables multiple users to experience VR while simultaneously moving the position and direction of their respective viewpoints within a VR space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-211835 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is expected that various communications between multiple users will be realized in the VR space. For example, it is expected that in the VR space, each user will use an avatar that they created individually to virtually go to an exhibition facility such as a museum, a theme park, travel, or participate in a common game. Therefore, it is desirable to record the scenery or a group photo of the avatars as a memory in the VR space by performing a predetermined operation in the real world.

[0006] An object of the present invention is to provide a terminal device, a method, and a program that can record memories of activities in a VR space. [Means for solving the problem]

[0007] A terminal device according to one aspect of the present invention includes a display device that displays a virtual reality used by a user, a calculation unit that calculates a relative position between the display device and a terminal device that is actually held by the user, a display control unit that displays a virtual terminal device in the virtual reality space in accordance with a control signal from the terminal device and changes a shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device, and a video data acquisition unit that acquires video data relating to an image in the space of the shooting range changed by the display control unit in the virtual reality space. The imaging device includes a storage unit that stores information, and a storage control unit that stores the imaging data of the imaging range acquired by the video data acquisition unit in the storage unit based on a save operation.

[0008] A method according to one aspect of the present invention includes a step of calculating a relative position between a display device that displays a virtual reality used by a user and a terminal device that is actually held by the user, a step of displaying a virtual terminal device in the virtual reality space according to a control signal from the terminal device, and a step of changing a shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device, a step of acquiring video data relating to an image in the space of the changed shooting range in the virtual reality space, and a step of saving the shooting data of the shooting range in a memory unit of the terminal device based on a save operation. Includes.

[0009] A program according to one aspect of the present invention causes a computer to execute the following steps: calculating the relative position between a display device that displays a virtual reality used by a user and a terminal device actually held by the user; displaying a virtual terminal device in the virtual reality space in accordance with a control signal from the terminal device, and changing the shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device; acquiring video data relating to an image in the space of the changed shooting range in the virtual reality space; and saving the shooting data of the shooting range in a memory unit of the terminal device based on a save operation. Effect of the Invention

[0010] According to the present invention, memories of activities in a VR space can be recorded. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of the configuration of a terminal device according to the embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an identifier displayed on the display unit. [Figure 4] FIG. 4 is a block diagram illustrating an example of the configuration of the display device according to the embodiment. [Diagram 5] FIG. 5 is a block diagram illustrating an example of the configuration of a server according to the embodiment. [Figure 6] FIG. 6 is a diagram for explaining a method of using the terminal device and the display device according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining a method of using the terminal device and the display device according to the embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process flow of the terminal device according to the embodiment. [Figure 9] FIG. 9 is a block diagram showing an example of the configuration of a terminal device according to the first modified example. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a terminal device according to the second modified example. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of a display device according to the second modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and when there are multiple embodiments, the present invention also includes a configuration in which each embodiment is combined. In the following embodiments, the same parts are given the same reference numerals, and duplicated explanations will be omitted. [Information Processing System] An information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an example of the configuration of the information processing system according to an embodiment.

[0013] As shown in FIG. 1, the information processing system 1 includes a terminal device 10, a display device 20, and a server 30. The terminal device 10, the display device 20, and the server 30 are communicably connected via a network N. The information processing system 1 is a system that can record memories in a VR space. The terminal device 10 is a device including, for example, a smartphone, a tablet terminal, and the like. The display device 20 is a device that can experience VR, for example. The display device 20 is a device including, for example, a head mounted display (HMD) that is worn on the user's head. For example, while wearing the display device 20 on the head, the user can experience a theme park in a VR space using an avatar with other users on the server 30. [Terminal Device] The configuration of a terminal device according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of a terminal device according to an embodiment.

[0014] 2, the terminal device 10 includes a display unit 12, an audio output unit 13, an operation unit 14, a storage unit 15, a communication unit 16, and a control unit 17. The display unit 12, the audio output unit 13, the operation unit 14, the storage unit 15, the communication unit 16, and the control unit 17 are each connected to each other via a bus B1.

[0015] The imaging unit 11 captures various images around the terminal device 10. For example, the imaging unit 11 captures an image of the display device 20. The imaging unit 11 includes an imaging element (not shown) and a circuit that generates image data based on the output of the imaging element. Examples of the imaging element include, but are not limited to, a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a CCD (Charge Coupled Device).

[0016] The display unit 12 displays various information. For example, the display unit 12 displays various images. For example, the display unit 12 displays an identifier for calculating the relative position between the terminal device 10 and the display device 20. The identifier will be described later. For example, the display unit 12 is configured with a display including a liquid crystal display (LCD) or an organic electro-luminescence (Organic EL) display.

[0017] The audio output unit 13 outputs various sounds. The audio output unit 13 can be realized by, for example, a speaker.

[0018] The operation unit 14 accepts various operations from the user on the terminal device 10. The operation unit 14 is composed of, for example, buttons, switches, a touch panel, and the like. The operation unit 14 accepts, for example, an operation to start or end communication with the display device 20. The operation unit 14 accepts, for example, an operation to capture and acquire an image displayed on the display device 20, and an operation to store the captured image in the storage unit 15.

[0019] The storage unit 15 stores various information. For example, the storage unit 15 stores image data of various objects such as scenery, buildings, and characters in the VR space displayed on the display device 20. The storage unit 15 may store image data rendered into a two-dimensional image, or may store only information specifying a three-dimensional object and a shooting range, or information specifying a shooting range. The storage unit 15 can be realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or a solid state drive.

[0020] The communication unit 16 executes communication between the terminal device 10 and an external device. The communication unit 16 executes communication with the display device 20, for example.

[0021] The control unit 17 controls the operation of each unit of the terminal device 10. The control unit 17 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing a program (for example, the program according to the present invention) stored in a storage unit (not shown) using a RAM or the like as a working area. The control unit 17 may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 17 may be realized by a combination of hardware and software.

[0022] The control unit 17 includes a calculation unit 171 , a video data acquisition unit 172 , a display control unit 173 , an output control unit 174 , an operation control unit 175 , an imaging control unit 176 , a storage control unit 177 , and a communication control unit 178 .

[0023] The calculation unit 171 calculates the relative position between the terminal device 10 and the display device 20. The calculation unit 171 calculates the relative distance between the terminal device 10 and the display device 20 based on, for example, the size of the identifier displayed on the display unit 12 captured by an external imaging device acquired by the video data acquisition unit 172. The calculation unit 171 may calculate the relative distance between the terminal device 10 and the display device 20 based on, for example, a table (not shown) in which the size of the identifier included in the captured video is associated with the relative distance between the terminal device 10 and the display device 20. The calculation unit 171 calculates the relative inclination of the terminal device 10 with respect to the display device 20 based on, for example, the degree and direction of distortion of the identifier displayed on the display unit 12. The calculation unit 171 calculates the relative position between the terminal device 10 and the display device 20 based on, for example, the relative distance between the terminal device 10 and the display device 20 and the relative inclination of the terminal device 10 with respect to the display device 20.

[0024] FIG. 3 is a diagram for explaining the identifier displayed on the display unit 12. As shown in FIG. 3, the calculation unit 171 calculates the relative distance between the terminal device 10 and the display unit 20 based on, for example, the size of the identifier 111 displayed on the display unit 12 captured by the imaging unit 21 of the display unit 20. The calculation unit 171 calculates the relative inclination of the terminal device 10 with respect to the display unit 20 based on, for example, the degree and direction of distortion of the identifier 111 displayed on the display unit 12 captured by the imaging unit 21 of the display unit 20. The calculation unit 171 calculates the relative position of the terminal device 10 and the display unit 20 based on, for example, the relative distance between the terminal device 10 and the display unit 20 and the relative inclination of the terminal device 10 with respect to the display unit 20. The calculation unit 171 may calculate the relative position of the terminal device 10 and the display unit 20 based on, for example, the size of the identifier 111 captured by another external camera. The identifier 111 is displayed, for example, at the four corners of the display unit 12. The identifier 111 is, for example, a QR code (registered trademark), but is not limited to this. Note that the identifier 111 may be displayed alone, for example, in the center of the display unit 12.

[0025] The calculation unit 171 may also calculate the relative position between the terminal device 10 and the display device 20 based on the imaging data of the display device 20 captured by the imaging unit 11. The calculation unit 171 may, for example, determine the size of the display device 20 included in the imaging data using a known image recognition process, and calculate the relative position between the terminal device 10 and the display device 20 based on the determined size. In this case, the calculation unit 171 may calculate the relative distance between the terminal device 10 and the display device 20 based on, for example, a table in which the size of the display device 20 and the relative distance between the terminal device 10 and the display device 20 are associated with each other. The calculation unit 171 may calculate the relative inclination of the terminal device 10 with respect to the display device 20 based on, for example, the degree and direction of distortion of the display device 20.

[0026] The calculation unit 171 may calculate the relative position between the terminal device 10 and the display device 20 based on the image data of an identifier such as a QR code provided on the housing of the display device 20 captured by the imaging unit 11. The calculation unit 171 may, for example, determine the size and distortion of the identifier included in the captured data using a known image recognition process, and calculate the relative distance between the terminal device 10 and the display device 20 and the relative inclination of the terminal device 10 with respect to the display device 20 based on the determined size. The calculation unit 171 may calculate the relative position between the terminal device 10 and the display device 20 by grasping the relative distance between the terminal device 10 and the display device 20 and the relative inclination of the terminal device 10 with respect to the display device 20 based on the determined size and distortion of the identifier. In this case, the calculation unit 171 may calculate the relative distance between the terminal device 10 and the display device 20 based on, for example, a table in which the size of the identifier and the relative distance between the terminal device 10 and the display device 20 are associated with each other. The calculation unit 171 may calculate the relative inclination of the terminal device 10 with respect to the display device 20 based on, for example, the degree and direction of distortion of the identifier.

[0027] The video data acquisition unit 172 acquires various types of video data. For example, the video data acquisition unit 172 acquires video data captured by a virtual terminal device in the VR space displayed on the display device 20 from the display device 20. For example, when the operation control unit 175 acquires an operation signal for capturing an image in the VR space displayed on the display device 20 input to the operation unit 14, the video data acquisition unit 172 acquires video data captured by a virtual terminal device in the VR space displayed on the display device 20. Shooting by a virtual terminal device in the VR space refers to generating an image capturing a predetermined range of the VR space from the position of a virtual camera provided in the virtual terminal device in the VR space. The virtual terminal device in the VR space may have the same shape and size as the terminal device 10, or may be different. When the terminal device 10 is provided with an imaging unit 11, the position of the virtual camera in the virtual terminal device may be the same as the position of the imaging unit 11 in the terminal device 10, or may be different. When the terminal device 10 includes the imaging unit 11, the characteristics of the virtual camera may be the same as or different from the characteristics of the imaging unit 11. The characteristics of the virtual camera may be appropriately changed by the user through an operation in the VR space, such as wide angle or telephoto (zoom in, zoom out). The user may be able to perform operations in the VR space through an operation screen displayed on the display unit of the virtual terminal device.

[0028] The display control unit 173 causes various images to be displayed on the display unit 12. For example, the display control unit 173 causes the display unit 12 to display an identifier for calculating the relative position between the terminal device 10 and the display device 20. The display control unit 173 displays, on the display unit 12, an image related to the image data captured in the VR space, which the image data acquisition unit 172 has acquired from the display device 20.

[0029] The display control unit 173 controls the shooting range displayed on the display unit of the virtual terminal device when shooting various objects in the VR space with a virtual terminal device in the VR space displayed on the display device 20, for example. The display control unit 173 controls the shooting range displayed on the display unit of the virtual terminal device in the VR space based on the relative positions of the real terminal device 10 and the display device 20, for example. Specifically, the display control unit 173 controls the shooting range displayed on the display unit of the virtual terminal device in the VR space to change according to the change in the relative positions of the real terminal device 10 and the display device 20, for example. The display control unit 173 outputs a control signal for changing the shooting range displayed on the virtual terminal device in the VR space to the display device 20 via the communication unit 16, for example.

[0030] The display control unit 173 generates an avatar of the user who uses the terminal device 10 and the display device 20, for example, to be displayed in the image displayed on the display device 20. The display control unit 173 generates the avatar, for example, based on image data of the face of the user who uses the terminal device 10 and the display device 20. The display control unit 173 may change each part constituting the face of the avatar, for example, based on operation information received from the user by the operation unit 14. The display control unit 173 outputs, for example, a control signal for displaying the generated avatar on the display device 20 to the display device 20 via the communication unit 16.

[0031] The output control unit 174 controls the audio output unit 13 to output audio.

[0032] The operation control unit 175 acquires an operation signal related to an operation input to the operation unit 14. For example, the operation control unit 175 outputs a control signal related to the acquired operation signal to control the operation of the terminal device 10. For example, the operation control unit 175 acquires an operation signal related to an operation for shooting an image in the VR space displayed on the display device 20 input to the operation unit 14.

[0033] The imaging control unit 176 controls the imaging unit 11. The imaging control unit 176 sets imaging conditions for the imaging unit 11 and causes the imaging unit 11 to capture an image. The imaging control unit 176 controls the imaging unit 11, for example, to capture an image of an identifier such as a QR code provided on the housing. Note that when an identifier such as a QR code is displayed on the display unit 12 of the terminal device 10, the imaging unit 11 and the imaging control unit 176 are not necessary.

[0034] The storage control unit 177 stores various data in the storage unit 15. For example, the storage control unit 177 stores in the storage unit 15 video data related to a video captured in the VR space and displayed on the display device 20, which is acquired by the video data acquisition unit 172.

[0035] The communication control unit 178 controls the communication unit 16 to control communication between the terminal device 10 and an external device. The communication control unit 178 controls the communication unit 16 to control communication between the terminal device 10 and the display device 20, for example. [Display device] The configuration of the display device according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the display device according to the embodiment.

[0036] 4, the display device 20 includes an imaging unit 21, a display unit 22, an audio output unit 23, an operation unit 24, and a communication unit 25. The imaging unit 21, the display unit 22, the audio output unit 23, the operation unit 24, and the communication unit 25 are each connected to each other via a bus B3.

[0037] In this embodiment, the display device 20 is described as a device including an HMD that a user wears on his / her head, but is not limited thereto. The display device 20 may be, for example, a display device that is installed on a desk in a room and includes a display including a liquid crystal display or an organic EL display.

[0038] The imaging unit 21 captures various images around the display device 20. For example, the imaging unit 21 captures an identifier 111 displayed on the display unit 12 of the terminal device 10. The imaging unit 21 includes an imaging element (not shown) and a circuit that generates video data based on the output of the imaging element. Examples of the imaging element include, but are not limited to, a CMOS image sensor and a CCD.

[0039] The display unit 22 displays various images. For example, the display unit 22 displays an image in a VR space. The display unit 22 is realized by, for example, an HMD.

[0040] The audio output unit 23 outputs various sounds. The audio output unit 23 can be realized by, for example, a speaker.

[0041] The operation unit 24 accepts various operations from a user on the display device 20. The operation unit 24 is composed of, for example, buttons, switches, etc. The operation unit 24 accepts, for example, an operation to start or end communication with the terminal device 10.

[0042] The communication unit 25 executes communication between the display device 20 and an external device. The communication unit 25 executes communication with, for example, the terminal device 10. The communication unit 25 executes communication with, for example, the server 30.

[0043] The control unit 26 controls the operation of each unit of the display device 20. The control unit 26 is realized, for example, by a CPU, an MPU, or the like executing a program stored in a storage unit (not shown) using a RAM or the like as a working area. The control unit 26 may be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 26 may be realized by a combination of hardware and software.

[0044] The control unit 26 includes an acquisition unit 261, a display control unit 262, an output control unit 263, an operation control unit 264, an imaging control unit 265, and a communication control unit 266. The acquisition unit 261, the display control unit 262, the output control unit 263, the operation control unit 264, the imaging control unit 265, and the communication control unit 266 are each connected to each other via a bus B4.

[0045] The acquisition unit 261 acquires various types of information. For example, the acquisition unit 261 acquires video data relating to the video of the identifier 111 displayed on the display unit 12 of the terminal device 10 captured by the imaging unit 21.

[0046] The display control unit 262 controls the image displayed on the display unit 22. For example, the display control unit 262 causes the display unit 22 to display an image related to VR. For example, the display control unit 262 causes the display unit 22 to display an image for allowing a plurality of users to experience a theme park or the like in a VR space. For example, the display control unit 262 changes the display range of the image displayed on the display unit 22 or displays an avatar in the image displayed on the display unit 22 according to a control signal received from the terminal device 10. For example, the display control unit 262 changes the shooting range displayed on the virtual terminal device in the VR space displayed on the display unit 22 according to a control signal received from the terminal device 10. The control signal received by the display control unit 262 from the terminal device 10 is, for example, a control signal generated based on a user's operation of an operation screen displayed on the display unit of the virtual terminal device in the VR space by operating the operation unit 14 of the terminal device 10. When the identifier 111 is displayed on the display unit 12 of the terminal device 10, the user is actually operating the terminal device 10 in his / her hand, but the identifier 111 is displayed on the display unit 12 and the operation screen is not displayed. On the other hand, a control signal generated based on the operation is transmitted from the terminal device 10 to the display device 20, and the display control unit 262 controls the operation screen displayed on the display unit of the virtual terminal device in the VR space corresponding to the terminal device 10 based on the control signal. The user perceives the behavior of the virtual operation screen visually from the display device 20, and perceives the actual operation by touch from the terminal device 10. The terminal device 10 may generate vibrations with an actuator (not shown) to provide feedback of the operation to the user.

[0047] The output control unit 263 outputs various sounds from the audio output unit 23. For example, the output control unit 263 outputs the sounds of other users in the VR space. For example, the output control unit 263 causes the audio output unit 23 to output the sounds of other users who are experiencing a theme park or the like together in the VR space.

[0048] The operation control unit 264 acquires an operation signal related to an operation input to the operation unit 24. The operation control unit 264 controls the operation of the display device 20, for example, by outputting a control signal related to the acquired operation signal.

[0049] The imaging control unit 265 controls the imaging unit 21. The imaging control unit 265 sets imaging conditions for the imaging unit 21 and causes the imaging unit 21 to capture an image.

[0050] The communication control unit 266 controls the communication unit 25 to control communication between the display device 20 and an external device. The communication control unit 266 controls, for example, the communication unit 25 to control communication between the terminal device 10 and the display device 20. The communication control unit 266 controls, for example, the communication unit 25 to control communication between the display device 20 and the server 30. [server] The configuration of the server according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the configuration of the server according to the embodiment.

[0051] As shown in FIG. 5, the server 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0052] The communication unit 31 executes communication between the server 30 and an external device. The communication unit 31 executes communication with, for example, the terminal device 10. The communication unit 31 executes communication with, for example, the display device 20.

[0053] The storage unit 32 stores various types of information. For example, the storage unit 32 stores map information in the VR space. The map information includes various types of information such as scenery, buildings, and characters in the VR space. The storage unit 32 can be realized by a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or a solid state drive.

[0054] The control unit 33 controls the operation of each unit of the server 30. The control unit 33 is realized, for example, by a CPU, an MPU, or the like executing a program stored in a storage unit (not shown) using a RAM or the like as a working area. The control unit 33 may be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 33 may be realized by a combination of hardware and software.

[0055] The control unit 33 generates a VR space based on the map information stored in the storage unit 32. The control unit 33 generates a VR space in which, for example, a virtual theme park can be experienced. By connecting the display device 20 and the server 30, the user of the display device 20 can experience, for example, a virtual theme park together with a plurality of users using an avatar. [How to use terminals and display devices] A method of using a terminal device and a display device according to an embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 and Fig. 7 are diagrams for explaining a method of using a terminal device and a display device according to an embodiment.

[0056] 6, the display device 20 is used by being worn on the head of a user U, for example. In other words, the user U can experience a virtual theme park in a VR space using the display device 20. For example, while using the display device 20, the user U can operate the terminal device 10 to take pictures of objects such as scenery, buildings, and characters as memories of the theme park in the VR space, or take pictures with multiple other users.

[0057] FIG. 7 shows the shooting data in the VR space. For example, by operating the terminal device 10, the user U can shoot an image including the avatar A1 of the user U and the avatars A2, A3, and A4 of other users in the VR space using a virtual terminal device in the VR space. The shot image may be a still image or a moving image. The display range of the shot image changes depending on the relative position between the terminal device 10 and the display device 20. In other words, the user U can change the shooting range of the VR space by adjusting the relative position between the terminal device 10 and the display device 20 in the real space. The shooting range in the VR space is displayed on the display unit of the virtual terminal device in the VR space. The shooting range displayed on the display unit of the virtual terminal device can be adjusted by adjusting the relative position between the terminal device 10 and the display device 20 in the real space. Specifically, by adjusting the relative positions of the terminal device 10 and the display device 20 in real space, the user U can adjust the shooting range in the VR space so that avatar A1, avatar A2, avatar A3, and avatar A4 appear in a single photograph. [Terminal device processing] A process flow of the terminal device according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of a process flow of the terminal device according to the embodiment.

[0058] First, the operation control unit 175 determines whether or not the shooting mode of the virtual camera in the VR space is set to the shooting mode for shooting with the out-camera via the operation unit 14 (step S10). If it is determined that the shooting mode is set to the shooting mode for shooting with the out-camera (step S10; Yes), the process proceeds to step S12. If it is determined that the shooting mode is not set to the shooting mode for shooting with the out-camera (step S10; No), the process proceeds to step S11.

[0059] If it is determined that the result of step S10 is No, the operation control unit 175 determines whether or not the shooting mode of the virtual camera in the VR space is set to the shooting mode for shooting with the in-camera via the operation unit 14 (step S11). If it is determined that the shooting mode is set to the shooting mode for shooting with the in-camera (step S11; Yes), the process proceeds to step S12. If it is determined that the shooting mode is not set to the shooting mode for shooting with the in-camera (step S11; No), the process proceeds to step S18.

[0060] If it is determined that the answer is Yes in step S10 and Yes in step S11, the calculation unit 171 calculates the relative position between the terminal device 10 and the display device 20 (step S12). Specifically, the calculation unit 171 calculates the relative position between the terminal device 10 and the display device 20 based on the size of the identifier 111 displayed on the display unit 12 of the terminal device 10 and captured by the imaging unit 21 of the display device 20. Then, the process proceeds to step S12.

[0061] The display control unit 173 recognizes the shooting range in the VR space displayed on the display device 20 based on the relative position calculated by the calculation unit 171 (step S13). Specifically, the display control unit 173 recognizes the shooting range displayed on the display unit of the virtual terminal device in the VR space. Then, the process proceeds to step S14.

[0062] The operation control unit 175 judges whether or not an operation to shoot the shooting range in the VR space recognized in step S13 has been accepted via the operation unit 14 (step S14). If it is judged that an operation to shoot the shooting range in the VR space has been accepted (step S14; Yes), the process proceeds to step S15. If it is judged that an operation to shoot the shooting range in the VR space has not been accepted (step S14; No), the process proceeds to step S18.

[0063] If it is determined as Yes in step S14, the video data acquisition unit 172 acquires shooting data related to the shooting range in the VR space from the display device 20 (step S15). Here, the display control unit 173 may display the video related to the shooting data in the VR space acquired by the video data acquisition unit 172 on the display unit 12. The shooting data may be a still image or a moving image. This allows the user to understand the video shot in the VR space. Then, the process proceeds to step S16.

[0064] The operation control unit 175 determines whether or not an operation to save the photographed data acquired in step S15 has been accepted via the operation unit 14 (step S16). If it is determined that an operation to save the photographed data has been accepted (step S16; Yes), the process proceeds to step S17. If it is determined that an operation to save the photographed data has not been accepted (step S16; No), the process proceeds to step S18.

[0065] If it is determined as Yes in step S16, the storage control unit 177 stores the photographed data acquired in step S15 in the storage unit 15 (step S17), and then the process proceeds to step S18.

[0066] The control unit 17 determines whether or not to end the process (step S18). Specifically, the operation control unit 175 determines to end the process when it receives an operation to end shooting or to turn off the power of the terminal device 10. When it is determined to end the process (step S18; Yes), the process of Fig. 8 ends. When it is determined not to end the process (step S18; No), the process proceeds to step S10.

[0067] As described above, in this embodiment, the shooting range in the VR space displayed on the display device 20 is shot by operating the terminal device 10, and the shot video data is stored in the terminal device 10. As a result, in this embodiment, the scenery in the VR space and memories of a virtual theme park can be recorded in the terminal device 10, making it easy to post photos shot in the VR space on a social network service (SNS).

[0068] In this embodiment, a user can experience a virtual theme park and communicate with other users using avatars in a VR space. In addition, in this embodiment, a user can take a group photo in the same VR space with his / her own avatar and the avatars of other users. [First Modification] A first modified example of this embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the configuration of a terminal device according to the first modified example.

[0069] As shown in FIG. 9, the terminal device 10A differs from the terminal device 10 shown in FIG. 2 in that a control unit 17A includes an attitude detection unit 179.

[0070] The posture detection unit 179 detects the posture of the user experiencing VR using the display device 20. The posture detection unit 179 detects the posture of each part including, for example, the user's head, both arms, and both legs. In the first modified example, for example, the video data acquisition unit 172 acquires shooting data of the user from an external shooting device (not shown). In this case, the posture detection unit 179 may detect the posture of the user based on the shooting data acquired by the video data acquisition unit 172. The posture detection unit 179 may detect the posture of the user using motion capture or other well-known techniques.

[0071] Display control unit 173 changes the posture of the user's avatar displayed on display device 20 based on the detection result of posture detection unit 179. For example, when posture detection unit 179 detects that the user is raising his / her right hand, display control unit 173 raises the right hand of the user's avatar. That is, display control unit 173 changes the posture of the avatar to match the posture of the user detected by posture detection unit 179.

[0072] As described above, the first modification of the present embodiment changes the posture of the avatar in the VR space in real time in accordance with the posture of the user in the real space. This allows the first modification of the present invention to easily change the posture of the avatar in the VR space, improving usability. [Second modified example] A second modified example of this embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing an example of the configuration of a terminal device according to the second modified example.

[0073] As shown in FIG. 10, the terminal device 10B differs from the terminal device 10 shown in FIG.

[0074] The sensor 18 includes various sensors. For example, the sensor 18 includes a sensor that detects the relative position between the terminal device 10B and the display device 20. Examples of the sensor 18 include, but are not limited to, a laser radar (for example, LIDAR: Laser Imaging Detection and Ranging), an infrared sensor including an infrared emitting unit and a light receiving sensor, and a ToF (Time of Flight) sensor.

[0075] The calculation unit 171 of the control unit 17B calculates the relative position between the terminal device 10B and the display device 20 based on the detection result by the sensor 18. That is, in the second modified example, the calculation unit 171 calculates the relative position between the terminal device 10B and the display device 20 by using a space grasping means different from the imaging unit 11.

[0076] As described above, in the second modified example of the present embodiment, the relative position between the terminal device 10B and the display device 20 is calculated based on the detection results of various sensors. This allows the second modified example to improve the degree of freedom in design. [Third Modification] A third modified example of this embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of the configuration of a display device according to the third modified example.

[0077] As shown in FIG. 11, the display device 20A differs from the display device 20 shown in FIG. 4 in that the display device 20A includes a sensor 27 and that the control unit 26A includes a calculation unit 267.

[0078] The sensor 27 includes various sensors. For example, the sensor 27 includes a sensor that detects the relative position between the terminal device 10B and the display device 20. Examples of the sensor 27 include, but are not limited to, a laser radar (for example, LIDAR), an infrared sensor including an infrared emitting unit and a light receiving sensor, and a ToF sensor.

[0079] The calculation unit 267 calculates the relative position between the terminal device 10 and the display device 20A based on the detection result by the sensor 27. That is, the relative position between the terminal device 10 and the display device 20A may be calculated by the display device 20A instead of the terminal device 10. In this case, the display device 20A calculates information on the calculated relative position between the terminal device 10 and the display device 20A to the terminal device 10 via the communication unit 25. Then, the display control unit 173 changes the shooting range displayed on the display unit of the virtual terminal device in the VR space displayed on the display device 20A based on the relative position between the terminal device 10 and the display device 20A calculated by the calculation unit 267, for example.

[0080] As described above, in the third modification of the present embodiment, the relative position between the terminal device 10 and the display device 20A is calculated by the display device 20A based on the detection result of the sensor. This allows the third modification to improve the degree of freedom in design.

[0081] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0082] 1. Information Processing Systems 10, 10A, 10B Terminal equipment 11,21 Imaging section 12,22 Display section 13,23 Audio output section 14,24 Operation section 15,32 Storage part 16,25,31 Communications Department 17, 17A, 17B, 26, 26A, 33 Control section 18,27 Sensors 20 Display device 30 Servers 171,267 Calculation section 172 Video data acquisition unit 173,262 Display control section 174,263 Output control section 175,264 Operation control section 176,265 Imaging control section 177 Memory control unit 178,266 Communication control section 179 Attitude detection unit 261 Acquisition Department

Claims

1. A calculation unit that calculates a relative position between a display device that displays a virtual reality used by a user and a terminal device that is actually held by the user; a display control unit that displays a virtual terminal device in the virtual reality space in accordance with a control signal from the terminal device, and changes a shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device; a video data acquisition unit that acquires video data relating to an image in a space of a shooting range changed by the display control unit in the virtual reality space; A storage unit that stores information; a storage control unit that stores the shooting data of the shooting range acquired by the video data acquisition unit in the storage unit based on a save operation.

2. an operation unit that receives an operation on the terminal device from the user and generates a control signal in response to the operation; The terminal device according to claim 1 , wherein the display control unit changes the shooting range in accordance with the control signal received from the terminal device.

3. The terminal device according to claim 1 , further comprising an actuator that generates vibrations to provide feedback of an operation to a user.

4. Calculating a relative position between a display device that displays a virtual reality used by a user and a terminal device that is actually held by the user; displaying a virtual terminal device in the virtual reality space in accordance with a control signal from the terminal device, and changing a shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device; a step of acquiring image data relating to an image in the space of the changed shooting range in the virtual reality space; and a step of storing the shooting data of the shooting range in a storage unit of the terminal device based on a storage operation. A method comprising:

5. Calculating a relative position between a display device that displays a virtual reality used by a user and a terminal device that is actually held by the user; displaying a virtual terminal device in the virtual reality space in accordance with a control signal from the terminal device, and changing a shooting range displayed on the virtual terminal device based on the relative position in the virtual reality space displayed on the display device; acquiring image data relating to an image in the space of the changed shooting range in the virtual reality space; storing the photographed data of the photographing range in a storage unit of the terminal device based on a storage operation; A program that causes a computer to execute the following.

Images