Information processing system, wearing device, and program

Abstract

To provide an information processing system, a wearing device, and a program configured to present a feeling similar to a feeling of walking in a vast virtual space to a user from user's walking in a limited real space, while allowing the user to view an object in the virtual space.SOLUTION: A HMD 1 is mounted on the head of a user P who walks in a walking region 5 including a handrail 3 prepared in a real space and a pole 4 prepared in a predetermined position on a line of the handrail 3 extended from positions of both ends of the handrail 3. The HMD captures an image of the real space according to an orientation of the head of the user P, acquires a moving status of the user P including the orientation of the head and a walking position of the user P walking in the walking region 5 on the basis of the captured image, generates an image of a virtual space, which is different from the real space, according to the acquired moving status, using data related image generation of the virtual space in which objects and passage images including images corresponding to the handrail 3 and the pole 4 stored in a virtual space data storage unit 22 are arranged, and displays the generated image of the virtual space.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, a mounting device, and a program. [Background technology]

[0002] In recent years, in attractions and video games at amusement facilities, etc., images are provided to users using a head-mounted display (HMD) worn on the head (for example, Patent Document 1). When providing images using an HMD, it is possible to display various images of a space different from the real space, which is the real world, in front of the user's eyes.

[0003] In addition, a spatial perception manipulation method has been disclosed that uses a technology called Redirected Walking, which creates an unnoticeable discrepancy between the amount of movement in reality and the amount of movement in virtual reality (VR), thereby unconsciously influencing walking and making it possible to present the sensation of walking in a large virtual space even in a small real space (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-4263 A [Patent Document 2] Patent Publication No. 2023-13650 Summary of the Invention [Problem to be solved by the invention]

[0005] When an image corresponding to the user's movements is displayed on the HMD, the user who moves around wearing the HMD can get the feeling that he or she is walking in a virtual space. However, the user wearing the HMD cannot see the real space. Therefore, it is necessary to devise a way to prevent the user from bumping into other users or walls in the real space. It is also desirable to provide a user with an image of a vast virtual space through the user's compact walking. Furthermore, some ingenuity is required to ensure that the images you want to show to the user are shown to the user.

[0006] Therefore, an object of the present invention is to provide an information processing system, a wearable device, and a program that give a user the sensation of walking in a vast virtual space instead of walking in a limited real space, and further enable the user to view objects in the virtual space. [Means for solving the problem]

[0007] The present invention solves the above problems by the following solving means. A first invention is an information processing system including a walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail, a wearing device to be worn on the head of a user walking in the walking area, and an information processing device that outputs an image of a virtual space different from the real space to the wearing device, wherein the handrail is provided at a position where the user's hands can follow when the user walks in the walking area, and the pole is for changing the user's direction of travel in the opposite direction by walking while being held by the user's hands in the walking area, and the wearing device includes an imaging unit that captures an image of the real space according to the orientation of the user's head, and an information processing device that outputs an image of the virtual space different from the real space to the wearing device. and a display unit that displays an image of the virtual space output from a processing device, wherein the information processing device comprises a memory unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and poles and objects are arranged, and a control unit that performs processing based on the captured images taken by the photographing unit, wherein the control unit acquires the user's movement status including the user's walking position and head direction in the walking area based on the captured images, and generates an image of the virtual space corresponding to the acquired movement status using the data in the memory unit, and the display unit outputs the image of the virtual space generated by the information processing device. A second invention is an information processing system according to the first invention, wherein the control unit determines the position of the user in the corridor image according to the movement status, and generates an image of the virtual space including the object observable from the determined position and the direction of the user's head using the data in the memory unit. A third invention is an information processing system in which, in the information processing system of the first or second invention, the control unit determines the user's rotational movement around the pole and movement along the handrail based on the captured image. A fourth invention is an information processing system according to the third invention, wherein, when the control unit determines that the user is rotating around the pole, it generates an image of the virtual space in which the passage image has been moved by rotating it in a rotational direction around the pole relative to the direction of travel by a predetermined angle that is different from a rotational movement angle based on the user's walking position. A fifth invention is an information processing system according to the fourth invention, wherein the control unit generates an image of the virtual space in a state in which the passage image has been further moved upward or downward. A sixth invention is an information processing system in which, in any of the third to fifth inventions, the control unit, when it determines that the user is moving along the handrail, generates an image of the virtual space in which the user has moved a distance in the passage image that is different from a distance moved based on the user's walking position. A seventh invention is an information processing system, in which, in any of the first to sixth inventions, the object is a 3DCG of an object in the real space, and the memory unit stores the data relating to the virtual space in which the passage image is placed in the vicinity of the object. An eighth invention is the information processing system according to the seventh invention, wherein the storage unit stores the data relating to the virtual space in which the passage image is arranged so as to surround the object. A ninth invention is an information processing system according to the seventh or eighth invention, wherein the object is a 3DCG image obtained by enlarging or reducing the object. A tenth invention is an information processing system in which, in any of the first to sixth inventions, the memory unit stores the data relating to the virtual space in which the passage image is placed in the vicinity of each of a plurality of the objects, and the plurality of objects are images relating to the same object in the real space, each of which is an image of a different size. An eleventh invention is an information processing system, in any of the first to sixth inventions, wherein the object is a 3DCG of an object, and the memory unit stores the data relating to the virtual space in which the passage image is arranged so as to pass through the inside of the object. A twelfth aspect of the present invention is an information processing system according to any one of the first to eleventh aspects of the present invention, wherein the information processing device is provided in the mounting device. A thirteenth invention is an information processing system, in which, in any of the first to eleventh inventions, the information processing device is provided separately from the wearing device, and the wearing device and the information processing device are connected so as to be able to communicate with each other. A fourteenth invention is a wearing device that is worn on the head of a user walking in a walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail, the handrail being provided at a position where the user's hands can follow when the user walks in the walking area, and the pole being held by the user in the handrail as the user walks in the walking area, thereby changing the user's direction of travel in the opposite direction, the wearing device comprising: a photographing unit that photographs the real space according to the orientation of the user's head; a display unit that displays an image of a virtual space different from the real space; a memory unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrail and the pole and objects are arranged; and a control unit that acquires the user's movement status including the walking position and head orientation of the user in the walking area based on the photographed image captured by the photographing unit, generates an image of the virtual space according to the acquired movement status using the data in the memory unit, and outputs the generated image of the virtual space to the display unit. A fifteenth invention is an information processing system including a walking area including a handrail provided in a real space and poles provided at predetermined positions on an extension line of the handrail from both ends of the handrail, and a wearing device to be worn on the head of a user walking in the walking area, wherein the handrail is provided at a position where the user's hands can follow when the user walks in the walking area, and the pole is a pole that is held by the user's hands in the walking area and is used to change the user's direction of travel in the opposite direction when walking, and a computer that is the wearing device of the information processing system includes an imaging unit that images the real space according to the direction of the user's head; The program is provided with a display unit that displays an image of a virtual space that is different from the real space, and a memory unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and poles and objects are arranged, and causes the computer to function as follows: based on the captured images taken by the photographing unit, the user's movement status including the user's walking position and head direction in the walking area is acquired based on the captured images; an image of the virtual space corresponding to the acquired movement status is generated using the data in the memory unit; and the generated image of the virtual space is output to the display unit. A sixteenth invention is an information processing system comprising: a walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail; and a wearing device to be worn on the head of a user walking in the walking area, wherein the handrail is provided at a position where the user's hands can follow when the user walks in the walking area, and the pole is held by the user's hands while walking in the walking area, thereby changing the user's moving direction in the opposite direction, and the wearing device comprises: an imaging unit that images the real space according to the orientation of the user's head; and a display unit that displays an image of a virtual space different from the real space. In addition, the computer connected to the mounting device of the information processing system in a communicable manner includes a memory unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and the poles and objects are arranged, and the program causes the computer to function as follows: based on the captured image taken by the photographing unit, the computer acquires the user's movement status including the user's walking position and head direction in the walking area based on the captured image; generates an image of the virtual space according to the acquired movement status using the data in the memory unit; and outputs the generated image of the virtual space to the display unit. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing system, a wearable device, and a program that give a user the sensation of walking in a vast virtual space instead of walking in a limited real space, and further enable the user to view objects in the virtual space. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall schematic diagram of an information processing system according to an embodiment of the present invention; [Diagram 2] 2A and 2B are a functional block diagram and a diagram showing an example of a storage unit of the HMD according to the present embodiment. [Diagram 3] 4A to 4C are diagrams illustrating examples of images output by the HMD according to the present embodiment. [Figure 4] 11 is a diagram showing an example of setting an aisle image in a virtual space in the information processing system according to the embodiment. FIG. [Diagram 5] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 6] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 7] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 9] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 10] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 11] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 12] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 13] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 14] FIG. 2 is a diagram showing an example of a virtual space in the information processing system according to the embodiment. [Figure 15] 10 is a flowchart showing a virtual space output process of the HMD according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited to this example. (Embodiment) FIG. 1 is an overall schematic diagram of an information processing system 100 according to this embodiment. FIG. 2 is a functional block diagram of the HMD 1 according to this embodiment and a diagram showing an example of the storage unit 20 of the HMD 1.

[0011] The information processing system 100 shown in FIG. 1 is a system including an HMD (head mounted display) 1 (wearable device, information processing device) and a walking area 5 including handrails 3 and poles 4. The HMD 1 is a device that, when worn by a user P on the head, covers the user P's eyes with the HMD 1, blocking the user P's field of vision from real space, and outputs an image of a virtual space to the user P's field of vision. The walking area 5 is an area in which the user P wearing the HMD 1 walks. The walking area 5 is an area around the handrail 3, and includes an area in which the user P can walk by holding the handrail 3. The walking area 5 is also an area around the pole 4, and includes an area in which the user P can walk with the pole 4 held by the hand of the user P.

[0012] The handrail 3 is provided at a height (e.g., about 0.8 m above the floor) that allows the user P to hold on to it when walking in a straight line. The handrail 3 has a length of, for example, about 4 m. The length of the handrail 3 may be a length that depends on the installation location of the walking area 5, but is about 3 m to 5 m. The pole 4 is provided at a predetermined position on the extension line of the handrail 3 from each end of the handrail 3. The predetermined position on the extension line of the handrail 3 is a distance that the user P can reach when he / she extends his / her hand while grasping the end of the handrail 3, and is desirably provided at a position at a distance that the user P will not hit the handrail 3 when walking one turn while holding the pole 4 in his / her hand. The distance is, for example, about 1 m from the end of the handrail 3. The pole 4 has a height of, for example, about 2 m to 3 m.

[0013] With the HMD1 attached to the head, the user P walks along the handrail 3, holding the handrail 3 with the right hand of the user P, and at the end of the handrail 3, switches from the handrail 3 to the pole 4 and rotates. By doing so, the user P can change the direction of travel from the previous direction to the opposite direction. After that, the user P switches from the pole 4 to the handrail 3, and walks holding the handrail 3 with the right hand of the user P. The user P repeats such an action of switching between the handrail 3 and the pole 4 and walking. By the user P moving back and forth in the walking area 5 using the handrail 3 and the pole 4, an image is output to the HMD1 as if the user P is moving on a passage image in a virtual space that is not a back and forth movement. Therefore, the user P can be given the illusion of moving along the passage image in a virtual space.

[0014] Next, the HMD 1 will be described. As shown in FIG. 2A, the HMD 1 includes a control unit 10, a storage unit 20, an imaging unit 26, and a display unit 27. The control unit 10 is a central processing unit (CPU) that controls the entire HMD 1. The control unit 10 appropriately reads and executes an operating system (OS) and application programs stored in the storage unit 20, thereby cooperating with the above-mentioned hardware and executing various functions.

[0015] The control unit 10 includes a captured image acquisition unit 11, a movement status acquisition unit 12, a virtual space image generation unit 13, and an image output unit . The photographed image acquiring section 11 acquires, from the photographing section 26, a photographed image that is an image photographed by the photographing section 26. The movement status acquisition unit 12 acquires the movement status of the user P based on the captured images acquired by the captured image acquisition unit 11. The movement status includes the walking position in the walking area 5 and the direction of the head of the user P. The movement status also includes a movement history from when the user P started walking in the walking area 5. Then, the movement status acquisition unit 12 determines and acquires the rotational movement around the pole 4 and the movement along the handrail 3 by the user P based on the captured images. At this time, the movement status acquisition unit 12 may further determine the movement based on the walking position.

[0016] The virtual space image generating unit 13 determines the position of the user P in the passage image according to the movement status acquired by the movement status acquiring unit 12, and generates an image of the virtual space including objects observable from the determined position and head direction of the user P. The virtual space image generating unit 13 uses the virtual space data in the virtual space data storage unit 22 in generating the image of the virtual space. The image output unit 14 outputs the image of the virtual space generated by the virtual space image generation unit 13 to the display unit 27.

[0017] The storage unit 20 is a storage area such as a hard disk, a semiconductor memory device, etc. for storing programs, data, etc. required for the control unit 10 to execute various processes. The storage unit 20 includes a program storage unit 21 and a virtual space data storage unit 22 . The program storage unit 21 is a storage area for storing various programs. The program storage unit 21 stores a processing program 21a for carrying out various functions executed by the control unit 10 of the HMD 1.

[0018] The virtual space data storage unit 22 is a storage area for storing various types of virtual space data. 2B shows an example of the virtual space data storage unit 22. The virtual space data storage unit 22 stores virtual space data in association with a data ID (IDentification). The data ID is identification information for identifying the virtual space data, and is, for example, arbitrarily determined by the information processing system 100. The contents of the virtual space data are descriptions of the contents of the virtual space. The data is the virtual space data itself. The data can be registered in the virtual space data storage unit 22 via an input / output device (not shown) that stores data and is connected to an input / output terminal or the like (not shown) provided in the HMD 1. Alternatively, the data can be registered in the virtual space data storage unit 22 via a communication interface (not shown) provided in the HMD 1 and received from the input / output device via network communication.

[0019] 2A is a camera that captures real space. The capture units 26 are provided on the outer surface opposite to the surface facing the head of the user P wearing the HMD 1, for example, near the positions corresponding to the left and right eyes of the user P wearing the HMD 1. The display unit 27 is a display device such as an LCD (Liquid Crystal Display), an organic EL display, etc. The display unit 27 is provided on the inner surface that faces the head of a user P wearing the HMD 1. Here, a computer refers to an information processing device equipped with a control unit, a storage device, etc., and the HMD 1 is an information processing device equipped with a control unit 10, a storage unit 20, etc., and is included in the concept of a computer.

[0020] Next, the processing in the information processing system 100 will be described with examples. FIG. 3 is a diagram showing an example of an image 30 output by the HMD 1 according to this embodiment. The image 30 shown in FIG. 3 is an image of the virtual space seen by the user P shown in FIG. 1. The image 30 includes a passage image 31. The passage image 31 includes a handrail image 33 and a pole image 34. The handrail image 33 and the pole image 34 are located at the same positions as the handrail 3 and the pole 4 in the real space. In other words, the handrail image 33 is adjusted so that it is located at the same position as the handrail 3 of the user P in the real space, so that the user P feels as if his / her hand is running along the handrail image 33. Then, when the user P walks along the handrail 3 with his / her right hand in the walking area 5 toward the pole 4, the image 30 is updated as if the user P is walking along the passage image 31 in the traveling direction. Here, the distance that the user P actually walks in the real space may be different from the distance that the user P walks in the virtual space shown to the user P.

[0021] Technology for making the distance traveled by user P in real space different from the distance traveled by user P in virtual space as seen by user P without creating a sense of incongruity is well known, and for example, the technology disclosed in the following document can be used. Reference 1: K. Matsumoto, Y. Ban, T. Narumi, Y. Yanase, T. Tanikawa and M. Hirose, Unlimited Corridor: Redirected Walking Techniques using Visuo Haptic Interaction, ACM SIGGRAPH Emerging Technologies, pp.1-2, 2016. Reference 2: R. Nagao, K. Matsumoto, T. Narumi, T. Tanikawa and M. Hirose, Ascending and Descending in Virtual Reality: Simple and Safe System Using Passive Haptics, IEEE Transactions on Visualization and Computer Graphics, vol.24, no.4, pp.1584-1593, 2018.

[0022] 3, the image 30 may include a progress mark 37 indicating a progress direction. When the position of the user P in the virtual space reaches a predetermined position, the control unit 10 may output a sound related to the virtual space corresponding to the position from a sound output unit (not shown) of the HMD 1, and the image 30 may include a sound mark 38 indicating the predetermined position.

[0023] Next, the setting of the passage image related to the creation of the virtual space data will be described with an example. FIG. 4 is a diagram showing an example of how a passage image is set in a virtual space in the information processing system 100 according to this embodiment. 4(A) shows an example of a passage image set in a vertically long virtual space 41. The virtual space 41 is a space with a width of 5 m in the X direction, a depth of 14 m in the Y direction, and a height of 3 m in the Z direction. The virtual space 41 includes straight passages 42a to 42d and rotation passages 43a to 43c. The rotation passages 43a to 43c can rotate at any angle, and this can be controlled by the number of revolutions in the real space. The straight passages 42a to 42d correspond to the walking area 5 in the real space where people walk along the handrails 3. The turning passages 43a to 43c correspond to the walking area 5 in the real space where people walk while holding onto the poles 4.

[0024] 4(B) shows an example of setting a passage image in a square virtual space 45. The virtual space 45 is a space that is about 7 m in the horizontal direction X and the depth direction Y, and about 3 m in the height direction Z. The virtual space 45 includes straight passages 46a to 46d and turning passages 47a to 47c. The virtual space 45 gradually rises while going around through the straight passages 46a to 46d and the turning passages 47a to 47c.

[0025] It should be noted that, not limited to the example shown in Fig. 4, in any virtual space in which a passage image is set, the straight passage can be made longer or shorter than the length of the passage in the real space. Furthermore, by combining the direct passage and the rotating passage, in any virtual space, a passage image that allows the user P to move within the virtual space can be provided.

[0026] Next, as an example in which the user P can view objects in the virtual space, an example of the arrangement of objects and passage images in the virtual space will be described. 5 to 14 are diagrams showing examples of a virtual space in the information processing system 100 according to this embodiment.

[0027] (Example 1. Experiencing buildings) 5 to 7 are examples of virtual space 50 in which the object is a 3DCG building. Virtual space 50 is an example in which the data ID of virtual space data storage unit 22 is "d01" (see FIG. 2(B)). In virtual space 50, a passage image 51 corresponding to walking area 5 (see FIG. 1) is placed in a position where the interior of the building, etc. can be viewed. FIG. 5A shows the configuration of a passage image 51 in a virtual space 50. FIG. 5B shows an object 53 in the virtual space 50 . 6 and 7 show an example of a virtual space 50 in which a passage image 51 and an object 53 are placed. 6 and 7, straight passage 52a in passage image 51 in Fig. 5(A) is a passage for entering the inside of the building from the corridor of object 53. Straight passages 52b to 52d in passage image 51 in Fig. 5(A) are passages arranged to enable viewing of the inside of object 53, making it possible to view the inside of object 53 from various viewpoints, as shown in Fig. 7. Straight passage 52e in passage image 51 in Fig. 5(A) is a passage that enables viewing of the exterior of object 53 from the outside of the building, which is object 53.

[0028] When user P puts on the HMD 1 and begins to walk along handrails 3 in walking area 5 in real space, the display unit 27 of HMD 1 can display a virtual space 50 as seen when walking along straight passage 52a, and user P can walk along the corridor of object 53 in virtual space 50 and approach a door at the boundary to the interior of the building. After that, the user P grips the pole 4 in the real space and rotates to change the moving direction in the virtual space 50 so that the user P can proceed along the straight passage 52b. Then, the user P walks along the handrail 3 in the real space, and the HMD 1 displays the user P walking along the straight passage 52b. Therefore, the user P can appreciate the inside of the object 53. The user P repeats a similar walk in the real space, and the user P grips the pole 4 and rotates to change the moving direction from the straight passage 52d to the straight passage 52e so that the user P can proceed. After changing the moving direction, the user P walks along the handrail 3, and the display displays the user P walking along the straight passage 52e, which is in a different position from the straight passage 52d. Then, the user P can appreciate the appearance of the object 53 on the left side of the moving direction.

[0029] In this way, by arranging object 53 of a building in the real space in virtual space 50, it is possible to provide user P with a feeling similar to appreciating an actual building. Therefore, there is no need to go to the location of the building. Furthermore, since the passage image 51 is arranged so as to surround three sides of the interior of the object 53, the user P can be provided with the opportunity to view the interior of the building from various directions while moving around. Furthermore, not only can the interior of object 53 actually be viewed, but also a viewing method that is not possible in reality can be provided, such as moving to a remote location to view the interior of object 53 and then immediately viewing the exterior of object 53.

[0030] (Example 2. Experiencing a giant painting) 8 and 9 are examples of a virtual space 60 in which the object is a 2DCG (image) of a painting. The virtual space 60 is an example in which the data ID of the virtual space data storage unit 22 is "d02" (see FIG. 2(B)). In the virtual space 60, a passage image 61 corresponding to the walking area 5 is configured in a substantially square shape (see FIG. 9), and the turning passage gradually increases in height as it turns and moves. In the virtual space 60, three objects 63a to 63c, which are the 2DCG of the painting as the object, a full-size object, a medium-size object, and a large-size object, are arranged in positions visible from the passage.

[0031] When user P wears the HMD 1 and starts walking along the handrail 3 in the walking area 5 in the real space, the display unit 27 of the HMD 1 can display a virtual space 60 that can be seen while walking along the passage image 61 from position 66. Then, the user P can closely view a full-size object 63a placed near the passage image 61. In addition, a medium-size object 63b is placed behind the full-size object 63a, and the user P can view the full-size object 63a together with the medium-size object 63b while comparing the two.

[0032] After that, the user P grips the pole 4 in the real space and rotates to change the direction of movement in the virtual space 60. At that time, it is possible to display the user P walking up the turning passage of the passage image 61. In this virtual space 60, when the user P grips the pole 4 and rotates one and a half times, the next straight passage that can be walked is displayed, and the direction of movement in the virtual space 60 is changed. Then, the user P walks along the handrail 3 in the real space, and it is possible to display on the HMD 1 that the user P is walking along the straight passage of the passage image 61. Then, the user P can view the medium-sized object 63b arranged nearby, and can also view the large-sized object 63c arranged far away.

[0033] In this way, by arranging objects 63a at the original size of a painting in the real space to enlarged object 63c in virtual space 60 and arranging corridor image 61 in the vicinity of each of objects 63a to 63c, user P can view each of objects 63a to 63c from multiple viewpoints, thereby enabling viewing of the painting from a variety of viewpoints.

[0034] (Example 3. Experiencing a giant 3D image) FIG. 10(A) is an example of a virtual space 70 in which an object is a 3D image of a three-dimensional image. The virtual space 70 is an example in which the data ID of the virtual space data storage unit 22 is "d03" (see FIG. 2(B)). In the virtual space 70, a passage image 71 corresponding to the walking area 5 is arranged so as to surround the periphery of the three-dimensional image of the object 73 in a substantially square shape. The turning passage of the passage image 71 gradually increases in height while rotating. The passage image 71 is also arranged so as to go around the object 73 twice. The object 73 is an enlarged version of an actual three-dimensional object. In this example, the object 73 is an enlarged version of an actual three-dimensional object by about 10 times.

[0035] When the user P wears the HMD 1 and starts walking along the handrail 3 in the walking area 5 in the real space, the display unit 27 of the HMD 1 can display the virtual space 70 seen when walking along the passage image 71 from position 76. The user P can then view, for example, the foot part of the object 73 in more detail, and view it from an angle looking up from below the object 73, as shown in FIG. 10(B). After that, when the user P moves back and forth in the walking area 5 in the real space, the display unit 27 of the HMD 1 can display the passage image 71 as if the user P is gradually ascending and walking around the object 73. The user P can then view, for example, the upper part of the object 73 including its face together with the passage image 71 of the virtual space 70 that the user P has passed through, as shown in FIG. 10(C), by looking backwards relative to the traveling direction.

[0036] In this way, by placing the object 73, which is an enlarged version of an object in the real space, in the virtual space 70, the user P can observe the object 73 and appreciate it in more detail than the actual object. In addition, since the passage image 71 is placed so as to surround the periphery of the object 73, an appreciation route that allows the user P to appreciate the object 73 while rotating around it can be realized. As a result, the object 73 can be displayed on the display unit 27 of the HMD 1 so as to be observable from the front, left and right directions, and from the rear direction, and further, the object 73 can be displayed so as to gradually progress from the bottom to the top. The user P can view the object 73 from the feet of the object 73 shown in Fig. 10(B) to finally the face of the object 73 shown in Fig. 10(C).

[0037] (Example 4. Experiencing a large 3D work -1) FIG. 11 is an example of a virtual space 80 in which the object is a 3DCG of a celestial globe, which is a three-dimensional work of art. The virtual space 80 is an example in which the data ID of the virtual space data storage unit 22 is "d04" (see FIG. 2(B)). In the virtual space 80, a passage image 81 corresponding to the walking area 5 is placed inside the celestial globe, which is an object 83. In addition, an object 84, which is a 3DCG of a globe, is placed near the passage image 81.

[0038] When user P wears HMD 1 and starts walking along handrail 3 in walking area 5 in real space, virtual space 80 that can be seen while walking along passage image 81 from position 86 can be displayed on display unit 27 of HMD 1. User P can then appreciate celestial globe object 83 from inside. After that, when user P moves back and forth in walking area 5 in real space, globe object 84 can be displayed up close on display unit 27 of HMD 1. User P can observe objects 83 and 84 and can sense the difference in size between the terrestrial globe and the celestial globe.

[0039] (Example 5. Experiencing a large 3D work -2) 12 to 14 are examples of virtual space 90 in which the object is a 3DCG of a celestial globe, which is a three-dimensional work of art. The virtual space 90 is an example in which the data ID of the virtual space data storage unit 22 is "d05" (see FIG. 2(B)). As shown in FIG. 12(A), in the virtual space 90, a passage image 91 corresponding to the walking area 5 is arranged so as to enter the inside of the celestial globe, which is an object 93, from the outside. And, as shown in FIG. 12(B), in the virtual space 90, the passage image 91 is arranged so as to go out from the inside of the object 93. Also, near the passage image 91 and outside the celestial globe, an object 94, which is a 3DCG of a globe, is arranged.

[0040] When user P wears HMD 1 and starts walking along handrail 3 in walking area 5 in real space, virtual space 90 that can be seen while walking along passage image 91 from position 96 shown in Fig. 12(A) can be displayed on display unit 27 of HMD 1. User P can then appreciate the appearance of celestial globe object 93 and terrestrial globe object 94. By observing object 93 and object 94, user P can sense the difference in size between the terrestrial globe and celestial globe, etc.

[0041] Thereafter, as the user P moves back and forth in the walking area 5 in the real space, he or she moves from the outside to the inside of the celestial globe object 93, and the interior of the celestial globe object 93 can be displayed on the display unit 27 of the HMD 1. Fig. 13 shows the interior of the celestial globe object 93. Fig. 14 shows a passage image 91 placed inside the object 93 in the virtual space 90. Note that, as shown in Figs. 13 and 14, the virtual space 90 may have an object 95, which is a 3DCG globe, placed inside the object 93. In this way, by using the HMD 1, the user P can view the object 93 from both the outside and the inside, and can get a sense of the scale of the object 93, etc.

[0042] Next, the process performed by the HMD 1 for outputting the above-described virtual space will be described. 15 is a flowchart showing the virtual space output process of the HMD 1 according to this embodiment, assuming that one piece of virtual space data stored in the virtual space data storage unit 22 of the HMD 1 has been selected. In step S (hereinafter, "step S" will be simply referred to as "S") 11 of FIG. 15, the captured image acquisition unit 11 of the HMD 1 acquires a captured image via the imaging unit .

[0043] In S12, the movement status acquisition unit 12 acquires the movement status of the user P in the real space based on the captured image. In S13, the virtual space image generating unit 13 generates an image of the virtual space according to the movement situation, using the virtual space data.

[0044] In S14, the image output unit 14 outputs the generated image of the virtual space to the display unit 27. In S15, the control unit 10 judges whether or not to end the image output of the virtual space. For example, when the image of the virtual space output by the HMD1 is the end of the corridor image, or when an end operation (such as turning off the power) is performed by a button or the like (not shown) of the HMD1, the control unit 10 judges to end the image output of the virtual space. If the output is to be ended (S15: YES), the control unit 10 ends this process. On the other hand, if the output is not to be ended (S15: NO), the control unit 10 shifts the process to S11 and repeats the process.

[0045] As described above, the HMD 1 of this embodiment has the following advantages. (1) The device comprises a photographing unit 26 that photographs the real space according to the direction of the head of a user P walking in a walking area 5 in the real space, a display unit 27 that displays an image of a virtual space different from the real space, a virtual space data storage unit 22 that stores data related to the generation of an image of a virtual space in which a passage image including an image corresponding to handrails 3 and poles 4 in the walking area 5 and objects are arranged, and a control unit 10 that performs processing based on the photographed image photographed by the photographing unit 26, and the control unit 10 acquires the movement status of the user P including the walking position and head direction of the user P in the walking area 5 based on the photographed image, generates an image of the virtual space according to the acquired movement status using data from the virtual space data storage unit 22, and the display unit 27 outputs the generated image of the virtual space. Therefore, when user P walks in the walking area 5, a virtual space is generated and displayed in which passage images, including images corresponding to the handrails 3 and poles 4 in the walking area 5, and objects are arranged according to the movement situation, so that user P can walk on the passage image in the virtual space and view the objects in the virtual space.

[0046] (2) The control unit 10 determines the position of user P in the corridor image based on the movement situation, and generates an image of a virtual space including objects observable from the determined position and the direction of user P's head using data from the virtual space data storage unit 22. Therefore, since an image of the virtual space at a position that reflects the movement status of the user P in the walking area 5 is generated, an image of the virtual space that matches the movement of the user P can be generated and displayed on the display unit 27.

[0047] (3) The control unit 10 determines the rotational movement around the pole 4 and the movement along the handrail 3 by the user P based on the captured image. Therefore, different processing can be performed depending on whether the user P is moving around the handrail 3 or the pole 4, and an image of the virtual space that the user P can intuitively feel can be presented.

[0048] (4) When the control unit 10 determines that the user P is moving in a rotational direction around the pole 4, it generates an image of the virtual space in which the corridor image has been rotated in a rotational direction around the pole 4 relative to the direction of travel by a predetermined angle that is different from the rotational movement angle based on the user P's walking position. Therefore, since the virtual space can be set to a state in which the direction of movement can be changed at various angles different from the rotational movement angle in the real space, it is possible to provide something that allows free movement in the virtual space.

[0049] (5) The control unit 10 generates an image of the virtual space in which the passage image has been further moved upward. Therefore, the virtual space can be made to move in a state that also includes changes in the upward direction, so that it is possible to provide something that allows free movement in the virtual space.

[0050] (6) When the control unit 10 determines that the user P is moving along the handrail 3, it generates an image of the virtual space in which the user P has moved a distance in the passage image that is different from the distance moved based on the user P's walking position. Therefore, the virtual space can be made to be a state in which movement is possible over distances different from those in the real space, providing something that allows free movement in the virtual space.

[0051] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. The above-mentioned embodiments and the modified forms described below can be used in appropriate combination, but detailed description will be omitted.

[0052] (Variations) (1) In the present embodiment, the processing is performed only by the HMD 1, but the present invention is not limited to this. For example, the HMD may be provided with an image capture unit and a display unit, and the control unit and storage unit may be performed by an information processing device such as a PC (personal computer) communicably connected to the HMD. In addition, the processing of the control unit and storage unit may be shared between the information processing device and the HMD. (2) In the present embodiment, the HMD 1 is used as an example of a wearable device, but the wearable device is not limited to this. For example, glasses or the like may be used.

[0053] (3) In the present embodiment, an example has been described in which an image of a virtual space in a state in which the passage image is moved upward is generated, but the present invention is not limited to this. An image of a virtual space in a state in which the passage image is moved downward may be generated. In this way, the virtual space can be moved in a state including a downward change, so that it is possible to provide a state in which movement in the virtual space is made even more flexible. (4) In the present embodiment, several examples of the virtual space of the information processing system 100 are illustrated, but the present invention is not limited to these examples. Any other examples may be used as long as the virtual space includes a passage image corresponding to the walking area 5 and an object that can be seen from the passage image. [Explanation of symbols]

[0054] 1 HMD 3. Handrails 4. Paul 5. Walking Area 10 Control section 20 Memory section 21a Processing Program 22 Virtual space data storage unit 26 Photography Department 27 Display section 41,45,50,60,70,80,90 Virtual Space 31,51,61,71,81,91 Passage images 53,63a,63b,63c,73,83,84,93,94,95 Objects 100 Information Processing Systems P User

Claims

1. A walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail; A wearing device that is worn on the head of a user walking in the walking area; an information processing device that outputs an image of a virtual space that is different from the real space to the wearing device; An information processing system comprising: The handrail is provided at a position where the user's hands can be supported when the user walks in the walking area, the pole is for changing the moving direction of the user in the opposite direction by being held by the user while walking in the walking area, The mounting device includes: An imaging unit that images the real space according to a direction of the user's head; a display unit that displays an image of the virtual space output from the information processing device; Equipped with The information processing device includes: a storage unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and the poles and objects are arranged; A control unit that performs processing based on the captured image captured by the imaging unit; Equipped with The control unit is acquiring a movement status of the user including a walking position and a head direction of the user in the walking area based on the captured image; generating an image of the virtual space according to the acquired movement status using the data in the storage unit; An information processing system, wherein the display unit outputs an image of the virtual space generated by the information processing device.

2. 2. The information processing system according to claim 1, The control unit determines the user's position in the corridor image according to the movement status, and generates an image of the virtual space including the objects observable from the determined position and the direction of the user's head using the data in the memory unit.

3. 2. The information processing system according to claim 1, The control unit determines the user's rotational movement around the pole and movement along the handrail based on the captured image.

4. 4. The information processing system according to claim 3, When the control unit determines that the user is rotating around the pole, it generates an image of the virtual space in which the passage image has been moved by rotating it in a rotational direction centered on the pole relative to the direction of travel by a predetermined angle that is different from a rotational movement angle based on the user's walking position.

5. 5. The information processing system according to claim 4, The control unit generates an image of the virtual space in a state in which the passage image has been further moved upward or downward.

6. 4. The information processing system according to claim 3, When the control unit determines that the user is moving along the handrail, it generates an image of the virtual space in which the user has moved a distance in the passage image that is different from a distance traveled based on the user's walking position.

7. In the information processing system according to any one of claims 1 to 6, the object is a 3DCG of an object in the real space, The storage unit stores the data relating to the virtual space in which the passage image is arranged near the object.

8. 8. The information processing system according to claim 7, The storage unit stores the data relating to the virtual space in which the passage image is arranged to surround the object.

9. 9. The information processing system according to claim 8, An information processing system, wherein the object is a 3DCG image obtained by enlarging or reducing the object.

10. In the information processing system according to any one of claims 1 to 6, the storage unit stores the data relating to the virtual space in which the passage image is arranged in the vicinity of each of the plurality of objects; An information processing system, wherein the multiple objects are images of a same object in the real space, each image having a different size.

11. In the information processing system according to any one of claims 1 to 6, The object is a 3DCG of an object, The storage unit stores the data relating to the virtual space in which the passage image is arranged so as to pass through the inside of the object.

12. 2. The information processing system according to claim 1, The information processing system includes an information processing device provided in the mounting device.

13. 2. The information processing system according to claim 1, the information processing device is provided separately from the mounting device, The mounting device and the information processing device are communicatively connected to each other.

14. A wearing device to be worn on the head of a user walking in a walking area including a handrail provided in a real space and poles provided at predetermined positions on an extension line of the handrail from both ends of the handrail, The handrail is provided at a position where the user's hands can be supported when the user walks in the walking area, the pole is for changing the moving direction of the user in the opposite direction by being held by the user while walking in the walking area, The mounting device includes: An imaging unit that images the real space according to a direction of the user's head; a display unit that displays an image of a virtual space different from the real space; a storage unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and the poles and objects are arranged; Acquire a movement status of the user, including a walking position and a head direction of the user in the walking area, based on a captured image captured by the imaging unit; generating an image of the virtual space according to the acquired movement status using the data in the storage unit; A control unit that outputs the generated image of the virtual space to the display unit; A mounting device comprising:

15. A walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail; A wearing device that is worn on the head of a user walking in the walking area; An information processing system comprising: The handrail is provided at a position where the user's hands can be supported when the user walks in the walking area, The pole is for changing the moving direction of the user in the opposite direction by being held by the user while walking in the walking area. A computer that is the mounting device of the information processing system, An imaging unit that images the real space according to a direction of the user's head; a display unit that displays an image of a virtual space different from the real space; a storage unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and the poles and objects are arranged; Equipped with The computer, As a process based on the captured image captured by the imaging unit, acquiring a movement status of the user including a walking position and a head direction of the user in the walking area based on the captured image; generating an image of the virtual space according to the acquired movement status using the data in the storage unit; A program for causing the device to function in such a way that the image of the generated virtual space is output to the display unit.

16. A walking area including a handrail provided in a real space and a pole provided at a predetermined position on an extension line of the handrail from both ends of the handrail; A wearing device that is worn on the head of a user walking in the walking area; An information processing system comprising: The handrail is provided at a position where the user's hands can be supported when the user walks in the walking area, the pole is for changing the moving direction of the user in the opposite direction by being held by the user while walking in the walking area, The mounting device includes: An imaging unit that images the real space according to a direction of the user's head; a display unit that displays an image of a virtual space different from the real space; Equipped with a computer connected to the mounting device of the information processing system in a communicable manner includes a storage unit that stores data related to image generation of the virtual space in which a passage image including images corresponding to the handrails and the poles and objects are arranged; The computer, As a process based on the captured image captured by the imaging unit, acquiring a movement status of the user including a walking position and a head direction of the user in the walking area based on the captured image; generating an image of the virtual space according to the acquired movement status using the data in the storage unit; A program for causing the device to function in such a way that the image of the generated virtual space is output to the display unit.

Images