Information processing device and floor height adjustment method

Abstract

To provide a technique for supporting setting a height of a floor surface in a virtual space.SOLUTION: An image generation device generates an adjustment screen image 300 for enabling a user wearing a head-mounted display to adjust a floor height of a virtual space that is displayed on the head-mounted display. The adjustment screen image 300 includes a body height image 302. The image generation device causes the adjustment screen image 300 to be displayed on the head-mounted display. When an operation to set the body height of a user for the body height image 302 in the adjustment screen image 300 is input, the image generation device sets the floor height of the virtual space in accordance with the body height of the user set for the body height image 302.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to data processing technology, and more particularly to an information processing device and a floor surface height adjustment method. [Background technology]

[0002] Image display systems that allow users wearing head-mounted displays to view a target space from any viewpoint are becoming widespread. For example, there is electronic content that realizes virtual reality (VR) by displaying a virtual three-dimensional space on the head-mounted display according to the user's line of sight. The use of head-mounted displays can also enhance the sense of immersion in the video and improve the operability of applications such as games. In addition, walk-through systems have been developed that allow users wearing head-mounted displays to virtually walk around a space displayed as a video by physically moving around. Summary of the Invention [Problem to be solved by the invention]

[0003] An image display system that presents VR images to a user wearing a head-mounted display may place various objects, such as game items, on a preset floor of a virtual space. If the height of the floor of the virtual space set in the image display system differs significantly from the height of the floor of the real space, the user may feel uncomfortable with the VR image, which may impair the sense of immersion in the VR experience.

[0004] The present invention has been made in view of these problems, and one object of the present invention is to provide a technique for assisting in setting the floor height of a virtual space. [Means for solving the problem]

[0005] In order to solve the above problem, an information processing device according to one aspect of the present invention is an adjustment screen for allowing a user wearing a head-mounted display to adjust the height of the floor of a virtual space displayed on the head-mounted display, and includes: an adjustment screen generation unit that generates an adjustment screen including an image of a height meter; a display control unit that displays the adjustment screen on the head-mounted display; and a floor setting unit that, when an operation to set the user's height is input for the image of the height meter on the adjustment screen, sets the height of the floor of the virtual space in accordance with the user's height set in the image of the height meter.

[0006] Another aspect of the present invention is a floor height adjustment method, which includes the steps of: generating an adjustment screen that is displayed on a head-mounted display to allow a user wearing a head-mounted display to adjust the height of a floor of a virtual space, the adjustment screen including an image of a height meter; displaying the adjustment screen on the head-mounted display; and, when an operation to set the user's height is input for the image of the height meter on the adjustment screen, setting the height of the floor of the virtual space in accordance with the user's height set in the image of the height meter.

[0007] Any combination of the above components, or conversion of the present invention between a system, a computer program, a recording medium on which a computer program is readably recorded, a data structure, etc., are also valid aspects of the present invention. [Effects of the Invention]

[0008] According to the present invention, it is possible to assist in setting the floor height of a virtual space. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the appearance of a head-mounted display according to an embodiment; [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of an image display system according to an embodiment. [Figure 3]FIG. 1 is a diagram for explaining an example of an image world that an image generating device displays on a head-mounted display. [Figure 4] FIG. 2 is a diagram illustrating an internal circuit configuration of the image generating device. [Figure 5] FIG. 2 is a diagram illustrating an internal circuit configuration of a head-mounted display. [Figure 6] FIG. 2 is a block diagram showing functional blocks of the image generating apparatus. [Figure 7] 10 is a flowchart showing the operation of the image generating device. [Figure 8] FIG. 10 is a diagram illustrating an example of an adjustment screen. [Figure 9] FIG. 10 is a diagram illustrating an example of an adjustment screen. DETAILED DESCRIPTION OF THE INVENTION

[0010] This embodiment relates to an image display system that displays application images on a head-mounted display worn on a user's head. The head-mounted display is also called a VR headset. FIG. 1 shows an example of the appearance of a head-mounted display 100 according to this embodiment. The head-mounted display 100 includes an output mechanism unit 102 and a wearing mechanism unit 104. The wearing mechanism unit 104 includes a wearing band 106 that, when worn by the user, wraps around the head and secures the device in place.

[0011] The output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the user wears the head mounted display 100, and is provided with a display panel inside so that it faces the eyes when worn. The display panel of the head mounted display 100 of the embodiment is not transparent. In other words, the head mounted display 100 of the embodiment is a light-opaque head mounted display.

[0012] The housing 108 may further include an eyepiece located between the eyes of the user wearing the head mounted display 100 and the display panel of the head mounted display 100, to expand the user's field of view. The head mounted display 100 may further include speakers or earphones at positions corresponding to the user's ears when worn. The head mounted display 100 also includes a built-in motion sensor that detects the translational and rotational movements of the head of the user wearing the head mounted display 100, as well as the position and posture at each time.

[0013] The head mounted display 100 also includes a stereo camera 110 on the front surface of the housing 108. The stereo camera 110 captures video of the surrounding real space in a field of view that corresponds to the user's line of sight. If the captured image is displayed immediately, the real space in the direction the user is facing can be seen as it is, which is known as video see-through. Furthermore, if a virtual object is drawn on the image of a real object captured in the captured image, augmented reality (AR) can be realized. There is no limit to the number of cameras that the image display system 10 can include, and the head mounted display 100 may include one camera, or three or more cameras.

[0014] 2 shows an example of the configuration of an image display system 10 according to an embodiment. The image display system 10 includes a head-mounted display 100, an image generation device 200, and a controller 140. The head-mounted display 100 is connected to the image generation device 200 via wireless communication. The image generation device 200 may further be connected to a server (not shown) via a network. In this case, the server may provide the image generation device 200 with data for an online application, such as a game in which multiple users can participate via the network.

[0015] The image generation device 200 is an information processing device that identifies the position of the viewpoint and the direction of the line of sight based on the position and posture of the head of a user wearing the head mounted display 100, generates a display image so as to provide a corresponding field of view, and outputs the display image to the head mounted display 100. The image generation device 200 may be a stationary game console, a PC, or a tablet terminal. The image generation device 200 is capable of executing various applications related to VR and AR, but the image generation device 200 of the embodiment generates a display image of a virtual world in which the game is set while progressing through an electronic game depicting a virtual world (hereinafter also referred to as a "VR game"), and displays the display image on the head mounted display 100.

[0016] The image generating device 200 may generate moving images for viewing or providing information, regardless of whether the moving images are from a virtual world or a real world, and display the moving images on the head-mounted display 100. The image generating device 200 may also display a panoramic image with a wide angle of view centered on the user's viewpoint on the head-mounted display 100, thereby providing the user with a deep sense of immersion in the displayed world.

[0017] The controller 140 is an input device (e.g., a game controller) that is held by the user and into which the user's operations are input. The user's operations include operations to control image generation in the image generation device 200 and operations to control image display in the head-mounted display 100. The controller 140 is connected to the image generation device 200 via wireless communication and transmits data indicating the user's operations to the image generation device 200. As a variation, one or both of the head-mounted display 100 and the controller 140 may be connected to the image generation device 200 via wired communication via a signal cable or the like.

[0018] The controller 140 includes buttons 142 and an analog stick 144 as components through which user operations are input. The buttons 142 include a directional button and a cross key. The analog stick 144 is also called a control stick, and is used to input a direction and a tilt amount by tilting. The tilt amount can also be considered as the angle at which the analog stick 144 is tilted.

[0019] 3 is a diagram illustrating an example of an image world that the image generation device 200 displays on the head-mounted display 100. In this example, a state is created in which the user 12 is in a room, which is a virtual space. As shown in the figure, objects such as walls, floors, windows, a table, and objects on the table are arranged in a world coordinate system that defines the virtual space. The image generation device 200 defines a view screen 14 in the world coordinate system according to the position of the viewpoint and the direction of the line of sight of the user 12, and draws a display image by representing the images of objects on the view screen.

[0020] The image generation device 200 acquires the position of the viewpoint and the direction of the line of sight of the user 12 (hereinafter, these may be collectively referred to as the "viewpoint") from the head-mounted display 100 at a predetermined rate, and changes the position and direction of the view screen 14 accordingly. This allows an image to be displayed on the head-mounted display 100 in a field of view corresponding to the user's viewpoint. The image generation device 200 can also generate stereo images with parallax and display the stereo images in the left and right regions of the display panel of the head-mounted display 100, allowing the user 12 to view a virtual space in three dimensions. This allows the user 12 to experience virtual reality as if they were in a room in the displayed world.

[0021] 4 shows the internal circuit configuration of the image generation device 200. The image generation device 200 includes a CPU (Central Processing Unit) 222, a GPU (Graphics Processing Unit) 224, and a main memory 226. These components are connected to one another via a bus 230. An input / output interface 228 is further connected to the bus 230. A communication unit 232, a storage unit 234, an output unit 236, an input unit 238, and a recording medium drive unit 240 are connected to the input / output interface 228.

[0022] The communication unit 232 includes a peripheral device interface such as USB or IEEE1394, and a network interface such as a wired LAN or wireless LAN. The storage unit 234 includes a hard disk drive, a nonvolatile memory, etc. The output unit 236 outputs data to the head mounted display 100. The input unit 238 accepts data input from the head mounted display 100 and also accepts data input from the controller 140. The recording medium drive unit 240 drives a removable recording medium such as a magnetic disk, an optical disk, or a semiconductor memory.

[0023] The CPU 222 controls the entire image generating device 200 by executing an operating system stored in the storage unit 234. The CPU 222 also executes various programs (e.g., VR game applications) that are read from the storage unit 234 or a removable recording medium and loaded into the main memory 226, or that are downloaded via the communication unit 232. The GPU 224 has the functions of a geometry engine and a rendering processor, performs drawing processing in accordance with drawing commands from the CPU 222, and outputs the drawing results to the output unit 236. One or both of the CPU 222 and the GPU 224 can also be called a processor. The main memory 226 is composed of RAM (Random Access Memory), and stores programs and data necessary for processing.

[0024] 5 shows the internal circuit configuration of the head mounted display 100. The head mounted display 100 includes a CPU 120, a main memory 122, a display unit 124, and an audio output unit 126. These units are connected to one another via a bus 128. An input / output interface 130 is further connected to the bus 128. To the input / output interface 130, a communication unit 132 including a wireless communication interface, a motion sensor 134, an eye tracking sensor 136, and the stereo camera 110 are connected.

[0025] The CPU 120 processes information acquired from each unit of the head mounted display 100 via the bus 128, and supplies display image and audio data acquired from the image generating device 200 to the display unit 124 and audio output unit 126. The main memory 122 stores programs and data necessary for processing by the CPU 120.

[0026] The display unit 124 includes a display panel such as a liquid crystal panel or an organic EL panel, and displays images in front of the eyes of a user wearing the head mounted display 100. The display unit 124 realizes stereoscopic vision by displaying a pair of stereo images on a left-eye display panel provided in front of the user's left eye and a right-eye display panel provided in front of the user's right eye. The display unit 124 may further include a pair of lenses (a lens for the left eye and a lens for the right eye) that are positioned between the display panel and the user's eyes when the head mounted display 100 is worn, and that expand the user's field of view.

[0027] The audio output unit 126 is composed of speakers or earphones provided at positions corresponding to the user's ears when the head mounted display 100 is worn, and allows the user to hear audio. The communication unit 132 is an interface for sending and receiving data to and from the image generating device 200, and realizes communication using known wireless communication technology such as Bluetooth (registered trademark).

[0028] The motion sensor 134 includes a gyro sensor and an acceleration sensor, and acquires the angular velocity and acceleration of the head mounted display 100. The eye tracking sensor 136 is a known sensor for eye tracking. Eye tracking, which can also be called gaze measurement, is a technology for detecting the position, movement, and gaze direction of the user's pupils (which can also be called eyeballs). For example, the eye tracking sensor 136 detects the position and movement of the user's pupils using infrared rays or the like.

[0029] As shown in Fig. 1, the stereo camera 110 is a pair of video cameras that capture the surrounding real space from left and right viewpoints in a field of view that corresponds to the user's viewpoint. An image of the user's surrounding space captured by the stereo camera 110 will be referred to as a "camera image" below. A camera image can be said to be an image of the real space in the user's line of sight (typically in front of the user), and can also be said to be an image that captures an object that exists in the user's line of sight.

[0030] The data transmitted from the head mounted display 100 to the image generating device 200 via the communication unit 132 includes the following contents. (1) Measurement value by motion sensor 134. (2) Measurements taken by eye tracking sensor 136. (3) Data of images (camera images) captured by the stereo camera 110.

[0031] Here, we will explain the problems with conventional techniques for setting the floor height of a virtual space and the features of the image display system 10 of the embodiment. A known conventional method for setting the floor height of a virtual space is to touch the floor in real space using a controller. However, the inventors felt that this conventional method is not universal from the perspective of accessibility, as it can be difficult for people who have difficulty bending their knees.

[0032] Therefore, the image display system 10 of the embodiment provides an adjustment screen, which is a user interface for allowing a user wearing the head-mounted display 100 to adjust the height of the floor of the virtual space displayed on the head-mounted display 100. The adjustment screen includes an image of a height chart. When an operation to set the user's height is input on the image of the height chart on the adjustment screen, the height of the floor of the virtual space is set according to the set height of the user. This can assist the user in appropriately setting the height of the floor of the virtual space. Also, a highly accessible method for setting the height of the floor of the virtual space can be provided.

[0033] 6 is a block diagram showing the functional blocks of the image generation device 200. The image generation device 200 executes various information processes such as the progress of the VR game and communication with the server, but the following mainly describes the functional blocks related to setting the height of the floor of the virtual space.

[0034] The multiple functional blocks shown in Fig. 6 can be realized in hardware by the configuration of the CPU 222, GPU 224, main memory 226, storage unit 234, etc. shown in Fig. 4, and can be realized in software by a computer program that implements the functions of the multiple functional blocks. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms using only hardware, only software, or a combination thereof, and are not limited to any one of them.

[0035] The image generating device 200 includes a data processing unit 250 and a data storage unit 252. The data storage unit 252 corresponds to the storage unit 234 in Fig. 4 and stores data referenced or updated by the data processing unit 250. For example, the data storage unit 252 stores image data of each element arranged on an adjustment screen described later in relation to Fig. 8 etc.

[0036] The data storage unit 252 also includes a play area storage unit 254. The play area storage unit 254 stores data related to the play area. The play area is an area in which a user wearing the head mounted display 100 can move while playing an application (e.g., a VR game). The play area can be said to be an area or range in the user's surrounding space (i.e., the real-world space surrounding the user) in which the user is permitted to move around while viewing a VR image (e.g., a three-dimensional image of a VR game).

[0037] The play area storage unit 254 may store data indicating the positions of points constituting the boundary of the play area (for example, coordinate values ​​of each point in a world coordinate system) as data related to the play area. In addition, the play area storage unit 254 of the embodiment further stores data indicating the height of the floor of the virtual space in which the VR game is played. The height of the floor of the virtual space can also be considered as the vertical distance from a reference position of the head-mounted display 100 to the floor.

[0038] The data processing unit 250 executes various types of data processing. The data processing unit 250 transmits and receives data to and from the head mounted display 100 and the controller 140 via the communication unit 232, the output unit 236, and the input unit 238 shown in Fig. 4. For example, the data processing unit 250 acquires camera images and sensor data transmitted from the head mounted display 100, and acquires data related to user operations transmitted from the controller 140.

[0039] The data processing unit 250 includes a system unit 260, an app execution unit 262, and a display control unit 264. The functions of the multiple functional blocks included in the data processing unit 250 may be implemented in a computer program. The processor (e.g., CPU 222 and GPU 224) of the image generating device 200 may perform the functions of the multiple functional blocks included in the data processing unit 250 by reading the computer program stored in a storage (e.g., memory unit 234) of the image generating device 200 into the main memory 226 and executing the program.

[0040] The App execution unit 262 reads data related to an application (a VR game in this embodiment) selected by the user from the data storage unit 252, and executes the application selected by the user. The App execution unit 262 generates a VR image showing the execution result of the VR game based on (1) the data related to the play area stored in the play area storage unit 254, (2) the camera image acquired by the system unit 260, (3) the position and orientation of the head mounted display 100 acquired by the system unit 260, and (4) the user's line of sight measured by the system unit 260. The VR image includes an image for the left eye and an image for the right eye.

[0041] The display control unit 264 transmits data of various VR images generated by the App execution unit 262 to the head mounted display 100, and causes the VR images to be displayed on the display unit 124 of the head mounted display 100. The display unit 124 of the head mounted display 100 displays the image for the left eye generated by the App execution unit 262 on a display panel for the left eye, and displays the image for the right eye generated by the App execution unit 262 on a display panel for the right eye.

[0042] The system unit 260 executes system processing related to the head mounted display 100. The system unit 260 provides common services to multiple applications (e.g., multiple VR games) for the head mounted display 100. The common services include, for example, providing play area data, providing camera images, providing information on the position and orientation of the head mounted display 100, and providing gaze measurement results.

[0043] The system unit 260 also executes processing related to setting the play area. The processing related to setting the play area includes processing to support adjustment of the floor height of the virtual space. The floor height of the virtual space can be referred to as the floor height of the play area, the floor height recognized by the App execution unit 262, or the floor height set in the VR game.

[0044] The system unit 260 includes a camera image acquisition unit 270 , a position and orientation acquisition unit 272 , a line-of-sight measurement unit 274 , a play area setting unit 276 , an adjustment screen generation unit 278 , and a floor setting unit 280 .

[0045] The camera image acquisition unit 270 acquires data of the camera image captured by the stereo camera 110 of the head mounted display 100 and transmitted from the head mounted display 100 .

[0046] The position and orientation acquisition unit 272 acquires the position and orientation of the head mounted display 100. The position and orientation acquisition unit 272 detects the position and orientation of the head mounted display 100 worn on the user's head at a predetermined rate based on the detection value of the motion sensor 134 of the head mounted display 100. The position and orientation of the head mounted display 100 can also be said to be the position and orientation of the head of the user wearing the head mounted display 100.

[0047] The position of the head mounted display 100 may be coordinates indicating the position where the head mounted display 100 exists in a three-dimensional space in the real world. The orientation of the head mounted display 100 may be the tilt of the head mounted display 100 in three axes, namely, the vertical direction, the horizontal direction, and the height direction. The position and orientation acquisition unit 272 may acquire the position and orientation of the head mounted display 100 based on a camera image transmitted from the head mounted display 100. Alternatively, the position and orientation acquisition unit 272 may acquire the position and orientation of the head mounted display 100 based on both the detection value of the motion sensor 134 of the head mounted display 100 and the camera image.

[0048] The gaze measurement unit 274 uses known eye tracking technology to detect the position, movement and gaze direction of the eyes of the user wearing the head mounted display 100 based on the detection values ​​of the eye tracking sensor 136 of the head mounted display 100.

[0049] The play area setting unit 276 executes various processes related to setting the play area. The play area setting unit 276 sets the play area based on the camera image acquired by the camera image acquisition unit 270 and the user's operation input via the controller 140. The play area setting unit 276 includes a function as a play area detection unit, and specifically, automatically detects the play area from the surrounding space of the user wearing the head mounted display 100 based on the camera image acquired by the camera image acquisition unit 270. The play area setting unit 276 stores data related to the detected and set play area in the play area storage unit 254.

[0050] The adjustment screen generation unit 278 generates data of an adjustment screen for allowing a user wearing the head mounted display 100 to adjust the height of the floor surface of the virtual space displayed on the head mounted display 100. The display control unit 264 transmits the data of the adjustment screen generated by the adjustment screen generation unit 278 to the head mounted display 100, and causes the display unit 124 of the head mounted display 100 to display the adjustment screen.

[0051] The floor setting unit 280 sets the height of the floor of the virtual space in response to an operation on the adjustment screen input using the buttons 142 or analog stick 144 of the controller 140, and stores data indicating the height of the floor of the virtual space in the play area storage unit 254. In the embodiment, when an operation to set the user's height is input using the image of a height gauge on the adjustment screen, the floor setting unit 280 sets the height of the floor of the virtual space in response to the height of the user.

[0052] The operation of the image generating device 200 configured as above will now be described. Fig. 7 is a flowchart showing the operation of the image generating device 200. Fig. 7 shows the operation executed when a user wearing the head mounted display 100 uses the controller 140 to select a menu for setting a play area from among a plurality of setting menus for the head mounted display 100 provided by the image generating device 200.

[0053] 7, a camera image acquisition unit 270 of the image generating device 200 sequentially acquires a plurality of images (camera images) captured by the stereo camera 110 of the head mounted display 100 and transmitted from the head mounted display 100. Furthermore, a position and orientation acquisition unit 272 of the image generating device 200 repeatedly acquires the position and orientation of the head mounted display 100 based on the images captured by the stereo camera 110 of the head mounted display 100 and / or measurements by the motion sensor 134. A gaze measurement unit 274 of the image generating device 200 detects the position, movement, and gaze direction of the eyeballs of the user wearing the head mounted display 100 based on measurements by the eye tracking sensor 136 of the head mounted display 100.

[0054] The play area setting unit 276 of the image generating device 200 automatically detects a play area in the space surrounding the user wearing the head-mounted display 100 based on camera images and motion sensor data acquired from the head-mounted display 100 (S10). For example, the play area setting unit 276 may estimate the 3D shape of the user's room using a known method based on the camera images and motion sensor data corresponding to the camera images, and estimate the 3D shape as the shape of the play area. The play area setting unit 276 stores play area data including coordinate values ​​of a point cloud that constitutes the boundary of the play area in the play area storage unit 254.

[0055] Also, in S10, the play area setting unit 276 detects a plane perpendicular to the vertical direction indicated by the motion sensor data based on the estimated shape of the play area, and estimates the result of combining the detected planes of the same height as the shape of the floor of the play area (i.e., the floor of the virtual space). The play area setting unit 276 estimates the height of the detected floor (in other words, the distance from the head-mounted display 100 to the floor) using a known method such as triangulation. The play area setting unit 276 further stores the estimated shape and height of the floor in the play area storage unit 254.

[0056] If the user selects to use a height meter UI (User Interface) to adjust the floor height (Y in S11), the adjustment screen generating unit 278 of the image generating device 200 generates data of an adjustment screen including a height meter image (S12). The display control unit 264 of the image generating device 200 causes the head-mounted display 100 to display the adjustment screen including the height meter image (S13).

[0057] Fig. 8 shows an example of the adjustment screen. The adjustment screen 300 displays an AR image using video see-through. Specifically, the adjustment screen 300 displays an image of a real-world space (here, the user's room) captured by the stereo camera 110 of the head-mounted display 100. The adjustment screen 300 in Fig. 8 shows the controller 140 held by the user.

[0058] The adjustment screen 300 also includes a height meter image 302 and a user image 304. The height meter image 302 includes a scale on the height meter. The scale on the height meter may be set so that the height of the floor surface detected by the play area setting unit 276 and stored in the play area storage unit 254 is set to 0, and the scale value increases vertically upward from the floor surface. The user image 304 is an image that simulates a user wearing the head mounted display 100. Hereinafter, the adjustment screen including the height meter image 302 and the user image 304 will also be referred to as a first adjustment screen 300a.

[0059] 8, the user inputs an operation to set the user's height on the height scale image 302. The operation to set the user's height on the height scale image 302 is an operation input to the button 142 or analog stick 144 of the controller 140, and is an operation to specify a position (which can also be considered a value) that matches the user's height on the scale of the height scale image 302. Specifically, the user aligns the top of the head of the user image 304 with the position that matches the user's height on the scale of the height scale image 302 by inputting an operation in the up or down direction on the button 142 or analog stick 144 of the controller 140, and inputs a setting completion operation.

[0060] In the embodiment, the height of the floor surface stored in the play area storage unit 254 is the vertical distance from a predetermined reference part of the head mounted display 100 (for example, the center of the front of the housing 108) to the detected floor surface. Also, the distance from the reference part of the head mounted display 100 to the top of the head of the user image 304 (hereinafter also referred to as "top of head distance") is a predetermined fixed value, for example, 10 centimeters.

[0061] Returning to Figure 7, when the user image 304 is aligned with the scale on the height meter image 302 on the first adjustment screen 300a and the setting completion operation is input (Y in S14), the floor setting unit 280 of the image generating device 200 saves the floor height set on the first adjustment screen 300a in the play area memory unit 254 (S18).

[0062] In this case, the floor setting unit 280 may set the height of the floor based on the height (e.g., 160 centimeters) remaining after subtracting the head-top distance (e.g., 10 centimeters) from the height (e.g., 170 centimeters) indicated by the scale of the height scale image 302 aligned with the top of the head of the user image 304. For example, the floor setting unit 280 may set the vertical distance from the reference position of the head mounted display 100 to the floor to 160 centimeters, or may set the height of the floor to minus 160 centimeters from the reference position of the head mounted display 100.

[0063] If the setting completion operation is not input on the first adjustment screen 300a and the gaze measurement unit 274 detects that the user's gaze is directed downward (for example, vertically downward) (N in S14), the floor setting unit 280 generates data for an adjustment screen 300 (hereinafter also referred to as the "second adjustment screen 300b") that shows an image of the floor surface in real space superimposed on the floor surface in the virtual space. The display control unit 264 displays the second adjustment screen 300b on the head-mounted display 100. If the user's gaze direction changes from the horizontal direction (a direction perpendicular to the vertical direction) to a downward direction, the floor setting unit 280 and the display control unit 264 switch the display target from the first adjustment screen 300a to the second adjustment screen 300b.

[0064] FIG. 9 also shows an example of the adjustment screen 300. FIG. 9 shows an example of a second adjustment screen 300b. The second adjustment screen 300b includes a real floor image 306 and a virtual floor image 308. The real floor image 306 is an image of the floor in real space captured by the stereo camera 110 of the head-mounted display 100. The virtual floor image 308 is an image showing the floor in virtual space detected by the play area setting unit 276. The adjustment screen generation unit 278 places the virtual floor image 308 at a position based on the height of the floor stored in the play area storage unit 254 (in other words, the vertical distance from the reference position of the head-mounted display 100 to the floor). Note that a predetermined transparency (also called transmittance) greater than 0 may be set for both the real floor image 306 and the virtual floor image 308, and they may be displayed semi-transparently so that the user can view both the real floor image 306 and the virtual floor image 308 regardless of which is on top.

[0065] The user's operation on the second adjustment screen 300b is an operation input to the button 142 or analog stick 144 of the controller 140, and is an operation for adjusting the height of the floor surface in the real space and the height of the floor surface in the virtual space. Specifically, the user adjusts the vertical position of the virtual floor image 308 by inputting an up or down operation to the button 142 or analog stick 144 of the controller 140 so that the real floor image 306 and the virtual floor image 308 exactly overlap, and inputs a setting completion operation.

[0066] Returning to FIG. 7, when a setting completion operation is input on the second adjustment screen 300b (Y in S15), the floor surface setting unit 280 saves the floor surface height set on the second adjustment screen 300b in the play area memory unit 254 (S18).

[0067] In S18, the floor setting unit 280 may derive the difference between the floor height based on the position of the virtual floor image 308 when the setting completion operation is input and the estimated value of the floor height (i.e., the value before adjustment) stored in the play area storage unit 254, and may update the floor height stored in the user image 304 to reflect the difference. For example, if the estimated value of the floor height is minus 150 centimeters from the reference position of the head mounted display 100 and the virtual floor image 308 is pressed down by 10 centimeters on the second adjustment screen 300b, the floor setting unit 280 may update the floor height to minus 160 centimeters from the reference position of the head mounted display 100.

[0068] If the setting completion operation is not input on the second adjustment screen 300b (N in S15), S18 is skipped and the processing in this figure ends. Note that, returning to S14, the display control unit 264 may continue displaying the first adjustment screen 300a or the second adjustment screen 300b depending on the direction of the user's line of sight.

[0069] The image generating device 200 further provides a method for adjusting the floor height of the virtual space that is different from the method using the adjustment screen 300 (first adjustment screen 300a and second adjustment screen 300b). This different method is a conventional method for setting the floor height of the virtual space, that is, a method for touching the controller 140 to the floor in real space. Instead of using a height measuring UI to adjust the floor height, the user can select the method of touching the controller 140 to the floor. This allows the user to set the floor height of the virtual space using a method that is easy for the user from among various methods. For example, a user who is familiar with the conventional method for setting the floor height of the virtual space can select the method of touching the controller 140 to the floor instead of using a height measuring UI.

[0070] When the user selects the method of touching the controller 140 to the floor surface in adjusting the floor height (N in S11), the display control unit 264 causes the head-mounted display 100 to display information instructing the user to place the controller 140 on the floor surface (S16). At this time, the display control unit 264 may cause the head-mounted display 100 to display an AR image by video see-through including the above-mentioned instruction information.

[0071] When it is detected that the controller 140 has moved to a position of a predetermined height (e.g., the floor surface in the real space) (Y in S17), the floor surface setting unit 280 estimates the height of the floor surface in the virtual space according to the height position of the controller 140, and stores data indicating the estimated floor surface height in the play area storage unit 254 (S18). The detection of the controller 140 having moved to a position of a predetermined height (e.g., the floor surface in the real space) and the estimation of the floor surface height triggered by this detection may be realized by known techniques. In this case, the position of the predetermined height means the lowest position to which the controller 140 has moved while adjusting the height of the floor surface in the virtual space. If it is not detected that the controller 140 has moved to a position of a predetermined height (e.g., the floor surface in the real space) (N in S17), the process of S18 is skipped and the flow in this figure is terminated. Note that it is also possible to wait until it is detected that the controller 140 has moved to a position of a predetermined height (e.g., the floor surface in the real space), and for example, to continue displaying information instructing the user to place the controller 140 on the floor surface.

[0072] When the adjustment of the floor height of the virtual space is completed and a user operation to instruct the start of the VR game is input, the App execution unit 262 of the image generation device 200 executes the VR game using the play area data stored in the play area storage unit 254. Based on the floor height of the virtual space stored in the play area storage unit 254, the App execution unit 262 generates a VR image in which various characters, buildings, items, etc. are arranged on the floor of the virtual space.

[0073] According to the image generating device 200 of the embodiment, the floor height of the virtual space is set in response to an operation of setting the user's height on the height scale image 302 on the adjustment screen 300 (specifically, an operation of specifying a position on the scale of the height scale image 302 that corresponds to the user's height). This makes it possible to provide a user interface that makes it easy to appropriately set the floor height of the virtual space through an intuitive operation. Furthermore, according to the image generating device 200, by changing the mode of the adjustment screen in response to the user's line of sight, it is possible to provide a user interface that makes it easy to appropriately set the floor height of the virtual space through an intuitive operation and that is adapted to the user's line of sight.

[0074] The present invention has been described above based on the embodiments. These embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component or each treatment process, and that such modifications are also within the scope of the present invention.

[0075] A modified example will now be described. At least some of the functions implemented in the image generation device 200 in the above embodiment may be implemented in the head-mounted display 100, or may be implemented in a server connected to the image generation device 200 via a network. For example, the head-mounted display 100 may have a function to generate data for various screens and images based on camera images and sensor measurement values. Furthermore, the server may have a function to generate data for various screens and images based on camera images and sensor measurement values, and the head-mounted display 100 may display screens and images generated by the server.

[0076] Any combination of the above-described examples and modifications is also useful as an embodiment of the present disclosure. A new embodiment resulting from a combination will have the combined effects of the combined examples and modifications. It will also be understood by those skilled in the art that the functions to be performed by each constituent element described in the claims can be realized by each component shown in the examples and modifications alone or in combination. [Explanation of symbols]

[0077] 100 head-mounted display, 140 controller, 142 button, 144 analog stick, 200 image generation device, 264 display control unit, 278 adjustment screen generation unit, 280 floor surface setting unit.

Claims

1. an adjustment screen generation unit that generates an adjustment screen including an image of a height meter, the adjustment screen allowing a user wearing a head-mounted display to adjust the height of a floor surface of a virtual space displayed on the head-mounted display; a display control unit that displays the adjustment screen on the head-mounted display; a floor setting unit that sets a floor height of the virtual space in accordance with the height of the user set in the image of the height meter when an operation for setting the height of the user is input for the image of the height meter on the adjustment screen; An information processing device comprising:

2. the operation of setting the user's height on the image of the height meter is an operation input to a button or stick of a controller, and is an operation of designating a position on the scale of the height meter that corresponds to the user's height; The information processing device according to claim 1 .

3. the adjustment screen generation unit generates an adjustment screen that shows an image of a floor surface in the real space superimposed on an image of the floor surface in the virtual space when the user's line of sight is directed downward; the floor setting unit sets the height of the floor of the virtual space in response to an operation input to a button or a stick of a controller, the operation being to match the height of the floor of the real space with the height of the floor of the virtual space; 3. The information processing device according to claim 1.

4. The user can select a method for adjusting the height of the floor surface of the virtual space other than a method using the adjustment screen.

4. The information processing device according to claim 1.

5. generating an adjustment screen including an image of a height gauge, the adjustment screen allowing a user wearing a head-mounted display to adjust the height of a floor of the virtual space displayed on the head-mounted display; displaying the adjustment screen on the head-mounted display; when an operation for setting the height of the user is inputted for the image of the height meter on the adjustment screen, setting the height of a floor surface of the virtual space in accordance with the height of the user set in the image of the height meter; The computer performs the floor height adjustment method.

6. a function of generating an adjustment screen including an image of a height meter, the adjustment screen allowing a user wearing a head-mounted display to adjust the height of a floor surface of a virtual space displayed on the head-mounted display; and a function of displaying the adjustment screen on the head-mounted display; a function of setting the height of the floor of the virtual space in accordance with the height of the user set in the image of the height meter when an operation for setting the height of the user is input for the image of the height meter on the adjustment screen; A computer program that enables a computer to achieve the above.

Images