Information Processing Apparatus, Information Processing Method, and Computer Program

Abstract

To provide a technique that supports setting of a play area where a user wearing a head-mounted display is allowed to move while playing an application.SOLUTION: An image creation device 200 automatically detects a play area based on a camera image capturing a peripheral space of a user and picked up by a camera of a head-mounted display 100. The image creation device 200 causes the head-mounted display 100 to display an image indicating the play area that is automatically detected. The image creation device 200 receives the user's operation to edit the play area from a controller 140, and changes the shape of the play area according to the operation.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to data processing technology, and more particularly to an information processing apparatus, an information processing method, and a computer program.

Background Art

[0002] An image display system in which a user wearing a head-mounted display can view a target space from a free viewpoint has become widespread. For example, there is known electronic content that realizes virtual reality (VR) by using a virtual three-dimensional space as a display target and displaying an image corresponding to the user's line-of-sight direction on the head-mounted display. By using a head-mounted display, it is possible to enhance the sense of immersion in the video and improve the operability of applications such as games. In addition, a walk-through system has been developed in which a user wearing a head-mounted display can virtually walk around in a space displayed as a video by physically moving.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The movable range of a user wearing a head-mounted display during play of an application needs to be restricted according to the size of the user's room and the environment such as furniture and luggage placed in the user's room.

[0004] The present invention has been made in view of such problems, and one object thereof is to provide a technique for assisting in setting a playable area in which a user wearing a head-mounted display can move during play of an application.

Means for Solving the Problems

[0005] To solve the above problems, an information processing apparatus according to an aspect of the present invention includes a play area detection unit that automatically detects a playable area in which a user wearing a head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display, a display control unit that causes the head-mounted display to display an image indicating the playable area automatically detected by the play area detection unit, and a play area editing unit that accepts an operation of the user for editing the play area and changes the shape of the play area according to the operation.

[0006] Another aspect of the present invention is an information processing method. This method includes steps of a computer automatically detecting a playable area in which a user wearing a head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display, causing the head-mounted display to display an image indicating the automatically detected playable area, accepting an operation of the user for editing the play area, and changing the shape of the play area according to the operation.

[0007] Note that any combination of the above components, and those obtained by converting the expression of the present invention among a system, a computer program, a recording medium in which the computer program is readable, a data structure, etc. are also effective as aspects of the present invention.

Advantages of the Invention

[0008] According to the present invention, it is possible to assist in setting a playable area in which a user wearing a head-mounted display can move during play of an application.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Embodiment for Carrying Out the Invention

[0010] This embodiment relates to an image display system that displays an application image on a head-mounted display worn on a user's head. FIG. 1 shows an external appearance example of the head-mounted display 100. The head-mounted display 100 of the embodiment is composed of an output mechanism unit 102 and a mounting mechanism unit 104. The mounting mechanism unit 104 includes a mounting band 106 that wraps around the head when worn by the user to achieve fixation of the device.

[0011] The output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the user wears the head-mounted display 100, and a display panel is provided inside so as to face the eyes during wearing. The display panel of the head-mounted display 100 of the embodiment is assumed to be non-transmissive. That is, the head-mounted display 100 of the embodiment is an optically non-transmissive head-mounted display.

[0012] Furthermore, an eyepiece lens that is located between the display panel and the user's eyes when the head-mounted display 100 is worn and expands the user's viewing angle may be provided inside the housing 108. The head-mounted display 100 may further include a speaker or earphone at a position corresponding to the user's ear during wearing. Also, the head-mounted display 100 incorporates a motion sensor and detects the translational and rotational movements of the head of the user wearing the head-mounted display 100, and thus the position and posture at each moment.

[0013] In addition, the head-mounted display 100 is provided with a stereo camera 110 on the front surface of the housing 108. The stereo camera 110 captures a moving image of the surrounding real space in a field of view corresponding to the user's line of sight. If the captured image is immediately displayed, so-called video see-through can be realized, where the state of the real space in the direction the user is facing can be seen as it is. Furthermore, if a virtual object is drawn on the image of the real object shown in the captured image, augmented reality (AR) can be realized.

[0014] Figure 2 shows a configuration example of the image display system 10 of the embodiment. The image display system 10 includes a head-mounted display 100, an image generation device 200, and a controller 140. The head-mounted display 100 is connected to the image generation device 200 by wireless communication. The image generation device 200 may be further connected to a server via a network. In that case, the server may provide data of an online application such as a game that multiple users can participate in via the network to the image generation device 200.

[0015] The image generation device 200 is an information processing device that identifies the position of the viewpoint and the direction of the line of sight based on the position and posture of the head of the user wearing the head-mounted display 100, and generates a display image so as to have a corresponding field of view and outputs it to the head-mounted display 100. For example, the image generation device 200 may generate a virtual world, which is the stage of the game, as a display image while advancing an electronic game, or may display a moving image for viewing or information providing regardless of whether it is a virtual world or the real world. Further, by displaying a panoramic image with a wide angle of view centered on the user's viewpoint on the head-mounted display 100, a deep sense of immersion in the display world can be given to the user. Note that the image generation device 200 may be a stationary game machine or a PC.

[0016] The controller 140 is a controller (for example, a game controller) that is held by the user's hand and to which user operations for controlling image generation in the image generation device 200 and image display in the head-mounted display 100 are input. The controller 140 is connected to the image generation device 200 by wireless communication. As a modification, one or both of the head-mounted display 100 and the controller 140 may be connected to the image generation device 200 by wired communication via a signal cable or the like.

[0017] FIG. 3 is a diagram for explaining an example of an image world that the image generation device 200 causes the head-mounted display 100 to display. In this example, the user 12 is creating a state of being in a room that is a virtual space. In the world coordinate system that defines the virtual space, as shown in the drawing, objects such as walls, floors, windows, tables, and objects on the table are arranged. The image generation device 200 defines the view screen 14 in the world coordinate system according to the position of the user 12's viewpoint and the direction of the line of sight, and draws a display image by representing an image of the object thereon.

[0018] The image generation device 200 acquires the position of the user 12's viewpoint and the direction of the line of sight (hereinafter, these may be collectively referred to as "viewpoint") from the head-mounted display 100 at a predetermined rate, and changes the position and direction of the view screen 14 accordingly. Thereby, an image can be displayed on the head-mounted display 100 in a visual field corresponding to the user's viewpoint. Further, if the image generation device 200 generates a stereoscopic image having parallax and displays the stereoscopic image in the left and right regions of the display panel of the head-mounted display 100, the virtual space can also be stereoscopically viewed by the user 12. Thereby, the user 12 can experience virtual reality as if being in the room of the display world.

[0019] FIG. 4 shows the internal circuit configuration of the image generation device 200. The image generation device 200 includes a CPU (Central Processing Unit) 222, a GPU (Graphics Processing Unit) 224, and a main memory 226. These respective units are interconnected via a bus 230. An input / output interface 228 is further connected to the bus 230. A communication unit 232, a storage unit 234, an output unit 236, an input unit 238, and a recording medium drive unit 240 are connected to the input / output interface 228.

[0020] The communication unit 232 includes peripheral device interfaces such as USB and IEEE 1394, and network interfaces such as wired LAN or wireless LAN. The storage unit 234 includes a hard disk drive, a non-volatile memory, and the like. The output unit 236 outputs data to the head-mounted display 100. The input unit 238 receives data input from the head-mounted display 100 and also receives data input from the controller 140. The recording medium drive unit 240 drives a removable recording medium such as a magnetic disk, an optical disk, or a semiconductor memory.

[0021] The CPU 222 controls the entire image generation device 200 by executing the operating system stored in the storage unit 234. Also, the CPU 222 executes various programs (such as VR game applications) read from the storage unit 234 or a removable recording medium, loaded into the main memory 226, or downloaded via the communication unit 232. The GPU 224 has the functions of a geometry engine and a rendering processor, performs rendering processing according to a drawing command from the CPU 222, and outputs the rendering result to the output unit 236. The main memory 226 is composed of a RAM (Random Access Memory) and stores programs and data necessary for processing.

[0022] FIG. 5 shows the internal circuit configuration of the head-mounted display 100. The head-mounted display 100 includes a CPU 120, a main memory 122, a display unit 124, and an audio output unit 126. These units are interconnected via a bus 128. An input / output interface 130 is further connected to the bus 128. A communication unit 132 including a wireless communication interface, a motion sensor 134, and a stereo camera 110 are connected to the input / output interface 130.

[0023] The CPU 120 processes the information acquired from each part of the head-mounted display 100 via the bus 128, and supplies the display image and audio data acquired from the image generation device 200 to the display unit 124 and the audio output unit 126. The main memory 122 stores programs and data necessary for the processing in the CPU 120.

[0024] The display unit 124 includes a display panel such as a liquid crystal panel or an organic EL panel, and displays an image in front of the eyes of the user wearing the head-mounted display 100. The display unit 124 may realize stereoscopic vision by displaying a pair of stereo images in areas corresponding to the left and right eyes. The display unit 124 may further include a pair of lenses that are located between the display panel and the user's eyes when the head-mounted display 100 is worn and expand the user's viewing angle.

[0025] The audio output unit 126 is composed of speakers or earphones provided at positions corresponding to the user's ears when the head-mounted display 100 is worn, and allows the user to hear sound. The communication unit 132 is an interface for transmitting and receiving data to and from the image generation device 200, and realizes communication by a known wireless communication technology such as Bluetooth (registered trademark). The motion sensor 134 includes a gyro sensor and an acceleration sensor, and acquires the angular velocity and acceleration of the head-mounted display 100.

[0026] As shown in FIG. 1, the stereo camera 110 is a pair of video cameras that capture the surrounding real space from the left and right viewpoints in a field of view corresponding to the user's viewpoint. An image captured by the stereo camera 110 and showing the user's surrounding space is hereinafter also referred to as a "camera image". It can also be said that the camera image is an image in which an object existing in the user's line of sight direction (typically the front of the user) is reflected. The measurement values by the motion sensor 134 and the data of the captured image (camera image) by the stereo camera 110 are transmitted to the image generation device 200 via the communication unit 132 as necessary.

[0027] In the image display system 10 of the embodiment, a play area is set that defines the range of the real world where a user wearing the head-mounted display 100 can move during the play of an application. The play area can be said to be an area or range within the surrounding space of the user (the real-world space spreading around the user) where the user is permitted to move around while viewing a virtual reality image (hereinafter also referred to as a "VR image"). When the user attempts to deviate from the play area or has deviated from the play area during the play of the application, the image display system 10 provides the user with a warning that prompts attention or prompts a return to the play area.

[0028] The above application is assumed to be a game application that displays a VR image on the head-mounted display 100, and is hereinafter also referred to as a "VR game". For example, the VR game may be a tennis game that displays a VR image showing a tennis court in a virtual world and changes the position of a character on the tennis court in the virtual world as the user moves (such as walking) in the real world.

[0029] The first feature of the image display system 10 of the embodiment will be described. As a first step, the image generation device 200 automatically detects a play area where a user wearing the head-mounted display 100 can move during the play of an application. As a second step after the first step, the image generation device 200 receives a user operation for editing the automatically detected play area and changes the shape of the play area according to the operation. This supports the user in efficiently setting a play area with an arbitrary shape.

[0030] The second feature of the image display system 10 of the embodiment will be described. When the user manually edits the play area, the image generation device 200 reduces the play area according to the user's operation. For example, the image generation device 200 provides a user interface for manually editing the play area and a user interface that can specify an area to be excluded from the play area. The image generation device 200 deletes the area specified by the user via the user interface from the play area. This supports the user in efficiently setting the desired play area.

[0031] The third feature of the image display system 10 of the embodiment will be described. The image generation device 200 automatically switches the operation mode of an application (a VR game in the embodiment) that generates a VR image in consideration of the set play area according to the size of the set play area. Specifically, the image generation device 200 determines whether to operate the application in a mode where the user can move or in a mode where the user cannot move according to the size of the set play area. Thereby, the operation mode of the application can be automatically switched according to the size of the play area.

[0032] FIG. 6 is a block diagram showing the functional blocks of the image generation device. As described above, the image generation device 200 executes general information processing such as the progress of the VR game and communication with the server. In FIG. 6, in particular, the functional blocks related to the setting of the play area are shown in detail. Note that at least a part of the functions of the image generation device 200 shown in FIG. 6 may be implemented in a server connected to the image generation device 200 via a network.

[0033] In addition, the multiple functional blocks shown in FIG. 6 can be realized in terms of hardware by the configurations such as the CPU 222, GPU 224, main memory 226, storage unit 234, etc. shown in FIG. 4, and in terms of software, they can be realized by a computer program that implements the functions of the multiple functional blocks. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof, and are not limited to any one of them.

[0034] The image generation device 200 includes a data processing unit 250 and a data storage unit 252. The data processing unit 250 executes various data processes. The data processing unit 250 transmits and receives data to and from the head-mounted display 100 and the controller 140 via the communication unit 232, output unit 236, and input unit 238 shown in FIG. 4. The data storage unit 252 stores data that is referred to or updated by the data processing unit 250.

[0035] The data storage unit 252 includes an App storage unit 254, a play area storage unit 256, and a map storage unit 258. The App storage unit 254 stores data of an application (a VR game in the embodiment) that generates VR images.

[0036] The play area storage unit 256 stores data related to the play area. The data related to the play area includes data indicating the positions of a point group that constitutes the boundary of the play area (for example, the coordinate values of each point in the world coordinate system).

[0037] The map storage unit 258 stores map data for estimating the position of the head-mounted display 100 (i.e., the position of the user wearing the head-mounted display 100). The map data of the embodiment is an image (feature points shown in the image) showing the room where the user plays a VR game, and includes a set of images (feature points shown in the image) whose positional relationship (position and direction) is known. Specifically, the map data includes a plurality of sets in which the position, line-of-sight direction, and key frame of the head-mounted display 100 are associated. Note that the map data may include other items necessary for so-called self-position estimation.

[0038] A certain set of key frames is data indicating the features of an image visible at the same set of position and line-of-sight direction. A key frame is an image generated based on an image captured by the stereo camera 110 of the head-mounted display 100 (hereinafter also referred to as a "camera image"), and is an image including a predetermined number or more of feature points. The number of feature points to be included in a key frame may be 24 or more. The feature points may include corners detected by a known corner detection method, or may be detected based on the luminance gradient. A key frame can also be said to be a set of partial images cut out from a camera image.

[0039] The data processing unit 250 includes a system unit 260, an App execution unit 290, and a display control unit 292. The functions of these multiple functional blocks may be implemented in a computer program. The CPU 222 and GPU 224 of the image generation device 200 may read the computer program from the storage unit 234 or a recording medium into the main memory 226 and execute it to exhibit the functions of the multiple functional blocks.

[0040] The App execution unit 290 reads the data of the application (VR game in the embodiment) selected by the user from the App storage unit 254 and executes the application selected by the user. The display control unit 292 transmits the data of various images (e.g., VR images and AR images) generated by the system unit 260 and the App execution unit 290 to the head-mounted display 100, and causes those images to be displayed on the display unit 124 (display panel) of the head-mounted display 100.

[0041] The system unit 260 executes system processing related to the head-mounted display 100. The system unit 260 provides common services for a plurality of applications (e.g., VR games) for the head-mounted display 100. The system unit 260 includes a camera image acquisition unit 262, a play area setting unit 264, a map generation unit 274, a mode determination unit 276, a notification unit 278, a position estimation unit 280, and a warning processing unit 282.

[0042] The camera image acquisition unit 262 acquires the camera image captured by the stereo camera 110 of the head-mounted display 100, which is transmitted from the head-mounted display 100.

[0043] The play area setting unit 264 executes various processes for assisting in setting the play area. The play area setting unit 264 sets the play area based on the camera image acquired by the camera image acquisition unit 262 and the user operation input via the controller 140. The play area setting unit 264 includes a play area detection unit 266, a floor setting unit 268, a play area editing unit 270, and a temporary setting unit 272.

[0044] The play area detection unit 266 automatically detects a play area from the surrounding space of the user wearing the head-mounted display 100 based on the camera image acquired by the camera image acquisition unit 262. The floor level setting unit 268 accepts an operation of the user for editing the height of the floor surface detected as the play area by the play area detection unit 266, and changes the height of the floor surface according to the operation.

[0045] The play area editing unit 270 accepts an operation of the user for editing the play area automatically detected by the play area detection unit 266, and changes the shape of the play area according to the operation. For example, the play area editing unit 270 reduces the play area automatically detected by the play area detection unit 266 according to the operation of the user.

[0046] When the size of the play area set by the user (that is, the play area after manual editing by the user) is smaller than a predetermined threshold, the temporary setting unit 272 temporarily sets a temporarily valid play area in place of the play area set by the user. The temporary play area is a play area where the user cannot move (in other words, a play area where the movement of the user is prohibited), and is a play area with a fixed shape.

[0047] In parallel with the play area detection unit 266 detecting the play area, the map generation unit 274 generates a map for estimating the position of the user based on the camera image acquired by the camera image acquisition unit 262. The map generation unit 274 stores the generated map data in the map storage unit 258. When a plurality of camera images obtained by mapping the surrounding space of the user from a plurality of directions and the number of camera images equal to or more than a predetermined number are input, the map generation unit 274 ends the generation of the map.

[0048] The mode determination unit 276 determines whether to operate the VR game in a mode where the user can move or in a mode where the user cannot move according to the size of the play area (in the embodiment, the play area after manual editing by the user) set by the play area setting unit 264. When the size of the play area is equal to or greater than a predetermined threshold, the mode determination unit 276 determines to operate the VR game in a mode where the user can move. When the size of the play area is less than the above threshold, the mode determination unit 276 determines to operate the VR game in a mode where the user cannot move.

[0049] The notification unit 278 notifies the App execution unit 290 of the information necessary for the execution of the VR game by the App execution unit 290. For example, when the mode determination unit 276 determines to operate the VR game in a mode where the user can move, the notification unit 278 transmits data indicating that the user can move to the VR game (in the embodiment, the App execution unit 290 that executes the VR game). Also, when the mode determination unit 276 determines to operate the VR game in a mode where the user cannot move, the notification unit 278 transmits data indicating that the user cannot move to the VR game (in the embodiment, the App execution unit 290 that executes the VR game).

[0050] The position estimation unit 280 estimates the position of the user in the real world, or in other words, the position of the user in the play area, based on the map data stored in the map storage unit 258 and the camera image acquired by the camera image acquisition unit 262. For example, the position estimation unit 280 may collate a plurality of key frames included in the map data with the camera image, and estimate the position of the user based on the collation result and the position and line-of-sight direction of the head-mounted display 100 associated with each key frame. Also, the position estimation unit 280 may estimate the position of the user using a known self-position estimation technique such as SLAM (Simultaneous Localization and Mapping). The warning processing unit 282 executes warning processing for the user according to the relationship between the boundary of the play area and the position of the user.

[0051] The operation of the image display system 10 configured as described above will be described. FIG. 7 is a flowchart showing the operation of the image generation device 200. This figure shows the operation of the image generation device 200 when setting the play area. The user can select the initial setting or resetting of the play area in the system setting menu of the head-mounted display 100. When the initial setting or resetting of the play area is selected, the play area setting unit 264 of the image generation device 200 causes the head-mounted display 100 to display a message prompting the user to look around through the display control unit 292.

[0052] FIG. 8 schematically shows the user's actions when setting the play area. In the room 30, the user wears the head-mounted display 100 on the head and moves while looking around. The user 32 in FIG. 8 indicates the user's position at this time. The arrow 34 indicates the direction of the user's line of sight. The play area 36 shows an example of the play area to be set. During the execution of S10-S12 (play area detection process and map generation process) described later, the play area setting unit 264 of the image generation device 200 causes the camera image acquired by the camera image acquisition unit 262 to be displayed on the display panel of the head-mounted display 100 through the display control unit 292, that is, performs a video see-through that shows the user the state of the real space in the direction the user is facing as it is. This enhances the safety of the user.

[0053] The head-mounted display 100 sequentially transmits data of a plurality of camera images to the image generation device 200. Each of the data of the plurality of camera images includes sensor data at the time of generating each camera image. The sensor data includes measurement values by the motion sensor 134 and includes, for example, the angular velocity and acceleration of the head-mounted display 100. Returning to FIG. 7, the camera image acquisition unit 262 of the image generation device 200 acquires the data of the camera image transmitted from the head-mounted display 100 (S10).

[0054] The play area detection unit 266 of the image generation device 200 automatically detects the play area in the user's surrounding space based on the camera image acquired in S10 (S11). Specifically, the play area detection unit 266 may estimate the 3D shape of the user's room by a known method based on the camera image and the sensor data corresponding to the camera image. Based on the estimated 3D shape of the room, the play area detection unit 266 detects a plane (typically the floor surface) perpendicular to the gravity direction indicated by the sensor data, and may detect, as the play area, the result of synthesizing a plurality of planes at the same detected height. The play area detection unit 266 stores play area data including the coordinate values of the point group constituting the boundary of the play area in the play area storage unit 256.

[0055] When detecting the play area, the play area detection unit 266 detects the height of the floor surface as the play area. The height of the floor surface may be, for example, the distance between the floor surface and the head-mounted display 100 in the gravity direction. The play area detection unit 266 stores data indicating the height of the floor surface in the play area storage unit 256. When the position of the head-mounted display 100 is the origin, the height of the floor surface may be -1 meter or the like.

[0056] In parallel with the process of S11, the map generation unit 274 of the image generation device 200 generates a map for estimating the user's position based on the camera image acquired in S10 (S12). FIG. 9 shows an example of a camera image, and FIG. 10 shows an example of a key frame. The map generation unit 274 extracts a plurality of feature points 44 included in the camera image 40 using a known method such as a corner detection method, and generates a key frame 42 indicating the extracted plurality of feature points 44. When the map generation unit 274 extracts 24 or more feature points 44 from one camera image 40, the key frame 42 including the 24 or more extracted feature points 44 is stored in the map storage unit 258 in association with the position and line-of-sight direction of the user specified by the sensor data.

[0057] The map generation unit 274 stores a plurality of criteria (hereinafter referred to as "bins") for deriving a score regarding the ratio covered by the input camera image (key frames generated from the camera image in the embodiment) in the surrounding space of the user. The plurality of bins are a plurality of criteria that divide the surrounding space of the user in different manners.

[0058] FIG. 11 shows a plurality of bins. Each of bin 50, bin 52, and bin 54 has the center at the position of the user and divides the surrounding space of the user with the gravity direction as the axis. Bin 50 is a criterion that divides the surrounding space of the user into four parts with a central angle of 90 degrees based on the line-of-sight direction of the user at the start of play area setting and the direction perpendicular to the line-of-sight direction. Bin 52 is shifted in phase by 45 degrees with respect to bin 50. Bin 54 is a criterion that divides the surrounding space of the user into 16 parts with a central angle of 22.5 degrees. Hereinafter, the divided areas in the surrounding space of the user defined by each of bin 50, bin 52, and bin 54 are referred to as "divided areas".

[0059] The map generation unit 274 derives a score regarding the ratio covered by a plurality of input camera images (key frames generated from the camera image in the embodiment) in the surrounding space of the user based on bin 50, bin 52, and bin 54. When the score becomes equal to or higher than a predetermined threshold value, the map generation unit 274 ends the map generation process.

[0060] Specifically, when the map generation unit 274 generates a key frame from the camera image, for each of bin 50, bin 52, and bin 54, it is assumed that the divided area that matches the line-of-sight direction where the key frame was obtained is filled, and the total number of filled divided areas in each bin is counted as the score. When the map generation unit 274 generates a new key frame, the number of newly filled divided areas in each bin is added to the score by the same algorithm.

[0061] Returning to FIG. 7, in the embodiment, if the score is less than the threshold value "10" (N in S13), the process returns to S10, and the detection process of the play area and the map generation process are continued. When the score becomes equal to or greater than the threshold value "10" (Y in S13), the play area detection unit 266 ends the detection process of the play area, and the map generation unit 274 ends the map generation process. The threshold value "10" of the score is a value that can be cleared if the user wearing the head-mounted display 100 can look around 180 degrees around himself / herself, but is defined as a value that cannot be cleared if the user does not look around.

[0062] After the execution of the play area detection process and the map generation process, the floor setting unit 268 of the image generation device 200 generates a floor adjustment screen based on the data indicating the height of the floor stored in the play area storage unit 256. The floor adjustment screen may include an AR image in which an object indicating the floor (for example, a semi-transparent lattice-shaped object) is superimposed on the camera image acquired by the camera image acquisition unit 262. The floor setting unit 268 causes the display panel of the head-mounted display 100 to display the floor adjustment screen via the display control unit 292. The floor setting unit 268 receives an operation of the user for adjusting the height of the floor input to the floor adjustment screen, and changes the height of the floor according to the operation of the user. The floor setting unit 268 stores the data indicating the changed height of the floor in the play area storage unit 256 (S14).

[0063] The play area editing unit 270 of the image generation device 200 generates an image indicating the play area automatically detected by the play area detection unit 266. Specifically, the play area editing unit 270 generates a play area editing screen based on the play area data stored in the play area storage unit 256. The play area editing screen includes an AR image in which an object indicating the play area is superimposed on the camera image acquired by the camera image acquisition unit 262. The display control unit 292 of the image generation device 200 causes the play area editing screen to be displayed on the display panel of the head-mounted display 100 (S15).

[0064] FIG. 12 shows an example of a play area editing screen. The play area editing screen 60 includes a play area 62 and a boundary 64. The play area 62 is an image showing a play area (typically a floor surface without obstacles), and may be, for example, an image showing a semi-transparent grid-like object. The boundary 64 is an image showing the boundary of the play area 62, and is an image that intersects the play area perpendicularly at the boundary of the play area 62. The boundary 64 may also be, for example, a semi-transparent grid-like object.

[0065] Returning to FIG. 7, the play area editing unit 270 receives an operation of a user who edits the play area 62 input to the play area editing screen 60. The play area editing unit 270 changes the shape of the play area 62 according to the operation of the user, that is, enlarges or reduces the play area 62 according to the operation of the user. The play area editing unit 270 updates the play area data stored in the play area storage unit 256 by storing the data of the play area 62 after the shape change (for example, the coordinate values of the point group constituting the boundary) in the play area storage unit 256 (S16).

[0066] FIGS. 13(a)-(d) are views of the play area editing screen 60 of FIG. 12 seen from above, and are views showing an example of editing for enlarging the play area 62. The controller cursor 66 is an object indicating a position specified by an operation input by a user wearing the head-mounted display 100 to the controller 140. For example, the controller 140 may have a function of a pointing device, and the play area editing unit 270 may display the controller cursor 66 at a position in the virtual space specified by the controller 140.

[0067] As shown in FIGS. 13(a) to 13(c), the user moves the controller cursor 66 on the play area editing screen 60 and draws a line that goes out of the play area 62 from within the play area 62 and then enters the play area 62 again as the movement locus of the controller cursor 66. In this case, as shown in FIG. 13(d), the play area editing unit 270 adds the area surrounded by the drawn line to the play area 62.

[0068] Now, consider the case where the user draws a line that goes out of the play area 62 from within the play area 62 and ends outside the play area 62 as the movement locus of the controller cursor 66. In this case, the play area editing unit 270 adds the area along the drawn line (in other words, the movement locus of the controller cursor 66) to the play area 62. For example, when the user ends the movement of the controller cursor 66 (in other words, the drawing of the line) at the position of the controller cursor 66 shown in FIG. 13(b), an area that extends thinly to the right from the original play area 62 is added to the play area 62.

[0069] FIGS. 14(a) to 14(d) show the details of the process of setting the play area on the play area editing screen 60 regarding the editing of the play area. As shown in FIG. 14(a), the play area editing unit 270 acquires the data of the play area 62 automatically detected by the play area detection unit 266. As shown in FIG. 14(b), the play area editing unit 270 grids the automatically detected play area 62, in other words, arranges the play area 62 on the grid.

[0070] As shown in FIG. 14(c), the play area editing unit 270 fills the grids inside the play area 62 with the objects 68. In this example, one object 68 is arranged in one grid. As shown in FIG. 14(d), the play area editing unit 270 extracts the boundary of the play area 62 based on the grids filled with the objects 68 and determines the boundary line of the play area 62.

[0071] Figures 15(a)-(e) show the details of the process of expanding the play area regarding the editing of the play area. Here, as shown in Figure 15(a), when the user presses the trigger of the controller 140 (i.e., the input part that triggers the display of the controller cursor 66), the position of the controller cursor 66 is inside the play area 62. The user moves the controller cursor 66 outside the play area 62 while keeping the trigger pressed, and then moves the controller cursor 66 back inside the play area 62 and releases the trigger. In this case, as shown in Figure 15(b), the play area editing unit 270 fills the grid inside the movement trajectory of the controller cursor 66 with the object 68 as the controller cursor 66 moves.

[0072] As shown in Figure 15(c), the play area editing unit 270 extracts the boundary of the play area 62 based on the grid filled with the object 68. As shown in Figure 15(d), the play area editing unit 270 determines the boundary of the play area 62 when the user releases the trigger. If there are multiple boundaries, the play area editing unit 270 selects the boundary with a larger range, in other words, the outer boundary. As shown in Figure 15(e), the play area editing unit 270 fills the grid inside the boundary line with the object 68.

[0073] Figures 16(a)-(d) are views of the play area editing screen 60 of Figure 12 seen from above, and show an example of editing to reduce the play area 62. As shown in Figures 16(a)-(c), the user moves the controller cursor 66 on the play area editing screen 60 and draws a line that enters the play area 62 from outside the play area 62 and then exits the play area 62 as a line indicating the movement trajectory of the controller cursor 66. In this case, as shown in Figure 16(d), the play area editing unit 270 deletes one of the play areas 62 divided by the drawn line.

[0074] In the embodiment, when the user is present in one of the divided play areas based on the position of the user estimated by the position estimation unit 280, the play area editing unit 270 detects this. When one of the divided play areas (for example, play area 62a in FIG. 16(c)) is an area where the user is present and the other (for example, play area 62b in FIG. 16(c)) is an area where the user is not present, the play area editing unit 270 deletes the area where the user is not present (for example, play area 62b).

[0075] As a modification, the play area editing unit 270 may leave the relatively large play area (for example, play area 62a) among the divided play areas and delete the relatively small play area (for example, play area 62b). Further, when the user is present in any of the divided play areas, the play area editing unit 270 deletes the play area where the user is not present among the divided play areas, and when the user is not present in any of the divided play areas, the play area editing unit 270 may delete the relatively small play area among the divided play areas.

[0076] Here, assume that the user draws a line that enters the play area 62 from outside the play area 62 and ends inside the play area 62 as a line indicating the movement trajectory of the controller cursor 66. In this case, the play area editing unit 270 deletes the area along the drawn line (in other words, the movement trajectory of the controller cursor 66) from the play area 62. For example, when the user ends the movement of the controller cursor 66 at the position of the controller cursor 66 shown in FIG. 16(b), the area through which the controller cursor 66 has passed is deleted from the original play area 62.

[0077] Figures 17(a)-(d) show the details of the process of reducing the play area regarding the editing of the play area. Here, as shown in Figure 17(a), when the user presses the trigger of the controller 140, the position of the controller cursor 66 is outside the play area 62. The user moves the controller cursor 66 inside the play area 62 while keeping the trigger pressed, and then moves the controller cursor 66 outside the play area 62 and releases the trigger. In this case, as shown in Figure 17(b), the play area editing unit 270 deletes the object 68 that fills the grid within the movement trajectory of the controller cursor 66 as the controller cursor 66 moves.

[0078] As shown in Figure 17(c), the play area editing unit 270 extracts the boundary of the play area 62 based on the grid filled with the object 68. As shown in Figure 17(d), when the user releases the trigger, the play area editing unit 270 determines only the boundary line of the play area 62 where the user exists and deletes the play area 62 where the user does not exist.

[0079] Returning to Figure 7. By the processing up to S16, it is the play area set by the play area setting unit 264, in other words, the data of the arbitrarily shaped play area (hereinafter also referred to as the "user-set play area") set by the user (for example, the coordinate values of the point group constituting the boundary of the play area) is stored in the play area storage unit 256. The mode determination unit 276 of the image generation device 200 derives the size and shape of the user-set play area based on the data of the user-set play area.

[0080] When the size of the user-set play area (including the shape here) is equal to or greater than a predetermined first threshold (Y in S17), the mode determination unit 276 determines that the user-set play area is a movable play area, and determines to operate the VR game in a mode where the user can move. In the embodiment, when a square with a length of 2 meters and a width of 2 meters is included in the user-set play area, it is determined that the size of the user-set play area is equal to or greater than the first threshold. The mode determination unit 276 stores data indicating that the user-set play area is a movable play area in the play area storage unit 256 (S18).

[0081] When the size of the user-set play area is less than the first threshold (N in S17), and the size of the user-set play area is equal to or greater than a predetermined second threshold smaller than the first threshold (Y in S19), the mode determination unit 276 determines that the user-set play area is a non-movable play area, and determines to operate the VR game in a mode where the user cannot move. In the embodiment, when a square with a length of 1 meter and a width of 1 meter is included in the user-set play area, it is determined that the size of the user-set play area is equal to or greater than the second threshold. The mode determination unit 276 stores data indicating that the user-set play area is a non-movable play area in the play area storage unit 256 (S20).

[0082] When the size of the user-set play area is less than the second threshold, that is, when a square with a length of 1 meter and a width of 1 meter is not included in the user-set play area (N in S19), the mode determination unit 276 determines to set a temporary play area. The temporary setting unit 272 causes the head-mounted display 100 to display a message indicating that a temporary setting area is to be set, and sets a temporary play area at the position specified by the user via the controller 140. The temporary play area is a play area with a fixed shape (in other words, a fixed size) that is prohibited from being edited by the user. The shape of the temporary play area in the embodiment is a circle with a radius of 1.5 meters.

[0083] When a temporary play area is set, the temporary setting unit 272 stores data related to the temporary play area in the play area storage unit 256 instead of the data related to the user-set play area that has been stored in the play area storage unit 256 until then. This data includes, for example, the coordinate values of the point group that constitutes the boundary of the temporary play area. Also, the temporary setting unit 272 stores data indicating that the play area is a non-movable play area in the play area storage unit 256 (S21).

[0084] FIG. 18 is also a flowchart showing the operation of the image generation device 200. This figure shows the operation of the image generation device 200 during the execution of a VR game. When the user starts a VR game on the image generation device 200 (Y in S30), the notification unit 278 transmits data of the play area (user-set play area or temporary play area) stored in the play area storage unit 256, for example, data indicating the shape and size of the play area, to the App execution unit 290. Also, the notification unit 278 transmits data indicating whether the above play area is a movable play area or a non-movable play area to the App execution unit 290 (S31).

[0085] In parallel with the process of S31, the notification unit 278 notifies the user whether movement is possible during the VR game play via the display control unit 292 (S32). Specifically, when it is determined by the mode determination unit 276 that the user operates the VR game in a movable mode, that is, when data indicating that the play area is a movable play area is stored in the play area storage unit 256, the display control unit 292 displays a message indicating that the user can move during the play of the VR game on the display panel of the head-mounted display 100.

[0086] On the other hand, when it is determined by the mode determination unit 276 that the user will operate the VR game in an immovable mode, that is, when data indicating that it is an immovable play area is stored in the play area storage unit 256, the display control unit 292 displays a message indicating that the user is immovable during the play of the VR game on the display panel of the head-mounted display 100. This message may be, for example, a message warning the user not to move during the play of the VR game.

[0087] The App execution unit 290 of the image generation device 200 reads out the program data of the VR game from the App storage unit 254 and starts the VR game (S33). For example, the App execution unit 290 may cause the user to acquire the game item by arranging the game item within the range of the play area (including the temporary play area) in the virtual space depicting the game world. The display control unit 292 displays the VR game image (for example, a VR image) generated by the App execution unit 290 on the display panel of the head-mounted display 100.

[0088] The camera image acquisition unit 262 of the image generation device 200 sequentially acquires the camera images sequentially transmitted from the head-mounted display 100, and the position estimation unit 280 sequentially estimates the position and the line-of-sight direction of the user in the real world (S34). The notification unit 278 notifies the App execution unit 290 of the position and the line-of-sight direction of the user estimated by the position estimation unit 280 (S35). The App execution unit 290 advances the VR game according to the estimated position and line-of-sight direction of the user (S36). For example, when it is a play area where the user can move, the App execution unit 290 may move the user character in the VR game according to the change in the position of the user in the real world (in other words, within the play area).

[0089] When the warning processing unit 282 of the image generation device 200 detects that the user's position in the real world has reached near the boundary of the play area, for example, when the distance from the head-mounted display 100 worn by the user to the boundary of the play area becomes equal to or less than a predetermined threshold value (for example, 30 cm) (Y in S37), it will detect this. The warning processing unit 282 will execute a predetermined warning process on the user (S38).

[0090] For example, when the user's position reaches near the boundary of the play area, the warning processing unit 282 may pass an image indicating the boundary of the play area (for example, the boundary 64 shown in FIG. 12) to the display control unit 292. The display control unit 292 may display, on the display panel of the head-mounted display 100, as a display image, an image in which an image indicating the boundary of the play area is superimposed on the game image generated by the App execution unit 290. Also, when the user's position in the real world reaches near the boundary of the play area or exceeds the boundary of the play area, the warning processing unit 282 may display a video see-through image on the display panel of the head-mounted display 100 via the display control unit 292. Further, when the user's position in the real world reaches near the boundary of the play area (for example, 30 cm), the warning processing unit 282 may first display an image indicating the boundary of the play area, and when the user's position in the real world reaches even nearer to the boundary of the play area (for example, 10 cm), it may display a video see-through image. On the other hand, if the user's position in the real world is not near the boundary of the play area (N in S37), S38 will be skipped.

[0091] When a predetermined end condition is satisfied, such as when the execution of the VR game is stopped by the user (Y in S39), the flow of this figure will end. If the end condition is not satisfied (N in S39), it will return to S34 and continue the VR game. If the VR game is not started (N in S30), the processing after S31 will be skipped.

[0092] According to the image display system 10 of the embodiment, after automatically detecting the play area based on the camera image captured by the head-mounted display 100, the user is allowed to manually edit the play area. Further, in the manual editing of the play area, a user interface (play area editing screen 60) that enables both enlargement and reduction of the play area is provided. Thereby, it is possible to assist the user in efficiently setting a play area of an arbitrary shape.

[0093] Also, according to the image display system 10 of the embodiment, according to the size of the set play area, it is automatically determined whether to operate the VR game in a mode where the user can move or in a mode where the user cannot move. In this way, by automatically switching the operation mode of the application according to the size of the set play area, it is possible to suppress the user from trying to set the play area many times, and to improve the safety of the user who wears the head-mounted display 100 and plays the application.

[0094] Also, the application that generates the VR image may arrange various objects (such as game items) in the play area, and it is desirable that the play area has a certain size. When the size of the user-set play area is small (less than the second threshold value described above), the image display system 10 of the embodiment sets a temporary play area with a fixed shape. Thereby, it is possible to secure a certain degree of size of the play area, and to prevent the world view created by the image display on the head-mounted display 100 (for example, the interest of the VR game) from being impaired due to the play area being too small.

[0095] As described above, the present invention has been described based on the embodiments. It is understood by those skilled in the art that these embodiments are illustrative, and various modifications are possible for each component or combination of each processing process, and such modifications are also within the scope of the present invention.

[0096] A modification regarding the manual editing of the play area will be described. Figures 19(a)-(d) show modified examples of editing to expand the play area. Similar to the embodiment, as shown in FIGS. 19(a) to (c), the user moves the controller cursor 66 on the play area editing screen 60 and draws a line that goes out of the play area 62 from within the play area 62 and then enters the play area 62 as a line indicating the movement locus of the controller cursor 66. In this case, as shown in FIG. 19(d), the play area editing unit 270 adds the area surrounded by the drawn line to the play area 62. Note that the width of the line indicating the movement locus of the controller cursor 66 in the modified example is thinner than the width of the line indicating the movement locus of the controller cursor 66 in the embodiment.

[0097] Here, consider the case where the user draws a line that goes out of the play area 62 from within the play area 62 and ends outside the play area 62 as a line indicating the movement locus of the controller cursor 66. In this case, in this modified example, the play area editing unit 270 maintains the shape of the play area 62 up to that point. In other words, the play area editing unit 270 cancels the drawn line, that is, does not expand the play area 62.

[0098] Figures 20(a)-(b) show modified examples of editing to reduce the play area. Similar to the embodiment, as shown in FIG. 20(a), the user moves the controller cursor 66 on the play area editing screen 60 and draws a line that enters the play area 62 from outside the play area 62 and then goes out of the play area 62 as a line indicating the movement locus of the controller cursor 66. In this case, as shown in FIG. 20(b), the play area editing unit 270 deletes one of the play areas 62 divided by the drawn line. When one of the divided play areas is the area where the user exists and the other is the area where the user does not exist, the play area editing unit 270 deletes the area where the user does not exist.

[0099] Here, consider a case where the user draws a line that enters the play area 62 from outside the play area 62 and ends inside the play area 62 as a line indicating the movement trajectory of the controller cursor 66. In this case, in this modification example, the play area editing unit 270 maintains the shape of the play area 62 up to that point. In other words, the play area editing unit 270 cancels the drawn line, that is, does not shrink the play area 62.

[0100] Any combination of the above-described embodiments and modification examples is also useful as an embodiment of the present disclosure. The new embodiments generated by the combination have the effects of the respective embodiments and modification examples combined. Also, it is understood by those skilled in the art that the functions to be achieved by each constituent element described in the claims are realized by a single one of the constituent elements shown in the embodiments and modification examples or by the cooperation thereof.

[0101] The technical ideas based on the descriptions of the embodiments and modification examples can be expressed as follows. [Item 1-1] A play area detection unit that automatically detects a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; A display control unit that causes the head-mounted display to display an image indicating the play area automatically detected by the play area detection unit; A play area editing unit that receives an operation of the user for editing the play area and changes the shape of the play area according to the operation; An information processing apparatus comprising: According to this information processing apparatus, it is possible to assist the user in efficiently setting a play area having an arbitrary shape. [Item 1-2] A map generation unit that generates a map for estimating the position of the user based on the camera image in parallel with the play area detection unit detecting the play area; The map generation unit is a plurality of camera images that reflect the surrounding space of the user from a plurality of directions. When camera images equal to or more than a predetermined number are input, the map generation process is terminated. The information processing apparatus according to item 1-1. According to this information processing apparatus, the accuracy of estimating the position of the user can be improved. [Item 1-3] The map generation unit derives a score regarding the ratio of the surrounding space of the user covered by a plurality of input camera images based on a plurality of criteria obtained by dividing the surrounding space of the user in different manners. When the score becomes equal to or more than a predetermined threshold value, the map generation process is terminated. The information processing apparatus according to item 1-2. According to this information processing apparatus, the coverage of the map for estimating the position of the user can be improved. [Item 1-4] A step of automatically detecting a playable area in which a user wearing the head-mounted display can move during play of an application based on a camera image reflecting the surrounding space of the user captured by a camera of the head-mounted display; A step of causing the head-mounted display to display an image indicating the automatically detected playable area; A step of receiving an operation of a user for editing the playable area and changing the shape of the playable area according to the operation; An information processing method executed by a computer. According to this information processing method, the computer can be made to assist the user in efficiently setting a playable area having an arbitrary shape. [Item 1-5] A function of automatically detecting a playable area in which a user wearing the head-mounted display can move during play of an application based on a camera image reflecting the surrounding space of the user captured by a camera of the head-mounted display; A function of causing the head-mounted display to display an image indicating the automatically detected playable area; A function that receives an operation of a user who edits the play area and changes the shape of the play area according to the operation, A computer program for realizing the above on a computer. According to this computer program, the computer can be assisted so that the user can efficiently set a play area with an arbitrary shape.

[0102] [Item 2-1] A storage unit that stores a play area in which a user wearing a head-mounted display can move during the play of an application in the surrounding space of the user, A display control unit that causes the head-mounted display to display an image indicating the play area stored in the storage unit, A play area editing unit that receives an operation of a user who edits the play area and reduces the play area according to the operation of the user, An information processing apparatus comprising: According to this information processing apparatus, it can assist the user in setting a play area with a desired shape. [Item 2-2] Further comprising a play area detection unit that automatically detects the play area based on a camera image showing the surrounding space of the user captured by the camera of the head-mounted display, The play area editing unit reduces the play area automatically detected by the play area detection unit according to the operation of the user. The information processing apparatus according to Item 2-1. According to this information processing apparatus, it can assist the user in efficiently setting a play area with a desired shape. [Item 2-3] When the user draws a line that enters the play area from outside the play area and exits the play area, the play area editing unit deletes one of the play areas divided by the line. The information processing apparatus according to Item 2-1 or 2-2. According to this information processing apparatus, a user interface that can delete unnecessary play areas by intuitive operations can be provided. [Item 2-4] When one of the play areas divided by the line is an area where the user exists and the other is an area where the user does not exist, the play area editing unit deletes the area where the user does not exist. The information processing apparatus according to Item 2-3. According to this information processing apparatus, a user interface that can delete unnecessary play areas by intuitive operations can be provided. [Item 2-5] When the user draws a line that enters the play area from outside the play area and ends inside the play area, the play area editing unit deletes the area along the line from the play area. The information processing apparatus according to Item 2-3 or 2-4. According to this information processing apparatus, a user interface that can delete unnecessary play areas by intuitive operations can be provided. [Item 2-6] When the user draws a line that enters the play area from outside the play area and ends inside the play area, the play area editing unit maintains the shape of the play area until then. The information processing apparatus according to Item 2-3 or 2-4. According to this information processing apparatus, it is possible to prevent the play area from being deleted against the user's intention. [Item 2-7] When the user draws a line that exits the play area from inside the play area and enters the play area, the play area editing unit adds the area surrounded by the line to the play area. The information processing apparatus according to Item 2-3. According to this information processing apparatus, a user interface that can expand the play area by intuitive operations can be provided. [Item 2-8] When the user draws a line that starts from within the play area, exits the play area, and ends outside the play area, the play area editing unit adds the area along the line to the play area. The information processing apparatus according to item 2-7. According to this information processing apparatus, a user interface that can expand the play area by an intuitive operation can be provided. [Item 2-9] When the user draws a line that starts from within the play area, exits the play area, and ends outside the play area, the play area editing unit maintains the shape of the play area up to that point. The information processing apparatus according to item 2-7. According to this information processing apparatus, it is possible to prevent the play area from being expanded against the user's intention. [Item 2-10] A computer including a storage unit that stores a play area in which a user wearing a head-mounted display can move during the play of an application in the surrounding space of the user. A step of causing the head-mounted display to display an image indicating the play area stored in the storage unit. A step of receiving an operation of the user for editing the play area and reducing the play area according to the operation of the user. An information processing method for executing. According to this information processing method, the computer can be assisted so that the user can set a play area having a desired shape. [Item 2-11] In a computer including a storage unit that stores a play area in which a user wearing a head-mounted display can move during the play of an application in the surrounding space of the user. A function of causing the head-mounted display to display an image indicating the play area stored in the storage unit. A function of receiving an operation of the user for editing the play area and reducing the play area according to the operation of the user. A computer program for realizing. According to this computer program, the computer can be assisted so that the user can set a play area of a desired shape.

[0103] [Item 3-1] A play area setting unit that sets a play area in which the user can move during the play of the application in the surrounding space of the user wearing the head-mounted display, A determination unit that determines to operate the application in a mode in which the user can move or determines to operate the application in a mode in which the user cannot move according to the size of the play area set by the play area setting unit; An information processing apparatus comprising: According to this information processing apparatus, the safety of a user wearing a head-mounted display and playing an application can be improved. [Item 3-2] When the size of the play area is equal to or greater than a predetermined first threshold value, the determination unit determines to operate the application in a mode in which the user can move, and when the size of the play area is less than the first threshold value, the determination unit determines to operate the application in a mode in which the user cannot move. The information processing apparatus according to Item 3-1. According to this information processing apparatus, the safety of a user wearing a head-mounted display and playing an application can be improved. [Item 3-3] When the size of the play area set by the user is less than a second threshold value that is smaller than the first threshold value, the play area setting unit sets a temporary play area that is a play area in which the user cannot move and has a fixed shape instead of the play area set by the user. The information processing apparatus according to Item 3-2. According to this information processing apparatus, it is possible to secure a certain size of the play area and prevent the play area from being too small to impair the creation of the worldview by the image display on the head-mounted display. [Item 3-4] When it is determined by the determination unit that the application is to be operated in a mode in which the user can move, the application is notified that the user can move, and when it is determined by the determination unit that the application is to be operated in a mode in which the user cannot move, the application is further provided with a notification unit that notifies the application that the user cannot move. The information processing apparatus according to any one of Items 3-1 to 3-3. According to this information processing apparatus, the application can be operated in a mode that matches the size of the play area. [Item 3-5] When it is determined by the determination unit that the application is to be operated in a mode in which the user cannot move, the information processing apparatus further includes a display control unit that causes the head-mounted display to display that the user cannot move during the play of the application. The information processing apparatus according to any one of Items 3-1 to 3-4. According to this information processing apparatus, it is possible to make the user recognize that the user cannot move during the play of the application. [Item 3-6] A step of setting a play area in which the user can move during the play of the application in the surrounding space of the user wearing the head-mounted display, A step of determining whether to operate the application in a mode in which the user can move or to operate the application in a mode in which the user cannot move according to the size of the set play area, An information processing method executed by a computer. According to this information processing method, it is possible to improve the safety of a user wearing a head-mounted display and playing an application. [Item 3-7] A function for setting a playable area in which a user wearing a head-mounted display can move during play of an application in the surrounding space of the user, A function for determining whether to operate the application in a mode in which the user can move or in a mode in which the user cannot move according to the size of the set playable area, A computer program for causing a computer to implement. According to this computer program, the safety of a user wearing a head-mounted display and playing an application can be enhanced.

Explanation of Signs

[0104] 10 Image display system, 100 Head-mounted display, 200 Image generation device, 256 Playable area storage unit, 264 Playable area setting unit, 266 Playable area detection unit, 270 Playable area editing unit, 272 Temporary setting unit, 274 Map generation unit, 276 Mode determination unit, 292 Display control unit.

Claims

1. A play area detection unit that automatically detects a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; A display control unit that causes the head-mounted display to display an image indicating the play area automatically detected by the play area detection unit; A play area editing unit that receives an operation of the user for editing the play area and changes the shape of the play area according to the operation; A map generation unit that generates a map for estimating the position of the user based on the camera image in parallel with the play area detection unit detecting the play area; Comprising: When the map generation unit receives a plurality of camera images showing the surrounding space of the user from a plurality of directions and the number of input camera images is equal to or more than a predetermined number, the map generation unit ends the map generation process. An information processing apparatus.

2. A play area detection unit that automatically detects a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; A display control unit that causes the head-mounted display to display an image indicating the play area automatically detected by the play area detection unit; A play area editing unit that receives an operation of the user for editing the play area and changes the shape of the play area according to the operation; A map generation unit that generates a map for estimating the position of the user based on the camera image in parallel with the play area detection unit detecting the play area; Comprising: The map generation unit derives a score regarding the ratio of the surrounding space of the user covered by a plurality of input camera images based on a plurality of criteria obtained by dividing the surrounding space of the user in different manners, and when the score becomes equal to or more than a predetermined threshold value, the map generation unit ends the map generation process. An information processing apparatus.

3. A step of automatically detecting a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; In parallel with detecting the play area, generating a map for estimating the position of the user based on the camera image; Displaying an image indicating the automatically detected play area on the head-mounted display; Receiving an operation of the user for editing the play area and changing the shape of the play area according to the operation; which is executed by a computer; In the generating step, when a plurality of camera images obtained by reflecting the surrounding space of the user from a plurality of directions are input and the number of input camera images is equal to or more than a predetermined number, the map generation process is terminated. An information processing method.

4. Automatically detecting a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; In parallel with detecting the play area, generating a map for estimating the position of the user based on the camera image; Displaying an image indicating the automatically detected play area on the head-mounted display; Receiving an operation of the user for editing the play area and changing the shape of the play area according to the operation; which is executed by a computer; In the generating step, based on a plurality of criteria obtained by dividing the surrounding space of the user in different manners, a score regarding the ratio of the surrounding space of the user covered by a plurality of input camera images is derived, and when the score becomes equal to or more than a predetermined threshold value, the map generation process is terminated. An information processing method.

5. A detection function for automatically detecting a play area in which a user wearing the head-mounted display can move during play of an application based on a camera image showing the surrounding space of the user captured by a camera of the head-mounted display; A function of displaying an image indicating the automatically detected play area on the head-mounted display; A function of receiving an operation of the user for editing the play area and changing the shape of the play area according to the operation; A generation function for generating a map for estimating the position of the user based on the camera image in parallel with the detection function detecting the play area; which is realized by a computer. The generation function is a plurality of camera images that reflect the surrounding space of the user from a plurality of directions. When the number of input camera images is equal to or greater than a predetermined number, the generation process of the map is terminated. A computer program. **Claim 6** A detection function that automatically detects a playable area in which a user wearing the head-mounted display can move during play of an application, based on camera images that reflect the surrounding space of the user captured by a camera of the head-mounted display; A function that causes the head-mounted display to display an image indicating the automatically detected playable area; A function that receives an operation by the user to edit the playable area and changes the shape of the playable area according to the operation; A generation function that generates a map for estimating the position of the user based on the camera images, in parallel with the detection function detecting the playable area; Implemented on a computer, The generation function derives a score regarding the ratio of the surrounding space of the user covered by a plurality of input camera images among the surrounding space of the user, based on a plurality of criteria that divide the surrounding space of the user in different manners. When the score becomes equal to or greater than a predetermined threshold value, the generation process of the map is terminated. A computer program.

Images