Information processing apparatus, information processing method, and computer program

Abstract

The image display system 10 includes a head-mounted display 100 and an image generation device 200. The image generation device 200 sets a game area in a space around a user who wears the head-mounted display 100, in which the user can move while playing an application. The image generation device 200 determines to operate the application in a mode that allows movement of the user according to the size of the set game area or determines to operate the application in a mode that does not allow movement of the user.

Description

Technical Field

[0001] This invention relates to a data processing technology, and more specifically to an information processing device, an information processing method, and a computer program. Background Technology

[0002] Image display systems that allow users wearing head-mounted displays to freely enjoy a target space from any viewpoint are widely used. For example, there are known electronic content that designates a virtual three-dimensional (3D) space as the display target and displays an image on a head-mounted display corresponding to the user's line of sight to achieve virtual reality (VR). The use of head-mounted displays can increase the feeling of immersion in video and improve the operability of applications such as games. A further roaming system has been developed that allows users wearing head-mounted displays to virtually walk around a space displayed as video by physical movement while the head-mounted display is attached. Summary of the Invention

[0003] [The problem the invention aims to solve]

[0004] The range of movement a user wearing a head-mounted display can make while playing an application needs to be limited based on the size of the user's room and the environment in the user's room (such as furniture and items).

[0005] This invention was developed in consideration of the aforementioned problems. The object of this invention is to provide a technique for determining an appropriate operating mode for an application used to generate VR images based on the size of the set game area.

[0006] [Solution to the problem]

[0007] To address the aforementioned problems, an information processing device according to one aspect of the present invention includes: a game area setting unit that sets a game area in the space surrounding a user wearing a head-mounted display, wherein the user can move within the game area while playing an application; and a determining unit that determines, based on the size of the game area set by the game area setting unit, whether to operate the application in a user-movable mode or to operate the application in a user-immovable mode.

[0008] Another aspect of the invention relates to an information processing method. This method is executed by a computer and includes the steps of setting up a game area in the space surrounding a user wearing a head-mounted display, in which the user can move while playing an application, and determining, based on the size of the set game area, whether to operate the application in a user-movable mode or a user-immovable mode.

[0009] Note that any combination of the above-mentioned constituent elements, as well as expressions of the present invention that are converted into systems, computer programs, recording media that readablely record computer programs, data structures, etc., are also valid as embodiments of the present invention.

[0010] [Beneficial effects of the invention]

[0011] This invention enables the determination of an appropriate operating mode for the application used to generate VR images based on the size of the set game area. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating an example of the appearance of a head-mounted display.

[0013] Figure 2 This is a diagram illustrating a configuration example of an image display system according to an embodiment.

[0014] Figure 3 This is a diagram used to illustrate an example of the world of images displayed on a head-mounted display by an image-generating device.

[0015] Figure 4 It is a diagram describing the internal circuit configuration of an image generation device.

[0016] Figure 5 This is a diagram depicting the internal circuitry of a head-mounted display.

[0017] Figure 6 It is a block diagram describing the functional blocks of an image generation device.

[0018] Figure 7 This is a flowchart illustrating the operation of an image generation device.

[0019] Figure 8 This is a diagram that schematically illustrates user behavior when a game area is set up.

[0020] Figure 9 This is a diagram illustrating an example of a camera image.

[0021] Figure 10 This is a diagram illustrating an example of a keyframe.

[0022] Figure 11 It is a diagram depicting multiple boxes.

[0023] Figure 12 This is an example image depicting the game area editing screen.

[0024] Figure 13 (a) to Figure 13 (d) is a diagram illustrating an example of editing used to expand the game area.

[0025] Figure 14 (a) to Figure 14 (d) is a diagram describing the details of the process used to set the game area in the game area editing screen.

[0026] Figure 15 (a) to Figure 15 (e) is a diagram describing the details of the processes used to expand the game area.

[0027] Figure 16 (a) to Figure 16 (d) is a diagram illustrating an example of editing used to reduce the game area.

[0028] Figure 17 (a) to Figure 17 (d) is a diagram describing the details of the processing used to reduce the game area.

[0029] Figure 18 This is a flowchart illustrating the operation of an image generation device.

[0030] Figure 19 (a) to Figure 19 (d) is a diagram describing the modifications made by the editor to expand the game area.

[0031] Figure 20 (a) to Figure 20 (b) shows the modifications made to the editor to reduce the game area. Detailed Implementation

[0032] This embodiment relates to an image display system that displays images of an application on a head-mounted display attached to a user's head. Figure 1 An example of the appearance of a head-mounted display 100 is described. The head-mounted display 100 according to an embodiment includes an output mechanism unit 102 and an attachment mechanism unit 104. The attachment mechanism unit 104 includes an attachment strap 106 that is attached around the user's head to secure the device when worn by the user.

[0033] The output mechanism unit 102 includes a housing 108, which is shaped to cover the user's left and right eyes when the head-mounted display 100 is attached to the user, and to house the display panel in an eye-facing position when attached. It is assumed that the display panel of the head-mounted display 100 according to the embodiment is opaque. In other words, the head-mounted display 100 according to the embodiment is an opaque head-mounted display.

[0034] The housing 108 may also house an eye lens so that it is positioned between the display panel and the user's eyes when the head-mounted display 100 is attached, thereby expanding the user's field of vision. The head-mounted display 100 may also include speakers and headphones at positions corresponding to the user's ears when attached. Furthermore, the head-mounted display 100 has built-in motion sensors to detect translational or rotational movements of the user's head, as well as the position and posture of the head at each step.

[0035] Furthermore, the head-mounted display 100 includes a stereo camera 110 on the front surface of the housing 108. The stereo camera 110 captures video of the surrounding real space within the user's field of vision, corresponding to their line of sight. Content commonly referred to as video perspective (which presents the state of real space in the direction the user is facing as is) can be achieved through the instantaneous display of the captured images. Additionally, augmented reality (AR) can be achieved by drawing images of virtual objects on top of images of real objects contained within the captured images.

[0036] Figure 2 An example configuration of an image display system 10 according to an embodiment is described. The image display system 10 includes a head-mounted display 100, an image generating device 200, and a controller 140. The head-mounted display 100 is wirelessly connected to the image generating device 200. The image generating device 200 can also be connected to a server via a network. In this case, the server can provide data from online applications (such as games that allow multiple users to participate via the network) to the image generating device 200.

[0037] The image generation device 200 is an information processing device that, with reference to the position and posture of the user's head wearing the head-mounted display 100, specifies the position of the viewpoint and the direction of the gaze, forms a display image corresponding to the specified position and direction of the field of view, and outputs the display image to the head-mounted display 100. For example, the image generation device 200 can form a display image of a virtual world corresponding to the stage of a video game while advancing the game, or display motion images for viewing or to provide information, regardless of whether the motion images are associated with the virtual world or the real world. Furthermore, the image generation device 200 can give the user a sense of deep immersion in the displayed world by displaying a panoramic image with a wide viewing angle around the user's viewpoint on the head-mounted display 100. Note that the image generation device 200 can be a stationary game console or a PC (personal computer).

[0038] The controller 140 is a controller (e.g., a game controller) held by a user and receiving input from operations performed by the user to control image generation by the image generation device 200 or image display by the head-mounted display 100. The controller 140 is connected to the image generation device 200 wirelessly. In a variant, either or both of the head-mounted display 100 and the controller 140 may be connected to the image generation device 200 via wired communication using a signal cable or the like.

[0039] Figure 3 This diagram illustrates an example of an image world displayed by a head-mounted display 100 under the operation of an image generation device 200. In this example, a state is created where the user 12 exists in a room corresponding to a virtual space. As shown, objects such as walls, floors, windows, tables, and items on the tables are arranged in a world coordinate system that defines the virtual space. The image generation device 200 defines a viewing screen 14 in this world coordinate system according to the viewpoint's position and the direction of the user 12's gaze, and renders images of the objects on the viewing screen 14 to draw and display an image.

[0040] The image generation device 200 acquires the position and direction of the user 12's viewpoint (hereinafter, these positions and directions will be collectively referred to as "viewpoint") from the head-mounted display 100 at a predetermined rate, and changes the position and direction of the viewing screen 14 according to the acquired position and direction. In this way, the head-mounted display 100 is allowed to display images within the field of view corresponding to the user's viewpoint. Furthermore, the image generation device 200 is also capable of presenting a three-dimensional view of the virtual space to the user 12 by forming stereoscopic images with parallax and displaying the stereoscopic images in the left and right areas of the display panel of the head-mounted display 100. In this way, the user 12 is allowed to experience virtual reality, where the user 12 feels as if he or she exists in a room of the displayed world.

[0041] Figure 4 The internal circuit configuration of the image generation device 200 is described. The image generation device 200 includes a CPU (Central Processing Unit) 222, a GPU (Graphics Processing Unit) 224, and a main memory 226. These units are interconnected via a bus 230. An input / output interface 228 is also connected to the bus 230. A communication unit 232, a storage unit 234, an output unit 236, an input unit 238, and a recording medium driving unit 240 are connected to the input / output interface 228.

[0042] Communication unit 232 includes peripheral interfaces such as USB (Universal Serial Bus) and IEEE (Institute of Electrical and Electronics Engineers) 1394, and network interfaces such as wired LAN (Local Area Network) and wireless LAN. Storage unit 234 includes hard disk drives, non-volatile memory, etc. Output unit 236 outputs data to head-mounted display 100. Input unit 238 receives data input from head-mounted display 100 and from controller 140. Recording medium driving unit 240 drives removable recording media such as magnetic disks, optical disks, or semiconductor memory.

[0043] The CPU 222 executes the operating system stored in the storage unit 234 to control the overall image generation device 200. Furthermore, the CPU 222 executes various programs (e.g., VR game applications) read from the storage unit 234 or a removable recording medium and loaded into the main memory 226, or downloaded via the communication unit 232. The GPU 224, which has geometry engine capabilities and a rendering processor, performs image rendering processing according to image rendering commands issued by the CPU 222 and outputs the rendering results to the output unit 236. The main memory 226 includes RAM (Random Access Memory) and stores the programs and data required for processing.

[0044] Figure 5 The internal circuitry of the head-mounted display 100 is described. The head-mounted display 100 includes a CPU 120, main memory 122, a display unit 124, and an audio output unit 126. These units are interconnected via a bus 128. An input / output interface 130 is also connected to the bus 128. A communication unit 132, including a wireless communication interface, a motion sensor 134, and a stereo camera 110 are connected to the input / output interface 130.

[0045] The CPU 120 processes information acquired from various units of the head-mounted display 100 via the bus 128, and provides display images and audio data acquired from the image generation device 200 to the display unit 124 and the audio output unit 126. The main memory 122 stores the programs and data required for processing by the CPU 120.

[0046] Display unit 124 includes a display panel such as a liquid crystal panel and an organic EL (electroluminescent) panel, and displays an image in front of the eyes of a user wearing head-mounted display 100. Display unit 124 can provide a three-dimensional view by displaying a pair of stereoscopic images in areas corresponding to the left and right eyes. Display unit 124 may also include a pair of lenses located between the display panel and the eyes of the user wearing head-mounted display 100 to expand the user's field of view.

[0047] The audio output unit 126 includes speakers and headphones positioned corresponding to the ears of a user wearing the head-mounted display 100 to allow the user to hear sound. The communication unit 132 is an interface for sending and receiving data to and from the image generation device 200, and communication is achieved via a known wireless communication technology such as Bluetooth (registered trademark). The motion sensor 134 includes a gyroscope sensor and an accelerometer, and acquires the angular velocity and acceleration of the head-mounted display 100.

[0048] like Figure 1 As shown, the stereo camera 110 includes a pair of video cameras for capturing images of the surrounding real space in the field of view corresponding to the user's viewpoint, respectively, from the left and right viewpoints. The images captured by the stereo camera 110 and indicating the space around the user will also be referred to hereinafter as "camera images," and are also considered to be images containing objects present in the user's line of sight (typically in front of the user). Measurements obtained by the motion sensor 134 and data regarding the captured images (camera images) obtained by the stereo camera 110 are transmitted to the image generation device 200 via the communication unit 132 as needed.

[0049] The image display system 10 according to an embodiment sets up a game area, which defines the range of the real world that a user wearing the head-mounted display 100 can move around in while playing an application. The game area is considered to be the area or range in which the user is allowed to move around while viewing and listening to virtual reality images (hereinafter also referred to as "VR images") in the space surrounding the user (the space of the real world extending around the user). If the user attempts to leave the game area or has already left the game area while playing the application, the image display system 10 issues a warning to the user to draw their attention or urge them to return to the game area.

[0050] The aforementioned application is assumed to be a game application that displays VR images on a head-mounted display 100, and will also be referred to as a "VR game" below. For example, a VR game could be a tennis game that displays VR images representing a tennis court in a virtual world and changes the character's position on the tennis court in the virtual world in association with the user's movement (e.g., walking) in the real world.

[0051] A first feature of the image display system 10 according to an embodiment will be described. In a first step, the image generating device 200 automatically detects a movable game area while a user wearing the head-mounted display 100 is playing an application. In a second step following the first step, the image generating device 200 receives an operation performed by the user to edit the automatically detected game area, and changes the shape of the game area according to the operation. In this way, the image generating device 200 helps the user effectively set a game area with a desired shape.

[0052] A second feature of the image display system 10 according to an embodiment will be described. The image generating device 200 reduces the game area based on operations performed by the user when the user manually edits the game area. For example, the image generating device 200 provides a user interface that is operated for manual editing of the game area and is also configured to receive the designation of areas to be excluded from the game area. The image generating device 200 removes the user-designated area from the game area via the user interface. In this way, the image generating device 200 helps the user efficiently set the desired game area.

[0053] A third feature of the image display system 10 according to an embodiment will be described. The image generating device 200 automatically switches the operation mode of the application that forms the VR image considering the set game area (the VR game according to the embodiment) based on the size of the game area. Specifically, the image generating device 200 determines whether to operate the application in a user-movable mode or a user-immobile mode based on the size of the set game area. In this way, the operation mode of the application can be automatically switched according to the size of the game area.

[0054] Figure 6 This is a block diagram describing the functional blocks of the image generation device. As described above, the image generation device 200 performs general information processing, such as advancing VR games and communicating with servers. Figure 6 The details of the functional blocks associated with the game area settings are described in particular. Note that... Figure 6 At least some of the functions of the image generating device 200 described herein can be installed on a server connected to the image generating device 200 via a network.

[0055] in addition, Figure 6 The multiple functional blocks described in the document can be accessed through, for example... Figure 4 The configuration of CPU 222, GPU 224, main memory 226, and storage unit 234 described herein is implemented in hardware and can also be implemented in software by a computer program combining the functions of multiple functional blocks. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various forms, such as hardware only, software only, and a combination of both. Thus, these functional blocks are not intended to be limited to any particular form.

[0056] The image generation apparatus 200 includes a data processing unit 250 and a data storage unit 252. The data processing unit 250 performs various types of data processing. The data processing unit 250 via... Figure 4The communication unit 232, output unit 236, and input unit 238 described herein send data to the head-mounted display 100 and the controller 140, and receive data from the head-mounted display 100 and the controller 140. The data storage unit 252 stores data referenced or updated by the data processing unit 250.

[0057] The data storage unit 252 includes an App storage unit 254, a game area storage unit 256, and a map storage unit 258. The App storage unit 254 stores data of the application (VR game according to the embodiment) that forms VR images.

[0058] The game area storage unit 256 stores data associated with the game area. The data associated with the game area includes data indicating the location of the point cloud that constitutes the boundary of the game area (e.g., the coordinate values ​​of the corresponding points in the world coordinate system).

[0059] Map storage unit 258 stores map data for estimating the position of head-mounted display 100 (i.e., the position of the user wearing head-mounted display 100). According to an embodiment, the map data corresponds to images (feature points contained in the images), each image indicating the room where the user is playing a VR game, and includes a set of images (feature points contained in the images), the positional relationships (position and orientation) of each image being elucidated. Specifically, the map data includes multiple sets of positions of head-mounted display 100, gaze directions, and keyframes associated with each other. Note that the map data may include other items commonly referred to as self-position estimation.

[0060] Keyframes included in a specific set correspond to data indicating features of an image viewed from a location within the same set and in the direction of the line of sight. A keyframe is an image generated using images captured by the stereoscopic camera 110 of the head-mounted display 100 (hereinafter also referred to as a "camera image"), and is also an image containing a predetermined number or more feature points. The number of feature points to be included in the keyframe can be 24 or more. Feature points may include corners detected by known corner detection methods and may be detected with reference to the slope of brightness. A keyframe is also considered as a set of partial images cut from the camera image.

[0061] The data processing unit 250 includes a system unit 260, an application execution unit 290, and a display control unit 292. The functions of these multiple functional blocks can be incorporated into a computer program. The CPU 222 and GPU 224 of the image generation device 200 can read the aforementioned computer program from the storage unit 234 or the recording medium into the main memory 226 and execute the computer program to perform the functions of the aforementioned multiple functional blocks.

[0062] The App execution unit 290 reads data of an application (a VR game according to an embodiment) selected by the user from the App storage unit 254 and executes the application selected by the user. The display control unit 292 sends data of various images (e.g., VR images and AR images) formed by the system unit 260 and the App execution unit 290 to the head-mounted display 100 and causes the display unit 124 (display panel) of the head-mounted display 100 to display these images.

[0063] System unit 260 performs system processing associated with head-mounted display 100. System unit 260 provides common services for multiple applications of head-mounted display 100 (e.g., VR games). System unit 260 includes camera image acquisition unit 262, game area setting unit 264, map generation unit 274, mode determination unit 276, notification unit 278, position estimation unit 280, and warning processing unit 282.

[0064] The camera image acquisition unit 262 acquires camera images sent from the head-mounted display 100 and captured by the stereo camera 110 of the head-mounted display 100.

[0065] The game area setting unit 264, which performs various processes to assist in setting up the game area, sets the game area by referring to camera images acquired by the camera image acquisition unit 262 and operations input by the user via the controller 140. The game area setting unit 264 includes a game area detection unit 266, a floor surface setting unit 268, a game area editing unit 270, and a temporary setting unit 272.

[0066] The game area detection unit 266 automatically detects the game area from the space surrounding the user wearing the head-mounted display 100, referring to the camera image acquired by the camera image acquisition unit 262. The floor surface setting unit 268 receives an operation performed by the user to edit the height of the floor surface detected as the game area by the game area detection unit 266, and changes the height of the floor surface according to the operation.

[0067] The game area editing unit 270 receives an operation performed by the user to edit the game area automatically detected by the game area detection unit 266, and changes the shape of the game area according to the operation. For example, the game area editing unit 270 reduces the game area automatically detected by the game area detection unit 266 according to the operation received from the user.

[0068] If the game area set by the user has a size smaller than a predetermined threshold, the temporary setting unit 272 sets a temporarily effective temporary game area instead of the game area set by the user (i.e., the game area available after being manually edited by the user). The temporary game area is a game area that the user cannot move (in other words, a game area that the user is prohibited from moving), and it is also a game area with a fixed shape.

[0069] While the game area detection unit 266 detects the game area, the map generation unit 274 generates a map to estimate the user's position by referring to camera images acquired by the camera image acquisition unit 262. The map generation unit 274 stores the generated map data in the map storage unit 258. When the input is a predetermined or greater number of camera images indicating the space around the user and captured in multiple directions, the map generation unit 274 terminates map generation.

[0070] The mode determination unit 276 determines whether to operate the VR game in a user-movable mode or a user-immovable mode based on the size of the game area set by the game area setting unit 264 (the game area available after manual editing by the user according to the embodiment). The mode determination unit 276 determines to operate the VR game in a user-movable mode if the size of the game area is a predetermined threshold or larger, or determines to operate the VR game in a user-immovable mode if the size of the game area is smaller than the threshold.

[0071] The notification unit 278 notifies the App execution unit 290 of the information required for the App execution unit 290 to execute the VR game. For example, if the mode determination unit 276 determines that the VR game is operated in a user-movable mode, the notification unit 278 sends data indicating user mobility to the VR game (the App execution unit 290 executes the VR game according to the embodiment). Conversely, if the mode determination unit 276 determines that the VR game is operated in a user-non-movable mode, the notification unit 278 sends data indicating user non-movability to the VR game (the App execution unit 290 executes the VR game according to the embodiment).

[0072] The position estimation unit 280 estimates the user's position in the real world, i.e., the user's position within the game area, by referring to map data stored in the map storage unit 258 and camera images acquired by the camera image acquisition unit 262. For example, the position estimation unit 280 may use camera images to examine multiple keyframes included in the map data and estimate the user's position by referring to the examination results, the position of the head-mounted display 100, and the gaze direction associated with the corresponding keyframes. Alternatively, the position estimation unit 280 may estimate the user's position by using a known self-position estimation technique such as SLAM (Simultaneous Localization and Mapping). The warning processing unit 282 performs warning processing for the user based on the relationship between the boundaries of the game area and the user's position.

[0073] The operation of the image display system 10 configured as described above will be described.

[0074] Figure 7 This is a flowchart illustrating the operation of the image generation device 200. The diagram shows the operation of the image generation device 200 performed to set the game area. The user is allowed to select default settings or reset the game area from the system settings menu of the head-mounted display 100. When the default settings or reset of the game area are selected, the game area setting unit 264 of the image generation device 200 causes the head-mounted display 100 to display a message prompting the user to look around using the display control unit 292.

[0075] Figure 8 The illustration depicts user actions performed to set up a game area. The user attaches a head-mounted display 100 to his or her head and moves around the room 30 while looking around. Figure 8 Each user 32 described indicates the user's current position. Each of the arrows 34 indicates the user's line of sight. Game area 36 indicates an example of the game area to be set. During the execution of S10 to S12 (game area detection processing and map generation processing) described below, the game area setting unit 264 of the image generation device 200 displays the camera image acquired by the camera image acquisition unit 262 on the display panel of the head-mounted display 100 by using the display control unit 292; in other words, it performs video perspective to present the state of the real space to the user in the direction the user is facing. In this way, user safety is increased.

[0076] The head-mounted display 100 sequentially sends data from multiple camera images to the image generation device 200. Each of the multiple data points from the multiple camera images includes sensor data generated during the formation of the corresponding camera image. The sensor data includes measurements obtained by the motion sensor 134, such as the angular velocity and acceleration of the head-mounted display 100. (See again...) Figure 7As described, the camera image acquisition unit 262 of the image generation device 200 acquires data of the camera image sent from the head-mounted display 100 (S10).

[0077] The game area detection unit 266 of the image generation device 200 automatically detects the game area in the space surrounding the user (S11) by referring to the camera image acquired in S10. Specifically, the game area detection unit 266 can estimate the 3D shape of the user's room using known methods by referring to the camera image and sensor data corresponding to the camera image. The game area detection unit 266 can detect a plane (typically the floor surface) perpendicular to the direction of gravity indicated by the sensor data by referring to the estimated 3D shape of the room, and detect a composite result of multiple detected planes having the same height as the game area. The game area detection unit 266 stores game area data, including the coordinate values ​​of the point cloud constituting the boundary of the game area, in the game area storage unit 256.

[0078] During the detection of the game area, the game area detection unit 266 detects the height of the floor surface corresponding to the game area. The height of the floor surface can be the distance between the floor surface and the head-mounted display 100 in the direction of gravity. The game area detection unit 266 stores data representing the height of the floor surface in the game area storage unit 256. When the head-mounted display 100 is located at the origin, the height of the floor surface can be, for example, -1 meter.

[0079] The map generation unit 274 of the image generation device 200 generates a map for estimating the user's location simultaneously with the processing in S11, referring to the camera images acquired in S10 (S12). Figure 9 Examples describing camera images, while Figure 10 An example describing a keyframe. Map generation unit 274 extracts multiple feature points 44 included in camera image 40 using known methods such as corner detection methods, and generates keyframes 42 indicating the extracted feature points 44. In the case of extracting 24 or more feature points 44 from a camera image 40, map generation unit 274 stores keyframes 42 including the 24 or more extracted feature points 44 in map storage unit 258, while associating keyframes 42 with the user's position and gaze direction specified by sensor data.

[0080] The map generation unit 274 stores multiple references (hereinafter referred to as "bins"), each reference being used to derive a score indicating the coverage of the user's surrounding space by an input camera image (a keyframe generated from the camera image according to an embodiment). The multiple bins are multiple references that divide the user's surrounding space in different ways.

[0081] Figure 11Several boxes are described. Each of boxes 50, 52, and 54 centers the user's position and divides the space around the user using an axis aligned with the direction of gravity. Box 50 is a reference point for dividing the space around the user into four zones, each with a central angle of 90 degrees, referencing the user's line of sight at the start of the game area setup and a direction perpendicular to the line of sight. Box 52 is a reference point shifted 45 degrees from box 50. Box 54 is a reference point for dividing the space around the user into 16 zones, each with a central angle of 22.5 degrees. The areas created by dividing the space around the user and specifying them by the corresponding boxes 50, 52, and 54 will be referred to hereinafter as "divided areas".

[0082] Map generation unit 274 derives scores from bins 50, 52, and 54 indicating the coverage of the space surrounding the user by multiple input camera images (keyframes generated from the camera images according to an embodiment). If the score becomes a predetermined threshold or higher, map generation unit 274 terminates the map generation process.

[0083] Specifically, when generating keyframes from camera images, map generation unit 274 assumes that for each of bins 50, 52, and 54, a segmented region consistent with the viewing direction of the already acquired keyframe is filled, and the total number of filled segmented regions is counted as the score for each bin. When generating new keyframes, map generation unit 274 adds the number of newly filled segmented regions to the score of each bin using a similar algorithm.

[0084] again Figure 7 As described, when the score is below the threshold "10" ("No" in S13), the process returns to S10 to continue the game area detection processing and map generation processing in the embodiment. If the score becomes the threshold "10" or higher ("Yes" in S13), the game area detection unit 266 ends the game area detection processing, and the map generation unit 274 ends the map generation processing. The score threshold "10" is determined to be a value reached when a user wearing the head-mounted display 100 looks 180 degrees in all directions, but not reached when the user does not look around.

[0085] After performing game area detection processing and map generation processing, the floor surface setting unit 268 of the image generation device 200 generates a floor surface adjustment screen by referencing data indicating the height of the floor surface and storing it in the game area storage unit 256. The floor surface adjustment screen may include an AR image formed by overlaying objects indicating the floor surface (e.g., translucent and mesh objects) onto a camera image acquired by the camera image acquisition unit 262. The floor surface setting unit 268 displays the floor surface adjustment screen on the display panel of the head-mounted display 100 using the display control unit 292. The floor surface setting unit 268 receives user input to the floor surface adjustment screen to adjust the height of the floor surface, and changes the height of the floor surface according to the user input. The floor surface setting unit 268 stores data indicating the changed height of the floor surface in the game area storage unit 256 (S14).

[0086] The game area editing unit 270 of the image generation device 200 forms an image representing the game area automatically detected by the game area detection unit 266. Specifically, the game area editing unit 270 generates a game area editing screen with reference to game area data stored in the game area storage unit 256. The game area editing screen includes an AR image formed by overlaying an object indicating the game area onto a camera image acquired by the camera image acquisition unit 262. The display control unit 292 of the image generation device 200 causes the display panel of the head-mounted display 100 to display the game area editing screen (S15).

[0087] Figure 12 An example of a game area editing screen is described. Game area editing screen 60 includes a game area 62 and a boundary 64. Game area 62 is an image indicating the game area (typically a floor surface without obstacles). For example, game area 62 could be an image indicating a semi-transparent, grid-like object. Boundary 64 is an image indicating the boundary of game area 62, and is also an image that perpendicularly crosses the game area at its boundary. For example, boundary 64 could also be a semi-transparent, grid-shaped object.

[0088] Refer again Figure 7 As described, the game area editing unit 270 receives user input on the game area editing screen 60 to edit the game area 62. The game area editing unit 270 changes the shape of the game area 62 according to the user input, that is, it expands or shrinks the game area 62 according to the user input. The game area editing unit 270 stores the data of the game area 62 that has undergone shape changes (e.g., the coordinate values ​​of the point cloud constituting the boundary) in the game area storage unit 256 to update the game area data stored in the game area storage unit 256 (S16).

[0089] Figure 13 (a) to Figure 13 (d) is viewed from above. Figure 12 The illustration shows the game area editing screen 60 and describes an example of editing for expanding the game area 62. The controller cursor 66 is an object that indicates the location specified by an operation input by a user wearing the head-mounted display 100 to the controller 140. For example, the controller 140 may function as a pointing device, and the game area editing unit 270 may display the controller cursor 66 in virtual space at the location specified by the controller 140.

[0090] like Figure 13 (a) to Figure 13 As shown in (c), the user moves the controller cursor 66 in the game area editing screen 60 to draw a line extending from the inside of the game area 62 to the outside of the game area 62 and back to the inside of the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, as... Figure 13 As shown in (d), the game area editing unit 270 adds the area surrounded by the drawn lines to the game area 62.

[0091] Here, it is assumed that the user draws a line extending from the inside of the game area 62 to the outside of the game area 62 and ending at the outside of the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, the game area editing unit 270 adds an area extending along the drawn line (i.e., the movement trajectory of the controller cursor 66) to the game area 62. For example, when the user... Figure 13 (b) If the movement of the controller cursor 66 ends at the position shown in the diagram (i.e., drawing a line), a narrow area extending to the right from the original game area 62 is added to the game area 62.

[0092] Figure 14 (a) to Figure 14 (d) shows details of the process for setting the game area in the game area editing screen 60 used for editing the game area. For example... Figure 14 As shown in (a), the game area editing unit 270 acquires data of the game area 62 automatically detected by the game area detection unit 266. For example... Figure 14 As shown in (b), the game area editing unit 270 converts the automatically detected game area 62 into a grid, that is, arranges the game area 62 on the grid.

[0093] like Figure 14 As shown in (c), the game area editing unit 270 fills the grid within the game area 62 with object 68. According to this example, an object 68 is placed at a grid intersection. Figure 14As shown in (d), the game area editing unit 270 extracts the boundary of the game area 62 by referring to the grid filled with objects 68, and establishes the boundary line of the game area 62.

[0094] Figure 15 (a) to Figure 15 (e) describes the details of the processes used to expand the game area for editing the game area. In this case, such as... Figure 15 As shown in (a), when the user presses the trigger of controller 140 (i.e., the input unit that provides the opportunity to display controller cursor 66), the position of controller cursor 66 is within the game area 62. The user moves controller cursor 66 outside the game area 62 while pressing the trigger, and further moves controller cursor 66 inside the game area 62 and releases the trigger. In this case, as... Figure 15 As shown in (b), the game area editing unit 270 fills the grid within the movement trajectory of the controller cursor 66 with objects 68 associated with the movement of the controller cursor 66.

[0095] like Figure 15 As shown in (c), the game area editing unit 270 extracts the boundary of the game area 62 with reference to the mesh filled with object 68. Figure 15 As shown in (d), when the user releases the trigger, the game area editing unit 270 establishes the boundary of the game area 62. In the case of multiple boundaries, the game area editing unit 270 selects to define the boundary with the larger range, i.e., the outer boundary. Figure 15 As shown in (e), the game area editing unit 270 fills the grid within the boundary with object 68.

[0096] Figure 16 (a) to Figure 16 (d) is viewed from above. Figure 12 An illustration of the game area editing screen 60 is provided, along with an example of editing for shrinking the game area 62. (Example:) Figure 16 (a) to Figure 16 As shown in (c), the user moves the controller cursor 66 in the game area editing screen 60 to draw a line extending from the outside of the game area 62 to the inside of the game area 62, leaving the game area 62 on the outside, as a line indicating the movement trajectory of the controller cursor 66. In this case, as... Figure 16 As shown in (d), the game area editing unit 270 deletes one of the parts of the game area 62 that are separated by the drawn lines.

[0097] According to an embodiment, when a user is present in one of the separate game areas, the game area editing unit 270 detects this state by referring to the user's position estimated by the position estimation unit 280. In one of the separate game areas (e.g., Figure 16(c) Game area 62a) and another (e.g., Figure 16 In cases where game area 62b in (c) corresponds to an area where the user exists and an area where the user does not exist, the game area editing unit 270 deletes the area where the user does not exist (e.g., game area 62b).

[0098] During modification, the game area editing unit 270 can retain relatively large separate game areas (e.g., game area 62a) and delete relatively small separate game areas (e.g., game area 62b). Furthermore, if the user exists in any of the separate game areas, the game area editing unit 270 can delete one or more separate game areas where the user does not exist. If the user does not exist in any of the separate game areas, the game area editing unit 270 can delete relatively small separate game areas.

[0099] Here, it is assumed that the user draws a line extending from the outside of the game area 62 to the inside of the game area 62 and ending inside the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, the game area editing unit 270 deletes the area extending along the drawn line (i.e., the movement trajectory of the controller cursor 66) from the game area 62. For example, when the user... Figure 16 (b) If the movement of controller cursor 66 ends at the position shown in the original game area 62, the area that controller cursor 66 has passed through is deleted from the original game area 62.

[0100] Figure 17 (a) to Figure 17 (d) describes the details of the processing used to reduce the game area for game area editing. In this case, such as Figure 17 As shown in (a), when the user presses the trigger of controller 140, the controller cursor 66 is located outside the game area 62. The user moves the controller cursor 66 inside the game area 62 while pressing the trigger, and further moves the controller cursor 66 outside the game area 62 and releases the trigger. In this case, as... Figure 17 As shown in (b), the game area editing unit 270 deletes objects 68 that fill the grid within the movement trajectory of the controller cursor 66 in association with the movement of the controller cursor 66.

[0101] like Figure 17 As shown in (c), the game area editing unit 270 extracts the boundary of the game area 62 with reference to the mesh filled with object 68. Figure 17 As shown in (d), when the user releases the trigger, the game area editing unit 270 only establishes the boundary of the game area 62 where the user exists, and deletes the game area 62 where the user does not exist.

[0102] Refer again Figure 7 The process up to S16 involves the following steps: Data of the game area set by the game area setting unit 264 (e.g., coordinate values ​​of the point cloud constituting the boundary of the game area) is stored in the game area storage unit 256. The game area set by the game area setting unit 264 is the game area set by the user and having a desired shape (hereinafter also referred to as the "user-set game area"). The pattern determination unit 276 of the image generation device 200 derives the size and shape of the user-set game area based on the data of the user-set game area.

[0103] If the size (including shape) of the user-defined game area is a predetermined first threshold or greater (as in S17), the mode determination unit 276 determines that the user-defined game area is a movable game area and determines to operate the VR game in a user-movable mode. According to an embodiment, if the user-defined game area contains a square with a vertical length of 2 meters and a horizontal length of 2 meters, the size of the user-defined game area is determined to be a first threshold or greater. The mode determination unit 276 stores data indicating that the user-defined game area is a movable game area in the game area storage unit 256 (S18).

[0104] If the size of the user-defined game area is less than a first threshold (N in S17) but equal to or greater than a second threshold (Y in S19), the mode determination unit 276 determines that the user-defined game area is a non-movable game area and determines that the VR game is operated in a non-movable user mode. According to an embodiment, if the user-defined game area contains a square with a vertical length of 1 meter and a horizontal length of 1 meter, the size of the user-defined game area is determined to be the second threshold or greater. The mode determination unit 276 stores data indicating that the user-defined game area is a non-movable game area in the game area storage unit 256 (S20).

[0105] If the size of the user-defined game area is less than a second threshold, that is, if the user-defined game area does not contain a square with a vertical length of 1 meter and a horizontal length of 1 meter ("No" in S19), the mode determination unit 276 determines to set a temporary game area. The temporary setting unit 272 causes the head-mounted display 100 to display a message indicating that a temporary setting area should be set, and sets the temporary game area at the location specified by the user via the controller 140. The temporary game area is a game area with a fixed shape (i.e., a fixed size) and corresponds to an area where user editing is prohibited. According to an embodiment, the temporary game area has a circular shape with a radius of 1.5 meters.

[0106] When a temporary game area is set up, the temporary setting unit 272 stores the data associated with the temporary game area in the game area storage unit 256, instead of the data associated with the user-set game area previously stored in the game area storage unit 256. For example, the data associated with the temporary game area includes the coordinate values ​​of the point cloud that constitutes the boundary of the temporary game area. In addition, the temporary setting unit 272 stores data indicating that the temporary game area is a non-movable game area in the game area storage unit 256 (S21).

[0107] Figure 18 This is also a flowchart illustrating the operation of the image generation device 200. The diagram shows the operation of the image generation device 200 during the execution of a VR game. When a user starts a VR game using the image generation device 200 (Yes in S30), the notification unit 278 sends data (such as data indicating the shape and size of the game area) stored in the game area storage unit 256 to the App execution unit 290 regarding the game area (user-set game area or temporary game area). Furthermore, the notification unit 278 sends data indicating whether the aforementioned game area is a movable or non-movable game area to the App execution unit 290 (S31).

[0108] Simultaneously with the processing in S31, the notification unit 278 notifies the user whether the user is mobile or immobile during VR game play via the display control unit 292 (S32). Specifically, when the mode determination unit 276 determines that the VR game is operated in a user-mobile mode, that is, when the data indicating that the game area is mobile is stored in the game area storage unit 256, the display control unit 292 causes the display panel of the head-mounted display 100 to display a message indicating that the user is mobile during VR game play.

[0109] On the other hand, when the mode determination unit 276 determines that the VR game is being operated in a user-immobile mode, that is, when data indicating that the game area is an immobile game area is stored in the game area storage unit 256, the display control unit 292 causes the display panel of the head-mounted display 100 to display a message indicating that the user cannot move while playing the VR game. For example, this message could be a warning message prohibiting movement while playing the VR game.

[0110] The App execution unit 290 of the image generation device 200 reads the program data of the VR game from the App storage unit 254 and starts the VR game (S33). For example, the App execution unit 290 can arrange game items within the game area (including temporary game area) in the virtual space that renders the game world to allow the user to access these items. The display control unit 292 causes the display panel of the head-mounted display 100 to display the image of the VR game (e.g., VR image) generated by the App execution unit 290.

[0111] The camera image acquisition unit 262 of the image generation device 200 sequentially acquires camera images sequentially transmitted from the head-mounted display 100, and the position estimation unit 280 sequentially estimates the user's position and gaze direction in the real world (S34). The notification unit 278 notifies the App execution unit 290 of the user's position and gaze direction estimated by the position estimation unit 280 (S35). The App execution unit 290 advances the VR game based on the thus estimated user position and gaze direction (S36). For example, when the user is in a movable game area, the App execution unit 290 can move the user character in the VR game according to changes in the user's position in the real world (i.e., in the game area).

[0112] When a user's location in the real world is close to the boundary of the game area, such as when the distance from the head-mounted display 100 attached to the user to the boundary of the game area is a predetermined threshold (e.g., 30 cm) or shorter ("Yes" in S37), the warning processing unit 282 of the image generation device 200 detects this state. The warning processing unit 282 performs predetermined warning processing on the user (S38).

[0113] For example, if the user's position is close to the boundary of the game area, the warning processing unit 282 can provide the display control unit 292 with an indication of the boundary of the game area (e.g., Figure 12The image of the boundary 64 described in the image is displayed. The display control unit 292 can cause the display panel of the head-mounted display 100 to display an image formed by superimposing an image indicating the boundary of the game area onto the game image formed by the App execution unit 290. Furthermore, if the user's position in the real world is close to or crosses the boundary of the game area, the warning processing unit 282 can cause the display panel of the head-mounted display 100 to display a video perspective image by using the display control unit 292. In addition, the warning processing unit 282 can first cause the display panel of the head-mounted display 100 to display an image indicating the boundary of the game area when the user's position in the real world is close to the boundary of the game area (e.g., 30 cm), and can cause the display panel of the head-mounted display 100 to display a video perspective image when the user's position in the real world moves further toward the boundary of the game area (e.g., 10 cm). Meanwhile, if the user's position in the real world is not close to the boundary of the game area ("No" in S37), the process skips S38.

[0114] If a predetermined termination condition is met, such as the user stopping the VR game (Yes in S39), the process in this diagram ends. If the termination condition is not met (No in S39), the process returns to S34 to continue the VR game. If the VR game has not started (No in S30), the processing in S31 and subsequent steps are skipped.

[0115] According to an embodiment, the image display system 10 automatically detects the game area by referring to camera images captured by the head-mounted display 100, and then allows the user to manually edit the game area. Furthermore, for manual editing of the game area, the image display system 10 provides a user interface (game area editing screen 60) that enables the expansion and contraction of the game area. In this way, the image display system 10 can help the user effectively set a game area with a desired shape.

[0116] Furthermore, according to the embodiment, the image display system 10 automatically determines whether to operate the VR game in a user-movable mode or a user-immobile mode based on the size of the set game area. This configuration, which can automatically switch the application's operation mode according to the size of the set game area, can reduce repeated attempts by the user to set the game area and improve the safety of users wearing the head-mounted display 100 and playing the application.

[0117] Furthermore, in some cases, applications that form VR images arrange various objects (e.g., game items) in the game area. Therefore, preferably, the game area has at least a certain size. In cases where the user-set game area has a small size (less than the second threshold mentioned above), the image display system 10 according to the embodiment sets up a temporary game area with a fixed shape. In this way, the game area has at least a certain size, and any impairment to the creation of the world view realized by the image display on the head-mounted display 100 (e.g., the excitement of the VR game) caused by an excessively small game area is avoided.

[0118] The invention has been described with reference to embodiments. These embodiments are presented by way of example. Those skilled in the art will understand that various modifications can be made to the combinations of the various components or processes, and these modifications are also included within the scope of the invention.

[0119] This section will explain the modifications associated with manual editing of the game area.

[0120] Figure 19 (a) to Figure 19 (d) shows the modifications made to the editor used to expand the game area. For example... Figure 19 (a) to Figure 19 As shown in (c), similar to the embodiment, the user moves the controller cursor 66 in the game area editing screen 60 to draw a line extending from the inside of the game area 62 to the outside of the game area 62 and back to the inside of the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, as... Figure 19 As shown in (d), the game area editing unit 270 adds the area surrounded by the drawn lines to the game area 62. Note that in the modification, the width of the line indicating the movement trajectory of the controller cursor 66 is smaller than the width of the line indicating the movement trajectory of the controller cursor 66 in the embodiment.

[0121] Here, it is assumed that the user draws a line extending from the inside of the game area 62 to the outside of the game area 62 and ending at the outside of the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, the game area editing unit 270 maintains the previous shape of the game area 62 in this modification. In other words, the game area editing unit 270 cancels the drawn line, that is, it does not expand the game area 62.

[0122] Figure 20 (a) and Figure 20 (b) shows the modifications made to the editor for reducing the game area. For example... Figure 20As shown in (a), similar to the embodiment, the user moves the controller cursor 66 in the game area editing screen 60 to draw a line extending from the outside of the game area 62 to the inside of the game area 62, leaving the game area 62 outside as a line indicating the movement trajectory of the controller cursor 66. In this case, as... Figure 20 As shown in (b), the game area editing unit 270 deletes one of the portions of the game area 62 separated by the drawn lines. In cases where one of the separated game areas corresponds to an area where the user exists and an area where the user does not exist, the game area editing unit 270 deletes the area where the user does not exist.

[0123] Here, it is assumed that the user draws a line extending from the outside of the game area 62 to the inside of the game area 62 and ending inside the game area 62, as a line indicating the movement trajectory of the controller cursor 66. In this case, the game area editing unit 270 maintains the previous shape of the game area 62 in this modification. In other words, the game area editing unit 270 cancels the drawn line, that is, it does not reduce the size of the game area 62.

[0124] Any combination of the above embodiments and modifications can also serve as a mode for implementing this disclosure. The new modes resulting from such combinations possess the respective advantageous effects provided by the embodiments and the modifications thus combined. Furthermore, those skilled in the art will understand that the functions performed by the respective constituent elements described in the claims are achieved by the respective constituent elements included individually or collaboratively in the embodiments and modifications.

[0125] The technical ideas described in the embodiments and modifications can be expressed in the following items.

[0126] [Item 1-1]

[0127] An information processing device, comprising:

[0128] The game area detection unit automatically detects the game area that a user wearing a head-mounted display can move around in while playing the application, with reference to camera images. The camera images indicate the space around the user and are captured by the head-mounted display's camera.

[0129] The display control unit causes the head-mounted display to show an image representing the game area automatically detected by the game area detection unit; and

[0130] The game area editing unit receives operations performed by the user to edit the game area and changes the shape of the game area according to the operation.

[0131] This information processing device can help users effectively set up game areas with the desired shape.

[0132] [Items 1-2]

[0133] The information processing equipment according to item 1-1 also includes:

[0134] The map generation unit, while the game area detection unit detects the game area, generates a map to estimate the user's position by referring to camera images.

[0135] When the input is a predetermined or larger number of camera images, which are used to indicate the space around the user and captured in multiple directions, the map generation unit ends the map generation process for generating a map.

[0136] This information processing device can improve the accuracy of user location estimation.

[0137] [Items 1-3]

[0138] According to the information processing device of item 1-2, the map generation unit refers to multiple benchmarks that divide the space around the user in different forms, derives a score indicating the coverage of the space around the user by multiple input camera images, and terminates the map generation process when the score becomes a predetermined threshold or higher.

[0139] This information processing device can increase the completeness of maps used to estimate user locations.

[0140] [Items 1-4]

[0141] An information processing method executed by a computer, the method comprising:

[0142] The steps for automatically detecting movable game areas by a user wearing a head-mounted display while playing the application refer to camera images, which indicate the space around the user and are captured by the head-mounted display's camera.

[0143] The steps of displaying an image representing an automatically detected game area on a head-mounted display; and

[0144] The steps to receive user-executed actions to edit the game area and change the shape of the game area accordingly.

[0145] This information processing method allows computers to help users effectively set up game areas with the desired shape.

[0146] [Items 1-5]

[0147] A computer program for enabling a computer to:

[0148] The feature automatically detects the game area that the user can move around in while playing the application, by referring to the space around the user wearing the head-mounted display and the camera images captured by the head-mounted display's camera.

[0149] The function of enabling the head-mounted display to show an image representing the automatically detected game area; and

[0150] This function receives user-executed actions to edit the game area and changes the shape of the game area accordingly.

[0151] This computer program allows the computer to help the user effectively set up a game area with the desired shape.

[0152] [Item 2-1]

[0153] An information processing device, comprising:

[0154] The storage unit is located in the gaming area within the space surrounding the user wearing the head-mounted display, where the user can move around while playing the application.

[0155] The display control unit causes the head-mounted display to show an image representing the game area stored in the storage unit; and

[0156] The game area editing unit receives operations performed by the user to edit the game area and reduces the game area according to the user's operations.

[0157] This information processing device can help users effectively set up game areas with the desired shape.

[0158] [Item 2-2]

[0159] The information processing equipment according to item 2-1 also includes:

[0160] The game area detection unit automatically detects the game area by referencing camera images. These camera images indicate the space around the user and are captured by the head-mounted display's camera.

[0161] The game area editing unit reduces the game area automatically detected by the game area detection unit based on the user's actions.

[0162] This information processing device can help users set up game areas with the desired shape.

[0163] [Items 2-3]

[0164] According to the information processing device of item 2-1 or 2-2, in the case where the user draws a line extending from the outside of the game area to the inside of the game area and leaving the game area on the outside, the game area editing unit deletes a portion of the game area separated by the line.

[0165] This information processing device can provide a user interface that allows users to delete unnecessary game areas through intuitive operation.

[0166] [Items 2-4]

[0167] According to the information processing device of item 2-3, in the case where one and the other of the line-separated portions of the game area correspond to the area where the user exists and the area where the user does not exist, respectively, the game area editing unit deletes the area where the user does not exist.

[0168] This information processing device can provide a user interface that allows users to delete unnecessary game areas through intuitive operation.

[0169] [Items 2-5]

[0170] According to the information processing device of item 2-3 or 2-4, in the case where the user draws a line extending from the outside of the game area to the inside of the game area and ending inside the game area, the game area editing unit deletes the area extending from the game area along the line.

[0171] This information processing device can provide a user interface that allows users to delete unnecessary game areas through intuitive operation.

[0172] [Items 2-6]

[0173] According to the information processing device of item 2-3 or 2-4, in the case where the user draws a line extending from the outside of the game area to the inside of the game area and ending inside the game area, the game area editing unit maintains the previous shape of the game area.

[0174] This information processing device can prevent the deletion of game areas against the user's will.

[0175] [Items 2-7]

[0176] According to the information processing device of item 2-3, when the user draws a line extending from inside the game area to outside the game area and back to inside the game area, the game area editing unit adds the area surrounded by the line to the game area.

[0177] This information processing device can provide a user interface that allows for intuitive operation and expansion of the game area.

[0178] [Items 2-8]

[0179] According to the information processing device of item 2-7, when the user draws a line extending from inside the game area to outside the game area and ending outside the game area, the game area editing unit adds the area extending along the line to the game area.

[0180] This information processing device can provide a user interface that allows for intuitive operation and expansion of the game area.

[0181] [Items 2-9]

[0182] According to the information processing device of item 2-7, when the user draws a line extending from inside the game area to outside the game area and ending outside the game area, the game area editing unit maintains the previous shape of the game area.

[0183] This information processing device can prevent the expansion of game areas that violate user intent.

[0184] [Item 2-10]

[0185] An information processing method executed by a computer, the computer including a storage unit storing information in a game area in the space surrounding a user wearing a head-mounted display, the game area being movable by the user while playing an application, the method comprising:

[0186] The steps of displaying an image representing a game area stored in a storage unit on a head-mounted display; and

[0187] Receive the user's command to edit the game area, and then shrink the game area based on the user's command.

[0188] This information processing method allows the computer to help the user set up a game area with the desired shape.

[0189] [Item 2-11]

[0190] A computer program for enabling a computer to include a storage unit containing a game area (where the user can move around while playing an application) stored in the space surrounding a user wearing a head-mounted display:

[0191] The ability to display images representing the game area stored in the storage unit on a head-mounted display; and

[0192] It has the function of receiving user-executed actions to edit the game area and shrinking the game area based on the user-executed actions.

[0193] This computer program allows the computer to help the user set up a game area with the desired shape.

[0194] [Item 3-1]

[0195] An information processing device, comprising:

[0196] A game area setting unit, wherein the game area setting unit sets a movable game area in the space surrounding the user while playing the application; and

[0197] The determining unit determines, based on the size of the game area set by the game area setting unit, whether to operate the application in a user-movable mode or in a user-unmovable mode.

[0198] This information processing device can increase the security of users who wear head-mounted displays and use applications.

[0199] [Item 3-2]

[0200] According to the information processing device of item 3-1, the determining unit determines to operate the application in a user-movable mode when the size of the game area is a predetermined first threshold or higher, and determines to operate the application in a user-non-movable mode when the size of the game area is less than the first threshold.

[0201] This information processing device can increase the security of users who wear head-mounted displays and use applications.

[0202] [Item 3-3]

[0203] According to the information processing device of item 3-2, when the game area set by the user has a size smaller than a second threshold, a temporary game area is set by the game area setting unit instead of the game area set by the user. The temporary game area is a game area that the user cannot move and has a fixed shape, and the second threshold is smaller than the first threshold.

[0204] This information processing device can ensure a game area of ​​a certain size and prevent damage to the creation of the world view displayed on the head-mounted display due to the game area being too small.

[0205] [Items 3-4]

[0206] The information processing equipment according to any one of items 3-1 to 3-3 further includes:

[0207] The notification unit notifies the application of the fact that the user is mobile when the determining unit determines that the application is operated in a user mobile mode, and notifies the application of the fact that the user is not mobile when the determining unit determines that the application is operated in a user non-mobile mode.

[0208] The information processing device can operate the application in a mode suitable for the size of the game area.

[0209] [Items 3-5]

[0210] The information processing device according to any one of items 3-1 to 3-4 further includes:

[0211] When the determining unit determines that the application is being operated in a user immobile mode, the display control unit causes the head-mounted display to show the fact that the user is immobile while playing the application.

[0212] This information processing device can notify users that they cannot move while playing the application.

[0213] [Items 3-6]

[0214] An information processing method executed by a computer, the method comprising:

[0215] The steps of setting up a gaming area in the space surrounding a user wearing a head-mounted display, where the user can move around while playing the application; and

[0216] Based on the size of the set game area, determine the steps to operate the application in either a user-movable mode or a user-immovable mode.

[0217] This information processing method can increase the security of users who wear head-mounted displays and use applications.

[0218] [Items 3-7]

[0219] A computer program for enabling a computer to:

[0220] The ability to create a gaming area in the space surrounding a user wearing a head-mounted display, where the user can move around while playing the application; and

[0221] The function of operating the application in either a user-movable mode or a user-immovable mode is determined based on the size of the set game area.

[0222] This computer program can increase the security of users who wear head-mounted displays and use applications.

[0223] [Industrial Applicability]

[0224] This invention applies to devices or systems that enable head-mounted displays to show images of applications.

[0225] [List of reference numerals]

[0226] 10: Image display system

[0227] 100: Head-mounted display

[0228] 200: Image generation device

[0229] 256: Game area storage unit

[0230] 264: Game Area Setting Unit

[0231] 266: Game Area Detection Unit

[0232] 270: Game Area Editing Unit

[0233] 272: Temporary setup unit

[0234] 274: Map Generation Unit

[0235] 276: Pattern Determination Unit

[0236] 292: Display control unit.

Claims

1. An information processing device, comprising: A game area setting unit sets up a game area in the space around a user wearing a head-mounted display, where the user can move while playing the application; as well as The determining unit determines, based on the size of the game area set by the game area setting unit, whether to operate the application in a user-movable mode or in a user-immovable mode. Wherein, if the game area set by the user has a size smaller than a first threshold, the game area setting unit sets a temporary game area to replace the game area set by the user. The temporary game area is a game area that the user cannot move and has a fixed shape.

2. The information processing device according to claim 1, wherein, The determining unit determines to operate the application in the user-movable mode if the size of the game area is a predetermined second threshold or greater, and determines to operate the application in the user-non-movable mode if the size of the game area is less than the second threshold, wherein the first threshold is less than the second threshold.

3. The information processing device according to claim 1, further comprising: The notification unit notifies the application that the user is mobile when the determining unit determines that the application is operated in the user mobile mode, and notifies the application that the user is not mobile when the determining unit determines that the application is operated in the user non-mobile mode.

4. The information processing device according to claim 1, further comprising: If the determining unit determines that the application is being operated in the user immobile mode, the display control unit causes the head-mounted display to show that the user is immobile while playing the application.

5. An information processing method executed by a computer, the method comprising: A gaming area is set up in the space around a user wearing a head-mounted display, where the user can move while playing the application; as well as Based on the size of the set game area, determine whether to operate the application in a user-movable mode or a user-immovable mode. Where the game area set by the user has a size smaller than a first threshold, a temporary game area is set to replace the game area set by the user. The temporary game area is a game area that the user cannot move and has a fixed shape.

6. A computer-readable storage medium comprising computer program instructions, which, when executed by one or more processors, cause the one or more processors to perform: A gaming area is set up in the space surrounding a user wearing a head-mounted display, where the user can move while playing the application; and Based on the size of the set game area, determine whether to operate the application in a user-movable mode or a user-immovable mode. in, If the game area set by the user has a size smaller than a first threshold, a temporary game area is set to replace the game area set by the user. The temporary game area is a game area that the user cannot move and has a fixed shape.

Images