Image display system, image display program, display control device, and image display method

Abstract

To provide an image display system capable of improving operability of an object placed in a virtual space.SOLUTION: An exemplary image display system includes a goggle device having a display unit. A virtual camera and a user interface is placed in a virtual space. The posture of the virtual camera in the virtual space is controlled according to the posture of the goggle device. When the goggle device is turned by a given angle or more in a pitch direction, the user interface is moved to the front of the virtual camera in a yaw direction.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] The present invention relates to an image display system, an image display program, a display control device, and an image display method capable of displaying an image.

Background Art

[0002] As a prior art, there is an image display system that arranges an object in a virtual space and displays the object by moving within the virtual space and directing the line of sight of a virtual camera toward the object (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, in order to display an object, for example, a user needs to perform an operation of directing the line of sight of a virtual camera toward the object, and there is room for improvement from the viewpoint of improving the operability of the object.

[0005] Therefore, an object of the present invention is to provide an image display system, an image display program, a display control device, and an image display method capable of improving the operability of an object arranged in a virtual space.

Means for Solving the Problems

[0006] In order to solve the above problems, the present invention employs the following configuration.

[0007] The image display system of the present invention is an image display system including a goggle device, and includes an object placement means, a display control means, an acquisition means, a camera rotation means, a determination means, and a control means. The object placement means places an object in a virtual space. The display control means causes an image captured by a virtual camera in the virtual space to be displayed on a display unit of the goggle device. The acquisition means acquires at least a rotation angle in the yaw direction and a rotation angle in the pitch direction of the goggle device. The camera rotation means rotates the virtual camera in the yaw direction of the virtual space according to the rotation angle in the yaw direction of the goggle device, and rotates the virtual camera in the pitch direction of the virtual space according to the rotation angle in the pitch direction of the goggle device. The determination means determines whether or not the rotation angle in the pitch direction of the goggle device is within a predetermined range. When the rotation angle in the pitch direction of the goggle device is outside the predetermined range, the control means performs at least one of movement of the object and control of the virtual camera in the virtual space so that the object is located in front of the virtual camera in the yaw direction.

[0008] According to the above, when the rotation angle in the pitch direction of the goggle device is outside the predetermined range, for example, the object can be moved or the virtual camera can be controlled so that the object is located in front of the virtual camera in the yaw direction. Thereby, when the user rotates the goggle device by a predetermined angle or more in the pitch direction, for example, the user can more easily view the object from the front. Therefore, the operability of the object can be improved.

[0009] In another configuration, while the rotation angle in the pitch direction of the goggle device is outside the predetermined range, the control means may perform at least one of movement of the object and control of the virtual camera according to the rotation of the goggle device in the yaw direction so that the object continues to be located in front of the virtual camera in the yaw direction.

[0010] According to the above, while the rotation angle of the goggle device in the pitch direction is outside the predetermined range, for example, even when the goggle device is rotated in the yaw direction, an object can be positioned in the front in the yaw direction.

[0011] Also, in another configuration, when the rotation angle of the goggle device in the pitch direction is within the predetermined range, the control means may not perform the movement of the object and the control of the virtual camera for positioning the object in front of the virtual camera in the yaw direction.

[0012] According to the above, when the rotation angle of the goggle device in the pitch direction is within the predetermined range, for example, the position of the object can be continuously maintained.

[0013] Also, in another configuration, the predetermined range may be a range including an angle at which the goggle device is in a horizontal state.

[0014] According to the above, for example, when the goggle device is in a horizontal state, the position of the object can be continuously maintained, for example.

[0015] Also, in another configuration, the image display system may further include a selection means for selecting the object according to the rotation angle of the goggle device. When the rotation angle of the goggle device in the pitch direction is outside the predetermined range, the selection of the object by the selection means may be disabled.

[0016] According to the above, an object can be selected according to the rotation angle of the goggle device, and at least one of the movement of the object and the control of the virtual camera can be performed when the object cannot be selected.

[0017] In another configuration, the image display system may further include second determination means for determining whether or not the yaw rotation angle of the goggle device satisfies a predetermined condition. Further, when it is determined by the second determination means that the yaw rotation angle of the goggle device satisfies the predetermined condition, the image display system may further include second control means for performing at least one of movement of the object and control of the virtual camera in the virtual space so that the object is located in front of the virtual camera in the yaw direction.

[0018] According to the above, based on the yaw rotation angle of the goggle device, it is possible to move the object or control the virtual camera.

[0019] In another configuration, the determination means may determine that the pitch rotation angle of the goggle device is outside the predetermined range when the depression angle of the goggle device is equal to or greater than a predetermined threshold.

[0020] According to the above, when the goggle device is facing downward, it is possible to move the object or control the virtual camera. Note that the determination means may determine that the pitch rotation angle of the goggle device is outside the predetermined range when the depression angle of the goggle device is a predetermined threshold, or may determine that it is within the predetermined range.

[0021] In another configuration, the determination means may determine that the pitch rotation angle of the goggle device is outside the predetermined range when the elevation angle of the goggle device is equal to or greater than a predetermined threshold.

[0022] According to the above, when the goggle device is facing upward, it is possible to move the object or control the virtual camera. Note that the determination means may determine that the pitch rotation angle of the goggle device is outside the predetermined range when the elevation angle of the goggle device is a predetermined threshold, or may determine that it is within the predetermined range.

[0023] In another configuration, when the depression angle of the goggle device is equal to or greater than a threshold value, or when the elevation angle is equal to or greater than a threshold value, the determination means may determine that the rotation angle of the goggle device in the pitch direction is outside the predetermined range.

[0024] According to the above, when the goggle device is facing up or down, the movement of the object or the control of the virtual camera can be performed. Note that the determination means may determine that the rotation angle of the goggle device in the pitch direction is outside the predetermined range when the depression angle of the goggle device is equal to the threshold value or the elevation angle is equal to the threshold value, or may determine that it is within the predetermined range.

[0025] In another configuration, when the rotation angle of the goggle device in the pitch direction is outside the predetermined range, the control means may move the object to the front of the virtual camera in the yaw direction while maintaining the height direction position of the object in the virtual space.

[0026] According to the above, the object can be moved to the front of the virtual camera in the yaw direction while maintaining the height direction position of the object. As a result, when the user faces straight ahead, the object can be viewed from the front, and the operability of the object can be improved.

[0027] In another configuration, the object may be a user interface that can be operated by the user.

[0028] According to the above, the operability of the user interface can be improved.

[0029] In another configuration, when the rotation angle of the goggle device in the pitch direction becomes within the predetermined range after becoming outside the predetermined range, at least one of the movement of the object and the control of the virtual camera may be performed so that the object is positioned in front of the virtual camera in the yaw direction.

[0030] According to the above, when the rotation angle of the goggle device in the pitch direction is outside a predetermined range and then within the predetermined range, the object can be positioned in front of the virtual camera in the yaw direction. As a result, for example, when the user rotates the goggle device by a predetermined angle or more in the pitch direction and then returns the rotation of the goggle device in the pitch direction and views the goggle device, the user can view the object from the front.

[0031] An image display system according to another embodiment is an image display system including a goggle device, and includes an object placement unit, a display control unit, an acquisition unit, a camera rotation unit, a determination unit, and a control unit. The object placement unit places an object in a virtual space. The display control unit causes an image captured by a virtual camera in the virtual space to be displayed on a display unit of the goggle device. The acquisition unit acquires at least the rotation angle of the goggle device in the yaw direction and the rotation angle in the pitch direction. The camera rotation unit rotates the virtual camera in the yaw direction of the virtual space according to the rotation angle of the goggle device in the yaw direction, and rotates the virtual camera in the pitch direction of the virtual space according to the rotation angle of the goggle device in the pitch direction. The determination unit determines whether the rotation angle of the goggle device in the pitch direction is within a predetermined range. The control unit, when the rotation angle of the goggle device in the pitch direction becomes within the predetermined range after being outside the predetermined range, performs at least one of moving the object and controlling the virtual camera in the virtual space so that the object is positioned in front of the virtual camera in the yaw direction.

[0032] According to the above, when the rotation angle of the goggle device in the pitch direction is outside a predetermined range and then within the predetermined range, the object can be positioned in front of the virtual camera in the yaw direction. As a result, for example, when the user rotates the goggle device by a predetermined angle or more in the pitch direction and then returns the rotation of the goggle device in the pitch direction and views the goggle device, the user can view the object from the front. Therefore, the operability of the object can be improved.

[0033] Further, another invention may be an image display program that causes a processor of a device for displaying an image on a display unit of the goggle device to function as each of the above means. Further, another invention may be a display control device for displaying an image on a display unit of the goggle device, the device including each of the above means. Further, another invention may be an image display method performed in the above image display system including the goggle device.

Advantages of the Invention

[0034] According to the present invention, for example, by rotating the goggle device in the pitch direction, an object can be moved to the front of the virtual camera in the yaw direction, and the operability of the object can be improved.

Brief Description of the Drawings

[0035]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 7A

Figure 7B

Figure 7C

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0036] (Configuration of the Image Display System) Hereinafter, with reference to the drawings, the image display system 1 of this embodiment will be described. The image display system 1 of this embodiment is a system that allows a user to experience virtual reality (VR). FIG. 1 is a diagram showing an example of the image display system 1 in this embodiment. FIG. 2 is a diagram showing an example of the appearance of the information processing device 2 having the display unit 21 included in the image display system 1.

[0037] As shown in FIG. 1, the image display system 1 includes a goggle device 10. The goggle device 10 is held by both or one of the user's hands and applied to the user's face so as to cover the user's left and right eyes.

[0038] The goggle device 10 includes an upper surface 11, a right side surface 12, a lower surface 13, a left side surface 14, and a bottom surface 15. The goggle body is formed by these surfaces 11 to 15. Further, on the bottom surface 15 of the goggle device 10, a substantially circular left opening 16L is provided at a position corresponding to the user's left eye, and a substantially circular right opening 16R is provided at a position corresponding to the user's right eye. Lenses are fitted in the left opening 16L and the right opening 16R, respectively. Further, a partition surface 17 that divides the inner space of the goggle device 10 into left and right is provided in the inner space surrounded by the surfaces 11 to 15 of the goggle device 10.

[0039] The goggle device 10 also includes a display unit 21. Specifically, an information processing device 2 having the display unit 21 is provided on the back side of the bottom surface 15 of the goggle body. A part of the display unit 21 of the information processing device 2 provided on the back side of the bottom surface 15 is visually recognized by the user through the left opening 16L and the right opening 16R via the lenses. As shown in FIG. 2, the display unit 21 is a horizontally long substantially rectangular display screen. The left region 21L surrounded by the dashed line on the left side of FIG. 2 is visually recognized by the user's left eye through the lens provided in the left opening 16L, and the right region 21R surrounded by the dashed line on the right side is visually recognized by the user's right eye through the lens provided in the right opening 16R. When the user holds the goggle device 10 and applies it to the face, the user's left eye is substantially surrounded by the upper surface 11, the lower surface 13, the left side surface 14, and the partition surface 17, and the user's right eye is substantially surrounded by the upper surface 11, the lower surface 13, the right side surface 12, and the partition surface 17. Therefore, while the left eye image displayed in the left region 21L of the display unit 21 is visually recognized by the user's left eye, the surrounding environment and the right eye image displayed in the right region 21R are difficult to be visually recognized by the left eye. Also, while the right eye image displayed in the right region 21R of the display unit 21 is visually recognized by the user's right eye, the surrounding environment and the left eye image displayed in the left region 21L are difficult to be visually recognized by the right eye.

[0040] The goggle device 10 is constituted by the goggle body and the display unit 21. The information processing device 2 having the display unit 21 is detachably attached to the goggle body. As the information processing device 2 that can be detachably attached to the goggle body, a tablet terminal, a smartphone, a portable game device, or the like may be used.

[0041] Although details will be described later, the information processing device 2 generates a left eye image of the virtual space as seen from the left virtual camera and a right eye image of the virtual space as seen from the right virtual camera. The information processing device 2 displays the generated left eye image and right eye image in the left region 21L and the right region 21R of the display unit 21, respectively. Thereby, the user can visually recognize a stereoscopic image and experience virtual reality (VR) as if the user exists in the virtual space.

[0042] FIG. 3 is a diagram showing an example of the functional configuration of the information processing apparatus 2. As shown in FIG. 3, in addition to the display unit 21, the information processing apparatus 2 includes a processor 20, an inertial sensor 22, a DRAM 23, and a nonvolatile memory 24. The processor 20 includes a CPU and a GPU. The CPU performs the processes described later in cooperation with the DRAM 23, and the GPU generates images (left-eye image and right-eye image) according to instructions from the CPU. The generated left-eye image and right-eye image are respectively displayed in the left region 21L and the right region 21R of the display unit 21. Note that the CPU and the GPU may be mounted on separate chips, or they may be mounted on one chip as a SoC (System-on-a-chip).

[0043] Also, the inertial sensor 22 is a sensor for detecting the posture of the information processing apparatus 2. Specifically, the inertial sensor 22 includes an angular velocity sensor and an acceleration sensor. The angular velocity sensor detects the angular velocity around a predetermined three axes (for example, the XYZ axes shown in FIG. 2). Also, the acceleration sensor detects the acceleration around a predetermined three axes (for example, the XYZ axes shown in FIG. 2).

[0044] Also, the nonvolatile memory 24 is a storage device that stores a program for performing the processes described later, and may be, for example, a flash memory. Note that the nonvolatile memory 24 may be any storage device such as a magnetic disk or an optical disk.

[0045] FIG. 4 is a diagram showing a state when the user uses the goggle device 10. As shown in FIG. 4, the user holds the goggle device 10 on which the information processing apparatus 2 is mounted and looks into the display unit 21. When the user turns the face or the whole body in the left-right direction (horizontal direction; or also referred to as the "yaw direction") or in the up-down direction (vertical direction; or also referred to as the "pitch direction") in this state, the posture of the goggle device 10 (information processing apparatus 2) changes from the reference posture. The reference posture is, for example, the posture of the goggle device 10 when the user faces forward. For example, the reference posture may be a posture in which the downward direction (negative direction of the Y axis in FIG. 2) along the display unit 21 (bottom surface 15) of the goggle device 10 is parallel to the direction of gravity.

[0046] The information processing device 2 calculates the posture of the information processing device 2 (goggle device 10) based on the angular velocity value and / or acceleration value detected by the inertial sensor 22. According to the posture of the information processing device 2 (goggle device 10) in the real space, the postures of the left virtual camera and the right virtual camera in the virtual space are controlled.

[0047] A virtual object is arranged in the virtual space, and the user can view a stereoscopic image of the virtual space including the virtual object. In the present embodiment, a user interface (UI) object is arranged in the virtual space as an example of the virtual object.

[0048] FIG. 5 is a diagram showing an example of the virtual space VS. As shown in FIG. 5, an xyz orthogonal coordinate system is set in the virtual space VS. The x-axis is the horizontal axis of the virtual space VS. The y-axis is the height axis of the virtual space VS. The z-axis is perpendicular to the x-axis and the y-axis and is the depth axis of the virtual space.

[0049] A left virtual camera VCL and a right virtual camera VCR are arranged in the virtual space VS. The left virtual camera VCL and the right virtual camera VCR are arranged in the virtual space at an interval similar to the interval between the left and right eyes of an average user. By visually recognizing the left-eye image and the right-eye image of the virtual space seen from the left virtual camera VCL and the right virtual camera VCR by the user's left eye and right eye respectively, the user can view the stereoscopic image of the virtual space. Hereinafter, the left virtual camera VCL and the right virtual camera VCR may be collectively referred to as "virtual camera VC".

[0050] In addition, a UI object 30 is arranged in the virtual space VS. The UI object 30 is a virtual object for the user to operate or display information presented to the user. For example, the UI object 30 provides a menu function for the user. For example, the UI object 30 is displayed during the execution of a predetermined game application, and the user can use the UI object 30 to perform a predetermined operation in the game. The UI object 30 is, for example, a plate-shaped object and is arranged in a posture perpendicular to the xz plane at a position of a predetermined height in the virtual space.

[0051] The UI object 30 includes icons 31 to 34 that can be selected by the user. For example, the icon 34 selected by the user may be displayed in a different manner from other icons. The method for the user to select any one of the icons 31 to 34 is arbitrary. For example, a pointer 37 (see FIG. 7A) is displayed at a predetermined position (for example, the center of the image) of the stereoscopic image displayed on the display unit 21. When the pointer is within the display area of the icon, the icon may be processed as being selected by the user. In this case, when the user selects a desired icon, the user changes the posture of the goggle device 10 so that the pointer enters the display area of the desired icon. Further, for example, the user may use a handheld operation device connected to the information processing device 2 wirelessly or by wire to control the position of the pointer displayed in the stereoscopic image. For example, the operation device includes a direction indicating unit (for example, an analog stick, a cross key, a button, etc.) capable of indicating a direction, and the position of the pointer may be controlled based on an operation on the direction indicating unit.

[0052] Each icon may be, for example, an icon for determining an operation in the game or an icon for using a predetermined item. Further, each icon may be an icon for performing an operation (for example, an operation to return to the previous screen, an operation for determination) on the menu screen provided by the UI object 30. Further, each icon may be an icon for selecting and starting a predetermined application.

[0053] Further, the UI object 30 includes an information display object 35. The information display object 35 is an object for providing information to the user using characters, images, etc.

[0054] Note that in the virtual space, not only games but also any other activities may be performed. For example, simulations of specific tasks (e.g., tasks of driving an airplane or a car) may be performed. Depending on the activities performed in the virtual space, the functions of each icon included in the UI object 30 (operations performed using the UI object 30) are different.

[0055] (Movement control of UI object 30) Next, the control of the UI object 30 when the user turns the bottom surface 15 of the goggle device 10 in the left - right direction (yaw direction) or the up - down direction (pitch direction) while looking at the display unit 21 of the goggle device 10 will be described.

[0056] FIG. 6A is a view of the virtual space as seen from above when the goggle device 10 is maintained in the reference posture. FIG. 6B is a view of the virtual space as seen from above when the goggle device 10 is rotated in the yaw direction (left direction) from the reference posture. FIG. 6C is a view of the virtual space as seen from above when the goggle device 10 is further rotated in the yaw direction (left direction) from the state of FIG. 6B.

[0057] FIG. 7A is an example of an image visible to the user, and shows an example of an image of the virtual space seen from the virtual camera VC (VCL or VCR) in the posture shown in FIG. 6A. FIG. 7B is an example of an image visible to the user, and shows an example of an image of the virtual space seen from the virtual camera VC in the posture shown in FIG. 6B. FIG. 7C is an example of an image visible to the user, and shows an example of an image of the virtual space seen from the virtual camera VC in the posture shown in FIG. 6C. Note that in FIGS. 7A to 7C, planar rectangular images are shown, but actually the images shown in FIGS. 7A to 7C are stereoscopic images and are substantially circular in shape like the left region 21L and the right display region 21R.

[0058] A camera coordinate system (XYZ coordinate system) fixed to the virtual camera VC is set for the virtual camera VC. The X-axis is the axis in the right direction of the virtual camera VC, the Y-axis is the axis in the upward direction of the virtual camera VC, and the Z-axis is the axis in the line-of-sight direction of the virtual camera VC. As shown in FIG. 6A, when the goggle device 10 (information processing device 2) is maintained in the reference posture, the virtual camera VC is facing the UI object 30.

[0059] In the state shown in FIG. 6A, an image as shown in FIG. 7A is displayed on the display unit 21. The user visually recognizes the stereoscopic image as shown in FIG. 7A by visually recognizing the left-eye image and the right-eye image displayed on the display unit 21. As shown in FIG. 7A, the UI object 30 is displayed in the front, and the background of the virtual space is displayed behind the UI object 30. Further, for example, a pointer 37 is set and displayed at a position that is the center of the stereoscopic image. The user controls the posture of the goggle device 10 so that the pointer 37 is located in the display area of the UI object 30, and uses the pointer 37 to select the selectable icons 31 to 34 in the UI object 30. Note that the pointer 37 may not be displayed on the screen and may be set inside the information processing device 2.

[0060] When the goggle device 10 is rotated in the yaw direction (for example, the left direction) in the real space, as shown in FIG. 6B, the virtual camera VC also rotates in the yaw direction (for example, the left direction) in the virtual space. At this time, an image as shown in FIG. 7B is displayed. Specifically, when the goggle device 10 is rotated to the left, the UI object 30 is displayed so as to be located on the right side of the front, and a part of the UI object 30 is not displayed.

[0061] When the goggle device 10 is further rotated leftward from the state shown in FIG. 6B, as shown in FIG. 6C, the virtual camera VC further rotates leftward. At this time, an image as shown in FIG. 7C is displayed on the display unit 21. Specifically, when the goggle device 10 is further rotated leftward from the state shown in FIG. 6B without inverting it rightward, the UI object 30 exits the imaging range of the virtual camera VC and is no longer displayed. In this case, the background of the virtual space other than the UI object 30 is displayed.

[0062] When the user further rotates the goggle device 10 leftward from the state shown in FIG. 6C, the user can visually recognize the virtual space further to the left.

[0063] Here, for example, in the posture shown in FIG. 6C, when the user inverts the goggle device 10 rightward by a predetermined angle, the UI object 30 moves to be positioned in front of the virtual camera VC.

[0064] FIG. 8 is a diagram showing how the UI object 30 moves to the front of the virtual camera VC when the goggle device 10 is inverted rightward by a predetermined angle in the posture shown in FIG. 6C. In FIG. 8, the dashed line 30' indicates the UI object before movement, and the solid line 30 indicates the UI object after movement.

[0065] As shown in FIG. 8, when the goggle device 10 rotates, for example, in the left direction and then reverses by a predetermined angle in the right direction, the UI object 30 moves to the front (front in the left - right direction) of the virtual camera VC. For example, the UI object 30 moves leftward along a circle centered on the intermediate position between the left virtual camera VCL and the right virtual camera VCR while changing its orientation. That is, the UI object 30 moves leftward on a plane parallel to the xz - plane while maintaining the distance from the virtual camera VC. The UI object 30 moves while changing its orientation so as to face the virtual camera VC. Therefore, when the virtual camera VC is not rotating in the pitch direction in the virtual space, the line of sight of the virtual camera VC becomes perpendicular to the moved UI object 30. When the UI object 30 moves to the front of the virtual camera VC, a stereoscopic image as shown in FIG. 7A is displayed on the display unit 21.

[0066] The UI object 30 moves a predetermined distance per frame time (for example, 1 / 60 second). Therefore, when the goggle device 10 rotates, for example, in the left direction and then reverses by a predetermined angle in the right direction, the UI object 30 does not move instantaneously to the front of the virtual camera VC, but moves over a certain period of time (for example, several to several tens of frame times). Note that the UI object 30 may be moved instantaneously (in one - frame time) to the front of the virtual camera VC.

[0067] Note that even when a part of the UI object 30 is displayed and the other part is not displayed as shown in FIG. 7B, when the goggle device 10 reverses by a predetermined angle in the right direction, the UI object 30 may be moved to the front of the virtual camera VC. Also, even when the UI object 30 is displayed at the edge of the screen and the entire UI object 30 is displayed, when the goggle device 10 reverses by a predetermined angle in the right direction, the UI object 30 may be moved to the front of the virtual camera VC.

[0068] Also, when the pointer 37 is located within the display area of the UI object 30, the UI object 30 is not moved. When the pointer 37 is not located within the display area of the UI object 30 and the Google device 10 is inverted by a predetermined angle, the UI object 30 may be moved.

[0069] As described above, in this embodiment, when the virtual camera VC simply rotates in one direction (e.g., the left direction) in the yaw direction, and for example, if the UI object 30 merely exits the imaging range of the virtual camera VC, the UI object 30 is not moved to the front of the virtual camera VC. After the virtual camera VC rotates in one direction (e.g., the left direction) in the yaw direction, when the virtual camera VC rotates (inverts) by a predetermined angle in the other direction (e.g., the right direction) in the yaw direction, the UI object 30 is moved to the front of the virtual camera VC. Hereinafter, the movement of the UI object 30 will be described in detail.

[0070] (Details of the movement of the UI object 30) FIG. 9 is a diagram showing an example of the area where the UI object 30 is moved. In FIG. 9, when the UI object 30 is located in front of the virtual camera VC, a diagram conceptually dividing the virtual space into a plurality of areas based on the virtual camera VC is shown.

[0071] In this embodiment, the processing related to the movement of the UI object 30 differs depending on whether the line of sight of the virtual camera VC is in any of the "UI area", "area B", and "area A" shown in FIG. 9.

[0072] When the line of sight of the virtual camera VC is in the "UI area" shown in FIG. 9, the UI object 30 is displayed on the display unit 21, and the UI object 30 does not move in the virtual space according to the posture change of the goggle device 10. The "UI area" is an area based on the UI object 30 and is an area corresponding to the UI object 30. Specifically, taking the angle of the line of sight of the virtual camera VC when the UI object 30 is located in front of the virtual camera VC (front in the left-right and up-down directions) as 0 degrees, when the angles of the line of sight of the virtual camera VC in the yaw direction and pitch direction in the virtual space are within a predetermined threshold, it is determined that the line of sight of the virtual camera VC is in the "UI area" shown in FIG. 9. More specifically, when the absolute value of the rotation angle of the line of sight of the virtual camera VC in the yaw direction is less than the first threshold (Ty), and the absolute value of the rotation angle of the line of sight of the virtual camera VC in the pitch direction is less than the second threshold (Tx), it is determined that the line of sight of the virtual camera VC is in the "UI area" shown in FIG. 9.

[0073] Here, the UI area is defined by the first threshold and the second threshold. For example, the UI area may coincide with the display area of the UI object 30. In this case, for example, when the rotation angle of the line of sight of the virtual camera VC in the yaw direction reaches the first threshold (Ty), the entire UI object 30 is not displayed as shown in FIG. 7C. Also, the UI area may be smaller than the display area of the UI object 30. In this case, when the rotation angle of the line of sight of the virtual camera VC in the yaw direction reaches the first threshold (Ty), a part of the UI object 30 is displayed as shown in FIG. 7B, for example. Note that even when the rotation angle of the line of sight of the virtual camera VC in the yaw direction reaches the first threshold (Ty), all of the UI object 30 may be displayed. Also, the UI area defined by the first threshold and the second threshold may be larger than the display area of the UI object 30.

[0074] Also, the UI area may be an area where the pointer 37 in the stereoscopic image is located within the display area of the UI object 30. That is, the UI area may be an area formed by the trajectory when the pointer 37 moves along the outer periphery of the display area of the UI object 30.

[0075] When the line of sight of the virtual camera VC is in the B area shown in FIG. 9, "movement processing in the B area" is performed. That is, when the absolute value of the rotation angle in the yaw direction of the line of sight of the virtual camera VC is greater than or equal to the first threshold value (Ty), and the absolute value of the rotation angle in the pitch direction of the line of sight of the virtual camera VC is less than the second threshold value (Tx), "movement processing in the B area" is performed. In the "movement processing in the B area", "judgment as to whether or not to move the UI object 30" is performed. Hereinafter, this judgment will be specifically described with reference to FIGS. 10 and 11.

[0076] FIG. 10 is a view of the virtual camera VC as seen from above in the virtual space, and shows an example when the virtual camera VC is rotated to the left and then to the right. FIG. 11 is a view of the virtual camera VC as seen from the lateral direction in the virtual space, and shows an example when the virtual camera VC is rotated downward and then upward.

[0077] As shown in FIG. 10, when the line of sight of the virtual camera VC rotates beyond the first threshold value "Ty" in the left direction (positive direction) in the yaw direction, for example, the line of sight of the virtual camera VC enters the B region. After the line of sight of the virtual camera VC enters the B region, the change amount "y1" in the left direction and the change amount "y2" in the right direction are calculated. These change amounts "y1" in the left direction and "y2" in the right direction are relative rotation amounts after the line of sight of the virtual camera VC enters the B region. When the line of sight of the virtual camera reciprocates in the left and right directions, the change amount "y1" in the left direction and the change amount "y2" in the right direction are respectively accumulated. For example, after the line of sight of the virtual camera VC enters the B region, if the line of sight of the virtual camera VC rotates in the order of "10" degrees in the left direction, "2" degrees in the right direction, "5" degrees in the left direction, and "3" degrees in the right direction, the cumulative value of the change amount "y1" in the left direction is "10 + 5 = 15" degrees, and the cumulative value of the change amount "y2" in the right direction is "2 + 3 = 5" degrees. Among the cumulative value of the change amount "y1" in the left direction and the cumulative value of the change amount "y2" in the right direction, the smaller value (the cumulative value of the change amount y2 in the right direction) is stored as the change amount Ry in the left and right directions. In this case, the cumulative value of "5" degrees of the change amount "y2" in the right direction is stored as the change amount Ry in the left and right directions.

[0078] The same applies to the rotation in the pitch direction. As shown in FIG. 11, after the line of sight of the virtual camera VC enters the B region (that is, after the line of sight of the virtual camera VC rotates by the first threshold value or more in the yaw direction), it is assumed that it rotates "x1" degrees in the downward direction (negative direction) in the pitch direction, and then rotates "x2 (<x1)" degrees in the upward direction.

[0079] Note that in FIG. 11, the case where the rotation angle in the pitch direction in the virtual space when the line of sight of the virtual camera VC enters the B region is "0" degrees is shown. When the line of sight of the virtual camera VC enters the B region, the line of sight of the virtual camera VC may be rotated by a predetermined angle in the pitch direction in the virtual space. In this case, the change amount in the downward direction from the time when the line of sight of the virtual camera VC enters the B region is "x1".

[0080] Of the cumulative value of the downward change amount "x1" and the cumulative value of the upward change amount "x2", the smaller value (the cumulative value of the upward change amount x2) is stored as the vertical change amount Rx. For example, after the line of sight of the virtual camera VC enters the B region, if the line of sight of the virtual camera VC rotates downward by "10" degrees, upward by "2" degrees, downward by "3" degrees, and upward by "1" degree in that order, the cumulative value of the downward change amount "x1" is "10 + 3 = 13" degrees, and the cumulative value of the upward change amount "x2" is "2 + 1 = 3" degrees. Of the cumulative value of the downward change amount "x1" and the cumulative value of the upward change amount "x2", the smaller value (the cumulative value of the upward change amount x2) is stored as the vertical change amount Rx. In this case, the cumulative value of the upward change amount "x2", which is "3" degrees, is stored as the vertical change amount Rx.

[0081] Here, regarding the yaw direction or the pitch direction, when the line of sight of the virtual camera VC continues to rotate in one direction (for example, the left direction in the yaw direction), the change amount in the other direction (for example, the right direction in the yaw direction) is "0". Therefore, if the smaller value among the cumulative values of the rotation amount in one direction and the rotation amount in the other direction exceeds "0" degrees, it means that the rotation of the line of sight of the virtual camera VC has changed from one direction to the other direction. On the other hand, if the smaller value among the cumulative values of the rotation amount in one direction and the rotation amount in the other direction is "0" degrees, it means that the line of sight of the virtual camera continues to rotate in one direction or the line of sight of the virtual camera has not rotated.

[0082] Therefore, in the present embodiment, based on the smaller values Rx and Ry, it is detected whether the rotation of the line of sight of the virtual camera VC has changed (i.e., reversed) from one direction to the other direction. When a change from one direction to the other direction is detected, the UI object 30 is moved to the front of the virtual camera VC.

[0083] Specifically, when the sum of the change amount Ry in the left-right direction and the change amount Rx in the up-down direction is equal to or greater than a predetermined value (for example, "6" degrees), the UI object 30 is moved to the front of the virtual camera VC. On the other hand, when this sum is less than the predetermined value, the UI object 30 is not moved. For example, when the line of sight of the virtual camera VC is in the B region, if the line of sight of the virtual camera VC rotates in the order of "10 degrees to the left" → "3 degrees to the right" → "20 degrees to the left", the change amount Ry in the left-right direction is "3" degrees, and the change amount Rx in the up-down direction is "0" degrees. In this case, since "Rx + Ry" is less than the predetermined value, the UI object 30 is not moved to the front of the virtual camera VC.

[0084] On the other hand, when the line of sight of the virtual camera VC is in the B region, if the line of sight of the virtual camera VC rotates in the order of "10 degrees to the left" → "3 degrees to the right" → "20 degrees to the left" → "3 degrees to the right", the change amount Ry in the left-right direction is "6" degrees. In this case, since "Rx + Ry" is equal to or greater than the predetermined value, the UI object 30 is moved to the front of the virtual camera VC. Also, when the line of sight of the virtual camera VC rotates in an oblique direction, the rotation direction is divided into components in the left-right direction (yaw direction) and the up-down direction (pitch direction), and the above Rx and Ry are calculated respectively. For example, when the line of sight of the virtual camera VC rotates to "the upper left direction (10 degrees in the left direction component, 10 degrees in the up direction component)", and then rotates to "the lower right direction (3 degrees in the right direction component, 3 degrees in the down direction component)", Rx becomes "3" degrees and Ry becomes "3" degrees. In this case, since "Rx + Ry" is equal to or greater than the predetermined value, the UI object 30 is moved to the front of the virtual camera VC.

[0085] As described above, in the "movement process in area B", after the virtual camera VC rotates in one direction in the yaw or pitch direction, it is detected that the virtual camera VC rotates in the opposite direction. When the virtual camera VC rotates in the opposite direction and the accumulated value of the rotation angle in the opposite direction becomes equal to or greater than a predetermined value, the UI object 30 moves to the front of the virtual camera VC. That is, when the line of sight of the virtual camera VC changes from one direction to the other direction in a predetermined rotation direction (yaw direction or pitch direction), and the rotation angle in the other direction becomes equal to or greater than a predetermined value, the UI object 30 is moved.

[0086] In addition, when the line of sight of the virtual camera VC is in area B, even if "Rx + Ry" is less than the predetermined value, if a predetermined operation is performed by the user, the UI object 30 may be moved to the front of the virtual camera VC. For example, when a predetermined button or location on the goggle device 10 is pressed, tapped, or struck, the UI object 30 may be moved to the front of the virtual camera VC. Also, for example, when the acceleration sensor in the inertial sensor 22 of the goggle device 10 detects an acceleration value equal to or greater than a predetermined value, the UI object 30 may be moved to the front of the virtual camera VC.

[0087] Returning to FIG. 9, when the line of sight of the virtual camera VC is in area A, that is, when the absolute value of the rotation angle of the line of sight of the virtual camera VC in the pitch direction exceeds the second threshold value (Tx), the "movement process in area A" is performed. In the "movement process in area A", the determination in area B described above is not performed, and the UI object 30 is moved so that the UI object 30 is always positioned in front of the virtual camera VC with respect to the left - right direction. Here, it is assumed that the "movement process in area A" is performed when the depression angle or elevation angle of the virtual camera VC (goggle device 10) exceeds the second threshold value, but the threshold value for the depression angle and the threshold value for the elevation angle do not have to be the same value, and they may be different.

[0088] FIG. 12 is a diagram showing how the UI object 30 moves when the line of sight of the virtual camera VC is in area A.

[0089] In FIG. 12, 30' indicated by a dashed line is the UI object before movement, and 30 indicated by a solid line is the UI object after movement. As shown in FIG. 12, for example, when the line of sight of the virtual camera VC is in area A above the UI object 30, the UI object 30 moves so as to be positioned in front of the virtual camera VC in the left-right direction. The rotation angle of the virtual camera VC in the yaw direction (left-right direction) with respect to the UI object 30 after movement is "0" degrees. That is, with reference to the line-of-sight direction of the virtual camera VC, the UI object 30 after movement is not shifted in the left-right direction. Therefore, when the line of sight of the virtual camera VC is rotated downward (when the virtual camera VC is rotated so that the rotation angle in the pitch direction of the virtual camera VC becomes "0" degrees), the UI object 30 is positioned in front of the virtual camera VC.

[0090] Here, assume that after the line of sight of the virtual camera VC enters area B, the UI object 30 does not move in the "movement process in area B", and the line of sight of the virtual camera VC enters area A. In this case, when the line of sight of the virtual camera VC enters area A, the UI object 30 starts to move by the "movement process in area A". In this case, the UI object 30 may move to the front of the virtual camera VC in the left-right direction over a certain period of time (for example, several to several tens of frame times), or may move instantaneously (in 1 frame time).

[0091] Also, when the line of sight of the virtual camera VC enters area A from the state where it is in the "UI area", the UI object 30 moves by the "movement process in area A". When the virtual camera VC rotates in the left-right direction when the line of sight of the virtual camera VC is in area A, the UI object 30 moves in accordance with the rotation of the virtual camera VC in the left-right direction. That is, the UI object 30 always moves so as to follow the rotation of the virtual camera VC in the left-right direction while the virtual camera is rotating in the left-right direction.

[0092] FIG. 13A is a diagram showing an example of an image displayed on the display unit 21 when the line of sight of the virtual camera VC rotates downward and enters the A region. FIG. 13B is a diagram showing an example of an image displayed on the display unit 21 when the virtual camera VC rotates leftward from the state of FIG. 13A.

[0093] As shown in FIG. 13A, when the line of sight of the virtual camera VC rotates downward (i.e., when the goggle device 10 rotates downward), a part of the lower side of the UI object 30 is displayed in the upper region of the screen. The UI object 30 is displayed at the center of the screen in the left-right direction. Further, in the background of the virtual space, for example, there is a virtual object 40, and the virtual object 40 is displayed at the left end of the screen.

[0094] In this state, when the line of sight of the virtual camera VC rotates leftward (i.e., when the goggle device 10 rotates leftward), as shown in FIG. 13B, the position of the UI object 30 in the left-right direction does not change. This is because, by the movement process in the above A region, the UI object 30 moves within the virtual space according to the rotation of the virtual camera VC in the yaw direction. On the other hand, the virtual object 40 existing in the background of the virtual space is displayed on the right side of the position in FIG. 13A. While the user continues to rotate the goggle device 10 leftward, the position of the UI object 30 on the screen does not change, and the background of the virtual space moves rightward.

[0095] Note that the sizes and shapes of the "UI area", "A area", and "B area" shown in FIG. 9 may vary according to the size and shape of the displayed UI object 30. Also, there are multiple types of UI objects 30, and the sizes and shapes of the respective areas may vary according to the type of UI object 30. Further, the sizes and shapes of the respective areas may vary according to the scene (background scene) of the virtual space. In this case, the above-described first threshold value (Ty) and second threshold value (Tx) will vary according to the sizes and shapes of the respective areas. Also, the "UI area" does not necessarily have to match the size of the UI object 30. For example, there may be a margin around the UI object 30, and an area including the area where the UI object 30 is displayed and the surrounding margin may be set as the "UI area". Conversely, the "UI area" may be smaller than the area where the UI object 30 is displayed.

[0096] As described above, in the present embodiment, when the rotation angles in the yaw direction and pitch direction of the virtual camera VC in the virtual space (the rotation angles in the yaw direction and pitch direction of the goggle device 10) exceed a predetermined threshold value, movement processing of the UI object 30 is performed.

[0097] Specifically, when the absolute value of the rotation angle in the yaw direction of the virtual camera VC in the virtual space is greater than the first threshold value (Ty) and the absolute value of the rotation angle in the pitch direction is less than the second threshold value (Tx) (when the line of sight of the virtual camera VC is in the B area), "movement processing in the B area" is performed. In the "movement processing in the B area", a determination is made as to whether to move the UI object 30 based on the rotation angles in the yaw direction and pitch direction of the virtual camera VC. Specifically, when the rotation in the yaw direction of the virtual camera VC changes from one direction to the other direction, a value Ry obtained by accumulating the rotation angle in the other direction is calculated. Also, when the rotation in the pitch direction of the virtual camera VC changes from one direction to the other direction, a value Rx obtained by accumulating the rotation angle in the other direction is calculated. Then, when the sum of Rx and Ry is equal to or greater than a predetermined value, it is determined that the UI object 30 is to be moved, and the UI object 30 is moved to the front of the virtual camera VC.

[0098] There are times when the user wants to view the UI object 30 in the virtual space, and there are also times when the user wants to view objects in the virtual space that are different from the UI object 30 (background or other objects in the virtual space). When the line of sight of the virtual camera VC (the posture of the goggle device 10) continues to rotate in one direction, it is considered that the user wants to view a virtual space different from the UI object 30. For this reason, in the present embodiment, when the line of sight of the virtual camera VC continues to rotate in one direction (for example, to the left in terms of the yaw direction, and upward in terms of the pitch direction) or is stationary, the UI object 30 is not moved to the front of the virtual camera VC. Therefore, the user can overlook the virtual space other than the UI object 30 by continuously rotating the goggle device 10, for example, to the left.

[0099] On the other hand, when the line of sight of the virtual camera VC reverses from the direction away from the UI object 30 to the direction approaching the UI object 30, there is a possibility that the user wants to view the UI object 30 more than other objects in the virtual space. Therefore, in the present embodiment, when the line of sight of the virtual camera VC reverses from one direction (the direction away from the UI object 30) to another direction, and further, when the rotation amount in the other direction becomes equal to or greater than a predetermined value, the UI object 30 is moved to the front of the virtual camera VC.

[0100] By performing such control, in the present embodiment, it is possible to allow the user to view a virtual space different from the UI object 30, and at the same time, move the UI object 30 existing in the virtual space to the front to make it easier to operate (view) the UI object 30.

[0101] For example, when the virtual camera VC faces a direction different from that of the UI object 30 and the UI object 30 is out of the imaging range of the virtual camera VC, it is also conceivable to move the UI object 30 to the front of the virtual camera VC. However, in this case, since the UI object 30 moves to the front of the user when it is out of the imaging range of the virtual camera VC, the user cannot view the virtual space other than the UI object 30.

[0102] In contrast, in this embodiment, the goggle device 10 is continuously rotated in one direction (the direction away from the UI object 30) in the yaw direction, or when it is stationary, the UI object 30 does not move to the front of the virtual camera VC, so the user can view the virtual space other than the UI object 30. When the user wants to view or operate the UI object 30 while looking around the virtual space, the user can move the UI object 30 to the front of the user, for example, by inverting the goggle device 10 in the other direction in the yaw direction or by rocking the goggle device 10 up and down. Thereby, for example, an icon within the UI object 30 can be easily selected using a pointer, and the operability of the UI object 30 can be improved in the VR space. Even when at least a part of the UI object 30 is displayed and the UI object 30 is displayed at the edge of the screen, the UI object 30 can be moved to the front by simply inverting the goggle device 10 by a predetermined angle, and the operability can be improved.

[0103] Also, in this embodiment, since the UI object 30 is moved within the virtual space, it may be possible to reduce VR sickness.

[0104] In addition, in the present embodiment, when the rotation angle of the virtual camera VC in the virtual space in the pitch direction is greater than the second threshold value (Tx) (when the line of sight of the virtual camera VC is in the A region), "movement processing in the A region" is performed. In the "movement processing in the A region", the UI object 30 always moves to the front (front in the left-right direction) of the virtual camera VC in accordance with the rotation of the virtual camera VC in the yaw direction. Therefore, after the line of sight of the virtual camera VC enters the A region, when the display unit 21 is viewed with the goggle device 10 in the reference posture, the UI object 30 always comes to be located in front of the user. For example, after the user places the goggle device 10 on the surface of the real space (a surface parallel to the ground, such as a table) so that the bottom surface 15 of the goggle device 10 faces downward or upward, and then grips the goggle device 10 again to view the display unit 21, the UI object 30 always appears to be located in front of the user.

[0105] Also, for example, it is assumed that two users experience VR using one goggle device 10. After one user experiences VR using the goggle device 10, the goggle device 10 is handed to the other user. At this time, the posture of the goggle device 10 changes, for example, so that the bottom surface 15 faces downward. In this case, if the UI object 30 does not move by the movement processing in the A region, when the other user grips the goggle device 10 to view the display unit 21, the UI object 30 is not necessarily in front of the other user. If the UI object 30 does not move by the movement processing in the A region, the UI object 30 maintains the position when the goggle device 10 was handed to the other user. Therefore, when the other user views the display unit 21 of the goggle device 10, the UI object 30 may be located behind the other user or may be located in the right direction. However, in the present embodiment, since the UI object 30 moves by the movement processing in the A region, when the other user views the display unit 21 of the goggle device 10, the UI object 30 comes to be located in front of the other user. Therefore, the operability is improved for the other user.

[0106] (Details of Processing) Next, a specific example of the processing performed in the information processing apparatus 2 will be described. First, the data stored in the information processing apparatus 2 will be described.

[0107] FIG. 14 is a diagram showing an example of the data stored in the information processing apparatus 2 (DRAM 23 thereof). As shown in FIG. 14, the information processing apparatus 2 stores a predetermined program, angular velocity data, virtual camera data, UI data, cumulative values X1 and X2, cumulative values Y1 and Y2, change amounts Rx and Ry. In addition to these, various data such as data related to objects arranged in the virtual space and operation data according to the user's operations are stored.

[0108] The program is a program for executing the processing described later. The program is stored, for example, in the non-volatile memory 24 or an external storage medium, and is read from the non-volatile memory 24 or the external storage medium into the DRAM 23. Note that the program may be acquired from another device via a network (e.g., LAN, WAN, Internet, etc.).

[0109] The angular velocity data is data related to the angular velocity output by the inertial sensor 22.

[0110] The virtual camera data is data related to the positions and postures of the left virtual camera VCL and the right virtual camera VCR.

[0111] The UI data is data related to the position, type of the UI object 30, icons 31 to 34 included in the UI object 30, the information display object 35, etc.

[0112] The cumulative value X1 is a value used in the movement process in the B region, and is a value obtained by accumulating the downward change amount x1 (see FIG. 11). The cumulative value X2 is a value used in the movement process in the B region, and is a value obtained by accumulating the upward change amount x2 (see FIG. 11).

[0113] The cumulative value Y1 is a value used in the movement process in the B region and is a value obtained by cumulatively adding the amount of change y1 in the left direction (see FIG. 10). Further, the cumulative value Y2 is a value used in the movement process in the B region and is a value obtained by cumulatively adding the amount of change y2 in the right direction (see FIG. 10).

[0114] The amount of change Rx is a value used in the movement process in the B region and is the smaller value of the cumulative values X1 and X2 of the amount of change in the vertical direction (pitch direction) of the line of sight of the virtual camera VC. Further, the amount of change Ry is a value used in the movement process in the B region and is the smaller value of the cumulative values Y1 and Y2 of the amount of change in the horizontal direction (yaw direction) of the line of sight of the virtual camera VC. Rx and Ry are initially set to "0".

[0115] (Explanation of the main flow) Next, the details of the main process performed in the information processing apparatus 2 will be described. FIG. 15 is a flowchart showing an example of the main process performed in the processor 20 of the information processing apparatus 2. The process shown in FIG. 15 is performed by the CPU or GPU of the information processing apparatus 2 executing a predetermined program. Note that in FIG. 15, only the process related to the movement of the above-described UI object 30 is shown, and other processes (for example, processes corresponding to user operations in the virtual space) are omitted.

[0116] As shown in FIG. 15, the processor 20 first performs an initial process (step S100). In the initial process, first, initialization regarding the posture of the goggle device 10 (information processing apparatus 2) is performed. For example, the user is instructed to place the goggle device 10 including the information processing apparatus 2 on a table or the like, and the posture of the goggle device 10 is initialized. Further, in the initial process, an xyz coordinate system is set in the virtual space, and the UI object 30, the left virtual camera VCL, the right virtual camera VCR, the background, and other objects are arranged in the virtual space. After the process of step S100, the processor 20 repeatedly executes the processes of steps S101 to S109 at a predetermined frame time (for example, 1 / 60 second).

[0117] After the process of step S100, the processor 20 acquires data (for example, an angular velocity value) from the inertial sensor 22 (step S101).

[0118] Next, the processor 20 performs an attitude calculation process based on the data from the inertial sensor 22 acquired in step S101 (step S102). Specifically, the processor 20 calculates the attitude of the goggle device 10 (information processing device 2) based on the angular velocity value from the inertial sensor 22. The processor 20 calculates the change in attitude from the initialization in step S100 by integrating the angular velocity value from the inertial sensor 22, and acquires the attitude of the goggle device 10. Further, the processor 20 sets the attitude of the virtual camera VC (left virtual camera CVL and right virtual camera VCR) in the virtual space according to the calculated attitude of the goggle device 10. For example, the processor 20 sets the attitude of each virtual camera VC so that the attitudes of the left virtual camera VCL and the right virtual camera VCR in the virtual space coincide with the attitude of the goggle device 10 in the real space. Thereby, for example, when the goggle device 10 rotates 5 degrees to the left, the left virtual camera VCL and the right virtual camera VCR also rotate 5 degrees to the left.

[0119] Subsequently, the processor 20 determines whether or not the rotation angle in the pitch direction of the line of sight of the virtual camera VC is equal to or greater than a second threshold value (Tx) based on the attitude calculated in step S102 (step S103). When it is determined that the rotation angle in the pitch direction of the line of sight of the virtual camera VC is equal to or greater than the second threshold value (Tx) (step S103: YES), the processor 20 performs a movement process in the A region (step S104). When the process of step S104 is executed, the processor 20 then executes the process of step S108.

[0120] Specifically, in step S104, the processor 20 moves the UI object 30 so as to be in front in the left - right direction of the virtual camera VC. The processor 20 moves the UI object 30 on a plane parallel to the xz plane of the virtual space while maintaining the height of the UI object 30 in the virtual space. For example, the processor 20 moves the UI object 30 along a circle centered on the position of the virtual camera VC (for example, the middle of the left virtual camera VCL and the right virtual camera VCR) so that the UI object 30 is positioned in front in the left - right direction of the virtual camera VC. Thereby, the UI object 30 moves as shown in FIG. 12. Note that the processor 20 may move the UI object 30 by a predetermined distance in one frame time. In this case, the process of step S104 is repeatedly executed for each frame time, and the state in which the UI object 30 moves in front in the left - right direction of the virtual camera VC is displayed on the display unit 21.

[0121] On the other hand, when it is determined that the rotation angle of the virtual camera VC in the pitch direction of the line of sight is less than the second threshold value (Tx) (step S103: NO), the processor 20 determines whether the rotation angle of the virtual camera VC in the yaw direction of the line of sight is greater than or equal to the first threshold value (Ty) (step S105).

[0122] When it is determined that the rotation angle of the virtual camera VC in the yaw direction of the line of sight is greater than or equal to the first threshold value (Ty) (step S105: YES), the processor 20 performs a movement process in the B region (step S106). Details of the movement process in this B region will be described later. When the process of step S106 is executed, the processor 20 then executes the process of step S108.

[0123] When it is determined that the rotation angle in the yaw direction of the line of sight of the virtual camera VC is less than the first threshold value (Ty) (step S105: NO), the processor 20 performs UI processing (step S107). In this UI processing, selection processing of each icon in the UI object 30 and processing after selection are performed. For example, when a pointer 37 located at a predetermined position (e.g., the center) of the stereoscopic image is within the display area of the icon 31 in the UI object 30, the icon 31 is selected. Each icon is assigned processing (e.g., screen switching, operations in a game, startup of other applications, etc.) when selected. The processor 20 performs processing according to the selected icon. Note that in step S107, the processor 20 resets the change amounts Rx, Ry, and the cumulative values X1, X2, Y1, Y2 calculated in step S106 described later to "0" respectively. When the processing in step S107 is executed, the processor 20 then executes the processing in step S108.

[0124] In step S108, the processor 20 performs image generation processing. Specifically, the processor 20 generates a left-eye image by imaging the virtual space using the left virtual camera VCL, and generates a right-eye image by imaging the virtual space using the right virtual camera VCR.

[0125] Then, the processor 20 causes the display unit 21 to display the left-eye image and the right-eye image generated in step S108 (step S109). Specifically, the processor 20 displays the left-eye image in the left region 21L and the right-eye image in the right region 21R. When the processing in step S109 is executed, the processor 20 executes the processing in step S101 again.

[0126] (Movement processing in the B region) Next, the movement processing in the B region of step S106 will be described. FIG. 16 is a flowchart showing an example of the movement processing in the B region of step S106.

[0127] As shown in FIG. 16, the processor 20 first calculates an upward change amount x1 and a downward change amount x2 with respect to the rotation angle in the pitch direction in the virtual space of the line of sight of the virtual camera VC (step S130). For example, the processor 20 determines whether the line of sight has changed upward or downward based on the posture of the virtual camera VC in the previous processing loop and the posture of the virtual camera VC in the current processing loop, and calculates the change amount x1 or x2. Then, the calculated change amount x1 or x2 is added to the cumulative value X1 or X2. For example, when the line of sight of the virtual camera VC continues to change upward, the cumulative value X1 increases, while the downward change amount x2 becomes "0" degrees, and the cumulative value X2 also becomes "0" degrees. Also, for example, as in the example of FIG. 11, when the virtual camera VC rotates downward by x1 degrees and then rotates upward by x2 degrees, "x1" is set for the cumulative value X1 and "x2" is set for the cumulative value X2, respectively.

[0128] Next, the processor 20 sets the smaller value of the cumulative values X1 and X2 updated in step S130 as the change amount Rx in the vertical direction (step S131).

[0129] Subsequently, the processor 20 calculates a leftward change amount y1 and a rightward change amount y2 with respect to the rotation angle in the yaw direction in the virtual space of the line of sight of the virtual camera VC (step S132). For example, the processor 20 determines whether the line of sight has changed leftward or rightward based on the posture of the virtual camera VC in the previous processing loop and the posture of the virtual camera VC in the current processing loop, and calculates the change amount y1 or y2. Then, the calculated change amount y1 or y2 is added to the cumulative value Y1 or Y2. For example, when the line of sight of the virtual camera VC continues to change leftward, the cumulative value Y1 increases, while the rightward change amount y2 becomes "0" degrees, and the cumulative value Y2 also becomes "0" degrees. Also, for example, as in the example of FIG. 10, when the virtual camera VC rotates leftward by y1 degrees and then rotates rightward by y2 degrees, "y1" is set for the cumulative value Y1 and "y2" is set for the cumulative value Y2, respectively.

[0130] Next, the processor 20 sets the smaller value of the cumulative values Y1 and Y2 updated in step S132 as the amount of change Ry in the horizontal direction (step S133).

[0131] Subsequently, the processor 20 determines whether the sum of Rx set in step S131 and Ry set in step S133 is greater than or equal to a predetermined value (step S134).

[0132] When it is determined that "Rx + Ry" is greater than or equal to the predetermined value (step S134: YES), the processor 20 moves the UI object 30 so as to be in front of the virtual camera VC (step S135). Specifically, the processor 20 moves the UI object 30 on a plane parallel to the xz plane of the virtual space while maintaining the height of the UI object 30 in the virtual space. For example, the processor 20 moves the UI object 30 along a circle centered on the position of the virtual camera VC (for example, the middle of the left virtual camera VCL and the right virtual camera VCR) so that the UI object 30 is located in front of the virtual camera VC in the horizontal direction. As a result, the UI object 30 moves as shown in FIG. 8.

[0133] When the process of step S135 is executed, or when it is determined as NO in step S134, the processor 20 determines whether a predetermined operation has been performed (step S136). Here, the "predetermined operation" may be, for example, an operation in which a predetermined button or location on the goggle device 10 is pressed, tapped, or struck.

[0134] When it is determined that a predetermined operation has been performed (step S136: YES), the processor 20 executes the process of step S137. The process of step S137 is the same as the process of step S135.

[0135] When the process of step S137 is executed, or when it is determined as NO in step S136, the processor 20 ends the process shown in FIG. 16 and returns to the process in FIG. 15.

[0136] Note that the processes shown in the above flowchart are merely examples, and the order and content of the processes, as well as the threshold values used for determination, may be changed as appropriate. Also, a threshold value may or may not be included in the determination. For example, the determination of "whether a certain value is equal to or greater than a threshold value" may be replaced with the determination of "whether a certain value is greater than the threshold value".

[0137] As described above, when the line of sight of the virtual camera VC is in the B region, in the movement process in the B region, it is detected that the line of sight of the virtual camera VC has changed from one direction to the other direction with respect to the yaw direction or the pitch direction. Specifically, when the value of Rx or Ry set in step S131 or step S133 exceeds "0" degrees, it means that the direction has changed from one direction to the other direction. When the line of sight of the virtual camera VC changes from one direction to the other direction, and the accumulated value (Rx and / or Ry) of the rotation angle in the other direction becomes equal to or greater than a predetermined value, the UI object 30 is moved to the front of the virtual camera VC. As a result, the user can look around the virtual space other than the UI object 30, and when the user wants to view the UI object 30, the user can move the UI object 30 to the front.

[0138] Also, when the line of sight of the virtual camera VC is in the A region, in the movement process in the A region, the UI object 30 is always moved so as to be in front with respect to the left - right direction of the virtual camera VC. As a result, for example, by turning the goggle device 10 upward or downward, the UI object 30 can be moved to the front.

[0139] (Modification example) The image processing of the present embodiment has been described above. However, the above embodiment is merely an example, and for example, the following modifications may be added.

[0140] For example, in the above embodiment, in the movement process in area A and the movement process in area B, the UI object 30 was moved so that the UI object 30 was located in front of the virtual camera VC. In other embodiments, the virtual camera VC may be controlled (change in posture and / or movement) so that the UI object 30 is located in front of the virtual camera VC. For example, the UI object 30 may be fixed and the posture of the virtual camera VC may be changed, or the virtual camera VC may be moved, so that the UI object 30 is located in front of the virtual camera VC. Also, in other embodiments, both the movement of the UI object 30 and the control of the virtual camera VC may be performed so that the UI object 30 is located in front of the virtual camera VC. That is, at least either one of the movement of the UI object 30 and the control of the virtual camera VC (change in posture and / or movement) may be performed so that the UI object 30 is located in front of the virtual camera VC. When the movement of the UI object 30 is performed, as shown in the above embodiment, the UI object 30 may be moved and rotated so that the UI object 30 faces the virtual camera VC.

[0141] Also, in the above embodiment, the UI object 30 was moved in front of the virtual camera VC when the condition was satisfied. In other embodiments, the UI object 30 may be moved so that at least a part of the UI object 30 is located within the imaging range of the virtual camera VC when the condition is satisfied, or the virtual camera VC may be controlled (for example, movement and / or rotation of the virtual camera), or both the movement of the UI object 30 and the control of the virtual camera VC may be performed.

[0142] In the above embodiment, in the movement process in the B region, when the rotation of the virtual camera VC in the yaw direction or the pitch direction changes from one direction to the other direction, it is determined whether the smaller value among the value obtained by accumulating the rotation angle in the other direction and the value obtained by accumulating the rotation angle in one direction is equal to or greater than a predetermined value. Specifically, it is determined whether the sum of the change amounts Ry and Rx is equal to or greater than a predetermined value. The determination of whether to move the UI object 30 may be performed by other methods. For example, in other embodiments, it may be determined whether either one of the change amounts Ry and Rx is equal to or greater than a predetermined value. Further, when the rotation of the virtual camera VC in the yaw direction or the pitch direction changes from one direction to the other direction, and the value obtained by accumulating the rotation angle in the other direction and / or the value obtained by accumulating the rotation angle in one direction is equal to or greater than a predetermined value, the UI object 30 may be moved.

[0143] In other embodiments, without accumulating the rotation angles in the other direction and the one direction, when the rotation angle in the other direction and / or the rotation angle in one direction becomes equal to or greater than a predetermined value, the UI object 30 may be moved. In this case, when the rotation of the virtual camera VC in one direction and the rotation in the other direction are repeated a plurality of times, if the rotation angle in the other direction or the rotation angle in one direction in each rotation is less than the predetermined value, the UI object 30 is not moved. On the other hand, when the rotation of the virtual camera VC in one direction and the rotation in the other direction are repeated a plurality of times, if the rotation angle in the other direction or the rotation angle in one direction in one rotation becomes equal to or greater than the predetermined value, the UI object 30 is moved.

[0144] Also, the determination of whether the change amount Ry regarding the yaw direction is equal to or greater than a predetermined value and the determination of whether the change amount Rx regarding the pitch direction is equal to or greater than a predetermined value may be performed separately. In this case, when either one of Rx and Ry becomes equal to or greater than a predetermined value, the UI object 30 is moved.

[0145] Also, the threshold values may be made different between the change amount Ry in the yaw direction and the change amount Rx in the pitch direction. For example, for the yaw direction, if Ry is equal to or greater than a first predetermined value (e.g., 4 to 8 degrees), the UI object 30 may be moved, and for the pitch direction, if Rx is equal to or greater than a second predetermined value (e.g., 10 to 20 degrees), which is greater than the first predetermined value, the UI object 30 may be moved. Further, these threshold values may be made different according to the type of the displayed UI object 30 and the scene (background scene) of the virtual space.

[0146] Also, in the above embodiment, regarding the yaw direction and the pitch direction, the first condition is that the line of sight of the virtual camera VC is reversed from one direction to the other direction. When the first condition is satisfied and further the second condition is satisfied (specifically, when the cumulative value of the rotation angle in the other direction becomes equal to or greater than a predetermined value), the UI object 30 is moved to the front of the virtual camera VC. In other embodiments, when the first condition is satisfied, that is, when the line of sight of the virtual camera VC is reversed from one direction to the other direction, the UI object 30 may be moved to the front of the virtual camera VC. In this case, regardless of the cumulative value, the UI object 30 moves to the front of the virtual camera VC at the moment when the line of sight of the virtual camera VC is reversed.

[0147] Also, in other embodiments, when the number of reversals of the line of sight of the virtual camera VC becomes equal to or greater than a predetermined number of times, the UI object 30 may be moved to the front of the virtual camera VC. For example, when the virtual camera VC rotates leftward and enters the B region and then repeats rotations rightward and leftward, the UI object 30 may be moved when the number of reversals (reversals rightward and leftward) becomes equal to or greater than a predetermined number of times.

[0148] Also, in other embodiments, when the line of sight of the virtual camera VC is reversed from one direction (e.g., leftward) to the other direction (e.g., rightward), the UI object 30 may be moved based on the angles in the other direction (rightward) and the one direction (leftward) after the reversal.

[0149] That is, in the movement process in the B area, the UI object 30 may be moved to the front of the virtual camera VC based on the fact that the line of sight of the virtual camera VC has reversed from one direction to the other direction. Here, "based on the fact that the line of sight of the virtual camera VC has reversed from one direction to the other direction" includes, for example, the following matters. · When the line of sight of the virtual camera VC reverses from one direction to the other direction in a predetermined rotation direction, and further, the rotation angle in the other direction or its cumulative value (the above Rx or Ry) becomes a predetermined value or more. · The number of reversals of the line of sight of the virtual camera VC in a predetermined rotation direction becomes a predetermined value. · The line of sight of the virtual camera VC changes from one direction to the other direction in a predetermined rotation direction.

[0150] Also, in the above embodiment, regarding whether the line of sight of the virtual camera VC is in either the "A area" or the "B area" shown in FIG. 9, it is determined based on the angle of the line of sight of the virtual camera VC, and the UI object 30 is moved according to each area. In other embodiments, not limited to the angle of the line of sight of the virtual camera VC, for example, the UI object 30 may be moved according to whether the position of the pointer is in either the "A area" or the "B area". For example, a pointer is displayed at a predetermined position (for example, the center) of the stereoscopic image, and when the pointer is in the A area, the above "movement process in the A area" may be performed, and when the pointer is in the B area, the above "movement process in the B area" may be performed.

[0151] Also, in the above embodiment, the UI object 30 is moved when the condition is satisfied. The object to be moved is not limited to the UI object 30, and may be any object. For example, when the condition is satisfied, a character object that conveys predetermined information to the user may be moved to the front of the virtual camera VC.

[0152] In the above embodiment, the posture of the goggle device 10 (information processing device 2) is detected based on data from the inertial sensor 22 incorporated in the information processing device 2. In other embodiments, the posture of the goggle device 10 may be detected by other methods. For example, the image display system 1 includes a camera that images the goggle device 10 from the outside, images the goggle device 10 or a marker attached to the goggle device 10 with the camera, and may acquire the posture of the goggle device 10 based on the captured image. Further, the goggle device 10 may include a camera, and the posture of the goggle device 10 may be acquired from changes in the image captured by the camera.

[0153] In the above embodiment, the rotation direction of the line of sight of the virtual camera VC is divided into a component in the left - right direction (yaw direction) and a component in the up - down direction (pitch direction) in the virtual space, and the UI object 30 is moved based on the fact that the line of sight of the virtual camera VC has changed from one direction to the other direction with respect to the component in the left - right direction or the up - down direction. For example, when the line of sight of the virtual camera VC changes from one direction (for example, the left direction) to the other direction (for example, the right direction) in the component in the left - right direction, the rotation amount in the other direction and / or the one direction in the component in the left - right direction is calculated, and when the calculated rotation amount is equal to or greater than a predetermined value (the predetermined value may or may not be included), the UI object 30 is moved. In other embodiments, the UI object 30 may be moved based on the fact that the line of sight of the virtual camera VC has changed from one direction (for example, the direction from the lower left to the upper right) to the other direction (for example, the direction from the upper right to the lower left) in the diagonal direction.

[0154] That is, the UI object 30 may be moved based on the fact that a predetermined component (for example, the left - right direction component, the up - down direction component, the diagonal direction component) of the rotation direction of the line of sight of the virtual camera VC has changed from one direction to the other direction.

[0155] Also, in the above-described embodiment, when the line of sight of the virtual camera VC is in the A region, the UI object 30 is moved following the rotation of the virtual camera VC in the left-right direction. In other embodiments, after the line of sight of the virtual camera VC exits the UI region and enters the A region, the UI object 30 may be moved to the front of the virtual camera VC in the left-right direction (yaw direction) (or the virtual camera VC may be controlled) at the timing when the line of sight of the virtual camera VC returns to the UI region. That is, if the UI object 30 is controlled to be visible in the front when the user removes the goggle device 10 (for example, after turning the bottom surface 15 of the goggle device 10 upward or downward) and then places the goggle device 10 on the face to view the display unit, the timing for moving the UI object 30 may be anytime. For example, when the line of sight of the virtual camera VC enters the A region, the UI object 30 may be moved to follow the rotation of the virtual camera VC in the left-right direction, or when the line of sight of the virtual camera VC enters the A region and then returns to the UI region, the UI object 30 may be moved to the front of the virtual camera VC in the left-right direction.

[0156] In the above-described embodiment, when the line of sight of the virtual camera VC is within the A region, the control is performed such that the UI object 30 is positioned in front of the virtual camera VC in the yaw direction. Further, when the line of sight of the virtual camera VC is within the B region, the control is performed such that the UI object 30 is positioned in front of the virtual camera VC on the condition that the line of sight of the virtual camera VC is reversed. The "front" as used herein includes not only the exact front but also the substantial front. The exact front means that the straight line extending the line of sight of the virtual camera VC intersects with a straight line parallel to the y-axis passing through the horizontal center of the UI object 30. In other words, the exact front means that the line of sight of the virtual camera VC is not shifted in the horizontal direction of the UI object 30. The substantial front includes the case where the line of sight of the virtual camera VC is shifted in the horizontal direction of the UI object 30. For example, "the UI object 30 is positioned in the substantial front of the virtual camera VC" may mean that a plane perpendicular to the xz plane passing through the line of sight of the virtual camera VC intersects with the UI object 30. In this case, when the rotation angle of the virtual camera VC in the pitch direction with respect to the UI object 30 is set to "0" degrees, the straight line extending the line of sight of the virtual camera VC hits the UI object 30. Further, "moving the UI object so that the UI object 30 is positioned in the substantial front of the virtual camera VC" means that when the rotation angle of the virtual camera VC in the pitch direction with respect to the UI object 30 is set to "0" degrees, at least a part of the UI object 30 enters the field of view of the virtual camera VC, and the rotation angle of the virtual camera VC in the yaw direction with respect to the UI object 30 becomes smaller than before the movement of the UI object. That is, the "front" in this specification means that the rotation angle of the virtual camera in the yaw direction with respect to the UI object (predetermined object) is a predetermined angle. The "predetermined angle" may be an angle at which at least a part of the predetermined object is included in the field of view of the virtual camera when the rotation angle of the virtual camera in the pitch direction with respect to the predetermined object is set to "0" degrees.

[0157] In the above embodiment, when the line of sight of the virtual camera VC is in the upper A region or the lower A region of the UI region, the movement process in the A region is performed. In other embodiments, the movement process in the A region may be performed only when the line of sight of the virtual camera VC is in the upper A region of the UI region. Conversely, the movement process in the A region may be performed only when the line of sight of the virtual camera VC is in the lower A region of the UI region.

[0158] In addition, the sizes and shapes of the regions shown in FIG. 9 are merely examples, and each region may have any other size and shape. Also, when the line of sight of the virtual camera VC is at the boundary of each region, it may be determined that the line of sight of the virtual camera VC is in the UI region, or it may be determined that the line of sight is in a region different from the UI region (i.e., the A region or the B region).

[0159] In the above embodiment, based on the two left virtual cameras VCL and the right virtual camera VCR, left-eye images and right-eye images having parallax with each other are generated. In other embodiments, an image may be generated based on one virtual camera, and the generated image may be deformed to generate left-eye images and right-eye images having parallax with each other. That is, in this specification, "generating left-eye images and right-eye images based on a virtual camera" includes both generating left-eye images and right-eye images based on a pair of left virtual cameras VCL and right virtual camera VCR (i.e., based on a plurality of virtual cameras) and generating left-eye images and right-eye images based on one virtual camera.

[0160] In the above-described embodiment, the display unit 21 of the goggle device 10 is detachable from the goggle main body. In other embodiments, the display unit 21 of the goggle device 10 may be fixed to the goggle main body. That is, the goggle device 10 may be configured such that a display unit for displaying a left-eye image and a right-eye image and an information processing device for generating the left-eye image and the right-eye image are integrated. Further, the display unit of the goggle device 10 is not limited to the display unit 21 as described above. For example, the display unit of the goggle device 10 may have two display units (a left-eye display unit visually recognized by the user's left eye and a right-eye display unit visually recognized by the user's right eye). Further, the display unit of the goggle device 10 may have an arbitrary shape. For example, the display unit itself of the goggle device 10 may be formed in a substantially circular shape (circular or elliptical) like the left opening 16L and the right opening 16R. Further, two left and right display units formed in a square or rectangular shape may be used as the display unit of the goggle device 10. Further, the display unit of the goggle device 10 may be a display device such as a liquid crystal display device or an organic EL display device, or may be a projection type display device that projects an image onto a projection surface.

[0161] Also, the configuration of the image display system 1 of the above embodiment is merely an example and is not limited to the above. For example, in the above embodiment, the goggle device 10 is configured by an information processing device 2 having a display unit 21 for displaying an image and a processor 20 for performing processing for generating an image, and a goggle body. That is, the image display system 1 is configured by the goggle device 10 including the goggle body and the information processing device 2. In other embodiments, a goggle device having a display unit (a goggle device integrated with the display unit) and an information processing device for performing processing such as generating an image may be configured as separate devices, and the image display system 1 may be configured by these multiple devices. In this case, the goggle device and the information processing device may be connected by wire or wirelessly, and the left-eye image and the right-eye image generated in the information processing device may be transmitted to the goggle device and visually recognized by the user. Further, the information processing device may perform processing related to the movement of the UI object 30 described above, transmit the result of the processing to the goggle device, and the goggle device may generate the left-eye image and the right-eye image and let the user visually recognize them. Also, the goggle device and the information processing device may be connected by a network (LAN, WAN, Internet, etc.).

[0162] Further, in the present embodiment, a goggle device in which the user peeks into the display unit while holding it by hand is used, but in other embodiments, the goggle device may be a head-mounted display that does not require the user to hold it by hand and is fixedly worn on the user's head.

[0163] Although the present invention has been described above, the above description is merely an exemplification of the present invention, and various improvements and modifications may be added.

Explanation of Reference Numerals

[0164] 1 Image display system 2 Information processing device 10 Goggle device 11 Upper surface 12 Right side surface 13 Lower surface 14 Left side surface 15 Bottom surface 16L Left opening 16R Right opening 17 Partition surface 20 Processor 21 Display unit 22 Inertial sensor 30 UI object 31 - 34 Icons 35 Information display object 37 Pointer VC Virtual camera VCL Left virtual camera VCR Right virtual camera

Claims

1. A goggle device, object placement means for placing an object in a virtual space, display control means for causing an image captured by a virtual camera in the virtual space to be displayed on a display unit of the goggle device, acquisition means for acquiring at least a rotation angle in the yaw direction and a rotation angle in the pitch direction of the goggle device, camera rotation means for rotating the virtual camera in the yaw direction of the virtual space according to the rotation angle in the yaw direction of the goggle device, and rotating the virtual camera in the pitch direction of the virtual space according to the rotation angle in the pitch direction of the goggle device, determination means for determining whether or not the rotation angle in the pitch direction of the goggle device is within a predetermined range, control means for performing at least one of moving the object in the left - right direction and controlling the virtual camera in the virtual space so that the object is located within the field of view of the virtual camera in the yaw direction when the rotation angle in the pitch direction of the goggle device is outside the predetermined range. An image display system comprising the same.

2. While the rotation angle in the pitch direction of the goggle device is outside the predetermined range, the control means performs at least one of the movement of the object and the control of the virtual camera according to the rotation of the goggle device in the yaw direction so that the object continues to be located within the field of view of the virtual camera in the yaw direction. The image display system according to Claim 1.

3. The control means, when the rotation angle in the pitch direction of the goggle device is within the predetermined range, does not perform the movement of the object and the control of the virtual camera for positioning the object within the field of view of the virtual camera in the yaw direction. The image display system according to Claim 1 or 2.

4. The predetermined range is a range including an angle at which the goggle device is in a horizontal state. The image display system according to Claim 3.

5. further comprising selection means for selecting the object based on at least the rotation angle in the pitch direction of the goggle device, when the rotation angle in the pitch direction of the goggle device is outside the predetermined range, selection of the object by the selection means is not possible. The image display system according to any one of Claims 1 to 4.

6. second determination means for determining whether or not the rotation angle in the yaw direction of the goggle device satisfies a predetermined condition, When it is determined by the second determination means that the rotation angle of the goggle device in the yaw direction satisfies the predetermined condition, at least one of the movement of the object and the control of the virtual camera is performed in the virtual space so that the object is located within the field of view of the virtual camera in the yaw direction. The image display system according to any one of claims 1 to 5, further comprising second control means.

7. The determination means determines that the rotation angle of the goggle device in the pitch direction is outside the predetermined range when the depression angle of the goggle device is equal to or greater than a predetermined threshold value. The image display system according to any one of claims 1 to 6.

8. The determination means determines that the rotation angle of the goggle device in the pitch direction is outside the predetermined range when the elevation angle of the goggle device is equal to or greater than a predetermined threshold value. The image display system according to any one of claims 1 to 7.

9. The determination means determines that the rotation angle of the goggle device in the pitch direction is outside the predetermined range when the depression angle of the goggle device is equal to or greater than a threshold value or the elevation angle of the goggle device is equal to or greater than a threshold value. The image display system according to any one of claims 1 to 6.

10. When the rotation angle of the goggle device in the pitch direction is outside the predetermined range, the control means moves the object into the field of view of the virtual camera in the yaw direction while maintaining the height direction position of the object in the virtual space. The image display system according to any one of claims 1 to 9.

11. The object is a user interface operable by a user. The image display system according to any one of claims 1 to 10.

12. An image display program executed by a processor of a device that displays an image on a display unit of a goggle device, the processor being Object placement means for placing an object in a virtual space, Display control means for displaying an image captured by a virtual camera in the virtual space on the display unit, Acquisition means for acquiring at least the rotation angle of the goggle device in the yaw direction and the rotation angle in the pitch direction, Camera rotation means for rotating the virtual camera in the yaw direction of the virtual space according to the rotation angle of the goggle device in the yaw direction and rotating the virtual camera in the pitch direction of the virtual space according to the rotation angle of the goggle device in the pitch direction, Determination means for determining whether or not the rotation angle of the goggle device in the pitch direction is within a predetermined range, and When the rotation angle of the goggle device in the pitch direction is outside the predetermined range, at least one of the left-right movement of the object in the virtual space and the control of the virtual camera is performed so that the object is located within the field of view of the virtual camera in the yaw direction, An image display program that functions as control means.

13. A display control device for displaying an image on a display unit of a goggle device, Object placement means for placing an object in a virtual space, Display control means for displaying an image captured by a virtual camera in the virtual space on the display unit, Acquisition means for acquiring at least the rotation angle in the yaw direction and the rotation angle in the pitch direction of the goggle device, Camera rotation means for rotating the virtual camera in the yaw direction of the virtual space according to the rotation angle of the goggle device in the yaw direction, and rotating the virtual camera in the pitch direction of the virtual space according to the rotation angle of the goggle device in the pitch direction, Determination means for determining whether or not the rotation angle of the goggle device in the pitch direction is within a predetermined range, When the rotation angle of the goggle device in the pitch direction is outside the predetermined range, control means for performing at least one of the left-right movement of the object in the virtual space and the control of the virtual camera so that the object is located within the field of view of the virtual camera in the yaw direction, A display control device comprising:

14. An image display method performed in an image display system including a goggle device, the image display system comprising: An object placement step of placing an object in a virtual space, A display control step of displaying an image captured by a virtual camera in the virtual space on a display unit of the goggle device, An acquisition step of acquiring at least the rotation angle in the yaw direction and the rotation angle in the pitch direction of the goggle device, A camera rotation step of rotating the virtual camera in the yaw direction of the virtual space according to the rotation angle of the goggle device in the yaw direction, and rotating the virtual camera in the pitch direction of the virtual space according to the rotation angle of the goggle device in the pitch direction, Determination means for determining whether or not the rotation angle of the goggle device in the pitch direction is within a predetermined range, A control step of performing at least one of the left-right movement of the object in the virtual space and the control of the virtual camera so that the object is located within the field of view of the virtual camera in the yaw direction when the rotation angle of the goggle device in the pitch direction is outside the predetermined range, and an image display method for causing the control step to be executed.

Images