Image display system, image display program, image display device, and image display method

Abstract

To provide an image display system capable of reducing possibility of VR sickness.SOLUTION: One example of an image display system includes a goggle device having a display unit. The image display system includes a virtual camera disposed in a virtual space. An angle of view of the virtual camera can be set to a first angle of view and a second angle of view smaller than the first angle of view. By setting the angle of view of the virtual camera to the second angle of view, part of the virtual space is enlarged and displayed. The image display system sets a parallax between a left-eye image and a right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and sets the parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to an image display system capable of displaying stereoscopic images, an image display program, an image display device, and an image display method. [Background technology]

[0002] Prior art includes a display control system that places a virtual camera in a virtual space, generates images for the left eye and the right eye based on the virtual camera, and allows the user to view the images with their left and right eyes, respectively, thereby allowing the user to view a stereoscopic image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-135771 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a user is allowed to view a stereoscopic image using a goggle device and experience virtual reality (VR), there is room for improvement in terms of reducing the possibility of VR sickness.

[0005] Therefore, an object of the present invention is to provide an image display system, an image display program, an image display device, and an image display method that can reduce the possibility of VR sickness. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention employs the following configuration.

[0007] The image display system of the present invention includes a goggle device, a virtual camera arrangement means, a view angle setting means, an image generation means, a display control means, and a parallax setting means. The virtual camera arrangement means arranges a virtual camera in a virtual space. The view angle setting means sets the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle. The image generation means generates a left eye image and a right eye image, which are images of the virtual space included in the view angle of the virtual camera and have a parallax with respect to each other. The display control means displays the left eye image and the right eye image on a display unit of the goggle device. The parallax setting means sets the parallax of the left eye image and the right eye image to a first parallax when the view angle of the virtual camera is set to the first view angle, and sets the parallax of the left eye image and the right eye image to a second parallax smaller than the first parallax when the view angle of the virtual camera is set to the second view angle. When the angle of view of the virtual camera is set to the first angle of view, the image generation means generates the image for the left eye and the image for the right eye, which have the first parallax relative to each other, and when the angle of view of the virtual camera is set to the second angle of view, the image generation means generates the image for the left eye and the image for the right eye, which are images that are enlarged portions of the virtual space and have the second parallax relative to each other.

[0008] According to the above, a part of the virtual space can be enlarged and displayed by setting the angle of view of the virtual camera to the second angle of view. When enlarging a part of the virtual space, the parallax between the left eye image and the right eye image is made smaller than normal. This weakens the stereoscopic effect when enlarging, and reduces the possibility of VR sickness.

[0009] The virtual camera may include a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image. The parallax setting means may set a virtual distance between the left-eye virtual camera and the right-eye virtual camera to a first distance when the angle of view of the virtual camera is set to the first angle of view, and may set a virtual distance between the left-eye virtual camera and the right-eye virtual camera to a second distance shorter than the first distance when the angle of view of the virtual camera is set to the second angle of view.

[0010] According to the above, by narrowing the distance between the left-eye virtual camera and the right-eye virtual camera when the virtual space is expanded, it is possible to reduce the parallax between the left-eye image and the right-eye image.

[0011] The angle of view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The parallax setting means may continuously shorten a virtual distance between the left-eye virtual camera and the right-eye virtual camera when the angle of view of the virtual camera changes continuously from the first angle of view to the second angle of view.

[0012] According to the above, by continuously shortening the distance between the left-eye virtual camera and the right-eye virtual camera in response to continuously expanding a portion of the virtual space, it is possible to prevent a sudden change in the three-dimensional effect, thereby reducing the possibility of VR sickness.

[0013] The parallax setting means may linearly change the virtual distance between the left-eye virtual camera and the right-eye virtual camera in response to a change in the angle of view of the virtual camera.

[0014] According to the above, the distance between the left-eye virtual camera and the right-eye virtual camera can be changed linearly in response to changes in the angle of view of the virtual cameras, thereby weakening the three-dimensional effect so as not to cause discomfort to the user.

[0015] Furthermore, the parallax setting means may set the second parallax to substantially zero when the angle of view of the virtual camera is set to the second angle of view.

[0016] According to the above, when a part of the virtual space is enlarged to the maximum, a flat image can be displayed, thereby reducing the possibility of VR sickness.

[0017] Furthermore, the angle of view setting means may further set the angle of view of the virtual camera to a third angle of view smaller than the first angle of view and larger than the second angle of view. When the angle of view of the virtual camera is set to the third angle of view, the parallax setting means may further set the parallax of the left eye image and the right eye image to a third parallax smaller than the first parallax and larger than the second parallax. When the angle of view of the virtual camera is set to the third angle of view, the image generating means may generate the left eye image and the right eye image, which are images obtained by enlarging a part of the virtual space and have the third parallax from each other,

[0018] Based on the above, the angle of view of the virtual camera can be set to a third angle of view between the first angle of view and the second angle of view, and a part of the virtual space can be enlarged.

[0019] The angle of view setting means may reduce the angle of view of the virtual camera within a range from the first angle of view to the second angle of view. The image generating means may generate the left eye image and the right eye image by increasing a magnification ratio of a part of the virtual space in response to the reduction in the angle of view of the virtual camera. The parallax setting means may reduce the parallax of the left eye image and the right eye image in response to the increase in the magnification ratio.

[0020] According to the above, the parallax between the left eye image and the right eye image can be reduced as the magnification ratio of the virtual space increases, thereby reducing the possibility of VR sickness during magnification.

[0021] The goggle device may further include a lens for allowing the left eye image and the right eye image to be viewed by a left eye and a right eye of a user, respectively. The first angle of view may be set to a viewing angle of the user using the lens.

[0022] According to the above, when the virtual space is not enlarged, the user's viewing angle and the viewing angle of the virtual camera can be matched, allowing the user to view an image of the VR space that does not feel unnatural.

[0023] The image display system may further include an object arrangement means for arranging a predetermined virtual object in the virtual space. The display control means may generate the left eye image and the right eye image including an image of the predetermined object in a first size when the angle of view of the virtual camera is set to the first angle of view, and may generate the left eye image and the right eye image including an image of the predetermined object in a second size larger than the first size when the angle of view of the virtual camera is set to the second angle of view.

[0024] Based on the above, it is possible to enlarge and display a predetermined object in a virtual space, and it is also possible to reduce the possibility of VR sickness when the object is enlarged.

[0025] Furthermore, the position of the virtual camera in the virtual space may be the same when the angle of view of the virtual camera is set to the first angle of view and when the angle of view is set to the second angle of view.

[0026] According to the above, a part of the virtual space can be enlarged by changing the angle of view of the virtual camera while maintaining the position of the virtual camera. This allows the part of the virtual space to be enlarged and reduces the possibility of VR sickness at the time of enlargement.

[0027] The image display system may further include a detection means for detecting an attitude of the goggle device, a virtual camera attitude control means for controlling an attitude of the virtual camera based on the attitude of the goggle device, and a change setting means for setting a degree of change in the attitude of the virtual camera in response to a change in the attitude of the goggle device. When the angle of view of the virtual camera is set to the second angle of view, if the attitude of the goggle device changes, the virtual camera attitude control means may control the attitude of the virtual camera so as to approach the attitude of the goggle device by the degree of change set by the change setting means.

[0028] According to the above, when a part of the virtual space is enlarged and displayed, if the attitude of the goggle device changes, the attitude of the virtual camera can be changed so as to approach the attitude of the goggle device by a set degree. This makes it possible to reduce the change in the attitude of the virtual camera VC even if the attitude of the goggle device changes, and to prevent shaking of the virtual space due to, for example, camera shake. This makes it possible to reduce the possibility of VR sickness.

[0029] In addition, when the angle of view of the virtual camera is set to the second angle of view, the change setting means may set the degree of change in attitude of the virtual camera in response to a change in attitude of the goggle device to be smaller than when the angle of view of the virtual camera is set to the first angle of view.

[0030] According to the above, when a portion of the virtual space is enlarged, the degree of change in the attitude of the virtual camera is smaller than usual, thereby reducing the possibility of VR sickness when a portion of the virtual space is enlarged and displayed.

[0031] The image display system may further include a detection means for detecting an attitude of the goggle device, and a virtual camera attitude control means for controlling an attitude of the virtual camera based on the attitude of the goggle device. When the angle of view of the virtual camera is set to the first angle of view, the virtual camera attitude control means may control the attitude of the virtual camera in accordance with the attitude of the goggle device after the change when the attitude of the goggle device changes, and when the angle of view of the virtual camera is set to the second angle of view, the virtual camera attitude control means may perform a correction process for reducing the change in the attitude of the virtual camera when the attitude of the goggle device changes, and control the attitude of the virtual camera in accordance with the correction process.

[0032] According to the above, during normal times, no correction process is performed, and when the attitude of the goggle device changes, the attitude of the virtual camera is controlled according to the attitude of the goggle device after the change. On the other hand, during enlargement, a correction process is performed to reduce the change in the attitude of the virtual camera. Therefore, during normal times, the attitude of the virtual camera can be controlled to match the movement of the user, thereby reducing the possibility of VR sickness. On the other hand, during enlargement, by reducing the change in the attitude of the virtual camera, it is possible to prevent shaking of the virtual space due to, for example, camera shake, and to reduce the possibility of VR sickness.

[0033] The image generating means may generate a first left-eye image and a first right-eye image representing a part of the virtual space when the angle of view of the virtual camera is set to the first angle of view, and may generate a second left-eye image and a second right-eye image that are enlarged portions of the virtual space when the angle of view of the virtual camera is set to the second angle of view. The display control means may display each of the first left-eye image and the first right-eye image in a display area of ​​a first size when the angle of view of the virtual camera is set to the first angle of view, and display each of the second left-eye image and the second right-eye image in a display area of ​​a second size smaller than the first size when the angle of view of the virtual camera is set to the second angle of view.

[0034] Based on the above, by reducing the display areas of the left eye image and the right eye image which are enlarged portions of the virtual space, the possibility of VR sickness can be reduced.

[0035] The display area of ​​the second size may be an area having a shape substantially similar to that of the display area of ​​the first size.

[0036] According to the above, even when the angle of view of the virtual camera is set to the second angle of view, a left eye image and a right eye image that are approximately similar to those when the angle of view is set to the first angle of view are displayed, so that when a portion of the virtual space is enlarged and displayed, the user is not made to feel uncomfortable, and the possibility of VR sickness is reduced.

[0037] The display area of ​​the second size may be a central area excluding at least an outer periphery of the display area of ​​the first size.

[0038] According to the above, when a part of the virtual space is enlarged and displayed, the central parts of the left eye image and the right eye image are displayed, excluding the peripheral parts. When the left eye image and the right eye image are viewed through the lens of the goggle device, the peripheral parts look strange, but since the peripheral parts are excluded, the possibility of VR sickness can be reduced.

[0039] The angle of view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The display control means may continuously reduce display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, in response to the angle of view of the virtual camera being continuously changed from the first angle of view to the second angle of view.

[0040] As described above, by continuously enlarging a portion of the virtual space and continuously reducing the display area of ​​the left eye image and the right eye image, it is possible to prevent sudden changes in the display of the virtual space, thereby reducing the possibility of VR sickness.

[0041] The angle of view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The display control means may change display areas of a left-eye image and a right-eye image, which are enlarged portions of the virtual space, into substantially similar shapes and continuously reduce the display areas in response to the continuous change in the angle of view of the virtual camera from the first angle of view to the second angle of view.

[0042] According to the above, since the left eye image and the right eye image having substantially similar shapes are displayed while the angle of view of the virtual camera is continuously changed from the first angle of view to the second angle of view, it is possible to prevent the user from feeling uncomfortable when a part of the virtual space is continuously enlarged and to prevent a sudden change from occurring in the display of the virtual space, thereby reducing the possibility of VR sickness.

[0043] Another image display system of the present invention includes a goggle device, a virtual camera positioning means for positioning a virtual camera in a virtual space, an image generation means for generating an image for a left eye and an image for a right eye having parallax with respect to each other, which are images of the virtual space based on the virtual camera, a display control means for displaying the image for the left eye and the image for the right eye on a display unit of the goggle device, an enlargement setting means for setting a part of the virtual space to be displayed enlarged more than usual, and a parallax setting means for setting the parallax of the image for the left eye and the image for the right eye to a first parallax if the enlarged display has not been set by the enlargement setting means, and for setting the parallax of the image for the left eye and the image for the right eye to a second parallax smaller than the first parallax if the enlarged display has been set by the enlargement setting means. When the enlarged display is not set, the image generating means generates the left eye image and the right eye image having the first parallax relative to each other, and when the enlarged display is set, the image generating means generates the left eye image and the right eye image, which are images that are enlarged portions of the virtual space and have the second parallax relative to each other.

[0044] Another invention may be an image display program that causes a processor of a device that displays an image on a display unit of a goggle device to function as each of the above means. Another invention may be an image display device that displays an image on a display unit of a goggle device and that includes each of the above means. Another invention may be an image display method that is performed in the image display system that includes a goggle device. Effect of the Invention

[0045] According to the present invention, it is possible to enlarge and display a part of a virtual space, and it is also possible to reduce the possibility of VR sickness. [Brief description of the drawings]

[0046] [Figure 1] FIG. 1 shows an example of a state in which a left controller 3 and a right controller 4 are attached to a main unit 2. [Diagram 2] FIG. 1 shows an example of a state in which the left controller 3 and the right controller 4 are removed from the main unit 2. [Diagram 3] Six-sided views showing an example of the main unit 2 [Figure 4] Six-sided diagram showing an example of the left controller 3 [Diagram 5] A block diagram showing an example of the internal configuration of the main unit 2. [Figure 6] A block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. [Figure 7] FIG. 1 is a perspective view showing an example of the appearance of a goggle device 150. [Figure 8] FIG. 1 is a front view showing an example of a state in which the main unit 2 is attached to the goggle device 150. [Figure 9] FIG. 1 is a perspective view showing an example of the appearance of a camera device 200. [Figure 10] FIG. 1 is a diagram showing an example of a state in which a camera device 200 is attached to a goggle device 150 including a main body device 2. [Figure 11] FIG. 1 shows an example of a user viewing an image displayed on the image display system 100. [Figure 12]FIG. 1 is a diagram showing an example of a virtual space constructed in the main device 2. [Figure 13] FIG. 13 is a diagram showing an example of an image displayed on the display 12 of the main device 2 in another scene. [Figure 14] FIG. 14 is a diagram showing an example of the left-eye image and the right-eye image shown in FIG. 13 viewed through a left-eye lens 153L and a right-eye lens 153R, respectively. [Figure 15] A diagram showing an example of the angle of view of the left virtual camera VCL in a normal state and an image for the left eye generated at that angle of view. [Figure 16] FIG. 13 is a diagram showing an example of the angle of view of the left virtual camera VCL during zoom-in and a left-eye image generated at that angle of view. [Figure 17] FIG. 13 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view A that is smaller than the first angle of view and larger than the second angle of view. [Figure 18] FIG. 13 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view B that is smaller than the third angle of view A and larger than the second angle of view. [Figure 19] FIG. 13 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view C that is smaller than the third angle of view B and larger than the second angle of view. [Figure 20] FIG. 13 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a second angle of view. [Figure 21A] FIG. 13 is a diagram showing an example of a user's viewing angle when using a lens 153. [Figure 21B] FIG. 13 is a diagram showing an example of the relationship between the direction along the angle of view of the virtual camera and the line of sight of the user when the angle of view of the virtual camera is changed to a second angle of view; [Figure 22] FIG. 1 is a diagram showing an example of the relationship between the angle of view of a virtual camera VC and the display area of ​​a left-eye image and a right-eye image; [Figure 23] FIG. 1 is a diagram showing an example of the relationship between the angle of view of a virtual camera VC and the inter-camera distance d. [Figure 24] FIG. 13 is a diagram showing an example of a change in the attitude of the main device 2 and a change in the attitude of the virtual camera VC when camera shake correction is not performed. [Diagram 25] FIG. 13 is a diagram showing an example of a change in the attitude of the main device 2 and a change in the attitude of the virtual camera VC when image stabilization is performed. [Figure 26] FIG. 1 is a diagram showing an example of data stored in the memory (mainly the DRAM 85) of the main device 2. [Figure 27] 1 is a flowchart showing an example of processing performed by the processor 81 of the main unit 2. [Figure 28] Flowchart showing an example of the enlarged image display process in step S107 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] Hereinafter, an image display system 100 (see FIG. 11) of this embodiment will be described with reference to the drawings. The image display system 100 of this embodiment is a system that allows a user to experience virtual reality (VR). For example, a predetermined game may be played in a VR space using the image display system 100. First, a game system 1 (an example of an information processing system) included in the image display system 100 will be described.

[0048] (Explanation of Game System 1) An example of the game system 1 includes a main unit (information processing device; in this embodiment, functions as a game device main unit) 2, a left controller 3, and a right controller 4. The left controller 3 and the right controller 4 are each detachable from the main unit 2. In other words, the game system 1 can be used as an integrated device by attaching the left controller 3 and the right controller 4 to the main unit 2. The game system 1 can also be used as a separate device from the main unit 2, the left controller 3, and the right controller 4 (see FIG. 2). Below, the hardware configuration of the game system 1 of this embodiment will be described, followed by a description of the image display system 100 of this embodiment.

[0049] Fig. 1 is a diagram showing an example of a state in which a left controller 3 and a right controller 4 are attached to a main unit 2. As shown in Fig. 1, the left controller 3 and the right controller 4 are each attached to and integrated with the main unit 2. The main unit 2 is a device that executes various processes (e.g., game processes) in the game system 1. The main unit 2 includes a display 12. The left controller 3 and the right controller 4 are devices that include an operation unit that allows the user to perform input.

[0050] Fig. 2 is a diagram showing an example of a state in which the left controller 3 and the right controller 4 have been removed from the main unit 2. As shown in Figs. 1 and 2, the left controller 3 and the right controller 4 are detachable from the main unit 2. In the following, the left controller 3 and the right controller 4 may be collectively referred to as "controller."

[0051] Fig. 3 is a six-sided view showing an example of the main unit 2. As shown in Fig. 3, the main unit 2 includes a substantially plate-shaped housing 11. In this embodiment, the main surface of the housing 11 (in other words, the front surface, i.e., the surface on which the display 12 is provided) is generally rectangular in shape.

[0052] 3, the main unit 2 includes a display 12 provided on a main surface of a housing 11. The display 12 displays an image generated by the main unit 2. In this embodiment, the display 12 is a liquid crystal display (LCD). However, the display 12 may be any type of display device.

[0053] The main unit 2 also has a left side terminal 17, which is a terminal through which the main unit 2 performs wired communication with the left controller 3, and a right side terminal 21 through which the main unit 2 performs wired communication with the right controller 4.

[0054] As shown in FIG. 3, the main unit 2 includes a slot 23. The slot 23 is provided on the upper side of the housing 11. The slot 23 has a shape that allows a predetermined type of storage medium to be attached thereto. The predetermined type of storage medium is, for example, a storage medium (e.g., a dedicated memory card) dedicated to the game system 1 and the same type of information processing device. The predetermined type of storage medium is used, for example, to store data used in the main unit 2 (e.g., application save data, etc.) and / or programs executed in the main unit 2 (e.g., application programs, etc.). The main unit 2 also includes a power button 28.

[0055] FIG. 4 is a six-sided view showing an example of the left controller 3.

[0056] The left controller 3 includes an analog stick 32. As shown in Fig. 4, the analog stick 32 is provided on the main surface of the housing 31. The analog stick 32 can be used as a direction input unit that can input a direction.

[0057] The left controller 3 is equipped with various operation buttons. The left controller 3 is equipped with four operation buttons 33 to 36 (specifically, a right direction button 33, a down direction button 34, an up direction button 35, and a left direction button 36) on the main surface of the housing 31. The left controller 3 is also equipped with a first L button 38 and a ZL button 39 on the upper left of the side of the housing 31. The left controller 3 is also equipped with a second L button 43 and a second R button 44 on the side of the housing 31 that is attached when attached to the main unit 2. These operation buttons are used to give instructions according to various programs (for example, OS programs and application programs) executed on the main unit 2.

[0058] In addition, the left controller 3 is equipped with a terminal 42 that enables the left controller 3 to communicate with the main unit 2 via wire.

[0059] The right controller 4 also has an analog stick and a number of buttons, similar to the left controller 3. A description of the right controller 4 will be omitted.

[0060] FIG. 5 is a block diagram showing an example of the internal configuration of the main unit 2. As shown in FIG.

[0061] The main unit 2 includes a processor 81. The processor 81 is an information processing unit that executes various types of information processing executed in the main unit 2, and may be composed of only a CPU (Central Processing Unit), or may be composed of a SoC (System-on-a-chip) including multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function. The processor 81 executes various types of information processing by executing an information processing program (e.g., a game program) stored in a storage unit (specifically, an internal storage medium such as a flash memory 84, or an external storage medium inserted in the slot 23, etc.).

[0062] The main unit 2 includes a flash memory 84 and a dynamic random access memory (DRAM) 85 as examples of internal storage media built into the main unit 2. The flash memory 84 and the DRAM 85 are connected to the processor 81. The flash memory 84 is a memory used mainly for storing various data (which may be programs) saved in the main unit 2. The DRAM 85 is a memory used for temporarily storing various data used in information processing.

[0063] The main device 2 includes a slot interface (hereinafter abbreviated as "I / F") 91. The slot I / F 91 is connected to the processor 81. The slot I / F 91 is connected to the slot 23, and reads and writes data from and to a predetermined type of storage medium (e.g., a dedicated memory card) inserted in the slot 23 in response to an instruction from the processor 81.

[0064] The processor 81 appropriately reads and writes data from and to the flash memory 84, DRAM 85, and each of the above storage media to execute the above information processing.

[0065] The main unit 2 includes a network communication unit 82. The network communication unit 82 is connected to the processor 81. The network communication unit 82 communicates with an external device via a network (specifically, wireless communication). In this embodiment, the network communication unit 82 connects to a wireless LAN and communicates with an external device using a method conforming to the Wi-Fi standard as a first communication mode. The network communication unit 82 also performs wireless communication with other main units 2 of the same type using a predetermined communication method (for example, communication using a unique protocol or infrared communication) as a second communication mode. Note that the wireless communication using the second communication mode enables wireless communication with other main units 2 located within a closed local network area, and realizes a function that enables so-called "local communication" in which data is transmitted and received by directly communicating between multiple main units 2.

[0066] The main unit 2 includes a controller communication unit 83. The controller communication unit 83 is connected to the processor 81. The controller communication unit 83 performs wireless communication with the left controller 3 and / or the right controller 4. Any communication method may be used between the main unit 2 and the left controller 3 and right controller 4, but in this embodiment, the controller communication unit 83 performs communication with the left controller 3 and the right controller 4 in accordance with the Bluetooth (registered trademark) standard.

[0067] The processor 81 is connected to the above-mentioned left side terminal 17, right side terminal 21, and lower terminal 27. When the processor 81 performs wired communication with the left controller 3, it transmits data to the left controller 3 via the left side terminal 17, and receives operation data from the left controller 3 via the left side terminal 17. When the processor 81 performs wired communication with the right controller 4, it transmits data to the right controller 4 via the right side terminal 21, and receives operation data from the right controller 4 via the right side terminal 21.

[0068] The display 12 is also connected to the processor 81. The processor 81 displays on the display 12 an image generated (for example, by executing the above-mentioned information processing) and / or an image acquired from the outside.

[0069] The main body device 2 also includes an acceleration sensor 89 as an inertial sensor. In this embodiment, the acceleration sensor 89 detects the magnitude of acceleration along three predetermined axes (for example, the x, y and z axes shown in FIG. 1). The acceleration sensor 89 may detect acceleration in one or two axes.

[0070] The main body device 2 also includes an angular velocity sensor 90 as an inertial sensor. In this embodiment, the angular velocity sensor 90 detects angular velocities around three predetermined axes (for example, the x, y and z axes shown in FIG. 1). The angular velocity sensor 90 may detect angular velocities around one axis or two axes.

[0071] The acceleration sensor 89 and the angular velocity sensor 90 are connected to the processor 81, and the detection results of the acceleration sensor 89 and the angular velocity sensor 90 are output to the processor 81. The processor 81 can calculate information related to the movement and / or attitude of the main unit 2 based on the detection results of the acceleration sensor 89 and the angular velocity sensor 90.

[0072] The main unit 2 includes a power control unit 97 and a battery 98. The power control unit 97 is connected to the battery 98 and the processor 81. Although not shown, the power control unit 97 is also connected to each unit of the main unit 2 (specifically, each unit that receives power from the battery 98, the left terminal 17, and the right terminal 21). The power control unit 97 controls the supply of power from the battery 98 to each of the above-mentioned units based on instructions from the processor 81.

[0073] Furthermore, battery 98 is connected to lower terminal 27. When an external charging device (e.g., a cradle) is connected to lower terminal 27 and power is supplied to main unit 2 via lower terminal 27, battery 98 is charged with the supplied power.

[0074] Fig. 6 is a block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. Note that details of the internal configuration of the main unit 2 are omitted in Fig. 6 because they are shown in Fig. 5.

[0075] The left controller 3 includes a communication control unit 101 that communicates with the main unit 2, in addition to the above-mentioned buttons and analog stick 32. As shown in FIG. 6, the communication control unit 101 is connected to each component including the terminal 42. In this embodiment, the communication control unit 101 can communicate with the main unit 2 both by wired communication via the terminal 42 and by wireless communication not via the terminal 42. The communication control unit 101 controls the communication method that the left controller 3 uses with the main unit 2. That is, when the left controller 3 is attached to the main unit 2, the communication control unit 101 communicates with the main unit 2 via the terminal 42. Also, when the left controller 3 is detached from the main unit 2, the communication control unit 101 performs wireless communication with the main unit 2 (specifically, the controller communication unit 83). The wireless communication between the controller communication unit 83 and the communication control unit 101 is performed according to, for example, the Bluetooth (registered trademark) standard.

[0076] The left controller 3 also includes a memory 102, such as a flash memory. The communication control unit 101 is configured with, for example, a microcomputer (also called a microprocessor), and executes firmware stored in the memory 102 to perform various processes.

[0077] The left controller 3 also includes an inertial sensor. Specifically, the left controller 3 includes an acceleration sensor 104. The left controller 3 also includes an angular velocity sensor 105. In this embodiment, the acceleration sensor 104 detects the magnitude of acceleration along three predetermined axes (for example, the x, y and z axes shown in FIG. 4). The acceleration sensor 104 may detect acceleration in one or two axial directions. In this embodiment, the angular velocity sensor 105 detects angular velocity around three predetermined axes (for example, the x, y and z axes shown in FIG. 4). The angular velocity sensor 105 may detect angular velocity around one or two axes. The acceleration sensor 104 and the angular velocity sensor 105 are each connected to the communication control unit 101. The detection results of the acceleration sensor 104 and the angular velocity sensor 105 are repeatedly output to the communication control unit 101 at appropriate timing.

[0078] The communication control unit 101 acquires information related to the input (specifically, information related to the operation, or the detection results by the sensors) from each input unit (specifically, each button, analog stick 32, each sensor 104 and 105). The communication control unit 101 transmits operation data including the acquired information (or information obtained by performing a specified process on the acquired information) to the main unit 2. The operation data is repeatedly transmitted once every specified time. The interval at which the information related to the input is transmitted to the main unit 2 may or may not be the same for each input unit.

[0079] The left controller 3 also includes a power supply unit 108. In this embodiment, the power supply unit 108 has a battery and a power control circuit. Although not shown, the power control circuit is connected to the battery and to each unit of the left controller 3 (specifically, each unit that receives power from the battery).

[0080] Like the left controller 3, the right controller 4 also includes a communication control unit 111, a memory 112, an acceleration sensor 114, an angular velocity sensor 115, and a power supply unit 118. A description of the right controller 4 will be omitted.

[0081] (Description of Image Display System 100) Next, an example of an image display system 100 will be described with reference to Fig. 7 to Fig. 11. The image display system 100 of this embodiment is composed of a goggle device 150 (Fig. 7) to which a main body device 2 is attached, and a camera device 200 (Fig. 9). Below, the goggle device 150 and the camera device 200 will each be described, and the image display system 100 composed of the goggle device 150 and the camera device 200 will be described.

[0082] Fig. 7 is a perspective view showing an example of the appearance of the goggle device 150. Fig. 8 is a front view showing an example of a state in which the main body device 2 is attached to the goggle device 150. Fig. 9 is a perspective view showing an example of the appearance of the camera device 200. Fig. 10 is a diagram showing an example of a state in which the camera device 200 is attached to the goggle device 150 including the main body device 2. Fig. 11 is a diagram showing an example of a state in which a user views an image displayed on the image display system 100.

[0083] 7, the goggle device 150 has a goggle body 151, a lens frame member 152, and lenses 153 (a left-eye lens 153L and a right-eye lens 153R). Here, the goggle device 150, which is an example of a device constituting an image display system, is worn by fitting to the face of a user so as to cover the left and right eyes of the user, and has a function of blocking at least a part of external light and a function of supporting the user's stereoscopic vision by a pair of lenses.

[0084] The goggle body 151 has a mounting portion that fixes the main body device 2 in a detachable manner by contacting the front, back, top, and bottom of the main body device 2. The goggle body 151 has a front contact portion that contacts a part of the front (the surface on which the display 12 is provided) of the main body device 2, a back contact portion that contacts the back of the main body device 2, an upper contact portion that contacts the top surface of the main body device 2, and a lower contact portion that contacts the bottom surface of the main body device 2. The mounting portion is formed by a gap formed by being surrounded by the front contact portion, the back contact portion, the upper contact portion, and the lower contact portion. The mounting portion has an opening on the left side and / or the right side so that it can be mounted from the left side or right side of the main body device 2. Then, as shown in FIG. 8, for example, the main body device 2 is mounted from the opening on the left side to form the goggle device 150 including the main body device 2. In addition, left and right openings are formed in the front abutment portion of the goggle body 151 to enable the user to view the display images (image for the left eye and image for the right eye) on the display 12 when the main unit 2 is worn.

[0085] A lens frame member 152 is fixed to the front side of the front contact portion of the goggle body 151 (the front side of the paper in FIG. 7). The lens frame member 152 is a member for fixing a pair of a left-eye lens 153L and a right-eye lens 153R. The lens frame member 152 has left and right openings. The positions and sizes of the left and right openings of the lens frame member 152 are approximately the same as the positions and sizes of the left and right openings provided in the front contact portion of the goggle body 151. The left-eye lens 153L and the right-eye lens 153R are fitted and fixed in the left and right openings of the lens frame member 152, respectively. The distance between the centers of the left-eye lens 153L and the right-eye lens 153R is set to the distance between the left and right eyes of an average user.

[0086] Further, the goggle device 150 is provided with a recess 154 for accommodating the user's nose. When the user wears the goggle device 150 so that the nose is accommodated in the recess 154, the left-eye lens 153L is positioned in front of the user's left eye, and the right-eye lens 153R is positioned in front of the user's right eye.

[0087] As shown in FIG. 8, when the main device 2 is attached to the goggle body 151, the left-eye image IML displayed in the left display area (area surrounded by a dashed line) of the display 12 is viewed by the left eye of the user through the left-eye lens 153L. The left-eye image is an approximately circular image with a part (a part close to the right-eye image) being straight. The left-eye lens 153L is circular, and allows the left-eye image IML to be viewed by the left eye of the user. The left eye of the user is surrounded by the left side, upper surface, and lower surface of the goggle body 151, and a partition surface (not shown) that separates the left and right sides. For this reason, the left eye of the user can view the left eye image, while the surrounding environment and the right-eye image are difficult to view.

[0088] Further, the right-eye image IMR displayed in the right display area (area surrounded by a dashed line) of the display 12 is viewed by the right eye of the user through the right-eye lens 153R. The right-eye image is an approximately circular image with a portion (a portion close to the left-eye image) being straight. The right-eye lens 153R is circular and allows the right-eye image IMR to be viewed by the user's right eye. The user's right eye is surrounded by the right side, upper surface, lower surface, and partition surface of the goggle body 151. For this reason, the right-eye image is viewed by the user's right eye, while the surrounding environment and the left-eye image are difficult to view.

[0089] The shape of the image for the left eye and the image for the right eye may be a circle, an ellipse, an approximately circular or elliptical shape that is a deformed portion of a circle or ellipse, or a polygon, star, etc.

[0090] As shown in FIG. 9, the image display system 100 includes a camera device 200. The camera device 200 includes a camera body 201 and a cylindrical part 205. The camera device 200 has a shape like a real camera as a whole. The camera device 200 is a pseudo camera that does not have a real imaging element or optical lens. The cylindrical part 205 is provided on the front side (the z-axis positive side in FIG. 9) of the camera device 200. The cylindrical part 205 is a substantially cylindrical member. The central axis direction of the cylindrical part 205 substantially coincides with the line of sight of the user when the user wears the image display system 100. That is, when the user wears the image display system 100, the cylindrical part 205 is formed so as to extend from the front side of the image display system 100 to the line of sight of the user. The cylindrical part 205 is configured to be rotatable in the roll direction (around the central axis of the cylindrical part 205). For example, the cylindrical portion 205 is configured to be rotatable by 90 degrees in the roll direction from the normal state shown in Fig. 9. The cylindrical portion 205 imitates, for example, a telephoto zoom lens of an actual camera.

[0091] The cylindrical portion 205 has an opening 206 at the tip. The above-mentioned left controller 3 (or the right controller 4) is inserted into the opening 206. For example, the left controller 3 is inserted into the opening 206 to detect the rotation angle of the cylindrical portion 205 in the roll direction. When the cylindrical portion 205 is not rotated, the opening 206 has a mountain-shaped shape with the center higher upward (y-axis direction in FIG. 9) than the left and right. When the cylindrical portion 205 is not rotated, the left controller 3 is inserted into the opening 206 in the longitudinal direction (y-axis direction in FIG. 4) so ​​that the main surface of the housing 31 of the left controller 3 faces upward. Since the opening 206 has a mountain-shaped shape, the left controller 3 having the analog stick 32 protruding in the main surface direction can be inserted into the opening 206. Furthermore, when the cylindrical portion 205 is rotated in the roll direction, the left controller 3 inserted inside the opening 206 also rotates around the y-axis; however, the analog stick 32 comes into contact with the mountain-shaped portion, thereby fixing the inserted left controller 3 in place and preventing the left controller 3 from moving inside the opening 206.

[0092] The upper view of Fig. 10 shows the camera device 200 on the opposite side to the cylindrical portion 205. As shown in Fig. 10, the camera main body portion 201 includes an upper surface portion 202, a right side portion 203, and a left side portion 204. (A part of) the goggle device 150 to which the main body device 2 is attached is fitted into a recess formed by the upper surface portion 202, the right side portion 203, and the left side portion 204. For example, at least a portion of the goggle device 150 corresponding to the main body device 2 is fitted into a recess formed by the upper surface portion 202, the right side portion 203, and the left side portion 204. In this way, the image display system 100 is configured.

[0093] As shown in FIG. 11, a user holds an image display system 100 including a goggle device 150 and a camera device 200, and views a left-eye image and a right-eye image displayed on a display 12 of a main unit 2. Specifically, the main unit 2 defines a virtual space, and generates a left-eye image and a right-eye image having a parallax with respect to each other based on a left virtual camera and a right virtual camera. The main unit 2 displays the generated left-eye image and right-eye image in a left display region and a right display region of the display 12, respectively. This allows the user to view a stereoscopic image, and experience virtual reality (VR) as if the user were present in the virtual space.

[0094] (Virtual space display) Next, an image displayed in the image display system 100 of this embodiment will be described. The image display system 100 of this embodiment allows the user to experience VR, and also provides a zoom-in (close-up) function for enlarging a part of the VR space. In the following, first, the virtual space defined by the main unit 2 will be described, and then the zoom-in function in the VR space will be described.

[0095] 12 is a diagram showing an example of a virtual space constructed in the main device 2. An XYZ Cartesian coordinate system is set in the virtual space VS. The X-axis is the horizontal axis of the virtual space VS. The Y-axis is the height axis of the virtual space VS. The Z-axis is the axis perpendicular to the X-axis and Y-axis, and is the depth axis of the virtual space.

[0096] A left virtual camera VCL and a right virtual camera VCR are arranged in the virtual space VS. The heights of the left virtual camera VCL and the right virtual camera VCR may be set to the height of an average human's line of sight. The line of sight directions of the left virtual camera VCL and the right virtual camera VCR are set to the same direction. In the following description, when the left virtual camera VCL and the right virtual camera VCR are not distinguished from each other, the left virtual camera VCL and the right virtual camera VCR may be collectively referred to as a "virtual camera VC."

[0097] The attitude of the virtual camera VC is controlled so as to match the attitude of the image display system 100 (goggle device 150) in the real space. For example, the main device 2 calculates the attitude of the main device 2 based on the angular velocity value and / or acceleration value detected by the angular velocity sensor 90 and / or the acceleration sensor 89. Specifically, the main device 2 calculates the change in the attitude of the main device from the time of initialization by integrating the angular velocity value from the angular velocity sensor 90. The main device 2 controls the attitudes of the left virtual camera VCL and the right virtual camera VCR in the virtual space according to the calculated attitude. For example, when the main device 2 is in an attitude in which a line perpendicular to the display 12 is parallel to the ground, the virtual camera VC faces in a direction parallel to the XZ plane of the virtual space. When the user rotates the goggle device 150 (main device 2) 90 degrees in the yaw direction (left-right direction) from this attitude, the virtual camera VC also rotates 90 degrees in the yaw direction (left-right direction) in the virtual space.

[0098] In addition, various virtual objects are arranged in the virtual space VS. The types of virtual objects arranged vary depending on, for example, the game scene. For example, in the example shown in FIG. 12, a character object 300, a table object 302, and a cylinder object 301 on the table object are arranged in the virtual space VS. The user visually recognizes the left eye image and the right eye image, which are obtained by viewing the virtual space from the left virtual camera VCL and the right virtual camera VCR, with the left eye and the right eye, respectively, to visually recognize a stereoscopic image of the virtual space VS. Note that in this specification, the term "for" does not mean that it is used only for that purpose, and for example, the left eye image or the right eye image may be visually recognized with both eyes.

[0099] Here, a left eye image and a right eye image displayed on the display 12 in normal times (i.e., when not zooming in) will be described. Fig. 13 is a diagram showing an example of an image displayed on the display 12 of the main unit 2 in another scene. In Fig. 13, for example, a scene of a virtual room is displayed.

[0100] As shown in FIG. 13, the left display area, which is substantially circular and surrounded by a dashed line, displays an image for the left eye, which is a virtual space viewed from the left virtual camera VCL. The right display area, which is substantially circular and surrounded by a dashed line, displays an image for the right eye, which is a virtual space viewed from the right virtual camera VCR. The left display area and the right display area are areas of the same size and are symmetrical about the center in the left-right direction of the display 12. Note that areas other than the left display area and the right display area (hatched areas in FIG. 13) are areas that are invisible or difficult to see when viewed by a user through the lens 153. These invisible or difficult to see areas may be referred to as "non-visible areas" below. For example, a black image is displayed in the non-visible area. Also, the outlines of the left display area and the right display area (boundaries with the non-visible areas) are displayed indistinctly by blurring processing.

[0101] The left-eye image and the right-eye image include images of a carpet 400, a window 401, and a ceiling 402 as images of virtual objects. The left-eye image displayed in the left display area of ​​the display 12 is corrected taking into consideration that it is viewed by the left eye of the user through the left-eye lens 153L. That is, when it is displayed on the display 12 as the left-eye image, the left-eye image is distorted as a whole taking into consideration the optical characteristics (lens distortion (distortion aberration)) of the left-eye lens 153L.

[0102] For example, carpet 400 is placed in the virtual space as a rectangular object. Window 401 is a rectangular object with a long side parallel to the X-axis and a short side parallel to the Y-axis of the virtual space. Ceiling 402 is an object with a linear pattern drawn in the virtual space. For example, the pattern of ceiling 402 includes straight lines parallel to the Z-axis of the virtual space. In an image generated based on virtual camera VC, if no distortion according to the characteristics of left-eye lens 153L is added, the straight line portions of each object will be straight lines.

[0103] When generating the left-eye image, the main device 2 does not directly render the virtual space seen from the left virtual camera VCL, but generates the left-eye image by adding distortion according to the characteristics of the left-eye lens 153L. Therefore, the left-eye image displayed on the display 12 has an overall distorted shape, and each object has a distorted shape. For example, the image of the carpet 400 in the left-eye image has a more distorted shape the closer it is to the outer periphery of the left-eye image. Also, the window 401 has a more distorted shape the closer it is to the outer periphery of the left-eye image. Also, the pattern of the ceiling 402 is curved rather than straight in the left-eye image, and the distortion is greater the closer it is to the outer periphery.

[0104] In this way, the degree of distortion is greater in the peripheral portion of the left eye image than in the central portion. This is because, when the left eye image is viewed through the left eye lens 153L, the peripheral portion of the image is distorted more than the central portion of the image. Taking into account such characteristics of the left eye lens 153L, the main unit 2 distorts the entire image so that the distortion becomes greater in the peripheral portion of the image.

[0105] The right-eye image is also an image that has been distorted in accordance with the characteristics of the right-eye lens 153R, similar to the left-eye image. Note that there is a parallax between the left-eye image and the right-eye image. For example, when comparing the left-eye image and the right-eye image, the window 401 is shifted to the left side in the right-eye image compared to the left-eye image.

[0106] Hereinafter, the process of adding distortion to an image taking into account the characteristics of the lens 153 may be referred to as "distortion correction."

[0107] The user views the left-eye image and the right-eye image shown in Fig. 13 through left-eye lens 153L and right-eye lens 153R. When the user views these images, the distortion applied to each image is cancelled by lens 153, and the images appear natural to the user.

[0108] FIG. 14 is a diagram showing an example of the left eye image and the right eye image shown in FIG. 13 viewed through left eye lens 153L and right eye lens 153R, respectively.

[0109] As shown in Fig. 14, the left eye image and the right eye image viewed through lens 153 are natural images, not the overall distorted images shown in Fig. 13. For example, in Fig. 13, the straight line parts of each object are displayed as curved lines, but when viewed through lens 153, the straight line parts of each object are viewed as straight lines.

[0110] In this way, instead of displaying an image seen from the virtual camera VC in the virtual space as it is, an image that has been subjected to distortion correction according to the characteristics of the lens 153 is displayed in the display area of ​​the display 12. Then, when the user sees the left eye image and the right eye image displayed on the display 12 through the lens 153, the distortion of each image is canceled by the lens 153. Therefore, when the user sees the left eye image and the right eye image through the lens 153, the user recognizes them as natural stereoscopic images of the virtual space, and feels as if the virtual object existing in the virtual space is in the real space. For example, when a virtual object is placed 5 m away from the virtual camera VC in the virtual space, the user feels as if the virtual object actually exists 5 m away. Also, the user can change the attitude of the goggle device 150 to change the attitude of the left virtual camera VCL and the right virtual camera VCR, and look around in various directions in the virtual space.

[0111] (Enlarged view of virtual space) In the image display system 100 of this embodiment, the user can enlarge a part of the virtual space by rotating the cylindrical part 205. Specifically, the left controller 3 (or the right controller 4) provided inside the cylindrical part 205 rotates as the cylindrical part 205 rotates. The main unit 2 acquires the attitude of the left controller 3 based on the operation data from the left controller 3, and narrows the angle of view of the left virtual camera VCL and the right virtual camera VCR according to the acquired attitude of the left controller 3. This causes a part of the virtual space to be enlarged (zoomed in).

[0112] Fig. 15 is a diagram showing an example of the angle of view of the left virtual camera VCL in normal mode and an example of a left-eye image generated at the angle of view. Fig. 16 is a diagram showing an example of the angle of view of the left virtual camera VCL in zoom-in mode and an example of a left-eye image generated at the angle of view.

[0113] 15 and 16 show diagrams of the virtual space as viewed from above. Note that for the sake of explanation, the distortion described above is omitted in the left-eye image on the right side of Fig. 15 and Fig. 16.

[0114] 15, the angles of view of the left virtual camera VCL and the right virtual camera VCR are usually set to a first angle of view (e.g., 90 degrees). Also, the distance (inter-camera distance) d between the left virtual camera VCL and the right virtual camera VCR is usually set to the distance between the left and right eyes of an average user.

[0115] When the angle of view of the left virtual camera VCL is set to the first angle of view, the character object 300, the cylinder object 301, and the table object 302 are included in the angle of view of the left virtual camera VCL. An image of the virtual space included in the first angle of view is generated as an image for the left eye. In this case, for example, an image for the left eye as shown in the diagram on the right side of FIG. 15 is generated. Specifically, the main unit 2 performs projective transformation (perspective projection) of the virtual space included in the first angle of view from the position of the left virtual camera VCL to generate an image for the left eye of a predetermined size. Note that when the image for the left eye is generated, the above-mentioned distortion correction is performed. The same is true for the image for the right eye.

[0116] When the user rotates the cylindrical part 205, the angles of view of the left virtual camera VCL and the right virtual camera VCR become narrower. For example, when the user rotates the cylindrical part 205 to the maximum extent, the angles of view of the left virtual camera VCL and the right virtual camera VCR are set to the second angle of view (for example, 30 degrees). When the angle of view of the virtual camera VC is set to the second angle of view, the entire cylindrical object 301 and a part of the table object 302 are included in the angle of view of the virtual camera VC. An image of a virtual space included in the angle of view of the virtual camera VC and having a predetermined size is generated. That is, when the angle of view of the virtual camera VC is set to the second angle of view, an enlarged left-eye image is generated by enlarging a part of the virtual space included when the angle of view of the virtual camera VC was set to the first angle of view. For example, as shown in FIG. 16, an image for the left eye is generated including an entire image of the cylindrical object 301 enlarged from that in FIG. 15 and an enlarged image of a part of the table object 302.

[0117] Furthermore, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is smaller than when the angle of view of the virtual camera VC is set to the first angle of view. That is, when the angle of view of the virtual camera VC is set to the second angle of view, the parallax between the left-eye image and the right-eye image is smaller than when the angle of view of the virtual camera VC is set to the first angle of view. In this embodiment, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is set to "0", and the parallax between the left-eye image and the right-eye image is zero.

[0118] In this embodiment, the angle of view of the virtual camera VC changes continuously from a first angle of view (e.g., 90 degrees) to a second angle of view (e.g., 30 degrees) according to the rotation angle of the cylindrical part 205 (left controller 3). Also, the inter-camera distance d changes continuously from the first angle of view to the second angle of view according to the rotation angle of the cylindrical part 205 (left controller 3).

[0119] In this embodiment, while the angle of view of the virtual camera VC changes continuously from the first angle of view to the second angle of view, a part of the virtual space is continuously enlarged and the display areas of the left eye image and the right eye image are continuously reduced. Below, the left eye image and the right eye image displayed on the display 12 when the angle of view of the virtual camera VC changes from the first angle of view to the second angle of view will be described with reference to Figs. 17 to 20.

[0120] FIG. 17 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view A smaller than the first angle of view and larger than the second angle of view. FIG. 18 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view B smaller than the third angle of view A and larger than the second angle of view. FIG. 19 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view C smaller than the third angle of view B and larger than the second angle of view. FIG. 20 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to the second angle of view.

[0121] As shown in FIG. 17, when the angle of view of the virtual camera VC is set to the third angle of view A (for example, 50 degrees), an image in which a part of the virtual space is enlarged is displayed. For example, in FIG. 15, a character object 300, a cylinder object 301, and a table object 302 are displayed, and the cylinder object 301 is displayed relatively small, but in FIG. 17, the whole of the enlarged cylinder object 301 and a part of the enlarged table object 302 are displayed. That is, when the angle of view of the virtual camera VC is set to the third angle of view A, the displayed sizes of the cylinder object 301 and the table object 302 are larger than when the angle of view of the virtual camera VC is set to the first angle of view. When the angle of view of the virtual camera VC is set to the third angle of view A (for example, 50 degrees), the display areas of the enlarged left-eye image and right-eye image are the same size as the display areas when the angle of view is set to the first angle of view. In addition, the non-visible areas other than the left display area and the right display area are filled with black. The outlines of the left and right display areas (boundaries with the non-visual area) are displayed in a blurred manner.

[0122] As shown in FIG. 18, when the angle of view of the virtual camera VC is set to the third angle of view B (for example, 45 degrees), a part of the virtual space is further enlarged. For example, in FIG. 18, the cylindrical object 301 and the table object 302 are further enlarged and displayed than in FIG. 17. In this case, the display area of ​​the enlarged left-eye image and right-eye image is smaller than that in FIG. 17. In FIG. 18, the left display area and the right display area when the angle of view of the virtual camera VC is set to the first angle of view are shown by dashed lines, and the left display area and the right display area when the angle of view of the virtual camera VC is set to the third angle of view B are shown by solid lines. The area surrounded by the dashed line and the solid line is filled with black, similar to the non-visual area, so that the display area of ​​the left-eye image and the right-eye image, which are part of the virtual space enlarged, becomes smaller. In other words, the outer peripheral parts of the left-eye image and the right-eye image are cut, so that the display area of ​​these images is reduced. The left-eye image and the right-eye image, which are enlarged portions of the virtual space, are images of the same size and are line-symmetric.

[0123] The left and right display areas are substantially circular when the angle of view of the virtual camera VC is the first angle of view (90 degrees), and the left and right display areas are also substantially circular when the angle of view of the virtual camera VC is the third angle of view B (45 degrees), for example, substantially a regular dodecagon. The left and right display areas at the angle of view B are areas excluding the outer periphery of the left and right display areas at the first angle of view. In addition, when the left and right display areas are reduced, the above-mentioned blurring process is not performed, and the contours of the left and right display areas are clear. Note that even when the angle of view of the virtual camera VC is reduced, the contours of the left and right display areas may be made unclear.

[0124] Also, as shown in FIG. 19, when the angle of view of the virtual camera VC is set to the third angle of view C (for example, 35 degrees), a part of the virtual space is further enlarged. The objects 301, 302 in the left eye image and the left eye image in FIG. 19 are displayed larger than the objects 301, 302 in the left eye image in FIG. 18. Also, the display areas of the left eye image and the right eye image in which a part of the virtual space is enlarged are further reduced than in FIG. 18. That is, the left display area and the right display area when the angle of view of the virtual camera VC is set to the third angle of view C (35 degrees) are smaller than the left display area and the right display area when the angle of view of the virtual camera VC is set to the third angle of view B (45 degrees). In other words, when the angle of view of the virtual camera VC is set to the third angle of view C, the peripheral parts of the left eye image and the right eye image are cut larger than when the angle of view of the virtual camera VC is set to the third angle of view B. The shapes of the left and right display areas at the third angle of view C are approximately regular dodecagons, similar to the third angle of view B. Even when the angle of view of virtual camera VC is set to the third angle of view C, the left eye image and right eye image, which are enlarged portions of the virtual space, are the same size and line-symmetrical images.

[0125] Also, as shown in FIG. 20, when the angle of view of the virtual camera VC is set to the second angle of view (for example, 30 degrees), a part of the virtual space is further enlarged. The objects 301 and 302 in the left eye image and the left eye image in FIG. 20 are larger than the objects 301 and 302 in the left eye image in FIG. 19. Also, the display areas of the left eye image and the right eye image in which a part of the virtual space is enlarged are further reduced than in FIG. 19. That is, the left display area and the right display area when the angle of view is set to the second angle of view (30 degrees) are smaller than the left display area and the right display area when the angle of view is set to the third angle of view C (35 degrees). The shapes of the left display area and the right display area at the second angle of view are approximately regular dodecagons, similar to the third angle of view C. Even when the angle of view of the virtual camera VC is set to the second angle of view, the left eye image and the right eye image in which a part of the virtual space is enlarged are images of the same size and line symmetry.

[0126] In this manner, in this embodiment, a portion of the virtual space is enlarged (zoomed in) by reducing the angle of view of the virtual camera VC. The smaller the angle of view of the virtual camera VC, the greater the enlargement rate of the virtual space, and the smaller the display area of ​​the left eye image and the right eye image. In other words, the smaller the angle of view of the virtual camera VC (the more a portion of the virtual space is enlarged), the more the peripheral portions of the left eye image and the right eye image are cut off.

[0127] 17 to 20, the distortion correction causes the distortion to be greater toward the outer periphery of the left-eye image and the right-eye image, and as described below, the outer periphery of the left-eye image and the right-eye image gives the user a feeling of discomfort. However, in this embodiment, the outer periphery of the left-eye image and the right-eye image is cut, so that the discomfort felt by the user can be reduced, and the possibility of VR sickness can be reduced.

[0128] In addition, the shape of the left eye image and the right eye image when the angle of view of the virtual camera VC is set to the first angle of view is approximately circular, and the shape of the left eye image and the right eye image when the angle of view of the virtual camera VC is set to the third angle of view B to the second angle of view is approximately a regular dodecagon. It can be said that the shapes of the left eye image and the right eye image are approximately similar shapes until the angle of view of the virtual camera VC changes from the first angle of view to the second angle of view. Here, the shape of the left eye image and the right eye image when zoomed in is not limited to approximately a regular dodecagon. For example, the shape of the left eye image and the right eye image when zoomed in may be approximately a regular decagon, approximately a regular octagon, approximately a regular hexagon, or approximately a regular pentagon. That is, in this specification, "approximately similar shapes" is not limited to the completely same shape, but includes shapes that are similar but strictly different.

[0129] Here, when changing the angle of view of the virtual camera VC from the first angle of view to the second angle of view, it is possible to keep the size of the display area constant without cutting off the outer periphery of the left eye image and the right eye image as described above. However, reducing the angle of view of the virtual camera VC while maintaining the size of the display area of ​​the left eye image and the right eye image may cause VR sickness.

[0130] Specifically, when the angle of view of the left virtual camera VCL and the right virtual camera VCR is reduced and a part of the virtual space is enlarged, the appearance becomes optically incorrect, which may cause VR sickness. The reason for this will be explained below.

[0131] Fig. 21A is a diagram showing an example of the viewing angle of a user using lens 153. Fig. 21B is a diagram showing an example of the relationship between the direction along the angle of view of the virtual camera and the line of sight of the user when the angle of view of the virtual camera is changed to a second angle of view.

[0132] As shown in FIG. 21A, in the image display system 100 of this embodiment, when the user views the display 12 through the lens 153, the viewing angle of one eye of the user is adjusted to, for example, 90 degrees. For example, the left end of the lens 153 is 45 degrees to the left of the position of the user's eyes in the left-right direction, and the right end of the lens 153 is 45 degrees to the right, and the top end of the lens 153 is 45 degrees above the position of the user's eyes in the up-down direction, and the bottom end of the lens 153 is 45 degrees below the position of the user's eyes. The first angle of view of the virtual camera VC in normal times coincides with the viewing angle of the user using the lens 153. Since the viewing angle of one eye of a person is about 90 degrees to 100 degrees, when the user views the display 12 of the goggle device 150, most of the field of view is covered.

[0133] 21B, when the angle of view (viewing angle) of the virtual camera VC is set to, for example, 30 degrees, an image in a 30-degree range as seen from the virtual camera VC is stretched and displayed on the display 12, and the user sees the stretched image. If the 30-degree range as seen from the virtual camera VC is expanded to a 90-degree viewing range, the user will see an incorrect image.

[0134] Specifically, when the peripheral portions of the left-eye image and the right-eye image are not cut, for example, a virtual object or a line at a 15-degree angle from the virtual camera VC appears to be located at a 45-degree angle from the user's eyes. That is, the angle (45 degrees) when the end of the stretched left-eye image and the right-eye image is viewed from the viewpoint of the user wearing the goggle device 150 is larger than the angle (15 degrees) when the direction of the virtual space along the second angle of view is viewed from the virtual camera VC. For this reason, the virtual space appears differently from the actual appearance, and the user may feel a sense of incongruity. For example, the part that is visible when an object is actually at a 15-degree angle from the user is different from the part that is visible when the object is actually at a 45-degree angle from the user. For example, when an object is actually located at a 15-degree angle, the back part of the object is visible, but when the object is actually located at a 45-degree angle, the back part may not be visible. Therefore, when the virtual space seen from the 30-degree angle of view of the virtual camera VC is expanded to a 90-degree field of view, the object appears differently from the actual appearance from the user. Such a difference in appearance becomes larger toward the outer periphery of the lens 153. In addition, when the angle of view of the virtual camera VC and the field of view of the user wearing the goggle device 150 (the display angle of view of the goggle device 150) match, when the virtual camera VC is rotated at a certain angle, the virtual object in the virtual space also appears to move by the same angle. On the other hand, when the angle of view of the virtual camera VC and the display angle of view of the goggle device 150 differ, when the virtual camera VC is rotated at a certain angle, the virtual object in the virtual space does not move by the same angle. For example, when the angle of view of the virtual camera VC is set to 30 degrees, when the user rotates his / her head 30 degrees to the right, the object in front of him / her rotates 90 degrees to the right. In this way, when the angle of view of the virtual camera VC differs from the display angle of view of the goggle device 150, the virtual object appears unnatural to the user or moves unnaturally. This may cause VR sickness.

[0135] Also, for example, it is conceivable to generate an image of the virtual space seen from the virtual camera VC, and perform distortion correction according to the characteristics of the lens 153 according to the position from the center of the generated image (the display position when displayed on the display 12). Since distortion correction according to the position from the center of the image is performed, when the image displayed on the display 12 is viewed through the lens 153, the distortion of the lens 153 and the distortion due to the distortion correction cancel each other out, and the image is visually recognized by the user as an image without distortion. Therefore, whether the angle of view of the virtual camera VC is set to the first angle of view or the second angle of view, the user basically visually recognizes an image without distortion. However, for example, the position and distance of the user's eyes are not always constant, and there may be some deviation between the position of the user's eyes, the position of the lens 153, and the position of the image displayed on the display 12. Due to these position deviations, the distortion of the lens 153 and the distortion due to the distortion correction may not match. Since the distortion is larger closer to the outer periphery of the lens, the deviation between the distortion of the lens 153 and the distortion due to the distortion correction also becomes larger toward the outer periphery. When the angle of view of the virtual camera VC is set to the second angle and expanded, the object appears distorted around the periphery of the image, which makes the difference from the real appearance large and may cause VR sickness.

[0136] When the angle of view of the virtual camera VC is reduced to enlarge an image, the closer to the outer periphery, the more unnatural the image appears, which may cause VR sickness. Therefore, in this embodiment, in order to reduce VR sickness, as shown in Figs. 17 to 20, when the angle of view of the virtual camera VC is reduced to enlarge a part of the virtual space, the outer periphery of the left eye image and the right eye image is cut. Specifically, the smaller the angle of view of the virtual camera VC, the larger the area of ​​the outer periphery to be cut (in other words, the smaller the display area of ​​the left eye image and the right eye image).

[0137] This allows a part of the virtual space to be enlarged, and reduces VR sickness caused by the enlargement. Specifically, it is possible to prevent the user from viewing the image of the virtual space corresponding to the outer periphery where the difference from the real appearance becomes large, and to prevent the user from feeling uncomfortable.

[0138] In addition, by cutting out the outer periphery of the left eye image and the right eye image, the user becomes aware that part of the virtual space is expanding, and views the images with that awareness, making it less likely for VR sickness to occur.

[0139] In addition, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge the image, the inter-camera distance d is changed according to the angle of view of the virtual camera VC. Specifically, the smaller the angle of view of the virtual camera VC (the higher the enlargement rate of the virtual space), the smaller the inter-camera distance d is. This can reduce VR sickness. That is, when the angle of view of the virtual camera VC is reduced to enlarge the image, the image will look optically incorrect as described above, and the three-dimensional effect will also be different from the reality. For this reason, if the inter-camera distance d is set to the same as normal when the angle of view of the virtual camera VC is reduced to enlarge the image, the user will feel a three-dimensional effect different from the reality, which may cause VR sickness. Therefore, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge the image, the inter-camera distance d is reduced (that is, the parallax between the left eye image and the right eye image is reduced). This can weaken the three-dimensional effect and reduce VR sickness.

[0140] Fig. 22 is a diagram showing an example of the relationship between the angle of view of virtual camera VC and the display area of ​​the left-eye image and the right-eye image. Fig. 23 is a diagram showing an example of the relationship between the angle of view of virtual camera VC and the inter-camera distance d. In Figs. 22 and 23, the horizontal axis indicates the magnification ratio (angle of view of virtual camera VC), and the magnification ratio increases (angle of view decreases) in the rightward direction.

[0141] As shown in FIG. 22, when the angle of view of the virtual camera VC is between the first angle of view (e.g., 90 degrees) and the third angle of view A, the area of ​​the display area of ​​the left-eye image and the right-eye image is constant (see FIG. 17). When the angle of view of the virtual camera VC is smaller than the third angle of view A, the area of ​​the display area is smaller (see FIG. 18 and FIG. 19). The smaller the angle of view of the virtual camera VC, the greater the rate of reduction in the area of ​​the display area. For example, the area of ​​the display area is reduced relatively gradually until the angle of view of the virtual camera VC reaches the third angle of view C, and when the angle of view of the virtual camera VC becomes smaller than the third angle of view C, the area of ​​the display area is reduced abruptly. When the angle of view of the virtual camera VC is set to the smallest second angle of view (e.g., 30 degrees), the display area is smallest (see FIG. 20).

[0142] 23, the inter-camera distance d decreases while the angle of view of the virtual camera VC changes from the first angle of view (e.g., 90 degrees) to the second angle of view. For example, the inter-camera distance d decreases linearly as the angle of view decreases (the image enlarges), and when the angle of view of the virtual camera VC is set to the smallest second angle of view, the inter-camera distance d becomes "0."

[0143] In addition, the inter-camera distance d does not need to change linearly, but may change curvilinearly in response to changes in the angle of view of the virtual camera VC (i.e., the rate of increase or decrease in the inter-camera distance d may change while the angle of view of the virtual camera VC changes).

[0144] (Image stabilization when zooming in) Next, the image stabilization function of this embodiment will be described. In this embodiment, image stabilization is performed when a part of the virtual space is enlarged by reducing the angle of view of the virtual camera VC. Image stabilization is a function that reduces changes in the attitude of the virtual camera VC caused by camera shake. This image stabilization will be described below.

[0145] Fig. 24 is a diagram showing an example of a change in the attitude of the main unit 2 and a change in the attitude of the virtual camera VC when no image stabilization is performed, and Fig. 25 is a diagram showing an example of a change in the attitude of the main unit 2 and a change in the attitude of the virtual camera VC when image stabilization is performed.

[0146] For example, in this embodiment, the orientation of the main unit 2 and the virtual camera VC is represented by a vector. For example, the orientation of the main unit 2 is represented by a quaternion q having components representing rotations in the x, y, and z axis directions and a component representing a rotation angle. The horizontal axis of Fig. 24 and Fig. 25 indicates time, and the vertical axis indicates the vector q representing the orientation.

[0147] As shown in Fig. 24, when the attitude of the main unit 2 changes between times t0 and t5, the attitude of the virtual camera VC usually coincides with the attitude of the main unit 2 at each time. That is, in this embodiment, when the angle of view of the virtual camera VC is set to the first angle of view, image stabilization is not performed. When the main unit 2 detects a change in its own attitude, it changes the attitude of the virtual camera VC so that it coincides with its own attitude. Note that the main unit 2 may calculate its own attitude at a frequency of, for example, 200 Hz.

[0148] On the other hand, as shown in FIG. 25, when the angle of view of the virtual camera VC is smaller than the first angle of view, camera shake correction is performed. When camera shake correction is performed, the attitude of the virtual camera VC changes to approach (follow) the attitude of the main body device 2. When the attitude of the main body device 2 changes, the attitude of the virtual camera VC changes to match the attitude of the main body device 2 over a predetermined time. Therefore, even if the attitude of the main body device 2 changes suddenly, the attitude of the virtual camera VC does not change suddenly, but changes to approach the attitude of the main body device 2 over a certain amount of time. For example, if the virtual camera VC matches the attitude of the main body device 2 at time t0 and the attitude of the main body device 2 changes at time t1, the attitude of the virtual camera VC at time t1 does not completely match the attitude of the main body device 2. The attitude of the virtual camera VC changes by an amount of change smaller than the amount of change in the attitude of the main body device 2. Furthermore, when the attitude of the main body device 2 changes at time t2, the attitude of the virtual camera VC at time t2 does not completely match the attitude of the main body device 2, but changes to approach the attitude of the main body device 2. Thereafter, if the attitude of main device 2 at time t3 is the same as that at time t2, the attitude of virtual camera VC at time t3 approaches even closer to that of main device 2. Then, if the attitude of main device 2 at time t4 is the same as that at time t2, the attitude of virtual camera VC at time t4 will coincide with the attitude of main device 2.

[0149] In this embodiment, the smaller the angle of view of the virtual camera VC (the higher the magnification ratio of the virtual space), the greater the degree of camera shake correction. Here, the degree of camera shake correction is a value indicating how much the change in the attitude of the virtual camera VC is delayed with respect to the change in the attitude of the main unit 2. The smaller the degree of camera shake correction, the faster the attitude of the virtual camera VC approaches the attitude of the main unit 2, and the greater the degree of camera shake correction, the slower the attitude of the virtual camera VC approaches the attitude of the main unit 2. For example, when the angle of view of the virtual camera VC is set to the third angle of view A (first angle of view<third angle of view A<second angle of view), the main unit 2 may set the attitude of the virtual camera VC to 90% of the attitude of the main unit when it detects a change in its own attitude. Also, when the angle of view of the virtual camera VC is set to the second angle of view, the main unit 2 may set the attitude of the virtual camera VC to 80% of the attitude of the main unit when it detects a change in its own attitude. When the angle of view of the virtual camera VC is set to the first angle of view, the degree of correction is "0", and when the main unit 2 detects a change in the attitude of the device itself, it sets the attitude of the virtual camera VC to 100% of the attitude of the device itself.

[0150] Usually, in VR, if the movement of the virtual camera is not matched with the movement of the user (the movement of the goggle device 150, etc.) and the attitude of the virtual camera is corrected, the movement of the real user and the movement of the image in the VR space seen by the user will not match, which may cause VR sickness. For this reason, in the field of VR, the movement of the user is usually matched with the movement of the virtual camera. However, in the image display system 1 of this embodiment, a function of enlarging a part of the VR space by narrowing the angle of view of the virtual camera VC is provided. When the angle of view of the virtual camera VC is narrowed, if the attitude of the virtual camera VC is changed to match the change in the attitude of the goggle device 150, the VR space may shake due to a slight movement of the hand (shake), which may cause VR sickness. For this reason, in this embodiment, when the angle of view of the virtual camera VC is narrowed, the influence of the shake is reduced by the shake correction. As a result, even if the angle of view of the virtual camera VC is narrowed to enlarge a part of the virtual space, VR sickness can be reduced.

[0151] Note that the method of bringing the attitude of the virtual camera closer to the attitude of the main body device 2 is not limited to the above. For example, the attitude of the virtual camera VC may be set based on an attitude (e.g., a weighted average) calculated from the latest attitude of the main body device 2 and the immediately preceding attitude of the main body device 2.

[0152] As another method of camera shake correction, when the change in the attitude of the main body device 2 is equal to or less than a predetermined threshold, the attitude of the virtual camera VC may not be changed, and when the change in the attitude of the main body device 2 exceeds the predetermined threshold, the attitude of the virtual camera VC may be changed. In this case, the smaller the angle of view of the virtual camera VC, the larger the predetermined threshold may be. For example, when the angle of view of the virtual camera VC is a first angle of view, the predetermined threshold may be set to "0", and when the angle of view of the virtual camera VC is a second angle of view, the predetermined threshold may be set to a value larger than "0".

[0153] Regardless of the method of image stabilization, when the angle of view of virtual camera VC is the first angle of view, image stabilization is not performed, and when the attitude of main body device 2 changes, the attitude of virtual camera VC may be set according to the attitude of main body device 2 after the change. On the other hand, when the angle of view of virtual camera VC is the second angle of view, image stabilization is performed to reduce the change in the attitude of virtual camera VC when the attitude of main body device 2 changes, and the attitude of virtual camera VC may be set according to the result of the image stabilization.

[0154] As described above, in this embodiment, the angle of view of the virtual camera VC is reduced to generate a left-eye image and a right-eye image in which a part of the virtual space is enlarged, and a part of the display area of ​​the enlarged left-eye image and the right-eye image is cut. Since the peripheral parts of the enlarged left-eye image and the right-eye image are cut, it is possible to prevent the user from viewing the part where the difference from the real appearance becomes large, and VR sickness can be reduced. In addition, by continuously reducing the area of ​​the display area of ​​the left-eye image and the right-eye image in response to continuously enlarging a part of the virtual space, it is possible to prevent the user from feeling uncomfortable and to prevent a sudden change from occurring in the display of the virtual space. This can reduce the possibility of VR sickness. In addition, it is possible to make the user aware that a part of the virtual space is enlarged, and to make the user view the image with that awareness, making it difficult for VR sickness to occur. Therefore, it is possible to enlarge a part of the VR space and reduce VR sickness.

[0155] In addition, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge a part of the virtual space, the distance between the left virtual camera VCL and the right virtual camera VCR is shortened. This weakens the stereoscopic effect and reduces VR sickness. In addition, by continuously reducing the inter-camera distance d in response to continuously enlarging a part of the virtual space, it is possible to prevent a sudden change in the stereoscopic effect and prevent the user from feeling uncomfortable. This reduces the possibility of VR sickness.

[0156] In addition, in this embodiment, camera shake correction is performed when a part of the virtual space is enlarged. This makes it possible to reduce, for example, the shaking of the VR space caused by camera shake, and to reduce VR sickness. On the other hand, by not performing camera shake correction under normal circumstances, it is possible to control the attitude of the virtual camera VC according to the change in the attitude of the main unit 2. This makes it possible to match the movement of the user with the movement of the virtual camera VC, and to reduce the possibility of VR sickness.

[0157] In this embodiment, a part of the virtual space is enlarged by rotating the cylindrical portion 205, which imitates a telephoto zoom lens of a real camera, in the roll direction. Such an operation is closer to operating a real camera than, for example, a case where a user holds a controller and presses an operation button, tilts an analog stick, or changes the attitude of the controller. By enlarging the virtual space with such an operation similar to the operation of a real camera, it is possible to give the user a sensation of operating a real camera, and VR sickness can be reduced.

[0158] When enlarging a part of the virtual space, instead of reducing the angle of view of the virtual camera VC, it is possible to move the virtual camera VC in the virtual space (to move closer to the enlarged target). In this case, if there is another virtual object in the movement path of the virtual camera VC, the virtual camera VC may pass through or collide with the other virtual object. Furthermore, the user may visually feel as if the user himself is moving even though he is not actually moving. This may cause the user to feel uncomfortable and cause VR sickness.

[0159] In addition, when enlarging a part of the virtual space, it is possible to move the virtual objects and other background closer to the virtual camera VC without moving the virtual camera VC. In this case, the virtual space itself moves, which may cause VR sickness.

[0160] However, in this embodiment, when enlarging a part of the virtual space, the virtual camera VC and the virtual space itself are not moved, but the angle of view of the virtual camera VC is reduced, so that no problems due to such movement of the virtual camera VC or virtual space occur.

[0161] (Processing details) Next, a specific example of the processing performed in the main unit 2 will be described. First, the data stored in the main unit 2 will be described.

[0162] Fig. 26 is a diagram showing an example of data stored in the memory (mainly the DRAM 85) of the main unit 2. As shown in Fig. 26, the memory of the main unit 2 stores a predetermined program, main unit sensor data, main unit attitude data, controller attitude data, virtual object data, virtual camera data, left eye image data, and right eye image data. In addition to these, various other data are stored in the main unit 2.

[0163] The predetermined program is a program for executing the processing according to the flowchart described later. The predetermined program is stored in, for example, the flash memory 84 or a memory card inserted into the slot 23, and is read into the DRAM 85 at the start of the processing. The predetermined program may be obtained from another device via a network (for example, a LAN, a WAN, the Internet, etc.).

[0164] The main body sensor data is data output from the acceleration sensor 89 and the angular velocity sensor 90 of the main body device 2, and includes data related to acceleration and data related to angular velocity. The main body sensor data is output from the acceleration sensor 89 and the angular velocity sensor 90 at predetermined time intervals (for example, 1 / 800 second intervals) and stored in memory.

[0165] The main body attitude data is data calculated based on main body sensor data (data related to acceleration and / or data related to angular velocity) and indicates the attitude of the main body device 2. For example, the main body device 2 calculates a vector representing the attitude of the main body device 2 based on the angular velocity value from the angular velocity sensor 90, and stores the vector in memory as the main body attitude data. The main body attitude data is also calculated at a predetermined time interval (for example, 1 / 200 second intervals).

[0166] The controller attitude data is data related to the attitude of the controllers (left controller 3 and right controller 4). The main unit 2 acquires controller attitude data from each controller at a predetermined time interval (for example, 1 / 200 second intervals) and stores the acquired controller attitude data in memory. The main unit 2 may acquire angular velocity values ​​and / or acceleration values ​​detected by the inertial sensors (angular velocity sensors and acceleration sensors) of each controller from each controller as controller attitude data. Alternatively, each controller may calculate an attitude based on the angular velocity values ​​and / or acceleration values, and transmit data related to the calculated attitude to the main unit 2.

[0167] The virtual object data is data relating to various virtual objects arranged in a virtual space, and includes data on the position, shape, etc. of each virtual object.

[0168] The virtual camera data is data relating to the left virtual camera VCL and the right virtual camera VCR, and includes data indicating the position, attitude, and angle of view of each virtual camera VC.

[0169] The left-eye image data is an image of the virtual space generated based on the left virtual camera VCL, and is data related to the left-eye image displayed in the left display area of ​​the display 12. The right-eye image data is an image of the virtual space generated based on the right virtual camera VCR, and is data related to the right-eye image displayed in the right display area of ​​the display 12.

[0170] (Flowchart explanation) Next, a detailed description will be given of the processing performed in the main unit 2. Fig. 27 is a flowchart showing an example of the processing performed in the processor 81 of the main unit 2. Note that Fig. 27 shows only the above-mentioned processing, and omits other processing (for example, game processing according to the user's actions performed in a virtual space, etc.).

[0171] As shown in FIG. 27, the processor 81 first performs an initial process (step S100). In the initial process, the attitude of the image display system 100 (main unit 2) is initialized. For example, in the initial process, an instruction is given to a user to place the image display system 100 including the main unit 2 on a table or the like, and the attitude of the main unit 2 is initialized. In the initial process, an XYZ coordinate system is set in the virtual space, and each virtual object, the left virtual camera VCL, the right virtual camera VCR, and the like are arranged in the virtual space. After the process of step S100, the processor 81 repeatedly executes the processes from step S101 onward at predetermined frame times (for example, 1 / 60 seconds).

[0172] After the process of step S100, the processor 81 acquires main body sensor data (step S101). In addition, in step S101, the processor 81 acquires controller attitude data.

[0173] Next, the processor 81 calculates the attitude of the main body device 2 based on the main body sensor data acquired in step S101 (step S102). Specifically, the processor 81 acquires the attitude of the main body device 2 (goggle device 150) by integrating the latest angular velocity value output by the angular velocity sensor 90 to calculate the attitude. The acquired attitude of the main body device 2 is stored in the memory as main body attitude data.

[0174] Next, the processor 81 determines whether or not the cylindrical portion 205 of the camera device 200 has rotated in the roll direction (step S103). Specifically, the processor 81 determines whether or not the left controller 3 (or the right controller 4) provided inside the cylindrical portion 205 has rotated around the y axis based on the controller attitude data acquired in step S101.

[0175] When it is determined that the cylindrical part 205 (left controller 3) is not rotating (step S103: NO), the processor 81 sets the attitude of the virtual camera VC according to the attitude of the main body device 2 acquired (calculated) in step S102 (step S104). Specifically, the processor 81 sets the attitudes of the left virtual camera VCL and the right virtual camera VCR so as to match the attitude of the main body device 2, and stores them in the memory as attitude data of the virtual camera data. Note that, when the cylindrical part 205 is not rotating, the processor 81 sets the positions of the left virtual camera VCL and the right virtual camera VCR so that the inter-camera distance d is a predetermined value (for example, the distance between the left and right eyes of an average person), and stores them in the memory as position data of the virtual camera data. Also, when the cylindrical part 205 is not rotating, the processor 81 sets the angles of view of the left virtual camera VCL and the right virtual camera VCR to a first angle of view (for example, 90 degrees), and stores them in the memory as angle of view data of the virtual camera data.

[0176] Next, the processor 81 generates a left-eye image and a right-eye image based on the left virtual camera VCL and the right virtual camera VCR (step S105). Here, the angle of view of the virtual camera VC is set to the first angle of view, and an image (a left-eye image and a right-eye image) of a virtual space having a generally circular normal size is generated. In addition, in step S105, the processor 81 performs distortion correction according to the characteristics of the lens 153. As a result, a left-eye image and a right-eye image that have been subjected to distortion correction are generated.

[0177] Following step S105, the processor 81 causes the left eye image and the right eye image generated in step S105 to be displayed on the display 12 (step S106). As a result, the left eye image and the right eye image in the virtual space are displayed in the left display region and the right display region, respectively, of the display 12. Note that a black image is displayed in the non-visual region.

[0178] On the other hand, when it is determined that the cylindrical portion 205 is rotating (step S103: YES), the processor 81 performs an enlarged image display process (step S107). The enlarged image display process of step S107 will be described later.

[0179] When the process of step S106 has been executed or when the process of step S107 has been executed, the processor 81 determines whether or not to end the process shown in Fig. 27 (step S108). For example, when a game is played in a VR space, the processor 81 determines to end the process shown in Fig. 27 when the game is cleared or when an instruction to end the game is given by the user. When the determination is NO in step S108, the processor 81 executes the process of step S101 again.

[0180] (Enlarged image display processing) Next, the enlarged image display process of step S107 will be described in detail with reference to Fig. 28. Fig. 28 is a flow chart showing an example of the enlarged image display process of step S107.

[0181] 28, the processor 81 sets the angle of view of the virtual camera VC (step S121). Specifically, the processor 81 sets the angles of view of the left virtual camera VCL and the right virtual camera VCR in accordance with the rotation angle of the left controller 3 (the cylindrical portion 205) around the y axis.

[0182] Next, the processor 81 enables the image stabilization function (step S122). Specifically, the processor 81 sets the degree of image stabilization according to the angle of view set in step S121, and enables the image stabilization function. The processor 81 sets the degree of correction so that the degree of image stabilization increases as the angle of view of the virtual camera VC decreases (i.e., as the magnification rate increases).

[0183] Following step S122, the processor 81 sets the attitude of the virtual camera VC based on the attitude of the main unit 2 acquired (calculated) in step S102 (step S123). Specifically, the processor 81 sets the attitudes of the left virtual camera VCL and the right virtual camera VCR according to the degree of image stabilization set in step S122 so as to approach the attitude of the main unit 2. As a result, the smaller the angle of view of the virtual camera VC (the higher the magnification ratio), the more slowly the attitude of the virtual camera VC approaches the attitude of the main unit 2.

[0184] Next, the processor 81 sets the inter-camera distance d in accordance with the angle of view set in step S121 (step S124).

[0185] Next, the processor 81 generates a left eye image and a right eye image by enlarging a part of the virtual space based on the left virtual camera VCL and the right virtual camera VCR (step S125). Here, a left eye image and a right eye image of normal size by enlarging a part of the virtual space are generated. In addition, in step S125, the processor 81 performs distortion correction according to the characteristics of the lens 153. As a result, a left eye image and a right eye image that have been subjected to distortion correction are generated. The distortion correction in this step S125 is the same process as the distortion correction in step S105 described above.

[0186] Following step S125, the processor 81 determines whether or not the angle of view of the virtual camera VC set in step S121 is equal to or smaller than a predetermined value (the third angle of view A in FIG. 22) (step S126). Note that the processor 81 may determine whether or not the angle of view of the virtual camera VC is less than the predetermined value in step S126.

[0187] When it is determined that the angle of view of the virtual camera VC is equal to or smaller than a predetermined value (step S126: YES), the processor 81 reduces the left and right display areas in accordance with the angle of view of the virtual camera VC (step S127). For example, the processor 81 reduces the display area of ​​the virtual space image by filling in the outer periphery of the normal-sized virtual space image (left-eye image and right-eye image) generated in the processing of step S125 with black. The smaller the angle of view of the virtual camera VC, the smaller the area of ​​the display area of ​​the virtual space image is made by the processor 81.

[0188] Next, the processor 81 causes the display 12 to display the left eye image and the right eye image (step S128). As a result, the left eye image and the right eye image, which are enlarged portions of the virtual space with the outer periphery cut off, are displayed in the left display region and the right display region, respectively, of the display 12. Note that a black image is displayed in the non-visual region.

[0189] On the other hand, if the angle of view of the virtual camera VC is not equal to or smaller than the predetermined value (step S126: NO), the processor 81 causes the display 12 to display the left-eye image and the right-eye image of normal size generated in the process of step S125 (step S129). Here, the process of reducing the display area of ​​the left and right images as in step S127 is not performed. As a result, the left-eye image and the right-eye image, which are the same size as the images in step S106 but are an enlarged portion of the virtual space, are displayed. Note that a black image is displayed in the non-visual area.

[0190] When the process of step S128 has been executed, or when the process of step S129 has been executed, the processor 81 ends the process shown in Fig. 28, and returns the process to Fig. 27. This concludes the description of the flowcharts shown in Fig. 27 and Fig. 28.

[0191] It should be noted that the processes shown in the above flowcharts are merely examples, and the order and content of the processes may be changed as appropriate.

[0192] (Modification) The image display system of this embodiment has been described above, but the above embodiment is merely an example, and the following modifications may be made, for example.

[0193] For example, in the above embodiment, a left-eye image and a right-eye image having a parallax are generated based on a pair of a left virtual camera VCL and a right virtual camera VCR. In another embodiment, an image may be generated based on one virtual camera, and the generated image may be deformed to generate a left-eye image and a right-eye image having a parallax. That is, in this specification, "generating a left-eye image and a right-eye image based on a virtual camera" includes both generating a left-eye image and a right-eye image based on a pair of a left virtual camera VCL and a right virtual camera VCR, and generating a left-eye image and a right-eye image based on one virtual camera.

[0194] Even when generating a left-eye image and a right-eye image having a parallax with each other using one virtual camera without using left and right virtual cameras, the left-eye image and the right-eye image are generated so that the parallax is different between normal and enlarged. Specifically, in normal, a left-eye image and a right-eye image having a first parallax are generated based on one virtual camera, and in enlarged, a left-eye image and a right-eye image having a second parallax smaller than the first parallax are generated based on one virtual camera.

[0195] In the above embodiment, generally circular images are displayed as the left eye image and the right eye image in the normal state, and generally similar images (e.g., generally regular dodecagonal images) are displayed in the enlarged state. The shapes of the left eye image and the right eye image are merely examples, and images of any other shape may be displayed. For example, polygonal left eye image and right eye image may be displayed in the normal state, and left eye image and right eye image with a small display area may be displayed in the enlarged state.

[0196] In the above embodiment, the outer periphery of the left eye image and the right eye image, which are obtained by enlarging a part of the virtual space, is painted black to cut the outer periphery, thereby reducing the display area of ​​the left eye image and the right eye image. In other embodiments, other methods may be used to reduce the display area of ​​the left eye image and the right eye image, which are obtained by enlarging a part of the virtual space. For example, an image of a predetermined color may be displayed on the outer periphery of the left eye image and the right eye image, which are obtained by enlarging a part of the virtual space, or the outer periphery may be made semi-transparent or blurred, so that the image of the virtual space corresponding to the outer periphery cannot be or is difficult to see. Also, the outer periphery of the left eye image and the right eye image, which are obtained by enlarging a part of the virtual space, may be hidden. Also, for example, the display area of ​​the left eye image and the right eye image, which are obtained by enlarging a part of the virtual space, may be reduced by arranging an occlusion object (for example, a donut-shaped object that is transparent at the center and occludes the outer periphery) that blocks a part of the field of view of the virtual camera VC. When a virtual space including such an occluding object is captured by a virtual camera, an image of a part of the virtual space displayed in normal times is enlarged and displayed in the center part, while the peripheral part is occluded by the occluding object, and the part of the virtual space displayed in normal times is not displayed (or is difficult for the user to see). The image (peripheral image) in the peripheral part of the left eye image and the right eye image may be an image in which the virtual space is made semi-transparent, an image in which the virtual space is blurred, an image of an occluding object arranged in the virtual space, or an image of a predetermined color that is not arranged in the virtual space. In other words, the image for the left eye and the image for the right eye may be generated by any other method as long as a part of the virtual space displayed in normal times is enlarged and displayed, and other parts of the virtual space displayed in normal times are displayed so as to be invisible or difficult to see when enlarged.

[0197] In the above embodiment, when enlarged display is set (for example, in the case of the second angle of view), a predetermined image (peripheral image, for example, a black image) is displayed on the outer periphery of the left eye image and the right eye image which are enlarged portions of the virtual space, and when enlarged display is not set (for the first angle of view), the predetermined image is not displayed on the outer periphery of the left eye image and the right eye image. In other embodiments, even when enlarged display is not set, a predetermined image may be displayed on the outer periphery of the left eye image and the right eye image. In this case, when enlarged display is not set, the predetermined image displayed on the outer periphery may be smaller than when enlarged display is set.

[0198] In the above embodiment, the angle of view of the virtual camera is set to be smaller than the normal first angle of view by rotating the cylindrical portion 205 (a part of the virtual space is displayed in a more enlarged scale than normal). In other embodiments, the angle of view of the virtual camera may be displayed in a larger scale by any other operation. For example, the magnification ratio of a part of the virtual space may be changed continuously by operating a button or analog stick of the left controller 3 or the right controller 4. In this case, the distance between the cameras (parallax between the left eye image and the right eye image) may also be changed continuously in accordance with the change in the magnification ratio of a part of the virtual space. In addition, the zoom-up may be set (i.e., set to the second angle of view) by operating a button (or analog stick) of the controller, and the zoom-up may be released (i.e., set to the first angle of view) by operating the same or a different button (or analog stick). In this case, the distance between the cameras (parallax between the left eye image and the right eye image) may also be changed in accordance with the setting and release of the zoom-up.

[0199] In this specification, "continuously changing" the angle of view or inter-camera distance (parallax) of the virtual camera includes changing it curvilinearly as shown in FIG. 22, and changing it linearly as shown in FIG. 23.

[0200] In the above embodiment, the angle of view of the virtual camera VC is changed continuously from the first angle of view to the second angle of view, thereby continuously enlarging a part of the virtual space. In another embodiment, the angle of view of the virtual camera VC may be set to the first angle of view or a second angle of view smaller than the first angle of view. That is, the angle of view of the virtual camera VC may be set to the first angle of view during normal operation, and the angle of view of the virtual camera VC may be set to the second angle of view during enlargement. In this case, when the angle of view of the virtual camera VC is set to the second angle of view, the area of ​​the display region of the left eye image and the right eye image is made smaller than when the angle of view of the virtual camera VC is set to the first angle of view.

[0201] In the above embodiment, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is set to zero. That is, when the magnification is maximum, the parallax between the left eye image and the right eye image is set to zero. In another embodiment, when the angle of view of the virtual camera VC is set to the second angle of view, the parallax between the left eye image and the right eye image may be set to substantially zero. Here, the state where the parallax is substantially zero includes a state where there is no parallax at all and a state where there is a small parallax but the image appears to the user as being substantially flat.

[0202] In the above embodiment, a part of the virtual space is enlarged by reducing the angle of view of the virtual camera VC. In other embodiments, a part of the virtual space may be enlarged by other methods. In this case, the camera distance d may be shortened in response to enlarging a part of the virtual space. For example, an image of the virtual space may be generated based on the virtual camera VC set to the first angle of view, a part of the generated image may be cut out and enlarged, and the enlarged image may be displayed as an image for the left eye and an image for the right eye. In this case, the parallax of the image for the left eye and the image for the right eye obtained by enlarging a part of the virtual space is made smaller than the parallax before enlargement. Even when enlarging a part of the virtual space by other methods in this way, the enlargement ratio of the virtual space may be continuously changed, and the parallax (for example, the camera distance d) of the image for the left eye and the image for the right eye may be reduced in response to the enlargement ratio.

[0203] In the above embodiment, the attitude of the goggle device 150 (main device 2) is detected based on data from an inertial sensor (angular velocity sensor 90 and / or acceleration sensor 89) built into the main device 2. In other embodiments, the attitude of the goggle device 150 may be detected by other methods. For example, the image display system 100 may include a camera that captures an image of the goggle device 150 from the outside, capture an image of the goggle device 150 or a marker attached to the goggle device 150 with the camera, and obtain the attitude of the goggle device 150 based on the captured image. Alternatively, the goggle device 150 may include a camera, and the attitude of the goggle device 150 may be obtained from a change in the image captured by the camera.

[0204] The image display system 100 is not limited to the above-mentioned configuration, and may have other configurations. For example, the goggle device 150 and the camera device 200 may be integrally configured so as not to be separated. That is, in the above-mentioned embodiment, the goggle device 150 has a detachable cylindrical part 205, but in other embodiments, the goggle device 150 may have a fixed cylindrical part 205. For example, the cylindrical part 205 may be provided on the back side of the goggle device. The goggle device may be used in various states, such as a type that fits the user's face by being held by the user, a type that fits the user's face by being fixed to the user's head, or a type in which the user looks into the device in a placed state. The goggle device included in the image display system may function as a so-called head-mounted display by being worn on the user's head with the main device 2 attached, and may have a helmet-type shape in addition to a goggle-type shape.

[0205] In the above configuration, the goggle device and the main unit 2 are detachable, but in other configurations, the goggle device and the main unit 2 may be integrated together. That is, the goggle device may be provided with a display for displaying a left-eye image and a right-eye image, and a processor for generating images.

[0206] In the above embodiment, the display 12 of the goggle device 150 is detachable from the goggle device 150. In other embodiments, the display unit of the goggle device may be fixed to the goggle device. The display unit of the goggle device is not limited to the rectangular display 12 as described above. For example, the display unit of the goggle device may have two display units (a left-eye display unit viewed by the left eye of the user and a right-eye display unit viewed by the right eye of the user). The display unit of the goggle device may have any shape. For example, the display unit of the goggle device itself may be formed in a substantially circular shape (circular or elliptical). Two left and right display units formed in a square or rectangular shape may be used as the display unit of the goggle device. The display unit of the goggle device may be a display device such as a liquid crystal display device or an organic EL display device, or may be a projection type display device that projects an image onto a projection surface.

[0207] The configuration of the image display system 1 of the above embodiment is merely an example, and is not limited to the above. For example, in the above embodiment, the goggle device is configured by the main device 2 having the display 12 for displaying an image and the processor 81 for performing processing for generating an image, and the goggle body. That is, the image display system 100 is configured by the goggle device including the goggle body and the main device 2. In another embodiment, the goggle device having a display unit and the information processing device performing processing for generating an image may be configured as separate devices, and the image display system 100 may be configured by these multiple devices. In this case, the goggle device and the information processing device may be connected by wire or wirelessly, and the left eye image and the right eye image generated in the information processing device may be transmitted to the goggle device and viewed by the user. Also, the information processing device may perform the above-mentioned processing, transmit the result of the processing to the goggle device, and the goggle device may generate the left eye image and the right eye image to be viewed by the user. Also, the goggle device and the information processing device may be connected by a network (LAN, WAN, Internet, etc.).

[0208] In addition, in this embodiment, a goggle device is used in which the user looks into the display while holding it in their hands, but in other embodiments, the goggle device may be a head-mounted display that is fixedly worn on the user's head.

[0209] Although the present invention has been described above, the above description is merely an example of the present invention, and various improvements and modifications may be made. [Explanation of symbols]

[0210] 1. Game System 2. Main Unit 3 Left Controller 4 Right Controller 12 Display 81 Processor 89 Acceleration Sensor 90 Angular Rate Sensor 100 Image display system 150 Goggle device 200 Camera Equipment 205 Cylindrical part VC Virtual Camera VCL Left Virtual Camera VCR Right Virtual Camera

Claims

1. A goggle device; a virtual camera arrangement means for arranging a virtual camera in a virtual space; a view angle setting means for setting an angle of view of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image having a parallax with each other, the left-eye image and the right-eye image being images of the virtual space included in an angle of view of the virtual camera; a display control means for causing the left eye image and the right eye image to be displayed on a display unit of the goggle device; a parallax setting means for setting a parallax between the left eye image and the right eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and for setting a parallax between the left eye image and the right eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, The image generating means includes: generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; generating the left-eye image and the right-eye image, which are images obtained by enlarging a part of the virtual space when the angle of view of the virtual camera is set to the second angle of view, and which have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when an angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance to set the first parallax; An image display system that sets the second parallax by setting a virtual distance between the left eye virtual camera and the right eye virtual camera to a second distance shorter than the first distance when the angle of view of the virtual camera is set to the second angle of view.

2. the angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, The parallax setting means The image display system of claim 1 , wherein when the angle of view of the virtual camera changes continuously from the first angle of view to the second angle of view, the virtual distance between the left-eye virtual camera and the right-eye virtual camera is continuously shortened.

3. The parallax setting means 3. The image display system according to claim 2, wherein the virtual distance between the left-eye virtual camera and the right-eye virtual camera is changed linearly in response to a change in an angle of view of the virtual camera.

4. the angle of view setting means reduces the angle of view of the virtual camera within a range from the first angle of view to the second angle of view, the image generating means generates the left eye image and the right eye image by increasing a magnification ratio of a part of the virtual space in response to a reduction in the angle of view of the virtual camera; 4. The image display system according to claim 1, wherein the parallax setting means reduces the parallax between the left-eye image and the right-eye image as the magnification ratio increases.

5. the goggle device includes lenses for allowing the left eye image and the right eye image to be viewed by a left eye and a right eye of a user, respectively; The image display system according to claim 1 , wherein the first angle of view is set to a viewing angle of the user using the lens.

6. further comprising an object placement means for placing a predetermined virtual object in the virtual space; The display control means generating the left-eye image and the right-eye image including an image of the predetermined virtual object having a first size when the angle of view of the virtual camera is set to the first angle of view; 6. The image display system according to claim 1, wherein when the angle of view of the virtual camera is set to the second angle of view, the image for the left eye and the image for the right eye are generated, the image including an image of the specified virtual object having a second size larger than the first size.

7. 7. The image display system according to claim 1, wherein the position of the virtual camera in the virtual space is the same when the angle of view of the virtual camera is set to the first angle of view and when the angle of view is set to the second angle of view.

8. A detection means for detecting the attitude of the goggle device; a virtual camera attitude control means for controlling an attitude of the virtual camera based on an attitude of the goggle device; and a change setting means for setting a degree of change in the attitude of the virtual camera in response to a change in the attitude of the goggle device, The virtual camera attitude control means 8. An image display system as described in any one of claims 1 to 7, wherein when the angle of view of the virtual camera is set to the second angle of view and the attitude of the goggle device changes, the attitude of the virtual camera is controlled so as to approach the attitude of the goggle device with a degree of change set by the change setting means.

9. A detection means for detecting the attitude of the goggle device; a virtual camera attitude control means for controlling an attitude of the virtual camera based on an attitude of the goggle device, The virtual camera attitude control means when an attitude of the goggle device changes in a case where the angle of view of the virtual camera is set to the first angle of view, controlling an attitude of the virtual camera in accordance with the attitude of the goggle device after the change; The image display system of claim 7, wherein when the angle of view of the virtual camera is set to the second angle of view, when the attitude of the goggle device changes, a correction process is performed to reduce the change in attitude of the virtual camera, and the attitude of the virtual camera is controlled in accordance with the correction process.

10. The image generating means includes: generating a first left-eye image and a first right-eye image representing a portion of the virtual space when the angle of view of the virtual camera is set to the first angle of view; generating a second left-eye image and a second right-eye image by enlarging a part of the virtual space when the angle of view of the virtual camera is set to the second angle of view; The display control means When the angle of view of the virtual camera is set to the first angle of view, the first left-eye image and the first right-eye image are displayed in a display area of ​​a first size, 10. An image display system as described in any one of claims 1 to 9, wherein when the angle of view of the virtual camera is set to the second angle of view, the second left eye image and the second right eye image are each displayed in a display area of ​​a second size smaller than the first size.

11. 11. The image display system according to claim 10, wherein the display area of ​​the second size is an area having a shape substantially similar to that of the display area of ​​the first size.

12. 12. The image display system according to claim 10, wherein the display area of ​​the second size is a central area excluding at least an outer periphery of the display area of ​​the first size.

13. the angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, 13. The image display system according to claim 10, wherein the display control means continuously reduces the display areas of the left eye image and the right eye image, which are enlarged portions of the virtual space, in response to the angle of view of the virtual camera being continuously changed from the first angle of view to the second angle of view.

14. the angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, The display control means 14. The image display system according to claim 10, wherein the display areas of the left eye image and the right eye image, which are enlarged portions of the virtual space, are changed to be approximately similar shapes and the display areas are continuously reduced in size in response to the angle of view of the virtual camera being continuously changed from the first angle of view to the second angle of view.

15. An image display program executed by a processor of a device that displays an image on a display unit of a goggle device, the processor comprising: a virtual camera arrangement means for arranging a virtual camera in a virtual space; a view angle setting means for setting an angle of view of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image having a parallax with each other, the left-eye image and the right-eye image being included in an angle of view of the virtual camera, in order to display the left-eye image and the right-eye image on a display unit of the goggle device; a parallax setting unit that sets a parallax between the left eye image and the right eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and that sets a parallax between the left eye image and the right eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, The image generating means includes: generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; generating, when a field angle of the virtual camera is set to the second field angle, the left-eye image and the right-eye image, which are images obtained by enlarging a part of the virtual space and have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when an angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance to set the first parallax; An image display program that sets the second parallax by setting a virtual distance between the left eye virtual camera and the right eye virtual camera to a second distance shorter than the first distance when the angle of view of the virtual camera is set to the second angle of view.

16. An image display method performed in an image display system including a goggle device, the image display system comprising: a virtual camera placement step of placing a virtual camera in a virtual space; a view angle setting step of setting an angle of view of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating step of generating a left-eye image and a right-eye image, the left-eye image and the right-eye image having a parallax with each other, of the virtual space included in an angle of view of the virtual camera, to be displayed on a display unit of the goggle device; a parallax setting step of setting a parallax between the left eye image and the right eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and setting a parallax between the left eye image and the right eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view; In the image generating step, generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; generating, when a field angle of the virtual camera is set to the second field angle, the left-eye image and the right-eye image, which are images obtained by enlarging a part of the virtual space and have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; In the disparity setting step, when an angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance to set the first parallax; An image display method, in which when the angle of view of the virtual camera is set to the second angle of view, the second parallax is set by setting a virtual distance between the left eye virtual camera and the right eye virtual camera to a second distance shorter than the first distance.

Images