Image display system, image display program, image display device, and image display method

Abstract

To provide an image display system capable of reducing possibility of VR sickness.SOLUTION: One example of an image display system includes a goggle device having a display unit. The image display system includes a virtual camera disposed in a virtual space. An angle of view of the virtual camera can be set to a first angle of view and a second angle of view smaller than the first angle of view. By setting the angle of view of the virtual camera to the second angle of view, part of the virtual space is enlarged and displayed. The image display system sets a parallax between a left-eye image and a right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and sets the parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to an image display system, an image display program, an image display device, and an image display method that are capable of displaying stereoscopic images. [Background technology]

[0002] Prior art includes a display control system that places a virtual camera in a virtual space, generates an image for the left eye and an image for the right eye based on the virtual camera, and allows the user to view a stereoscopic image with their left eye and right eye, respectively (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-135771 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a user is allowed to view a stereoscopic image using a goggle device and experience virtual reality (VR), there is room for improvement in terms of reducing the possibility of VR sickness.

[0005] Therefore, an object of the present invention is to provide an image display system, an image display program, an image display device, and an image display method that can reduce the possibility of VR sickness. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention employs the following configuration.

[0007] The image display system of the present invention includes a goggle device, a virtual camera positioning means, a field of view setting means, an image generation means, a display control means, and a parallax setting means. The virtual camera positioning means positions a virtual camera in a virtual space. The field of view setting means sets the field of view of the virtual camera to at least one of a first field of view and a second field of view smaller than the first field of view. The image generation means generates a left-eye image and a right-eye image, which are images of the virtual space included in the field of view of the virtual camera and have parallax with respect to each other. The display control means displays the left-eye image and the right-eye image on a display unit of the goggle device. When the field of view of the virtual camera is set to the first field of view, the parallax setting means sets the parallax of the left-eye image and the right-eye image to a first parallax, and when the field of view of the virtual camera is set to the second field of view, the parallax of the left-eye image and the right-eye image to a second parallax smaller than the first parallax. When the angle of view of the virtual camera is set to the first angle of view, the image generation means generates the image for the left eye and the image for the right eye, which have the first parallax relative to each other, and when the angle of view of the virtual camera is set to the second angle of view, the image generation means generates the image for the left eye and the image for the right eye, which are images that are enlarged portions of the virtual space and have the second parallax relative to each other.

[0008] According to the above, by setting the angle of view of the virtual camera to the second angle of view, a portion of the virtual space can be enlarged and displayed. When enlarging a portion of the virtual space, the parallax between the left-eye image and the right-eye image is made smaller than normal. This reduces the stereoscopic effect when enlarged, thereby reducing the possibility of VR sickness.

[0009] The virtual camera may include a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image. The parallax setting means may set a virtual distance between the left-eye virtual camera and the right-eye virtual camera to a first distance when the angle of view of the virtual camera is set to the first angle of view, and may set a virtual distance between the left-eye virtual camera and the right-eye virtual camera to a second distance shorter than the first distance when the angle of view of the virtual camera is set to the second angle of view.

[0010] According to the above, when the virtual space is expanded, the distance between the left-eye virtual camera and the right-eye virtual camera is narrowed, thereby making it possible to reduce the parallax between the left-eye image and the right-eye image.

[0011] Furthermore, the angle-of-view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The parallax setting means may continuously shorten the virtual distance between the left-eye virtual camera and the right-eye virtual camera when the angle of view of the virtual camera continuously changes from the first angle of view to the second angle of view.

[0012] According to the above, by continuously shortening the distance between the left-eye virtual camera and the right-eye virtual camera in response to continuously enlarging a portion of the virtual space, it is possible to prevent sudden changes in the three-dimensional effect, thereby reducing the possibility of VR sickness.

[0013] The parallax setting means may linearly change the virtual distance between the left-eye virtual camera and the right-eye virtual camera in accordance with a change in the angle of view of the virtual camera.

[0014] According to the above, the distance between the left-eye virtual camera and the right-eye virtual camera can be changed linearly in accordance with changes in the angle of view of the virtual camera, thereby weakening the three-dimensional effect so that it does not feel strange to the user.

[0015] Furthermore, the parallax setting means may set the second parallax to substantially zero when the angle of view of the virtual camera is set to the second angle of view.

[0016] According to the above, when a part of the virtual space is enlarged to the maximum, a flat image can be displayed, thereby reducing the possibility of VR sickness.

[0017] Furthermore, the angle of view setting means may further set the angle of view of the virtual camera to a third angle of view smaller than the first angle of view and larger than the second angle of view. When the angle of view of the virtual camera is set to the third angle of view, the parallax setting means may further set the parallax between the left-eye image and the right-eye image to a third parallax smaller than the first parallax and larger than the second parallax. When the angle of view of the virtual camera is set to the third angle of view, the image generating means may generate the left-eye image and the right-eye image, which are images obtained by enlarging a portion of the virtual space and have the third parallax from each other.

[0018] Based on the above, the angle of view of the virtual camera can be set to a third angle of view between the first angle of view and the second angle of view, and a part of the virtual space can be enlarged.

[0019] The angle-of-view setting means may reduce the angle of view of the virtual camera within a range from the first angle of view to the second angle of view. The image generating means may increase a magnification rate of a portion of the virtual space in response to the reduction in the angle of view of the virtual camera, thereby generating the left-eye image and the right-eye image. The parallax setting means may reduce the parallax of the left-eye image and the right-eye image in response to the increase in the magnification rate.

[0020] Based on the above, the parallax between the image for the left eye and the image for the right eye can be reduced as the magnification rate of the virtual space increases, thereby reducing the possibility of VR sickness during magnification.

[0021] The goggle device may further include lenses for allowing the left-eye image and the right-eye image to be viewed by the left eye and the right eye, respectively, of a user, and the first angle of view may be set to a viewing angle of the user using the lenses.

[0022] According to the above, when the virtual space is not enlarged, the user's viewing angle and the virtual camera's viewing angle can be matched, allowing the user to view a natural image of the VR space.

[0023] The image display system may further include an object placement unit that places a predetermined virtual object in the virtual space. When the angle of view of the virtual camera is set to the first angle of view, the display control unit may generate the left-eye image and the right-eye image including an image of the predetermined object at a first size, and when the angle of view of the virtual camera is set to the second angle of view, the display control unit may generate the left-eye image and the right-eye image including an image of the predetermined object at a second size larger than the first size.

[0024] Based on the above, it is possible to enlarge and display a predetermined object in a virtual space, and also to reduce the possibility of VR sickness when the object is enlarged.

[0025] Furthermore, the position of the virtual camera in the virtual space may be the same when the angle of view of the virtual camera is set to the first angle of view and when the angle of view is set to the second angle of view.

[0026] According to the above, it is possible to enlarge a part of the virtual space by changing the angle of view of the virtual camera while maintaining the position of the virtual camera. This allows the part of the virtual space to be enlarged and reduces the possibility of VR sickness at the time of enlargement.

[0027] The image display system may further include a detection means for detecting an attitude of the goggle device, a virtual camera attitude control means for controlling an attitude of the virtual camera based on the attitude of the goggle device, and a change setting means for setting a degree of change in the attitude of the virtual camera in response to a change in the attitude of the goggle device. When the angle of view of the virtual camera is set to the second angle of view and the attitude of the goggle device changes, the virtual camera attitude control means may control the attitude of the virtual camera so that the attitude approaches the attitude of the goggle device by the degree of change set by the change setting means.

[0028] According to the above, when a portion of the virtual space is enlarged and displayed, if the attitude of the goggle device changes, the attitude of the virtual camera VC can be changed so as to approach the attitude of the goggle device by a set degree. This makes it possible to reduce the change in the attitude of the virtual camera VC even when the attitude of the goggle device changes, and to prevent shaking of the virtual space due to, for example, camera shake. This reduces the possibility of VR sickness.

[0029] In addition, when the angle of view of the virtual camera is set to the second angle of view, the change setting means may set the degree of change in the attitude of the virtual camera in response to a change in the attitude of the goggle device to be smaller than when the angle of view of the virtual camera is set to the first angle of view.

[0030] According to the above, when a portion of the virtual space is enlarged, the degree of change in the posture of the virtual camera is smaller than normal, thereby reducing the possibility of VR sickness when enlarging and displaying a portion of the virtual space.

[0031] The image display system may further include a detection unit that detects an attitude of the goggle device, and a virtual camera attitude control unit that controls an attitude of the virtual camera based on the attitude of the goggle device. When the angle of view of the virtual camera is set to the first angle of view, the virtual camera attitude control unit may control the attitude of the virtual camera in accordance with the attitude of the goggle device after the change when the attitude of the goggle device changes, and when the angle of view of the virtual camera is set to the second angle of view, the virtual camera attitude control unit may perform a correction process to reduce the change in the attitude of the virtual camera when the attitude of the goggle device changes, and control the attitude of the virtual camera in accordance with the correction process.

[0032] According to the above, during normal operation, no correction processing is performed, and when the attitude of the goggle device changes, the attitude of the virtual camera is controlled according to the changed attitude of the goggle device. On the other hand, during enlargement, correction processing is performed to reduce changes in the attitude of the virtual camera. Therefore, during normal operation, the attitude of the virtual camera is controlled to match the user's movements, thereby reducing the possibility of VR sickness. On the other hand, during enlargement, by reducing changes in the attitude of the virtual camera, it is possible to prevent shaking of the virtual space due to, for example, camera shake, and reduce the possibility of VR sickness.

[0033] Furthermore, the image generation means may generate a first left-eye image and a first right-eye image representing a portion of the virtual space when the angle of view of the virtual camera is set to the first angle of view, and may generate a second left-eye image and a second right-eye image that are enlarged portions of the virtual space when the angle of view of the virtual camera is set to the second angle of view. The display control means may display each of the first left-eye image and the first right-eye image in a display area of ​​a first size when the angle of view of the virtual camera is set to the first angle of view, and may display each of the second left-eye image and the second right-eye image in a display area of ​​a second size smaller than the first size when the angle of view of the virtual camera is set to the second angle of view.

[0034] Based on the above, by reducing the display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, it is possible to reduce the possibility of VR sickness.

[0035] The display area of ​​the second size may be an area having a shape substantially similar to that of the display area of ​​the first size.

[0036] According to the above, even when the angle of view of the virtual camera is set to the second angle of view, the left eye image and the right eye image are displayed in a shape that is approximately similar to when the angle of view is set to the first angle of view, so that when a portion of the virtual space is enlarged and displayed, the user does not feel uncomfortable, and the possibility of VR sickness can be reduced.

[0037] The display area of ​​the second size may be a central area of ​​the display area of ​​the first size excluding at least the outer periphery thereof.

[0038] According to the above, when a portion of the virtual space is enlarged and displayed, the central portions of the left-eye image and the right-eye image are displayed, excluding the peripheral portions. When the left-eye image and the right-eye image are viewed through the lenses of the goggle device, the peripheral portions appear more unnatural, but because the peripheral portions are excluded, the possibility of VR sickness can be reduced.

[0039] The angle-of-view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The display control means may continuously reduce display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, in response to the continuously changing angle of view of the virtual camera from the first angle of view to the second angle of view.

[0040] According to the above, by continuously enlarging a portion of the virtual space and continuously reducing the display area of ​​the image for the left eye and the image for the right eye, it is possible to prevent sudden changes from occurring in the display of the virtual space, thereby reducing the possibility of VR sickness.

[0041] The angle-of-view setting means may be capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view. The display control means may change display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, into substantially similar shapes and continuously reduce the display areas in response to the continuous change of the angle of view of the virtual camera from the first angle of view to the second angle of view.

[0042] According to the above, while the angle of view of the virtual camera is continuously changing from the first angle of view to the second angle of view, the left-eye image and the right-eye image having substantially similar shapes are displayed, so that when a part of the virtual space is continuously enlarged and displayed, it is possible to prevent the user from feeling uncomfortable and to prevent abrupt changes in the display of the virtual space, thereby reducing the possibility of VR sickness.

[0043] Another image display system of the present invention includes a goggle device, a virtual camera positioning means for positioning a virtual camera in a virtual space, an image generation means for generating images of the virtual space based on the virtual camera, an image for the left eye and an image for the right eye having parallax with respect to each other, a display control means for displaying the image for the left eye and the image for the right eye on a display unit of the goggle device, an enlargement setting means for setting a part of the virtual space to be displayed at a larger magnification than normal, and a parallax setting means for setting the parallax of the image for the left eye and the image for the right eye to a first parallax if the enlargement setting means has not set the enlarged display, and for setting the parallax of the image for the left eye and the image for the right eye to a second parallax smaller than the first parallax if the enlargement setting means has set the enlarged display. When the enlarged display is not set, the image generation means generates the image for the left eye and the image for the right eye, which have the first parallax relative to each other, and when the enlarged display is set, the image generation means generates the image for the left eye and the image for the right eye, which are images that are enlarged portions of the virtual space and have the second parallax relative to each other.

[0044] Another invention may be an image display program that causes a processor of a device that displays an image on a display unit of a goggle device to function as each of the above means. Another invention may be an image display device that displays an image on a display unit of a goggle device and that includes each of the above means. Another invention may be an image display method performed in the image display system that includes a goggle device. [Effects of the Invention]

[0045] According to the present invention, it is possible to enlarge and display a part of a virtual space, and also to reduce the possibility of VR sickness. [Brief explanation of the drawings]

[0046] [Figure 1] FIG. 1 shows an example of a state in which the left controller 3 and the right controller 4 are attached to the main unit 2. [Figure 2] FIG. 10 shows an example of a state in which the left controller 3 and the right controller 4 are detached from the main unit 2. [Figure 3] Six-sided views showing an example of the main unit 2 [Figure 4] Six-sided diagram showing an example of the left controller 3 [Figure 5] A block diagram showing an example of the internal configuration of the main unit 2. [Figure 6] A block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. [Figure 7] FIG. 1 is a perspective view showing an example of the appearance of a goggle device 150. [Figure 8] FIG. 1 is a front view showing an example of a state in which the main body device 2 is attached to the goggle device 150. [Figure 9] FIG. 1 is a perspective view showing an example of the appearance of a camera device 200. [Figure 10] FIG. 1 is a diagram showing an example of how a camera device 200 is attached to a goggle device 150 including a main device 2. [Figure 11] FIG. 1 is a diagram showing an example of a user viewing an image displayed on the image display system 100. [Figure 12]FIG. 1 is a diagram showing an example of a virtual space constructed in the main device 2. [Figure 13] FIG. 10 is a diagram showing an example of an image displayed on the display 12 of the main device 2 in another scene. [Figure 14] 14 is a diagram showing an example of the left-eye image and the right-eye image shown in FIG. 13 viewed through a left-eye lens 153L and a right-eye lens 153R, respectively. FIG. [Figure 15] A diagram showing an example of the angle of view of the left virtual camera VCL in normal operation and an image for the left eye generated at that angle of view. [Figure 16] FIG. 10 is a diagram showing an example of the angle of view of the left virtual camera VCL when zoomed in and an image for the left eye generated at that angle of view. [Figure 17] FIG. 10 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view A that is smaller than the first angle of view and larger than the second angle of view. [Figure 18] FIG. 10 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view B that is smaller than the third angle of view A and larger than the second angle of view. [Figure 19] FIG. 10 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view C that is smaller than the third angle of view B and larger than the second angle of view. [Figure 20] FIG. 10 shows an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a second angle of view. [Figure 21A] FIG. 10 is a diagram showing an example of a user's viewing angle when using a lens 153. [Figure 21B] FIG. 10 is a diagram showing an example of the relationship between the direction along the angle of view of the virtual camera and the line of sight of the user when the angle of view of the virtual camera is changed to a second angle of view; [Figure 22] FIG. 10 is a diagram showing an example of the relationship between the angle of view of a virtual camera VC and the display area of ​​an image for the left eye and an image for the right eye. [Figure 23] FIG. 1 is a diagram showing an example of the relationship between the angle of view of a virtual camera VC and the inter-camera distance d. [Figure 24] FIG. 10 is a diagram showing an example of a change in the attitude of the main device 2 and a change in the attitude of the virtual camera VC when image stabilization is not performed. [Figure 25] FIG. 10 is a diagram showing an example of a change in the attitude of the main device 2 and a change in the attitude of the virtual camera VC when image stabilization is performed. [Figure 26] FIG. 1 is a diagram showing an example of data stored in the memory (mainly the DRAM 85) of the main device 2. [Figure 27] 10 is a flowchart showing an example of processing performed by the processor 81 of the main unit 2. [Figure 28] Flowchart showing an example of the enlarged image display process in step S107 DETAILED DESCRIPTION OF THE INVENTION

[0047] Hereinafter, an image display system 100 (see FIG. 11) of this embodiment will be described with reference to the drawings. The image display system 100 of this embodiment is a system that allows a user to experience virtual reality (VR). For example, a predetermined game may be played in a VR space using the image display system 100. First, a game system 1 (an example of an information processing system) included in the image display system 100 will be described.

[0048] (Game System 1 explanation) An example of a game system 1 includes a main unit (information processing device; in this embodiment, it functions as a game device main unit) 2, a left controller 3, and a right controller 4. The left controller 3 and the right controller 4 are each detachable from the main unit 2. In other words, the game system 1 can be used as an integrated device by attaching the left controller 3 and the right controller 4 to the main unit 2. The game system 1 can also be used with the main unit 2, the left controller 3, and the right controller 4 separate from each other (see FIG. 2). Below, the hardware configuration of the game system 1 of this embodiment will be described, followed by a description of the image display system 100 of this embodiment.

[0049] FIG. 1 is a diagram showing an example of a state in which a left controller 3 and a right controller 4 are attached to a main unit 2. As shown in FIG. 1, the left controller 3 and the right controller 4 are each attached to and integrated with the main unit 2. The main unit 2 is a device that executes various processes (e.g., game processes) in the game system 1. The main unit 2 is equipped with a display 12. The left controller 3 and the right controller 4 are devices that have operation units that allow the user to perform inputs.

[0050] Fig. 2 is a diagram showing an example of the state in which the left controller 3 and the right controller 4 are detached from the main unit 2. As shown in Figs. 1 and 2, the left controller 3 and the right controller 4 are detachable from the main unit 2. Note that, below, the left controller 3 and the right controller 4 may be collectively referred to as "controllers."

[0051] Fig. 3 is a six-sided view showing an example of the main unit 2. As shown in Fig. 3, the main unit 2 includes a substantially plate-shaped housing 11. In this embodiment, the main surface of the housing 11 (in other words, the front surface, i.e., the surface on which the display 12 is provided) is generally rectangular.

[0052] 3, the main unit 2 includes a display 12 provided on the main surface of the housing 11. The display 12 displays images generated by the main unit 2. In this embodiment, the display 12 is a liquid crystal display (LCD). However, the display 12 may be any type of display device.

[0053] The main unit 2 also has a left terminal 17, which is a terminal for the main unit 2 to communicate with the left controller 3 via a wired connection, and a right terminal 21, which is a terminal for the main unit 2 to communicate with the right controller 4 via a wired connection.

[0054] As shown in FIG. 3, the main unit 2 includes a slot 23. The slot 23 is provided on the upper side of the housing 11. The slot 23 has a shape that allows a predetermined type of storage medium to be inserted therein. The predetermined type of storage medium is, for example, a storage medium (e.g., a dedicated memory card) dedicated to the game system 1 and the same type of information processing device. The predetermined type of storage medium is used, for example, to store data used by the main unit 2 (e.g., application save data, etc.) and / or programs executed by the main unit 2 (e.g., application programs, etc.). The main unit 2 also includes a power button 28.

[0055] FIG. 4 is a six-sided view showing an example of the left controller 3.

[0056] The left controller 3 includes an analog stick 32. As shown in Fig. 4, the analog stick 32 is provided on the main surface of the housing 31. The analog stick 32 can be used as a direction input unit that can input directions.

[0057] The left controller 3 is equipped with various operation buttons. The left controller 3 is equipped with four operation buttons 33 to 36 (specifically, a right button 33, a down button 34, an up button 35, and a left button 36) on the main surface of the housing 31. The left controller 3 also has a first L button 38 and a ZL button 39 on the upper left of the side of the housing 31. The left controller 3 is also equipped with a second L button 43 and a second R button 44 on the side of the housing 31 that is attached to the main unit 2. These operation buttons are used to issue instructions according to various programs (for example, OS programs and application programs) executed on the main unit 2.

[0058] The left controller 3 also includes a terminal 42 for wired communication between the left controller 3 and the main unit 2.

[0059] The right controller 4 also has an analog stick and multiple buttons, just like the left controller 3. A description of the right controller 4 will be omitted.

[0060] FIG. 5 is a block diagram showing an example of the internal configuration of the main unit 2. As shown in FIG.

[0061] The main unit 2 includes a processor 81. The processor 81 is an information processing unit that executes various types of information processing executed in the main unit 2, and may be composed of, for example, only a CPU (Central Processing Unit), or may be composed of an SoC (System-on-a-chip) that includes multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function. The processor 81 executes various types of information processing by executing an information processing program (for example, a game program) stored in a storage unit (specifically, an internal storage medium such as flash memory 84, or an external storage medium inserted into slot 23, etc.).

[0062] The main device 2 includes a flash memory 84 and a DRAM (Dynamic Random Access Memory) 85 as examples of internal storage media built into the main device 2. The flash memory 84 and the DRAM 85 are connected to the processor 81. The flash memory 84 is a memory used primarily to store various types of data (which may be programs) saved in the main device 2. The DRAM 85 is a memory used to temporarily store various types of data used in information processing.

[0063] The main device 2 includes a slot interface (hereinafter abbreviated as "I / F") 91. The slot I / F 91 is connected to the processor 81. The slot I / F 91 is connected to the slot 23, and reads and writes data from and to a predetermined type of storage medium (e.g., a dedicated memory card) inserted into the slot 23 in accordance with instructions from the processor 81.

[0064] The processor 81 reads and writes data from and to the flash memory 84, DRAM 85, and the above-mentioned storage media as appropriate, to execute the above-mentioned information processing.

[0065] The main unit 2 includes a network communication unit 82. The network communication unit 82 is connected to the processor 81. The network communication unit 82 communicates with external devices via a network (specifically, wireless communication). In this embodiment, the network communication unit 82 connects to a wireless LAN and communicates with external devices using a method conforming to the Wi-Fi standard as a first communication mode. The network communication unit 82 also performs wireless communication with other main units 2 of the same type using a predetermined communication method (e.g., communication using a proprietary protocol or infrared communication) as a second communication mode. Note that wireless communication using the second communication mode enables wireless communication with other main units 2 located within a closed local network area, and realizes a function that enables so-called "local communication," in which data is transmitted and received by direct communication between multiple main units 2.

[0066] The main unit 2 is equipped with a controller communication unit 83. The controller communication unit 83 is connected to the processor 81. The controller communication unit 83 performs wireless communication with the left controller 3 and / or right controller 4. Any communication method may be used between the main unit 2 and the left controller 3 and right controller 4, but in this embodiment, the controller communication unit 83 performs communication with the left controller 3 and right controller 4 in accordance with the Bluetooth (registered trademark) standard.

[0067] The processor 81 is connected to the above-mentioned left side terminal 17, right side terminal 21, and lower side terminal 27. When the processor 81 performs wired communication with the left controller 3, it transmits data to the left controller 3 via the left side terminal 17 and receives operation data from the left controller 3 via the left side terminal 17. When the processor 81 performs wired communication with the right controller 4, it transmits data to the right controller 4 via the right side terminal 21 and receives operation data from the right controller 4 via the right side terminal 21.

[0068] The display 12 is also connected to the processor 81. The processor 81 displays on the display 12 an image generated (for example, by executing the above-described information processing) and / or an image acquired from the outside.

[0069] The main unit 2 also includes an acceleration sensor 89 as an inertial sensor. In this embodiment, the acceleration sensor 89 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in FIG. 1). The acceleration sensor 89 may also detect acceleration along one or two axes.

[0070] The main unit 2 also includes an angular velocity sensor 90 as an inertial sensor. In this embodiment, the angular velocity sensor 90 detects angular velocities around three predetermined axes (for example, the x, y, and z axes shown in FIG. 1). The angular velocity sensor 90 may also detect angular velocities around one axis or two axes.

[0071] The acceleration sensor 89 and the angular velocity sensor 90 are connected to the processor 81, and the detection results of the acceleration sensor 89 and the angular velocity sensor 90 are output to the processor 81. The processor 81 can calculate information related to the movement and / or attitude of the main unit 2 based on the detection results of the acceleration sensor 89 and the angular velocity sensor 90.

[0072] The main device 2 includes a power control unit 97 and a battery 98. The power control unit 97 is connected to the battery 98 and the processor 81. Although not shown, the power control unit 97 is also connected to each part of the main device 2 (specifically, each part that receives power from the battery 98, the left terminal 17, and the right terminal 21). The power control unit 97 controls the power supply from the battery 98 to each of the above parts based on instructions from the processor 81.

[0073] Furthermore, battery 98 is connected to lower terminal 27. When an external charging device (e.g., a cradle) is connected to lower terminal 27 and power is supplied to main device 2 via lower terminal 27, battery 98 is charged with the supplied power.

[0074] 6 is a block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. Note that details of the internal configuration of the main unit 2 are omitted in FIG. 6 because they are shown in FIG. 5.

[0075] In addition to the above-mentioned buttons and analog stick 32, the left controller 3 is equipped with a communication control unit 101 that communicates with the main unit 2. As shown in FIG. 6 , the communication control unit 101 is connected to various components, including the terminal 42. In this embodiment, the communication control unit 101 can communicate with the main unit 2 both via wired communication via the terminal 42 and via wireless communication without using the terminal 42. The communication control unit 101 controls the communication method used by the left controller 3 with the main unit 2. That is, when the left controller 3 is attached to the main unit 2, the communication control unit 101 communicates with the main unit 2 via the terminal 42. When the left controller 3 is detached from the main unit 2, the communication control unit 101 performs wireless communication with the main unit 2 (specifically, the controller communication unit 83). Wireless communication between the controller communication unit 83 and the communication control unit 101 is performed in accordance with, for example, the Bluetooth (registered trademark) standard.

[0076] The left controller 3 also includes a memory 102, such as a flash memory. The communication control unit 101 is configured, for example, by a microcomputer (also called a microprocessor), and executes firmware stored in the memory 102 to perform various processes.

[0077] The left controller 3 also includes an inertial sensor. Specifically, the left controller 3 includes an acceleration sensor 104. The left controller 3 also includes an angular velocity sensor 105. In this embodiment, the acceleration sensor 104 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in FIG. 4). The acceleration sensor 104 may detect acceleration along one or two axial directions. In this embodiment, the angular velocity sensor 105 detects angular velocity around three predetermined axes (for example, the x, y, and z axes shown in FIG. 4). The angular velocity sensor 105 may detect angular velocity around one or two axes. The acceleration sensor 104 and the angular velocity sensor 105 are each connected to the communication control unit 101. The detection results of the acceleration sensor 104 and the angular velocity sensor 105 are repeatedly output to the communication control unit 101 at appropriate timing.

[0078] The communication control unit 101 acquires information about the input (specifically, information about the operation or the detection results by the sensors) from each input unit (specifically, each button, analog stick 32, and each sensor 104 and 105). The communication control unit 101 transmits operation data including the acquired information (or information obtained by performing predetermined processing on the acquired information) to the main unit 2. The operation data is repeatedly transmitted once every predetermined time. The interval at which the information about the input is transmitted to the main unit 2 may or may not be the same for each input unit.

[0079] The left controller 3 also includes a power supply unit 108. In this embodiment, the power supply unit 108 has a battery and a power control circuit. Although not shown, the power control circuit is connected to the battery and to each unit of the left controller 3 (specifically, each unit that receives power from the battery).

[0080] Like the left controller 3, the right controller 4 also includes a communication control unit 111, a memory 112, an acceleration sensor 114, an angular velocity sensor 115, and a power supply unit 118. A description of the right controller 4 will be omitted.

[0081] (Explanation of image display system 100) Next, an example of an image display system 100 will be described with reference to Fig. 7 to Fig. 11. The image display system 100 of this embodiment is configured by a goggle device 150 (Fig. 7) to which a main body device 2 is attached, and a camera device 200 (Fig. 9). Below, the goggle device 150 and the camera device 200 will be described separately, and the image display system 100 configured by the goggle device 150 and the camera device 200 will be described.

[0082] Fig. 7 is a perspective view showing an example of the appearance of goggle device 150. Fig. 8 is a front view showing an example of a state in which main device 2 is attached to goggle device 150. Fig. 9 is a perspective view showing an example of the appearance of camera device 200. Fig. 10 is a diagram showing an example of a state in which camera device 200 is attached to goggle device 150 including main device 2. Fig. 11 is a diagram showing an example of a state in which a user views an image displayed on image display system 100.

[0083] 7, goggle device 150 has goggle body 151, lens frame member 152, and lenses 153 (left eye lens 153L and right eye lens 153R). Here, goggle device 150, which is an example of a device constituting an image display system, is worn by fitting to the face of a user so as to cover the left and right eyes of the user, and has a function of blocking at least a part of external light and a function of supporting the user's stereoscopic vision with a pair of lenses.

[0084] The goggle body 151 has an attachment portion that detachably fixes the main body device 2 by contacting the front, back, top, and bottom surfaces of the main body device 2. The goggle body 151 has a front abutment portion that contacts a portion of the front surface (the surface on which the display 12 is provided) of the main body device 2, a back abutment portion that contacts the back surface of the main body device 2, an top abutment portion that contacts the top surface of the main body device 2, and a bottom abutment portion that contacts the bottom surface of the main body device 2. The attachment portion is formed by a gap surrounded by the front abutment portion, the back abutment portion, the top abutment portion, and the bottom abutment portion. The attachment portion has an opening on the left and / or right side so that it can be attached from the left or right side of the main body device 2. As shown in FIG. 8 , for example, the main body device 2 is attached through the opening on the left side to form a goggle device 150 including the main body device 2. In addition, the front contact portion of the goggle body 151 has left and right openings formed to allow the user to view the display images (images for the left eye and right eye) on the display 12 when the main body device 2 is worn.

[0085] A lens frame member 152 is fixed to the front side of the front abutment portion of the goggle body 151 (the front side of the paper in FIG. 7). The lens frame member 152 is a member for fixing a pair of left-eye lens 153L and right-eye lens 153R. The lens frame member 152 has left and right openings. The positions and sizes of the left and right openings of this lens frame member 152 approximately match the positions and sizes of the left and right openings provided in the front abutment portion of the goggle body 151. The left-eye lens 153L and the right-eye lens 153R are fitted and fixed into the left and right openings, respectively, of this lens frame member 152. The distance between the centers of the left-eye lens 153L and the right-eye lens 153R is set to the distance between the left and right eyes of an average user.

[0086] Furthermore, the goggle device 150 is provided with a recess 154 for accommodating the user's nose. When the user wears the goggle device 150 so that the nose is accommodated in the recess 154, the left-eye lens 153L is positioned in front of the user's left eye, and the right-eye lens 153R is positioned in front of the user's right eye.

[0087] As shown in FIG. 8, when the main device 2 is attached to the goggle main body 151, the left-eye image IML displayed in the left display area (area surrounded by a dashed line) of the display 12 is viewed by the left eye of the user through the left-eye lens 153L. The left-eye image is a substantially circular image with a portion (a portion close to the right-eye image) being straight. The left-eye lens 153L is circular and causes the left-eye image IML to be viewed by the user's left eye. The user's left eye is surrounded by the left side, top, and bottom surfaces of the goggle main body 151, and a partition surface (not shown) that separates the left and right sides. For this reason, the left-eye image is viewed by the user's left eye, while the surrounding environment and the right-eye image are difficult to view.

[0088] Furthermore, the right-eye image IMR displayed in the right display region (region surrounded by a dashed line) of the display 12 is viewed by the user's right eye via the right-eye lens 153R. The right-eye image is a substantially circular image with a portion (a portion close to the left-eye image) being straight. The right-eye lens 153R is circular and allows the right-eye image IMR to be viewed by the user's right eye. The user's right eye is surrounded by the right side, top surface, bottom surface, and partition surface of the goggle body 151. For this reason, the right-eye image is viewed by the user's right eye, while the surrounding environment and the left-eye image are difficult to view.

[0089] The shapes of the image for the left eye and the image for the right eye may be circular, elliptical, or may be approximately circular or elliptical, which is a deformation of a part of a circle or ellipse, or may be polygonal, star-shaped, etc.

[0090] As shown in FIG. 9 , the image display system 100 includes a camera device 200. The camera device 200 includes a camera body 201 and a cylindrical portion 205. The camera device 200 has an overall shape similar to that of a real camera. The camera device 200 is a pseudo camera that does not have an actual imaging element or optical lens. The cylindrical portion 205 is provided on the front side of the camera device 200 (the positive z-axis direction side in FIG. 9 ). The cylindrical portion 205 is a substantially cylindrical member. The central axis direction of the cylindrical portion 205 substantially coincides with the line of sight of the user when the user wears the image display system 100. In other words, when the user wears the image display system 100, the cylindrical portion 205 is formed to extend from the front side of the image display system 100 in the line of sight of the user. The cylindrical portion 205 is configured to be rotatable in a roll direction (around the central axis of the cylindrical portion 205). For example, the cylindrical portion 205 is configured to be rotatable by 90 degrees in the roll direction from the normal state shown in Fig. 9. The cylindrical portion 205 is modeled after, for example, a telephoto zoom lens of an actual camera.

[0091] The cylindrical portion 205 has an opening 206 at its tip. The left controller 3 (or the right controller 4) is inserted into the opening 206. For example, the left controller 3 is inserted into the opening 206 to detect the rotation angle of the cylindrical portion 205 in the roll direction. When the cylindrical portion 205 is not rotated, the opening 206 has a mountain-like shape with the center higher upward (in the y-axis direction in FIG. 9 ) than the left and right sides. When the cylindrical portion 205 is not rotated, the left controller 3 is inserted into the opening 206 in the longitudinal direction (in the y-axis direction in FIG. 4 ) so that the main surface of the housing 31 of the left controller 3 faces upward. Because the opening 206 has a mountain-like shape, the left controller 3 having the analog stick 32 protruding in the direction of the main surface can be inserted into the opening 206. Furthermore, when the cylindrical portion 205 is rotated in the roll direction, the left controller 3 inserted inside the opening 206 also rotates around the y-axis, but the analog stick 32 comes into contact with the mountain-shaped portion, thereby fixing the inserted left controller 3 in place and preventing the left controller 3 from moving inside the opening 206.

[0092] The upper view of Fig. 10 shows the camera device 200 on the opposite side to the cylindrical portion 205. As shown in Fig. 10, the camera main body portion 201 includes an upper surface portion 202, a right side surface portion 203, and a left side surface portion 204. The upper surface portion 202, the right side surface portion 203, and the left side surface portion 204 form a recess in which (a part of) the goggle device 150 to which the main body device 2 is attached is fitted. For example, at least the portion of the goggle device 150 corresponding to the main body device 2 is fitted into the recess formed by the upper surface portion 202, the right side surface portion 203, and the left side surface portion 204. In this manner, the image display system 100 is configured.

[0093] As shown in FIG. 11 , a user holds an image display system 100 including a goggle device 150 and a camera device 200 and views a left-eye image and a right-eye image displayed on a display 12 of a main unit 2. Specifically, the main unit 2 defines a virtual space and generates a left-eye image and a right-eye image having a parallax therebetween based on a left virtual camera and a right virtual camera. The main unit 2 displays the generated left-eye image and right-eye image in a left display region and a right display region of the display 12, respectively. This allows the user to view a stereoscopic image and experience virtual reality (VR) as if the user were present in the virtual space.

[0094] (Virtual space display) Next, an image displayed in the image display system 100 of this embodiment will be described. The image display system 100 of this embodiment allows the user to experience VR and also provides a zoom-in (close-up) function for enlarging a part of the VR space. Below, first, the virtual space defined by the main device 2 will be described, and then the zoom-in function in the VR space will be described.

[0095] 12 is a diagram showing an example of a virtual space constructed in the main device 2. An XYZ Cartesian coordinate system is set in the virtual space VS. The X axis is the horizontal axis of the virtual space VS. The Y axis is the height axis of the virtual space VS. The Z axis is the axis perpendicular to the X axis and Y axis, and is the depth axis of the virtual space.

[0096] A left virtual camera VCL and a right virtual camera VCR are placed in the virtual space VS. The heights of the left virtual camera VCL and the right virtual camera VCR may be set to the same height as the average human eye level. The line of sight of the left virtual camera VCL and the right virtual camera VCR is set to the same direction. In the following description, when there is no need to distinguish between the left virtual camera VCL and the right virtual camera VCR, the left virtual camera VCL and the right virtual camera VCR may be collectively referred to as "virtual camera VC."

[0097] The attitude of the virtual camera VC is controlled so as to match the attitude of the image display system 100 (goggle device 150) in real space. For example, the main device 2 calculates the attitude of the main device 2 based on the angular velocity values ​​and / or acceleration values ​​detected by the angular velocity sensor 90 and / or the acceleration sensor 89. Specifically, the main device 2 calculates the change in its own attitude from the time of initialization by integrating the angular velocity values ​​from the angular velocity sensor 90. The main device 2 controls the attitudes of the left virtual camera VCL and the right virtual camera VCR in the virtual space according to the calculated attitude. For example, when the main device 2 is oriented such that a line perpendicular to the display 12 is parallel to the ground, the virtual camera VC faces in a direction parallel to the XZ plane in the virtual space. When the user rotates the goggle device 150 (main device 2) 90 degrees in the yaw direction (left / right direction) from this attitude, the virtual camera VC also rotates 90 degrees in the yaw direction (left / right direction) in the virtual space.

[0098] Furthermore, various virtual objects are placed in the virtual space VS. The types of virtual objects placed vary depending on, for example, the game scene. For example, in the example shown in FIG. 12, a character object 300, a table object 302, and a cylindrical object 301 on the table object are placed in the virtual space VS. The user views a left-eye image and a right-eye image, which are images of the virtual space viewed from the left virtual camera VCL and the right virtual camera VCR, with their left and right eyes, respectively, to view a stereoscopic image of the virtual space VS. Note that in this specification, the term "for" does not mean that it is used exclusively for that purpose; for example, the left-eye image or the right-eye image may be viewed with both eyes.

[0099] Here, the left-eye image and the right-eye image displayed on the display 12 in normal times (i.e., when not zoomed in) will be described. Fig. 13 is a diagram showing an example of an image displayed on the display 12 of the main unit 2 in another scene. In Fig. 13, for example, a scene of a virtual room is displayed.

[0100] As shown in FIG. 13, a left-eye image of the virtual space as seen from the left virtual camera VCL is displayed in the left display area, which is a substantially circular area surrounded by a dashed line. A right-eye image of the virtual space as seen from the right virtual camera VCR is displayed in the right display area, which is a substantially circular area surrounded by a dashed line. The left and right display areas are areas of the same size and are symmetrical about the center of the left-right direction of display 12. Areas other than the left and right display areas (hatched areas in FIG. 13) are invisible or difficult to see when viewed by a user through lens 153. These invisible or difficult to see areas may be referred to as "non-visible areas" below. A black image, for example, is displayed in the non-visible areas. The outlines of the left and right display areas (boundaries with the non-visible areas) are blurred to make the boundaries unclear.

[0101] The left-eye image and the right-eye image include, as images of virtual objects, images of a carpet 400, a window 401, and a ceiling 402. The left-eye image displayed in the left display area of ​​the display 12 is corrected taking into consideration that it is viewed by the left eye of the user through the left-eye lens 153L. That is, when it is displayed on the display 12 as the left-eye image, the left-eye image is distorted overall taking into consideration the optical characteristics (lens distortion (distortion aberration)) of the left-eye lens 153L.

[0102] For example, carpet 400 is placed in the virtual space as a rectangular object. Window 401 is a rectangular object with a long side parallel to the X axis of the virtual space and a short side parallel to the Y axis. Ceiling 402 is an object with a linear pattern drawn in the virtual space. For example, the pattern on ceiling 402 includes straight lines parallel to the Z axis of the virtual space. In an image generated based on virtual camera VC, if distortion according to the characteristics of left-eye lens 153L is not added, the straight line portions of each object will be straight lines.

[0103] When generating the left-eye image, the main device 2 does not simply render the virtual space seen from the left virtual camera VCL, but rather adds distortion according to the characteristics of the left-eye lens 153L to generate the left-eye image. As a result, the left-eye image displayed on the display 12 has a distorted shape overall, and each object has a distorted shape. For example, the image of carpet 400 in the left-eye image is more distorted the closer it is to the periphery of the left-eye image. Similarly, the image of window 401 is more distorted the closer it is to the periphery of the left-eye image. Furthermore, the pattern of ceiling 402 is curved rather than straight in the left-eye image, and the closer it is to the periphery, the greater the distortion.

[0104] In this way, the degree of distortion is greater in the peripheral portion of the left-eye image than in the central portion. This is because, when the left-eye image is viewed through the left-eye lens 153L, the peripheral portion of the image is more distorted than the central portion. Taking into account such characteristics of the left-eye lens 153L, the main unit 2 distorts the entire image so that the distortion becomes greater in the peripheral portion of the image.

[0105] Like the image for the left eye, the image for the right eye is also an image to which distortion has been added according to the characteristics of the right-eye lens 153R. Note that there is a parallax between the image for the left eye and the image for the right eye. For example, when comparing the image for the left eye and the image for the right eye, the window 401 is shifted to the left side in the image for the right eye compared to the image for the left eye.

[0106] Hereinafter, the process of adding distortion to an image in consideration of the characteristics of the lens 153 may be referred to as "distortion correction."

[0107] The user views the left-eye image and right-eye image shown in Fig. 13 through left-eye lens 153L and right-eye lens 153R. When the user views these images, the distortions added to the images are canceled out by lens 153, making the images appear natural to the user.

[0108] FIG. 14 is a diagram showing an example of the left-eye image and the right-eye image shown in FIG. 13 viewed through left-eye lens 153L and right-eye lens 153R, respectively.

[0109] As shown in Fig. 14, the left eye image and the right eye image viewed through lens 153 are natural images, not the totally distorted images shown in Fig. 13. For example, in Fig. 13, the straight line portions of each object are displayed as curved lines, but when viewed through lens 153, the straight line portions of each object are viewed as straight lines.

[0110] In this way, rather than displaying an image of the virtual space viewed from the virtual camera VC as is, an image that has undergone distortion correction according to the characteristics of the lens 153 is displayed in the display area of ​​the display 12. When the user views the left-eye image and the right-eye image displayed on the display 12 through the lens 153, the distortion of each image is canceled by the lens 153. Therefore, when the user views the left-eye image and the right-eye image through the lens 153, they perceive them as natural stereoscopic images of the virtual space, and the user feels as if the virtual object existing in the virtual space is in real space. For example, if a virtual object is placed 5 meters away from the virtual camera VC in the virtual space, the user feels as if the virtual object is actually present 5 meters away. Furthermore, the user can change the orientation of the left virtual camera VCL and the right virtual camera VCR by changing the orientation of the goggle device 150, thereby viewing various directions in the virtual space.

[0111] (Enlarged view of virtual space) In image display system 100 of this embodiment, the user can enlarge a portion of the virtual space by rotating cylindrical portion 205. Specifically, rotating cylindrical portion 205 rotates left controller 3 (or right controller 4) provided inside cylindrical portion 205. Main unit 2 acquires the orientation of left controller 3 based on operation data from left controller 3, and narrows the angle of view of left virtual camera VCL and right virtual camera VCR according to the acquired orientation of left controller 3. This enlarges (zooms in) a portion of the virtual space.

[0112] Fig. 15 is a diagram showing an example of the angle of view of the left virtual camera VCL in normal mode and an example of a left-eye image generated at that angle of view. Fig. 16 is a diagram showing an example of the angle of view of the left virtual camera VCL in zoom-in mode and an example of a left-eye image generated at that angle of view.

[0113] 15 and 16 show views of the virtual space viewed from above. Note that for the sake of explanation, the distortion described above is omitted in the left-eye image on the right side of Fig. 15 and Fig. 16.

[0114] 15, the angles of view of the left virtual camera VCL and the right virtual camera VCR are usually set to a first angle of view (for example, 90 degrees). Also, the distance (inter-camera distance) d between the left virtual camera VCL and the right virtual camera VCR is usually set to the distance between the left and right eyes of an average user.

[0115] When the angle of view of the left virtual camera VCL is set to the first angle of view, character object 300, cylinder object 301, and table object 302 are included within the angle of view of the left virtual camera VCL. An image of the virtual space included within the first angle of view is generated as an image for the left eye. In this case, for example, an image for the left eye such as that shown in the diagram on the right side of FIG. 15 is generated. Specifically, main unit 2 performs projective transformation (perspective projection) of the virtual space included within the first angle of view from the position of left virtual camera VCL to generate an image for the left eye of a predetermined size. Note that when the image for the left eye is generated, the distortion correction described above is performed. The same applies to the image for the right eye.

[0116] When the user rotates cylindrical portion 205, the angles of view of left virtual camera VCL and right virtual camera VCR become narrower. For example, when the user rotates cylindrical portion 205 to the maximum extent, the angles of view of left virtual camera VCL and right virtual camera VCR are set to a second angle of view (e.g., 30 degrees). When the angle of view of virtual camera VC is set to the second angle of view, the entire cylindrical object 301 and a portion of table object 302 are included in the angle of view of virtual camera VC. An image of a predetermined size is generated as an image of the virtual space included in the angle of view of virtual camera VC. That is, when the angle of view of virtual camera VC is set to the second angle of view, an enlarged left-eye image is generated by enlarging a portion of the virtual space included when the angle of view of virtual camera VC was set to the first angle of view. For example, as shown in FIG. 16, a left-eye image is generated that includes an entire image of cylindrical object 301 enlarged compared to that in FIG. 15 and an enlarged image of a portion of table object 302.

[0117] Furthermore, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is smaller than when the angle of view of the virtual camera VC is set to the first angle of view. That is, when the angle of view of the virtual camera VC is set to the second angle of view, the parallax between the left-eye image and the right-eye image is smaller than when the angle of view of the virtual camera VC is set to the first angle of view. In this embodiment, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is set to "0", and the parallax between the left-eye image and the right-eye image is zero.

[0118] In this embodiment, the angle of view of the virtual camera VC changes continuously from a first angle of view (e.g., 90 degrees) to a second angle of view (e.g., 30 degrees) according to the rotation angle of the cylindrical portion 205 (left controller 3). Also, the inter-camera distance d changes continuously from the first angle of view to the second angle of view according to the rotation angle of the cylindrical portion 205 (left controller 3).

[0119] In this embodiment, while the angle of view of the virtual camera VC changes continuously from the first angle of view to the second angle of view, a portion of the virtual space is continuously enlarged, and the display areas of the left-eye image and the right-eye image are continuously reduced. Below, with reference to Figures 17 to 20, the left-eye image and the right-eye image displayed on the display 12 when the angle of view of the virtual camera VC changes from the first angle of view to the second angle of view will be described.

[0120] Fig. 17 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view A that is smaller than the first angle of view and larger than the second angle of view. Fig. 18 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view B that is smaller than the third angle of view A and larger than the second angle of view. Fig. 19 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to a third angle of view C that is smaller than the third angle of view B and larger than the second angle of view. Fig. 20 is a diagram showing an example of a left-eye image and a right-eye image displayed on the display 12 when the angle of view of the virtual camera VC is set to the second angle of view.

[0121] As shown in FIG. 17, when the angle of view of the virtual camera VC is set to the third angle of view A (e.g., 50 degrees), an image in which a portion of the virtual space is enlarged is displayed. For example, in FIG. 15, a character object 300, a cylinder object 301, and a table object 302 are displayed, with the cylinder object 301 displayed relatively small, whereas in FIG. 17, the entire enlarged cylinder object 301 and a portion of the enlarged table object 302 are displayed. That is, when the angle of view of the virtual camera VC is set to the third angle of view A, the displayed sizes of the cylinder object 301 and the table object 302 are larger than when the angle of view of the virtual camera VC is set to the first angle of view. When the angle of view of the virtual camera VC is set to the third angle of view A (e.g., 50 degrees), the display areas of the enlarged left-eye image and right-eye image are the same size as the display areas when the angle of view is set to the first angle of view. In addition, non-visible areas other than the left display area and the right display area are filled in black. The outlines of the left and right display areas (boundaries with the non-visual area) are blurred to make the boundaries unclear.

[0122] As shown in FIG. 18, when the angle of view of the virtual camera VC is set to a third angle of view B (for example, 45 degrees), a portion of the virtual space is further enlarged. For example, in FIG. 18, the cylindrical object 301 and the table object 302 are displayed at a greater enlargement than in FIG. 17. In this case, the display areas of the enlarged left-eye image and right-eye image are smaller than in FIG. 17. In FIG. 18, the left display area and the right display area when the angle of view of the virtual camera VC is set to the first angle of view are shown by dashed lines, and the left display area and the right display area when the angle of view of the virtual camera VC is set to the third angle of view B are shown by solid lines. The area surrounded by the dashed and solid lines is filled in black, similar to the non-visual area, thereby reducing the display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space. In other words, the outer peripheries of the left-eye image and the right-eye image are cut off, thereby reducing the display areas of these images. The left-eye image and the right-eye image, which are enlarged portions of the virtual space, are images of the same size and line symmetry.

[0123] When the angle of view of the virtual camera VC is the first angle of view (90 degrees), the left and right display areas are approximately circular. When the angle of view of the virtual camera VC is the third angle of view B (45 degrees), the left and right display areas are also shaped similarly to circles, for example, approximately dodecagonal. The left and right display areas at angle of view B are areas excluding the outer peripheries of the left and right display areas at the first angle of view. Furthermore, when the left and right display areas are reduced, the blurring process described above is not performed, and the contours of the left and right display areas remain clear. Note that even when the angle of view of the virtual camera VC is reduced, the contours of the left and right display areas may be blurred.

[0124] 19, when the angle of view of virtual camera VC is set to third angle of view C (e.g., 35 degrees), a portion of the virtual space is further enlarged. Objects 301 and 302 in the left-eye image and the right-eye image in FIG. 19 are displayed larger than objects 301 and 302 in the left-eye image in FIG. 18. Furthermore, the display areas of the left-eye image and the right-eye image, in which a portion of the virtual space is enlarged, are further reduced than in FIG. 18. That is, the left and right display areas when the angle of view of virtual camera VC is set to third angle of view C (35 degrees) are smaller than the left and right display areas when the angle of view of virtual camera VC is set to third angle of view B (45 degrees). In other words, when the angle of view of virtual camera VC is set to third angle of view C, the outer peripheral portions of the left-eye image and the right-eye image are cut off to a greater extent than when the angle of view of virtual camera VC is set to third angle of view B. The shapes of the left and right display areas at the third angle of view C are approximately regular dodecagons, just like at the third angle of view B. Even when the angle of view of virtual camera VC is set to the third angle of view C, the left eye image and right eye image, which are enlarged portions of the virtual space, are the same size and line-symmetrical.

[0125] 20, when the angle of view of virtual camera VC is set to a second angle of view (e.g., 30 degrees), a portion of the virtual space is further enlarged. Objects 301 and 302 in the left-eye image and the right-eye image in FIG. 20 are larger than objects 301 and 302 in the left-eye image in FIG. 19. The display areas of the left-eye image and the right-eye image, in which a portion of the virtual space is enlarged, are further reduced than in FIG. 19. That is, the left display area and the right display area when the angle of view is set to the second angle of view (30 degrees) are smaller than the left display area and the right display area when the angle of view is set to the third angle of view C (35 degrees). The shapes of the left display area and the right display area at the second angle of view are approximately regular dodecagons, similar to those at the third angle of view C. Even when the angle of view of virtual camera VC is set to the second angle of view, the left-eye image and the right-eye image, in which a portion of the virtual space is enlarged, are images of the same size and line symmetry.

[0126] In this manner, in this embodiment, a portion of the virtual space is enlarged (zoomed in) by reducing the angle of view of the virtual camera VC. The smaller the angle of view of the virtual camera VC, the greater the magnification rate of the virtual space, and the smaller the display area of ​​the left-eye image and the right-eye image. In other words, the smaller the angle of view of the virtual camera VC (the more a portion of the virtual space is enlarged), the more the outer peripheral portions of the left-eye image and the right-eye image are cut off.

[0127] 17 to 20, the distortion correction causes greater distortion toward the outer periphery of the left-eye image and the right-eye image, and as will be described later, the outer periphery of the left-eye image and the right-eye image creates a more uncomfortable appearance for the user. However, in this embodiment, the outer periphery of the left-eye image and the right-eye image is cut, thereby reducing the user's sense of discomfort and the possibility of VR sickness.

[0128] Note that when the angle of view of virtual camera VC is set to the first angle of view, the shapes of the left-eye image and the right-eye image are approximately circular, and when the angle of view of virtual camera VC is set to the third angle of view B to the second angle of view, the shapes of the left-eye image and the right-eye image are approximately regular dodecagons. It can be said that the shapes of the left-eye image and the right-eye image are approximately similar until the angle of view of virtual camera VC changes from the first angle of view to the second angle of view. Here, the shapes of the left-eye image and the right-eye image when zoomed in are not limited to approximately regular dodecagons. For example, the shapes of the left-eye image and the right-eye image when zoomed in may be approximately decagons, approximately regular octagons, approximately regular hexagons, or approximately regular pentagons. In other words, in this specification, "approximately similar shapes" are not limited to completely identical shapes, but include shapes that are similar but strictly different.

[0129] Here, when changing the angle of view of the virtual camera VC from the first angle of view to the second angle of view, it is possible to keep the size of the display area constant without cutting off the outer periphery of the left-eye image and the right-eye image as described above. However, if the angle of view of the virtual camera VC is reduced while maintaining the size of the display area of ​​the left-eye image and the right-eye image, it may cause VR sickness.

[0130] Specifically, if the angle of view of the left virtual camera VCL and the right virtual camera VCR is reduced and a portion of the virtual space is enlarged, the appearance may be optically incorrect, which may cause VR sickness. The reason for this is explained below.

[0131] Fig. 21A is a diagram showing an example of the viewing angle of a user using lens 153. Fig. 21B is a diagram showing an example of the relationship between the direction along the angle of view of the virtual camera and the line of sight of the user when the angle of view of the virtual camera is changed to a second angle of view.

[0132] As shown in FIG. 21A , in the image display system 100 of this embodiment, when a user views the display 12 through the lens 153, the viewing angle of one eye of the user is adjusted to, for example, 90 degrees. For example, in the horizontal direction, the left end of the lens 153 is 45 degrees to the left of the user's eye position, and the right end of the lens 153 is 45 degrees to the right. In the vertical direction, the upper end of the lens 153 is 45 degrees above the user's eye position, and the lower end of the lens 153 is 45 degrees below the user's eye position. The first angle of view of the virtual camera VC in normal operation matches the viewing angle of the user using the lens 153. Because the viewing angle of one eye of a person is approximately 90 degrees to 100 degrees, when a user views the display 12 of the goggle device 150, most of the field of view is covered.

[0133] 21B, if the angle of view (field of view) of virtual camera VC is set to, for example, 30 degrees, the image within the 30-degree range seen from virtual camera VC is stretched and displayed on display 12, and the user sees the stretched image. If the 30-degree range seen from virtual camera VC is expanded to a 90-degree field of view, the user will see an incorrect image.

[0134] Specifically, if the outer peripheries of the left-eye image and the right-eye image are not cut off, a virtual object or line located at a 15-degree angle from the virtual camera VC will appear to be located at a 45-degree angle from the user's eyes. That is, the angle (45 degrees) at which the edges of the stretched left-eye image and right-eye image are viewed from the viewpoint of a user wearing goggle device 150 is larger than the angle (15 degrees) at which the virtual space is viewed from virtual camera VC along the second angle of view. This causes the virtual space to appear differently from the actual appearance, which may cause the user to feel uncomfortable. For example, the portion of an object that is actually visible when it is located at a 15-degree angle from the user will differ from the portion that is visible when the object is actually located at a 45-degree angle from the user. For example, when an object is actually located at a 15-degree angle, the back portion of the object will be visible, but when the object is actually located at a 45-degree angle, the back portion may not be visible. Therefore, when the virtual space viewed from the 30-degree angle of view of virtual camera VC is expanded to a 90-degree field of view, the object will appear differently from the actual appearance to the user. This difference in appearance becomes greater toward the outer periphery of the lens 153. Furthermore, if the angle of view of the virtual camera VC and the field of view of the user wearing the goggle device 150 (the display angle of view of the goggle device 150) match, when the virtual camera VC is rotated by a certain angle, the virtual object in the virtual space also appears to move by the same angle. On the other hand, if the angle of view of the virtual camera VC and the display angle of view of the goggle device 150 differ, when the virtual camera VC is rotated by a certain angle, the virtual object in the virtual space does not move by the same angle. For example, if the angle of view of the virtual camera VC is set to 30 degrees, when the user rotates their head 30 degrees to the right, an object that was in front of them will rotate 90 degrees to the right. In this way, if the angle of view of the virtual camera VC differs from the display angle of view of the goggle device 150, the virtual object will look unnatural to the user and move unnaturally. This may cause VR sickness.

[0135] Further, for example, it is conceivable to generate an image of a virtual space as seen from the virtual camera VC, and perform distortion correction according to the characteristics of the lens 153 depending on the position from the center of the generated image (the display position when displayed on the display 12). Because distortion correction is performed according to the position from the center of the image, when the image displayed on the display 12 is viewed through the lens 153, the distortion of the lens 153 and the distortion caused by the distortion correction cancel each other out, and the user sees the image as an undistorted image. Therefore, whether the angle of view of the virtual camera VC is set to the first angle of view or the second angle of view, the user basically sees an image undistorted. However, for example, the position and distance of the user's eyes are not always constant, and there may be some deviation between the position of the user's eyes, the position of the lens 153, and the position of the image displayed on the display 12. Due to these positional deviations, the distortion of the lens 153 and the distortion caused by the distortion correction may not match. Because the distortion is greater closer to the outer periphery of the lens, the deviation between the distortion of the lens 153 and the distortion caused by the distortion correction also becomes greater as the image approaches the outer periphery. When the angle of view of the virtual camera VC is set to the second angle and enlarged, the object appears distorted and hangs over the outer edge of the image, which makes the difference from the real appearance larger and may cause VR sickness.

[0136] When the angle of view of the virtual camera VC is reduced to enlarge an image, the closer to the outer periphery, the more unnatural the image appears, which may cause VR sickness. Therefore, in this embodiment, in order to reduce VR sickness, when the angle of view of the virtual camera VC is reduced to enlarge a part of the virtual space, the outer periphery of the left-eye image and the right-eye image is cut as shown in Figures 17 to 20. Specifically, the smaller the angle of view of the virtual camera VC, the larger the area of ​​the outer periphery to be cut (in other words, the smaller the display area of ​​the left-eye image and the right-eye image).

[0137] This allows for the expansion of a portion of the virtual space and reduces VR sickness caused by the expansion. Specifically, it is possible to prevent the user from viewing the image of the virtual space corresponding to the outer periphery, which is significantly different from the real appearance, and to prevent the user from feeling uncomfortable.

[0138] In addition, by cutting out the outer edges of the image for the left eye and the image for the right eye, the user becomes aware that part of the virtual space is expanding, and the user views the image with that awareness, making it less likely that they will experience VR sickness.

[0139] Furthermore, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge an image, the inter-camera distance d is changed according to the angle of view of the virtual camera VC. Specifically, the smaller the angle of view of the virtual camera VC (the higher the magnification rate of the virtual space), the smaller the inter-camera distance d is made. This can reduce VR sickness. That is, when the angle of view of the virtual camera VC is reduced to enlarge an image, the image appears optically incorrect, as described above, and the three-dimensional effect differs from reality. For this reason, if the inter-camera distance d is kept the same as normal when the angle of view of the virtual camera VC is reduced to enlarge an image, the user will experience a three-dimensional effect that differs from reality, which may cause VR sickness. Therefore, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge an image, the inter-camera distance d is reduced (that is, the parallax between the left-eye image and the right-eye image is reduced). This can weaken the three-dimensional effect and reduce VR sickness.

[0140] Fig. 22 is a diagram showing an example of the relationship between the angle of view of virtual camera VC and the display area of ​​the left-eye image and the right-eye image. Fig. 23 is a diagram showing an example of the relationship between the angle of view of virtual camera VC and the inter-camera distance d. In Figs. 22 and 23, the horizontal axis represents the magnification factor (the angle of view of virtual camera VC), and the magnification factor increases (the angle of view becomes smaller) as one moves to the right.

[0141] As shown in FIG. 22, when the angle of view of virtual camera VC is between the first angle of view (e.g., 90 degrees) and the third angle of view A, the area of ​​the display area for the left-eye image and the right-eye image is constant (see FIG. 17). When the angle of view of virtual camera VC is smaller than the third angle of view A, the area of ​​the display area becomes smaller (see FIGS. 18 and 19). The smaller the angle of view of virtual camera VC, the greater the rate at which the area of ​​the display area decreases. For example, the area of ​​the display area decreases relatively gradually until the angle of view of virtual camera VC reaches the third angle of view C, and when the angle of view of virtual camera VC becomes smaller than the third angle of view C, the area of ​​the display area decreases abruptly. When the angle of view of virtual camera VC is set to the smallest second angle of view (e.g., 30 degrees), the display area becomes smallest (see FIG. 20).

[0142] 23, the inter-camera distance d decreases while the angle of view of virtual camera VC changes from the first angle of view (for example, 90 degrees) to the second angle of view. For example, the inter-camera distance d decreases linearly as the angle of view decreases (the image enlarges), and when the angle of view of virtual camera VC is set to the smallest second angle of view, the inter-camera distance d becomes "0".

[0143] In addition, the inter-camera distance d does not have to change linearly, but may change curvilinearly in accordance with the change in the angle of view of the virtual camera VC (i.e., the rate of increase or decrease of the inter-camera distance d may change while the angle of view of the virtual camera VC changes).

[0144] (Image stabilization when zooming in) Next, the image stabilization function of this embodiment will be described. In this embodiment, image stabilization is performed when a portion of the virtual space is enlarged by reducing the angle of view of the virtual camera VC. Image stabilization is a function that reduces changes in the posture of the virtual camera VC due to camera shake. This image stabilization will be described below.

[0145] Fig. 24 is a diagram showing an example of a change in the attitude of the main unit 2 and a change in the attitude of the virtual camera VC when image stabilization is not performed, and Fig. 25 is a diagram showing an example of a change in the attitude of the main unit 2 and a change in the attitude of the virtual camera VC when image stabilization is performed.

[0146] For example, in this embodiment, the orientations of the main unit 2 and virtual camera VC are represented by vectors. For example, the orientation of the main unit 2 is represented by a quaternion q having components representing rotations in the x-, y-, and z-axis directions and a component representing the rotation angle. In Figures 24 and 25, the horizontal axis represents time, and the vertical axis represents the vector q representing the orientation.

[0147] As shown in FIG. 24, when the attitude of the main device 2 changes between times t0 and t5, the attitude of the virtual camera VC usually matches the attitude of the main device 2 at each time. That is, in this embodiment, when the angle of view of the virtual camera VC is set to the first angle of view, image stabilization is not performed. When the main device 2 detects a change in its own attitude, it changes the attitude of the virtual camera VC so that it matches its own attitude. Note that the main device 2 may calculate its own attitude at a frequency of, for example, 200 Hz.

[0148] On the other hand, as shown in FIG. 25, when the angle of view of virtual camera VC is smaller than the first angle of view, image stabilization is performed. When image stabilization is performed, the attitude of virtual camera VC changes to approach (follow) the attitude of main unit 2. If the attitude of main unit 2 changes, the attitude of virtual camera VC changes to match the attitude of main unit 2 over a predetermined time. Therefore, even if the attitude of main unit 2 changes suddenly, the attitude of virtual camera VC does not change suddenly, but changes to approach the attitude of main unit 2 over a certain period of time. For example, if virtual camera VC matches the attitude of main unit 2 at time t0 and the attitude of main unit 2 changes at time t1, the attitude of virtual camera VC at time t1 does not completely match the attitude of main unit 2. The attitude of virtual camera VC changes by an amount smaller than the amount of change in the attitude of main unit 2. Furthermore, if the attitude of main unit 2 changes at time t2, the attitude of virtual camera VC at time t2 does not completely match the attitude of main unit 2, but changes to approach the attitude of main unit 2. Thereafter, if the orientation of main device 2 at time t3 is the same as that at time t2, the orientation of virtual camera VC at time t3 will further approach that of main device 2. Then, if the orientation of main device 2 at time t4 is the same as that at time t2, the orientation of virtual camera VC at time t4 will coincide with that of main device 2.

[0149] In this embodiment, the smaller the angle of view of the virtual camera VC (the higher the magnification ratio of the virtual space), the greater the degree of image stabilization. Here, the degree of image stabilization is a value indicating how much the change in the attitude of the virtual camera VC is delayed relative to a change in the attitude of the main unit 2. The smaller the degree of image stabilization, the faster the attitude of the virtual camera VC approaches that of the main unit 2, and the greater the degree of image stabilization, the slower the attitude of the virtual camera VC approaches that of the main unit 2. For example, when the angle of view of the virtual camera VC is set to the third angle of view A (first angle of view < third angle of view A < second angle of view), the main unit 2 may set the attitude of the virtual camera VC to 90% of its own attitude when it detects a change in its own attitude. Also, when the angle of view of the virtual camera VC is set to the second angle of view, the main unit 2 may set the attitude of the virtual camera VC to 80% of its own attitude when it detects a change in its own attitude. When the angle of view of the virtual camera VC is set to the first angle of view, the degree of correction is "0", and when the main device 2 detects a change in the attitude of its own device, it sets the attitude of the virtual camera VC to 100% of the attitude of its own device.

[0150] Typically, in VR, if the movement of the virtual camera is not matched with the movement of the user (such as the movement of the goggle device 150) and the posture of the virtual camera is corrected, the movement of the real user will not match the movement of the image in the VR space seen by the user, which may cause VR sickness. For this reason, in the field of VR, the movement of the user is usually matched with the movement of the virtual camera. However, the image display system 1 of this embodiment provides a function for enlarging a portion of the VR space by narrowing the angle of view of the virtual camera VC. When the angle of view of the virtual camera VC is narrowed and the posture of the virtual camera VC is changed to match the change in posture of the goggle device 150, slight hand movements (shake) may cause the VR space to shake, potentially causing VR sickness. For this reason, in this embodiment, when the angle of view of the virtual camera VC is narrowed, the effects of hand shake are reduced by image stabilization. As a result, VR sickness can be reduced even when the angle of view of the virtual camera VC is narrowed and a portion of the virtual space is enlarged.

[0151] Note that the method for bringing the attitude of the virtual camera closer to the attitude of the main body device 2 is not limited to the above. For example, the attitude of the virtual camera VC may be set based on an attitude (e.g., a weighted average) calculated from the latest attitude of the main body device 2 and the immediately preceding attitude of the main body device 2.

[0152] As another method of image stabilization, the attitude of the virtual camera VC may not be changed when the change in the attitude of the main body device 2 is equal to or less than a predetermined threshold, and the attitude of the virtual camera VC may be changed when the change in the attitude of the main body device 2 exceeds the predetermined threshold. In this case, the smaller the angle of view of the virtual camera VC, the larger the predetermined threshold may be. For example, when the angle of view of the virtual camera VC is a first angle of view, the predetermined threshold may be set to "0," and when the angle of view of the virtual camera VC is a second angle of view, the predetermined threshold may be set to a value greater than "0."

[0153] In either method of image stabilization, when the angle of view of virtual camera VC is the first angle of view, image stabilization is not performed, and when the attitude of main body device 2 changes, the attitude of virtual camera VC may be set in accordance with the attitude of main body device 2 after the change. On the other hand, when the angle of view of virtual camera VC is the second angle of view, image stabilization is performed to reduce the change in the attitude of virtual camera VC when the attitude of main body device 2 changes, and the attitude of virtual camera VC may be set in accordance with the result of the image stabilization.

[0154] As described above, in this embodiment, by reducing the angle of view of the virtual camera VC, left-eye and right-eye images are generated by enlarging a portion of the virtual space, and portions of the display areas of the enlarged left-eye and right-eye images are cut off. Because the peripheral portions of the enlarged left-eye and right-eye images are cut off, the user is prevented from viewing portions that significantly differ from the actual appearance, thereby reducing VR sickness. Furthermore, by continuously reducing the area of ​​the display areas of the left-eye and right-eye images in response to the continuous enlargement of a portion of the virtual space, the user is prevented from feeling uncomfortable and sudden changes are prevented from occurring in the display of the virtual space. This reduces the possibility of VR sickness. Furthermore, the user is made aware that a portion of the virtual space is being enlarged, allowing the user to view the images with that awareness, making VR sickness less likely to occur. Therefore, it is possible to enlarge a portion of the VR space and reduce VR sickness.

[0155] Furthermore, in this embodiment, when the angle of view of the virtual camera VC is reduced to enlarge a portion of the virtual space, the distance between the left virtual camera VCL and the right virtual camera VCR is shortened. This reduces the three-dimensional effect and reduces VR sickness. Furthermore, by continuously reducing the inter-camera distance d in response to continuously enlarging a portion of the virtual space, it is possible to prevent a sudden change in the three-dimensional effect and prevent the user from feeling uncomfortable. This reduces the possibility of VR sickness.

[0156] Furthermore, in this embodiment, camera shake correction is performed when a portion of the virtual space is enlarged. This can reduce, for example, shaking in the VR space due to camera shake, thereby reducing VR sickness. On the other hand, by not performing camera shake correction under normal circumstances, the attitude of the virtual camera VC can be controlled in accordance with changes in the attitude of the main unit 2. This allows the movement of the user to match the movement of the virtual camera VC, reducing the possibility of VR sickness.

[0157] Furthermore, in this embodiment, a portion of the virtual space is enlarged by rotating the cylindrical portion 205, which is modeled after the telephoto zoom lens of a real camera, in the roll direction. Such an operation is closer to operating a real camera than, for example, a user holding a controller and pressing an operation button, tilting an analog stick, or changing the attitude of the controller. Enlarging the virtual space with such an operation similar to operating a real camera can give the user the sensation of operating a real camera, thereby reducing VR sickness.

[0158] When enlarging a portion of the virtual space, instead of reducing the angle of view of the virtual camera VC, it is possible to move the virtual camera VC within the virtual space (bring it closer to the object to be enlarged). In this case, if there is another virtual object in the path of movement of the virtual camera VC, the virtual camera VC may pass through or collide with the other virtual object. Furthermore, the user may visually feel as if they are moving even though they are not actually moving. This may cause a sense of discomfort to the user, which may lead to VR sickness.

[0159] Also, when enlarging a part of the virtual space, it is possible to move the virtual objects or other background closer to the virtual camera VC without moving the virtual camera VC. In this case, the virtual space itself moves, which may cause VR sickness.

[0160] However, in this embodiment, when enlarging a portion of the virtual space, the virtual camera VC and the virtual space itself are not moved, but the angle of view of the virtual camera VC is reduced, so no problems arise due to such movement of the virtual camera VC or virtual space.

[0161] (Processing details) Next, a specific example of processing performed in the main unit 2 will be described. First, data stored in the main unit 2 will be described.

[0162] 26 is a diagram showing an example of data stored in the memory (mainly DRAM 85) of the main unit 2. As shown in FIG. 26, the memory of the main unit 2 stores a predetermined program, main unit sensor data, main unit attitude data, controller attitude data, virtual object data, virtual camera data, image data for the left eye, and image data for the right eye. In addition to these, various other data are also stored in the main unit 2.

[0163] The predetermined program is a program for executing the processing according to the flowchart described below. The predetermined program is stored, for example, in flash memory 84 or a memory card inserted into slot 23, and is read into DRAM 85 when processing starts. Note that the predetermined program may be obtained from another device via a network (for example, a LAN, a WAN, the Internet, etc.).

[0164] The main body sensor data is data output from the acceleration sensor 89 and angular velocity sensor 90 of the main body device 2, and includes data related to acceleration and data related to angular velocity. The main body sensor data is output from the acceleration sensor 89 and angular velocity sensor 90 at predetermined time intervals (for example, 1 / 800 second intervals) and stored in memory.

[0165] The main body attitude data is data that indicates the attitude of the main body device 2, calculated based on main body sensor data (data related to acceleration and / or data related to angular velocity). For example, the main body device 2 calculates a vector that represents the attitude of the main body device 2 based on the angular velocity value from the angular velocity sensor 90, and stores the vector in memory as the main body attitude data. The main body attitude data is also calculated at predetermined time intervals (for example, 1 / 200 second intervals).

[0166] The controller attitude data is data related to the attitude of the controllers (left controller 3 and right controller 4). The main unit 2 acquires controller attitude data from each controller at predetermined time intervals (for example, 1 / 200 second intervals) and stores the acquired controller attitude data in memory. The main unit 2 may acquire from each controller angular velocity values ​​and / or acceleration values ​​detected by the inertial sensors (angular velocity sensors and acceleration sensors) of each controller as controller attitude data. Alternatively, each controller may calculate its attitude based on the angular velocity values ​​and / or acceleration values, and transmit data related to the calculated attitude to the main unit 2.

[0167] The virtual object data is data relating to various virtual objects arranged in a virtual space, and includes data such as the position and shape of each virtual object.

[0168] The virtual camera data is data relating to the left virtual camera VCL and the right virtual camera VCR, and includes data indicating the position, attitude, and angle of view of each virtual camera VC.

[0169] The left-eye image data is an image of the virtual space generated based on the left virtual camera VCL, and is data relating to the image for the left eye displayed in the left display area of ​​the display 12. The right-eye image data is an image of the virtual space generated based on the right virtual camera VCR, and is data relating to the image for the right eye displayed in the right display area of ​​the display 12.

[0170] (Flowchart explanation) Next, details of the processing performed in the main unit 2 will be described. Fig. 27 is a flowchart showing an example of the processing performed in the processor 81 of the main unit 2. Note that Fig. 27 shows only the processing described above, and omits other processing (for example, game processing performed in virtual space in response to user actions, etc.).

[0171] As shown in FIG. 27, processor 81 first performs an initial process (step S100). In the initial process, the attitude of image display system 100 (main unit 2) is initialized. For example, in the initial process, a user is instructed to place image display system 100 including main unit 2 on a table or the like, and the attitude of main unit 2 is initialized. Also in the initial process, an XYZ coordinate system is set in the virtual space, and each virtual object, left virtual camera VCL, right virtual camera VCR, etc. are arranged in the virtual space. After the process of step S100, processor 81 repeatedly executes the processes from step S101 onwards at predetermined frame time intervals (for example, 1 / 60 seconds).

[0172] After the process of step S100, processor 81 acquires main body sensor data (step S101). In step S101, processor 81 also acquires controller attitude data.

[0173] Next, the processor 81 calculates the attitude of the main body device 2 based on the main body sensor data acquired in step S101 (step S102). Specifically, the processor 81 acquires the attitude of the main body device 2 (goggle device 150) by calculating the attitude by integrating the latest angular velocity value output by the angular velocity sensor 90. The acquired attitude of the main body device 2 is stored in memory as main body attitude data.

[0174] Next, the processor 81 determines whether the cylindrical portion 205 of the camera device 200 has rotated in the roll direction (step S103). Specifically, the processor 81 determines whether the left controller 3 (or the right controller 4) provided inside the cylindrical portion 205 has rotated around the y axis, based on the controller attitude data acquired in step S101.

[0175] If it is determined that the cylindrical portion 205 (left controller 3) is not rotating (step S103: NO), the processor 81 sets the attitude of the virtual camera VC according to the attitude of the main unit 2 acquired (calculated) in step S102 (step S104). Specifically, the processor 81 sets the attitudes of the left virtual camera VCL and the right virtual camera VCR so that they match the attitude of the main unit 2, and stores this in memory as attitude data of the virtual camera data. Note that, if the cylindrical portion 205 is not rotating, the processor 81 sets the positions of the left virtual camera VCL and the right virtual camera VCR so that the inter-camera distance d is a predetermined value (e.g., the distance between the left and right eyes of an average person), and stores this in memory as position data of the virtual camera data. Note that, if the cylindrical portion 205 is not rotating, the processor 81 sets the angles of view of the left virtual camera VCL and the right virtual camera VCR to a first angle of view (e.g., 90 degrees), and stores this in memory as angle-of-view data of the virtual camera data.

[0176] Next, processor 81 generates a left-eye image and a right-eye image based on left virtual camera VCL and right virtual camera VCR (step S105). Here, the angle of view of virtual camera VC is set to the first angle of view, and an approximately circular, normal-sized image of the virtual space (left-eye image and right-eye image) is generated. In addition, in step S105, processor 81 performs distortion correction according to the characteristics of lens 153. As a result, a distortion-corrected left-eye image and right-eye image are generated.

[0177] Following step S105, the processor 81 causes the left-eye image and the right-eye image generated in step S105 to be displayed on the display 12 (step S106). As a result, the left-eye image and the right-eye image of the virtual space are displayed in the left display region and the right display region, respectively, of the display 12. Note that a black image is displayed in the non-visible region.

[0178] On the other hand, if it is determined that the cylindrical portion 205 is rotating (step S103: YES), the processor 81 performs an enlarged image display process (step S107). The enlarged image display process of step S107 will be described later.

[0179] When the process of step S106 has been executed or when the process of step S107 has been executed, processor 81 determines whether or not to end the process shown in Fig. 27 (step S108). For example, when a game is played in a VR space, processor 81 determines to end the process shown in Fig. 27 when the game is cleared or when an instruction to end the game is given by the user. When the determination in step S108 is NO, processor 81 executes the process of step S101 again.

[0180] (Enlarged image display processing) Next, the enlarged image display process of step S107 will be described in detail below. Fig. 28 is a flowchart showing an example of the enlarged image display process of step S107.

[0181] 28, the processor 81 sets the angle of view of the virtual camera VC (step S121). Specifically, the processor 81 sets the angle of view of the left virtual camera VCL and the right virtual camera VCR in accordance with the rotation angle of the left controller 3 (cylindrical portion 205) around the y axis.

[0182] Next, processor 81 enables the image stabilization function (step S122). Specifically, processor 81 sets the degree of image stabilization according to the angle of view set in step S121, and enables the image stabilization function. Processor 81 sets the degree of correction so that the smaller the angle of view of virtual camera VC (i.e., the higher the magnification rate), the higher the degree of image stabilization.

[0183] Following step S122, processor 81 sets the attitude of virtual camera VC based on the attitude of main body device 2 acquired (calculated) in step S102 (step S123). Specifically, processor 81 sets the attitudes of left virtual camera VCL and right virtual camera VCR according to the degree of image stabilization set in step S122 so as to approach the attitude of main body device 2. As a result, the smaller the angle of view of virtual camera VC (the higher the magnification rate), the more slowly the attitude of virtual camera VC approaches the attitude of main body device 2.

[0184] Next, the processor 81 sets the inter-camera distance d in accordance with the angle of view set in step S121 (step S124).

[0185] Next, processor 81 generates left-eye and right-eye images by enlarging a portion of the virtual space based on left virtual camera VCL and right virtual camera VCR (step S125). Here, left-eye and right-eye images of normal size are generated by enlarging a portion of the virtual space. Furthermore, processor 81 performs distortion correction in step S125 in accordance with the characteristics of lens 153. As a result, distortion-corrected left-eye and right-eye images are generated. The distortion correction in step S125 is the same process as the distortion correction in step S105 described above.

[0186] Following step S125, the processor 81 determines whether the angle of view of the virtual camera VC set in step S121 is equal to or smaller than a predetermined value (third angle of view A in FIG. 22) (step S126). Note that the processor 81 may determine whether the angle of view of the virtual camera VC is less than the predetermined value in step S126.

[0187] If it is determined that the angle of view of virtual camera VC is equal to or smaller than a predetermined value (step S126: YES), processor 81 reduces the left and right display areas in accordance with the angle of view of virtual camera VC (step S127). For example, processor 81 reduces the display area of ​​the virtual space image by filling in the outer periphery of the normal-sized virtual space image (image for the left eye and image for the right eye) in black, generated in the processing of step S125. Processor 81 reduces the area of ​​the display area of ​​the virtual space image as the angle of view of virtual camera VC becomes smaller.

[0188] Next, processor 81 causes display 12 to display the left eye image and the right eye image (step S128). As a result, the left eye image and the right eye image, which are enlarged portions of the virtual space with the outer periphery cut off, are displayed in the left display region and the right display region, respectively, of display 12. Note that a black image is displayed in the non-visible region.

[0189] On the other hand, if the angle of view of virtual camera VC is not equal to or less than the predetermined value (step S126: NO), processor 81 causes display 12 to display the normal-sized left-eye image and right-eye image generated in the process of step S125 (step S129). Here, the process of reducing the display area of ​​the left and right images as in step S127 is not performed. As a result, the left-eye image and right-eye image, which are images of the same size as in step S106 but which are enlarged portions of the virtual space, are displayed. Note that a black image is displayed in the non-visible area.

[0190] When the process of step S128 has been executed or when the process of step S129 has been executed, processor 81 ends the process shown in Fig. 28 and returns the process to Fig. 27. This concludes the description of the flowcharts shown in Fig. 27 and Fig. 28.

[0191] The processes shown in the above flowcharts are merely examples, and the order and content of the processes may be changed as appropriate.

[0192] (Variation) The image display system of this embodiment has been described above, but the above embodiment is merely an example, and the following modifications may be made, for example.

[0193] For example, in the above embodiment, a left-eye image and a right-eye image having a parallax with respect to each other are generated based on a pair of a left virtual camera VCL and a right virtual camera VCR. In other embodiments, images may be generated based on a single virtual camera, and the generated images may be transformed to generate left-eye images and right-eye images having a parallax with respect to each other. In other words, in this specification, "generating left-eye images and right-eye images based on virtual cameras" includes both generating left-eye images and right-eye images based on a pair of a left virtual camera VCL and a right virtual camera VCR, and generating left-eye images and right-eye images based on a single virtual camera.

[0194] Even when generating a left-eye image and a right-eye image having a parallax with respect to each other using a single virtual camera without using left and right virtual cameras, the left-eye image and the right-eye image are generated so that the parallax differs between the normal state and the enlarged state. Specifically, in the normal state, a left-eye image and a right-eye image having a first parallax are generated based on the single virtual camera, and in the enlarged state, a left-eye image and a right-eye image having a second parallax smaller than the first parallax are generated based on the single virtual camera.

[0195] In the above embodiment, generally circular images are displayed as the left-eye image and the right-eye image in the normal state, and generally similar images (e.g., generally regular dodecagonal images) are displayed in the enlarged state. The shapes of the left-eye image and the right-eye image are merely examples, and images of any other shape may be displayed. For example, polygonal left-eye image and right-eye image may be displayed in the normal state, and generally similar images with smaller display areas may be displayed in the enlarged state.

[0196] In the above embodiment, the outer periphery of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, is painted black to cut off the outer periphery and reduce the display area of ​​the left-eye image and the right-eye image. In other embodiments, other methods may be used to reduce the display area of ​​the left-eye image and the right-eye image, which are enlarged portions of the virtual space. For example, an image of a predetermined color may be displayed on the outer periphery of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, or the outer periphery may be made semi-transparent or blurred, making it difficult or impossible to view the image of the virtual space corresponding to the outer periphery. Furthermore, the outer periphery of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, may be hidden. For example, the display area of ​​the left-eye image and the right-eye image, which are enlarged portions of the virtual space, may be reduced by placing an occlusion object (e.g., a donut-shaped object that is transparent in the center and occludes the outer periphery) that blocks part of the field of view of the virtual camera VC. When a virtual space including such an occluding object is captured with a virtual camera, the central portion of the image is an enlarged version of the image of the virtual space normally displayed, while the peripheral portion is occluded by the occluding object, making the portion of the virtual space normally displayed invisible (or difficult for the user to see). The images in the peripheral portions of the left-eye image and the right-eye image (peripheral images) may be images of a semi-transparent virtual space, images of a blurred virtual space, images of an occluding object located in the virtual space, or images of a predetermined color not located in the virtual space. In other words, the left-eye image and the right-eye image may be generated by any other method as long as a portion of the virtual space normally displayed is enlarged and displayed, and other portions of the virtual space normally displayed are displayed so as to be invisible or difficult to see when enlarged.

[0197] Furthermore, in the above embodiment, when enlarged display is set (for example, in the case of the second angle of view), a predetermined image (peripheral image, for example, a black image) is displayed on the periphery of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, and when enlarged display is not set (for the first angle of view), the predetermined image is not displayed on the periphery of the left-eye image and the right-eye image. In other embodiments, even when enlarged display is not set, a predetermined image may be displayed on the periphery of the left-eye image and the right-eye image. In this case, when enlarged display is not set, the predetermined image displayed on the periphery may be smaller than when enlarged display is set.

[0198] In the above embodiment, the angle of view of the virtual camera is set to be smaller than the normal first angle of view by rotating the cylindrical portion 205 (setting to display a portion of the virtual space more enlarged than normal). In other embodiments, the setting to display a portion of the virtual space enlarged may be performed by any other operation. For example, the magnification of a portion of the virtual space may be continuously changed by operating a button or analog stick on the left controller 3 or the right controller 4. In this case, the inter-camera distance (parallax between the left-eye image and the right-eye image) may also be continuously changed in accordance with the change in the magnification of a portion of the virtual space. Furthermore, zoom-in may be set (i.e., set to the second angle of view) by operating a button (or analog stick) on the controller, and zoom-in may be canceled (i.e., set to the first angle of view) by operating the same or a different button (or analog stick). In this case, the inter-camera distance (parallax between the left-eye image and the right-eye image) may also be changed in accordance with the setting and cancellation of zoom-in.

[0199] In this specification, "continuously changing" the angle of view or inter-camera distance (parallax) of the virtual camera includes changing it in a curved manner as shown in FIG. 22, and changing it linearly as shown in FIG. 23.

[0200] In the above embodiment, the angle of view of the virtual camera VC is continuously changed from the first angle of view to the second angle of view, thereby continuously enlarging a portion of the virtual space. In other embodiments, the angle of view of the virtual camera VC may be set to the first angle of view or a second angle of view smaller than the first angle of view. That is, the angle of view of the virtual camera VC may be set to the first angle of view during normal operation and to the second angle of view during enlargement. In this case, when the angle of view of the virtual camera VC is set to the second angle of view, the area of ​​the display region for the left-eye image and the right-eye image is made smaller than when the angle of view of the virtual camera VC is set to the first angle of view.

[0201] In the above embodiment, when the angle of view of the virtual camera VC is set to the second angle of view, the inter-camera distance d is set to zero. That is, when the magnification is maximum, the parallax between the left-eye image and the right-eye image is set to zero. In other embodiments, when the angle of view of the virtual camera VC is set to the second angle of view, the parallax between the left-eye image and the right-eye image may be set to substantially zero. Here, a state in which there is substantially no parallax includes a state in which there is no parallax at all, and a state in which there is a small parallax but the image appears substantially flat to the user.

[0202] In the above embodiment, a portion of the virtual space is enlarged by reducing the angle of view of the virtual camera VC. In other embodiments, a portion of the virtual space may be enlarged by other methods. In this case, the inter-camera distance d may be shortened in response to the enlargement of the portion of the virtual space. For example, an image of the virtual space may be generated based on the virtual camera VC set to a first angle of view, a portion of the generated image may be cropped and enlarged, and the enlarged image may be displayed as an image for the left eye and an image for the right eye. In this case, the parallax between the image for the left eye and the image for the right eye, which are enlarged portions of the virtual space, may be made smaller than the parallax before enlargement. Even when enlarging a portion of the virtual space by other methods like this, the enlargement ratio of the virtual space may be continuously changed, and the parallax between the image for the left eye and the image for the right eye (e.g., the inter-camera distance d) may be reduced in response to the enlargement ratio.

[0203] In the above embodiment, the attitude of the goggle device 150 (main device 2) is detected based on data from an inertial sensor (angular velocity sensor 90 and / or acceleration sensor 89) built into the main device 2. In other embodiments, the attitude of the goggle device 150 may be detected by other methods. For example, the image display system 100 may include a camera that captures an image of the goggle device 150 from the outside, and the camera may capture an image of the goggle device 150 or a marker attached to the goggle device 150, and the attitude of the goggle device 150 may be acquired based on the captured image. Alternatively, the goggle device 150 may include a camera, and the attitude of the goggle device 150 may be acquired from changes in the image captured by the camera.

[0204] Furthermore, the image display system 100 is not limited to the above-described configuration and may have other configurations. For example, the goggle device 150 and the camera device 200 may be configured as an inseparable integral unit. That is, in the above embodiment, the goggle device 150 has a detachable cylindrical portion 205. However, in other embodiments, the goggle device 150 may have a fixed cylindrical portion 205. For example, the cylindrical portion 205 may be provided on the back side of the goggle device. Furthermore, the goggle device may be used in various states, such as a type that fits the user's face by being held by the user, a type that fits the user's face by being fixed to the user's head, or a type in which the user looks into the device in a placed state. Furthermore, the goggle device included in the image display system may function as a so-called head-mounted display by being worn on the user's head with the main device 2 attached, and may have a helmet shape in addition to a goggle shape.

[0205] In the above configuration, the goggle device and the main device 2 are detachable, but in other configurations, the goggle device and the main device 2 may be configured as an integrated unit. That is, the goggle device may be provided with a display for displaying an image for the left eye and an image for the right eye, and a processor for generating the images.

[0206] In the above embodiment, the display 12 of the goggle device 150 is detachable from the goggle device 150. In other embodiments, the display unit of the goggle device may be fixed to the goggle device. The display unit of the goggle device is not limited to the rectangular display 12 described above. For example, the display unit of the goggle device may have two display units (a left-eye display unit viewed by the user's left eye and a right-eye display unit viewed by the user's right eye). The display unit of the goggle device may have any shape. For example, the display unit of the goggle device itself may be formed in a substantially circular (circular or elliptical) shape. Two left and right display units formed in a square or rectangular shape may be used as the display unit of the goggle device. The display unit of the goggle device may be a display device such as a liquid crystal display device or an organic EL display device, or may be a projection-type display device that projects an image onto a projection surface.

[0207] Furthermore, the configuration of the image display system 1 in the above embodiment is merely an example and is not limited to the above. For example, in the above embodiment, the goggle device is configured with the main device 2 having the display 12 for displaying images and the processor 81 for performing processing to generate images, and the goggle body. That is, the image display system 100 is configured with the goggle device including the goggle body and the main device 2. In other embodiments, the goggle device having a display unit and the information processing device performing processing to generate images may be configured as separate devices, and the image display system 100 may be configured with these multiple devices. In this case, the goggle device and the information processing device may be connected by wire or wirelessly, and the left-eye image and the right-eye image generated in the information processing device may be transmitted to the goggle device and viewed by the user. Alternatively, the information processing device may perform the above-described processing and transmit the results of the processing to the goggle device, and the goggle device may generate the left-eye image and the right-eye image to be viewed by the user. Alternatively, the goggle device and the information processing device may be connected by a network (such as a LAN, a WAN, or the Internet).

[0208] In addition, in this embodiment, a goggle device is used in which the user holds the device in their hand and looks into the display unit, but in other embodiments, the goggle device may be a head-mounted display that is fixedly worn on the user's head.

[0209] The present invention has been described above, but the above description is merely an example of the present invention, and various improvements and modifications may be made thereto. [Explanation of symbols]

[0210] 1. Game System 2 Main unit 3 Left Controller 4 Right Controller 12 Display 81 processors 89 Acceleration Sensor 90 Angular rate sensor 100 Image Display System 150 Goggle device 200 Camera Equipment 205 Cylindrical part VC Virtual Camera VCL Left Virtual Camera VCR Right Virtual Camera

Claims

1. a goggle device; an attitude acquisition means for acquiring an attitude of the goggle device; a virtual camera placement means for placing a virtual camera in a virtual space; a virtual camera attitude control means for controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting means for setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image having a parallax with each other, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera; a display control means for displaying the left-eye image and the right-eye image on a display unit of the goggle device; a parallax setting means for setting a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and for setting a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, The image generating means generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, generating the left-eye image and the right-eye image, which are images obtained by enlarging a part of the virtual space and have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; The virtual camera attitude control means When the angle of view of the virtual camera is set to the first angle of view, when an attitude of the goggle device changes, the attitude of the virtual camera is controlled in accordance with the attitude of the goggle device after the change; When the angle of view of the virtual camera is set to the second angle of view, an image display system performs a correction process to reduce the change in the attitude of the virtual camera when the attitude of the goggle device changes, and controls the attitude of the virtual camera in accordance with the correction process.

2. the angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, The parallax setting means The image display system of claim 1, wherein the virtual distance between the left-eye virtual camera and the right-eye virtual camera is continuously shortened when the angle of view of the virtual camera continuously changes from the first angle of view to the second angle of view.

3. The parallax setting means The image display system according to claim 2 , wherein the virtual distance between the left-eye virtual camera and the right-eye virtual camera is changed linearly in response to a change in the angle of view of the virtual camera.

4. the angle-of-view setting means reduces the angle of view of the virtual camera within a range from the first angle of view to the second angle of view, the image generating means generates the left-eye image and the right-eye image by increasing a magnification ratio of a part of the virtual space in response to a reduction in the angle of view of the virtual camera; 4. The image display system according to claim 1, wherein the parallax setting means reduces the parallax between the left-eye image and the right-eye image as the magnification ratio increases.

5. the goggle device includes lenses for allowing the left eye image and the right eye image to be viewed by the left eye and the right eye of a user, respectively; The image display system according to claim 1 , wherein the first angle of view is set to a viewing angle of the user using the lens.

6. further comprising object placement means for placing a predetermined virtual object in the virtual space; The display control means When the angle of view of the virtual camera is set to the first angle of view, the left-eye image and the right-eye image are generated, each including an image of the predetermined virtual object having a first size; 6. The image display system of claim 1, wherein when the angle of view of the virtual camera is set to the second angle of view, the image for the left eye and the image for the right eye are generated, the image including an image of the specified virtual object having a second size larger than the first size.

7. An image display system as described in any one of claims 1 to 6, wherein when the angle of view of the virtual camera is changed from the first angle of view to the second angle of view, the position of the virtual camera in the imaging direction is not changed, and the virtual distance between the left-eye virtual camera and the right-eye virtual camera is changed from the first distance to the second distance.

8. a change setting unit that sets a degree of change in the attitude of the virtual camera in response to a change in the attitude of the goggle device; The virtual camera attitude control means 8. The image display system according to claim 1, wherein when the angle of view of the virtual camera is set to the second angle of view and the attitude of the goggle device changes, the attitude of the virtual camera is controlled so as to approach the attitude of the goggle device by a degree of change set by the change setting means.

9. The image generating means generating a first left-eye image and a first right-eye image representing a part of the virtual space when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, a second left-eye image and a second right-eye image are generated by enlarging a part of the virtual space; The display control means When the angle of view of the virtual camera is set to the first angle of view, the first left-eye image and the first right-eye image are displayed in a display area of ​​a first size; 9. An image display system according to claim 1, wherein when the angle of view of the virtual camera is set to the second angle of view, the second left eye image and the second right eye image are each displayed in a display area of ​​a second size smaller than the first size.

10. 10. The image display system according to claim 9, wherein the display area of ​​the second size is a central area of ​​the display area of ​​the first size excluding at least an outer periphery thereof.

11. the angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, The image display system according to claim 9 or 10, wherein the display control means continuously reduces the display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, in accordance with the continuous change in the angle of view of the virtual camera from the first angle of view to the second angle of view.

12. An image display program executed by a processor of a device that displays an image on a display unit of a goggle device, the processor comprising: an attitude acquisition means for acquiring an attitude of the goggle device; a virtual camera placement means for placing a virtual camera in a virtual space; a virtual camera attitude control means for controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting means for setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera, and having a parallax therebetween, to be displayed on a display unit of the goggle device; a parallax setting unit that sets a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and that sets a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view; The image generating means generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, the left-eye image and the right-eye image are images that are enlarged portions of the virtual space, and the left-eye image and the right-eye image have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; The virtual camera attitude control means When the angle of view of the virtual camera is set to the first angle of view, when an attitude of the goggle device changes, the attitude of the virtual camera is controlled in accordance with the attitude of the goggle device after the change; an image display program that, when the angle of view of the virtual camera is set to the second angle of view, performs a correction process to reduce the change in the attitude of the virtual camera when the attitude of the goggle device changes, and controls the attitude of the virtual camera in accordance with the correction process.

13. An image display method performed in an image display system including a goggle device, the image display system comprising: an attitude acquisition step of acquiring an attitude of the goggle device; a virtual camera placement step of placing a virtual camera in a virtual space; a virtual camera attitude control step of controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting step of setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generation step of generating a left-eye image and a right-eye image, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera and having a parallax therebetween, to be displayed on a display unit of the goggle device; a parallax setting step of setting a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and setting a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, In the image generating step, generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, the left-eye image and the right-eye image are images that are enlarged portions of the virtual space, and the left-eye image and the right-eye image have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; In the disparity setting step, when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; In the virtual camera attitude control step, When the angle of view of the virtual camera is set to the first angle of view, when an attitude of the goggle device changes, the attitude of the virtual camera is controlled in accordance with the attitude of the goggle device after the change; an image display method, in which, when the angle of view of the virtual camera is set to the second angle of view, a correction process is performed to reduce the change in the attitude of the virtual camera when the attitude of the goggle device changes, and the attitude of the virtual camera is controlled in accordance with the correction process.

14. A goggle device; an attitude acquisition means for acquiring an attitude of the goggle device; a virtual camera placement means for placing a virtual camera in a virtual space; a virtual camera attitude control means for controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting means for setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image having a parallax with each other, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera; a display control means for displaying the left-eye image and the right-eye image on a display unit of the goggle device; a parallax setting means for setting a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and for setting a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, The image generating means generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, generating the left-eye image and the right-eye image, which are images obtained by enlarging a part of the virtual space and have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; The image generating means generating a first left-eye image and a first right-eye image representing a part of the virtual space when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, a second left-eye image and a second right-eye image are generated by enlarging a part of the virtual space; The display control means When the angle of view of the virtual camera is set to the first angle of view, the first left-eye image and the first right-eye image are displayed in a display area of ​​a first size; An image display system that, when the angle of view of the virtual camera is set to the second angle of view, displays each of the second left eye image and the second right eye image in a display area of ​​a second size that is smaller than the first size.

15. An image display system as described in Claim 14, wherein the display area of ​​the second size is a central area excluding at least the outer peripheral portion of the display area of ​​the first size.

16. The angle-of-view setting means is capable of continuously changing the angle of view of the virtual camera from the first angle of view to the second angle of view, The image display system of claim 14 or 15, wherein the display control means continuously reduces the display areas of the left-eye image and the right-eye image, which are enlarged portions of the virtual space, in accordance with the continuous change in the angle of view of the virtual camera from the first angle of view to the second angle of view.

17. An image display program executed by a processor of a device that displays an image on a display unit of a goggle device, the processor comprising: an attitude acquisition means for acquiring an attitude of the goggle device; a virtual camera placement means for placing a virtual camera in a virtual space; a virtual camera attitude control means for controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting means for setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generating means for generating a left-eye image and a right-eye image, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera, and having a parallax therebetween, to be displayed on a display unit of the goggle device; a parallax setting unit that sets a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and that sets a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view; The image generating means generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, the left-eye image and the right-eye image are images that are enlarged portions of the virtual space, and the left-eye image and the right-eye image have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; The parallax setting means when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; The image generating means generating a first left-eye image and a first right-eye image representing a part of the virtual space when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, a second left-eye image and a second right-eye image are generated by enlarging a part of the virtual space; The display control means When the angle of view of the virtual camera is set to the first angle of view, the first left-eye image and the first right-eye image are displayed in a display area of ​​a first size; An image display program that, when the angle of view of the virtual camera is set to the second angle of view, displays each of the second left eye image and the second right eye image in a display area of ​​a second size that is smaller than the first size.

18. An image display method carried out in an image display system including a goggle device, comprising: an attitude acquisition step of acquiring an attitude of the goggle device; a virtual camera placement step of placing a virtual camera in a virtual space; a virtual camera attitude control step of controlling the attitude of the virtual camera in accordance with the attitude of the goggle device; a view angle setting step of setting the view angle of the virtual camera to at least one of a first view angle and a second view angle smaller than the first view angle; an image generation step of generating a left-eye image and a right-eye image, the left-eye image and the right-eye image being images of the virtual space included in the angle of view of the virtual camera and having a parallax therebetween, to be displayed on a display unit of the goggle device; a parallax setting step of setting a parallax between the left-eye image and the right-eye image to a first parallax when the angle of view of the virtual camera is set to the first angle of view, and setting a parallax between the left-eye image and the right-eye image to a second parallax smaller than the first parallax when the angle of view of the virtual camera is set to the second angle of view, In the image generating step, generating the left-eye image and the right-eye image having the first parallax with respect to each other when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, the left-eye image and the right-eye image are images that are enlarged portions of the virtual space, and the left-eye image and the right-eye image have the second parallax with respect to each other; the virtual camera includes a left-eye virtual camera for generating the left-eye image and a right-eye virtual camera for generating the right-eye image; In the disparity setting step, when the angle of view of the virtual camera is set to the first angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a first distance, thereby setting the first parallax; when the angle of view of the virtual camera is set to the second angle of view, a virtual distance between the left-eye virtual camera and the right-eye virtual camera is set to a second distance that is shorter than the first distance, thereby setting the second parallax; In the image generating step, generating a first left-eye image and a first right-eye image representing a part of the virtual space when the angle of view of the virtual camera is set to the first angle of view; When the angle of view of the virtual camera is set to the second angle of view, a second left-eye image and a second right-eye image are generated by enlarging a part of the virtual space; In the display control step, When the angle of view of the virtual camera is set to the first angle of view, the first left-eye image and the first right-eye image are displayed in a display area of ​​a first size; An image display method in which, when the angle of view of the virtual camera is set to the second angle of view, the second left eye image and the second right eye image are each displayed in a display area of ​​a second size smaller than the first size.

Images