Image display system, image display program, image display method and display device

Abstract

To provide an image display system, an image display program, an image display method and a display device that can improve convenience associated with presentation of a user interface image.SOLUTION: An image display system comprises: a display device that has a display screen; and a goggle device that can wear the display device. The image display system displays a first image including a content image being a non-stereoscopic image and a first user interface image on a display screen in a first display mode, and displays a second image including a content image which is composed of a left-eye image and a right-eye image having a parallax to each other, and a second user interface image corresponding to the first user interface image, on the display screen in a second display mode. The image display system displays the second user interface image at a position being different from a position on the display screen where the first user interface image is displayed in the first display mode, in the second display mode.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to an image display system, an image display program, an image display method, and a display device capable of displaying stereoscopic images. [Background technology]

[0002] Conventionally, there has been a stereoscopic image display device that displays a stereoscopic image by allowing a user to view two images with a parallax between them with their left and right eyes (see, for example, Patent Document 1). For example, Patent Document 1 discloses a device in which a smartphone is housed in a goggle device that can be worn by a user, and the user can view a stereoscopic image displayed on the display screen of the smartphone through the goggle device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-198651 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the stereoscopic image display device disclosed in Patent Document 1 does not take into consideration the presentation of a user interface image that accepts touch operations by the user when displaying a stereoscopic image on the display screen, and therefore there is room for improvement in the convenience of the method of presenting the user interface image.

[0005] Therefore, an object of the present invention is to provide an image display system, an image display program, an image display method, and a display device that can improve the convenience of presenting user interface images. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention may adopt, for example, the following configurations: It should be understood that when interpreting the claims, the scope should be interpreted solely by the claims, and in the event of a contradiction between the claims and this section, the claims shall take precedence.

[0007] One configuration example of an image display system of the present invention includes a display device having a display screen for displaying images and a goggle device on which the display device can be worn. The image display system includes a mode setting means, a first display control means, and a second display control means. The mode setting means sets the display device to a first display mode or a second display mode different from the first display mode. The first display control means causes the display screen to display a first image, which includes a content image that is a non-stereoscopic image and a first user interface image, on the first display mode. The second display control means causes the display screen to display a second image, which includes a content image consisting of a left-eye image and a right-eye image that have parallax between them, and a second user interface image corresponding to the first user interface image, on the display screen in the second display mode. The second display control means causes the second user interface image to be displayed in a position on the display screen different from the position at which the first user interface image is displayed in the first display mode.

[0008] According to the above, when the display mode is switched, the user interface image is displayed at a different position, which makes it possible to improve convenience in terms of presenting the user interface image.

[0009] The image display system further includes a detection means for detecting whether the display device is attached to the goggle device or whether it is in the process of being attached. In this case, the mode setting means may switch the display device to the second display mode based on the detection result of the detection means when the display device is set to the first display mode.

[0010] According to the above, the display mode is switched based on the state in which the display device is attached to the goggle device or the state in which the display device is being attached, so that seamless switching of the display mode is possible.

[0011] The display device may also include an illuminance sensor. In this case, the goggle device may also include a light-blocking unit. The light-blocking unit blocks light from reaching the illuminance sensor of the display device when the display device is attached to the goggle device or in the process of being attached to the goggle device. The detection means may detect whether the display device is attached to the goggle device or in the process of being attached to the goggle device based on the detection result of the illuminance sensor.

[0012] Based on the above, it is possible to easily detect whether the display device is in an attached state or in an in-attachment state, based on the light-blocking state of the display device.

[0013] The display device may also include a touch panel provided on the display screen. In this case, a first user interface image displayed on the display screen may be capable of providing operational instructions in response to a touch operation on the touch panel. A second user interface image displayed on the display screen may be capable of providing operational instructions in response to a touch operation on the touch panel when the display device is attached to a goggle device.

[0014] Based on the above, it is possible to improve convenience in presenting a user interface image that accepts a touch operation.

[0015] Furthermore, the second display control means may cause an image corresponding to the content image displayed as a non-stereoscopic image by the first display control means to be displayed on the display screen as a content image consisting of an image for the left eye and an image for the right eye.

[0016] Based on the above, it is possible to seamlessly switch between a stereoscopic image and a non-stereoscopic image in the same content image.

[0017] In addition, in the second display mode, the second display control means may convert the content image that was displayed in the first display mode immediately before being set to the second display mode into a stereoscopic image consisting of an image for the left eye and an image for the right eye, and display it on the display screen.

[0018] Based on the above, it is possible to seamlessly switch between a stereoscopic image and a non-stereoscopic image in the same content image.

[0019] In addition, the second display control means may, in the second display mode, display an image for the left eye in a first area of ​​the display screen, display an image for the right eye in a second area of ​​the display screen different from the first area, and display a second user interface image in a third area of ​​the display screen different from the first and second areas.

[0020] According to the above, it is possible to prevent the first and second areas for displaying a stereoscopic image from being soiled by touch operations, and it is also possible to prevent fingers for touching the second user interface image from coming into view when viewing a stereoscopic image.

[0021] The second display control means may also display, as the second user interface image, a user interface image that has substantially the same function as the first user interface image but a different shape.

[0022] Based on the above, it is possible to display a user interface image having a shape suitable for operation and stereoscopic image display.

[0023] In addition, in the second display mode, the second display control means may adjust the shape of the second user interface image to fit the shape of a third area of ​​the display screen that is different from the first area of ​​the display screen that displays an image for the left eye and the second area of ​​the display screen that displays an image for the right eye, and display the second user interface image in the third area.

[0024] Based on the above, it is possible to display a user interface image in an appropriate shape.

[0025] The second display control means may set a third area above or below the display screen between the first area and the second area on the display screen.

[0026] Based on the above, it is possible to display a user interface image that does not interfere with the stereoscopic image display.

[0027] The second display control means may set a third area below the display screen between the first area and the second area on the display screen.

[0028] Based on the above, it is possible to display a user interface image that does not interfere with the stereoscopic image display and that is easy to operate.

[0029] The goggle device may also have an opening that exposes at least a third region, which is a part of the display screen, to the outside when the display device is attached to the goggle device.

[0030] Based on the above, a touch operation of touching a user interface image can be performed while wearing the goggle device.

[0031] The opening may be formed at a position corresponding to the nose of a user when the user wears the goggle device.

[0032] According to the above, the opening can be formed without impairing the light blocking property.

[0033] Furthermore, in the first display mode, the first display control means may display the first user interface image so as to be superimposed on the content image displayed on the display screen.

[0034] Based on the above, a relatively large content image can be displayed without being affected by the display of the user interface image.

[0035] Furthermore, the second display control means may cause the second user interface image to be displayed on the display screen as a non-stereoscopic image in the second display mode.

[0036] Based on the above, the operation of touching the user interface image becomes easy.

[0037] The display device may further include a display device-side connection section electrically connectable to another device. In this case, the goggle device may include a goggle device-side connection section electrically connectable to the display device-side connection section. The detection means may detect whether the display device is attached to the goggle device or whether it is in the process of being attached in response to the connection between the display device-side connection section and the goggle device-side connection section.

[0038] Based on the above, it is possible to reliably detect that the display device has been attached to the goggle device, or that the display device is in the process of being attached.

[0039] The present invention may also be embodied in the form of an image display program, an image display method, and a display device. [Effects of the Invention]

[0040] According to the present invention, when the display mode is switched, the user interface image is displayed in a different position, thereby improving the convenience of presenting the user interface image. [Brief explanation of the drawings]

[0041] [Figure 1] A diagram showing the left controller 3 and right controller 4 attached to the main unit 2. [Figure 2]FIG. 10 shows an example of a state in which the left controller 3 and the right controller 4 are detached from the main unit 2. [Figure 3] Six-sided views showing an example of the main unit 2 [Figure 4] Six-sided diagram showing an example of the left controller 3 [Figure 5] Six-sided diagram showing an example of the right controller 4 [Figure 6] A block diagram showing an example of the internal configuration of the main unit 2. [Figure 7] A block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. [Figure 8] FIG. 1 is a perspective view showing an example of the appearance of a goggle device 150. [Figure 9] FIG. 1 is a front view showing an example of a state in which the main body device 2 is attached to the goggle device 150. [Figure 10] FIG. 1 is a front view showing an example of a state of the main device 2 attached to the goggle device 150. [Figure 11] FIG. 10 is a diagram showing an example of the shape of a front contact portion 151b that comes into contact with a part of the front surface of the main body device 2. [Figure 12] FIG. 1 is a diagram showing an example of the internal structure of a goggle device 150. [Figure 13] FIG. 1 is a side view showing an example of a state of the main device 2 attached to the goggle device 150. [Figure 14] FIG. 10 is a diagram showing an example of a user viewing an image displayed on an image display system. [Figure 15] FIG. 10 is a diagram showing an example of a user holding an image display system. [Figure 16] 10A and 10B are diagrams showing examples of images displayed on the main unit 2 in a stereoscopic display mode and a non-stereoscopic display mode, respectively; [Figure 17] FIG. 10 is a diagram showing an example of a data area set in the DRAM 85 of the main device 2. [Figure 18] A flowchart showing an example of game processing executed by the game system 1. DETAILED DESCRIPTION OF THE INVENTION

[0042] An image display system according to an example of this embodiment will be described below. The example of the image display system according to this embodiment is configured by a game system 1 (a main unit 2 included in the game system 1 as a minimum configuration) and a goggle device 150. The example of the game system 1 includes a main unit (information processing device; in this embodiment, it functions as a game device main unit) 2, a left controller 3, and a right controller 4. The left controller 3 and the right controller 4 are each detachable from the main unit 2. In other words, the game system 1 can be used as an integrated device by attaching the left controller 3 and the right controller 4 to the main unit 2. The game system 1 can also be used by separating the main unit 2 from the left controller 3 and the right controller 4 (see FIG. 2). The hardware configuration of the game system 1 according to this embodiment will be described below, followed by a description of the control of the game system 1 according to this embodiment.

[0043] FIG. 1 is a diagram showing an example of a state in which a left controller 3 and a right controller 4 are attached to a main unit 2. As shown in FIG. 1, the left controller 3 and the right controller 4 are each attached to and integrated with the main unit 2. The main unit 2 is a device that executes various processes (e.g., game processes) in the game system 1. The main unit 2 is equipped with a display 12. The left controller 3 and the right controller 4 are devices that have operation units that allow the user to perform inputs.

[0044] Fig. 2 is a diagram showing an example of the state in which the left controller 3 and the right controller 4 are detached from the main unit 2. As shown in Figs. 1 and 2, the left controller 3 and the right controller 4 are detachable from the main unit 2. Note that, below, the left controller 3 and the right controller 4 may be collectively referred to as "controllers."

[0045] Fig. 3 is a six-sided view showing an example of the main unit 2. As shown in Fig. 3, the main unit 2 includes a substantially plate-shaped housing 11. In this embodiment, the main surface of the housing 11 (in other words, the front surface, i.e., the surface on which the display 12 is provided) is generally rectangular.

[0046] The shape and size of the housing 11 are arbitrary. As an example, the housing 11 may be of a portable size. Furthermore, the main unit 2 alone or an integrated device in which the left controller 3 and right controller 4 are attached to the main unit 2 may be a portable device. Furthermore, the main unit 2 or the integrated device may be a handheld device. Furthermore, the main unit 2 or the integrated device may be a portable device.

[0047] 3, the main unit 2 includes a display 12 provided on the main surface of the housing 11. The display 12 displays images generated by the main unit 2. In this embodiment, the display 12 is a liquid crystal display (LCD). However, the display 12 may be any type of display device.

[0048] The main device 2 also includes a touch panel 13 on the screen of the display 12. In this embodiment, the touch panel 13 is of a type that allows multi-touch input (for example, a capacitance type). However, the touch panel 13 may be of any type, and may be of a type that allows single-touch input (for example, a resistive type).

[0049] The main unit 2 is provided with a speaker (i.e., speaker 88 shown in FIG. 6) inside the housing 11. As shown in FIG. 3, speaker holes 11a and 11b are formed on the main surface of the housing 11. The output sound of the speaker 88 is output from these speaker holes 11a and 11b, respectively.

[0050] The main unit 2 also has a left terminal 17, which is a terminal for the main unit 2 to communicate with the left controller 3 via a wired connection, and a right terminal 21, which is a terminal for the main unit 2 to communicate with the right controller 4 via a wired connection.

[0051] As shown in FIG. 3, the main unit 2 includes a slot 23. The slot 23 is provided on the upper side of the housing 11. The slot 23 has a shape that allows a predetermined type of storage medium to be inserted therein. The predetermined type of storage medium is, for example, a storage medium (e.g., a dedicated memory card) dedicated to the game system 1 and the same type of information processing device. The predetermined type of storage medium is used, for example, to store data used by the main unit 2 (e.g., application save data, etc.) and / or programs executed by the main unit 2 (e.g., application programs, etc.). The main unit 2 also includes a power button 28.

[0052] The main unit 2 has a lower terminal 27. The lower terminal 27 is a terminal through which the main unit 2 communicates with the cradle. In this embodiment, the lower terminal 27 is a USB connector (more specifically, a female connector). When the all-in-one device or the main unit 2 alone is placed on the cradle, the game system 1 can display images generated and output by the main unit 2 on a stationary monitor. In this embodiment, the cradle also has the function of charging the all-in-one device or the main unit 2 alone that is placed on it. The cradle also has the function of a hub device (specifically, a USB hub).

[0053] The main device 2 includes an illuminance sensor 29. In this embodiment, the illuminance sensor 29 is provided below the main surface of the housing 11 and detects the illuminance (brightness) of light incident from the main surface side of the housing 11. Note that, depending on the illuminance of light detected by the illuminance sensor 29, the display 12 can be adjusted to an appropriate brightness to display an image, and in this embodiment, whether the main device 2 is attached to the goggle device 150 (described later) is determined based on the detected illuminance.

[0054] FIG. 4 is a six-sided view showing an example of the left controller 3. As shown in FIG. 4, the left controller 3 includes a housing 31. In this embodiment, the housing 31 has a vertically long shape, that is, a shape that is long in the up-down direction (i.e., the y-axis direction shown in FIGS. 1 and 4). The left controller 3 can also be held in a vertically long orientation when detached from the main unit 2. The housing 31 has a shape and size that allows it to be held in one hand, particularly the left hand, when held in a vertically long orientation. The left controller 3 can also be held in a horizontally long orientation. When the left controller 3 is held in a horizontally long orientation, it may be held with both hands.

[0055] The left controller 3 includes an analog stick 32. As shown in FIG. 4, the analog stick 32 is provided on the main surface of the housing 31. The analog stick 32 can be used as a direction input unit that can input directions. By tilting the analog stick 32, the user can input a direction corresponding to the tilt direction (and input a magnitude corresponding to the tilt angle). Note that instead of an analog stick, the left controller 3 may be equipped with a cross key or a slide stick that can perform slide inputs as a direction input unit. In this embodiment, input can be made by pressing the analog stick 32.

[0056] The left controller 3 is equipped with various operation buttons. The left controller 3 is equipped with four operation buttons 33 to 36 (specifically, a right button 33, a down button 34, an up button 35, and a left button 36) on the main surface of the housing 31. The left controller 3 also is equipped with a record button 37 and a - (minus) button 47. The left controller 3 is equipped with a first L button 38 and a ZL button 39 on the upper left of the side of the housing 31. The left controller 3 is also equipped with a second L button 43 and a second R button 44 on the side of the housing 31 that is attached to the main unit 2. These operation buttons are used to issue instructions according to various programs (for example, OS programs and application programs) executed on the main unit 2.

[0057] The left controller 3 also includes a terminal 42 for wired communication between the left controller 3 and the main unit 2.

[0058] FIG. 5 is a six-sided view showing an example of the right controller 4. As shown in FIG. 5, the right controller 4 includes a housing 51. In this embodiment, the housing 51 has a vertically long shape, that is, a shape that is long in the up-down direction. The right controller 4 can also be held in a vertically long orientation when detached from the main unit 2. The housing 51 has a shape and size that allows it to be held in one hand, particularly the right hand, when held in a vertically long orientation. The right controller 4 can also be held in a horizontally long orientation. When the right controller 4 is held in a horizontally long orientation, it may be held with both hands.

[0059] Like the left controller 3, the right controller 4 is equipped with an analog stick 52 as a directional input unit. In this embodiment, the analog stick 52 has the same configuration as the analog stick 32 of the left controller 3. The right controller 4 may also be equipped with a cross key or a slide stick that allows slide input, instead of an analog stick. Like the left controller 3, the right controller 4 is equipped with four operation buttons 53 to 56 (specifically, an A button 53, a B button 54, an X button 55, and a Y button 56) on the main surface of the housing 51. The right controller 4 is also equipped with a + (plus) button 57 and a home button 58. The right controller 4 is also equipped with a first R button 60 and a ZR button 61 on the upper right side of the housing 51. Like the left controller 3, the right controller 4 is also equipped with a second L button 65 and a second R button 66.

[0060] The right controller 4 also includes a terminal 64 for wired communication between the right controller 4 and the main unit 2.

[0061] Fig. 6 is a block diagram showing an example of the internal configuration of main unit 2. In addition to the configuration shown in Fig. 3, main unit 2 includes components 81-91, 97, and 98 shown in Fig. 6. Some of these components 81-91, 97, and 98 may be mounted on an electronic circuit board as electronic components and housed in housing 11.

[0062] The main unit 2 includes a processor 81. The processor 81 is an information processing unit that executes various types of information processing executed in the main unit 2, and may be composed of, for example, only a CPU (Central Processing Unit), or may be composed of an SoC (System-on-a-chip) that includes multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function. The processor 81 executes various types of information processing by executing an information processing program (for example, a game program) stored in a storage unit (specifically, an internal storage medium such as flash memory 84, or an external storage medium inserted into slot 23, etc.).

[0063] The main device 2 includes a flash memory 84 and a DRAM (Dynamic Random Access Memory) 85 as examples of internal storage media built into the main device 2. The flash memory 84 and the DRAM 85 are connected to the processor 81. The flash memory 84 is a memory used primarily to store various types of data (which may be programs) saved in the main device 2. The DRAM 85 is a memory used to temporarily store various types of data used in information processing.

[0064] The main device 2 includes a slot interface (hereinafter abbreviated as "I / F") 91. The slot I / F 91 is connected to the processor 81. The slot I / F 91 is connected to the slot 23, and reads and writes data from and to a predetermined type of storage medium (e.g., a dedicated memory card) inserted into the slot 23 in accordance with instructions from the processor 81.

[0065] The processor 81 reads and writes data from and to the flash memory 84, DRAM 85, and the above-mentioned storage media as appropriate, to execute the above-mentioned information processing.

[0066] The main unit 2 includes a network communication unit 82. The network communication unit 82 is connected to the processor 81. The network communication unit 82 communicates with external devices via a network (specifically, wireless communication). In this embodiment, the network communication unit 82 connects to a wireless LAN and communicates with external devices using a method conforming to the Wi-Fi standard as a first communication mode. The network communication unit 82 also performs wireless communication with other main units 2 of the same type using a predetermined communication method (e.g., communication using a proprietary protocol or infrared communication) as a second communication mode. Note that wireless communication using the second communication mode enables wireless communication with other main units 2 located within a closed local network area, and realizes a function that enables so-called "local communication," in which data is transmitted and received by direct communication between multiple main units 2.

[0067] The main unit 2 is equipped with a controller communication unit 83. The controller communication unit 83 is connected to the processor 81. The controller communication unit 83 performs wireless communication with the left controller 3 and / or right controller 4. Any communication method may be used between the main unit 2 and the left controller 3 and right controller 4, but in this embodiment, the controller communication unit 83 performs communication with the left controller 3 and right controller 4 in accordance with the Bluetooth (registered trademark) standard.

[0068] The processor 81 is connected to the left terminal 17, right terminal 21, and lower terminal 27. When performing wired communication with the left controller 3, the processor 81 transmits data to the left controller 3 via the left terminal 17 and receives operation data from the left controller 3 via the left terminal 17. When performing wired communication with the right controller 4, the processor 81 transmits data to the right controller 4 via the right terminal 21 and receives operation data from the right controller 4 via the right terminal 21. When performing wired communication with the right controller 4, the processor 81 transmits data to the cradle via the lower terminal 27. As described above, in this embodiment, the main unit 2 can perform both wired and wireless communication with the left controller 3 and the right controller 4. When an integrated device in which the left controller 3 and the right controller 4 are attached to the main unit 2 or the main unit 2 alone is attached to the cradle, the main unit 2 can output data (e.g., image data and audio data) to a stationary monitor or the like via the cradle.

[0069] Here, the main unit 2 can communicate simultaneously (in other words, in parallel) with multiple left controllers 3. The main unit 2 can also communicate simultaneously (in other words, in parallel) with multiple right controllers 4. Therefore, multiple users can simultaneously input to the main unit 2 using their own sets of left controllers 3 and right controllers 4. For example, a first user can input to the main unit 2 using a first set of left controllers 3 and right controllers 4, while a second user can simultaneously input to the main unit 2 using a second set of left controllers 3 and right controllers 4.

[0070] The main device 2 includes a touch panel controller 86, which is a circuit that controls the touch panel 13. The touch panel controller 86 is connected between the touch panel 13 and the processor 81. Based on a signal from the touch panel 13, the touch panel controller 86 generates data indicating, for example, the position where a touch input was made, and outputs the data to the processor 81.

[0071] The display 12 is also connected to the processor 81. The processor 81 displays on the display 12 an image generated (for example, by executing the above-described information processing) and / or an image acquired from the outside.

[0072] The main unit 2 includes a codec circuit 87 and speakers (specifically, a left speaker and a right speaker) 88. The codec circuit 87 is connected to the speakers 88 and the audio input / output terminal 25, and is also connected to the processor 81. The codec circuit 87 is a circuit that controls the input and output of audio data to and from the speakers 88 and the audio input / output terminal 25.

[0073] The main unit 2 also includes an acceleration sensor 89. In this embodiment, the acceleration sensor 89 detects the magnitude of acceleration along three predetermined axes (for example, the x, y, and z axes shown in FIG. 1). The acceleration sensor 89 may also detect acceleration along one or two axes.

[0074] The main body device 2 also includes an angular velocity sensor 90. In this embodiment, the angular velocity sensor 90 detects angular velocities around three predetermined axes (for example, the x, y, and z axes shown in FIG. 1). Note that the angular velocity sensor 90 may also detect angular velocities around one axis or two axes.

[0075] The acceleration sensor 89 and the angular velocity sensor 90 are connected to the processor 81, and the detection results of the acceleration sensor 89 and the angular velocity sensor 90 are output to the processor 81. The processor 81 can calculate information related to the movement and / or attitude of the main unit 2 based on the detection results of the acceleration sensor 89 and the angular velocity sensor 90.

[0076] Illuminance sensor 29 is connected to processor 81, and the detection result of illuminance sensor 29 is output to processor 81. Processor 81 can calculate information about the brightness around main unit 2 based on the detection result of illuminance sensor 29.

[0077] The main device 2 includes a power control unit 97 and a battery 98. The power control unit 97 is connected to the battery 98 and the processor 81. Although not shown, the power control unit 97 is also connected to each part of the main device 2 (specifically, each part that receives power from the battery 98, the left terminal 17, and the right terminal 21). The power control unit 97 controls the power supply from the battery 98 to each of the above parts based on instructions from the processor 81.

[0078] Furthermore, battery 98 is connected to lower terminal 27. When an external charging device (e.g., a cradle) is connected to lower terminal 27 and power is supplied to main device 2 via lower terminal 27, battery 98 is charged with the supplied power.

[0079] Figure 7 is a block diagram showing an example of the internal configuration of the main unit 2, left controller 3, and right controller 4. Note that details of the internal configuration of the main unit 2 are omitted in Figure 7 because they are shown in Figure 6.

[0080] The left controller 3 is equipped with a communication control unit 101 that communicates with the main unit 2. As shown in FIG. 7 , the communication control unit 101 is connected to each component, including the terminal 42. In this embodiment, the communication control unit 101 can communicate with the main unit 2 both via wired communication via the terminal 42 and via wireless communication without using the terminal 42. The communication control unit 101 controls the method of communication between the left controller 3 and the main unit 2. That is, when the left controller 3 is attached to the main unit 2, the communication control unit 101 communicates with the main unit 2 via the terminal 42. When the left controller 3 is detached from the main unit 2, the communication control unit 101 communicates wirelessly with the main unit 2 (specifically, with the controller communication unit 83). Wireless communication between the controller communication unit 83 and the communication control unit 101 is performed in accordance with, for example, the Bluetooth (registered trademark) standard.

[0081] The left controller 3 also includes a memory 102, such as a flash memory. The communication control unit 101 is configured, for example, by a microcomputer (also called a microprocessor), and executes firmware stored in the memory 102 to perform various processes.

[0082] The left controller 3 includes buttons 103 (specifically, buttons 33 to 39, 43, 44, and 47). The left controller 3 also includes an analog stick (referred to as "stick" in FIG. 7) 32. Each button 103 and analog stick 32 repeatedly outputs information related to operations performed on the button 103 and analog stick 32 to the communication control unit 101 at appropriate timing.

[0083] The communication control unit 101 acquires information about the input (specifically, information about the operation or the detection results by the sensors) from each input unit (specifically, each button 103, analog stick 32, and each sensor 104 and 105). The communication control unit 101 transmits operation data including the acquired information (or information obtained by performing a predetermined process on the acquired information) to the main unit 2. The operation data is repeatedly transmitted once every predetermined time. The interval at which the information about the input is transmitted to the main unit 2 may or may not be the same for each input unit.

[0084] By transmitting the above operation data to the main unit 2, the main unit 2 can obtain the input made to the left controller 3. In other words, the main unit 2 can determine the operation of each button 103 and analog stick 32 based on the operation data.

[0085] The left controller 3 is equipped with a power supply unit 108. In this embodiment, the power supply unit 108 has a battery and a power control circuit. Although not shown, the power control circuit is connected to the battery and to each part of the left controller 3 (specifically, each part that receives power from the battery).

[0086] As shown in FIG. 7, the right controller 4 is equipped with a communication control unit 111 that communicates with the main unit 2. The right controller 4 also has a memory 112 that is connected to the communication control unit 111. The communication control unit 111 is connected to each component, including the terminal 64. The communication control unit 111 and memory 112 have the same functions as the communication control unit 101 and memory 102 of the left controller 3. Therefore, the communication control unit 111 can communicate with the main unit 2 both via wired communication via the terminal 64 and via wireless communication that does not use the terminal 64 (specifically, communication in accordance with the Bluetooth (registered trademark) standard), and controls the method of communication between the right controller 4 and the main unit 2.

[0087] The right controller 4 has input units similar to those of the left controller 3. Specifically, it has buttons 113 and an analog stick 52. These input units have the same functions as those of the left controller 3 and operate in the same manner.

[0088] The right controller 4 is equipped with a power supply unit 118. The power supply unit 118 has the same functions as the power supply unit 108 of the left controller 3 and operates in the same manner.

[0089] Next, with reference to FIGS. 8 to 15, a goggle device 150 will be described as an example of a device that constitutes an image display system by wearing the game system 1 (specifically, the main body device 2). FIG. 8 is a perspective view showing an example of the appearance of the goggle device 150. FIG. 9 is a front view showing an example of a state in which the main body device 2 is worn on the goggle device 150. FIG. 10 is a front view showing an example of a state in which the main body device 2 is worn on the goggle device 150. FIG. 11 is a diagram showing an example of the shape of a front surface abutment 151b that comes into contact with a part of the front surface of the main body device 2. FIG. 12 is a diagram showing an example of the internal structure of the goggle device 150. FIG. 13 is a side view showing an example of a state in which the main body device 2 is worn on the goggle device 150. FIG. 14 is a diagram showing an example of a user viewing an image displayed on the image display system. FIG. 15 is a diagram showing an example of a user holding the image display system. In addition, Figure 11 is a view from the same direction as Figure 10, with parts of the goggle device 150 (part of the main body 151, lens frame member 152, lens 153, and plate-shaped member 154) removed (transparent state) to clearly show the front abutment portion 151b.

[0090] 8 to 13, goggle device 150 includes main body 151, lens frame member 152, lens 153, and plate-shaped member 154. Here, the goggle device, which is an example of a device constituting an image display system, is not limited to the configuration described below as long as it is worn by fitting to the user's face so as to cover the user's left and right eyes, has the function of blocking at least a portion of external light, and has the function of supporting the user's stereoscopic vision with a pair of lenses. For example, the goggle device may be used in various ways, such as a type that is held by the user and fits to the user's face, a type that is fixed to the user's head and fits to the user's face, or a type that the user looks into while the device is placed on the user's head. Furthermore, the goggle device may function as a so-called head-mounted display by being worn on the user's head with main body device 2 attached, and may have a helmet shape in addition to a goggle shape. In the following description of goggle device 150, a goggle-type goggle device that is held by the user and worn to fit the user's face will be used.

[0091] The main body 151 has an attachment portion that detachably fixes the main body device 2 by contacting the front, rear, top, and bottom surfaces of the main body device 2. The attachment portion has a front abutment portion that contacts a portion of the front surface (the surface on which the display 12 is provided) of the main body device 2, a rear abutment portion that contacts the rear surface of the main body device 2, an upper abutment portion that contacts the top surface of the main body device 2, and a lower abutment portion that contacts the bottom surface of the main body device 2, and is formed in a rectangular tube shape with open left and right sides and a gap formed between the front abutment portion, rear abutment portion, upper abutment portion, and lower abutment portion. The attachment portion has open sides (the side on the positive x-axis direction and the side on the negative x-axis direction in the figure) so that it can be attached from the left or right side of the main body device 2. 9, when the goggle device 150 is attached through the opening on the right side of the main body device 2, the front contact portion contacts the front surface of the main body device 2, the back contact portion contacts the back surface of the main body device 2, the upper contact portion contacts the upper surface of the main body device 2, and the lower contact portion contacts the lower surface of the main body device 2. Note that, as shown in FIG. 11, the front contact portion 151b of the main body 151 has an opening formed therein so as not to obstruct at least the view of the images (images for the left eye and the right eye) displayed on the display 12 when the main body device 2 is attached.

[0092] 9 and 13, main body device 2 is attached to goggle device 150 by sliding it from the left or right side of main body device 2 into the gap in the attachment portion of main body 151 along the front abutment portion, rear abutment portion, upper abutment portion, and lower abutment portion of the attachment portion. Furthermore, main body device 2 can be removed from goggle device 150 by sliding it left or right along the front abutment portion, rear abutment portion, upper abutment portion, and lower abutment portion of the attachment portion while attached to goggle device 150. In this way, main body device 2 can be detachably attached to goggle device 150.

[0093] The lens frame member 152 is fixed to the opening formed on the front surface of the main body 151. The lens frame member 152 has a pair of lens frames that are opened so as not to obstruct the field of view of the display images (left-eye image IML and right-eye image IMR) displayed on the display 12 of the main body device 2 attached to the main body 151. Furthermore, the outer edges formed on the top, bottom, left, and right of the lens frame member 152 are formed with bonding surfaces for bonding to the main body device 2, and a V-shaped recess is formed in the center of the outer edge formed on the lower side to come into contact with the nose of the user wearing the goggle device 150.

[0094] The lens 153 is composed of a pair of left-eye lens 153L and right-eye lens 153R, for example, a pair of Fresnel lenses. The left-eye lens 153L and the right-eye lens 153R are each fitted into a lens frame of the lens frame member 152. Specifically, the left-eye lens 153L is fitted into one of the lens frames that is open so as not to obstruct the field of view of the left-eye image IML displayed on the display 12 of the main unit 2 attached to the main body 151, so that the user can see the left-eye image IML when looking into the left-eye lens 153L with their left eye. The right-eye lens 153R is fitted into the other lens frame that is open so as not to obstruct the field of view of the right-eye image IMR displayed on the display 12 of the main unit 2 attached to the main body 151, so that the user can see the right-eye image IMR when looking into the right-eye lens 153R with their right eye. Note that left-eye lens 153L and right-eye lens 153R may typically be circular or elliptical magnifying lenses, and may be lenses that distort an image to allow a user to view it. For example, left-eye lens 153L may distort a left-eye image IML (described below) displayed in a distorted circular or elliptical shape in the direction opposite to the distortion of the image to allow the user to view it, and right-eye lens 153R may distort a right-eye image IMR (described below) displayed in a distorted circular or elliptical shape in the direction opposite to the distortion of the image to allow the user to view it, thereby providing a stereoscopic view of the image. Furthermore, left-eye lens 153L and right-eye lens 153R may be configured to be integrally formed.

[0095] The main body 151 has an abutment portion that protrudes outward from the front side of the main body 151 so as to surround the outer edge of the lens frame member 152 in a rectangular cylindrical shape. The end face of the abutment portion that protrudes outward from the front side is disposed on the near side of the lens 153 when the lens 153 is viewed from outside the goggle device 150, and the end face is disposed on the nearest side (the negative z-axis direction side) of the goggle device 150 when the main body device 2 is worn. The abutment portion of the main body 151 has an end face shape that fits the user's face (typically, the area around the user's eyes) when the user looks into the goggle device 150 with the main body device 2 worn, and has the function of fixing the positional relationship between the user's eyes and the lens 153 by the end face abutting against the user's face.

[0096] Furthermore, the contact portions can block external light from reaching the left-eye lens 153L and the right-eye lens 153R when viewing a stereoscopic image displayed on the display 12 using an image display system. This can improve the sense of immersion of the user viewing the stereoscopic image displayed on the display 12. Note that the light blocking provided by the contact portions does not have to completely block external light. For example, as shown in FIG. 15, a recess may be formed in a portion of the cylindrical contact portion. Note that the recess in the contact portion illustrated in FIG. 15 is formed at a position below the midpoint between the left-eye lens 153L and the right-eye lens 153R, which is the position where the nose of a user viewing a stereoscopic image displayed on the display 12 will come into contact. In other words, the recess in the contact portion can prevent the contact portion from coming into strong contact with the user's nose, thereby reducing discomfort caused by the contact of the contact portion with the nose even if the light blocking ability is somewhat reduced.

[0097] As shown in FIG. 10 , the plate-like member 154 is fixed inside the main body 151 between the lens frame member 152 and the display 12 when the main body device 2 is attached to the attachment portion of the main body 151. For example, the plate-like member 154 is shaped so that a portion thereof conforms to the V-shaped recess in the lens frame member 152, and is disposed like a wall (hereinafter referred to as a first wall portion) connecting the recess and the display 12 of the attached main body device 2. The space surrounded by the first wall portion serves as an opening 154h that exposes a portion of the display 12 of the main body device 2 attached to the main body 151 to the outside and functions as an operation window that allows a user to touch and operate the portion through the space. Note that the first wall portion of the plate-like member 154 may have an opening as shown in FIG. 12 .

[0098] 12, for example, plate-like member 154 is provided vertically between left-eye lens 153L and right-eye lens 153R, and is arranged as a wall (hereinafter referred to as a second wall portion) connecting the recess and display 12 of attached main body device 2. When main body device 2 is attached to main body 151, the second wall portion is arranged so as to separate the left-eye image IML and the right-eye image IMR displayed on display 12, and functions as a partition wall provided between the left-eye image IML and the right-eye image IMR. Plate-like member 154 is provided by extending the first wall portion to the second wall portion, and the first wall portion and the second wall portion are formed by an integrated member.

[0099] 10 , 13 , 14 , and 15 , the image display system is configured by mounting a main device 2 on a goggle device 150. In this embodiment, the main device 2 is mounted so that the entire main device 2 is covered by the goggle device 150. When the main device 2 is mounted on the goggle device 150, only the left-eye image IML displayed in the left region of the display 12 can be seen through the left-eye lens 153L, and only the right-eye image IMR displayed in the right region of the display 12 can be seen through the right-eye lens 153R. Therefore, a user of the image display system can view the left-eye image IML and the right-eye image IMR by viewing the left-eye lens 153L with his / her left eye and the right-eye lens 153R with his / her right eye. Therefore, by displaying the left-eye image IML and the right-eye image IMR with parallax on the display 12, a stereoscopic image that gives the user a sense of three-dimensionality can be displayed.

[0100] 14 and 15 , when a user holds an image display system in which main device 2 is attached to goggle device 150 and views a stereoscopic image displayed on display 12, the user can hold the left side of goggle device 150 with main device 2 attached in the left hand and the right side of goggle device 150 with the right hand. By holding the left and right sides of goggle device 150 in this way, the user can maintain a stable wearing state of main device 2.

[0101] Furthermore, even when the main body device 2 is attached to the goggle device 150, the image display system allows a user to touch a part of the touch panel 13 provided on the screen of the display 12 (a third area of ​​the display 12, which will be described later) through the opening 154h formed by the first wall portion of the plate-like member 154. The image display system can calculate information about the movement and / or attitude of the main body device 2, i.e., the movement and / or attitude of the image display system including the goggle device 150, based on the detection results of the acceleration sensor 89 and / or the angular velocity sensor 90 provided in the main body device 2. Therefore, the image display system can calculate the attitude of the user's head looking into the goggle device 150 with the main body device 2 attached, based on the direction of gravity. Furthermore, when the attitude or direction of the user's head looking into the goggle device 150 with the main body device 2 attached changes, the image display system can calculate the changed direction or angle. Furthermore, when a user looking into the goggle device 150 with the main body device 2 attached applies a vibration similar to hitting the image display system, the image display system can detect the vibration. Therefore, when the main device 2 is attached to the goggle device 150 and the stereoscopic image displayed on the display 12 is viewed through the left eye lens 153L and the right eye lens 153R, a play style is realized in which touch operations are possible through the opening 154h, operations based on the posture of the image display system based on the gravity direction, operations to change the posture of the image display system, and operations to apply vibration to the image display system.

[0102] When using the image display system of this embodiment, operation may be performed using at least one of the left controller 3 and the right controller 4 detached from the main unit 2. For example, when operating the image display system using the left controller 3, the user holds the goggle device 150 to which the main unit 2 is attached in the right hand while viewing a stereoscopic image displayed on the display 12, and holds the detached left controller 3 alone in the left hand to perform operation. In this case, operation information performed on the left controller 3 and / or the right controller 4 detached from the main unit 2 is transmitted to the main unit 2 via wireless communication with the main unit 2. Specifically, operation information performed on the left controller 3 is wirelessly transmitted from the communication control unit 101 of the left controller 3 and received by the controller communication unit 83 of the main unit 2. Operation information performed on the right controller 4 is wirelessly transmitted from the communication control unit 111 of the right controller 4 and received by the controller communication unit 83 of the main unit 2.

[0103] As described above, in this embodiment, by attaching the main device 2 to the goggle device 150, a portable image display system can be configured that the user can hold and view a stereoscopic image. Furthermore, in the image display system of this embodiment, the user views a stereoscopic image displayed on the display 12 of the main device 2 while placing the user's face against the goggle device 150. Therefore, the positional relationship between the stereo speakers (left speaker 88L and right speaker 88R) provided in the main device 2 and the user's ears is fixed, and the left and right speakers are located near the user's ears. Therefore, the main device 2 can output audio based on the positional relationship between the audio output device and the viewer's ears, without forcing the use of earphones or speakers. For example, the main device 2 can control a sound source using so-called stereophonic technology based on the positional relationship between the audio output device and the viewer's ears.

[0104] Next, images displayed on the main unit 2 will be described with reference to Fig. 9, Fig. 10, and Fig. 16. Fig. 16 shows examples of images displayed on the main unit 2 in the stereoscopic display mode and the non-stereoscopic display mode.

[0105] The image display system in this embodiment is set to either a stereoscopic display mode used when the main device 2 is attached to the goggle device 150 and an image displayed on the display 12 is stereoscopically viewed, or a non-stereoscopic display mode used when the main device 2 is detached from the goggle device 150 and an image displayed on the display 12 is viewed directly for non-stereoscopic viewing. The image display system then displays an image corresponding to the set mode on the display 12 of the main device 2. Here, the stereoscopic image for stereoscopic viewing may be an image for right eye and an image for left eye, which have parallax between them, viewed by the user with the right and left eyes, so that the user can view the image stereoscopically. In this case, the non-stereoscopic image for non-stereoscopic viewing is an image other than the two-image display (stereoscopic display) described above, and may typically be an image viewed by the user with the right and left eyes. In this embodiment, the non-stereoscopic display mode is used as an example of the first display mode. In addition, in this embodiment, the stereoscopic display mode is used as an example of the second display mode.

[0106] In the stereoscopic display mode, the image display system configures a content image to be displayed (for example, an image for displaying a part of a virtual space or a real space) with a left-eye image IML and a right-eye image IMR having parallax therebetween, and displays the left-eye image IML in a left region of the display 12 and the right-eye image IML in a right region of the display 12. Specifically, as shown in FIG. 9 , in the stereoscopic display mode, the left-eye image IML is a substantially elliptical region viewable by the left-eye lens 153L when the main device 2 is attached to the goggle device 150, and is displayed in a first region that is a part of the left region of the display 12. In addition, in the stereoscopic display mode, the right-eye image IMR is a substantially elliptical region viewable by the right-eye lens 153R when the main device 2 is attached to the goggle device 150, and is displayed in a second region that is a part of the right region of the display 12.

[0107] Here, as described above, when the main body device 2 is attached to the goggle device 150, the second wall portion of the plate-shaped member 154 is disposed between the left-eye image IML displayed in the first area and the right-eye image IMR displayed in the second area of ​​the display 12. Therefore, the left-eye image IML and the right-eye image IMR are separated by the second wall portion of the plate-shaped member 154 as a partition, and it is possible to prevent the right-eye image IMR from being viewed through the left-eye lens 153L or the left-eye image IML from being viewed through the right-eye lens 153R.

[0108] As an example, images of a virtual space viewed from a pair of virtual cameras (a left virtual camera and a right virtual camera) arranged in the virtual space with a parallax between them are generated as a left-eye image (IML) and a right-eye image (IMR), respectively. The pair of virtual cameras are arranged in the virtual space to correspond to the orientation of the main unit 2 relative to the direction of gravity in real space. The pair of virtual cameras then change their orientation in the virtual space to correspond to changes in the orientation of the main unit 2 in real space, and the line-of-sight direction of the virtual cameras is controlled according to the orientation of the main unit 2. This allows a user wearing the image display system to change the display range of the stereoscopically viewed virtual space by changing the orientation of the main unit 2 (image display system) as if looking around the surroundings, thereby providing an experience of being in the location of the virtual cameras. Note that in this embodiment, the main unit 2 aligns the direction of gravitational acceleration acting on the main unit 2 with the direction of gravity in the virtual space acting on the virtual cameras, and also aligns the amount of change in the orientation of the main unit 2 with the amount of change in the line-of-sight direction of the virtual cameras. This increases the realism of the operation of looking around the stereoscopic virtual space depending on the orientation of the main device 2.

[0109] The image display system also displays on the display 12 a user interface image IMU for receiving a touch operation on the touch panel 13 of the main unit 2. For example, the user interface image IMUa displayed in the stereoscopic display mode is displayed in a display area of ​​the display 12 that can be touched via the opening 154h of the goggle device 150. For example, as described above, the opening 154h is formed by being surrounded by the first wall portion of the plate-like member 154, and enables a touch operation to be performed on a part of the display 12 of the main unit 2 attached to the goggle device 150 (specifically, an area near the center of the bottom of the display 12) from the V-shaped recess in the lens frame member 152 that abuts against the user's nose. As an example, as shown in FIG. 9 , the opening 154h enables a touch operation to be performed on a third area of ​​the display 12 that is located below the display 12 and is sandwiched between the first area and the second area, even when the main unit 2 is attached to the goggle device 150.

[0110] For example, in the stereoscopic display mode, two user interface images IMUa1 and IMUa2 are displayed in the third region of the display 12. As an example, the user interface image IMUa1 is an operation icon that, when a display position of the user interface image IMUa1 is touched via the touch panel 13, issues an operation instruction to retry the game from the beginning. The user interface image IMUa2 is an operation icon that, when a display position of the user interface image IMUa1 is touched via the touch panel 13, issues an operation instruction to end the game. The two user interface images IMUa1 and IMUa2 are displayed side by side in a size that fits the shape of the third region in the third region of the display 12. This allows the user to touch one of the two user interface images IMUa1 and IMUa2 through the opening 154h, thereby enabling multiple operation instructions by touch operation even when the main unit 2 is attached to the goggle device 150. The two user interface images IMUa1 and IMUa2 may be displayed near the third region, which allows touch operation by exposing a portion of the display 12 to the outside. That is, a portion of the two user interface images IMUa1 and / or IMUa2 may be displayed outside the third area. In this embodiment, the user interface image IMUa is used as an example of the second user interface image.

[0111] The two user interface images IMUa1 and IMUa2 are images displayed in the stereoscopic display mode, but are displayed in a third region on the display 12 of the main device 2 attached to the goggle device 150, outside the first region visible to the user's left eye and the second region visible to the user's right eye. Therefore, the user interface images IMUa1 and IMUa2 displayed in the third region are displayed outside the field of view of the user viewing through the goggle device 150. Furthermore, because they are not composed of two images with parallax between them, they are displayed as non-stereoscopic images that cannot be viewed stereoscopically. Furthermore, because the user interface images IMUa1 and IMUa2, which are the target of touch operations, are displayed outside the first and second regions for displaying stereoscopic images, touch operations on the first and second regions are reduced. Therefore, the first and second regions for displaying stereoscopic images can be prevented from being soiled by touch operations on the display 12, and fingers for touch operations can be prevented from entering the field of view while viewing stereoscopic images.

[0112] In another embodiment, the user interface images IMUa1 and IMUa2 may be displayed on the display 12 as stereoscopic images that can be viewed stereoscopically in the stereoscopic display mode. In this case, the user interface images IMUa1 and IMUa2 are displayed on the display 12 as stereoscopic images by being composed of two images each having parallax, and typically, one image for stereoscopic viewing is displayed in a portion of the first area, and the other image for stereoscopic viewing is displayed in a portion of the second area.

[0113] As shown in Figure 16, in the non-stereoscopic display mode, the image display system constructs the content image to be displayed as a single image IMS, which is a non-stereoscopic image, and displays the single image IMS over the entire display area of ​​the display 12, as an example.

[0114] As an example, an image of a virtual space viewed from a single virtual camera disposed in the virtual space is generated as a single image IMS. The single virtual camera is disposed in the virtual space so as to correspond to the orientation of the main unit 2 relative to the direction of gravity in the real space. The single virtual camera then changes its orientation in the virtual space so as to correspond to changes in the orientation of the main unit 2 in the real space, and the line-of-sight direction of the virtual camera is controlled in accordance with the orientation of the main unit 2. This allows a user holding the main unit 2 detached from the goggle device 150 to change the display range of the virtual space displayed on the display 12 and view the virtual space by changing the orientation of the main unit 2 as if looking around, thereby providing an experience of being in the location of the virtual camera. Note that in this embodiment, even in the non-stereoscopic display mode, the main unit 2 aligns the direction of gravitational acceleration acting on the main unit 2 with the direction of gravity in the virtual space acting on the virtual camera, and aligns the amount of change in the orientation of the main unit 2 with the amount of change in the line-of-sight direction of the virtual camera. This increases realism even when looking around a non-stereoscopic virtual space depending on the orientation of the main unit 2. In this embodiment, a single image IMS is used as an example of a non-stereoscopic image.

[0115] Furthermore, the user interface image IMUb displayed in the non-stereoscopic display mode is displayed on the display 12, for example, superimposed on a content image (single image IMS) displayed on the display 12. For example, as shown in FIG. 16, two user interface images IMUb1 and IMUb2 are displayed on the display 12 even in the non-stereoscopic display mode. As an example, the user interface image IMUb1 is an image corresponding to the user interface image IMUa1, and is an operation icon that issues an operation instruction to retry the game from the beginning when the display position is touched via the touch panel 13. The user interface image IMUb2 is an image corresponding to the user interface image IMUa2, and is an operation icon that issues an operation instruction to end the game when the display position is touched via the touch panel 13. Here, the image corresponding to the user interface image IMUa has a different design and / or size but is substantially identical in function to the user interface image IMUa (for example, the operation instruction content when touched is the same). Note that the image corresponding to the user interface image IMUa may not only be substantially identical in function to the user interface image IMUa displayed in the non-stereoscopic display mode, but may also be the same in design and size, i.e., completely identical. Note that in this embodiment, the user interface image IMUb is used as an example of the first user interface image.

[0116] The two user interface images IMUb1 and IMUb2 are displayed in corner regions (for example, the upper left corner region and the upper right corner region) of the display 12 that are different from the third region. Note that the two user interface images IMUb1 and IMUb2 are not limited by the region where touch operations are possible, and therefore can be enlarged more than the user interface image IMUa displayed in the stereoscopic display mode, and can be displayed in a size and shape that is easy for the user to touch, and in a position where the visibility of the content image (single image IMS) is unlikely to be impaired by touch operations.

[0117] The image display system in this embodiment can automatically switch between the stereoscopic display mode and the non-stereoscopic display mode based on a detection result of whether the main device 2 is attached to the goggle device 150 or is in an in-attachment state. For example, the main device 2 is provided with an illuminance sensor 29 that detects the illuminance (brightness) of light incident from the main surface side of the housing 11. Based on the illuminance detection result by the illuminance sensor 29, it can detect whether the main device 2 is attached to the goggle device 150 or is in an in-attachment state. Specifically, when the main device 2 is attached to the goggle device 150 or is in an in-attachment state, the illuminance detected by the illuminance sensor 29 becomes dark. Therefore, by setting a threshold value capable of detecting the darkened illuminance and detecting whether the illuminance is equal to or greater than the threshold value, it becomes possible to detect whether the main device 2 is attached to the goggle device 150 or is in an in-attachment state. Here, the "attached state" detected by the main device 2 based on the illuminance detection result by the illuminance sensor 29 means that the main device 2 is completely attached to the goggle device 150. Furthermore, based on the illuminance detection results by the illuminance sensor 29, the "being in the middle of being worn" detected by the main unit 2 means that the main unit 2 is in a state preceding the state in which it is completely worn on the goggle device 150.

[0118] There are several possible modes for detecting whether main device 2 is being worn or in the process of being worn by illuminance sensor 29. As a first example, as shown in Fig. 11, in the wearing section of goggle device 150 that detachably fixes main device 2, front surface abutment portion 151b of main body 151 that comes into contact with part of the front surface (the surface on which display 12 is provided) of main device 2 in the worn state is formed so as to cover the light receiving surface and light receiving hole of illuminance sensor 29, and whether main device 2 is being worn or in the process of being worn to goggle device 150 is detected by front surface abutment portion 151b covering the light receiving surface and light receiving hole of illuminance sensor 29. As an example, if the light receiving surface or light receiving hole of the illuminance sensor 29 is provided in a corner of the main device 2, when the main device 2 is inserted into the wearing section of the goggle device 150 from the corner side, the light receiving surface or light receiving hole will be covered by the front abutment portion 151b at the beginning of the wearing operation, making it possible to detect that the main device 2 is in an "in progress of wearing" state based on the illuminance detection result by the illuminance sensor 29. Also, when the main device 2 is inserted into the wearing section of the goggle device 150 from the side opposite to the corner where the light receiving surface or light receiving hole of the illuminance sensor 29 is provided, the light receiving surface or light receiving hole will be covered by the front abutment portion 151b at the end of the wearing operation, making it possible to detect that the main device 2 is in an "in progress of wearing" state based on the illuminance detection result by the illuminance sensor 29. As another example, if the light receiving surface or light receiving hole of illuminance sensor 29 is provided on the lower central side of main device 2, when main device 2 is inserted into the wearing portion of goggle device 150, the light receiving surface or light receiving hole will be covered by front abutment portion 151b in the early to middle stages of the wearing operation regardless of the direction of insertion, so it becomes possible to detect that main device 2 is "in the middle of being worn" based on the illuminance detection result by illuminance sensor 29. In the first example above, front abutment portion 151b of main body 151 is used as an example of a light blocking portion.

[0119] As a second example, even if the front contact portion 151b is not formed to cover the light receiving surface or light receiving hole of the illuminance sensor 29, the relative darkness inside the main body 151 can be used to detect whether the main body 2 is in an attached state or in an in-attachment state to the goggle device 150. For example, a state in which external light is incident on the main body 151 of the goggle device 150 via the left eye lens 153L and the right eye lens 153R can be considered. Even in this state, the attached state of the main body 2 is detected by detecting that the illuminance is darker than outside the main body 151. As an example, if the light receiving surface or light receiving hole of the illuminance sensor 29 is provided in a corner of the main body 2, when the main body 2 is inserted into the attachment portion of the goggle device 150 from the corner side, the light receiving surface or light receiving hole is inserted into the main body 151 at the beginning of the attachment operation, so that it is possible to detect that the main body 2 is in an “in-attachment state” based on the illuminance detection result by the illuminance sensor 29. Furthermore, when main device 2 is inserted into the wearing section of goggle device 150 from the side opposite the corner where the light receiving surface and light receiving hole of illuminance sensor 29 are provided, the light receiving surface and light receiving hole are inserted into main body 151 at the end of the wearing operation, making it possible to detect that main device 2 is in a "wearing state" based on the illuminance detection result by illuminance sensor 29. As another example, if the light receiving surface and light receiving hole of illuminance sensor 29 are provided at the center of the lower part of main device 2, when main device 2 is inserted into the wearing section of goggle device 150, the light receiving surface and light receiving hole are inserted into main body 151 at the beginning to middle of the wearing operation regardless of the insertion direction, making it possible to detect that main device 2 is in a "wearing in progress" state based on the illuminance detection result by illuminance sensor 29. Note that in the second example above, main body 151 is used as another example of a light-blocking section.

[0120] As a third example, even if the front contact portion 151b is not formed to cover the light receiving surface or light receiving hole of the illuminance sensor 29, the fact that the inside of the main body 151 becomes darker when the user wears the goggle device 150 to which the main body 2 is attached is used to detect that the main body 2 is attached to the goggle device 150. For example, when the user is not wearing the goggle device 150, it is conceivable that external light enters the main body 151 of the goggle device 150 through the left eye lens 153L and the right eye lens 153R. In this case, when the user wears the goggle device 150, the external light entering through the left eye lens 153L and the right eye lens 153R is blocked, and the illuminance inside the main body 151 becomes darker, thereby detecting the wearing state of the main body 2. As an example, when light incident on the light receiving surface or light receiving hole of illuminance sensor 29 disposed in main body 151 of goggle device 150 is blocked by main body 151 and the user's face, it is possible to detect that main body device 2 is "in a worn state" based on the illuminance detection result by illuminance sensor 29. Furthermore, in this example, it is possible to detect not only whether main body device 2 is in a worn state on goggle device 150, but also whether the user is wearing goggle device 150 to which main body device 2 is attached. Note that in the above third example, main body 151 is used as another example of a light blocking portion.

[0121] When the image display system in this embodiment determines that the main unit 2 is not attached to the goggle device 150, it sets the display mode of the main unit 2 to the non-stereoscopic display mode. On the other hand, when it determines that the main unit 2, which is set to the non-stereoscopic display mode, has changed from a state in which the main unit 2 is not attached to the goggle device 150 to a state in which the main unit 2 is attached, the main unit 2 changes the displayed non-stereoscopic content image to a stereoscopic image, thereby displaying the same content image on the display 12. As an example, when a virtual space image is generated by setting a single virtual camera in the virtual space to display a single image IMS in the non-stereoscopic display mode, the main unit 2 changes the single virtual camera to a pair of virtual cameras (a left virtual camera and a right virtual camera) having parallax between them without changing the position and line of sight of the single virtual camera, and sets the virtual cameras to display the left-eye image IML and the right-eye image IMR, thereby switching to generation of the virtual space image in the stereoscopic display mode. Furthermore, when the main unit 2 is set to the stereoscopic display mode and it is determined that the main unit 2 has changed from a state in which it is wearing the goggle device 150 to a state in which it is not wearing the goggle device 150, the main unit 2 changes the content image of the stereoscopic image being displayed to a non-stereoscopic image, thereby displaying the same content image corresponding to the content image of the stereoscopic image as a non-stereoscopic image on the display 12. As an example, when a pair of virtual cameras is set in the virtual space to display a left-eye image IML and a right-eye image IMR in the stereoscopic display mode to generate a virtual space image, the main unit 2 changes the pair of virtual cameras to a single virtual camera without changing the positions and line-of-sight directions, and sets a virtual camera for displaying the single image IMS, thereby switching to generation of a virtual space image in the non-stereoscopic display mode. In this way, the only difference between a content image of a non-stereoscopic image (e.g., a virtual space image) corresponding to a content image of a stereoscopic image (e.g., a virtual space image) and a content image of a stereoscopic image (e.g., a virtual space image) corresponding to a content image of a non-stereoscopic image (e.g., a virtual space image) is whether the image is a stereoscopic image or a non-stereoscopic image.However, the content image of a non-stereoscopic image corresponding to the content image of a stereoscopic image, or the content image of a stereoscopic image corresponding to the content image of a non-stereoscopic image, may have different display ranges, and typically the content image of a stereoscopic image may have a narrower display range than the content image of a non-stereoscopic image.

[0122] Furthermore, when the display mode is switched, the image display system of this embodiment changes the size, shape, and position of the user interface image IMU and displays it on the display 12. For example, when the main unit 2 is set to the non-stereoscopic display mode and it is determined that the main unit 2 has changed from a non-attached state to an attached state of the goggle device 150, the main unit 2 changes the shape of the user interface images IMUb1 and IMUb2, which are displayed overlapping the content image in the corner region of the display 12, to user interface images IMUa1 and IMUa2, and moves the display position into a third region of the display 12, thereby displaying the user interface image IMU with the same function. Furthermore, when the main unit 2 is set to the stereoscopic display mode and it is determined that the main unit 2 has changed from a state in which it is attached to the goggle device 150 to a state in which it is not attached, the main unit 2 changes the shape of the user interface images IMUa1 and IMUa2 displayed in the third area of ​​the display 12 to user interface images IMUb1 and IMUb2, and moves the display position so that it overlaps with the content image in the corner area of ​​the display 12, thereby displaying the user interface image IMU with the same function.

[0123] In the above-described embodiment, the example in which whether the main device 2 is attached to the goggle device 150 or in the middle of being attached is detected based on the illuminance detection result by the illuminance sensor 29 is used. However, whether the main device 2 is attached to the goggle device 150 or in the middle of being attached may be detected based on other detection results. As an example, whether the main device 2 is attached to the goggle device 150 or in the middle of being attached may be detected based on a detection result of an electrical connection between a connection terminal provided on the main device 2 and a connection terminal provided on the goggle device 150 when the main device 2 is in the attached state or the middle of being attached, or a detection result of a predetermined switch mechanism provided on the main device 2 being turned on or off when the main device 2 is in the attached state or the middle of being attached. As another example, whether the main device 2 is attached to the goggle device 150 or in the middle of being attached may be detected by determining whether a predetermined image is captured or whether the luminance of the captured image is equal to or greater than a threshold based on an imaging result obtained by an imaging means (image sensor) provided on the main device 2. As another example, when the main unit 2 is attached to the goggle device 150 or is in the process of being attached, the user may be prompted to perform a predetermined operation, and based on whether the predetermined operation has been performed, it may be detected whether the main unit 2 is attached to the goggle device 150 or is in the process of being attached.

[0124] In the above-described embodiment, the third area is set below the center of the display 12 sandwiched between the first area and the second area, enabling a touch operation on the third area. However, the third area may be set in another area of ​​the display 12. As a first example, the third area may be set above the display 12, above the center of the display 12 sandwiched between the first area and the second area. As a second example, the third area may be set above the first area and the left edge of the display 12 (i.e., the upper left corner area of ​​the display 12) or below the first area (i.e., the lower left corner area of ​​the display 12). As a third example, the third area may be set above the second area and the right edge of the display 12 (i.e., the upper right corner area of ​​the display 12) or below the second area (i.e., the lower right corner area of ​​the display 12) between the second area and the right edge of the display 12. Regardless of the area in which the third area is set, the same operations as those described above can be performed by displaying a user interface image IMUa that matches the shape of the third area in the third area and forming an opening 154h that allows touch operation within the third area in the goggle device 150. Note that when the third area is set between the first area and the second area, the third area may be shifted to the left or right from the midpoint between the first area and the second area.

[0125] Furthermore, in the above-described embodiment, the function of the user interface image is used as an example of what is displayed on the display 12 to accept touch operations on the touch panel 13, but it may also be an image that does not accept touch operations. For example, the user interface image may be text information or icons that present some information to the user but cannot be touched. As an example, the user interface image may be an image that displays instructions corresponding to button operations, stick operations, or operations to move the controller body on the left controller 3 or the right controller 4.

[0126] The left-eye image IML and the right-eye image IMR may also be displayed outside the display area of ​​the display 12 visible through the left-eye lens 153L and the right-eye lens 153R (typically, outside the first area and / or outside the second area), and may also be partially displayed within the third area where touch operation is possible. The left-eye image IML and the right-eye image IMR may also be displayed in an area smaller than the display area of ​​the display 12 visible through the left-eye lens 153L and the right-eye lens 153R (typically, the first area and / or the second area).

[0127] Next, an example of specific processing executed by the game system 1 in this embodiment will be described with reference to Figures 17 and 18. Figure 17 is a diagram showing an example of a data area set in the DRAM 85 of the main unit 2 in this embodiment. In addition to the data shown in Figure 17, the DRAM 85 also stores data used in other processing, but detailed description thereof will be omitted.

[0128] The program storage area of ​​the DRAM 85 stores various programs Pa executed by the game system 1. In this embodiment, the various programs Pa include a communication program for wireless communication between the left controller 3 and the right controller 4 described above, and an application program for performing information processing (e.g., game processing) based on data acquired from the operation units (left controller 3, right controller 4, touch panel 13, acceleration sensor 89, angular velocity sensor 90) and the illuminance sensor 29. The various programs Pa may be stored in advance in the flash memory 84, or may be acquired from a storage medium detachable from the game system 1 (e.g., a predetermined type of storage medium inserted in the slot 23) and stored in the DRAM 85, or may be acquired from another device via a network such as the Internet and stored in the DRAM 85. The processor 81 executes the various programs Pa stored in the DRAM 85.

[0129] Furthermore, the data storage area of ​​DRAM 85 stores various types of data used in processes such as communication processes and information processes executed in game system 1. In this embodiment, DRAM 85 stores operation data Da, angular velocity data Db, acceleration data Dc, illuminance data Dd, posture data De, operation object data Df, virtual camera data Dg, left-eye virtual space image data Dh, right-eye virtual space image data Di, stereoscopic UI (user interface) image data Dj, non-stereoscopic virtual space image data Dk, non-stereoscopic UI (user interface) image data Dm, and image data Dn, etc.

[0130] The operation data Da is operation data acquired as appropriate from the left controller 3 and / or the right controller 4 or the touch panel 13. As described above, the operation data transmitted from the left controller 3 and / or the right controller 4 includes information about inputs from the input units (specifically, the buttons, analog sticks, and sensors) (specifically, information about the operation and detection results from the sensors). In this embodiment, the operation data is transmitted from the left controller 3 and / or the right controller 4 via wireless communication at a predetermined cycle, and the operation data Da is updated as appropriate using the received operation data. The update cycle of the operation data Da may be every frame, which is the cycle of processing executed by the game system 1 (described later), or may be every cycle at which operation data is transmitted via wireless communication. Furthermore, operation data indicating that the touch panel 13 has been operated is acquired at each cycle of the processing, and is stored in and updated in the operation data Da in response to the acquisition.

[0131] The angular velocity data Db is data indicating the angular velocity occurring in the main body device 2 detected by the angular velocity sensor 90. For example, the angular velocity data Db includes data indicating the angular velocity occurring in the main body device 2 around the x, y, and z axes.

[0132] The acceleration data Dc is data indicating the acceleration occurring in the main device 2 detected by the acceleration sensor 89. For example, the acceleration data Dc includes data indicating the acceleration occurring in the main device 2 in the x, y, and z-axis directions.

[0133] The illuminance data Dd is data indicating the illuminance around the main device 2 detected by the illuminance sensor 29.

[0134] The attitude data De is data that indicates the attitude in real space of the main unit 2. For example, the attitude data De includes data that indicates an attitude that is used as a reference for a gravity vector that indicates gravitational acceleration occurring in the main unit 2, and data that indicates a change in the attitude of the main unit 2.

[0135] The operation object data Df is data that indicates the position, direction, posture, and movement in the virtual space of the object operated by the user.

[0136] The virtual camera data Dg is data that indicates the position, direction, viewing angle, magnification, etc. of a virtual camera (a pair of left and right virtual cameras in stereoscopic display mode, or a single virtual camera in non-stereoscopic display mode) set in the virtual space.

[0137] The left-eye virtual space image data Dh is data for generating the left-eye image IML in the stereoscopic display mode. The right-eye virtual space image data Di is data for generating the right-eye image IMR in the stereoscopic display mode. The stereoscopic UI image data Dj is data indicating the position, shape, size, etc. of the user interface image IMUa in the stereoscopic display mode.

[0138] The non-stereoscopic virtual space image data Dk is data for generating a single image IMS in the non-stereoscopic display mode. The non-stereoscopic UI image data Dm is data that indicates the position, shape, size, etc. of the user interface image IMUb in the non-stereoscopic display mode.

[0139] The image data Dn is data for displaying images (for example, images of virtual objects, user interface images, information images, field images, background images, etc.) on the display screen during the game.

[0140] Next, a detailed example of information processing (game processing) in this embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of game processing executed by the game system 1. In this embodiment, the series of processes shown in Fig. 18 are performed by the processor 81 executing a communication program and a predetermined application program (game program) included in the various programs Pa. In addition, the game processing shown in Fig. 18 can be started at any timing.

[0141] 18 is merely an example, and the order of the steps may be changed, or other processes may be executed in addition to (or instead of) the processes of the steps, as long as the same results are obtained. Furthermore, in this embodiment, the processes of the steps of the flowchart are described as being executed by the processor 81, but the processes of some of the steps in the flowchart may be executed by a processor other than the processor 81 or a dedicated circuit. Furthermore, some of the processes executed by the main unit 2 may be executed by another information processing device capable of communicating with the main unit 2 (for example, a server capable of communicating with the main unit 2 via a network). That is, the processes shown in FIG. 18 may be executed by cooperation between a plurality of information processing devices, including the main unit 2.

[0142] In FIG. 18 , processor 81 performs initial settings for game processing (step S200) and proceeds to the next step. For example, in the initial settings, processor 81 initializes parameters for performing the processing described below. As an example, processor 81 uses the acceleration data stored in acceleration data Dc to calculate the direction of a gravity vector of gravitational acceleration acting on main unit 2, sets an initial attitude of main unit 2 based on the direction of the gravity vector, and updates attitude data De. Furthermore, processor 81 sets an initial attitude of the virtual camera in virtual space and updates virtual camera data Dg so that the relationship between the direction of the gravity vector of gravitational acceleration acting on main unit 2 and the x, y, and z-axis directions of main unit 2 is the same as that of the x, y, and z-axes of main unit 2. Here, being in the same direction as the x, y, and z-axes of main unit 2 means that the positive z-axis direction (depth direction on the screen) based on the direction of gravitational acceleration in real space is the same as the line of sight of the virtual camera based on the direction of gravity in virtual space, and the positive x-axis direction (leftward direction on the screen) based on the direction of gravitational acceleration in real space is the same as the leftward direction of the virtual camera based on the direction of gravity in virtual space.

[0143] Next, the processor 81 acquires various data and updates the operation data Da, angular velocity data Db, acceleration data Dc, and illuminance data Dd (step S201), and proceeds to the next step. For example, the processor 81 acquires operation data from the left controller 3 and / or the right controller 4 and updates the operation data Da. The processor 81 also acquires touch operation data from the touch panel 13 and updates the operation data Da. The processor 81 also acquires inertial data (acceleration data and angular velocity data) from the inertial sensors (acceleration sensor 89 and angular velocity sensor 90) provided in the main unit 2 and updates the acceleration data Dc and angular velocity data Db. The processor 81 also acquires illuminance data from the illuminance sensor 29 and updates the illuminance data Dd.

[0144] Next, the processor 81 calculates the orientation of the main unit 2 (step S202) and proceeds to the next step. For example, the processor 81 uses the acceleration data and angular velocity data stored in the angular velocity data Db and acceleration data Dc to calculate the direction of the gravity vector of the gravitational acceleration acting on the main unit 2, and updates the orientation data De. The processor 81 also calculates the direction and amount of rotation of the main unit 2 from the initial orientation, and updates the orientation data De. For example, the processor 81 calculates the direction and amount of rotation about the x, y, and z-axis directions of the main unit 2 in the initial orientation, and updates the orientation data De. Note that the direction of rotation can be represented by the positive or negative value of the amount of rotation, so the orientation data De may store only data indicating the amount of rotation. For example, the processor 81 may calculate a new amount of rotation by adding the amount of rotation based on the angular velocity data acquired in this step S202 to the amount of rotation calculated in the previous processing step S202.

[0145] Next, the processor 81 performs a process of determining whether or not the main device 2 is attached to the goggle device 150 (step S203), and proceeds to the next step. For example, if the illuminance indicated by the illuminance data Dd indicates a low illuminance that is less than a threshold value for detecting the illuminance when the main device 2 is attached to the goggle device 150, the processor 81 determines in step S203 that the main device 2 is attached to the goggle device 150. The detection means executes a process of detecting whether or not the display device is attached to the goggle device, or whether or not it is in the process of being attached, and corresponds to the processor 81 that performs the process of step S203, for example.

[0146] Next, the processor 81 determines whether or not the display mode is stereoscopic (step S204). For example, if the processor 81 determines in the wearing determination process of step S203 that the main device 2 is worn on the goggle device 150, the processor 81 makes a positive determination in step S204 and performs processing in the stereoscopic display mode. On the other hand, if the processor 81 determines in the wearing determination process of step S203 that the main device 2 is not worn on the goggle device 150, the processor 81 makes a negative determination in step S204 and performs processing in the non-stereoscopic display mode. If the processor 81 determines to perform processing in the stereoscopic display mode, the processor 81 proceeds to step S205. On the other hand, if the processor 81 determines to perform processing in the non-stereoscopic display mode, the processor 81 proceeds to step S213. The mode setting means executes processing to set the display device to a first display mode or a second display mode different from the first display mode, and corresponds to the processor 81 performing the processing of step S204, for example.

[0147] In step S205, the processor 81 performs object movement processing and proceeds to the next step. For example, when an operation to move a control object in virtual space is being performed, the processor 81 references the operation data Da and sets the movement of the control object according to the operation. Then, based on the set movement of the control object, the processor 81 sets the position, direction, posture, movement, etc. of the control object in virtual space, and updates the control object data Df.

[0148] The object movement processing in step S205 may involve the following types of object movement control. As a first example, a predetermined control object is set as the operation target and the control object is moved. In this case, the predetermined control object is moved, moved, or deformed based on input to the input units of the left controller 3 and / or the right controller 4. As a second example, a control object is selected as the operation target based on an operation, and then the control object is moved. In this case, a control object located in a predetermined display position (for example, a control object overlapping a sign displayed in the center of the display screen) is set as the operation target, and the control object selected as the operation target is moved, moved, or deformed based on input to the input units of the left controller 3 and / or the right controller 4. As a third example, the control object set as the operation target is moved based on vibrations applied to the goggle device 150 to which the main unit 2 is attached. For example, the gravitational acceleration component is removed from the acceleration in the x, y, and z axes directions of the main unit 2 indicated by the acceleration data Dc, and if the acceleration after removal indicates that vibration of a predetermined magnitude or greater is being applied to the main unit 2, the operation object being operated is caused to move in response to the vibration. Note that any method may be used to extract the gravitational acceleration, and for example, the acceleration component occurring on average in the main unit 2 may be calculated and extracted as the gravitational acceleration.

[0149] Next, processor 81 performs a process of operating the pair of left and right virtual cameras (step S206) and proceeds to the next step. For example, processor 81 sets the orientations of the pair of left and right virtual cameras in the virtual space by rotating them from the initial orientation by the rotation amount calculated in step S202, and updates the virtual camera data Dg. For example, while fixing the positional relationship between the pair of left and right virtual cameras from the initial orientation, processor 81 rotates the virtual cameras around the left and right direction of the virtual cameras as an axis by the same amount as the rotation about the left and right axis direction (x-axis direction) of the main unit 2 calculated in step S202, rotates the virtual cameras around the up and down direction of the virtual cameras as an axis by the same amount as the rotation about the up and down axis direction (y-axis direction) of the main unit 2 calculated in step S202, and rotates the virtual cameras around the line of sight of the virtual cameras as an axis by the same amount as the rotation about the screen depth axis direction (z-axis direction) of the main unit 2 calculated in step S202, thereby setting the orientations of the pair of left and right virtual cameras in the virtual space.

[0150] Next, processor 81 performs processing to generate a virtual space image for the left eye (step S207), and proceeds to the next step. For example, processor 81 places a control object in the virtual space based on control object data Df. Then, processor 81 generates, as a virtual space image for the left eye, a virtual space image viewed from the left virtual camera of the pair of left and right virtual cameras set by virtual camera data Dg, and updates virtual space image data Dh for the left eye.

[0151] Next, the processor 81 performs processing to generate a virtual space image for the right eye (step S208), and proceeds to the next step. For example, the processor 81 generates a virtual space image seen from the right virtual camera of the pair of left and right virtual cameras set by the virtual camera data Dg as a virtual space image for the right eye, and updates the virtual space image data Di for the right eye.

[0152] Next, the processor 81 performs a process of generating a stereoscopic user interface image (step S209), and proceeds to the next step. For example, the processor 81 generates a stereoscopic user interface image that fits the shape of the third area (see FIG. 9) of the display 12, and updates the stereoscopic UI image data Dj.

[0153] Next, the processor 81 performs a display control process to display the virtual space image for the left eye in the first area of ​​the display 12 (step S210), and proceeds to the next step. For example, the processor 81 displays the virtual space image for the left eye set in the virtual space image data Dh as the left eye image IML over the entire first area of ​​the display 12 (see FIG. 9).

[0154] Next, the processor 81 performs a display control process to display the virtual space image for the right eye in the second area of ​​the display 12 (step S211), and proceeds to the next step. For example, the processor 81 displays the virtual space image for the right eye set in the virtual space image data Di as the image for the right eye IMR over the entire second area of ​​the display 12 (see FIG. 9).

[0155] Next, processor 81 performs display control processing to display the stereoscopic UI image in the third area of ​​display 12 (step S212), and proceeds to step S219. For example, processor 81 displays the user interface image set in the stereoscopic UI image data Dj as user interface image IMUa (e.g., two user interface images IMUa1 and IMUa2) at a predetermined position in the third area (see FIG. 9) of display 12. Note that the second display control means executes processing to display, on the display screen in the second display mode, a second image including a content image made up of a left-eye image and a right-eye image having parallax therebetween and a second user interface image corresponding to the first user interface image, and corresponds to processor 81 performing the processing of steps S205 to S212, for example.

[0156] On the other hand, if it is determined in step S204 that the display mode is non-stereoscopic, then in step S213, the processor 81 performs object movement processing and proceeds to the next step. For example, when an operation to move a control object in virtual space is being performed, the processor 81 references the operation data Da and sets the movement of the control object in accordance with the operation. Then, based on the set movement of the control object, the processor 81 sets the position, direction, posture, movement, etc. of the control object in virtual space, and updates the control object data Df. Note that the object movement processing in step S213 is similar to the object movement processing in step S205 described above, and therefore a detailed description thereof will be omitted.

[0157] Next, processor 81 performs processing to operate a single virtual camera (step S214) and proceeds to the next step. For example, processor 81 sets the orientation of the single virtual camera in the virtual space by rotating it from the initial orientation by the rotation amount calculated in step S202, and updates the virtual camera data Dg. For example, processor 81 sets the orientation of the single virtual camera in the virtual space by rotating the single virtual camera from the initial orientation about the left-right direction of the virtual camera by the same amount as the rotation about the left-right axis direction (x-axis direction) of the main body device 2 calculated in step S202, rotating it about the up-down direction of the virtual camera by the same amount as the rotation about the up-down axis direction (y-axis direction) of the main body device 2 calculated in step S202, and rotating it about the line of sight of the virtual camera by the same amount as the rotation about the screen depth axis direction (z-axis direction) of the main body device 2 calculated in step S202.

[0158] Next, processor 81 performs processing to generate a virtual space image (step S215), and proceeds to the next step. For example, processor 81 generates a virtual space image viewed from the virtual camera set by the virtual camera data Dg, and updates non-stereoscopic virtual space image data Dk.

[0159] Next, the processor 81 performs processing to generate a non-stereoscopic user interface image (step S216), and proceeds to the next step. For example, the processor 81 generates a non-stereoscopic user interface image to be superimposed on the single image IMS and displayed on the display 12 (see FIG. 16), and updates the non-stereoscopic UI image data Dm.

[0160] Next, processor 81 performs display control processing to display the virtual space image in the entire area of ​​display 12 (step S217), and proceeds to the next step. For example, processor 81 displays the virtual space image set in the non-stereoscopic virtual space image data Dk as a single image IMS in the entire area of ​​display 12 (see FIG. 16).

[0161] Next, processor 81 performs display control processing to superimpose non-stereoscopic UI images on the single image IMS (step S218), and proceeds to step S219. For example, processor 81 superimposes user interface images IMUb (e.g., two user interface images IMUb1 and IMUb2) set in non-stereoscopic UI image data Dm on the single image IMS in the upper left and upper right corner regions (see FIG. 16) of display 12. Note that first display control means executes processing to display a first image including a content image, which is a non-stereoscopic image, and a first user interface image on the display screen in the first display mode, and corresponds to processor 81 performing the processing of steps S213 to S218, for example.

[0162] In step S219, processor 81 performs user interface operation processing and proceeds to the next step. For example, when a touch operation is performed on touch panel 13, processor 81 refers to operation data Da and sets a user operation instruction corresponding to the touch operation in accordance with a user interface image displayed on display 12 that overlaps the position of the touch operation. Then, processor 81 performs processing in accordance with the set user operation instruction.

[0163] Next, processor 81 determines whether or not to end the game (step S220). Conditions for ending the game in step S220 include, for example, the game result being determined or the user performing an operation to end the game. If processor 81 does not end the game, it returns to step S201 and repeats the process, and if it ends the game, it ends the process according to this flowchart. Thereafter, the series of processes from step S201 to step S220 is repeatedly executed until it is determined in step S220 that the game is to end.

[0164] As described above, in this embodiment, when switching between the normal display mode (non-stereoscopic display mode) and the stereoscopic display mode, the user interface image is displayed in a different position depending on the set display mode, thereby improving convenience in presenting the user interface image. Furthermore, the display mode is automatically switched based on the state in which the main unit 2 is attached to the goggle device 150, thereby enabling seamless switching of the display mode. Furthermore, in this embodiment, in the stereoscopic display mode, the user interface image is displayed in a third region, which is different from the first and second regions of the display 12 and which displays the right-eye image and the left-eye image, respectively. Therefore, the user interface image can be displayed without interfering with the stereoscopic image display.

[0165] In the above-described embodiment, the image display system is configured by attaching the main device 2 having an information processing function to the goggle device 150. However, the image display system may be configured in other ways. As a first example, an image display system that displays stereoscopic images may be configured by providing the goggle device 150 with a control unit that performs the information processing (game processing) and generates images, and attaching a display device having an image display function to the goggle device 150. In this case, the control unit provided in the goggle device 150 outputs image data for displaying the stereoscopic image and the user interface image on the display device to the display device, thereby displaying the stereoscopic image and the user interface image on the display device. In the above-described first example, a mechanism for detecting the wearing status of the display device, a mechanism for detecting the attitude of the image display system, a mechanism for accepting user operations, and the like may be provided in either the display device or the goggle device 150. As a second example, a control device that is connected wirelessly or by wire to the display device, separate from the goggle device 150 to which the display device is attached, may be provided, and the image display system may be configured by the display device, the goggle device 150, and the control device. In this case, operation data, acceleration data, angular velocity data, illuminance data, etc. are output from the display device to the control device, and a content image and a user interface image in a display mode based on the illuminance data are output from the control device to the display device. Also in the second example, the mechanism for detecting the wearing status of the display device, the mechanism for detecting the attitude of the image display system, and the mechanism for accepting user operations may be provided in either the display device or the goggle device 150. The control device controls the display range of the content image to be displayed according to the operation data, acceleration data, and angular velocity data acquired from the display device and / or the goggle device 150, and outputs the content image to the display device. In this embodiment, the main unit 2 is used as an example of a display device.

[0166] It should be noted that in the above-described embodiment, the method for detecting the attitude of the main unit 2 is merely an example, and the attitude of the main unit 2 may be detected using other methods or other data. As an example, the main unit 2 and / or the goggle device 150 to which the main unit 2 is attached may be imaged from outside, and the attitude of the main unit 2 and / or the goggle device 150 may be detected using the image thus captured. Furthermore, the controller for controlling the operation of the operation object may be not only the left controller 3 or the right controller 4, but also another controller.

[0167] Furthermore, the game system 1 and / or the main unit 2 may be any device, such as a portable game device, any portable electronic device (PDA (Personal Digital Assistant), mobile phone, personal computer, camera, tablet, etc.), etc.

[0168] In the above-described embodiment, an example was used in which a stereoscopic image was displayed by displaying a left-eye image and a right-eye image, which have parallax relative to each other, on the left and right screens of the display 12, respectively. However, the left-eye image and the right-eye image may be displayed on separate screens. For example, if the display 12 provided on the main unit 2 is configured with multiple display screens, the left-eye image is displayed on one of the multiple display screens, and the right-eye image is displayed on another of the multiple display screens. In this case, the user can view a stereoscopic image through the goggle device 150 by viewing with their left eye the left-eye image displayed on one of the multiple display screens via the left-eye lens 153L and viewing with their right eye the right-eye image displayed on another of the multiple display screens via the right-eye lens 153R.

[0169] Furthermore, the stereoscopic images and non-stereoscopic images displayed on the main unit 2 may be displayed as game images by the processor 81 executing information processing (game processing) in response to user operations, or as moving images or still images by the processor 81 performing moving image playback or still image playback in response to user operations. In other words, the stereoscopic images and non-stereoscopic images displayed on the main unit 2 are generated by the processor 81 of the main unit 2 performing information processing (e.g., game processing, moving image playback processing, still image playback processing). However, at least a portion of the processing for generating the stereoscopic images and non-stereoscopic images may be performed by another device. For example, if the main unit 2 is configured to be able to communicate with yet another device (e.g., a server, another image display device, another game device, another mobile terminal, another information processing device), the above processing may be performed in cooperation with the other device. In this way, performing at least a portion of the above processing in another device enables processing similar to the above-described processing. Furthermore, the above-described information processing may be performed by one processor or by cooperation between multiple processors included in an information processing system configured with at least one information processing device. Furthermore, in the above embodiment, information processing can be performed by the processor 81 of the main unit 2 executing a predetermined program, but some or all of the above processing may also be performed by a dedicated circuit provided in the main unit 2.

[0170] According to the above-described modified example, the present invention can also be realized in a so-called cloud computing system configuration, or in a distributed wide area network or local network system configuration. For example, in a distributed local network system configuration, the above processing can be performed cooperatively between a stationary information processing device (stationary game device) and a portable information processing device (portable game device). Note that in these system configurations, there is no particular limitation on which device performs the above processing, and it goes without saying that the present invention can be realized regardless of the processing division.

[0171] Furthermore, the processing order, setting values, conditions used for judgment, etc. used in the information processing described above are merely examples, and it goes without saying that this embodiment can be realized even with other orders, values, and conditions.

[0172] The program may be supplied to the game system 1 not only through an external storage medium such as an external memory, but also through a wired or wireless communication line. The program may be pre-recorded in a nonvolatile storage device within the device. The information storage medium for storing the program may be a nonvolatile memory, a CD-ROM, a DVD, or similar optical disk-shaped storage media, a flexible disk, a hard disk, a magneto-optical disk, or a magnetic tape. The information storage medium for storing the program may also be a volatile memory for storing the program. Such a storage medium may be a recording medium readable by a computer or the like. For example, the various functions described above can be provided by having a computer or the like read and execute the program from such a recording medium.

[0173] Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit its scope. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. It is understood that the scope of the present invention should be interpreted solely by the claims. Furthermore, it is understood that those skilled in the art can implement equivalent scopes based on the description of the present invention and common technical knowledge from the description of specific embodiments of the present invention. Furthermore, it should be understood that the terms used in this specification are used in the same sense as commonly used in the art unless otherwise specified. Therefore, unless otherwise defined, all technical and technical terms used in this specification have the same meaning as commonly understood by those skilled in the art to which this invention belongs. In the event of a conflict, the present specification (including definitions) shall prevail. [Industrial Applicability]

[0174] As described above, the present invention can be used as an image display system, an image display program, an image display method, a display device, and the like, which can improve the convenience of presenting user interface images. [Explanation of symbols]

[0175] 1. Game System 2...Main unit 3...Left controller 4...Right controller 11. Housing 12...Display 29...Illuminance sensor 81...Processor 83...Controller communication section 85...DRAM 150...Goggle device 151...Main body 152...Lens frame member 153...Lens 154...Plate-shaped member

Claims

1. An image display system including a display device having a display screen for displaying an image and a goggle device on which the display device can be worn, a mode setting means for setting the display device to a first display mode or a second display mode different from the first display mode; a first display control means for causing the display screen to display a first image including a content image, which is a non-stereoscopic image, and a first user interface image in the first display mode; a second display control means for causing the display screen to display, in the second display mode, a second image including a content image including a left-eye image and a right-eye image having parallax therebetween and a second user interface image corresponding to the first user interface image; The second display control means, in the second display mode, displays the second user interface image at a position on the display screen different from a position at which the first user interface image is displayed in the first display mode.

2. Further comprising a detection means for detecting whether the display device is in a mounted state on the goggle device or in a partially mounted state, 2. The image display system according to claim 1, wherein the mode setting means switches the display device to the second display mode based on a detection result of the detection means when the display device is set to the first display mode.

3. The display device includes an illuminance sensor, the goggle device includes a light blocking unit that blocks light from reaching the illuminance sensor of the display device when the display device is in a worn state or in a half-worn state on the goggle device, 3. The image display system according to claim 2, wherein the detection means detects whether the display device is attached to the goggle device or is in the process of being attached based on a detection result by the illuminance sensor.

4. the display device includes a touch panel provided on the display screen, the first user interface image displayed on the display screen can provide an operation instruction in response to a touch operation on the touch panel; 4. The image display system according to claim 1, wherein the second user interface image displayed on the display screen is capable of providing operational instructions in response to a touch operation on the touch panel when the display device is attached to the goggle device.

5. 5. The image display system according to claim 1, wherein the second display control means displays on the display screen an image corresponding to a content image displayed as a non-stereoscopic image by the first display control means as a content image consisting of an image for the left eye and an image for the right eye.

6. 6. The image display system according to claim 1, wherein the second display control means, in the second display mode, converts a content image that was displayed in the first display mode immediately before being set to the second display mode into a stereoscopic image consisting of an image for the left eye and an image for the right eye, and displays the stereoscopic image on the display screen.

7. 7. The image display system of claim 1, wherein, in the second display mode, the second display control means displays the image for the left eye in a first area of ​​the display screen, displays the image for the right eye in a second area of ​​the display screen different from the first area, and displays the second user interface image in a third area of ​​the display screen different from the first area and the second area.

8. 8. The image display system according to claim 1, wherein the second display control means displays, as the second user interface image, a user interface image that is substantially the same in function as the first user interface image but different in shape.

9. 9. The image display system of claim 8, wherein the second display control means, in the second display mode, adjusts a shape of the second user interface image to fit a shape of a third area of ​​the display screen that is different from a first area of ​​the display screen that displays an image for the left eye and a second area of ​​the display screen that displays an image for the right eye, and displays the second user interface image in the third area.

10. 10. The image display system according to claim 7, wherein the second display control means sets the third area above or below the display screen between the first area and the second area on the display screen.

11. The image display system according to claim 10 , wherein the second display control means sets the third area below the display screen between the first area and the second area on the display screen.

12. The image display system according to any one of claims 7 and 9 to 11, wherein the goggle device has an opening that exposes at least the third area, which is a part of the display screen, to the outside when the display device is attached to the goggle device.

13. The image display system according to claim 12 , wherein the opening is formed at a position corresponding to a nose of a user when the user wears the goggle device.

14. 14. The image display system according to claim 1, wherein the first display control means, in the first display mode, displays the first user interface image so as to be superimposed on a content image displayed on the display screen.

15. 15. The image display system according to claim 1, wherein the second display control means, in the second display mode, causes the second user interface image to be displayed on the display screen as a non-stereoscopic image.

16. The display device further includes a display device side connection section that can be electrically connected to another device, the goggle device includes a goggle device side connection part electrically connectable to the display device side connection part, 3. The image display system according to claim 2, wherein the detection means detects whether the display device is attached to the goggle device or is in the process of being attached in response to the connection between the display device side connection section and the goggle device side connection section.

17. An image display program executed by a computer included in a display device having a display screen for displaying an image, The computer, a mode setting means for setting the display device to the first display mode or to a second display mode different from the first display mode; a first display control means for causing the display screen to display a first image including a content image, which is a non-stereoscopic image, and a first user interface image in the first display mode; functioning as a second display control means for displaying, on the display screen in the second display mode, a second image including a content image made up of a left-eye image and a right-eye image having parallax therebetween and a second user interface image corresponding to the first user interface image; The second display control means displays the second user interface image in the second display mode at a position on the display screen different from a position at which the first user interface image is displayed in the first display mode.

18. 1. An image display method for displaying an image on a display device having a display screen for displaying an image, comprising: a mode setting step of setting the display device to the first display mode or to a second display mode different from the first display mode; a first display control step of displaying a first image including a content image, which is a non-stereoscopic image, and a first user interface image on the display screen in the first display mode; a second display control step of displaying, on the display screen in the second display mode, a second image including a content image including a left-eye image and a right-eye image having parallax therebetween and a second user interface image corresponding to the first user interface image; The image display method, wherein in the second display control step, the second user interface image is displayed in a position on the display screen that is different from a position on the display screen where the first user interface image is displayed in the first display mode.

19. A display device having a display screen for displaying an image, a mode setting means for setting the display device to the first display mode or to a second display mode different from the first display mode; a first display control means for causing the display screen to display a first image including a content image, which is a non-stereoscopic image, and a first user interface image in the first display mode; a second display control means for causing the display screen to display, in the second display mode, a second image including a content image including a left-eye image and a right-eye image having parallax therebetween and a second user interface image corresponding to the first user interface image; The second display control means displays the second user interface image in the second display mode at a position on the display screen different from a position at which the first user interface image is displayed in the first display mode.

Images