Image display system, image display program, image display method, and display device

Abstract

To provide an image display system, an image display program, an image display method and a display device that can improve convenience associated with presentation of a user interface image.SOLUTION: An image display system comprises: a display device that has a display screen; and a goggle device that can wear the display device. The image display system displays a first image including a content image being a non-stereoscopic image and a first user interface image on a display screen in a first display mode, and displays a second image including a content image which is composed of a left-eye image and a right-eye image having a parallax to each other, and a second user interface image corresponding to the first user interface image, on the display screen in a second display mode. The image display system displays the second user interface image at a position being different from a position on the display screen where the first user interface image is displayed in the first display mode, in the second display mode.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] The present invention relates to an image display system, an image display program, an image display method, and a display device capable of displaying a stereoscopic image.

Background Art

[0002] Conventionally, there has been a stereoscopic image display device that displays a stereoscopic image by allowing a user to visually recognize two images with different parallaxes with the left and right eyes of the user (see, for example, Patent Document 1). For example, in Patent Document 1, a smartphone is housed in a goggle device that can be worn by a user, and the user can look into the stereoscopic image displayed on the display screen of the smartphone through the goggle device, thereby enabling the user to view a stereoscopic image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the stereoscopic image display device disclosed in Patent Document 1 does not consider at all presenting a user interface image that accepts a touch operation of the user when displaying a stereoscopic image on the display screen. Therefore, there is room for improving the convenience of the method of presenting the user interface image.

[0005] Therefore, an object of the present invention is to provide an image display system, an image display program, an image display method, and a display device capable of improving the convenience regarding the presentation of a user interface image.

Means for Solving the Problems

[0006] To achieve the above objectives, the present invention may adopt, for example, the following configuration. It should be understood that when interpreting the claims, the scope should be interpreted solely by the claims, and in the event of any conflict between the claims and the description in this section, the claims shall prevail.

[0007] One example of the configuration of the image display system of the present invention comprises a display device having a display screen for displaying images and a goggle device on which the display device can be attached. The image display system comprises a mode setting means, a first display control means, and a second display control means. The mode setting means sets the display device to a first display mode or a second display mode different from the first display mode. In the first display mode, the first display control means causes the display screen to display a first image including a content image which is a non-stereoscopic image and a first user interface image. In the second display mode, the second display control means causes the display screen to display a second image including a content image consisting of a left-eye image and a right-eye image which have parallax with respect to each other, and a second user interface image which corresponds to the first user interface image. In the second display mode, the second display control means causes the second user interface image to be displayed at a position on the display screen different from the position on which the first user interface image is displayed in the first display mode.

[0008] As described above, when switching display modes, the user interface image is displayed in a different position, which improves the convenience of presenting the user interface image.

[0009] Furthermore, the image display system further includes detection means. The detection means detects whether the display device is attached to the goggle device or whether it is in the process of being attached. In this case, the mode setting means may switch the display device to a second display mode based on the detection result of the detection means when the display device is set to a first display mode.

[0010] As described above, the display mode is switched based on the state of attachment or the in-progress state of attachment of the display device to the goggles, enabling seamless switching of display modes.

[0011] Furthermore, the display device may be equipped with an illuminance sensor. In this case, the goggle device may be equipped with a light-shielding section. The light-shielding section blocks the light from the display device to the illuminance sensor when the display device is attached to the goggle device or in the process of being attached. The detection means may detect whether the display device is attached to the goggle device or in the process of being attached, based on the detection result from the illuminance sensor.

[0012] According to the above, it is possible to easily detect whether the display device is installed or in the process of being installed based on the light-shielding state of the display device.

[0013] Furthermore, the display device may include a touch panel on the display screen. In this case, the first user interface image displayed on the display screen may allow operation instructions in response to touch operations on the touch panel. The second user interface image displayed on the display screen may allow operation instructions in response to touch operations on the touch panel when the display device is mounted on the goggles device.

[0014] According to the above, it becomes possible to improve the convenience of presenting user interface images that accept touch operations.

[0015] Furthermore, the second display control means may display on the display screen an image corresponding to the content image displayed as a non-stereoscopic image by the first display control means, as a content image consisting of a left-eye image and a right-eye image.

[0016] According to the above, it is possible to seamlessly switch between stereoscopic and non-stereoscopic images within the same content image.

[0017] Furthermore, the second display control means may, in the second display mode, convert the content image that was displayed in the first display mode immediately prior to setting the second display mode into a stereoscopic image consisting of a left-eye image and a right-eye image, and display it on the display screen.

[0018] According to the above, it is possible to seamlessly switch between stereoscopic and non-stereoscopic images within the same content image.

[0019] Furthermore, the second display control means may, in the second display mode, display an image for the left eye in a first area of ​​the display screen, display an image for the right eye in a second area of ​​the display screen different from the first area, and display a second user interface image in a third area of ​​the display screen different from the first and second areas.

[0020] According to the above, it is possible to prevent the first and second regions for displaying stereoscopic images from being defaced by touch operations, and to prevent fingers used to touch the second user interface image from entering the field of view while viewing the stereoscopic image.

[0021] Furthermore, the second display control means may display a user interface image as the second user interface image that has substantially the same function as the first user interface image but a different shape.

[0022] According to the above, it is possible to display a user interface image with a shape suitable for operation and stereoscopic image display.

[0023] Furthermore, in the second display mode, the second display control means may adjust the shape of the second user interface image to match the shape of a third area of ​​a display screen that is different from the shape of the first area of ​​the display screen that displays the image for the left eye and the second area of ​​the display screen that displays the image for the right eye, and then display the second user interface image in the third area.

[0024] According to the above, a user interface image with an appropriate shape can be displayed.

[0025] Further, the second display control means may set a third area above or below the display screen sandwiched by the first area and the second area on the display screen.

[0026] According to the above, a user interface image that does not interfere with stereoscopic image display can be displayed.

[0027] Further, the second display control means may set a third area below the display screen sandwiched by the first area and the second area on the display screen.

[0028] According to the above, a user interface image that does not interfere with stereoscopic image display and is easy to operate can be displayed.

[0029] Further, the goggle device may have an opening that exposes at least a part of the third area, which is a part of the display screen, to the outside when the display device is attached to the goggle device.

[0030] According to the above, a touch operation for touching the user interface image is possible in a state where the goggle device is worn.

[0031] Further, the opening may be formed at a position corresponding to the nose of the user when the user wears the goggle device.

[0032] According to the above, the opening can be formed without impairing the light shielding property.

[0033] Further, the first display control means may display a first user interface image by superimposing it on the content image displayed on the display screen in the first display mode.

[0034] According to the above, it is possible to display relatively large content images without being affected by the display of user interface images.

[0035] Furthermore, in the second display mode, the second display control means may display the second user interface image on the display screen as a non-stereoscopic image.

[0036] According to the above, touching the user interface image becomes easier.

[0037] Furthermore, the display device may further include a display device-side connection part that can be electrically connected to other devices. In this case, the goggle device may also include a goggle device-side connection part that can be electrically connected to the display device-side connection part. The detection means may detect whether the display device is attached to the goggle device, or whether it is in the process of being attached, depending on whether the display device-side connection part and the goggle device-side connection part are connected.

[0038] According to the above, it is possible to reliably detect whether the display device has been attached to the goggle device, or whether it is in the process of being attached.

[0039] Furthermore, the present invention may be implemented in the form of an image display program, an image display method, and a display device. [Effects of the Invention]

[0040] According to the present invention, when switching display modes, the user interface image is displayed in a different position, thereby improving the convenience of presenting the user interface image. [Brief explanation of the drawing]

[0041] [Figure 1] This diagram shows the main unit 2 with the left controller 3 and right controller 4 attached. [Figure 2]This diagram shows an example of the state in which the left controller 3 and right controller 4 have been removed from the main unit 2. [Figure 3] A six-view drawing showing an example of the main unit 2. [Figure 4] A six-view drawing showing an example of the left controller 3. [Figure 5] A six-view drawing showing an example of the right controller 4. [Figure 6] Block diagram showing an example of the internal configuration of the main unit 2. [Figure 7] Block diagram showing an example of the internal configuration of the main unit 2, left controller 3, and right controller 4. [Figure 8] Perspective view showing an example of the appearance of the goggle device 150. [Figure 9] Front view showing an example of the main unit 2 being attached to the goggle device 150. [Figure 10] Front view showing an example of the state of the main unit 2 attached to the goggle device 150. [Figure 11] This figure shows an example of the shape of the front contact portion 151b that comes into contact with a part of the front surface of the main unit 2. [Figure 12] A diagram showing an example of the internal structure of the goggle device 150. [Figure 13] A side view showing an example of the state of the main unit 2 attached to the goggle device 150. [Figure 14] This diagram shows an example of how a user views an image displayed on an image display system. [Figure 15] A diagram showing an example of a user grasping an image display system. [Figure 16] This figure shows examples of images displayed on the main unit 2 in stereoscopic display mode and non-stereoscopic display mode, respectively. [Figure 17] This diagram shows an example of a data area set in the DRAM 85 of the main unit 2. [Figure 18] A flowchart showing an example of game processing performed by game system 1. [Modes for carrying out the invention]

[0042] The following describes an example of an image display system according to this embodiment. The example of the image display system in this embodiment consists of a game system 1 (a main unit 2 included in the game system 1 as the minimum configuration) and a goggle device 150. The example of the game system 1 includes a main unit (information processing device; in this embodiment, it functions as the main unit of the game device) 2, a left controller 3, and a right controller 4. The left controller 3 and the right controller 4 are detachable from the main unit 2. In other words, the game system 1 can be used as an integrated device by attaching the left controller 3 and the right controller 4 to the main unit 2. Alternatively, the game system 1 can also be used with the main unit 2 and the left controller 3 and right controller 4 as separate units (see Figure 2). The following describes the hardware configuration of the game system 1 of this embodiment, followed by a description of the control of the game system 1 of this embodiment.

[0043] Figure 1 shows an example of the main unit 2 with the left controller 3 and right controller 4 attached. As shown in Figure 1, the left controller 3 and right controller 4 are attached to the main unit 2 and integrated together. The main unit 2 is a device that performs various processes (e.g., game processing) in the game system 1. The main unit 2 is equipped with a display 12. The left controller 3 and right controller 4 are devices equipped with operation parts for user input.

[0044] Figure 2 shows an example of the left controller 3 and right controller 4 being removed from the main unit 2. As shown in Figures 1 and 2, the left controller 3 and right controller 4 are detachable from the main unit 2. In the following, the left controller 3 and right controller 4 will be collectively referred to as "controllers".

[0045] Figure 3 is a six-view drawing showing an example of the main unit 2. As shown in Figure 3, the main unit 2 includes a roughly plate-shaped housing 11. In this embodiment, the main surface of the housing 11 (in other words, the front surface, i.e., the surface on which the display 12 is provided) is roughly rectangular in shape.

[0046] The shape and size of the housing 11 are arbitrary. For example, the housing 11 may be portable. The main unit 2 alone, or the integrated unit in which the left controller 3 and right controller 4 are attached to the main unit 2, may be a portable device. The main unit 2 or the integrated unit may be a handheld device. The main unit 2 or the integrated unit may also be a portable device.

[0047] As shown in Figure 3, the main unit 2 includes a display 12 provided on the main surface of the housing 11. The display 12 displays images generated by the main unit 2. In this embodiment, the display 12 is a liquid crystal display (LCD). However, the display 12 may be any type of display device.

[0048] Furthermore, the main unit 2 is equipped with a touch panel 13 on the screen of the display 12. In this embodiment, the touch panel 13 is of a type that allows multi-touch input (for example, a capacitive touch panel). However, the touch panel 13 may be of any type, for example, a type that allows single-touch input (for example, a resistive touch panel).

[0049] The main unit 2 is equipped with a speaker (i.e., speaker 88 shown in Figure 6) inside the housing 11. As shown in Figure 3, speaker holes 11a and 11b are formed on the main surface of the housing 11. The sound output from speaker 88 is emitted from these speaker holes 11a and 11b, respectively.

[0050] Furthermore, the main unit 2 is equipped with a left terminal 17, which is a terminal for the main unit 2 to communicate with the left controller 3 via wired connection, and a right terminal 21, which is for the main unit 2 to communicate with the right controller 4 via wired connection.

[0051] As shown in Figure 3, the main unit 2 is equipped with a slot 23. The slot 23 is located on the upper side of the housing 11. The slot 23 has a shape that allows a predetermined type of storage medium to be inserted. The predetermined type of storage medium is, for example, a storage medium (e.g., a dedicated memory card) specifically for the game system 1 and similar information processing devices. The predetermined type of storage medium is used, for example, to store data used by the main unit 2 (e.g., application save data, etc.) and / or programs executed by the main unit 2 (e.g., application programs, etc.). The main unit 2 is also equipped with a power button 28.

[0052] The main unit 2 is equipped with a lower terminal 27. The lower terminal 27 is a terminal for the main unit 2 to communicate with the cradle. In this embodiment, the lower terminal 27 is a USB connector (more specifically, a female connector). When the integrated device or the main unit 2 alone is placed on the cradle, the game system 1 can display the images generated and output by the main unit 2 on a stationary monitor. In this embodiment, the cradle also has the function of charging the integrated device or the main unit 2 alone that is placed on it. The cradle also has the function of a hub device (specifically, a USB hub).

[0053] The main unit 2 is equipped with an illuminance sensor 29. In this embodiment, the illuminance sensor 29 is located on the lower part of the main surface of the housing 11 and detects the illuminance (brightness) of light incident from the main surface side of the housing 11. Depending on the illuminance of the light detected by the illuminance sensor 29, the display 12 can be set to an appropriate brightness to display an image. In this embodiment, based on the detected illuminance, it is determined whether or not the main unit 2 is attached to the goggle device 150, which will be described later.

[0054] Figure 4 is a six-view drawing showing an example of the left controller 3. As shown in Figure 4, the left controller 3 includes a housing 31. In this embodiment, the housing 31 has a vertically elongated shape, that is, it is long in the vertical direction (i.e., in the y-axis direction as shown in Figures 1 and 4). The left controller 3 can also be held in a vertically elongated orientation when detached from the main device 2. The housing 31 is shaped and sized to be held with one hand, especially the left hand, when held in a vertically elongated orientation. The left controller 3 can also be held in a horizontally elongated orientation. When the left controller 3 is held in a horizontally elongated orientation, it may be held with both hands.

[0055] The left controller 3 is equipped with an analog stick 32. As shown in Figure 4, the analog stick 32 is provided on the main surface of the housing 31. The analog stick 32 can be used as a directional input unit that can input direction. The user can input direction (and magnitude according to the angle of tilt) by tilting the analog stick 32. In addition, the left controller 3 may be equipped with a directional pad or a slide stick that allows slide input instead of the analog stick as the directional input unit. Furthermore, in this embodiment, input by pressing the analog stick 32 is also possible.

[0056] The left controller 3 is equipped with various operation buttons. The left controller 3 has four operation buttons 33-36 (specifically, a right direction button 33, a down direction button 34, an up direction button 35, and a left direction button 36) on the main surface of the housing 31. In addition, the left controller 3 is equipped with a record button 37 and a minus button 47. The left controller 3 has a first L button 38 and a ZL button 39 on the upper left side of the side of the housing 31. Furthermore, the left controller 3 has a second L button 43 and a second R button 44 on the side of the housing 31 that is attached when mounted to the main unit 2. These operation buttons are used to give instructions according to various programs (e.g., OS programs and application programs) executed on the main unit 2.

[0057] Furthermore, the left controller 3 is equipped with a terminal 42 for wired communication between the left controller 3 and the main unit 2.

[0058] Figure 5 is a six-view drawing showing an example of the right controller 4. As shown in Figure 5, the right controller 4 includes a housing 51. In this embodiment, the housing 51 has a vertically elongated shape, that is, a shape that is long in the vertical direction. When the right controller 4 is detached from the main unit 2, it can also be held in a vertically elongated orientation. The housing 51 is shaped and sized to be held with one hand, especially the right hand, when held in a vertically elongated orientation. The right controller 4 can also be held in a horizontally elongated orientation. When the right controller 4 is held in a horizontally elongated orientation, it may be held with both hands.

[0059] The right controller 4, like the left controller 3, is equipped with an analog stick 52 as a directional input unit. In this embodiment, the analog stick 52 has the same configuration as the analog stick 32 of the left controller 3. Alternatively, the right controller 4 may be equipped with a directional pad or a slide stick capable of slide input instead of the analog stick. The right controller 4, like the left controller 3, is equipped with four operation buttons 53-56 (specifically, A button 53, B button 54, X button 55, and Y button 56) on the main surface of the housing 51. Furthermore, the right controller 4 is equipped with a + (plus) button 57 and a home button 58. The right controller 4 is also equipped with a first R button 60 and a ZR button 61 on the upper right side of the housing 51. The right controller 4, like the left controller 3, is also equipped with a second L button 65 and a second R button 66.

[0060] Furthermore, the right controller 4 is equipped with a terminal 64 for wired communication between the right controller 4 and the main unit 2.

[0061] Figure 6 is a block diagram showing an example of the internal configuration of the main unit 2. In addition to the configuration shown in Figure 3, the main unit 2 includes the components 81-91, 97, and 98 shown in Figure 6. Some of these components 81-91, 97, and 98 may be mounted on an electronic circuit board as electronic components and housed within the housing 11.

[0062] The main unit 2 includes a processor 81. The processor 81 is an information processing unit that performs various information processing operations performed in the main unit 2, and may consist of, for example, only a CPU (Central Processing Unit), or it may consist of an SoC (System-on-a-chip) that includes multiple functions such as CPU function and GPU (Graphics Processing Unit) function. The processor 81 performs various information processing operations by executing information processing programs (for example, game programs) stored in a storage unit (specifically, an internal storage medium such as flash memory 84, or an external storage medium installed in slot 23).

[0063] The main unit 2 includes, as an example of an internal storage medium built into itself, a flash memory 84 and a DRAM (Dynamic Random Access Memory) 85. The flash memory 84 and DRAM 85 are connected to the processor 81. The flash memory 84 is a memory mainly used to store various types of data (which may be programs) stored in the main unit 2. The DRAM 85 is a memory used to temporarily store various types of data used in information processing.

[0064] The main unit 2 is equipped with a slot interface (hereinafter abbreviated as "I / F") 91. The slot I / F 91 is connected to the processor 81. The slot I / F 91 is connected to slot 23 and reads and writes data to a predetermined type of storage medium (for example, a dedicated memory card) installed in slot 23, according to instructions from the processor 81.

[0065] The processor 81 performs the above-mentioned information processing by appropriately reading and writing data to and from the flash memory 84 and DRAM 85, as well as to each of the above-mentioned storage media.

[0066] The main unit 2 includes a network communication unit 82. The network communication unit 82 is connected to the processor 81. The network communication unit 82 communicates with external devices via a network (specifically, wirelessly). In this embodiment, the network communication unit 82 communicates with external devices by connecting to a wireless LAN using a method compliant with the Wi-Fi standard as a first communication mode. The network communication unit 82 also performs wireless communication with other main unit 2 of the same type using a predetermined communication method (for example, communication using a proprietary protocol or infrared communication) as a second communication mode. The wireless communication using the second communication mode is possible with other main unit 2 located within a closed local network area, and realizes a function that enables so-called "local communication" in which data is sent and received by communicating directly between multiple main unit 2.

[0067] The main unit 2 includes a controller communication unit 83. The controller communication unit 83 is connected to the processor 81. The controller communication unit 83 communicates wirelessly with the left controller 3 and / or the right controller 4. The communication method between the main unit 2 and the left controller 3 and the right controller 4 is arbitrary, but in this embodiment, the controller communication unit 83 communicates with the left controller 3 and with the right controller 4 in accordance with the Bluetooth® standard.

[0068] The processor 81 is connected to the left terminal 17, right terminal 21, and lower terminal 27 described above. When the processor 81 communicates with the left controller 3 via a wired connection, it transmits data to the left controller 3 via the left terminal 17 and receives operation data from the left controller 3 via the left terminal 17. When the processor 81 communicates with the right controller 4 via a wired connection, it transmits data to the right controller 4 via the right terminal 21 and receives operation data from the right controller 4 via the right terminal 21. When the processor 81 communicates with the cradle, it transmits data to the cradle via the lower terminal 27. Thus, in this embodiment, the main unit 2 can perform both wired and wireless communication with the left controller 3 and the right controller 4, respectively. Furthermore, when the left controller 3 and the right controller 4 are mounted on the main unit 2 as an integrated unit, or when the main unit 2 alone is mounted on the cradle, the main unit 2 can output data (e.g., image data and audio data) to a stationary monitor or the like via the cradle.

[0069] Here, the main unit 2 can communicate simultaneously (in other words, in parallel) with multiple left controllers 3. Furthermore, the main unit 2 can communicate simultaneously (in other words, in parallel) with multiple right controllers 4. Therefore, multiple users can simultaneously input to the main unit 2 using their respective sets of left controllers 3 and right controllers 4. For example, while the first user inputs to the main unit 2 using the first set of left controllers 3 and right controllers 4, the second user can input to the main unit 2 using the second set of left controllers 3 and right controllers 4.

[0070] The main unit 2 includes a touch panel controller 86, which is a circuit that controls the touch panel 13. The touch panel controller 86 is connected between the touch panel 13 and the processor 81. Based on signals from the touch panel 13, the touch panel controller 86 generates data indicating, for example, the position where a touch input occurred, and outputs it to the processor 81.

[0071] The display 12 is also connected to the processor 81. The processor 81 displays images generated (for example, by performing the above information processing) and / or images acquired from an external source on the display 12.

[0072] The main unit 2 includes a codec circuit 87 and speakers (specifically, a left speaker and a right speaker) 88. The codec circuit 87 is connected to the speakers 88 and the audio input / output terminals 25, as well as to the processor 81. The codec circuit 87 is a circuit that controls the input and output of audio data to the speakers 88 and the audio input / output terminals 25.

[0073] Furthermore, the main unit 2 is equipped with an acceleration sensor 89. In this embodiment, the acceleration sensor 89 detects the magnitude of acceleration along a predetermined three-axis direction (for example, the x, y, and z axes shown in Figure 1). Note that the acceleration sensor 89 may also detect acceleration in one axis direction or two axis directions.

[0074] Furthermore, the main unit 2 is equipped with an angular velocity sensor 90. In this embodiment, the angular velocity sensor 90 detects angular velocity around three predetermined axes (for example, the x, y, and z axes shown in Figure 1). The angular velocity sensor 90 may also detect angular velocity around one axis or two axes.

[0075] The acceleration sensor 89 and the angular velocity sensor 90 are connected to the processor 81, and the detection results from the acceleration sensor 89 and the angular velocity sensor 90 are output to the processor 81. Based on the detection results from the acceleration sensor 89 and the angular velocity sensor 90, the processor 81 can calculate information regarding the movement and / or orientation of the main unit 2.

[0076] The illuminance sensor 29 is connected to the processor 81, and the detection result from the illuminance sensor 29 is output to the processor 81. Based on the detection result from the illuminance sensor 29, the processor 81 can calculate information regarding the brightness around the main unit 2.

[0077] The main unit 2 comprises a power control unit 97 and a battery 98. The power control unit 97 is connected to the battery 98 and the processor 81. Although not shown in the figures, the power control unit 97 is also connected to various parts of the main unit 2 (specifically, the parts that receive power from the battery 98, the left terminal 17, and the right terminal 21). Based on commands from the processor 81, the power control unit 97 controls the power supply from the battery 98 to the aforementioned parts.

[0078] The battery 98 is also connected to the lower terminal 27. When an external charging device (for example, a cradle) is connected to the lower terminal 27 and power is supplied to the main unit 2 via the lower terminal 27, the supplied power charges the battery 98.

[0079] Figure 7 is a block diagram showing an example of the internal configuration of the main unit 2, the left controller 3, and the right controller 4. Note that the details of the internal configuration of the main unit 2 are shown in Figure 6 and are therefore omitted in Figure 7.

[0080] The left controller 3 includes a communication control unit 101 that communicates with the main unit 2. As shown in Figure 7, the communication control unit 101 is connected to each component, including the terminal 42. In this embodiment, the communication control unit 101 can communicate with the main unit 2 both by wired communication via the terminal 42 and by wireless communication without using the terminal 42. The communication control unit 101 controls the method of communication that the left controller 3 performs with the main unit 2. That is, when the left controller 3 is attached to the main unit 2, the communication control unit 101 communicates with the main unit 2 via the terminal 42. When the left controller 3 is detached from the main unit 2, the communication control unit 101 performs wireless communication with the main unit 2 (specifically, the controller communication unit 83). Wireless communication between the controller communication unit 83 and the communication control unit 101 is performed according to, for example, the Bluetooth® standard.

[0081] The left controller 3 also includes a memory 102, such as flash memory. The communication control unit 101 is composed of, for example, a microcontroller (also called a microprocessor) and performs various processes by executing firmware stored in the memory 102.

[0082] The left controller 3 is equipped with buttons 103 (specifically, buttons 33-39, 43, 44, and 47). The left controller 3 is also equipped with an analog stick (referred to as "stick" in Figure 7) 32. Each button 103 and the analog stick 32 repeatedly output information about the operations performed on them to the communication control unit 101 at appropriate intervals.

[0083] The communication control unit 101 acquires information related to input (specifically, information related to operation or detection results from sensors) from each input unit (specifically, each button 103, analog stick 32, and each sensor 104 and 105). The communication control unit 101 transmits operation data, including the acquired information (or information obtained by performing a predetermined processing on the acquired information), to the main unit 2. The operation data is transmitted repeatedly at a rate of once per predetermined time. The interval at which information related to input is transmitted to the main unit 2 may or may not be the same for each input unit.

[0084] When the above operation data is transmitted to the main unit 2, the main unit 2 can obtain the input made to the left controller 3. In other words, the main unit 2 can determine the operation of each button 103 and the analog stick 32 based on the operation data.

[0085] The left controller 3 includes a power supply unit 108. In this embodiment, the power supply unit 108 includes a battery and a power control circuit. Although not shown, the power control circuit is connected to the battery and to each part of the left controller 3 (specifically, each part that receives power from the battery).

[0086] As shown in Figure 7, the right controller 4 includes a communication control unit 111 that communicates with the main unit 2. The right controller 4 also includes a memory 112 connected to the communication control unit 111. The communication control unit 111 is connected to each component, including the terminal 64. The communication control unit 111 and the memory 112 have the same functions as the communication control unit 101 and memory 102 of the left controller 3. Therefore, the communication control unit 111 can communicate with the main unit 2 both by wired communication via the terminal 64 and by wireless communication without the terminal 64 (specifically, communication according to the Bluetooth® standard), and controls the method of communication that the right controller 4 performs with the main unit 2.

[0087] The right controller 4 is equipped with the same inputs as the left controller 3. Specifically, it is equipped with buttons 113 and an analog stick 52. These inputs have the same functions and operate in the same way as the inputs of the left controller 3.

[0088] The right controller 4 is equipped with a power supply unit 118. The power supply unit 118 has the same functions and operates in the same manner as the power supply unit 108 of the left controller 3.

[0089] Next, with reference to Figures 8 to 15, we will describe a goggle device 150, which is an example of a device that constitutes an image display system when the game system 1 (specifically, the main unit 2) is attached to it. Figure 8 is a perspective view showing an example of the external appearance of the goggle device 150. Figure 9 is a front view showing an example of the state in which the main unit 2 is attached to the goggle device 150. Figure 10 is a front view showing an example of the state in which the main unit 2 is attached to the goggle device 150. Figure 11 is a diagram showing an example of the shape of the front contact portion 151b that comes into contact with a part of the front surface of the main unit 2. Figure 12 is a diagram showing an example of the internal structure of the goggle device 150. Figure 13 is a side view showing an example of the state in which the main unit 2 is attached to the goggle device 150. Figure 14 is a diagram showing an example of a user viewing an image displayed on the image display system. Figure 15 is a diagram showing an example of a user grasping the image display system. Figure 11 is a view from the same direction as Figure 10, with a portion of the goggle device 150 (part of the main body 151, lens frame member 152, lens 153, and plate-shaped member 154) removed (transparent) in order to clearly show the front contact portion 151b.

[0090] In Figures 8 to 13, the goggle device 150 has a main body 151, a lens frame member 152, a lens 153, and a plate-shaped member 154. Here, the goggle device, which is an example of a device constituting an image display system, is not limited to the configuration described later, as long as it is worn to fit the user's face so as to cover the user's left and right eyes, has the function of blocking at least a portion of the external light, and has the function of supporting stereoscopic vision for the user with a pair of lenses. For example, the goggle device may be used in various ways, such as a type that fits the user's face when grasped by the user, a type that fits the user's face when fixed to the user's head, or a type in which the user looks into a mounted device. Furthermore, the goggle device may function as a so-called head-mounted display when worn on the user's head with the main body device 2 attached, and may have a helmet-type shape in addition to the goggle type. In the following description of the goggle device 150, a goggle-type goggle device that is grasped by the user and worn to fit the user's face will be used.

[0091] The main body 151 has mounting parts that detachably fix the main body device 2 by contacting its front, back, top, and bottom surfaces. The mounting parts have a front contact part that contacts a part of the front surface of the main body device 2 (the surface on which the display 12 is provided), a rear contact part that contacts the back surface of the main body device 2, a top contact part that contacts the top surface of the main body device 2, and a bottom contact part that contacts the bottom surface of the main body device 2. The mounting parts are formed in a rectangular tube shape with a gap formed by the front contact part, rear contact part, top contact part, and bottom contact part, and both left and right sides are open. The mounting parts have openings on both sides (the side on the positive x-axis direction and the side on the negative x-axis direction in the figure) so that they can be attached from the left side or the right side of the main body device 2. Then, as shown in Figure 9, when the goggle device 150 is attached to the main unit 2 through the opening on the right side, the front contact portion contacts the front of the main unit 2, the rear contact portion contacts the rear of the main unit 2, the top contact portion contacts the top of the main unit 2, and the bottom contact portion contacts the bottom of the main unit 2. As shown in Figure 11, the front contact portion 151b of the main unit 151 is formed with an opening that does not obstruct the view of the display images (left eye image and right eye image) of the display 12 when the main unit 2 is attached.

[0092] As shown in Figures 9 and 13, the main unit 2 is attached to the goggle device 150 by sliding it into the gap of the mounting portion of the main unit 151, along the front contact portion, rear contact portion, top contact portion, and bottom contact portion of the mounting portion, from the left or right side of the main unit 2. The main unit 2 can also be removed from the goggle device 150 by sliding it to the left or right along the front contact portion, rear contact portion, top contact portion, and bottom contact portion of the mounting portion. In this way, the goggle device 150 can be detachably fitted with the main unit 2.

[0093] The lens frame member 152 is fixed to the opening formed on the front surface of the main body 151. The lens frame member 152 has a pair of lens frames with openings that do not obstruct the view of the display images (left eye image IML and right eye image IMR) displayed on the display 12 of the main unit device 2 attached to the main body 151. In addition, the outer edges formed on the top, bottom, left, and right of the lens frame member 152 have bonding surfaces formed for joining to the main unit device 2, and a V-shaped recess is formed in the center of the lower outer edge for contact with the nose of the user wearing the goggle device 150.

[0094] The lens 153 is composed of a pair of left-eye lenses 153L and right-eye lenses 153R, which are, for example, a pair of Fresnel lenses. The left-eye lens 153L and the right-eye lens 153R are each fitted into the lens frames of the lens frame member 152. Specifically, the left-eye lens 153L is fitted into one lens frame which is opened so as not to obstruct the view of the left-eye image IML displayed on the display 12 of the main unit device 2 attached to the main unit 151, so that when the user looks through the left-eye lens 153L with their left eye, they can see the left-eye image IML. The right-eye lens 153R is fitted into the other lens frame which is opened so as not to obstruct the view of the right-eye image IMR displayed on the display 12 of the main unit device 2 attached to the main unit 151, so that when the user looks through the right-eye lens 153R with their right eye, they can see the right-eye image IMR. The left-eye lens 153L and the right-eye lens 153R can be typically circular or elliptical magnifying lenses, or lenses that distort the image for the user to perceive. For example, the left-eye lens 153L may distort the left-eye image IML (described later), which is displayed distorted in a circular or elliptical shape, in the opposite direction to the distortion of the image, and the right-eye lens 153R may distort the right-eye image IMR (described later), which is displayed distorted in a circular or elliptical shape, in the opposite direction to the distortion of the image, thereby enabling stereoscopic viewing. Furthermore, the left-eye lens 153L and the right-eye lens 153R may be integrally formed.

[0095] The main body 151 has a contact portion that protrudes outward from the front side of the main body 151 so as to surround the outer edge of the lens frame member 152 in a rectangular tube shape. The end face of the contact portion that protrudes outward from the front side is positioned on the front side of the lens 153 when the lens 153 is viewed from the outside of the goggle device 150, and when the main body device 2 is attached, this end face is positioned on the front side (negative z-axis side) of the goggle device 150. Furthermore, the contact portion of the main body 151 has an end face shape that fits the user's face (typically around the user's eyes) when the user looks into the goggle device 150 with the main body device 2 attached, and by contacting the user's face, this end face has the function of fixing the positional relationship between the user's eyes and the lens 153.

[0096] Furthermore, the contact portion can block external light from reaching the left-eye lens 153L and the right-eye lens 153R when viewing a stereoscopic image displayed on the display 12 using the image display system. This improves the user's sense of immersion when viewing the stereoscopic image displayed on the display 12. Note that the light shielding by the contact portion does not need to completely block external light. For example, as shown in Figure 15, a recess may be formed in a part of the cylindrical contact portion. The recess in the contact portion illustrated in Figure 15 is formed at a position below the midpoint between the left-eye lens 153L and the right-eye lens 153R, which is the position where the user's nose comes into contact with the stereoscopic image displayed on the display 12. In other words, the recess in the contact portion makes it possible to avoid strong contact between the contact portion and the user's nose, and even if the light shielding performance is slightly reduced, the discomfort caused by contact between the contact portion and the nose can be alleviated.

[0097] As shown in Figure 10, the plate-shaped member 154 is fixed inside the main body 151, between the lens frame member 152 and the display 12, when the main body device 2 is mounted on the mounting portion of the main body 151. For example, a part of the plate-shaped member 154 is shaped to follow the V-shaped recess in the lens frame member 152, and is positioned as a wall (hereinafter referred to as the first wall) connecting the recess and the display 12 of the mounted main body device 2. The space enclosed by the first wall becomes an opening 154h that exposes a part of the display 12 of the main body device 2 mounted on the main body 151 to the outside, and functions as an operation window that allows the user to touch and operate that part through the space. Note that a part of the first wall of the plate-shaped member 154 may be open, as shown in Figure 12.

[0098] Furthermore, as shown in Figure 12, the plate-shaped member 154 is, for example, erected vertically between the left-eye lens 153L and the right-eye lens 153R, and is positioned as a wall (hereinafter referred to as the second wall) connecting the recess and the display 12 of the main unit 2 to which it is mounted. When the main unit 2 is mounted on the main unit 151, the second wall is positioned to separate the left-eye image IML and the right-eye image IMR displayed on the display 12, and functions as a partition wall provided between the left-eye image IML and the right-eye image IMR. The plate-shaped member 154 is provided by extending the first wall to the second wall, and the first wall and the second wall are formed by an integrated member.

[0099] In Figures 10, 13, 14, and 15, the image display system is configured by attaching the main unit 2 to the goggle device 150. In this embodiment, the main unit 2 is attached so that it is completely covered by the goggle device 150. When the main unit 2 is attached to the goggle device 150, it is possible to see only the left-eye image IML displayed in the left area of ​​the display 12 through the left-eye lens 153L, and to see only the right-eye image IMR displayed in the right area of ​​the display 12 through the right-eye lens 153R. Therefore, the user of the image display system can view the left-eye image IML and the right-eye image IMR by looking at the left-eye lens 153L with their left eye and the right-eye lens 153R with their right eye. By displaying the left-eye image IML and the right-eye image IMR with parallax on the display 12, a three-dimensional image with a sense of depth can be displayed to the user.

[0100] As shown in Figures 14 and 15, when a user grasps the image display system with the main unit 2 attached to the goggle device 150 and views the stereoscopic image displayed on the display 12, they can grasp the left side of the goggle device 150 with their left hand and the right side of the goggle device 150 with their right hand. By grasping the left and right sides of the goggle device 150 in this way, the main unit 2 can be maintained in a stable position.

[0101] Furthermore, even when the main unit 2 is attached to the goggles device 150, the image display system allows touch operation of a portion of the touch panel 13 (the third area of ​​the display 12, described later) on the screen of the display 12 through the opening 154h formed by the first wall portion of the plate-shaped member 154. The image display system can also calculate information regarding the movement and / or orientation of the main unit 2, i.e., the movement and / or orientation of the image display system including the goggles device 150, based on the detection results of the acceleration sensor 89 and / or angular velocity sensor 90 provided on the main unit 2. Therefore, the image display system can calculate the orientation of the user's head relative to the direction of gravity when looking through the goggles device 150 with the main unit 2 attached. In addition, if the orientation or direction of the user's head changes when looking through the goggles device 150 with the main unit 2 attached, the image display system can calculate the changed direction and angle. Furthermore, if the user looking through the goggles device 150 with the main unit 2 attached applies a vibration such as hitting the image display system, the image display system can detect that vibration. Therefore, when viewing the stereoscopic image displayed on the display 12 through the left eye lens 153L and the right eye lens 153R with the main unit 2 attached to the goggle device 150, a play style is realized that allows for touch operation through the aperture 154h, operation based on the orientation of the image display system relative to the direction of gravity, operation to change the orientation of the image display system, and operation to vibrate the image display system.

[0102] In this embodiment, when using the image display system, it may be operated using at least one of the left controller 3 and the right controller 4, which are detached from the main unit 2. For example, when operating the image display system using the left controller 3, the user holds the goggle device 150 to which the main unit 2 is attached with their right hand while viewing the stereoscopic image displayed on the display 12, and simultaneously holds the detached left controller 3 by itself with their left hand to operate it. In this case, operation information performed on the left controller 3 and / or the right controller 4, which are detached from the main unit 2, is transmitted to the main unit 2 via wireless communication. Specifically, operation information performed on the left controller 3 is wirelessly transmitted from the communication control unit 101 of the left controller 3 and received by the controller communication unit 83 of the main unit 2. Operation information performed on the right controller 4 is wirelessly transmitted from the communication control unit 111 of the right controller 4 and received by the controller communication unit 83 of the main unit 2.

[0103] Thus, in this embodiment, by attaching the main unit 2 to the goggle device 150, a portable image display system can be configured that the user can hold and view stereoscopic images with. Furthermore, in this embodiment of the image display system, since the user views the stereoscopic image displayed on the display 12 of the main unit 2 while the user's face is in contact with the goggle device 150, the positional relationship between the stereo speakers (left speaker 88L and right speaker 88R) provided on the main unit 2 and the user's ears is fixed, and the left and right speakers are positioned near the user's ears. Therefore, the main unit 2 can output sound based on the positional relationship between the audio output device and the viewer's ears without forcing the use of earphones or speakers. For example, the main unit 2 can control the sound source using so-called stereophonic sound technology based on the positional relationship between the audio output device and the viewer's ears.

[0104] Next, the images displayed on the main unit 2 will be described with reference to Figures 9, 10, and 16. Figure 16 shows examples of images displayed on the main unit 2 in stereoscopic display mode and non-stereoscopic display mode, respectively.

[0105] In this embodiment, the image display system is set to either a stereoscopic display mode, which is used when the main unit 2 is attached to the goggle device 150 and the image displayed on the display 12 is viewed in 3D, or a non-stereoscopic display mode, which is used when the main unit 2 is removed from the goggle device 150 and the image displayed on the display 12 is viewed directly to avoid stereoscopic viewing. The image display system then displays the image corresponding to the set mode on the display 12 of the main unit 2. Here, the stereoscopic image for 3D viewing may be a right-eye image and a left-eye image with parallax between them, which the user views with their right and left eyes, respectively, to achieve 3D viewing. In this case, the non-stereoscopic image is an image other than the two image displays (stereoscopic display) described above, and is typically a single image viewed by the user with their right and left eyes. In this embodiment, the non-stereoscopic display mode is used as an example of the first display mode. In this embodiment, the stereoscopic display mode is used as an example of the second display mode.

[0106] In stereoscopic display mode, the image display system constructs the content image to be displayed (for example, an image to display a part of a virtual space or real space) using a left-eye image IML and a right-eye image IMR that have parallax with respect to each other. The left-eye image IML is displayed in the left area of ​​the display 12, and the right-eye image IML is displayed in the right area of ​​the display 12. Specifically, as shown in Figure 9, in stereoscopic display mode, the left-eye image IML is a roughly elliptical area visible to the left-eye lens 153L when the main unit 2 is attached to the goggle device 150, and is displayed in a first area that is part of the left area of ​​the display 12. Also, in stereoscopic display mode, the right-eye image IMR is a roughly elliptical area visible to the right-eye lens 153R when the main unit 2 is attached to the goggle device 150, and is displayed in a second area that is part of the right area of ​​the display 12.

[0107] As described above, when the main unit 2 is attached to the goggle device 150, the second wall of the plate-shaped member 154 is positioned between the left-eye image IML displayed in the first area of ​​the display 12 and the right-eye image IMR displayed in the second area. Therefore, the left-eye image IML and the right-eye image IMR are separated by the second wall of the plate-shaped member 154, preventing the right-eye image IMR from being viewed through the left-eye lens 153L, or the left-eye image IML from being viewed through the right-eye lens 153R.

[0108] As an example, images of the virtual space as seen from a pair of virtual cameras (left virtual camera and right virtual camera) with parallax placed in the virtual space are generated as the left-eye image IML and the right-eye image IMR, respectively. The pair of virtual cameras are positioned in the virtual space so as to correspond to the orientation of the main device 2 relative to the direction of gravity in real space. The pair of virtual cameras then change their orientation in the virtual space in accordance with the change in the orientation of the main device 2 in real space, and control the line of sight direction of the virtual cameras according to the orientation of the main device 2. As a result, the user wearing the image display system can change the display range of the stereoscopic virtual space by changing the orientation of the main device 2 (image display system) to look around, and can look around the stereoscopic virtual space, so that they can experience as if they were actually in the location of the virtual camera. In this embodiment, the main device 2 aligns the direction of the gravitational acceleration acting on the main device 2 with the direction of gravity in the virtual space acting on the virtual cameras, and also aligns the amount of change in the orientation of the main device 2 with the amount of change in the line of sight direction of the virtual cameras. This increases the realism of the operation when viewing the stereoscopic virtual space based on the orientation of the main unit 2.

[0109] Furthermore, the image display system displays a user interface image IMU on the display 12 to accept touch operations on the touch panel 13 of the main unit 2. For example, the user interface image IMUa displayed in stereoscopic display mode is displayed in the display area of ​​the display 12 that can be touch-operated via the opening 154h of the goggle device 150. For example, as described above, the opening 154h is formed surrounded by the first wall portion of the plate-shaped member 154, and allows touch operations on a part of the display 12 of the main unit 2 mounted on the goggle device 150 (specifically, the area near the lower center of the display 12) from a V-shaped recess in the lens frame member 152 that contacts the user's nose. As an example, as shown in Figure 9, the opening 154h allows touch operations on a third area set below the display 12, sandwiched between the first and second areas of the display 12, even when the main unit 2 is mounted on the goggle device 150.

[0110] For example, in stereoscopic display mode, two user interface images, IMUa1 and IMUa2, are displayed in the third area of ​​the display 12. As an example, user interface image IMUa1 is an operation icon that, when its display position is touched via the touch panel 13, instructs the user to retry the game from the beginning. User interface image IMUa2 is an operation icon that, when its display position is touched via the touch panel 13, instructs the user to exit the game. The two user interface images, IMUa1 and IMUa2, are displayed side by side in the third area of ​​the display 12, sized to fit the shape of the third area. This allows the user to perform multiple operations via touch by touching either of the two user interface images, IMUa1 or IMUa2, through the opening 154h, even when the main unit 2 is attached to the goggles device 150. The two user interface images, IMUa1 and IMUa2, may also be displayed near the third area, where touch operation is possible by exposing a portion of the display 12 to the outside. In other words, parts of the two user interface images IMUa1 and / or IMUa2 may be displayed outside the third region described above. In this embodiment, user interface image IMUa is used as an example of the second user interface image.

[0111] The two user interface images IMUa1 and IMUa2 are images displayed in stereoscopic display mode, but on the display 12 of the main unit 2 attached to the goggle device 150, they are displayed in a third region that is outside the first region visible to the user's left eye and the second region visible to the user's right eye. Therefore, the user interface images IMUa1 and IMUa2 displayed in the third region are displayed outside the user's field of view when viewed through the goggle device 150, and since they are not composed of two images with parallax, they are displayed as non-stereoscopic images that cannot be viewed in stereo. In addition, since the user interface images IMUa1 and IMUa2 that are the target of touch operation are displayed outside the first and second regions for displaying stereoscopic images, the first and second regions are less likely to be touched. Therefore, it is possible to prevent the first and second regions for displaying stereoscopic images from being soiled by touch operation of the display 12, and to prevent fingers used for touch operation from entering the field of view while viewing stereoscopic images.

[0112] In other embodiments, the user interface images IMUa1 and IMUa2 may be displayed on the display 12 as stereoscopic images that can be viewed in stereo in stereoscopic display mode. In this case, the user interface images IMUa1 and IMUa2 are each composed of two images with parallax to each other and are displayed on the display 12 as stereoscopic images, typically with one image for stereoscopic viewing displayed in a part of the first region and the other image for stereoscopic viewing displayed in a part of the second region.

[0113] As shown in Figure 16, in non-stereoscopic display mode, the image display system constructs the content image to be displayed as a single image IMS, which is a non-stereoscopic image, and displays the single image IMS across the entire display area of ​​the display 12 as an example.

[0114] As an example, an image of the virtual space as seen from a single virtual camera placed in the virtual space is generated as a single image IMS. The single virtual camera is positioned in the virtual space so as to correspond to the orientation of the main device 2 relative to the direction of gravity in real space. The single virtual camera then changes its orientation in the virtual space in accordance with the change in the orientation of the main device 2 in real space, and controls the line of sight of the virtual camera according to the orientation of the main device 2. As a result, a user holding the main device 2 after it has been removed from the goggle device 150 can change the display range of the virtual space shown on the display 12 by changing the orientation of the main device 2 to look around, thus providing an experience as if they were actually at the location of the virtual camera. In this embodiment, even in non-stereoscopic display mode, the main device 2 aligns the direction of the gravitational acceleration acting on the main device 2 with the direction of gravity in the virtual space acting on the virtual camera, and also aligns the amount of change in the orientation of the main device 2 with the amount of change in the line of sight of the virtual camera. This increases realism even when looking around the non-stereoscopic virtual space by changing the orientation of the main device 2. In this embodiment, a single-image IMS is used as an example of a non-stereoscopic image.

[0115] Furthermore, the user interface image IMUb displayed in non-stereoscopic display mode is superimposed on the content image (single image IMS) displayed on the display 12, for example, and displayed on the display 12. For example, as shown in Figure 16, even in non-stereoscopic display mode, two user interface images IMUb1 and IMUb2 are displayed on the display 12. As an example, user interface image IMUb1 is the image corresponding to user interface image IMUa1, and when its display position is touched via the touch panel 13, it is an operation icon that instructs the user to retry the game from the beginning. Similarly, user interface image IMUb2 is the image corresponding to user interface image IMUa2, and when its display position is touched via the touch panel 13, it is an operation icon that instructs the user to end the game. Here, the image corresponding to user interface image IMUa is indicated as having a different design and / or size, but with substantially the same functionality as user interface image IMUa (for example, the content of the operation instruction when touched is the same). Furthermore, the image corresponding to the user interface image IMUa may be not only substantially identical in function to the user interface image IMUa displayed in non-stereoscopic display mode, but also identical in design and size, i.e., completely identical. In this embodiment, the user interface image IMUb is used as an example of the first user interface image.

[0116] The two user interface images IMUb1 and IMUb2 are displayed in corner areas of the display 12 that are different from the third area mentioned above (for example, the upper left corner and the upper right corner). Since the two user interface images IMUb1 and IMUb2 are not limited by the area where touch operation is possible, they can be displayed larger than the user interface image IMUa displayed in stereoscopic display mode, and can be displayed in a size and shape that is easy for the user to touch, and in a position where the visibility of the content image (single image IMS) is not easily impaired by touch operation.

[0117] The image display system in this embodiment can automatically switch between the stereoscopic display mode and the non-stereoscopic display mode based on the detection result of whether the main unit 2 is attached to the goggle device 150 or in the process of being attached. For example, the main unit 2 is provided with an illuminance sensor 29 that detects the illuminance (brightness) of light incident from the main surface side of the housing 11, and based on the illuminance detection result from the illuminance sensor 29, it is possible to detect whether the main unit 2 is attached to the goggle device 150 or in the process of being attached. Specifically, when the main unit 2 is attached to the goggle device 150 or in the process of being attached, the illuminance detected by the illuminance sensor 29 becomes darker, so by setting a threshold that can detect the darkened illuminance and detecting whether it is above the threshold, it is possible to detect whether the main unit 2 is attached or in the process of being attached. Here, "being attached" as detected by the main unit 2 based on the illuminance detection result from the illuminance sensor 29 means that the main unit 2 is completely attached to the goggle device 150. Furthermore, based on the illuminance detection results from the illuminance sensor 29, the "in-progress state of attachment" detected by the main unit 2 means that the main unit 2 is in a state prior to being fully attached to the goggle device 150.

[0118] There are several possible ways for the illuminance sensor 29 to detect whether the main unit 2 is attached or in the process of being attached. As a first example, as shown in Figure 11, in the attachment part of the goggle device 150 that detachably fixes the main unit 2, if the front contact portion 151b of the main body 151 that comes into contact with a part of the front surface (the surface on which the display 12 is provided) of the attached main unit 2 is formed to cover the light-receiving surface and light-receiving holes of the illuminance sensor 29, then the presence or absence of the main unit 2 being attached to the goggle device 150 or in the process of being attached is detected by the state in which the front contact portion 151b covers the light-receiving surface and light-receiving holes of the illuminance sensor 29. For example, if the light-receiving surface and light-receiving holes of the illuminance sensor 29 are located in a corner of the main unit 2, when the main unit 2 is inserted into the mounting section of the goggle device 150 from the corner side, the light-receiving surface and light-receiving holes will be covered by the front contact portion 151b in the initial stages of the mounting operation. This makes it possible to detect that the main unit 2 is "in the process of being mounted" based on the illuminance detection result from the illuminance sensor 29. Alternatively, if the main unit 2 is inserted into the mounting section of the goggle device 150 from the side opposite to the corner where the light-receiving surface and light-receiving holes of the illuminance sensor 29 are located, the light-receiving surface and light-receiving holes will be covered by the front contact portion 151b in the final stages of the mounting operation. This makes it possible to detect that the main unit 2 is "attached" based on the illuminance detection result from the illuminance sensor 29. As another example, if the light-receiving surface and light-receiving holes of the illuminance sensor 29 are located on the lower central side of the main unit 2, when the main unit 2 is inserted into the mounting section of the goggle device 150, the light-receiving surface and light-receiving holes will be covered by the front contact portion 151b during the initial to middle stages of the mounting operation, regardless of the direction from which it is inserted. Therefore, it becomes possible to detect that the main unit 2 is "in the process of being mounted" based on the illuminance detection result from the illuminance sensor 29. In the first example above, the front contact portion 151b of the main unit 151 is used as an example of a light-shielding portion.

[0119] As a second example, even if the front contact portion 151b is not formed to cover the light-receiving surface or light-receiving hole of the illuminance sensor 29, it is possible to detect whether the main unit 2 is attached to the goggle device 150 or in the process of being attached by utilizing the fact that the inside of the main unit 151 is relatively dark. For example, it is conceivable that external light is entering the main unit 151 of the goggle device 150 through the left eye lens 153L and the right eye lens 153R, but even in this state, the attached state of the main unit 2 is detected by detecting that the illuminance is darker than the outside of the main unit 151. As an example, if the light-receiving surface or light-receiving hole of the illuminance sensor 29 is located in the corner of the main unit 2, when the main unit 2 is inserted into the attachment part of the goggle device 150 from the corner side, the light-receiving surface or light-receiving hole is inserted into the inside of the main unit 151 in the initial stages of the attachment operation, so it is possible to detect that the main unit 2 is in the "process of being attached" based on the illuminance detection result by the illuminance sensor 29. Furthermore, if the main unit 2 is inserted into the mounting section of the goggle device 150 from the side opposite to the corner where the light-receiving surface and light-receiving holes of the illuminance sensor 29 are located, the light-receiving surface and light-receiving holes are inserted into the main unit 150 at the end of the mounting operation, making it possible to detect that the main unit 2 is "in a mounted state" based on the illuminance detection result by the illuminance sensor 29. As another example, if the light-receiving surface and light-receiving holes of the illuminance sensor 29 are located on the lower central side of the main unit 2, when the main unit 2 is inserted into the mounting section of the goggle device 150, the light-receiving surface and light-receiving holes are inserted into the main unit 151 at the initial to middle stages of the mounting operation, regardless of the direction from which it is inserted, making it possible to detect that the main unit 2 is "in a state of being mounted" based on the illuminance detection result by the illuminance sensor 29. Note that in the second example above, the main unit 151 is used as another example of the light-shielding part.

[0120] As a third example, even if the front contact portion 151b is not formed to cover the light-receiving surface or light-receiving hole of the illuminance sensor 29, the presence of the main unit 2 on the goggle device 150 can be detected by utilizing the fact that the inside of the main unit 151 becomes darker when the user wears the goggle device 150 to which the main unit 2 is attached. For example, when the user is not wearing the goggle device 150, external light may enter the main unit 151 of the goggle device 150 through the left eye lens 153L and the right eye lens 153R. In this case, when the user wears the goggle device 150, the external light entering through the left eye lens 153L and the right eye lens 153R is blocked, and the presence of the main unit 2 can be detected by detecting that the illuminance inside the main unit 151 becomes even darker. For example, if the light incident on the light-receiving surface or light-receiving hole of the illuminance sensor 29 located inside the main body 151 of the goggle device 150 is blocked by the main body 151 and the user's face, it becomes possible to detect that the main body 2 is "in a wearing state" based on the illuminance detection result by the illuminance sensor 29. In this example, it is possible to detect not only whether the main body 2 is attached to the goggle device 150, but also whether the user is wearing the goggle device 150 to which the main body 2 is attached. Note that in the third example above, the main body 151 is used as another example of a light-blocking part.

[0121] In this embodiment, the image display system determines that the main unit 2 is not attached to the goggle device 150, and sets the display mode of the main unit 2 to non-stereoscopic display mode. On the other hand, if the main unit 2, which is set to non-stereoscopic display mode, determines that the main unit 2 has changed from being unattached to being attached to the goggle device 150, the main unit 2 displays the same content image on the display 12 by changing the content image of the non-stereoscopic image being displayed to a stereoscopic image. As an example, if the main unit 2 has set up a single virtual camera in the virtual space to display a single image IMS in non-stereoscopic display mode and is generating a virtual space image, it changes to a pair of virtual cameras (left virtual camera and right virtual camera) with parallax between them without changing the position and line of sight of the single virtual camera, and sets up virtual cameras to display the left-eye image IML and the right-eye image IMR, thereby switching to the generation of a virtual space image in stereoscopic display mode. Furthermore, in the main unit 2, which is set to stereoscopic display mode, if it is determined that the main unit 2 has changed from being attached to the goggle device 150 to being unattached, the main unit 2 changes the content image of the displayed stereoscopic image to a non-stereoscopic image, thereby displaying the same content image corresponding to the content image of the stereoscopic image as a non-stereoscopic image on the display 12. For example, if the main unit 2 has set up a pair of virtual cameras in virtual space to display the left eye image IML and the right eye image IMR in stereoscopic display mode and generates a virtual space image, the main unit 2 changes to a single virtual camera without changing the position and line of sight direction of the pair of virtual cameras, and sets up a virtual camera to display a single image IMS, thereby switching to the generation of a virtual space image in non-stereoscopic display mode. In this way, the content image of a non-stereoscopic image (e.g., a virtual space image) corresponding to the content image of a stereoscopic image (e.g., a virtual space image), or the content image of a stereoscopic image (e.g., a virtual space image) corresponding to the content image of a non-stereoscopic image (e.g., a virtual space image), is in any case only different in that it is either a stereoscopic image or a non-stereoscopic image.However, the display range of a non-stereoscopic content image corresponding to a stereoscopic content image, or a stereoscopic content image corresponding to a non-stereoscopic content image, may differ, and typically the display range of the stereoscopic content image may be narrower than that of the non-stereoscopic content image.

[0122] Furthermore, in this embodiment, when the display mode is switched, the image display system changes the size, shape, and position of the user interface image IMU and displays it on the display 12. For example, in the main unit 2, which is set to non-stereoscopic display mode, if it is determined that the main unit 2 has changed from a state where it is not attached to the goggle device 150 to a state where it is attached, the main unit 2 changes the shape of the user interface images IMUb1 and IMUb2, which are displayed overlapping with the content image in the corner area of ​​the display 12, to user interface images IMUa1 and IMUa2, and moves their display position into the third area of ​​the display 12, thereby displaying the user interface image IMU with the same function. Furthermore, in the main unit 2, which is set to stereoscopic display mode, if it is determined that the main unit 2 has changed from being attached to the goggle device 150 to being unattached, the main unit 2 changes the shape of the user interface images IMUa1 and IMUa2 displayed in the third area of ​​the display 12 to user interface images IMUb1 and IMUb2, and moves their display position so that they overlap with the content image in the corner area of ​​the display 12, thereby displaying the user interface image IMU of the same function.

[0123] In the above-described embodiment, an example was used in which the main unit 2 is detected to be attached to the goggle device 150 or in the process of being attached based on the illuminance detection result from the illuminance sensor 29. However, the main unit 2 may also be detected to be attached to the goggle device 150 or in the process of being attached based on other detection results. For example, the main unit 2 may be detected to be attached to the goggle device 150 or in the process of being attached based on the detection result of the connection terminals provided on the main unit 2 and the connection terminals provided on the goggle device 150 being electrically connected when the main unit 2 is attached or in the process of being attached, or on the detection result of a predetermined switch mechanism provided on the main unit 2 being turned on or off when the main unit 2 is attached or in the process of being attached. As another example, the main unit 2 may be detected to be attached to the goggle device 150 or in the process of being attached based on the imaging result from the imaging means (image sensor) provided on the main unit 2, by determining whether a predetermined image has been captured or whether the captured brightness is above a threshold. As another example, when the main unit 2 is attached to or in the process of being attached to the goggle device 150, the system may prompt the user to perform a predetermined operation, and based on whether the predetermined operation has been performed, it may detect whether the main unit 2 is attached to or in the process of being attached to the goggle device 150.

[0124] Furthermore, in the above-described embodiment, the third area is set below the center of the display 12, which is sandwiched between the first and second areas of the display 12, thereby enabling touch operation on the third area. However, the third area may be set in other areas of the display 12. As a first example, the third area may be set above the center of the display 12, which is sandwiched between the first and second areas of the display 12. As a second example, the third area may be set above (i.e., the upper left corner area of ​​the display 12) or below (i.e., the lower left corner area of ​​the display 12) the area sandwiched between the first area and the left edge of the display 12. As a third example, the third area may be set above (i.e., the upper right corner area of ​​the display 12) or below (i.e., the lower right corner area of ​​the display 12) the area sandwiched between the second area and the right edge of the display 12. Regardless of which region the third region is set in, the same operation as described above is possible by displaying a user interface image IMUa that matches the shape of the third region within the third region and by forming an opening 154h in the goggle device 150 that allows touch operation within the third region. Note that if the third region is set between the first and second regions, which are sandwiched between the first and second regions, it may be shifted to the left or right from the midpoint between the first and second regions.

[0125] Furthermore, in the above-described embodiment, the user interface image was used as an example to show a user interface image that is displayed on the display 12 to accept touch operations on the touch panel 13, but it may also be one that does not accept touch operations. For example, the above-described user interface image may be text information or icons that present some kind of information to the user, even though it cannot be touched. For example, the above-described user interface image may be an image that shows instructions corresponding to button operations on the left controller 3 or right controller 4, stick operations, or operations that move the controller body.

[0126] Furthermore, the left-eye image IML and the right-eye image IMR may be displayed outside the display area of ​​the display 12 visible by the left-eye lens 153L and the right-eye lens 153R (typically outside the first area and / or the second area), and a portion of them may also be displayed within the third area where touch operation is possible. In addition, the left-eye image IML and the right-eye image IMR may be displayed in an area smaller than the display area of ​​the display 12 visible by the left-eye lens 153L and the right-eye lens 153R (typically the first area and / or the second area).

[0127] Next, with reference to Figures 17 and 18, an example of a specific process performed by the game system 1 in this embodiment will be described. Figure 17 shows an example of a data area set in the DRAM 85 of the main unit 2 in this embodiment. In addition to the data shown in Figure 17, the DRAM 85 also stores data used in other processes, but a detailed explanation will be omitted.

[0128] The program memory area of ​​the DRAM 85 stores various programs Pa that are executed by the game system 1. In this embodiment, the various programs Pa include communication programs for wireless communication with the left controller 3 and right controller 4 described above, and application programs for performing information processing (e.g., game processing) based on data acquired from the operation unit (left controller 3, right controller 4, touch panel 13, acceleration sensor 89, angular velocity sensor 90) and the illuminance sensor 29. The various programs Pa may be pre-stored in the flash memory 84, acquired from a storage medium that can be attached to the game system 1 (e.g., a predetermined type of storage medium installed in slot 23) and stored in the DRAM 85, or acquired from other devices via a network such as the Internet and stored in the DRAM 85. The processor 81 executes the various programs Pa stored in the DRAM 85.

[0129] Furthermore, the data storage area of ​​the DRAM 85 stores various types of data used in communication processing, information processing, and other processes performed in the game system 1. In this embodiment, the DRAM 85 stores operation data Da, angular velocity data Db, acceleration data Dc, illuminance data Dd, posture data De, operation object data Df, virtual camera data Dg, virtual space image data for the left eye Dh, virtual space image data for the right eye Di, stereoscopic UI (user interface) image data Dj, non-stereoscopic virtual space image data Dk, non-stereoscopic UI (user interface) image data Dm, and image data Dn, among others.

[0130] The operation data Da is operation data acquired as appropriate from the left controller 3 and / or the right controller 4 and the touch panel 13, respectively. As described above, the operation data transmitted from the left controller 3 and / or the right controller 4 includes information about input from each input unit (specifically, each button, analog stick, and each sensor) (specifically, information about the operation and detection results from each sensor). In this embodiment, operation data is transmitted from the left controller 3 and / or the right controller 4 at predetermined intervals via wireless communication, and the operation data Da is updated as appropriate using the received operation data. The update cycle of the operation data Da may be updated every frame, which is the cycle of the processing executed by the game system 1 described later, or it may be updated every time the operation data is transmitted via wireless communication. In addition, operation data indicating that the touch panel 13 has been operated is acquired at each cycle of the above processing, and is stored in and updated in the operation data Da accordingly.

[0131] The angular velocity data Db is data indicating the angular velocity occurring in the main unit 2 as detected by the angular velocity sensor 90. For example, the angular velocity data Db includes data indicating the angular velocity around the x, y, and z axes occurring in the main unit 2.

[0132] The acceleration data Dc is data indicating the acceleration occurring in the main unit 2 as detected by the acceleration sensor 89. For example, the acceleration data Dc includes data indicating the acceleration in the x, y, and z axes occurring in the main unit 2.

[0133] Illuminance data Dd is data indicating the illuminance around the main unit 2 as detected by the illuminance sensor 29.

[0134] The posture data De represents the orientation of the main device 2 in real space. For example, posture data De includes data that represents the orientation used as the reference for the gravity vector, which represents the gravitational acceleration occurring on the main device 2, and data that represents the change in the orientation of the main device 2.

[0135] The operation object data Df is data that indicates the position, orientation, orientation, and movement of the object being manipulated by the user in the virtual space.

[0136] Virtual camera data Dg is data that indicates the position, orientation, field of view, magnification, etc., of the virtual cameras set up in the virtual space (a pair of left and right virtual cameras in stereoscopic display mode, and a single virtual camera in non-stereoscopic display mode).

[0137] The virtual spatial image data Dh for the left eye is data used to generate the left-eye image IML in stereoscopic display mode. The virtual spatial image data Di for the right eye is data used to generate the right-eye image IMR in stereoscopic display mode. The stereoscopic UI image data Dj is data that indicates the position, shape, size, etc., of the user interface image IMUa in stereoscopic display mode.

[0138] The non-stereoscopic virtual space image data Dk is data for generating a single image IMS in non-stereoscopic display mode. The non-stereoscopic UI image data Dm is data indicating the position, shape, size, etc., of the user interface image IMUb in non-stereoscopic display mode.

[0139] Image data Dn is data used to display images (for example, images of virtual objects, user interface images, information images, field images, background images, etc.) on the display screen during gameplay.

[0140] Next, with reference to Figure 18, a detailed example of information processing (game processing) in this embodiment will be described. Figure 18 is a flowchart showing an example of game processing executed by the game system 1. In this embodiment, the series of processes shown in Figure 18 are performed by the processor 81 executing communication programs and predetermined application programs (game programs) included in various programs Pa. Furthermore, the timing at which the game processing shown in Figure 18 begins is arbitrary.

[0141] The processing steps in the flowchart shown in Figure 18 are merely examples; the order of the steps can be changed, or other processing may be performed in addition to (or instead of) the processing of each step, as long as similar results can be obtained. Furthermore, in this embodiment, the processing of each step in the flowchart is described as being performed by the processor 81, but some of the processing steps in the flowchart may be performed by a processor other than the processor 81 or a dedicated circuit. In addition, some of the processing performed in the main unit 2 may be performed by other information processing devices that can communicate with the main unit 2 (for example, a server that can communicate with the main unit 2 via a network). In other words, each of the processes shown in Figure 18 may be performed by multiple information processing devices, including the main unit 2, working together.

[0142] In Figure 18, the processor 81 performs initial setup for game processing (step S200) and proceeds to the next step. For example, in the initial setup described above, the processor 81 initializes the parameters for the processing described below. As an example, the processor 81 uses the acceleration data stored in the acceleration data Dc to calculate the direction of the gravity vector of the gravitational acceleration acting on the main unit 2, sets the initial posture of the main unit 2 based on the direction of said gravity vector, and updates the posture data De. The processor 81 also sets the initial posture of the virtual camera in the virtual space and updates the virtual camera data Dg so that the direction of the gravity vector of the gravitational acceleration acting on the main unit 2 is the same as the relationship between the x, y, and z axes of the main unit 2. Here, being in the same direction as the x, y, and z axes of the main unit 2 means that the positive z-axis direction (screen depth direction) based on the direction of gravitational acceleration in real space is the same as the line of sight direction of the virtual camera based on the direction of gravity in the virtual space, and the positive x-axis direction (screen left direction) based on the direction of gravitational acceleration in real space is the same as the left direction of the virtual camera based on the direction of gravity in the virtual space.

[0143] Next, the processor 81 acquires various data and updates the operation data Da, angular velocity data Db, acceleration data Dc, and illuminance data Dd respectively (step S201), and proceeds to the next step. For example, the processor 81 acquires operation data from the left controller 3 and / or the right controller 4 and updates the operation data Da. The processor 81 also acquires touch operation data from the touch panel 13 and updates the operation data Da. Furthermore, the processor 81 acquires inertial data (accelerometer 89 and angular velocity sensor 90) provided in the main unit 2 and updates the acceleration data Dc and angular velocity data Db respectively. In addition, the processor 81 acquires illuminance data from the illuminance sensor 29 and updates the illuminance data Dd.

[0144] Next, the processor 81 calculates the attitude of the main unit 2 (step S202) and proceeds to the next step. For example, the processor 81 uses the acceleration data and angular velocity data stored in the angular velocity data Db and acceleration data Dc to calculate the direction of the gravity vector of the gravitational acceleration acting on the main unit 2 and updates the attitude data De. The processor 81 also calculates the rotation direction and amount of rotation of the main unit 2 from the initial attitude and updates the attitude data De. For example, the processor 81 calculates the rotation direction and amount of rotation of the main unit 2 with respect to the xyz axis directions in the initial attitude and updates the attitude data De. Note that the rotation direction can be expressed by the sign of the amount of rotation, so the attitude data De may only store data indicating the amount of rotation. For example, the processor 81 may add the amount of rotation based on the angular velocity data obtained in the current step S202 to the amount of rotation calculated in step S202 of the previous process and calculate a new amount of rotation.

[0145] Next, the processor 81 performs a process to determine whether the main unit 2 is attached to the goggle device 150 (step S203), and proceeds to the next step. For example, the processor 81 determines in step S203 that the main unit 2 is attached to the goggle device 150 if the illuminance indicated by the illuminance data Dd is a dim illuminance less than the threshold for detecting the illuminance when the main unit 2 is attached to the goggle device 150. The detection means performs a process to detect whether the display device is attached to the goggle device or is in the process of being attached, and as an example, corresponds to the processor 81 that performs the process in step S203.

[0146] Next, the processor 81 determines whether or not it is in stereoscopic display mode (step S204). For example, if the processor 81 determines in the attachment determination process in step S203 that the main unit 2 is attached to the goggle device 150, it makes an affirmative determination in step S204 and processes in stereoscopic display mode. On the other hand, if the processor 81 determines in the attachment determination process in step S203 that the main unit 2 is not attached to the goggle device 150, it makes a negative determination in step S204 and processes in non-stereoscopic display mode. If the processor 81 processes in stereoscopic display mode, it proceeds to step S205. On the other hand, if the processor 81 processes in non-stereoscopic display mode, it proceeds to step S213. The mode setting means performs the process of setting the display device to a first display mode or a second display mode different from the first display mode, and as an example, corresponds to the processor 81 that performs the processing in step S204.

[0147] In step S205, the processor 81 performs object motion processing and proceeds to the next step. For example, the processor 81 refers to the operation data Da and, if an operation is performed to move an operation object in the virtual space, sets the movement of the operation object according to that operation. Then, based on the set movement of the operation object, the processor 81 sets the position, direction, orientation, and motion of the operation object in the virtual space and updates the operation object data Df.

[0148] In the object motion processing in step S205 described above, the following object motion control methods are possible. As a first example, a predetermined operation object is used as the target of the operation, and the operation object is made to move. In this case, the predetermined operation object is moved, moved, or deformed based on the inputs to the input terminals of the left controller 3 and / or the right controller 4. As a second example, an operation object is selected as the target of the operation based on an operation, and then the operation object is made to move. In this case, an operation object placed at a predetermined display position (for example, an operation object that overlaps with a sign displayed in the center of the display screen) is used as the target of the operation, and the operation object selected as the target of the operation is moved, moved, or deformed based on the inputs to the input terminals of the left controller 3 and / or the right controller 4. As a third example, the operation object that has become the target of the operation is made to move based on vibrations transmitted to the goggle device 150 to which the main unit device 2 is attached. For example, if the acceleration data Dc indicates that the acceleration component of gravity is being applied to the main device 2, and the resulting acceleration indicates that vibrations of a predetermined magnitude or greater are being applied to the main device 2, the target object is operated in accordance with those vibrations. Any method can be used to extract the acceleration component of gravity; for example, the acceleration component occurring on average in the main device 2 may be calculated and this acceleration component may be extracted as the acceleration component of gravity.

[0149] Next, the processor 81 performs the process of operating the pair of left and right virtual cameras (step S206) and proceeds to the next step. For example, the processor 81 sets the orientation of the pair of left and right virtual cameras in the virtual space by rotating them from their initial orientation by the amount of rotation calculated in step S202, and updates the virtual camera data Dg. For example, with the relative positions of the pair of left and right virtual cameras fixed from their initial orientation, the processor 81 sets the orientation of the pair of left and right virtual cameras in the virtual space by rotating the virtual camera around its left-right axis by the same amount of rotation around the left-right axis (x-axis) of the main unit 2 calculated in step S202, rotating the virtual camera around its up-down axis by the same amount of rotation around the up-down axis (y-axis) of the main unit 2 calculated in step S202, and rotating the virtual camera around its line-of-view axis by the same amount of rotation around the screen depth axis (z-axis) of the main unit 2 calculated in step S202.

[0150] Next, the processor 81 performs the process of generating a virtual space image for the left eye (step S207) and proceeds to the next step. For example, the processor 81 places an operation object in the virtual space based on the operation object data Df. Then, the processor 81 generates a virtual space image for the left eye as the image viewed from the left virtual camera of the pair of left and right virtual cameras set by the virtual camera data Dg, and updates the virtual space image data Dh for the left eye.

[0151] Next, the processor 81 performs the process of generating a virtual space image for the right eye (step S208) and proceeds to the next step. For example, the processor 81 generates a virtual space image for the right eye as the image viewed from the right virtual camera of the pair of left and right virtual cameras set by the virtual camera data Dg, and updates the virtual space image data Di for the right eye.

[0152] Next, the processor 81 performs the process of generating a stereoscopic user interface image (step S209) and proceeds to the next step. For example, the processor 81 generates a stereoscopic user interface image that matches the shape of the third region of the display 12 (see Figure 9) and updates the stereoscopic UI image data Dj.

[0153] Next, the processor 81 performs display control processing to display the virtual space image for the left eye in the first area of ​​the display 12 (step S210), and then proceeds to the next step. For example, the processor 81 displays the virtual space image for the left eye, which is set as the virtual space image data for the left eye Dh, as the left eye image IML across the entire first area of ​​the display 12 (see Figure 9).

[0154] Next, the processor 81 performs display control processing to display the virtual space image for the right eye in the second area of ​​the display 12 (step S211), and then proceeds to the next step. For example, the processor 81 displays the virtual space image for the right eye, which is set in the virtual space image data for the right eye Di, as the right eye image IMR across the entire second area of ​​the display 12 (see Figure 9).

[0155] Next, the processor 81 performs display control processing to display the stereoscopic UI image in the third area of ​​the display 12 (step S212), and proceeds to step S219. For example, the processor 81 displays the user interface image set in the stereoscopic UI image data Dj as a user interface image IMUa (for example, two user interface images IMUa1 and IMUa2) at a predetermined position in the third area of ​​the display 12 (see Figure 9). The second display control means performs processing to display a second image on the display screen in the second display mode, which includes a content image consisting of a left-eye image and a right-eye image with parallax between them, and a second user interface image corresponding to the first user interface image. This corresponds, for example, to the processor 81 that performs the processing in steps S205-S212.

[0156] On the other hand, if it is determined in step S204 that the display mode is not stereoscopic, in step S213 the processor 81 performs object motion processing and proceeds to the next step. For example, the processor 81 refers to the operation data Da and, if an operation is performed to move an operation object in the virtual space, sets the movement of the operation object according to that operation. Then, based on the set movement of the operation object, the processor 81 sets the position, direction, orientation, and movement of the operation object in the virtual space and updates the operation object data Df. Note that the object motion processing in step S213 is the same as the object motion processing in step S205 described above, so a detailed explanation is omitted.

[0157] Next, the processor 81 performs the process of operating a single virtual camera (step S214) and proceeds to the next step. For example, the processor 81 sets the orientation of the single virtual camera in the virtual space by rotating it from the initial orientation by the amount of rotation calculated in step S202, and updates the virtual camera data Dg. For example, the processor 81 sets the orientation of the single virtual camera in the virtual space by rotating the single virtual camera from the initial orientation by the same amount of rotation around the left-right axis (x-axis) of the main unit 2 calculated in step S202, rotating the virtual camera up-down by the same amount of rotation around the up-down axis (y-axis) of the main unit 2 calculated in step S202, and rotating the virtual camera's line-of-view direction by the same amount of rotation around the screen depth axis (z-axis) of the main unit 2 calculated in step S202.

[0158] Next, the processor 81 performs the process of generating a virtual space image (step S215) and proceeds to the next step. For example, the processor 81 generates a virtual space image as seen from the virtual camera set by the virtual camera data Dg and updates the non-stereoscopic virtual space image data Dk.

[0159] Next, the processor 81 performs the process of generating a non-stereoscopic user interface image (step S216) and proceeds to the next step. For example, the processor 81 generates a non-stereoscopic user interface image that is superimposed on a single image IMS and displayed on the display 12 (see Figure 16), and updates the non-stereoscopic UI image data Dm.

[0160] Next, the processor 81 performs display control processing to display the virtual space image across the entire area of ​​the display 12 (step S217), and then proceeds to the next step. For example, the processor 81 displays the virtual space image set in the non-stereoscopic virtual space image data Dk as a single image IMS across the entire area of ​​the display 12 (see Figure 16).

[0161] Next, the processor 81 performs display control processing to superimpose the non-stereoscopic UI image onto a single image IMS (step S218), and proceeds to step S219. For example, the processor 81 superimposes the user interface image set in the non-stereoscopic UI image data Dm as a user interface image IMUb (for example, two user interface images IMUb1 and IMUb2) onto the single image IMS in the upper left and upper right corner areas of the display 12 (see Figure 16). The first display control means performs processing to display a first image on the display screen in the first display mode, which includes a content image that is a non-stereoscopic image and a first user interface image. This corresponds, for example, to the processor 81 that performs the processing in steps S213-S218.

[0162] In step S219, the processor 81 performs user interface operation processing and proceeds to the next step. For example, the processor 81 refers to the operation data Da and, if a touch operation is performed on the touch panel 13, sets a user operation instruction corresponding to the touch operation, according to the user interface image displayed on the display 12 that overlaps with the location of the touch operation. Then, the processor 81 performs processing according to the set user operation instruction.

[0163] Next, the processor 81 determines whether or not to terminate the game (step S220). Conditions for terminating the game in step S220 include, for example, the game result being determined or the user performing an operation to terminate the game. If the game is not terminated, the processor 81 returns to step S201 and repeats the process; if the game is terminated, it terminates the process according to the flowchart. From thereafter, the series of processes from steps S201 to S220 are repeatedly executed until it is determined in step S220 that the game should be terminated.

[0164] Thus, in this embodiment, when switching between the normal display mode (non-stereoscopic display mode) and the stereoscopic display mode, the user interface image is displayed in a different position depending on the set display mode, thereby improving the convenience of presenting the user interface image. Furthermore, since the display mode is switched automatically based on the mounting state of the main unit 2 to the goggle device 150, seamless switching of the display mode is possible. Moreover, in this embodiment, in the stereoscopic display mode, the user interface image is displayed in a third area different from the first and second areas of the display 12 where the right eye image and the left eye image are displayed, respectively, so that the user interface image does not interfere with the display of the stereoscopic image.

[0165] In the above-described embodiment, the image display system is configured by attaching a main unit 2 having an information processing function to the goggle device 150, but the image display system may be configured in other ways. As a first example, an image display system that displays stereoscopic images may be configured by providing a control unit that performs the above-mentioned information processing (game processing) and generates images in the goggle device 150, and attaching a display device having the function of displaying images to the goggle device 150. In this case, the control unit provided in the goggle device 150 outputs image data to the display device for displaying stereoscopic images and user interface images to the display device, thereby displaying stereoscopic images and user interface images on the display device. In the embodiment of the first example above, the mechanism for detecting the mounting status of the display device, the mechanism for detecting the orientation of the image display system, the mechanism for receiving user operations, etc., may be provided in either the display device or the goggle device 150. As a second example, a control device may be provided separately from the goggle device 150 to which the display device is attached, and the display device, the goggle device 150, and the control device may be configured to form the image display system. In this case, operation data, acceleration data, angular velocity data, and illuminance data are output from the display device to the control device, and content images and user interface images of the display mode based on the illuminance data are output from the control device to the display device. In the embodiment of the second example described above, the mechanism for detecting the mounting status of the display device, the mechanism for detecting the orientation of the image display system, and the mechanism for receiving user operations may be provided in either the display device or the goggle device 150. The control device then controls the display range of the content image to be displayed according to the operation data, acceleration data, and angular velocity data acquired from the display device and / or the goggle device 150, and outputs it to the display device. In this embodiment, the main unit 2 is used as an example of a display device.

[0166] In the above-described embodiment, the method for detecting the orientation of the main unit 2 is merely an example, and the orientation of the main unit 2 may be detected using other methods or other data. For example, the main unit 2 and / or the goggle device 150 on which the main unit 2 is attached may be imaged from the outside, and the orientation of the main unit 2 and / or the goggle device 150 may be detected using the image captured. Furthermore, the controller for controlling the operation of the manipulated object may be other controllers, not just the left controller 3 or the right controller 4.

[0167] Furthermore, the game system 1 and / or main unit 2 may be any device, including a portable game device, any portable electronic device (such as a PDA (Personal Digital Assistant), mobile phone, personal computer, camera, tablet, etc.).

[0168] Furthermore, in the above-described embodiment, a stereoscopic image was displayed by showing left-eye and right-eye images, which have parallax with respect to each other, on the left and right screens of the display 12, respectively. However, the left-eye and right-eye images may be displayed on separate screens. For example, if the display 12 provided on the main unit 2 is composed of multiple display screens, the left-eye image may be displayed on one of the multiple display screens, and the right-eye image on another of the multiple display screens. In this case, the user can view the stereoscopic image through the goggle device 150 by viewing the left-eye image displayed on one of the multiple display screens with their left eye via the left-eye lens 153L, and by viewing the right-eye image displayed on another of the multiple display screens with their right eye via the right-eye lens 153R.

[0169] Furthermore, the stereoscopic and non-stereoscopic images displayed on the main unit 2 are displayed as game images when the processor 81 performs information processing (game processing) in response to user operation, or as moving images and still images when the processor 81 performs video playback or still image playback in response to user operation. In other words, the stereoscopic and non-stereoscopic images displayed on the main unit 2 are generated by the processor 81 of the main unit 2 performing information processing (e.g., game processing, video playback processing, still image playback processing), but at least a part of the processing that generates these stereoscopic and non-stereoscopic images may be performed by other devices. For example, if the main unit 2 is configured to communicate with other devices (e.g., a server, another image display device, another game device, another mobile terminal, another information processing device), the above processing may be performed by the cooperation of these other devices. In this way, by having at least a part of the above processing performed by other devices, processing similar to the above processing becomes possible. Furthermore, the above information processing can be performed by the cooperation of one processor or multiple processors included in an information processing system composed of at least one information processing device. Furthermore, in the above embodiment, information processing can be performed by the processor 81 of the main unit 2 executing a predetermined program, but some or all of the above processing may be performed by a dedicated circuit provided in the main unit 2.

[0170] As described above, the invention can be realized in so-called cloud computing system configurations, distributed wide-area networks, and local network system configurations. For example, in a distributed local network system configuration, the above processing can be performed collaboratively between a stationary information processing device (stationary game device) and a portable information processing device (portable game device). It goes without saying that in these system configurations, there are no particular limitations on which device performs the above processing, and the invention can be realized regardless of how the processing is divided.

[0171] Furthermore, the processing order, set values, and conditions used in the information processing described above are merely examples, and it goes without saying that this embodiment can be realized even with other orders, values, and conditions.

[0172] Furthermore, the above program may be supplied to the game system 1 not only through an external storage medium such as external memory, but also to the device via a wired or wireless communication line. The program may also be pre-recorded in a non-volatile storage device inside the device. The information storage medium for storing the program may be a CD-ROM, DVD, or similar optical disc-type storage medium, a flexible disk, a hard disk, a magneto-optical disk, a magnetic tape, etc. Alternatively, the information storage medium for storing the program may be a volatile memory for storing the program. Such storage media can be described as recording media that can be read by a computer or the like. For example, by having a computer or the like read and execute the program on these recording media, the various functions described above can be provided.

[0173] Although the present invention has been described in detail above, the above description is merely illustrative in all respects and is not intended to limit its scope. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention. It is understood that the scope of the present invention should be interpreted solely by the claims. Furthermore, it is understood that those skilled in the art can implement an equivalent scope based on the description of the specific embodiments of the present invention and common technical knowledge. Furthermore, it should be understood that terms used herein are used in the sense commonly used in the art unless otherwise specified. Accordingly, unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. In case of any conflict, this specification (including definitions) shall prevail. [Industrial applicability]

[0174] As described above, the present invention can be used as an image display system, image display program, image display method, and display device, etc., which can improve the convenience of presenting user interface images. [Explanation of symbols]

[0175] 1…Game System 2…Main unit 3…Left controller 4…Right controller 11… Housing 12…Display 29... Illuminance sensor 81… Processor 83…Controller Communication Unit 85…DRAM 150... Goggle device 151...Main unit 152...Lens frame component 153... Lens 154…Plate-shaped member

Claims

1. An image display system comprising a display device having a display screen for displaying images and a goggle-type device to which the display device can be attached, Mode setting means for setting the display device to a first display mode or a second display mode different from the first display mode, In the first display mode, a first display control means causes the display screen to display a first image including a content image which is a non-stereoscopic image and a first user interface image, In the second display mode, a second display control means causes the display screen to display a second image which includes a content image consisting of a left-eye image and a right-eye image having parallax with respect to each other, and a second user interface image which is a non-stereoscopic image that is at least different in position from the first user interface image. The device includes a detection means for detecting whether the display device is in the process of being attached to the goggle-type device, The mode setting means, when the display device is set to the first display mode and the detection means detects that it is in the installation in progress, sets the display device from the first display mode to the second display mode. The second display control means is an image display system that, in the second display mode, displays the second user interface image in a third area of ​​the display screen, excluding the area composed of a first area for displaying the left eye image and a second area for displaying the right eye image on the display screen.

2. The image display system according to claim 1, wherein the detection means detects whether the display device is in the process of being attached to the goggle-type device based on the fact that a predetermined operation has been performed.

3. The display device is equipped with an illuminance sensor, The goggle-type device includes a light-shielding section that blocks light from the display device to the illuminance sensor when the display device is in the process of being attached to the goggle-type device. The image display system according to claim 1, wherein the detection means detects whether the display device is in the process of being attached to the goggle-type device based on the detection result from the illuminance sensor.

4. The image display system according to any one of claims 1 to 3, wherein the second display control means causes the display screen to display an image on which the content image is displayed as a non-stereoscopic image by the first display control means, as a content image consisting of a left-eye image and a right-eye image.

5. The image display system according to any one of claims 1 to 4, wherein the second display control means causes the content image that was displayed in the first display mode immediately before being set to the second display mode to be converted into a stereoscopic image consisting of a left-eye image and a right-eye image and displayed on the display screen.

6. The image display system according to any one of claims 1 to 5, wherein the second display control means causes a user interface image that is substantially identical in function to the first user interface image but differs in shape to be displayed as the second user interface image.

7. The image display system according to any one of claims 1 to 6, wherein the second display control means sets the third region below the display screen, sandwiched between the first region and the second region on the display screen.

8. The image display system according to any one of claims 1 to 7, wherein the goggle-type device has an opening that exposes at least the third region, which is a part of the display screen, to the outside when the display device is attached to the goggle-type device.

9. The image display system according to any one of claims 1 to 8, wherein the first display control means displays the first user interface image superimposed on the content image displayed on the display screen in the first display mode.

10. The display device further comprises a display device side connection portion that can be electrically connected to other devices, The goggle-type device includes a goggle-type device side connection part that can be electrically connected to the display device side connection part, The image display system according to claim 1, wherein the detection means detects whether the display device is in the process of being attached to the goggle-type device, in response to the connection between the display device side connection and the goggle-type device side connection.

11. An image display program that runs on a computer included in a display device that has a display screen for displaying images and can be attached to a goggle-type device, The aforementioned computer, Mode setting means for setting the display device to a first display mode or a second display mode different from the first display mode, In the first display mode, a first display control means causes the display screen to display a first image including a content image which is a non-stereoscopic image and a first user interface image, In the second display mode, a second display control means causes the display screen to display a second image which includes a content image consisting of a left-eye image and a right-eye image having parallax with respect to each other, and a second user interface image which is a non-stereoscopic image that is at least different in position from the first user interface image. The display device is configured to function as a detection means for detecting whether or not it is in the process of being attached to the goggle-type device. The mode setting means, when the display device is set to the first display mode and the detection means detects that it is in the installation in progress, sets the display device from the first display mode to the second display mode. The second display control means is an image display program that displays the second user interface image in the second display mode within a third area of ​​the display screen, excluding the area composed of a first area for displaying the left eye image and a second area for displaying the right eye image.

12. An image display method that has a display screen for displaying images and displays images on a display device that can be attached to a goggle-type device, A mode setting step of setting the display device to a first display mode or a second display mode different from the first display mode, In the first display mode, a first display control step is performed to cause the display screen to display a first image including a content image which is a non-stereoscopic image and a first user interface image, A second display control step in which the display screen is made to display a second image in the second display mode, which includes a content image consisting of a left-eye image and a right-eye image having parallax with respect to each other, and a second user interface image which is a non-stereoscopic image that is at least different in position from the first user interface image, The detection step includes detecting whether the display device is in the process of being attached to the goggle-type device, In the mode setting step, if the display device is set to the first display mode, and in the detection step it is detected that the device is in the installation in progress, the display device is set from the first display mode to the second display mode. An image display method in which, in the second display control step, the second user interface image is displayed in a third area of ​​the display screen, excluding the area composed of a first area for displaying the left eye image and a second area for displaying the right eye image on the display screen.

13. A display device having a screen for displaying images and that can be attached to a goggle-type device, Mode setting means for setting the display device to a first display mode or a second display mode different from the first display mode, In the first display mode, a first display control means causes the display screen to display a first image including a content image which is a non-stereoscopic image and a first user interface image, In the second display mode, a second display control means causes the display screen to display a second image which includes a content image consisting of a left-eye image and a right-eye image having parallax with respect to each other, and a second user interface image which is a non-stereoscopic image that is at least different in position from the first user interface image. The device includes a detection means for detecting whether the display device is in the process of being attached to the goggle-type device, The mode setting means, when the display device is set to the first display mode and the detection means detects that it is in the installation in progress, sets the display device from the first display mode to the second display mode. The second display control means is a display device that, in the second display mode, displays the second user interface image in a third area of ​​the display screen, excluding the area composed of a first area for displaying the left eye image and a second area for displaying the right eye image on the display screen.

Images