Information processing device and image generating method

Abstract

To provide a technology that realizes display control suitable for head-mounted displays. [Solution] A movement detection unit 30 detects the orientation of a head-mounted display worn on the user's head, and a gaze direction determination unit 32 determines the gaze direction according to the detected orientation of the head-mounted display. An image generation unit 34 generates an image based on the determined gaze direction, and an image provision unit 36 ​​provides the generated image to the head-mounted display. An instruction acquisition unit 24 acquires an instruction to switch the gaze direction from an input device 6. When the instruction acquisition unit 24 acquires the switching instruction, the gaze direction determination unit 32 changes the gaze direction by a predetermined angle.

Description

[Technical field]

[0001] The present invention relates to a technique for generating an image to be displayed on a head mounted display. [Background technology]

[0002] A head-mounted display (HMD) is worn on the user's head and provides the user with a virtual reality (VR) world. Recently, applications have been introduced that allow users to play games while viewing the screen displayed on the HMD. With conventional stationary displays such as televisions, the user's field of view extends beyond the screen, making it difficult for the user to concentrate on the screen or to feel immersed in the game. On the other hand, when wearing an HMD, the user only sees the image displayed on the HMD, which increases the sense of immersion in the image world and further enhances the entertainment value of the game. If the HMD is equipped with a head tracking function and the display screen is updated in conjunction with the user's head position, the sense of immersion in the image world can be further improved.

[0003] In recent years, omnidirectional cameras that take 360-degree panoramic photos in all directions have become popular. In addition, remotely controlled unmanned flying objects are also being developed, and by equipping such flying objects with multiple cameras, it is possible to take panoramic photos in all directions from the air. It is expected that the omnidirectional panoramic images taken in this way will be displayed on an HMD, and the display screen will be updated in conjunction with the user's head position using a head tracking function, giving the user the feeling that they are in the real place. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-95045 A Summary of the Invention [Problem to be solved by the invention]

[0005] As a recent trend to enhance the realism of images, there has been active development of HMDs with wider viewing angles. HMDs with viewing angles of over 80 degrees are already on the market, and HMDs with a wide viewing angle provide users with an image world that is comparable to the real world.

[0006] The inventor of the present invention has tried various image displays in a wide-angle HMD and has recognized that depending on how the image is displayed, the user may experience a feeling of motion sickness. In this specification, this feeling is called "motion sickness (simulator sickness)". Even if the same image does not cause motion sickness when viewed on a television screen, motion sickness may occur on the screen of an HMD. The inventor has also recognized that when an information element such as a menu is included in a displayed image, depending on how the information element is presented, it may cause a sense of discomfort to the user. Based on the knowledge gained from various trials, the inventor has come up with a display control suitable for an HMD.

[0007] The present invention has been made in view of these problems, and has a general purpose to provide a technique for realizing display control suitable for an HMD. [Means for solving the problem]

[0008] In order to solve the above problems, an information processing device according to an embodiment of the present invention includes a detection unit that detects the attitude of a device from a captured image of the device and / or sensor information detected by an attitude sensor of the device, a gaze direction determination unit that determines a gaze direction according to the detected attitude of the device, an image generation unit that generates an image based on the determined gaze direction, and an instruction acquisition unit that acquires an instruction to switch the gaze direction. When the instruction acquisition unit acquires the switching instruction, the gaze direction determination unit discontinuously changes the gaze direction or changes the gaze direction by a predetermined angle.

[0009] Another aspect of the image generating method of the present invention is an image generating method comprising the steps of detecting the attitude of a device from a captured image of the device and / or sensor information detected by an attitude sensor of the device, determining a gaze direction based on the detected attitude of the device, generating an image based on the determined gaze direction, and obtaining an instruction to switch the gaze direction, wherein the step of determining the gaze direction discontinuously changes the gaze direction or changes the gaze direction by a predetermined angle when the switching instruction is obtained.

[0010] In addition, any combination of the above components, and conversions between expressions of the present invention such as methods, devices, systems, computer programs, recording media on which computer programs are readably recorded, and data structures are also valid aspects of the present invention. Effect of the Invention

[0011] According to the present invention, it is possible to provide a technique for achieving display control suitable for an HMD. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of a rendering process in the information processing device. [Diagram 3] FIG. 2 is a diagram showing an example of the external shape of an HMD. [Figure 4] FIG. 2 is a diagram showing functional blocks of an HMD. [Diagram 5] FIG. 2 is a diagram illustrating functional blocks of the information processing device. [Figure 6] 4 is a diagram for explaining panoramic image data stored in an image storage unit; FIG. [Figure 7] FIG. [Figure 8] FIG. [Figure 9] FIG. [Figure 10] 2A and 2B are diagrams showing the top and rear side surfaces of the input device. [Figure 11] FIG. [Figure 12] 13A and 13B are diagrams showing display images before and after switching; [Figure 13] FIG. 13 is a diagram showing an example in which information elements are superimposed on a display image. [Figure 14] FIG. 13 shows how information elements move with a panoramic image. [Figure 15] FIG. 13 is a diagram showing information elements superimposed on a panoramic image. [Figure 16] FIG. 13 is a diagram showing information elements superimposed on a panoramic image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] 1 shows an example of the configuration of an information processing system 1 according to an embodiment. The information processing system 1 includes an information processing device 10, a head mounted display device (HMD) 100 worn on the head of a user, an input device 6 operated by the user with the user's fingers, an imaging device 7 that captures an image of the user wearing the HMD 100, and an output device 4 that displays an image.

[0014] In the embodiment, the information processing device 10 includes a processing device 12 and an output control device 14. Here, the processing device 12 is a terminal device that receives operation information of an input device 6 operated by a user and executes various applications such as games. The processing device 12 and the input device 6 may be connected by a cable or by a known wireless communication technology. The output control device 14 is a processing unit that outputs image data to the HMD 100. The output control device 14 and the HMD 100 may be connected by a cable or by a known wireless communication technology.

[0015] The imaging device 7 captures an image of the user wearing the HMD 100 and provides the captured image to the processing device 12. The imaging device 7 may be a stereo camera. As will be described later, the HMD 100 is equipped with a marker (tracking LED) for tracking the user's head, and the processing device 12 detects the movement of the HMD 100 based on the position of the captured marker. The HMD 100 is also equipped with an attitude sensor (acceleration sensor, gyro sensor), and the processing device 12 acquires sensor information detected by the attitude sensor from the HMD 100, thereby achieving high-precision tracking processing in conjunction with the use of the captured image of the marker.

[0016] In the information processing system 1, since the user views an image displayed on the HMD 100, the output device 4 is not necessarily required, but the output control device 14 or the processing device 12 may output the same image as the image displayed on the HMD 100 from the output device 4. This allows another user to view on the output device 4 the image that the user is viewing on the HMD 100. As will be described later, since the image displayed on the HMD 100 has been subjected to optical lens distortion correction, it is necessary to output an image that has not been subjected to distortion correction from the output device 4.

[0017] In the information processing system 1, the processing device 12, the output device 4, the input device 6, and the imaging device 7 may constitute a conventional game system. In this case, the processing device 12 may be a game device that executes an application such as a game, and the input device 6 may be a device that supplies user operation information to the processing device 12, such as a game controller, a keyboard, a mouse, or a joystick. By adding the output control device 14 and the HMD 100 to the components of this game system, the information processing system 1 that executes a virtual reality (VR) application is constituted.

[0018] The function of the output control device 14 may be incorporated into the processing device 12 as a part of the function of the VR application. In other words, the information processing device 10 may be composed of one processing device 12, or may be composed of the processing device 12 and the output control device 14. In the following, the functions of the processing device 12 and the output control device 14 necessary for realizing the VR application will be collectively described as the functions of the information processing device 10.

[0019] The information processing device 10 generates image data to be displayed on the HMD 100. In the embodiment, the information processing device 10 prepares a 360-degree panoramic image in all directions, up, down, left, and right, captured by an omnidirectional camera, and displays the image on the HMD 100, generated based on the line of sight determined from the posture of the HMD 100 worn on the user's head. The display content may be either a still image or a video. In addition, it is not limited to an image that has been actually captured, and may be an image that has been drawn in real time by a game application.

[0020] The HMD 100 is a display device that displays an image through an optical lens on a display panel located in front of the user's eyes when worn on the user's head. The HMD 100 independently displays an image for the left eye on the left half of the display panel, and an image for the right eye on the right half. These images form parallax images seen from left and right viewpoints, and the images can be viewed stereoscopically by displaying the images in the left and right areas obtained by dividing the display panel into two. Note that, in order for the user to view the display panel through an optical lens, the information processing device 10 supplies the HMD 100 with image data in which optical distortion caused by the lens has been corrected in advance. In the information processing device 10, the optical distortion correction process may be performed by either the processing device 12 or the output control device 14.

[0021] FIG. 2 is an explanatory diagram of the rendering process in the information processing device 10. In the VR application of the embodiment, a virtual environment is realized in which the user is located at the center of a sphere and the visible image changes as the user changes the direction of the line of sight. A content image, which is an image material, is pasted on the inner circumference of a virtual sphere centered on a central point 9 where the user is located. Here, the content image is a 360-degree panoramic image in all directions, up, down, left, and right, taken by an omnidirectional camera, and is pasted on the inner circumference of the virtual sphere so that the top and bottom of the content image and the top and bottom of the virtual sphere match. This aligns the top and bottom of the user's real world with the top and bottom of the image world provided to the HMD 100, and a VR application that reproduces a realistic image world is realized.

[0022] The information processing device 10 detects the rotation angle and inclination of the user's head (actually, the HMD 100) by performing head tracking processing of the user. Here, the rotation angle of the HMD 100 is the rotation angle with respect to a reference direction on a horizontal plane, and the reference direction may be set as the direction in which the HMD 100 faces when the power of the HMD 100 is turned on, for example. The inclination of the HMD 100 is the inclination angle with respect to the horizontal plane. A known technology may be used as the head tracking processing, and the information processing device 10 can detect the rotation angle and inclination of the HMD 100 only from sensor information detected by the attitude sensor of the HMD 100, and further detect the rotation angle and inclination of the HMD 100 with high accuracy by performing image analysis of the marker (tracking LED) of the HMD 100 captured by the imaging device 7.

[0023] The information processing device 10 determines the attitude of the virtual camera 8 in the virtual sphere according to the detected rotation angle and inclination of the HMD 100. The virtual camera 8 is disposed so as to capture an image of the inner circumferential surface of the virtual sphere from the center point 9 of the virtual sphere, and the information processing device 10 matches the detected rotation angle and inclination with the rotation angle and inclination of the optical axis of the virtual camera 8 in the virtual sphere. The information processing device 10 acquires the captured image 5 of the virtual camera 8, that is, performs rendering processing, performs optical distortion correction for the optical lens, and supplies image data to the HMD 100. Note that one virtual camera 8 is shown in FIG. 2, but in reality, two virtual cameras 8 for the left eye and the right eye are disposed, and image data for each is generated.

[0024] 3 shows an example of the external shape of the HMD 100. In this example, the HMD 100 is composed of an output mechanism unit 102 and a wearing mechanism unit 104. The wearing mechanism unit 104 includes a wearing band 106 that goes around the head when worn by the user to secure the HMD 100 to the head. The wearing band 106 is made of a material or has a structure that allows the length to be adjusted to fit the user's head circumference.

[0025] The output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the user wears the HMD 100, and includes a display panel inside at a position directly facing the eyes when the user wears the HMD 100. The display panel is realized by a liquid crystal panel, an organic EL panel, or the like. The housing 108 also includes a pair of left and right optical lenses that are positioned between the display panel and the user's eyes when the HMD 100 is worn, and that expand the user's viewing angle. The HMD 100 may further include speakers or earphones at positions corresponding to the user's ears when the HMD 100 is worn.

[0026] The outer surface of the housing 108 is provided with light-emitting markers 110a, 110b, 110c, and 110d. In this example, the light-emitting markers 110 are constituted by tracking LEDs, but other types of markers may be used as long as they are captured by the imaging device 7 and the information processing device 10 can perform image analysis. The number and arrangement of the light-emitting markers 110 are not particularly limited, but they need to be so numbered and arranged that the posture (rotation angle and inclination) of the HMD 100 can be detected by capturing the image and analyzing it, and in the example shown in the figure, the light-emitting markers 110 are provided at the four corners of the front of the housing 108. Furthermore, the light-emitting markers 110 may be provided on the side or rear of the wearing band 106 so that they can be captured even when the user turns their back to the imaging device 7.

[0027] The HMD 100 may be connected to the information processing device 10 by a cable or by a known wireless communication technology. The HMD 100 transmits sensor information detected by a posture sensor to the information processing device 10, and also receives image data generated by the information processing device 10 and displays it on a display panel.

[0028] 3 shows an immersive (non-transmissive) display device that completely covers both eyes, but it may be a transmissive display device. The shape may be a hat-type as shown in the figure, or it may be a glasses-type.

[0029] 4 shows functional blocks of the HMD 100. The control unit 120 is a main processor that processes and outputs various data such as image data, audio data, and sensor information, as well as commands. The storage unit 122 temporarily stores data and commands processed by the control unit 120. The attitude sensor 124 detects attitude information such as the rotation angle and inclination of the HMD 100. The attitude sensor 124 includes at least a three-axis acceleration sensor and a three-axis gyro sensor. The microphone 126 converts the user's voice into an electrical signal. The light-emitting marker 110 is an LED, and multiple LEDs are attached to the wearing band 106 or the housing 108 of the HMD 100.

[0030] The communication control unit 128 transmits data input from the control unit 120 to the external information processing device 10 by wired or wireless communication via a network adapter or an antenna. The communication control unit 128 also receives data from the information processing device 10 by wired or wireless communication via a network adapter or an antenna, and outputs the data to the control unit 120.

[0031] When the control unit 120 receives image data or audio data from the information processing device 10, it supplies the data to the display panel 130 for display, and also supplies the data to the audio output unit 132 for audio output. The control unit 120 also causes the communication control unit 128 to transmit sensor information from the attitude sensor 124 and audio data from the microphone 126 to the information processing device 10.

[0032] 5 shows functional blocks of the information processing device 10. The information processing device 10 includes a sensor information acquisition unit 20, a captured image acquisition unit 22, and an instruction acquisition unit 24 as input interfaces with the outside. The sensor information acquisition unit 20 acquires sensor information from the attitude sensor 124 of the HMD 100 at a predetermined cycle. The captured image acquisition unit 22 acquires captured images of the HMD 100 from the imaging device 7 at a predetermined cycle. For example, the imaging device 7 captures images every (1 / 60) seconds, and the captured image acquisition unit 22 acquires captured images every (1 / 60) seconds. The instruction acquisition unit 24 acquires instructions input by the user from the input device 6.

[0033] The information processing device 10 further includes a motion detection unit 30, a gaze direction determination unit 32, an image generation unit 34, and an image provision unit 36. The motion detection unit 30 detects the posture of the HMD 100 worn on the user's head. The gaze direction determination unit 32 determines the gaze direction according to the posture of the HMD 100 detected by the motion detection unit 30. The image generation unit 34 generates an image according to the detected posture of the HMD 100, specifically, generates an image based on the gaze direction determined by the gaze direction determination unit 32. The image provision unit 36 ​​provides the generated image to the HMD 100.

[0034] 5, each element described as a functional block performing various processes can be configured in hardware as a circuit block, memory, or other LSI, and in software as a program loaded into memory, etc. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware alone, software alone, or a combination thereof, and are not limited to any one of them.

[0035] The image storage unit 40 stores 360-degree panoramic image data captured in advance. The image storage unit 40 may store a plurality of content images, and the content images may be still images or moving images. The image storage unit 40 of the embodiment stores panoramic image data in all directions, up, down, left, and right, and the information processing device 10 provides an omnidirectional panoramic image to a user wearing the HMD 100. Therefore, when the user turns his / her head left or right (the user may turn his / her whole body left or right) and rotates the horizontal line of sight left or right, a panoramic image in the left or right direction is displayed on the display panel 130 of the HMD 100, and when the user tilts his / her head up or down and tilts the line of sight vertically, a panoramic image in the upward or downward direction is displayed on the display panel 130 of the HMD 100.

[0036] The information element storage unit 42 stores information elements to be included in the panoramic image. For example, the information elements may be menu items for changing the brightness of the content image, or menu items for selecting the content image that is an image material. When the user inputs an instruction to display the information elements to the input device 6, the image generating unit 34 includes the information elements in the image to be generated, and the information elements superimposed on the panoramic image are displayed on the display panel 130 of the HMD 100. Note that the information elements may be notification information for the user, and in this case, the information elements may be superimposed on the panoramic image without the user operating the input device 6.

[0037] Fig. 6 is a diagram for explaining the panoramic image data stored in the image storage unit 40. For convenience of explanation, Fig. 6 shows a part of the panoramic image data in all directions, up, down, left and right, and omits a part of the image downward from the horizontal plane and a part of the image in the left and right directions. As described with reference to Fig. 2, an image obtained by rendering the panoramic image pasted on the inner peripheral surface of the virtual sphere is displayed on the display panel 130 of the HMD 100, and the user changes the line of sight by changing the rotation angle and inclination of the HMD 100, so that the displayed panoramic image moves according to the line of sight.

[0038] In the HMD 100, the communication control unit 128 transmits sensor information acquired by the orientation sensor 124 to the information processing device 10 at a predetermined cycle. Furthermore, the imaging device 7 captures images of the HMD 100 at a predetermined cycle and transmits the captured images to the information processing device 10. With reference to Fig. 5, the sensor information acquisition unit 20 acquires sensor information of the orientation sensor 124 and supplies it to the motion detection unit 30. Furthermore, the captured image acquisition unit 22 acquires the captured image and supplies it to the motion detection unit 30.

[0039] The motion detection unit 30 detects the orientation of the HMD 100, thereby performing a head tracking process to detect the orientation of the head of the user wearing the HMD 100. This head tracking process is performed to link the field of view displayed on the display panel 130 of the HMD 100 to the orientation of the user's head. In the head tracking process of the embodiment, the rotation angle of the HMD 100 with respect to a horizontal reference direction and the tilt angle with respect to a horizontal plane are detected. The horizontal reference direction may be set as the direction in which the HMD 100 faces when the power of the HMD 100 is turned on, for example.

[0040] This head tracking process uses a known method, and the motion detection unit 30 may detect the rotation angle of the HMD 100 with respect to the horizontal reference direction and the tilt angle with respect to the horizontal plane only from the sensor information of the orientation sensor 124, but it is preferable to further use the image capture results of the light-emitting marker 110 for tracking to improve detection accuracy. The motion detection unit 30 detects the rotation angle and tilt angle at a predetermined cycle. For example, if the images supplied to the HMD 100 are 60 fps, it is preferable that the detection process of the motion detection unit 30 is also performed at a cycle of (1 / 60) seconds.

[0041] The line of sight direction determination unit 32 determines the line of sight direction according to the attitude of the HMD 100 detected by the motion detection unit 30. This line of sight direction is the line of sight direction of the user, and is also the line of sight direction (optical axis direction) of the virtual camera 8 arranged at the center point 9 of the virtual sphere (see FIG. 2). Here, the line of sight direction determination unit 32 may determine the rotation angle and tilt angle detected by the motion detection unit 30 as the line of sight direction (optical axis direction) of the virtual camera 8 as is, or may determine the line of sight direction of the virtual camera 8 by performing some kind of correction processing. For example, when stable sensor information is not provided to the motion detection unit 30 due to noise being added to the sensor information, the motion detection unit 30 may detect a vibrating motion even though the user's head is not moving. In such a case, the line of sight direction determination unit 32 may determine the line of sight direction by smoothing and correcting the motion detected by the motion detection unit 30.

[0042] In addition, since the human visual field is asymmetrical from top to bottom, with the lower side being slightly wider than the upper side, the line of sight direction determining unit 32 may determine the line of sight direction of virtual camera 8 by slightly tilting the tilt angle detected by the motion detecting unit 30 downward.

[0043] The image generating unit 34 generates an image according to the posture of the HMD 100 detected by the motion detecting unit 30, specifically, generates an image based on the line of sight direction of the virtual camera 8 determined by the line of sight direction determining unit 32. The image generating unit 34 defines a field of view for the left eye and a field of view for the right eye specified by the line of sight direction, and generates images for the left eye and the right eye by rendering them. At this time, the image generating unit 34 generates a panoramic image in which distortion caused by image light from the display panel passing through an optical lens is corrected.

[0044] The HMD 100 of the embodiment provides the user with a field of view of approximately 100 degrees in the horizontal direction and approximately 100 degrees in the vertical direction. With reference to Fig. 2, the captured image 5 in the virtual sphere is captured at a field of view of approximately 100 degrees in the horizontal direction and approximately 100 degrees in the vertical direction, and is displayed on the display panel 130 of the HMD 100. As described above, since the human field of view is slightly wider below than above, the optical lens and the display panel 130 in the HMD 100 may be tilted by 5 degrees with respect to the direction directly facing the eyes, and a vertical field of view of 45 degrees above and 55 degrees below may be realized by the arrangement of the optical lens and the display panel 130.

[0045] 7 shows a display image 200a generated by the image generating unit 34. In the following drawings, the display image is represented as an image cut out from the panoramic image in order to facilitate understanding of the positional relationship of the display image within the entire panoramic image.

[0046] The image generating unit 34 generates an image based on the line of sight direction 202a determined by the line of sight direction determining unit 32. In reality, the image generating unit 34 generates a display image for the left eye and a display image for the right eye by rendering them, and these display images are different images including parallax, but the generation of images for both eyes will not be specifically described below. The image providing unit 36 ​​provides the display image 200a generated by the image generating unit 34 to the HMD 100. In the HMD 100, the control unit 120 causes the display image 200a to be displayed on the display panel 130, so that the user can see the display image 200a displayed on the display panel 130.

[0047] FIG. 8 shows a display image 200b generated by the image generation unit 34. The line of sight direction determination unit 32 determines the line of sight direction according to the posture of the HMD 100, and the image generation unit 34 generates an image based on the determined line of sight direction. In this example, the user turns his / her head counterclockwise in the horizontal direction, and the line of sight changes continuously from line of sight direction 202a to line of sight direction 202b. Here, the user turns his / her head counterclockwise by about 60 degrees, and this rotational movement causes the image generation unit 34 to generate an image that moves a panoramic image continuously in the counterclockwise direction from the display image 200a at a cycle of (1 / 60) seconds. The image provision unit 36 ​​provides the generated image to the HMD 100 at a cycle of (1 / 60) seconds.

[0048] FIG. 9 shows a display image 200c generated by the image generating unit 34. The line of sight direction determining unit 32 determines the line of sight direction according to the posture of the HMD 100, and the image generating unit 34 generates an image based on the determined line of sight direction. In this example, the user tilts his / her head upward from a state in which the display image 200a is displayed on the display panel 130, and the line of sight changes continuously from line of sight 202a to line of sight 202c. Here, the user tilts his / her head upward by about 30 degrees, and this tilting action causes the image generating unit 34 to generate an image that moves a panoramic image continuously in an upward direction from the display image 200a at a cycle of (1 / 60) seconds. The image providing unit 36 ​​provides the generated image to the HMD 100 at a cycle of (1 / 60) seconds.

[0049] In this way, the user changes the line of sight by moving his / her head, and the information processing device 10 provides the HMD 100 with a panoramic image of the desired direction to be displayed on the display panel 130. Changing the line of sight by moving the head is the same as a movement in the real world, and matches the user's senses. At this time, the HMD 100 provides the user with a wide viewing angle, further enhancing the sense of immersion in the panoramic image.

[0050] On the other hand, for example, to view a panoramic image directly behind the user, the user must naturally face directly behind. In order to face directly behind, it is not enough to simply turn the neck, but the user must change the direction of the body. Therefore, if the user is sitting on a non-rotating chair such as a sofa, the user must stand up and change the direction of the body. Also, if the HMD 100 is a non-transparent display device, the user's eyes are covered by the housing 108 and the user cannot see the surrounding environment, so the user may feel some resistance to moving his / her feet to change the direction of the body.

[0051] Therefore, in the embodiment, the user is allowed to input an instruction to switch the line of sight from the input device 6, so that the user can change the line of sight direction by operating the input device 6 without moving his / her head. Note that, in the embodiment, the input device 6 is shown as a device separate from the HMD 100, but the input device 6 may be provided in the HMD 100, that is, may be formed as an operation input unit such as a button on the wearing band 106 or the housing 108 of the HMD 100.

[0052] FIG. 10(a) shows the top surface of the input device 6. A user operates the input device 6 by holding the left grip part 78b with the left hand and holding the right grip part 78a with the right hand. The input device 6 has a direction button 71, analog sticks 77a and 77b, and an operation button 76 on the top surface of the housing. The direction button 71 includes an up button 71a, a left button 71b, a down button 71c, and a right button 71d. The right analog stick 77a and the left analog stick 77b are tilted to input a direction and a tilt amount. The right analog stick 77a and the left analog stick 77b also function as push-down buttons that sink downward when pressed by the user and return to their original position when the user releases their hands. A touch pad 79 is provided in a flat area between the direction button 71 and the operation button 76 on the top surface of the housing. The touch pad 79 also functions as a push button that sinks downward when pressed by the user and returns to its original position when the user releases the button.

[0053] A home button 80 is provided between the right analog stick 77a and the left analog stick 77b. The home button 80 is used to power on the input device 6 and at the same time activate a communication function for wirelessly connecting to the information processing device 10. A SHARE button 81 is provided to the left of the touch pad 79. The SHARE button 81 is used to input instructions from the user to the OS or system software in the information processing device 10. An OPTIONS button 82 is provided to the right of the touch pad 79. The OPTIONS button 82 is used to input instructions from the user to an application (game) executed in the information processing device 10. Both the SHARE button 81 and the OPTIONS button 82 may be formed as push buttons.

[0054] 10(b) shows the rear side of the input device 6. A touch pad 79 is bent and extended from the top surface of the housing of the input device 6 on the upper side of the rear side of the housing, and a horizontally long light-emitting unit 85 is provided on the lower side of the rear side of the housing. The light-emitting unit 85 has red (R), green (G), and blue (B) LEDs, and lights up according to light emission color information transmitted from the information processing device 10. The upper R1 button 83a and L1 button 83b are configured as push buttons, and the lower R2 button 84a and L2 button 84b may be configured as pivotally supported trigger buttons.

[0055] In the embodiment, the left analog stick 77b among the operation members of the input device 6 is used to input an instruction to switch the line of sight direction. When the left analog stick 77b is tilted to the left, the line of sight direction moves to the left, and when tilted to the right, the line of sight direction moves to the right. In this way, the left analog stick 77b is suitable for the user to intuitively change the line of sight direction. Note that other operation members, such as the right analog stick 77a and the direction button 71, may be used to input an instruction to switch the line of sight direction.

[0056] The inventors have tried various methods of changing the line of sight direction in response to the operation of the left analog stick 77b. First, as one method of change processing, the line of sight direction is continuously moved according to the tilt direction of the left analog stick 77b, generating an image in which a panoramic image moves smoothly. This is a common method of changing the line of sight for games displayed on a television screen, and is considered to match the user's senses. Various speeds of image movement at that time were also tried.

[0057] As a result, it was found that when the panoramic image was moved continuously by operating the left analog stick 77b, the user felt motion sickness when viewing it. This was independent of the speed at which the image moved, and whether the image was moved slowly or quickly, the result was that motion sickness was induced, although the degree of motion differed from person to person.

[0058] On television screens, it is common to have a display image that moves continuously in one direction, and in this case, users do not usually get motion sickness. After examining this point, the inventors have found that one of the causes is the difference in the viewing angle.

[0059] As for a television screen, the user watches it from a certain distance, so the user's horizontal viewing angle with respect to the television screen is at most several tens of degrees. In contrast, in a wide viewing angle HMD, the user's viewing angle is close to 90 degrees or more (hereinafter, the horizontal viewing angle of the HMD 100 is set to about 100 degrees). In other words, in the HMD 100, the image moves continuously in a range of about 100 degrees (and further in the entire field of view), which makes it easy for the user to experience motion sickness, and it was found that this is a problem specific to the HMD 100. Therefore, the present inventor tried to move the panoramic image discontinuously rather than continuously by operating the left analog stick 77b, and as a result, he found that it is possible to prevent motion sickness from occurring. Therefore, in the embodiment, when the information processing device 10 acquires the operation of the left analog stick 77b as an instruction to switch the line of sight, the information processing device 10 controls the display so that the panoramic image is moved discontinuously, that is, by a predetermined angle in one switching (one step).

[0060] When the user tilts the left analog stick 77b of the input device 6, the instruction acquisition unit 24 acquires an instruction to switch the line of sight direction to the tilted direction from the input device 6. For example, when the left analog stick 77b is tilted once and then immediately returned, the instruction acquisition unit 24 acquires a single instruction to switch to the tilted direction. Also, when the left analog stick 77b continues to be tilted, the instruction acquisition unit 24 continuously acquires an instruction to switch to the tilted direction.

[0061] Whether the instruction acquisition unit 24 acquires the tilt operation of the left analog stick 77b as a single switching instruction or as continuous switching instructions is determined by whether the tilt time (the time from tilting to returning to the original position) exceeds a predetermined time. Specifically, the instruction acquisition unit 24 acquires it as a single switching instruction when the tilt time is less than the predetermined time, and acquires it as another switching instruction when the tilt time reaches the predetermined time. If the tilt operation continues thereafter, a switching instruction may be acquired each time a time shorter than the predetermined time has passed.

[0062] When the instruction acquisition unit 24 acquires an instruction to switch the line of sight direction, the line of sight direction determination unit 32 changes the line of sight direction by a predetermined angle. For example, when the left analog stick 77b is tilted leftward, the instruction acquisition unit 24 acquires an instruction to switch the line of sight direction to the left, and the line of sight direction determination unit 32 changes the line of sight direction counterclockwise by a predetermined angle. In the following, an example will be described in which the user inputs an instruction to switch the line of sight direction to the left to the input device 6 while the display image 200a shown in FIG. 7 is displayed on the display panel 130 of the HMD 100.

[0063] 11 shows a display image 200d generated by the image generation unit 34. When the instruction acquisition unit 24 acquires a switching instruction to rotate the line of sight direction counterclockwise, the line of sight direction determination unit 32 changes the line of sight direction counterclockwise by a predetermined angle. Here, the predetermined angle is set to an angle larger than 10 degrees and smaller than the horizontal viewing angle (100 degrees) of the HMD 100.

[0064] If the angle changed in one step is set to 10 degrees or less, for example, at least 18 steps are required to see an image directly behind (180 degree rotation), and the large number of steps can make the user feel a little frustrated. On the other hand, if the angle changed in one step is greater than the viewing angle of the HMD 100, a completely new image (an image with no overlap with the image before switching) is suddenly displayed on the display panel 130, and continuity with the image before switching cannot be guaranteed. Therefore, it is preferable that the angle changed in one step is set to an angle greater than 10 degrees and smaller than the viewing angle of the HMD 100.

[0065] More preferably, the change angle in one step is set to half or less of the viewing angle of the HMD 100. This allows at least half of the image before switching to be included in the image after switching, and the user can recognize the continuity of the image before and after switching. In order to make the image continuity more noticeable, it is preferable that the change angle in one step is small. The inventors have tried various change angles per step and have come to the conclusion that a change angle per step of 15 degrees or more and 30 degrees or less is most preferable in terms of the time required to reach a desired angle when continuously switching and image continuity. The example shown in FIG. 11 shows a state in which the change angle in one step is set to 22.5 degrees.

[0066] Generally, it is said that the central visual field of a human being is 40 to 45 degrees, and the peripheral visual field is about 200 degrees. By focusing on this central visual field, the change angle per step may be set to be smaller than the central visual field so that the same image remains in the central visual field before and after switching. From this point of view, it is meaningful to set the change angle per step to 30 degrees or less. It is preferable that the change angle per step is set to a value obtained by dividing 360 degrees by an integer N, and that the original display position can be returned to by the user inputting an integer number of switching instructions from the input device 6.

[0067] Fig. 12 shows display images before and after switching displayed on the display panel 130. Fig. 12(a) shows display image 200a, and Fig. 12(b) shows display image 200d. Display image 200a is rotated left by a change angle of 22.5 degrees in one step to produce display image 200d. By setting the change angle to 22.5 degrees out of a viewing angle of 100 degrees, 22.5% of the image changes before and after switching, but 77.5% remains in the images before and after switching. This allows the user to easily recognize the continuity of the images.

[0068] When the instruction acquisition unit 24 continuously acquires switching instructions, the line of sight direction determination unit 32 changes the line of sight direction by a predetermined angle. This change period may be a fixed value such as one second, or may be dynamically determined according to the tilt amount of the left analog stick 77b. In other words, the change period may be set to be shorter as the tilt amount increases.

[0069] In the embodiment, the switching instruction using the input device 6 is valid only in the left-right direction, and invalid in the up-down direction. This is because a person faces the horizontal direction when sitting in a chair or standing upright, and does not usually face upward or downward all the time, and such a sense of reality is respected in the VR application. It is also possible to enable the switching instruction in the up-down direction, and it may be possible to set whether to enable or disable the switching instruction in the up-down direction depending on the content image. For a content image in which the top and bottom are not noticeable, such as a photographed image of a starry sky in the sky, the switching instruction in the up-down direction may be enabled.

[0070] 12 shows display images 200a and 200d before and after switching when the change angle per step is 22.5 degrees, but some users may wish to change the angle larger or smaller than 22.5 degrees. Therefore, options for selecting the change angle per step may be displayed on the display panel 130, and the user may operate the input device 6 to select the option displayed on the display panel 130. In the embodiment, the user is provided with options of 15 degrees and 30 degrees in addition to 22.5 degrees.

[0071] In the following, a procedure for displaying options for the change angle per step in a state in which the display image 200a shown in FIG. 12(a) is displayed on the display panel 130 of the HMD 100 will be described. A user can display menu items on the display panel 130 by operating a predetermined input section of the input device 6. For example, a △ button 75 (see FIG. 10(a)) may be assigned to the menu display operation. When a user presses the △ button 75, the input device 6 transmits operation information indicating that the △ button 75 has been pressed to the information processing device 10. The instruction acquisition section 24 in the information processing device 10 acquires the pressing information of the △ button 75 as an instruction to display the menu item.

[0072] When the instruction acquisition unit 24 acquires an instruction to display a menu item, the image generation unit 34 performs a process of including the menu item in the panoramic image to be generated. Note that the menu item is an example of an information element to be presented to the user, and the information element may be other items or notification information. By the image generation unit 34 including an information element such as a menu in the panoramic image, the user can view the presented information element while viewing the panoramic image based on the line of sight on the display panel 130 of the HMD 100.

[0073] The information element storage unit 42 stores information elements to be included in the panoramic image. The image generation unit 34 reads out information elements corresponding to the display instruction from the information element storage unit 42, and superimposes a window in which the information elements are arranged on the panoramic image. The image provision unit 36 ​​provides the panoramic image including the menu items to the HMD 100.

[0074] Fig. 13(a) shows an example in which a menu window 204 is superimposed on a display image 200a. In Fig. 13(a), the panoramic image behind the menu window 204 is hidden by the menu window 204, but in practice, it is preferable that the menu window 204 is displayed transparently so that the user can also see the panoramic image behind the menu window 204.

[0075] In the menu window 204, a selection frame surrounding one item is displayed, and the user can move the selection frame by pressing the up button 71a or the down button 71c of the input device 6. By placing the selection frame on a desired item and pressing a confirmation button (e.g., the circle button 72), the information elements below that correspond to the selected item are displayed on the display panel 130.

[0076] In the information processing device 10, the instruction acquisition unit 24 acquires operation information of the direction button 71 as an instruction to select an item, and acquires operation information of the circle button 72 as an instruction to confirm the item. The image generation unit 34 moves the selection frame in accordance with the instruction to select an item, and in accordance with the instruction to confirm the item, reads out the corresponding information element from the information element storage unit 42 and includes it in the panoramic image.

[0077] Fig. 13(b) shows an example in which a selection window 206 is superimposed on a display image 200a. When "change the change angle per step" is selected in the menu window 204 in Fig. 13(a), a selection window 206 presenting options for the change angle per step is displayed. Like the menu window 204, the selection window 206 is preferably displayed transparently so that the user can see the panoramic image behind it. When the user selects and confirms any angle in the selection window 206, the change angle per step is changed.

[0078] In this way, the information elements are superimposed on the panoramic image and displayed on the display panel 130. In the information processing system 1, the panoramic image on the display panel 130 changes in response to a change in the attitude of the HMD 100 (i.e., movement in the line of sight) and operation of the left analog stick 77b, but the present inventor has considered how to control the display of information elements displayed together with the panoramic image in a manner suitable for the HMD 100.

[0079] First, the inventor tried a method of always displaying an information element in the center of the screen of the display panel 130. In this case, even if the line of sight direction is changed and the panoramic image behind the information element changes, the display position of the information element on the screen does not move, but is always displayed in a fixed position in the center of the screen. It was found that this display method gives the impression that the information element is moving within the panoramic image, creating a sense of incongruity.

[0080] Therefore, the inventor tried a method of not changing the position of an information element when it is included in a panoramic image. In other words, the position of the information element in the panoramic image is fixed, so that when the user changes the line of sight, the information element moves together with the panoramic image. It was found that fixing the relative positional relationship between the information element and the panoramic image gives the impression that the information element constitutes a part of the panoramic image, and does not cause discomfort to the user.

[0081] However, in this case, since the information element moves together with the panoramic image, if the movement in the line of sight is large, the information element may move off the display screen, causing the user to lose sight of the information element. In particular, since the input device 6 is assigned to menu operations while the menu is displayed, the user must change the line of sight to find the menu window that he or she has lost sight of, whether to continue or end the menu operation.

[0082] To solve this problem, the inventor devised a method in which, in principle, the relative positional relationship between an information element and a panoramic image is fixed, and the information element moves together with the panoramic image, while at the same time, when an information element is about to fall off the screen, the relative positional relationship between the information element and the panoramic image is changed before it falls off the screen.

[0083] In this method, the time point when the information element is displayed is used as a reference, and when the posture of the HMD 100 changes from that point, the image generating unit 34 generates an image so that the information element is displayed to move with the image if the amount of change in posture is smaller than a predetermined first angle, and the information element is displayed at a position in the image moved by a predetermined second angle if the amount of change in posture reaches the predetermined first angle. The amount of change in posture of the HMD 100 corresponds to the amount of change in the movement in the line of sight.

[0084] Specifically, the line of sight direction determination unit 32 monitors changes in the movement of the line of sight from the time when the information element is displayed, and if the movement of the line of sight is smaller than a predetermined first angle, the image generation unit 34 displays the information element so that it moves together with the panoramic image, whereas if the movement of the line of sight reaches the predetermined first angle, the image generation unit 34 displays the information element at a position in the panoramic image moved by a predetermined second angle. Below, an example will be described in which the line of sight direction determination unit 32 monitors the rotation angle of the horizontal line of sight as a change in the movement of the line of sight, and the image generation unit 34 controls the display of the information element.

[0085] For example, consider a case where the user turns his / her head horizontally counterclockwise from the point when the display of the information element starts, changing the line of sight to the left. In this case, the panoramic image is displayed to flow to the right, and therefore the displayed information element also flows to the right in the display panel 130. When the movement angle of the information element, that is, the rotation angle of the horizontal line of sight, reaches a predetermined first angle, the image generating unit 34 moves the information element to a position rotated to the left by a predetermined second angle in the panoramic image and displays it. This first angle is set to an angle at which all of the information elements are not completely out of the user's field of view, and therefore the information element is maintained in a state in which at least a part of it is always displayed on the display panel 130. Hereinafter, this first angle may be called the "position change reference angle" and the second angle may be called the "return angle".

[0086] In the following, FIG. 13(a) shows the state in which the menu window 204 is initially displayed, and an example will be described in which the user rotates the line of sight to the left from this initial display state. Fig. 14 shows how the menu window 204, which is an information element, moves together with the panoramic image. When the user turns his / her head horizontally counterclockwise, the motion detection unit 30 detects the attitude of the HMD 100, and the line of sight direction determination unit 32 rotates the line of sight direction counterclockwise according to the detected attitude of the HMD 100. Fig. 14(a) shows a display image 200d rotated 22.5 degrees left from the initial display state. As described above, the 22.5 degree rotation can also be realized by operating the left analog stick 77b, and the display image 200d may be displayed on the display panel 130 by tilting the left analog stick 77b to the left without the user moving his / her head.

[0087] Because the relative positions of the menu window 204 and the panoramic image are fixed, the menu window 204 moves with the panoramic image, and therefore on the display panel 130 the user sees the menu window 204 moving to the right.

[0088] Fig. 14(b) shows a display image 200e rotated 45 degrees left from the initial display state. To display the display image 200e, the user may rotate his / her head 45 degrees left, but the 45 degrees left rotation can also be achieved by tilting the left analog stick 77b twice to the left, or by turning the head 22.5 degrees left and tilting the left analog stick 77b once to the left. Compared to Fig. 14(a), it can be seen that the menu window 204 has moved further to the right.

[0089] 15(a) shows a display image 200f rotated left from the initial display state to just before the position change reference angle, which is set to an angle at which none of the information elements fall completely off the screen.

[0090] Therefore, the position change reference angle needs to be set to an angle smaller than the viewing angle of the HMD 100. If the position change reference angle is equal to or larger than the viewing angle, a situation will occur in which all information elements are completely off the screen. Considering the horizontal position change reference angle, since the horizontal viewing angle of the HMD 100 is 100 degrees, the position change reference angle needs to be set to an angle smaller than 100 degrees.

[0091] The inventors tested the appearance of variously set position change reference angles, and found that when the position change reference angle is set to be very small relative to the viewing angle of the HMD 100, the display elements end up giving the impression that they are following the line of sight. As a result of various experiments, they found that by setting the position change reference angle to an angle approximately half the viewing angle of the HMD 100, the unity of the display elements and the background panoramic image can be recognized, and a user interface that makes the display elements visible can be realized. In the embodiment, the position change reference angle (first angle) is set to 60 degrees, approximately half the viewing angle of the HMD 100 (50 degrees).

[0092] 15(b) shows a display image 200g in which the information element is rotated leftward by a second angle (60 degrees) in the panoramic image from the initial display state. In this way, when the change in the predetermined direction of the line of sight from the initial display state reaches the first angle (60 degrees), the image generating unit 34 rearranges the relative position of the information element and the panoramic image to a position changed by the second angle in the predetermined direction. Here, the second angle is set to the same angle as the first angle. After the rearrangement, the line of sight direction determining unit 32 monitors the change in the movement of the line of sight direction based on the line of sight direction at the time of rearrangement.

[0093] 15(a), only a part of the menu window 204 is displayed, but in Fig. 15(b), the entire menu window 204 is displayed again, which makes it easier for the user to operate the menu in the menu window 204. Also, as shown in Fig. 15(a), by performing display control so that the entire menu window 204 is not hidden from view, the user can operate the menu window 204 at any time while enjoying the panoramic image without losing sight of the menu window 204.

[0094] The above describes an example in which the first angle, which is the position change reference angle, is equal to the second angle, which is the return angle. When the first angle and the second angle are equal, if the line of sight direction changes by the first angle from the initial display state, the information element is rearranged to the center position displayed in the initial display state, as shown in FIG. 15(b). Therefore, if the user turns his / her head further counterclockwise in the horizontal direction, the menu window 204 moves from the center to the right. Therefore, the menu window 204 is displayed to the right of the center, which makes it slightly difficult to see.

[0095] Therefore, hereinafter, an example will be described in which the second angle, which is the return angle, is set to be larger than the first angle, which is the position change reference angle. The second angle is set to 60 degrees, and the first angle is set to 45 degrees.

[0096] Here, it is assumed that FIG. 14(b) shows a display image 200e that has been rotated left from the initial display state to just before the first angle (45 degrees). When the line of sight moves further left and the change in the predetermined direction of the line of sight reaches the first angle (45 degrees), the image generating unit 34 rearranges the information element and the panoramic image to a position where the relative position of the information element and the panoramic image is changed by a second angle (60 degrees) in the predetermined direction. Here, since the second angle is set to be larger than the first angle, the rearranged information element is returned beyond the central position of the initial display state, and is therefore displayed to the left of the central position. After the rearrangement, the line of sight direction determining unit 32 monitors the change in the movement of the line of sight direction based on the line of sight direction at the time of rearrangement.

[0097] 16 shows a display image 200h in which the information element is rotated left by a second angle (60 degrees) in the panoramic image from the initial display state. When the change in the predetermined direction of the line of sight from the initial display state reaches a first angle (45 degrees), the image generating unit 34 rearranges the information element in the panoramic image at a position changed in a direction returning the information element to the predetermined direction by the second angle (60 degrees).

[0098] By setting the second angle larger than the first angle, when the user continues to move the line of sight to the left, the menu window 204 gradually moves rightward toward the center. This makes it possible to improve the visibility of the information elements while allowing the user to recognize the unity of the display elements with the background panoramic image. Note that by setting the first angle larger than half the second angle, it is possible to prevent the amount of movement in the panoramic image from becoming too large when rearranging the display elements.

[0099] In the above, it has been described that when the line of sight change in the horizontal direction reaches the position change reference angle (first angle), the horizontal arrangement of the information element is changed by the return angle (second angle). The same applies to the vertical direction, and when the line of sight change in the vertical direction reaches the position change reference angle for the vertical direction, display control may be executed to change the vertical arrangement of the information element by the return angle for the vertical direction. Note that the position change reference angle for the vertical direction may be set smaller than the position change reference angle for the horizontal direction, and may be set to, for example, 30 degrees.

[0100] Next, consider where to initially display the information element. One method is to display it near the center of the image that is being displayed at the timing when the information element is to be displayed, as shown in Figure 13(a). This method is effective because it is easy for the user to understand intuitively.

[0101] As another method, candidates for the display position of the information element may be set in advance on the virtual sphere. In the coordinate system of the virtual sphere, when the horizontal reference direction on the horizontal plane is set to 0 degrees, the candidate rotation angles are 60 degrees, 120 degrees, 180 degrees, 240 degrees, and 300 degrees in the horizontal direction, and the candidate tilt angles are 0 degrees, ±30 degrees, and ±60 degrees in the vertical direction. When displaying the information element, the candidate rotation angle and candidate tilt angle that are closest to the line of sight direction may be extracted, and the information element may be displayed with the extracted rotation angle and tilt angle as the center. By predetermining candidates for the initial position in this way, the image generating unit 34 only needs to select the initial position from among the candidate positions, simplifying the display control.

[0102] The present invention has been described above based on the embodiments. The embodiments are merely illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention.

[0103] In FIG. 14(a), a display image 200d and a menu window 204 rotated 22.5 degrees left from the initial display state are shown, and it has been explained that the 22.5 degrees left rotation may be realized by turning the user's neck or by operating the left analog stick 77b. In a modified example, when the left analog stick 77b is operated, the relative positional relationship between the information element and the panoramic image is released, and the information element may be displayed in the center in the image switched by operating the left analog stick 77b, as shown in FIG. 13(a). This is because when switching images by operating the left analog stick 77b, the displayed image jumps and the continuity of the image is interrupted. In such a case, it is also possible to release the fixation of the relative positional relationship between the information element and the panoramic image and control the display of the information element.

[0104] In the embodiment, the display of the information element is controlled by using the position change reference angle and the return angle. In a modified example, the display of the information element may be controlled according to the type of the information element. For example, when the system issues an important warning to the user, the image generating unit 34 may always place the warning information element in the center of the display image.

[0105] As one function of the VR application, the image generating unit 34 may realize a mirror function. The mirror function is a function for including an image in the opposite direction of the line of sight in the displayed image. Instead of including an image exactly opposite the line of sight, only the rotation angle of the line of sight is inverted, that is, an image in the line of sight direction is rotated by 180 degrees without changing the inclination, and is included in the displayed image. This mirror function allows the user to see an image of the back side at the same height. [Explanation of symbols]

[0106] 1 Information processing system, 6 Input device, 8 Virtual camera, 10 Information processing device, 12 Processing device, 14 Output control device, 20 Sensor information acquisition unit, 22 Captured image acquisition unit, 24 Instruction acquisition unit, 30 Motion detection unit, 32 Line of sight direction determination unit, 34 Image generation unit, 36 Image provision unit, 40 Image storage unit, 42...information element memory unit, 100...HMD, 102...output mechanism unit, 104...wearing mechanism unit, 106...wearing band, 108...casing, 110...light-emitting marker, 120...control unit, 122...memory unit, 124...posture sensor, 126...microphone, 128...communication control unit, 130...display panel, 132...audio output unit.

Claims

1. a detection unit that detects the orientation of the device from a captured image of the device and / or sensor information detected by an orientation sensor of the device; a gaze direction determination unit that determines a gaze direction according to the detected attitude of the device; an image generating unit that generates an image based on a determined line of sight; An instruction acquisition unit that acquires an instruction to switch a line of sight, When the instruction acquisition unit acquires a switching instruction, the line-of-sight direction determination unit changes the line-of-sight direction by a predetermined angle.

2. When the instruction acquisition unit successively acquires a switching instruction, the line-of-sight direction determination unit changes the line-of-sight direction in a horizontal direction by a predetermined angle at a time.

2. The information processing apparatus according to claim 1,

3. The line of sight direction determining unit changes the line of sight direction by an angle greater than 10 degrees and less than or equal to 30 degrees.

2. The information processing apparatus according to claim 1,

4. The angle by which the line of sight direction determination unit changes the line of sight direction is set to a value obtained by dividing 360 degrees by an integer N.

2. The information processing apparatus according to claim 1,

5. detecting the orientation of the device from a captured image of the device and / or sensor information detected by an orientation sensor of the device; determining a gaze direction in response to the detected orientation of the device; generating an image based on the determined gaze direction; and acquiring a gaze direction switching instruction, The step of determining the line of sight direction includes changing the line of sight direction by a predetermined angle when a switching instruction is acquired.

13. An image generating method comprising:

6. On the computer, A function of detecting the orientation of the device from a captured image of the device and / or sensor information detected by an orientation sensor of the device; A function of determining a gaze direction according to the detected orientation of the device; A function for generating an image based on a determined gaze direction; A function of acquiring an instruction to switch the line of sight and a program for realizing the function, A program, characterized in that the function of determining the line of sight direction includes a function of changing the line of sight direction by a predetermined angle when a switching instruction is acquired.

Images