Information processing apparatus, information processing system, method for controlling information processing apparatus, and program

Abstract

To provide an information processing apparatus, an information processing system, a method for controlling the information processing apparatus, and a program that can easily display, while switching, an image when a user sees at the current position and a pseudo image when the user is to see at a designated position.SOLUTION: A HMD 101 has a CPU 201 that acquires a user's current position as a current position, presents a plurality of positions including the current position on a display 208, designates a position from among the plurality of positions according to the user's instruction, displays an image acquired by a photographing unit 207 through photographing in a real space on the display 208 as a real image indicating the user's field of view at the current position when the designated position is the current position, and when the designated position is not the current position, creates a pseudo image imitating the user's field of view at the designated position and displays the pseudo image on the display 208.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, a control method for an information processing device, and a program. [Background technology]

[0002] In recent years, technological innovation in mixed reality (MR) has been remarkable. Furthermore, with this technological innovation, MR devices such as head-mounted displays (HMDs) are becoming more common. HMDs are being used in a variety of MR content, such as sharing completed images at construction sites and announcing new products by superimposing computer graphics (CG) images at exhibition halls. In MR content that provides such mixed reality spaces, if a user wants to change the position from which they view virtual objects, walking is the primary method. However, from the user's perspective, it is inconvenient for the user to have to walk every time they repeatedly change the position from which they view virtual objects. As a related technology, for example, Patent Document 1 discloses a technology that records panoramic images in association with the position and orientation at the time of capture, and generates and displays panoramic images as seen from positions where the user has virtually walked around, depending on the user's head orientation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-204973 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 has the problem that although it can display a panoramic image at a position where the user has virtually walked around, it is not easy to immediately return to displaying a panoramic image at a previous position.

[0005] The present invention has been made in view of the above-mentioned problems, and aims to provide an information processing device, an information processing system, a control method for an information processing device, and a program that can easily switch between displaying an image as viewed from a user's current position and a pseudo image as viewed from a position specified by the user. [Means for solving the problem]

[0006] In order to achieve the above object, the information processing device of the present invention is characterized by comprising: a position acquisition means for acquiring the current position of the user as a current position; a presentation means for presenting a plurality of positions including the current position on a display unit; a designation means for designating a position from among the plurality of positions in accordance with a user's instruction; a first display control means for, when the position designated by the designation means is the current position, displaying on the display unit an image acquired by a photographing unit by photographing real space as a real image showing the user's field of view at the current position; a generation means for, when the position designated by the designation means is not the current position, generating a pseudo image simulating the user's field of view at the position designated by the designation means; and a second display control means for displaying the pseudo image on the display unit. [Effects of the Invention]

[0007] According to the present invention, it is possible to easily switch between and display an image as viewed from the user's current position and a pseudo image as viewed from a position designated by the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a conceptual diagram illustrating the present embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of an HMD according to the present embodiment. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of the HMD according to the first embodiment. [Figure 4]FIG. 2 is a diagram showing an example of a data combination in which position and orientation information and an image of real space (hereinafter referred to as a "real image") are linked in position and orientation image information of the HMD of the first embodiment. [Figure 5] 3A to 3C are diagrams illustrating an example of when the display is switched in the HMD of the first embodiment. [Figure 6] 4A and 4B are diagrams illustrating an example of a method for specifying a position by a position specifying means in the HMD of the first embodiment. [Figure 7] 10 is a flowchart showing a flow when position and orientation information and a real image are recorded in the position and orientation image information of the HMD according to the first embodiment. [Figure 8] 5 is a flowchart showing a flow when the display is switched between a real image and a pseudo image in the HMD of the first embodiment. [Figure 9] This is a flowchart showing the flow when the display is switched between a real image and a pseudo image in the HMD of the first embodiment, and is a flowchart specialized for the flow when a position is specified by a user's gesture. [Figure 10] FIG. 10 is a block diagram showing the functional configuration of an HMD according to a second embodiment. [Figure 11] 10 is a flowchart showing a flow when the display is switched between a mixed reality image and a mixed pseudo image in the HMD of the second embodiment. [Figure 12] 10 is a flowchart showing a display process of a mixed reality image according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in each embodiment. For example, each component constituting the present invention can be replaced with any configuration that can perform the same function. Any component may also be added. Any two or more configurations (features) of each embodiment can be combined. Not all combinations of features described in each embodiment are necessarily essential to the solution of the present invention. Furthermore, the configurations of each embodiment can be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In the following embodiments, the same components are described with the same reference numerals.

[0010] First, a configuration common to each embodiment will be described with reference to the drawings. FIG. 1 is a conceptual diagram illustrating this embodiment. 101 is a head-mounted information processing device, a video see-through HMD. 102 is a user using the HMD 101. 103 is a shelf (hereinafter referred to as a "real object") that actually exists in front of the user 102. If the HMD 101 is made small and lightweight, the imaging and display system and processing system of the HMD 101 may be separated, with the imaging and display system located in the HMD 101 and the processing system located in a small external box computer as an external component. In such a configuration, the small external box computer corresponds to the information processing device. The external component is not limited to a small external box computer, and may be, for example, a portable computer such as a notebook PC, tablet PC, or smartphone, or a stationary computer such as a desktop PC.

[0011] FIG. 1(a) is a diagram showing an example of a scene in which a user 102 acquires information indicating the position and orientation of the user 102 (hereinafter also referred to as "position and orientation information") and an image at that position and orientation using a gyro sensor and a camera mounted on an HMD 101. In this case, the HMD 101 is worn on the head of the user 102 and is able to acquire the position and orientation information of the user 102 using the gyro sensor. Therefore, the HMD 101 can acquire information indicating the position and orientation of the user 102 and an image at that position and orientation by capturing an image using the camera in addition to acquiring the information using the gyro sensor. The HMD 101 associates the position and orientation information acquired in this way with the image and records them in storage.

[0012] Reference numeral 104 denotes an image currently being acquired by a camera mounted on the HMD 101, which is an image currently being displayed on an electronic display mounted on the HMD 101 and which indicates the field of view of the user 102 wearing the HMD 101. Reference numeral 105 denotes an arrow used by the HMD 101 to guide the user 102 to other postures. This allows the user 102 to record images of various postures at the current position (hereinafter also referred to as "current position") in the HMD 101 by moving their head along the arrow 105.

[0013] FIG. 1(b) is a diagram showing an example of a scene in which the user 102 switches the display of the electronic display mounted on the HMD 101 from a mixed reality image 106 at the current position to a composite pseudo image 107 at a specified position (a recorded position 109, described later). Reference numeral 108 denotes a three-dimensional virtual object (hereinafter referred to as a "virtual object"). Reference numeral 109 denotes the position of the user 102 (hereinafter referred to as a "recorded position") indicated by the position and orientation information associated with the image and recorded as described above. The mixed reality image 106 is an image of the mixed reality space viewed by the user 102 wearing the HMD 101 at the current position, and is composed of the real object 103, the virtual object 108, and a mark 110 indicating the recorded position 109. The composite pseudo image 107 is an image of the pseudo mixed reality space as viewed by the user 102 wearing the HMD 101 at the recorded position 109, and is composed of the real object 103, the virtual object 108, and the like. It should be noted that the mark 110 indicating the recording position 109 is not included in the composite pseudo image 107 because the recording position 109 does not fit therein.

[0014] When the user 102 designates a mark 110 using a controller or a hand tracking function of the HMD 101, the mixed reality image 106 at the current position is switched to a composite pseudo image 107 at a recording position 109. Furthermore, when the user 102 makes a gesture such as waving his / her hand, the composite pseudo image 107 at the recording position 109 is switched to the mixed reality image 106 at the current position. Furthermore, when the user 102 utters a voice indicating the position name indicating the recording position 109, the mixed reality image 106 at the current position is switched to the composite pseudo image 107 at the recording position 109. In this way, the HMD 101 can switch between displaying the mixed reality image 106 at the current position and the composite pseudo image 107 at the recording position 109. However, in a first embodiment described later, a case in which the virtual object 108 does not exist will be described.

[0015] FIG. 2 is a diagram showing an example of the hardware configuration of the HMD 101 according to this embodiment. The CPU 201 is a system control unit that controls the entire HMD 101. The CPU 201 also executes an information processing program to implement information processing according to this embodiment. The ROM 202 is a read-only memory that stores programs and parameters that do not require modification, such as basic programs and initial data. The RAM 203 is a memory for temporarily storing input information and calculation results from information processing and image processing. The sensor 204 is a sensing component, such as a gyro sensor, that detects the position and orientation of the HMD 101, i.e., the position and orientation of the user 102.

[0016] The communication I / F 205 is an interface unit capable of sending and receiving data to and from an external camera, the cloud, etc. The input / output connection form via the communication I / F 205 includes both a local connection via USB (registered trademark) or Bluetooth (registered trademark) and an internet connection via Ethernet (registered trademark) or Wi-Fi (registered trademark). The recording unit 206 is a storage unit capable of writing and reading various information. Specifically, the recording unit 206 is a memory card, removable disk, or IC card that is detachable from the HMD 101, or a hard disk or memory card that is built into or external to the HMD 101.

[0017] The information processing program according to this embodiment is recorded in the recording unit 206, read from the recording unit 206 by the CPU 201, and further expanded into the RAM 203 for execution. The information processing program may be stored in the ROM 202. The recording unit 206 can record required data used by the information processing program executed by the CPU 201. The photographing unit 207 is an imaging device such as a camera mounted on the HMD 101. The display unit 208 is a display device such as an electronic display mounted on the HMD 101. The display unit 208 is configured as a stereo display corresponding to both eyes of the user 102. The operation unit 209 controls input to the HMD 101. The operation unit 209 includes a power button, a menu button, a controller, and the like. In the HMD 101, all hardware components are connected to a bus 210 and communicate with each other.

[0018] The image capturing system and processing system of the HMD 101 may be separated as external components from the HMD 101 to form a display system (specifically, the display unit 208) and an information processing system for the HMD 101. In such an information processing system, the image capturing system (component equivalent to the photographing unit 207) separated from the HMD 101 may be, for example, an image capturing device such as a camera held by the user 102 or a 360-degree camera.

[0019] First Embodiment The first embodiment will be described below with reference to FIGS. 3 to 9 in addition to the above-mentioned FIGS. 1 and 2. FIG. 3 is a block diagram showing the functional configuration of the HMD 101 according to the first embodiment. A real image acquisition means 301 acquires a real image using the imaging unit 207 of the HMD 101. A user position and orientation acquisition means 302 acquires position and orientation information of the user 102 using the sensor 204 of the HMD 101. A recording position guidance display means 303 applies image recognition technology or the like to the real image acquired by the real image acquisition means 301, and displays guidance on the shooting position and shooting range to the user 102 when the recording means 304 performs recording. The arrow 105 described above is also displayed by the recording position guidance display means 303.

[0020] The recording means 304 acquires an image at the position and orientation indicated by the position and orientation information acquired by the user position and orientation acquisition means 302 from the real image acquired by the real image acquisition means 301, links the image to the position and orientation information, and records the image in position and orientation image information 305. In this way, the position and orientation image information 305 having a plurality of combinations of data in which position and orientation information and real images are linked is constructed as a data set by the recording means 304. Note that the recording means 304 records the image in the position and orientation image information 305 in response to a signal from the user 102 to start recording or when the current position of the user 102 remains the same for a certain period of time.

[0021] FIG. 4 is a diagram showing an example of a combination of data in which position and orientation information and real images are linked in the position and orientation image information 305. The position and orientation image information 305 records multiple sets of position and orientation information of the user 102 and real images linked thereto. The real images are data-compressed in JPEG format and are represented as image information by file names in FIG. 4. The position and orientation information of the user 102 is divided into position information and orientation information and recorded in the position and orientation image information 305. The position information is represented by (x, y, z), and the orientation information is represented by (yaw, pitch, roll). Note that in various control systems of the HMD 101, the position and orientation of the position and orientation information is converted into a position and orientation relative to the current position of the user 102, based on the position and orientation at an arbitrary point in real space, before being used. Note that in the following description, the position and orientation information of the user 102 will be referred to as "position information" when focusing on position information, and as "orientation information" when focusing on orientation information.

[0022] Returning to the explanation of FIG. 3, the position presenting means 306 presents the positions indicated by the position information recorded in the position and orientation image information 305 and the current position of the user 102 using marks or a list. Note that the position presenting means 306 presents only one of the overlapping positions. The position designation means 307 designates one position from among the positions presented by the position presenting means 306 in response to an instruction from the user 102. The instruction from the user 102 is given using the operation unit 209 (for example, a controller) of the HMD 101, gestures, voice, etc.

[0023] The image generation means 308 generates a pseudo image. The pseudo image is a pseudo image of real space as viewed by the user 102 wearing the HMD 101 at the position specified by the position specification means 307. The image generation means 308 generates, as a pseudo image, a real image that is linked and recorded in the position and orientation image information 305 with the position information indicating the position specified by the position specification means 307. However, if different orientation information is linked to the position information indicating the position specified by the position specification means 307, the image generation means 308 generates the pseudo image using an image set by the user 102 or an image corresponding to the median of the different orientation information. Thereafter, on the display unit 208 of the HMD 101 displaying the pseudo image, an arrow is displayed to guide the user 102 to the orientation indicated by the orientation information linked to the real image used to generate the pseudo image.

[0024] The first display means 309 displays the real image acquired by the real image acquisition means 301, that is, the real image at the current position and orientation of the user 102, on the display unit 208 of the HMD 101. The second display means 310 displays the pseudo image generated by the image generation means 308 on the display unit 208 of the HMD 101. The switching means 311 switches to displaying the position specified by the position designation means 307. If the position specified by the position designation means 307 is the current position of the user 102, the switching means 311 switches to displaying the real image. If the position specified by the position designation means 307 is not the current position of the user 102, that is, if it is the position indicated by the position information recorded in the position and orientation image information 305, the switching means 311 switches to displaying the pseudo image.

[0025] Note that when a pseudo image is displayed on the display unit 208 of the HMD 101, it becomes difficult for the user 102 wearing the HMD 101 to walk. Therefore, when the current position of the user 102 changes, the CPU 201 (first change means) of the HMD 101 switches from displaying a pseudo image to displaying a real image using the switching means 311. Furthermore, the presentation of a position by the position presentation means 306 and the specification of a position by the position specification means 307 are also performed when the first display means 309 displays a real image and when the second display means 310 displays a pseudo image.

[0026] 5A and 5B are diagrams showing an example of when the display is switched. Fig. 5A is a diagram showing an example of a scene in which the user 102 switches the display of the display unit 208 of the HMD 101 worn on the head to display a real image or a pseudo image. A, B, and C indicate positions indicated by each piece of position information recorded in the position and orientation image information 305. In the following description, "position A," "position B," and "position C" are used as the names of the respective positions.

[0027] FIG. 5(b) is a diagram showing an example of a screen when a real image of the user 102 at his / her current position is displayed on the display unit 208 of the HMD 101. 501, 502, and 503 are marks displayed by the position presenting means 306. Mark 501 indicates the location of position A, mark 502 indicates the location of position B, and mark 503 indicates the location of position C. 504 is a window showing the position at which the display unit 208 of the HMD 101 is displaying the field of view of the user 102. Hereinafter, the position of the user 102 whose field of view is displayed on the display unit 208 of the HMD 101 will be referred to as the "display position." Window 504 in FIG. 5(b) displays "Display position: 0. Current position." Note that window 504 is displayed by the position presenting means 306.

[0028] Reference numeral 505 denotes a list displayed by the position presenting means 306. Numbers and position names are displayed in association with each other in the list 505. The list 505 is an interactive 3D UI, but may also be a 2D UI. This also applies to the marks 501, 502, and 503. Reference numeral 506 denotes a hand icon displayed at the position of the hand of the user 102 detected by the hand tracking function of the HMD 101. When the hand icon 506 presses, for example, "2. Position B" or the mark 502 on the list 505 by a gesture of the user 102, the screen of FIG. 5(b) transitions to the screen of FIG. 5(c).

[0029] 5(c) is a diagram showing an example of a screen when a pseudo image at position B is displayed on the display unit 208 of the HMD 101. In this case, a mark 503 is displayed on the screen of FIG. 5(c). Also, "Display position: 2. Position B" is displayed in a window 504. Note that when the hand icon 506 is gestured by the user 102 to press, for example, "3. Position C" on the list 505 or the mark 503, a pseudo image at position C is displayed on the display unit 208 of the HMD 101.

[0030] FIG. 6 is a diagram showing an example of a method for specifying a position by the position specification means 307. Methods for specifying a position by the position specification means 307 include specification using the operation unit 209 (e.g., a controller) of the HMD 101, specification using a gesture, and specification using voice. Of these, FIG. 6 shows specification using a gesture and specification using voice. In the case of specification using a gesture, the position specification means 307 specifies one position from among the positions presented by the position presentation means 306 by recognizing a gesture by the user 102 using the hand tracking function of the HMD 101. When the hand icon 506 of the user 102 presses any number or position name on the list 505 with a hand movement, the position indicated by the pressed number or position name is specified. When the hand icon 506 of the user 102 presses any of the marks 501, 502, and 503 with a hand movement, the position indicated by the pressed mark is specified. When the user 102 waves their hand, the current position of the user 102 is specified. When the user 102 claps his / her hands (a predetermined gesture), the previously specified position is designated.

[0031] In the case of specification by voice, the position specification means 307 specifies one position from the positions presented by the position presentation means 306 by recognizing the voice of the user 102 using the voice recognition function of the HMD 101. When the user 102 speaks a number in the list 505, the position indicated by the number is specified. When the user 102 speaks a position name in the list 505, the position indicated by the position name is specified. When the user 102 speaks "go back," the current position of the user 102 is specified. When the user 102 speaks "return to previous display position" (in a predetermined voice), the position specified one position before is specified. Note that if the specified position is the current position of the user 102, a real image at the current position of the user 102 is displayed on the display unit 208 of the HMD 101. If the specified position is not the current position of the user 102, that is, if the specified position is a position indicated by position information recorded in the position and orientation image information 305, a pseudo image at the position is displayed on the display unit 208 of the HMD 101.

[0032] FIG. 7 is a flowchart showing the flow when position and orientation information and a real image are recorded in the position and orientation image information 305 of the HMD 101. The flowchart of FIG. 7 is realized by the CPU 201 reading out a program recorded in the recording unit 206, expanding it in the RAM 203, and executing it. When the flowchart of FIG. 7 starts, in step S701, the CPU 201 starts acquiring a real image using the real image acquisition means 301. Furthermore, the CPU 201 (guidance display means) causes the recording position guidance display means 303 to display guidance on the shooting position and shooting range on the display unit 208 of the HMD 101. Furthermore, the CPU 201 (first guiding means) causes the recording position guidance display means 303 to display an arrow 105 (first information) on the display unit 208 of the HMD 101. In step S702, the CPU 201 (information acquisition means) acquires the current position and orientation information of the user 102 using the user position and orientation acquisition means 302.

[0033] In step S703, the CPU 201 causes the recording means 304 to acquire a real image at the position and orientation indicated by the position and orientation information acquired in step S702 from the real image acquired by the real image acquisition means 301. Furthermore, the CPU 201 (association recording means) causes the recording means 304 to associate the real image acquired in this manner with the position and orientation information acquired in step S702 and record it in the position and orientation image information 305. Note that, as described above, the recording means 304 performs recording in step S703 in response to a cue from the user 102 to start recording or in response to the current position of the user 102 remaining the same for a certain period of time. Thereafter, the flowchart in FIG. 7 ends.

[0034] FIG. 8 is a flowchart showing a flow when the display is switched between a real image and a pseudo image in the HMD 101. The flowchart of FIG. 8 (control method of an information processing device) is realized by the CPU 201 (computer) reading out a program recorded in the recording unit 206, expanding it in the RAM 203, and executing it. This also applies to the flowchart of FIG. 9 described later. When the flowchart of FIG. 8 starts, in step S801, the CPU 201 starts acquiring a real image using the real image acquisition means 301. In step S802, the CPU 201 causes the first display means 309 to display the real image acquired by the real image acquisition means 301 on the display unit 208 of the HMD 101. As a result, a real image of the current position of the user 102 is displayed on the display unit 208 of the HMD 101. In step S803, the CPU 201 (position acquisition means) acquires the current position of the user 102 from the position and orientation information acquired by the user position and orientation acquisition means 302 (position acquisition step).

[0035] In step S804, the CPU 201 (presentation means) causes the position presenting means 306 to display marks 501, 502, 503, a list 505, and the like (presentation step). As a result, the position presenting means 306 presents the positions indicated by the position information recorded in the position and orientation image information 305 and the current position of the user 102 using the marks 501, 502, 503, and the list 505. Note that if multiple pieces of position information indicating the same position are recorded in the position and orientation image information 305, the position presenting means 306 presents only one of the positions. In step S805, the CPU 201 (designation means) causes the position designation means 307 to designate a position in response to an instruction from the user 102 using the operation unit 209 (e.g., a controller) of the HMD 101, a gesture, voice, or the like (designation step). In this way, the position designation means 307 acquires the position designated by the user 102 (hereinafter referred to as the "designated position") from among the positions presented by the position presenting means 306.

[0036] In step S806, the CPU 201 determines, using the position designation means 307, whether the designated position is different from the display position. If the CPU 201 determines, using the position designation means 307, that the designated position is different from the display position, the process proceeds to step S807. On the other hand, if the CPU 201 determines, using the position designation means 307, that the designated position is not different from the display position, the process returns to step S803. As a result, the determination process of step S806 is repeated until the designated position is different from the display position. In step S807, the CPU 201 determines, using the position designation means 307, whether the designated position is the current position of the user 102. If the CPU 201 determines, using the position designation means 307, that the designated position is the current position of the user 102, the process proceeds to step S814, which will be described later. On the other hand, if the CPU 201 determines, using the position designation means 307, that the designated position is not the current position of the user 102, the process proceeds to step S808. That is, if the position specified by the user 102 is the position indicated by the position information recorded in the position and orientation image information 305, the process proceeds to step S808.

[0037] In step S808, the CPU 201 (generation means) causes the image generation means 308 to generate a pseudo image (generation process). At that time, the image generation means 308 generates the pseudo image using a real image that is associated with and recorded in the position and orientation image information 305 for the position information indicating the specified position acquired in step S805. In step S809, the CPU 201 causes the switching means 311 to switch the display of the real image at the current position of the user 102 to the display of the pseudo image generated in step S808. In step S810, the CPU 201 (second display control means) causes the second display means 310 to display the pseudo image switched in step S809 on the display unit 208 of the HMD 101 (second display control process). As a result, the pseudo image at the specified position acquired in step S805 is displayed on the display unit 208 of the HMD 101.

[0038] In step S811, the CPU 201 acquires current posture information of the user 102 using the user position / posture acquisition means 302. In step S812, the CPU 201 determines whether the posture information acquired in step S811 by the image generation means 308 matches the posture information associated with the real image used to generate the pseudo image in step S808. If the CPU 201 determines that the posture information acquired in step S811 by the image generation means 308 matches the posture information associated with the real image used to generate the pseudo image in step S808, the process returns to step S803.

[0039] On the other hand, if the CPU 201 determines that the posture information acquired by the image generation means 308 in step S811 does not match the posture information associated with the real image used to generate the pseudo image in step S808, the process proceeds to step S813. In step S813, the CPU 201 (second guiding means) causes the image generation means 308 to display an arrow (second information) on the display unit 208 of the HMD 101. Note that the arrow is displayed in this case to guide the user 102 to the posture indicated by the posture information associated with the real image used by the image generation means 308 to generate the pseudo image. Thereafter, the process returns to step S811. Note that, as described above, when a pseudo image is displayed on the display unit 208 of the HMD 101, the CPU 201 switches from displaying the pseudo image to displaying the real image by the switching means 311 when the current position of the user 102 changes.

[0040] In step S814, the CPU 201 switches the display of the pseudo image to the display of the real image acquired by the real image acquisition means 301, using the switching means 311. In step S815, the CPU 201 (first display control means) causes the first display means 309 to display the real image switched in step S814 on the display unit 208 of the HMD 101 (first display control step). As a result, the display unit 208 of the HMD 101 displays the real image at the current position of the user 102. Thereafter, the process returns to step S803.

[0041] 9 is a flowchart showing the flow when the display is switched between a real image and a pseudo image in the HMD 101, specifically focusing on the flow when a position is specified by a gesture by the user 102. When the flowchart in FIG. 9 starts, in step S901, the CPU 201 acquires the gesture of the user 102 using the position specification means 307. In step S902, the CPU 201 determines, using the position specification means 307, whether the gesture of the user 102 is a pressing action. If the CPU 201 determines, using the position specification means 307, that the gesture of the user 102 is a pressing action, the process proceeds to step S903. On the other hand, if the CPU 201 determines, using the position specification means 307, that the gesture of the user 102 is not a pressing action, the process proceeds to step S909, which will be described later.

[0042] In step S903, the CPU 201 determines whether the position specified by the position specification means 307 with a pressing gesture by the user 102 (hereinafter referred to as the "specified position") is the current position of the user 102. If the CPU 201 determines that the specified position is the current position of the user 102 using the position specification means 307, the process proceeds to step S907, which will be described later. On the other hand, if the position specification means 307 determines that the specified position is not the current position of the user 102, the process proceeds to step S904. In other words, if the specified position is the position indicated by the position information recorded in the position and orientation image information 305, the process proceeds to step S904.

[0043] Steps S904 to S906 are similar to steps S808 to S810 in FIG. 8, and therefore detailed description thereof will be omitted. Thereafter, the process returns to step S901. Furthermore, steps S907 to S908 are similar to steps S814 to S815 in FIG. 8, and therefore detailed description thereof will be omitted. Thereafter, the process returns to step S901. In step S909, the CPU 201 determines, using the position designation means 307, whether the gesture of the user 102 is a waving motion. If the CPU 201 determines, using the position designation means 307, that the gesture of the user 102 is a waving motion, the process proceeds to step S907 described above. On the other hand, if the CPU 201 determines, using the position designation means 307, that the gesture of the user 102 is not a waving motion, the process proceeds to step S910.

[0044] In step S910, the CPU 201 determines, using the position designation means 307, whether the gesture of the user 102 is a clapping motion. If the CPU 201 determines, using the position designation means 307, that the gesture of the user 102 is a clapping motion, the process proceeds to step S911. On the other hand, if the CPU 201 determines, using the position designation means 307, that the gesture of the user 102 is not a clapping motion, the process returns to step S901. In step S911, the CPU 201 determines whether the position designated immediately before by the position designation means 307 is the current position of the user 102. If the CPU 201 determines that the position designated immediately before by the position designation means 307 is the current position of the user 102, the process proceeds to step S907 described above. As a result, a real image of the current position of the user 102 is displayed on the display unit 208 of the HMD 101.

[0045] On the other hand, if the CPU 201 determines that the position specified previously by the position specifying means 307 is not the current position of the user 102, the process proceeds to step S912. That is, if the position specified previously is the position indicated by the position information recorded in the position and orientation image information 305, the process proceeds to step S912. In step S912, the CPU 201 sets the position specified previously by the position specifying means 307 as the specified position. Thereafter, the process proceeds to step S904 described above. As a result, a pseudo image at the position specified previously is displayed on the display unit 208 of the HMD 101. Note that when a position is specified by the voice of the user 102, the display is switched between the real image and the pseudo image in the same manner as in the flowchart of FIG. 9. This concludes the description of the first embodiment.

[0046] Second Embodiment A second embodiment will be described below with reference to FIGS. 10 and 11. Note that in the second embodiment, differences from the first embodiment will be described. In the HMD 101 of the first embodiment, a real image at the current position of the user 102 and a pseudo image at a position designated by the user 102 are switched and displayed. However, when the user 102 wearing the HMD 101 is experiencing mixed reality, the user 102 will see a virtual object that exists in a virtual space of mixed reality. Therefore, in the second embodiment, an example will be described in which a virtual object is displayed in a real image at the current position of the user 102 or a pseudo image at a position designated by the user 102.

[0047] 10 is a block diagram showing the functional configuration of the HMD 101 in the second embodiment. A virtual object position and orientation acquisition means 1001 acquires three-dimensional position and orientation information of a virtual object in a mixed reality virtual space. A virtual CG rendering means 1002 acquires a CG image by rendering virtual CG using orientation information of the user 102 at a position specified by the position specification means 307 and position and orientation information of the virtual object.

[0048] The image synthesis means 1003 synthesizes the pseudo image generated by the image generation means 308 with the CG image generated by the virtual CG rendering means 1002. In this way, the image synthesis means 1003 acquires a synthesized image (hereinafter referred to as a "composite pseudo image") by synthesizing the pseudo image and the CG image at the position specified by the position specification means 307. The third display means 1004 displays a mixed reality image at the current position and posture of the user 102 on the display unit 208 of the HMD 101, using the real image acquired by the real image acquisition means 301 and the CG image acquired by the virtual CG rendering means 1002. The fourth display means 1005 displays the composite pseudo image acquired by the image synthesis means 1003 on the display unit 208 of the HMD 101. In other words, the fourth display means 1005 displays the composite pseudo image at the position specified by the position specification means 307 on the display unit 208 of the HMD 101.

[0049] The switching means 1006 switches to displaying the position specified by the position specifying means 307. If the position specified by the position specifying means 307 is the current position of the user 102, the switching means 1006 switches to displaying a mixed reality image. If the position specified by the position specifying means 307 is not the current position of the user 102, that is, if the position is the position indicated by the position information recorded in the position and orientation image information 305, the switching means 1006 switches to displaying a composite pseudo image. Note that if a composite pseudo image is displayed on the display unit 208 of the HMD 101, it becomes difficult for the user 102 wearing the HMD 101 to walk. Therefore, if the current position of the user 102 changes, the CPU 201 (second change means) of the HMD 101 switches from displaying the composite pseudo image to displaying the mixed reality image by the switching means 1006. Furthermore, the presentation of a position by the position presentation means 306 and the designation of a position by the position designation means 307 are also performed when the third display means 1004 displays a mixed reality image and when the fourth display means 1005 displays a mixed pseudo image.

[0050] FIG. 11 is a flowchart showing a flow when the display is switched between a mixed reality image and a mixed pseudo image in the HMD 101. The flowchart of FIG. 11 (control method of an information processing device) is realized by the CPU 201 (computer) reading out a program recorded in the recording unit 206, expanding it in the RAM 203, and executing it. When the flowchart of FIG. 11 starts, in step S1101, the CPU 201 starts acquiring a real image using the real image acquisition means 301. In step S1102, the CPU 201 performs display processing of the mixed reality image. FIG. 12 is a flowchart showing the display processing of the mixed reality image in step S1102. When the flowchart of FIG. 12 starts, in step S1201, the CPU 201 acquires current position and orientation information of the user 102 using the user position and orientation acquisition means 302.

[0051] In step S1202, the CPU 201 acquires three-dimensional position and orientation information of the virtual object using the virtual object position and orientation acquisition means 1001. In step S1203, the CPU 201 acquires a CG image at the current position of the user 102 using the virtual CG rendering means 1002. In step S1204, the CPU 201 displays a mixed reality image at the current position and orientation of the user 102 on the display unit 208 of the HMD 101 using the third display means 1004. At this time, the third display means 1004 uses the real image acquired by the real image acquisition means 301 and the CG image acquired in step S1203. As a result, a mixed reality image at the current position of the user 102 is displayed on the display unit 208 of the HMD 101. Thereafter, the process returns to the flowchart of FIG. 11 , and proceeds to step S1103.

[0052] Returning to the description of FIG. 11, steps S1103 to S1105 are the same as steps S803 to S805 in FIG. 8, and therefore detailed description thereof will be omitted. In step S1106, the CPU 201 determines, using the position designation means 307, whether the designated position differs from the display position. If the CPU 201 determines, using the position designation means 307, that the designated position differs from the display position, the process proceeds to step S1107. On the other hand, if the CPU 201 determines, using the position designation means 307, that the designated position does not differ from the display position, the process proceeds to step S1119, which will be described later. In step S1107, the CPU 201 uses the virtual object position and orientation acquisition means 1001 to acquire three-dimensional position and orientation information of the virtual object. In step S1108, the CPU 201 (rendering means) uses the virtual CG rendering means 1002 to acquire a CG image at the designated position acquired in step S1105 (rendering step).

[0053] In step S1109, the CPU 201 determines, via the position designation means 307, whether the designated position is the current position of the user 102. If the CPU 201 determines, via the position designation means 307, that the designated position is the current position of the user 102, the process proceeds to step S1117, which will be described later. On the other hand, if the CPU 201 determines, via the position designation means 307, that the designated position is not the current position of the user 102, the process proceeds to step S1110. In other words, if the position designated by the user 102 is the position indicated by the position information recorded in the position and orientation image information 305, the process proceeds to step S1110.

[0054] In step S1110, the CPU 201 (generation means) generates a pseudo image using the image generation means 308 (generation process). At that time, the image generation means 308 generates the pseudo image using a real image that is linked and recorded in the position and orientation image information 305 with the position information indicating the specified position acquired in step S1105. In step S1111, the CPU 201 (synthesis means) uses the image synthesis means 1003 to synthesize the pseudo image acquired in step S1110 with the CG image acquired in step S1108 (synthesis process). As a result, the image synthesis means 1003 acquires a composite pseudo image at the specified position acquired in step S1105.

[0055] In step S1112, the CPU 201 switches the display of the mixed reality image at the current position of the user 102 to the display of the composite pseudo image acquired in step S1111, using the switching means 1006. In step S1113, the CPU 201 (fourth display control means) causes the fourth display means 1005 to display the composite pseudo image switched in step S1112 on the display unit 208 of the HMD 101 (fourth display control step). As a result, the display unit 208 of the HMD 101 displays the composite pseudo image at the specified position acquired in step S1105.

[0056] Steps S1114 to S1116 are the same as steps S811 to S813 in FIG. 8, and therefore detailed description thereof will be omitted. In step S1117, the CPU 201 switches the display of the composite pseudo image to the display of a mixed reality image using the switching means 1006. In step S1118, the CPU 201 (third display control means) causes the third display means 1004 to display the mixed reality image on the display unit 208 of the HMD 101 (third display control step). At this time, the third display means 1004 uses the real image acquired by the real image acquisition means 301 and the CG image acquired in step S1108. As a result, a mixed reality image at the current position of the user 102 is displayed on the display unit 208 of the HMD 101. Thereafter, the process returns to step S1102.

[0057] In step S1119, the CPU 201 determines whether a composite pseudo image is being displayed by the fourth display means 1005. If the CPU 201 determines that a composite pseudo image is being displayed by the fourth display means 1005, the process returns to step S1103. That is, if a composite pseudo image is being displayed on the display unit 208 of the HMD 101, the process returns to step S1103. On the other hand, if the CPU 201 determines that a composite pseudo image is not being displayed by the fourth display means 1005, the process returns to S1102. That is, if a mixed reality image is being displayed on the display unit 208 of the HMD 101, the process returns to step S1102. This concludes the description of the second embodiment.

[0058] <Summary> As described above, in the HMD 101 of the first embodiment, the user 102 can switch between displaying a real image at the current position and a pseudo image at the specified position by specifying one of the presented positions, without having to move on foot. Similarly, in the HMD 101 of the second embodiment, the user 102 can switch between displaying a mixed reality image at the current position and a mixed pseudo image at the specified position by specifying one of the presented positions, without having to move on foot. Therefore, the HMD 101 of the first and second embodiments can easily switch between displaying an image as viewed by the user 102 at the current position and a pseudo image as viewed at the position specified by the user 102.

[0059] <Other> Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and various modifications and variations are possible within the spirit and scope of the present invention. For example, the position designation unit 307 may designate one of the positions presented by the position presentation unit 306 using a ray extending from a controller held by the user 102 or a ray extending from the hand of the user 102. Furthermore, the position designation unit 307 may designate one of the positions presented by the position presentation unit 306 by analyzing the line of sight of the user 102 using the eye tracking function of the HMD 101. Furthermore, when designating a previously designated position in response to the user 102 clapping their hands or uttering a command to "return to the previous display position," the position designation unit 307 may exclude the current position of the user 102 from the previously designated positions. Furthermore, in the position and orientation image information 305, the real image linked to the position and orientation information of the user 102 is a still image compressed in JPEG format, but it may also be a video of the same angle.

[0060] The present invention can also be realized by a process in which a program that realizes one or more functions of each of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a dedicated processor (e.g., a circuit such as an ASIC or FPGA) that realizes one or more functions. Furthermore, the present invention can also be realized by a combination of a general-purpose processor and a dedicated processor. Note that the term "processor" refers to a processor in a broad sense and includes both general-purpose processors and dedicated processors. Furthermore, the operations of a processor may not only be performed by a single processor, but may also be performed in cooperation with multiple processors located at physically separate locations.

[0061] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) A location acquisition means for acquiring the current location of the user as a current location; a display unit for displaying a plurality of positions including the current position on a display unit; a designation means for designating a position from among the plurality of positions in response to an instruction from a user; a first display control means for displaying, on the display unit, an image acquired by an imaging unit by imaging a real space as a real image showing a field of view of the user at the current position when the position designated by the designation means is the current position; a generating means for generating a pseudo image that resembles the user's field of view at the position designated by the designating means when the position designated by the designating means is not the current position; and second display control means for displaying the pseudo image on the display unit. (Configuration 2) The information processing device according to configuration 1, further comprising a first change means for changing the display of the display unit from the pseudo image to the real image when the current position changes while the pseudo image is being displayed on the display unit. (Configuration 3) A location acquisition means for acquiring the current location of the user as a current location; a display unit for displaying a plurality of positions including the current position on a display unit; a designation means for designating a position from among the plurality of positions in response to an instruction from a user; a rendering means for rendering a CG (Computer Graphics) image representing a virtual object within the user's field of view at the position designated by the designation means; a third display control means for displaying, on the display unit, a mixed reality image using an image acquired by an imaging unit by imaging a real space and the CG image as a real image showing the user's field of view at the current position when the position designated by the designation means is the current position; a generating means for generating a pseudo image that resembles the user's field of view at the position designated by the designating means when the position designated by the designating means is not the current position; a synthesis means for synthesizing the pseudo image and the CG image to obtain a composite pseudo image; and fourth display control means for displaying the composite pseudo image on the display unit. (Configuration 4) An information processing device according to configuration 3, characterized in that it comprises a second change means for changing the display of the display unit from the composite pseudo image to the mixed reality image when the current position changes while the composite pseudo image is being displayed on the display unit. (Configuration 5) An information processing device described in any one of configurations 1 to 4, characterized in that the presentation means presents the multiple positions on the display unit by displaying marks at the locations of each of the multiple positions in the image being displayed on the display unit. (Configuration 6) The information processing device described in any one of configurations 1 to 5, characterized in that the presentation means presents the multiple locations on the display unit by displaying a list on the display unit in which the location names of each of the multiple locations are arranged. (Configuration 7) The information processing device according to any one of configurations 1 to 6, wherein the designation means designates a position from among the plurality of positions in response to an instruction given by a user operating a controller. (Configuration 8) The information processing device according to any one of configurations 1 to 7, wherein the designation means designates a position from among the plurality of positions in response to an instruction by a gesture from the user. (Configuration 9) The information processing device according to configuration 8, wherein the designation means designates the previously designated position among the plurality of positions when the user's gesture is a predetermined gesture. (Configuration 10) The information processing device according to any one of configurations 1 to 9, wherein the designation means designates a position from among the plurality of positions in response to a voice instruction from a user. (Configuration 11) The information processing device according to configuration 10, wherein the designation means designates the previously designated position among the plurality of positions when the user's voice is a predetermined voice. (Configuration 12) An information processing device according to any one of configurations 1 to 11, characterized in that the designation means designates a position from among the plurality of positions in response to an instruction by a ray extending from a user's hand or a ray extending from a controller. (Configuration 13) The information processing device according to any one of configurations 1 to 12, wherein the designation means designates a position from among the plurality of positions in response to an instruction by the user's line of sight. (Configuration 14) An information acquisition means for acquiring position information and posture information indicating the position and posture of a user; a linking recording means for linking the position information and the posture information acquired by the information acquiring means with the real image when the real image is acquired, and recording the linked information, The information processing device described in any one of configurations 1 to 13, characterized in that the generation means generates the pseudo-image using the real image that is recorded in association with the position information indicating the position specified by the specification means. (Configuration 15) The information processing device described in Configuration 14 is characterized in that, when different pieces of posture information are linked to the position information indicating the position specified by the designation means, the generation means generates the pseudo-image using an image set by the user or an image corresponding to the median of the different pieces of posture information. (Configuration 16) The information processing device according to configuration 14 or 15, wherein when the current position remains the same for a certain period of time, recording is performed by the linking recording means. (Configuration 17) The information processing device according to any one of configurations 14 to 16, characterized in that there are a plurality of combinations of data obtained by linking the position information and the posture information with the real image. (Configuration 18) An information processing device described in any one of configurations 14 to 17, characterized in that it is provided with a guidance display means for displaying guidance on the display unit indicating the shooting range or shooting position when the real image recorded by the linked recording means is acquired by the shooting unit. (Configuration 19) An information processing device described in any one of configurations 14 to 18, characterized in that it is provided with a first guidance means for displaying on the display unit first information for guiding the user to a posture when the real image recorded by the linked recording means is acquired by the photographing unit. (Configuration 20) An information processing device described in any one of configurations 14 to 19, characterized in that it includes a second guidance means for displaying on the display unit second information for guiding the user to a posture indicated by the posture information recorded in association with the real image used by the generation means. (Configuration 21) The information processing device according to any one of configurations 1 to 20, wherein the information processing device is an HMD (Head Mounted Display) including the imaging unit and the display unit. (System 1) An information processing device according to any one of configurations 1 to 21; The imaging unit; an information processing system comprising the display unit. (System 2) The information processing system according to configuration 22, wherein the photographing unit is a camera carried by the user. (System 3) The information processing system according to configuration 22, wherein the photographing unit is a 360-degree camera. (Method 1) A location acquisition step of acquiring the current location of the user as the current location; a display step of displaying a plurality of positions including the current position on a display unit; a designation step of designating a position from among the plurality of positions in response to an instruction from a user; a first display control step of displaying, on the display unit, an image acquired by an imaging unit by imaging a real space as a real image showing a field of view of the user at the current position when the position designated by the designation step is the current position; a generating step of generating a pseudo image that resembles the user's field of view at the position specified by the specifying step, when the position specified by the specifying step is not the current position; a second display control step of displaying the pseudo image on the display unit. (Method 2) a location acquisition step of acquiring the current location of the user as the current location; a display step of displaying a plurality of positions including the current position on a display unit; a designation step of designating a position from among the plurality of positions in response to an instruction from a user; a rendering step of rendering a CG image showing a virtual object that is within the user's field of view at the position designated by the designation step; a third display control step of displaying, on the display unit, a mixed reality image using an image acquired by an imaging unit by imaging the real space and the CG image as a real image showing the user's field of view at the current position when the position designated by the designation step is the current position; a generating step of generating a pseudo image that resembles the user's field of view at the position specified by the specifying step, when the position specified by the specifying step is not the current position; a synthesis step of synthesizing the pseudo image and the CG image to obtain a composite pseudo image; a fourth display control step of displaying the composite pseudo image on the display unit. (Program 1) A program for causing a computer to execute each means of the information processing device according to any one of configurations 1 to 21. [Explanation of symbols]

[0062] 101 HMD (information processing device) 201 CPU (position acquisition means) (presentation means) (designation means) (first display control means) (generation means) (second display control means) 207 Photography Department 208 Display section

Claims

1. a location acquisition means for acquiring the current location of the user as a current location; a display unit for displaying a plurality of positions including the current position on a display unit; a designation means for designating a position from among the plurality of positions in response to an instruction from a user; a first display control means for displaying, on the display unit, an image acquired by an imaging unit by imaging a real space as a real image showing a field of view of the user at the current position when the position designated by the designation means is the current position; a generating means for generating a pseudo image that resembles the user's field of view at the position designated by the designating means when the position designated by the designating means is not the current position; and second display control means for displaying the pseudo image on the display unit.

2. 2. The information processing apparatus according to claim 1, further comprising a first change means for changing the display on the display unit from the pseudo image to the real image when the current position changes while the pseudo image is being displayed on the display unit.

3. a location acquisition means for acquiring the current location of the user as a current location; a display unit for displaying a plurality of positions including the current position on a display unit; a designation means for designating a position from among the plurality of positions in response to an instruction from a user; a rendering means for rendering a CG (Computer Graphics) image representing a virtual object within the user's field of view at the position designated by the designation means; a third display control means for displaying, on the display unit, a mixed reality image using an image acquired by an imaging unit by imaging a real space and the CG image as a real image showing the user's field of view at the current position when the position designated by the designation means is the current position; a generating means for generating a pseudo image that resembles the user's field of view at the position designated by the designating means when the position designated by the designating means is not the current position; a synthesis means for synthesizing the pseudo image and the CG image to obtain a composite pseudo image; and fourth display control means for displaying the composite pseudo image on the display unit.

4. 4. The information processing apparatus according to claim 3, further comprising a second change means for changing the display on the display unit from the composite pseudo image to the mixed reality image when the current position changes while the composite pseudo image is being displayed on the display unit.

5. 4. The information processing apparatus according to claim 1, wherein the presenting means presents the plurality of positions on the display unit by displaying marks at the locations of the plurality of positions in the image being displayed on the display unit.

6. 4. The information processing apparatus according to claim 1, wherein the presenting means presents the plurality of locations on the display unit by displaying a list on the display unit in which the location names of the plurality of locations are arranged.

7. 4. The information processing apparatus according to claim 1, wherein the designation means designates a position from among the plurality of positions in response to an instruction given by a user operating a controller.

8. 4. The information processing apparatus according to claim 1, wherein the designation means designates a position from among the plurality of positions in response to an instruction by a gesture of the user.

9. 9. The information processing apparatus according to claim 8, wherein the designation means designates the previously designated position among the plurality of positions when the user's gesture is a predetermined gesture.

10. 4. The information processing apparatus according to claim 1, wherein the designation means designates a position from among the plurality of positions in response to a voice instruction from a user.

11. 11. The information processing apparatus according to claim 10, wherein the designation means designates the previously designated position among the plurality of positions when the user's voice is a predetermined voice.

12. 4. The information processing apparatus according to claim 1, wherein the designation means designates a position from among the plurality of positions in response to an instruction by a ray extending from a user's hand or a controller.

13. 4. The information processing apparatus according to claim 1, wherein the designation means designates a position from among the plurality of positions in response to an instruction by the user's line of sight.

14. an information acquisition means for acquiring position information and posture information indicating the position and posture of a user; a linking recording means for linking the position information and the posture information acquired by the information acquiring means with the real image when the real image is acquired, and recording the linked information, 4. The information processing apparatus according to claim 1, wherein the generating means generates the pseudo image using the real image that is recorded in association with the position information indicating the position designated by the designating means.

15. The information processing device according to claim 14, characterized in that, when different pieces of posture information are linked to the position information indicating the position specified by the specification means, the generation means generates the pseudo image using an image set by the user or an image corresponding to the median of the different pieces of posture information.

16. 15. The information processing apparatus according to claim 14, wherein when the current position remains the same for a certain period of time, the linking and recording means records the current position.

17. The information processing apparatus according to claim 14, wherein there are a plurality of combinations of data obtained by linking the position information and the posture information with the real image.

18. 15. The information processing device according to claim 14, further comprising a guidance display means for displaying guidance on the display unit indicating the shooting range or shooting position when the real image recorded by the linked recording means is acquired by the shooting unit.

19. 15. The information processing device according to claim 14, further comprising a first guidance means for displaying on the display unit first information for guiding the user to assume a posture when the real image recorded by the linked recording means is acquired by the photographing unit.

20. The information processing device according to claim 14, further comprising a second guiding means for displaying on the display unit second information for guiding the user to a posture indicated by the posture information recorded in association with the real image used by the generating means.

21. 4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a head mounted display (HMD) including the image capturing unit and the display unit.

22. The information processing device according to claim 1 or 3; The imaging unit; an information processing system comprising the display unit.

23. 23. The information processing system according to claim 22, wherein the photographing unit is a camera carried by a user.

24. 23. The information processing system according to claim 22, wherein the photographing unit is a 360-degree camera.

25. a position acquisition step of acquiring the current position of the user as a current position; a display step of displaying a plurality of positions including the current position on a display unit; a designation step of designating a position from among the plurality of positions in response to an instruction from a user; a first display control step of displaying, on the display unit, an image acquired by an imaging unit by imaging a real space as a real image showing a field of view of the user at the current position when the position designated by the designation step is the current position; a generating step of generating a pseudo image that resembles the user's field of view at the position specified by the specifying step, when the position specified by the specifying step is not the current position; a second display control step of displaying the pseudo image on the display unit.

26. a position acquisition step of acquiring the current position of the user as a current position; a display step of displaying a plurality of positions including the current position on a display unit; a designation step of designating a position from among the plurality of positions in response to an instruction from a user; a rendering step of rendering a CG image showing a virtual object within the user's field of view at the position specified by the specifying step; a third display control step of displaying, on the display unit, a mixed reality image using an image acquired by an imaging unit by imaging the real space and the CG image as a real image showing the user's field of view at the current position when the position designated by the designation step is the current position; a generating step of generating a pseudo image that resembles the user's field of view at the position specified by the specifying step, when the position specified by the specifying step is not the current position; a synthesis step of synthesizing the pseudo image and the CG image to obtain a composite pseudo image; a fourth display control step of displaying the composite pseudo image on the display unit.

27. 4. A program for causing a computer to execute each means of the information processing apparatus according to claim 1.

Images