Electronic device, method for controlling electronic device, and program

Abstract

This technology provides a method for displaying virtual images effectively with minimal processing load. [Solution] The electronic device of the present invention comprises a first acquisition means for acquiring a background image, a second acquisition means for acquiring a virtual image, and a synthesis means for generating a composite image to be displayed on a display unit by combining the virtual image with the background image. The synthesis means is characterized in that, at a first time, it synthesizes the virtual image to a first position on the background image, and at a second time after the first time, without acquiring a new virtual image, it synthesizes the virtual image synthesized on the background image at the first time to a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time to the second time.

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method for an electronic device, and a program.

Background Art

[0002] As technologies for seamlessly and real-time fusing the real space and the virtual space, Mixed Reality (MR) technology and Augmented Reality (AR) technology are known. There is also Virtual Reality (VR) technology that provides only a virtual space. These technologies are used in, for example, a Head-Mounted Display (HMD).

[0003] For a user of an HMD, a virtual object may be displayed as if it were present at a fixed position. Generally, information in the real space is analyzed, a coordinate system of the virtual space associated with the real space is determined, and a virtual image is generated and displayed based on an operation of self-position estimation in the coordinate system.

[0004] Patent Document 1 discloses a technique of setting a reference plane in the real space and aligning the position of the corresponding plane of a virtual object with the reference plane. Patent Document 2 discloses a technique of determining the depth of a virtual object in consideration of the depth of the real space.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, conventional technology requires the repeated generation (updating and regeneration) of virtual images based on the results of self-localization, resulting in a high processing load. This problem cannot be solved even by using the technologies disclosed in Patent Documents 1 and 2.

[0007] The present invention aims to provide a technology that can suitably display virtual images with a low processing load. [Means for solving the problem]

[0008] The electronic device of the present invention comprises a first acquisition means for acquiring a background image, a second acquisition means for acquiring a virtual image, and a synthesis means for generating a composite image to be displayed on a display unit by combining the virtual image with the background image, wherein the synthesis means combines the virtual image at a first position on the background image at a first time, and at a second time after the first time, without acquiring a new virtual image, combines the virtual image combined with the background image at the first time at a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time to the second time. [Effects of the Invention]

[0009] According to the present invention, virtual images can be displayed effectively with a low processing load. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration and processing flow of Embodiment 1. [Figure 2] This block diagram shows the first configuration and processing flow that supplements Embodiment 1. [Figure 3] This block diagram shows a second configuration and processing flow that supplements Embodiment 1. [Figure 4] This figure shows an example of a conventional virtual image synthesis process. [Figure 5] This figure shows an example of a conventional virtual image synthesis process. [Figure 6] This figure shows an example of the virtual image synthesis process according to Embodiment 1. [Figure 7] This figure shows an example of the virtual image synthesis process according to Embodiment 2. [Figure 8] This figure shows an example of the virtual image synthesis process according to Embodiment 3. [Modes for carrying out the invention]

[0011] <Embodiment 1> Embodiment 1 of the present invention will now be described with reference to the drawings. Figure 1 is a block diagram showing the configuration and processing flow of the electronic device 100 according to Embodiment 1. The electronic device 100 is a display device such as a head-mounted display (HMD).

[0012] The imaging unit 11 is, for example, a camera provided on the electronic device 100, and acquires a real image (background image) by imaging the real space. The generation unit 12 generates a virtual image that includes a virtual object. The electronic device 100 has one imaging unit 11 and one generation unit 12, but the number of imaging units and generation units is not particularly limited. The imaging unit and generation unit may be provided on an external device. In the following description, the background image acquired by the imaging unit 11 is assumed to be a real image obtained by imaging the real space. However, the background image is not limited to a real image, and a generation unit that generates an image of a virtual space as a background image may be used instead of the imaging unit 11. Also, in the following description, the virtual image acquired by the generation unit 12 is assumed to be an image of a PC screen, which is a virtual object. However, the virtual image is not limited to an image of a PC screen, and images of various virtual objects may be used as virtual images.

[0013] Input unit 13 acquires a background image from the imaging unit 11 and outputs it to the synthesis unit 15. Input unit 14 acquires a virtual image from the generation unit 12 and outputs it to the synthesis unit 15. The electronic device 100 has two input units, input units 13 and 14, but the number of input units may be changed depending on the number of imaging units and generation units, etc.

[0014] The synthesizing unit 15 synthesizes the background image output from the input unit 13 and the virtual image output from the input unit 14. In the synthesizing unit 15, by synthesizing the virtual image with the background image, a synthesized image can be generated as if a virtual object exists in the real space. The method (process) of synthesizing the background image and the virtual image will be described later using FIG. 6.

[0015] The display unit 16 displays the synthesized image generated by the synthesizing unit 15 to the user. Note that the display unit 16 may be provided in an external device. The display unit 16 may be part or all of an HMD connected to the electronic device 100. In that case, the electronic device 100 may be an information processing device such as a PC. The display unit 16 is not limited to an HMD, and may be part or all of a display device that can be moved by the user, such as a handheld display (HHD), a tablet, or a smartphone.

[0016] The control unit 17 controls the entire electronic device 100. The control unit 17 detects the movement of the electronic device 100 and issues an instruction to the synthesizing unit 15. For example, the control unit 17 continuously acquires information regarding the position and orientation of the display unit 16 from the time when the control unit 17 starts processing (for example, when the electronic device 100 is activated). Based on the acquired information regarding the position and orientation of the display unit 16, the control unit 17 determines the relative position and orientation of the virtual object with respect to the display unit 16 and the display size of the virtual object, and stores the information regarding the display of the virtual object in the memory 18 described later. The control unit 17 reads the information regarding the display of the virtual object from the memory 18 and issues an instruction to the synthesizing unit 15 on how to display the virtual object in terms of position, orientation, and display size.

[0017] The memory 18 is a storage unit that is connected to each part of the electronic device 100 via a communication bus and can store (memorize) information regarding the position and orientation of the display unit 16, information regarding the position and orientation of the virtual object, and various images.

[0018] So far, the imaging unit 11, the generation unit 12, the input units 13, 14, the synthesis unit 15, the display unit 16, the control unit 17, and the memory 18 have been described. However, all of these components may be provided in one device, or some of the components may be provided in separate devices. For example, as described above, the imaging unit 11 and the generation unit 12 may be provided in an external device. In that case, the input units 13, 14 may be input I / Fs for acquiring data from the external device.

[0019] Note that the method for estimating the position and orientation of the HMD is not particularly limited. For example, the position and orientation of the HMD may be estimated using an inertial sensor such as an acceleration sensor, or the position and orientation of the HMD may be estimated using an image of the real space. This will be described in detail below with reference to FIGS. 2 and 3.

[0020] FIG. 2 is a block diagram showing the first configuration and the processing flow for supplementing Embodiment 1. In the description of FIG. 2, the operations of the imaging unit 11, the generation unit 12, the input units 13, 14, the synthesis unit 15, the display unit 16, the control unit 17, and the memory 18 are the same as those described above, and thus the description thereof will be omitted.

[0021] The inertial measurement unit 21 is an inertial sensor that acquires information regarding the position or orientation of the display unit 16 by detecting the movement of the display unit 16. The inertial measurement unit 21 (inertial sensor) may be an IMU (Inertial Measurement Unit) that includes a three-axis acceleration sensor and a three-axis angular velocity sensor, etc. In the inertial measurement unit 21, it is possible to detect the direction of gravity by the acceleration sensor, or to detect the change in the orientation of the display unit 16 using the angular velocity sensor. The inertial measurement unit 21 outputs the acquired information regarding the position and orientation of the display unit 16 to the control unit 17. Based on the information acquired from the inertial measurement unit 21, the control unit 17 issues an instruction to the synthesis unit 15 to synthesize the virtual image with the background image.

[0022] Figure 3 is a block diagram showing a second configuration and processing flow that supplements Embodiment 1. In explaining Figure 3, the operation of the imaging unit 11, generation unit 12, input units 13, 14, synthesis unit 15, display unit 16, control unit 17, and memory 18 is the same as described above, so the explanation will be omitted.

[0023] The image analysis unit 31 analyzes the background image captured by the imaging unit 11 and detects the movement of the display unit 16 to obtain information regarding the position and orientation of the display unit 16. The image analysis unit 31 can perform optical flow (motion vector detection) to detect movement by analyzing the direction in which feature points in the captured background image have moved from the image of the previous frame. However, although this explanation describes a case where the movement of the imaging unit 11 is accompanied by the movement of the display unit 16 because the imaging unit 11 is provided in the electronic device 100, the movement of the display unit 16 may also be detected by capturing and analyzing the display unit 16 from an external source. The image analysis unit 31 outputs the acquired information regarding the position and orientation of the display unit 16 to the control unit 17. Based on the information acquired from the image analysis unit 31, the control unit 17 instructs the synthesis unit 15 to synthesize a virtual image onto the background image.

[0024] When using the inertial measurement unit 21, the processing load for calculations to acquire information about the position and orientation of the display unit 16 is small, but there is a possibility that movements unrelated to the user's intentions may be detected when the user viewing the display unit 16 is in a vehicle or the like. Also, when using the image analysis unit 31, movement can be stably detected in bright places where imaging is possible with the imaging unit 11. However, motion may not be detected reliably in dark places where imaging is difficult. For this reason, both the inertial measurement unit 21 and the image analysis unit 31 may be combined to acquire information regarding the position and orientation of the display unit 16.

[0025] Using Figures 4 and 5, we will explain a conventional method for combining composite images. Figures 4 and 5 show an example of a conventional synthesis process (synthesis method) for combining a background image and a virtual image. The background image 41 is a real image captured by the imaging unit, and it shows a human face.

[0026] The virtual image 42 is an image generated by the generation unit, and a virtual object 421 (the region of the virtual object) corresponding to the PC screen exists within the virtual image 42 (upper left). Furthermore, the region of the virtual image 42 other than the region of the virtual object 421 is considered transparent.

[0027] The composite image 43 is an image created by the composite unit by combining the background image 41 and the virtual image 42. When the process of compositing the virtual image 42 onto the background image 41 (alpha blending using weights corresponding to the transparency of the virtual image 42) is performed, the pixel values ​​of the background image 41 are used in the transparent areas of the virtual image 42. In the area of ​​the virtual object 421, the pixel values ​​of the virtual image 42 are used. As a result, the user sees the virtual object 421 in front of the face of the person reflected in the background image 41. In order to make the background image 41 visible through the virtual object 421, the pixel values ​​obtained by combining the pixel values ​​of the background image 41 and the pixel values ​​of the virtual image 42 may be used in the area of ​​the virtual object 421. The background image 41 and the virtual image 42 are the same size, and for example, the background image 41 and the virtual image 42 are composited so that the upper left corner of the background image 41 and the upper left corner of the virtual image 42 overlap.

[0028] Background image 51 is a real-world image taken when the imaging unit is angled slightly upward compared to the field of view of background image 41, and the faces of the people in the image have moved downward from their positions in background image 41.

[0029] The virtual image 52 is an image regenerated by the generation unit based on a change in the orientation of the imaging unit so that it is slightly angled upward compared to the field of view of the virtual image 42. In the virtual image 52, the virtual object 521, which corresponds to the PC screen, is positioned lower than its position in the composite image 43 (the position of the virtual object 421) due to self-localization calculations such as SLAM.

[0030] The composite image 53 is an image created by combining the background image 51 and the virtual image 52. The process for generating the composite image 53 is the same as the process for generating the composite image 43. When the image transitions from composite image 43 to composite image 53, the virtual object corresponding to the PC screen is processed to follow the left side of the person's face shown in the background image, based on the change in the display unit's orientation. As a result, the user perceives the virtual object corresponding to the PC screen as being in a fixed position. However, this conventional compositing process involves a heavy processing load because self-position estimation calculations and virtual image regeneration are performed each time the background image transitions.

[0031] Figure 6 shows an example of the composite processing of a background image and a virtual image according to Embodiment 1. The background image 61 is a real image captured with the display unit 16 and the associated imaging unit 11 facing slightly upwards, similar to the background image 51, and the background image 61 is the same image as the background image 51.

[0032] The virtual image 62 is an image of only the region of the virtual object 621 generated by the generation unit 12. The virtual image 62 is stored in the memory 18 after being generated by the generation unit 12. The virtual image 62 represents the virtual object 621 which corresponds to the PC screen. Even if the position and orientation of the display unit 16 changes, the virtual image 62 is not regenerated.

[0033] The extended virtual image 63 is an image obtained by extending the virtual image 62 such that the area other than the virtual object 621 becomes a transparent area. If the virtual image 62 input to the input unit 14 and stored in the memory 18 is smaller than the background image 61, the input unit 14 will input an image larger than the background image 61. To achieve this, a transparent area is added to the virtual image 62. As a result, the input unit 14 generates an extended virtual image 63 and stores the extended virtual image 63 in the memory 18. The control unit 17 reads information regarding the display of the virtual object 621 from the memory 18 and determines which area of ​​the extended virtual image 63 stored in the memory 18 the composite unit 15 will read (cut out). The area 631 in the extended virtual image 63 is the same size as the background image 61 and is the area to be read by the composite unit 15. If the virtual image 62 is larger than the background image 61, the control unit 17 instructs the composite unit 15 to read a portion of the virtual image 62.

[0034] The composite image 64 is an image created by combining the background image 61 and the region 631. In the composite image 64, a virtual object 621 is displayed to the left of the human face shown in the background image 61. The compositing unit 15 changes the position of the region 631 cut out from the extended virtual image 63 in accordance with the change in the position and orientation of the display unit 16 in accordance with the change in time. After compositing the region 631 with the background image 61, the compositing unit 15 updates the composite image 64 by changing the position of the region 631 in accordance with the change in the position and orientation of the display unit 16. Assume that at the first time, a first region the same size as the background image 61 is cut out from the extended virtual image 63 and composited with the background image 61. At the second time, which is after the first time, without acquiring a new virtual image, a second region, which is the first region moved in accordance with the change in the position and orientation of the display unit 16 from the first time to the second time, is cut out from the extended virtual image 63 and composited with the background image 61.

[0035] In this way, a composite image 64 similar to the composite image 53 can be created without regenerating the virtual image 62. Since the method of Embodiment 1 does not require the regeneration of the virtual image 62, the virtual image can be displayed suitably with a small processing load.

[0036] <Embodiment 2> Hereinafter, Embodiment 2 of the present invention will be described with reference to Figure 7. Figure 7 is a diagram showing an example of the synthesis process between a background image and a virtual image according to Embodiment 2. In Embodiment 2, the configuration of the apparatus is the same as in Embodiment 1. In describing Embodiment 2, the differences from the synthesis process according to Embodiment 1 will be explained, and detailed explanations of similar points will be omitted.

[0037] Background image 71 is the same image as background image 61. Virtual image 72 is the same image as virtual image 62. Virtual object 721 is the same virtual object as virtual object 621. The imaging unit 11 also stores the captured background image 71 in the memory 18.

[0038] If the virtual image 72 input to the input unit 14 is smaller than the background image 71, the control unit 17 instructs the compositing unit 15 to directly overwrite the background image 71 stored in the memory 18 with the virtual image 72. The control unit 17 reads information regarding the display of the virtual object 721 from the memory 18 and determines the area of ​​the background image 71 to which the virtual image 72 will be written. If the virtual image 72 is larger than the background image 71, the control unit 17 instructs the compositing unit 15 to directly overwrite a portion of the background image 71 stored in the memory 18 with the virtual image 72. The compositing unit 15 overwrites the background image 71 stored in the memory 18 with the virtual image 72 according to the instructions from the control unit 17. As a result, the composite image 74 is generated in the memory 18. The compositing unit 15 changes the position where the virtual image 72 is written according to the change in the position and orientation of the display unit 16 in accordance with the change in time. The compositing unit 15 updates the composite image 74 by writing the virtual image 72 to the background image 71 and then changing the writing position of the virtual image 72 in accordance with the change in the position and orientation of the display unit 16. Assume that at the first time step, the virtual image 72 is composited to the first position of the background image 71. At the second time step, which is later than the first time step, no new virtual image is acquired. At the second time step, the virtual image 72 that was composited to the background image 71 at the first time step is composited to the second position of the background image 71, which has been moved from its first position in accordance with the change in the position and orientation of the display unit 16 from the first time step to the second time step.

[0039] In this way, it is possible to create a composite image 74 similar to the composite image 53 without regenerating the virtual image 72. Since the method of Embodiment 2 does not require the regeneration of the virtual image 72, the virtual image can be displayed suitably with a small processing load.

[0040] <Embodiment 3> Hereinafter, Embodiment 3 of the present invention will be described with reference to Figure 8. Figure 8 is a diagram showing an example of the synthesis process between a background image and a virtual image according to Embodiment 3. In Embodiment 3, the configuration of the apparatus is the same as in Embodiment 1. In describing Embodiment 3, the differences from the synthesis process according to Embodiment 1 will be explained, and detailed explanations of similar points will be omitted.

[0041] Background image 81 is the same image as background image 61. Virtual image 82 is the same image as virtual image 62. Virtual object 821 is the same virtual object as virtual object 621.

[0042] The composite image 83 is an image created by combining the background image 81 and the virtual image 82. The composite image 83 is a composite image generated by a dedicated circuit for combining multiple images, which is provided by the combining unit 15. The control unit 17 reads information regarding the display of the virtual object 821 from the memory 18 and determines the area in which the virtual image 82 will be combined by the circuit. If the virtual image 82 input to the input unit 14 is smaller than the background image 81, the control unit 17 instructs the combining unit 15 to combine the virtual image 82 by superimposing it onto the background image 81. If the virtual image 82 is larger than the background image 81, the control unit 17 instructs the combining unit 15 to combine the virtual image 82 by superimposing a portion of it onto the background image 81. In the composite image 83, the virtual object 821 is displayed to the left of the human face captured in the background image 81. The combining unit 15 changes the position in which the virtual image 82 is combined onto the background image 81 according to the change in the position and orientation of the display unit 16 in accordance with the change in time. The synthesis unit 15 synthesizes the virtual image 82 with the background image 81, and then updates the synthesized image 83 by changing the synthesis position of the virtual image 82 in accordance with the change in the position and orientation of the display unit 16.

[0043] In this way, it is possible to create a composite image 83 similar to the composite image 53 without regenerating the virtual image 72. Since there is no need to regenerate the virtual image 82 in the method of Embodiment 3, the virtual image can be displayed suitably with a small processing load.

[0044] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

[0045] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). Multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) may share the processing to control the entire device.

[0046] Furthermore, the above-mentioned processors are processors in a broad sense, including general-purpose processors and specialized processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Specialized processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0047] Furthermore, although embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0048] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit that implements one or more functions.

[0049] This embodiment includes the following configurations, methods, and programs. (Composition 1) A first acquisition means for acquiring a background image, A second acquisition means for acquiring a virtual image, A synthesis means that generates a composite image to be displayed on the display unit by compositing the virtual image onto the background image. It has, The synthesis means synthesizes the virtual image to a first position on the background image at a first time, and at a second time, later than the first time, without acquiring a new virtual image, synthesizes the virtual image synthesized on the background image at the first time to a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time to the second time. An electronic device characterized by the following features. (Configuration 2) The aforementioned virtual image consists of a region of a virtual object and a transparent region, and is larger than the aforementioned background image. The combining means, at the first time, cuts out a first region of the same size as the background image from the virtual image and combines it with the background image, and at the second time, cuts out a second region from the virtual image, which is obtained by moving the first region in accordance with the change in the position and orientation of the display unit from the first time to the second time, and combines it with the background image. The electronic device according to configuration 1, characterized by the features described above. (Composition 3) The aforementioned virtual image is an image of only the region of the virtual object. The electronic device according to configuration 1, characterized by the features described above. (Composition 4) The first acquisition means acquires the background image and stores it in the storage unit. The synthesis means stores the virtual image in the storage unit so as to overwrite a portion of the background image. The electronic device according to configuration 3, characterized by the features described above. (Composition 5) The synthesis means includes a dedicated circuit for compositing the virtual image onto the background image. The electronic device according to configuration 3, characterized by the features described above. (Composition 6) The system further includes an inertial sensor that detects the movement of the display unit and acquires information regarding the position and orientation of the display unit. An electronic device according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The system further includes an image analysis means for acquiring information regarding the position and orientation of the display unit by analyzing the background image. An electronic device according to any one of configurations 1 to 5, characterized by the above. (Composition 8) The aforementioned background image is an image captured from real space. An electronic device according to any one of configurations 1 to 7, characterized by the features described herein. (Composition 9) The aforementioned background image is an image of a virtual space. An electronic device according to any one of configurations 1 to 7, characterized by the features described herein. (method) The first acquisition step is to obtain the background image, The second acquisition step involves obtaining a virtual image, A synthesis step is to generate a composite image to be displayed on the display unit by combining the virtual image with the background image. It has, The synthesis step involves, at a first time step, compositing the virtual image onto a first position on the background image, and at a second time step, later than the first time step, compositing the virtual image composited onto the background image at the first time step onto a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time step to the second time step, without acquiring a new virtual image. A method for controlling electronic equipment characterized by the following features. (program) A program for causing a computer to function as one of the electronic devices described in any one of configurations 1 to 9. [Explanation of Symbols]

[0050] 11: Imaging unit 12: Generation unit 15: Synthesis unit

Claims

1. A first acquisition means for acquiring a background image, A second acquisition means for acquiring a virtual image, A synthesis means that generates a composite image to be displayed on the display unit by compositing the virtual image onto the background image. It has, The synthesis means synthesizes the virtual image to a first position on the background image at a first time, and at a second time, which is later than the first time, without acquiring a new virtual image, synthesizes the virtual image synthesized on the background image at the first time to a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time to the second time. An electronic device characterized by the following features.

2. The aforementioned virtual image consists of a region of a virtual object and a transparent region, and is larger than the aforementioned background image. The combining means, at the first time, cuts out a first region of the same size as the background image from the virtual image and combines it with the background image, and at the second time, cuts out a second region from the virtual image, which is obtained by moving the first region in accordance with the change in the position and orientation of the display unit from the first time to the second time, and combines it with the background image. The electronic device according to feature 1.

3. The aforementioned virtual image is an image of only the region of the virtual object. The electronic device according to feature 1.

4. The first acquisition means acquires the background image and stores it in the storage unit. The synthesis means stores the virtual image in the storage unit so as to overwrite a portion of the background image. The electronic device according to feature 3.

5. The synthesis means includes a dedicated circuit for compositing the virtual image onto the background image. The electronic device according to feature 3.

6. The system further includes an inertial sensor that detects the movement of the display unit and acquires information regarding the position and orientation of the display unit. The electronic device according to feature 1.

7. The system further includes an image analysis means for acquiring information regarding the position and orientation of the display unit by analyzing the background image. The electronic device according to feature 1.

8. The aforementioned background image is an image captured from real space. The electronic device according to feature 1.

9. The aforementioned background image is an image of a virtual space. The electronic device according to feature 1.

10. The first acquisition step is to obtain the background image, The second acquisition step involves acquiring a virtual image, By combining the virtual image with the background image, a composite image is generated to be displayed on the display unit. The synthesis steps that are performed It has, The synthesis step involves, at a first time step, compositing the virtual image onto a first position on the background image, and at a second time step, later than the first time step, compositing the virtual image composited onto the background image at the first time step onto a second position on the background image, which has been moved from its first position in accordance with the change in the position and orientation of the display unit from the first time step to the second time step, without acquiring a new virtual image. A method for controlling electronic equipment characterized by the following features.

11. A program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 9.

Images