Electronic equipment, control methods and programs for electronic equipment

Abstract

This provides a technique for better controlling the computational load when rendering virtual shadows. [Solution] The electronic device of the present invention is characterized by comprising: estimation means capable of performing estimation processing to estimate the position of a light source; and drawing means that, in the first case where the estimation means does not perform the estimation processing, executes a first drawing process to draw a virtual shadow representing a shadow on a virtual object; and in the second case where the estimation means performs the estimation processing, executes a second drawing process to draw the virtual shadow with a smaller computational load than the first drawing process.

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method for an electronic device, and a program.

Background Art

[0002] As a technology for seamlessly integrating the real world and the virtual world in real time, Mixed Reality (MR) technology is known. The MR technology is used, for example, in a video see-through type Head Mounted Display (HMD). In the video see-through type HMD using the MR technology, a real space corresponding to the field of view of a user wearing the HMD is imaged by a video camera or the like. Then, a composite image obtained by superimposing a CG (Computer Graphics) image on the obtained image of the real space is presented (displayed) to the user.

[0003] When synthesizing a captured image of the real space and a virtual image drawn in the virtual space, light source information may be acquired, and the light source information of the real image may be reflected in the virtual image. Further, based on the acquired light source information, a virtual shadow object (virtual shadow) may be drawn on a virtual object included in the virtual image.

[0004] In Patent Document 1, a technique for photographing a real space and estimating light source information based on the information acquired by the photographing is described.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] In Patent Document 1, the light source information estimation unit generates a celestial spherical image based on the 3D position and orientation data of the camera in 3D space, and estimates light source information (light source estimation) based on the celestial spherical image. Here, if the light source conditions are changed, it is necessary to regenerate the celestial spherical image. Therefore, when providing a high-resolution virtual shadow to the user, it is necessary to perform the rendering process of the high-resolution virtual shadow and the light source estimation process simultaneously, which increases the computational load.

[0007] This invention provides a technique for more effectively controlling the computational load when rendering virtual shadows. [Means for solving the problem]

[0008] The electronic device of the present invention is characterized by comprising: estimation means capable of performing estimation processing to estimate the position of a light source; and drawing means that, in a first case where the estimation means does not perform the estimation processing, executes a first drawing process to draw a virtual shadow representing a shadow on a virtual object; and in a second case where the estimation means performs the estimation processing, executes a second drawing process to draw the virtual shadow with a smaller computational load than the first drawing process. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a technique for more appropriately controlling the computational load when rendering virtual shadows. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram shows the positional relationship between the user and the light source of the HMD according to Embodiment 1. [Figure 2] This figure shows an example of the functional configuration of an electronic device according to Embodiment 1. [Figure 3] This figure shows an example of the hardware configuration of the information processing device according to Embodiment 1. [Figure 4] This is a flowchart of the light source estimation process according to Embodiment 1. [Figure 5]This is a diagram illustrating the virtual shadow according to Embodiment 1. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0012] The following describes the technology used to provide users with a realistic MR (Mixed Reality) space. To provide users with a realistic MR space, it is desirable that the optical properties (color, brightness, shadows, and reflections, etc.) represented in the virtual image match those captured in the real image when combining the real image and the virtual image. The degree to which these two optical properties match is called "optical consistency." To obtain the optical properties of the real image, a process (light source estimation process) is performed to estimate (obtain) light source information such as the type of light source, brightness, and position and orientation by analyzing the real image. Furthermore, optical consistency is enhanced by drawing high-resolution virtual shadow objects on the virtual image based on the results of the light source estimation process.

[0013] One technique for estimating light sources involves using an image called an environment map, which can be generated by imaging real space. Furthermore, environment maps can be used to simulate reflections of virtual objects in virtual images.

[0014] One known method for rendering virtual shadow objects (virtual shadows) representing shadows on virtual objects onto virtual images is shadow mapping, which generates a texture called a shadow map. In shadow mapping, the depth (distance) from the light source to the virtual object is obtained by rendering the virtual object from the position of the light source. By comparing this depth with the depth from the light source to each virtual object, it is possible to determine which areas are illuminated and which are not. The virtual shadows are then rendered using the shadow map generated in this way. Since the shadow map is a texture that stores depth values ​​in the content displayed on one screen (frame buffer), increasing the resolution of the shadow map can render shadows that are closer to reality. However, using shadow mapping is computationally intensive and time-consuming, which can lead to inconsistencies between the state of the real world and the state of the MR space. As a result, users may feel a sense of incongruity in the MR space they experience.

[0015] Therefore, the following describes a technique for enabling users to experience a realistic MR space by appropriately controlling the computational load when rendering shadow objects.

[0016] <Embodiment 1> Figure 1 is a diagram showing the positional relationship between the user (experiencer) and the light source 102 of the HMD 101 according to Embodiment 1. The HMD 101 is a head-mounted display device worn by the user on their head. The HMD 101 is a video see-through type HMD that presents the user with a mixed reality (MR) space in which the real and virtual spaces are fused, based on images captured from the real space. However, the HMD 101 is not limited to a video see-through type HMD. For example, it may be an optical see-through type HMD that allows the user to perceive the MR space by superimposing (displaying) a virtual image on a transparent display medium that allows the real space to be seen through. Also, the HMD 101 may be a display device such as a handheld display (HHD).

[0017] The light source 102 is a light source arranged in the real space. In Embodiment 1, the description will proceed with the light source 102 as a point light source, but it is not limited thereto. The virtual object 103 is a virtual object displayed on the HMD 101 (display unit 202 described later) and arranged at a distance visible to the user.

[0018] FIG. 2 is a diagram showing a functional configuration example of the HMD 101 and the information processing apparatus 200, which are electronic devices according to Embodiment 1. The HMD 101 and the information processing apparatus 200 are connected by wire or wirelessly. The HMD 101 and the information processing apparatus 200 may be configured as one electronic device, or each may be a separate device. The HMD 101 includes an imaging unit 201 and a display unit 202.

[0019] The imaging unit 201 is a camera for acquiring an imaging image (real image) representing the real space by imaging the real space. The imaging unit 201 may have two eyes of lenses that enable imaging of a right image and a left image with a parallax. The imaging unit 201 outputs the real image to an image acquisition unit 203 described later.

[0020] The display unit 202 is provided with a display element that presents (displays) an image to the right eye and a display element that presents an image to the left eye. By performing "binocular VR display" using the right-eye VR image and the left-eye VR image with a parallax from each other on the left and right display elements, the user can stereoscopically view the VR image. An EL (Electro Luminescence) panel, an LCD, or the like can be applied to the display element, but it is not limited thereto.

[0021] The information processing apparatus 200 may be an external device of the HMD 101 exemplified by a personal computer or the like, or may be a processing unit built in the HMD 101. The information processing apparatus 200 includes functions of an image acquisition unit 203, a measurement unit 204, a light source estimation control unit 205, a light source estimation unit 206, a virtual shadow control unit 207, a virtual data storage unit 208, a virtual image generation unit 209, and a composite image generation unit 210.

[0022] The image acquisition unit 203 acquires the real image captured by the imaging unit 201. The image acquisition unit 203 outputs the real image to the measurement unit 204, the light source estimation control unit 205, and the composite image generation unit 210.

[0023] The measurement unit 204 acquires information regarding the position and orientation of the HMD 101 by detecting features such as points and lines from the real image. However, the method of acquiring information regarding the position and orientation of the HMD 101 is not limited to this. For example, the measurement unit 204 may acquire information regarding the position and orientation of the HMD 101 by further equipping the HMD 101 with sensors for detecting position and orientation (such as an infrared reflection sensor, an ultrasonic sensor, a magnetic sensor, or a depth sensor).

[0024] The light source estimation control unit 205 instructs the light source estimation unit 206 to start, end, and continue the light source estimation process. The light source estimation control unit 205 determines whether or not a light source is included in the real image by detecting areas (highlight areas) in the real image where the brightness value is higher than a threshold. Based on the determination of whether or not a light source is included in the real image, the light source estimation control unit 205 decides whether or not to start the light source estimation process.

[0025] The light source estimation unit 206 performs light source information estimation processing (light source estimation processing) based on the real image and information regarding the position and orientation of the HMD 101. Generally, light source information in computer graphics includes the type of light source (parallel light source, point light source, spotlight, etc.), the brightness of the light source, the color of the light source, the position and orientation of the light source, and the size of the light source. The light source estimation unit 206 estimates the position and brightness of the light source from the real image. However, the light source information estimated by the light source estimation unit 206 is This is not the only example; for instance, it could also include the size, shape, color, movement, speed, and trajectory of a light source.

[0026] The virtual shadow control unit 207 outputs the information necessary for rendering the virtual shadow to the virtual image generation unit 209 based on the light source information estimation result (light source estimation result) from the light source estimation unit 206.

[0027] The virtual data storage unit 208 is a storage unit that stores data of the virtual object 103.

[0028] The virtual image generation unit 209 generates a virtual space (virtual image) based on the data of the virtual object 103, virtual shadow control information, and light source estimation results. The virtual image is an image representing the virtual space (virtual object 103) as seen from the position and orientation measured by the measurement unit 204. Note that the technique for generating a virtual image as seen from a predetermined position is a known technique, so a detailed explanation will be omitted. The virtual image generation unit 209 changes the method of drawing the virtual shadow based on the light source estimation results.

[0029] The composite image generation unit 210 generates an image in the MR space (composite image) by superimposing a virtual image onto a real image. The composite image generation unit 210 outputs the composite image to the display unit 202. By outputting the composite image to the display unit 202, the composite image generation unit 210 displays the composite image on the display unit 202.

[0030] Figure 3 shows an example of the hardware configuration of the information processing device 200. The information processing device 200 has a CPU 301, a storage device 302, a ROM 303, a RAM 304, an input I / F 305, and an output I / F 306.

[0031] The CPU 301 controls the storage device 302, ROM 303, RAM 304, input I / F 305, and output I / F 306, which are connected via the bus 300. The CPU 301 reads programs stored in ROM 303 and RAM 304 and executes them.

[0032] The storage device 302 stores programs and data for controlling the information processing device 200.

[0033] ROM303 stores the operating system (OS), device drivers, and boot program.

[0034] The RAM 304 temporarily stores programs and data loaded from the storage device 302 and ROM 303. The RAM 304 has a work area for each process to be executed by the CPU 301.

[0035] Input I / F 305 is an interface for receiving signals input from devices such as the HMD 101. Input signals in a format that the information processing device 200 can process are input to Input I / F 305 from devices such as the HMD 101 to which the information processing device 200 is connected. Output I / F 306 is an interface for outputting signals to devices such as the HMD 101. Output signals in a format that the HMD 101 can process are transmitted to the HMD 101 and other devices connected to the information processing device 200.

[0036] Figure 4 is a flowchart of the light source estimation process for estimating (acquiring) light source information according to Embodiment 1. The explanation of Figure 4 describes how the information processing device 200 controls the computation load of the virtual shadow according to the computation load status of the light source estimation process, thereby appropriately controlling the computation load. In the flowchart of Figure 4, the CPU 301 of the information processing device 200 processes a program. This is achieved by executing and controlling each component of the information processing device 200.

[0037] In step S400, the image acquisition unit 203 acquires a real image captured by the imaging unit 201 of the real space.

[0038] In step S401, the measurement unit 204 acquires information regarding the position and orientation of the HMD 101 based on the real-world image acquired from the image acquisition unit 203.

[0039] In step S402, the light source estimation control unit 205 determines whether the light source estimation unit 206 is currently performing the light source estimation process. If the light source estimation control unit 205 determines that the light source estimation unit 206 is currently performing the light source estimation process, it proceeds to step S404. If it determines that the light source estimation process is not currently being performed, it proceeds to step S403.

[0040] In step S403, the light source estimation control unit 205 determines whether or not to start the light source estimation process. In Embodiment 1, the light source estimation unit 206 determines whether or not the light source 102 is included in the real image based on the highlight region in the real image. Specifically, if the light source estimation unit 206 detects that the area of ​​the highlight region in the real image is greater than or equal to a threshold, it may determine that the light source 102 is included in the real image. Alternatively, if the light source estimation unit 206 determines that a highlight region exists in the real image, it may determine that the light source 102 is included in the real image. If the light source estimation unit 206 determines that the light source 102 is included in the real image, the light source estimation control unit 205 proceeds to step S404; if it determines that the light source 102 is not included in the real image, it proceeds to step S408. However, if no light source information exists in the first place, the light source estimation control unit 205 may start the light source estimation process.

[0041] In step S403, the light source estimation unit 206 may determine whether the light source 102 is included in the real image based on whether or not it detects a marker that has been placed in advance near the light source 102 in the real image. For example, a marker created with a two-dimensional barcode or the like may be attached near the light source 102. Then, the light source estimation unit 206 may determine whether the light source 102 is included in the real image if the marker is detected in the real image. Furthermore, the light source estimation unit 206 may determine whether or not the light source 102 is included in the real image by detecting the shape of the light source 102 from the real image or by detecting the distance between the HMD 101 and the light source 102.

[0042] In step S404, the light source estimation unit 206 performs light source estimation processing based on the real image and information regarding the position and orientation of the HMD 101. In Embodiment 1, since a point light source is used which does not require consideration of the direction of the light source, the light source estimation unit 206 estimates the brightness and position and orientation of the light source 102.

[0043] The brightness of light source 102 is estimated to be, for example, the brightness of the brightest area in the highlight region. For example, within a preset brightness range (e.g., 0 cd / m²).2 ~1000 cd / m² 2 When the range is expressed as a grayscale value (Y value) from 0 to 100, the brightness of the light source 102 is estimated to be the grayscale value corresponding to the brightness value of the brightest place in the highlight region. The position and orientation of the light source 102 is a relative position and orientation with respect to the position and orientation of the HMD 101. The position of the light source 102 is estimated to be the position of the point with the highest brightness value in the extracted highlight region. The light source estimation unit 206 outputs the information of the light source estimation result (estimated brightness and estimated position and orientation) obtained by the light source estimation process in step S404 to the virtual shadow control unit 207.

[0044] In step S405, the light source estimation control unit 205 determines whether or not to terminate the light source estimation process of the light source estimation unit 206. If the light source estimation result is output from the light source estimation unit 206, the light source estimation control unit 205 instructs the light source estimation unit 206 to terminate the light source estimation process. If no light source estimation result is output from step 6, the light source estimation unit 206 continues the light source estimation process. If the light source estimation control unit 205 determines that the light source estimation process of the light source estimation unit 206 should be terminated, the process proceeds to step S406. If it determines that the light source estimation process should not be terminated, the process proceeds to step S407.

[0045] In step S406, the virtual shadow control unit 207 updates the light source information held by the virtual image generation unit 209 with light source information based on the light source estimation result acquired in step S404.

[0046] In step S407, the virtual shadow control unit 207 performs drawing processing to draw a virtual shadow using a simple drawing method. In Embodiment 1, a texture to be used for a virtual shadow with an arbitrary transparency is stored in the storage device 302 in advance, and the virtual shadow control unit 207 reads the texture from the storage device 302 and outputs it to the virtual image generation unit 209. The virtual image generation unit 209 performs a simple shadow representation by placing the input texture on the bottom surface of the virtual object 103. This drawing method does not take into account "light source direction and light source intensity" and "positional relationship between the light source 102 and the virtual object 103," which are necessary for drawing high-quality virtual shadows, so it is possible to add a simple virtual shadow that does not take optical consistency into consideration.

[0047] However, the methods for simply adding virtual shadows are not limited to those described above. For example, an arbitrary virtual light source may be assumed in advance (e.g., a directional light illuminating from above the virtual object), and the virtual shadow control unit 207 may pre-render the virtual shadow to the virtual object. Furthermore, to prevent the rendering of virtual shadows that are far removed from reality and thus impair the user's MR experience, the virtual shadow control unit 207 may present blurred virtual shadows to the user by applying filtering to them. In addition, if there is no light source information, the virtual shadow control unit 207 does not need to render virtual shadows.

[0048] In step S408, the virtual shadow control unit 207 performs a drawing process to draw a high-quality virtual shadow (a virtual shadow with high optical consistency) compared to the virtual shadow in step S407. In step S408, shadow mapping technology is used to draw the high-quality virtual shadow.

[0049] This section describes how to render high-quality virtual shadows using shadow mapping technology. The shadow map contains distance information (depth value) from the light source to the virtual object. When the virtual shadow control unit 207 renders a virtual object in the virtual space from the viewpoint of the light source, it stores the distance information (depth value) from the light source to the virtual object in the frame buffer. This generates the shadow map. Next, the virtual shadow control unit 207 calculates the distance from each pixel of the virtual object to the light source in the virtual space from the position and orientation information of the HMD 101 measured by the measurement unit 204. The virtual shadow control unit 207 compares the distance from each pixel of the virtual object to the light source with the distance recorded in the shadow map to determine whether the virtual object is in the virtual shadow. The result of determining whether the virtual object is in the virtual shadow is referenced in the rendering process of the virtual shadow in the virtual image. By increasing the sampling value when generating the shadow map, the virtual shadow control unit 207 can increase the resolution of the shadow map, thereby suppressing jagged edges that occur at the edges of the virtual shadow. Techniques that increase the sampling value can be computationally intensive when there are multiple virtual objects and / or light sources, but they are widely used as methods for creating dynamic virtual shadows.

[0050] In step S408, shadow mapping technology is used to render high-quality virtual shadows. Although used in step S407, a shadow map technique may also be used to generate a virtual shadow. In step S407, the computational load can be reduced by using a lower-resolution shadow map compared to the shadow map generated in step S408. However, this is not the only way to reduce the computational load in step S407 compared to step S408. For example, the distance at which the virtual shadow is drawn may be reduced, or the coefficient used for drawing the virtual shadow may be reduced.

[0051] In step S408, the virtual shadow control unit 207 may define a temporary virtual light source and generate a high-resolution shadow map from that light source in order to generate high-quality virtual shadows. Alternatively, it may define a temporary virtual light source and implement a method for rendering high-quality shadows, such as ray tracing or IBL (Image-based lighting).

[0052] The virtual shadow control unit 207 may draw a virtual shadow based on the light source estimation result of the light source estimation unit 206, or it may draw a virtual shadow that is not based on the light source estimation result. Furthermore, if light source estimation has never been performed and no light source information exists, there is no information to generate a virtual shadow, so the virtual shadow control unit 207 will not generate a virtual shadow.

[0053] Furthermore, regardless of whether or not light source estimation is performed, the rendering method for virtual objects (objects other than virtual shadows) may be the same. In other words, whether or not light source estimation is performed, the virtual image generation unit 209 may render virtual objects (objects other than virtual shadows) using the same process. This allows the user to see virtual objects of a certain quality, regardless of whether or not light source estimation is performed.

[0054] Figures 5(A) and (B) are schematic diagrams showing the state in which a virtual shadow is drawn by the virtual shadow control unit 207. The virtual shadow 501 in Figure 5(A) is a virtual shadow drawn using a simple drawing method. The virtual shadow 501 is a circular texture with a transparent layer and is placed on the bottom surface of the virtual object 103 by the virtual shadow control unit 207. The method of drawing the virtual shadow 501 shown in Figure 5(A) is computationally intensive because it only involves drawing a circular texture on the bottom surface of the virtual object 103.

[0055] The virtual shadow 502 in Figure 5(B) is a high-quality virtual shadow texture rendered using shadow mapping technology. The virtual shadow 502 is a virtual shadow rendered by the virtual shadow control unit 207, which generates a high-resolution shadow map based on the light source estimation results. The method for rendering the virtual shadow 502 shown in Figure 5(B) is computationally more demanding than the method for rendering the virtual shadow 501 shown in Figure 5(A), but it can render a more realistic virtual shadow.

[0056] According to Embodiment 1, a simple virtual shadow is drawn during the execution of the light source estimation process or in the frame immediately following the completion of the light source estimation process, while a high-quality virtual shadow is drawn when the light source estimation process is not in progress. This allows for more appropriate control of the computational load of the light source estimation process and the computational load required to draw realistic, high-quality virtual shadows.

[0057] <Modified form of Embodiment 1> In Embodiment 1, light source estimation processing was performed from real-world images acquired by the HMD 101, but light source estimation processing may also be performed from a texture map (environment map) generated based on real-world images. The environment map is generated as a texture to be applied to the surface of a virtual object (such as a cube or sphere) when the virtual object is placed in real space, based on one or more real-world images captured by the imaging unit 201. Light source estimation processing may also be performed by estimating light source information from the environment map.

[0058] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). Multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) may share the processing to control the entire device.

[0059] Furthermore, the above-mentioned processors are processors in a broad sense, including general-purpose processors and specialized processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Specialized processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0060] Furthermore, although embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0061] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit that implements one or more functions.

[0062] This embodiment includes the following configurations, methods, and programs. (Composition 1) An estimation means capable of performing an estimation process to estimate the position of a light source, In the first case where the estimation means does not perform the estimation process, the drawing means executes a first drawing process to draw a virtual shadow representing the shadow of a virtual object, and in the second case where the estimation means performs the estimation process, the drawing means executes a second drawing process to draw the virtual shadow with a smaller computational load than the first drawing process. has An electronic device characterized by the following features. (Configuration 2) In the second case, the estimation means performs the estimation process based on the image captured by the imaging unit. The electronic device according to configuration 1, characterized by the features described above. (Composition 3) In the second case, the estimation means performs the estimation process based on information regarding the position and orientation of the imaging unit. The electronic device according to configuration 2, characterized by the features described above. (Composition 4) The estimation means is, If the aforementioned image includes the aforementioned light source, the estimation process is performed. If the aforementioned image does not include the aforementioned light source, the estimation process is not performed. The electronic device according to configuration 2 or 3, characterized by the above. (Composition 5) The drawing means is In the first drawing process, the virtual shadow is drawn using a first shadow map that includes distance information from the light source to the virtual object. In the second drawing process described above, the virtual shadow is drawn using a second shadow map with a lower resolution than the first shadow map. An electronic device according to any one of configurations 1 to 4, characterized by the features described herein. (Composition 6) In the second drawing process, the drawing means draws the virtual shadow which has been blurred by applying a filter. The electronic device according to configuration 5, characterized by the features described herein. (Composition 7) The drawing means is In the first drawing process described above, the virtual shadow is drawn using a shadow map that includes distance information from the light source to the virtual object. In the second drawing process described above, the virtual shadow is drawn without using the shadow map. An electronic device according to any one of configurations 1 to 4, characterized by the features described herein. (Composition 8) The drawing means is In the first drawing process, the virtual shadow is drawn using a texture map generated using the image captured by the imaging unit. In the second drawing process described above, the virtual shadow is drawn without using the texture map. An electronic device according to any one of configurations 2 to 4, characterized by the features described herein. (method) An estimation step that allows for estimation processing to estimate the position of a light source, In the first case where the estimation process is not performed in the estimation step, a first drawing process is performed to draw a virtual shadow representing the shadow of the virtual object; in the second case where the estimation process is performed in the estimation step, a second drawing process is performed to draw the virtual shadow with a lower computational load than the first drawing process; has A method for controlling electronic equipment characterized by the following features. (program) A program for causing a computer to function as one of the electronic devices described in any one of configurations 1 to 8. [Explanation of Symbols]

[0063] 206: Light source estimation unit 207: Virtual shadow control unit

Claims

1. An estimation means capable of performing an estimation process to estimate the position of a light source, In the first case where the estimation means does not perform the estimation process, the drawing means executes a first drawing process to draw a virtual shadow representing the shadow of the virtual object, and in the second case where the estimation means performs the estimation process, the drawing means executes a second drawing process to draw the virtual shadow with a smaller computational load than the first drawing process. has An electronic device characterized by the following features.

2. In the second case, the estimation means performs the estimation process based on the image captured by the imaging unit. The electronic device according to feature 1.

3. In the second case, the estimation means performs the estimation process based on information regarding the position and orientation of the imaging unit. The electronic device according to feature 2.

4. The estimation means is, If the aforementioned image includes the aforementioned light source, the estimation process is performed. If the aforementioned image does not include the aforementioned light source, the estimation process is not performed. The electronic device according to feature 2.

5. The drawing means is In the first drawing process, the virtual shadow is drawn using a first shadow map that includes distance information from the light source to the virtual object. In the second drawing process, the virtual shadow is drawn using a second shadow map with a lower resolution than the first shadow map. The electronic device according to feature 1.

6. In the second drawing process, the drawing means draws the virtual shadow which has been blurred by applying a filter. The electronic device according to feature 5.

7. The drawing means is In the first drawing process, the virtual shadow is drawn using a shadow map that includes distance information from the light source to the virtual object. In the second drawing process described above, the virtual shadow is drawn without using the shadow map. The electronic device according to feature 1.

8. The drawing means is In the first drawing process, the virtual shadow is drawn using a texture map generated using the image captured by the imaging unit. In the second drawing process described above, the virtual shadow is drawn without using the texture map. The electronic device according to feature 2.

9. An estimation step that allows for estimation processing to estimate the position of a light source, In the first case where the estimation process is not performed in the estimation step, the shadow of the virtual object is displayed. A drawing step is performed which includes a first drawing process to draw the virtual shadow, and in the case of a second drawing process which includes an estimation process in the estimation step, a second drawing process which includes a drawing process which draws the virtual shadow with a lower computational load than the first drawing process. has A method for controlling electronic equipment characterized by the following features.

10. A program for causing a computer to function as one of the means of an electronic device according to any one of claims 1 to 8.

Images