Electronic device and control method therefor

Abstract

An electronic device is disclosed. The electronic device comprises: a mirror display; a sensor; a memory for storing at least one instruction; and one or more processors connected to the mirror display, the sensor and the memory, wherein the one or more processors execute the at least one instruction so as to: identify, on the basis of data acquired through the sensor, space information corresponding to a space in which a user is positioned and gaze information of the user; identify the display size and the display position of a virtual object on the basis of the space information and the gaze information of the user; and control the mirror display such that the virtual object is displayed on the basis of the identified display size and display position of the virtual object.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device having a mirror display and a method for controlling the same.

[0002] Advances in electronic technology have led to the development and proliferation of various types of electronic devices. In particular, display devices, used in a variety of settings, including homes, offices, and public spaces, have been continuously evolving in recent years.

[0003] For example, mirror displays, which offer both mirroring and display functions, enable a variety of services. For example, mirror displays function as mirrors through reflectivity when no images are displayed, and when images are displayed, they display images through transmittance, allowing the user to see the images.

[0004] For example, a mirror display may display virtual objects in some areas even when operating in mirror mode. However, simply displaying virtual objects may result in a sense of incongruity between the real object reflected in the mirror display and the virtual object displayed on the mirror display due to differences in focus or lack of three-dimensionality.

[0005] An electronic device according to one or more embodiments includes a mirror display, a sensor, a memory storing at least one command, and at least one processor connected to the mirror display, the sensor, and the memory. The at least one processor can control the mirror display to identify space information corresponding to a space where a user is located and gaze information of the user based on data acquired through the sensor by executing the at least one command, identify a display size and a display position of a virtual object based on the space information and the gaze information of the user, and display the virtual object based on the display size and the display position of the identified virtual object.

[0006] The one or more processors according to one or more embodiments may, by executing the at least one command, identify distance information between the user and the electronic device based on data acquired through the sensor, and identify a display size and a display position of the virtual object based on the spatial information, the user's gaze information, and the distance information between the user and the electronic device.

[0007] According to one or more embodiments, the one or more processors, by executing the at least one command, can re-identify the display shape, display size, and display position of the virtual object based on the changed information when at least one of the user's gaze information or the distance information between the user and the electronic device is identified as having changed.

[0008] The one or more processors according to one or more embodiments may, by executing the at least one command, identify distance information between the user and the electronic device based on data acquired through the sensor, identify virtual body data of the user based on data acquired through the sensor, and control the mirror display to display real body data of the user based on the distance information between the user and the electronic device and the virtual body data of the user.

[0009] The one or more processors according to one or more embodiments may, by executing the at least one command, identify distance information between the user and the electronic device based on data acquired through the sensor, and control the mirror display to display a virtual object corresponding to the user based on the distance information between the user and the electronic device and the user's previous body data.

[0010] The one or more processors according to one or more embodiments can, by executing the at least one command, identify the size and position of a real object included in a space where the user is located based on data acquired through the sensor, and identify the display size and display position of a virtual object included in the space where the user is located based on the size and position of the real object and the user's gaze information.

[0011] The one or more processors according to one or more embodiments, by executing the at least one command, identify the size and direction of the shadow of the virtual object based on context information of the electronic device, the display size and the display position of the virtual object, and control the mirror display to display the shadow of the virtual object based on the size and direction of the shadow of the virtual object, wherein the context information may include at least one of time information and light source information.

[0012] The one or more processors according to one or more embodiments can identify a type of the virtual object based on a type of a real object included in a space where the user is located, and identify a display size and display position of the virtual object based on a size and position of the real object by executing the at least one command.

[0013] The one or more processors according to one or more embodiments can, by executing the at least one command, identify a shadow direction and ratio of the real object included in a space where the user is located based on a size of the real object and a direction of a light source, and control the mirror display to display a shadow of the virtual object based on the shadow direction and ratio of the real object.

[0014] The one or more processors according to one or more embodiments may, by executing the at least one command, identify gaze information of each of the plurality of users based on data acquired through the sensor, identify distance information between each of the plurality of users and the electronic device based on data acquired through the sensor, and identify a display size and a display position of the virtual object based on the spatial information, the gaze information of the plurality of users, and the distance information between the plurality of users and the electronic device.

[0015] A method for controlling an electronic device according to one or more embodiments includes a step of identifying spatial information corresponding to a space where a user is located and gaze information of the user based on data acquired through a sensor, a step of identifying a display size and a display position of a virtual object based on the spatial information and the gaze information of the user, and a step of displaying the virtual object through a mirror display based on the display size and display position of the identified virtual object.

[0016] A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device according to one or more embodiments, cause the electronic device to perform an operation, the operation comprising: a step of identifying space information corresponding to a space where a user is located and gaze information of the user based on data acquired through a sensor; a step of identifying a display size and a display position of a virtual object based on the space information and the gaze information of the user; and a step of displaying the virtual object through a mirror display based on the display size and the display position of the identified virtual object.

[0017] Aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0018] FIG. 1 is a drawing for explaining a usage form of a mirror display according to one or more embodiments.

[0019] Figure 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0020] FIGS. 3A and 3B are drawings illustrating the structure of a mirror display according to one or more embodiments.

[0021] FIG. 4 is a diagram showing transmission characteristic information according to one embodiment of the present invention.

[0022] FIG. 5 is a block diagram specifically illustrating a configuration of an electronic device according to one or more embodiments.

[0023] FIG. 6 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0024] FIGS. 7A and 7B are drawings illustrating a method for displaying a virtual object based on a mirror image according to one or more embodiments.

[0025] FIG. 8 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0026] FIGS. 9A and 9B are drawings for explaining a method of displaying a virtual object according to one or more embodiments.

[0027] FIG. 10 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0028] FIGS. 11, 12 and 13 are drawings for explaining a method of displaying a shadow of a virtual object according to one or more embodiments.

[0029] FIG. 14 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0030] FIG. 15 is a diagram illustrating a method for displaying fitness information according to one or more embodiments.

[0031] FIG. 16 is a diagram illustrating a method for displaying fitness information according to one or more embodiments.

[0032] FIG. 17 is a diagram illustrating a method for displaying virtual objects for multiple users according to one or more embodiments.

[0033] FIG. 18 is a drawing for explaining a method for displaying virtual objects for an interior experience according to one or more embodiments.

[0034] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0035] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0036] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0037] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0038] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0039] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0040] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0041] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0042] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0043] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "module" or "part" that needs to be implemented as specific hardware.

[0044] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0045] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0046] FIG. 1 is a drawing for explaining a usage form of a mirror display according to one or more embodiments.

[0047] An electronic device according to one embodiment of the present invention can be implemented as various types of mirror electronic devices that can be installed in various locations requiring a mirror and provide mirror functionality while transmitting information. Here, "Mirror Display" is a compound word of "Mirror," meaning a mirror, and "Display," meaning the work of visually expressing information.

[0048] As shown in Fig. 1, the mirror display (10) can be implemented in a form in which a switchable mirror (12) is added to a conventional display panel (11).

[0049] The display panel (11) can be implemented as a display panel such as an LCD (Liquid Crystal Display Panel), OLED (Organic Light Emitting Diodes), LCoS (Liquid Crystal on Silicon), or DLP (Digital Light Processing).

[0050] Fig. 1 is for explaining the usage form of a general mirror display (10). The user sees the image displayed on the display panel (11) constituting the mirror display (10) through the switchable mirror (12), and at the same time, when the image is not displayed, the user sees the light reflected by the switchable mirror (12) from an external light source, so that it can be used as a mirror. In addition, the mirror display (10) may operate so that the mirror function is provided in some areas and the display function is provided in other areas. For example, according to one example, even when the mirror display operates in mirror mode, a virtual object may be displayed in some areas. In this case, it is necessary to process the virtual object displayed on the mirror display so that it is provided without a sense of incongruity with the real object reflected on the mirror display.

[0051] Below, various embodiments are described that provide virtual objects displayed on a mirror display without any sense of incongruity with real objects reflected on the mirror display.

[0052] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments.

[0053] According to FIG. 2, the electronic device (100) includes a mirror display (110), a sensor (120), a memory (130), and one or more processors (140). According to an example, the electronic device (100) may be implemented as a switchable mirror device that provides a mirror function and a display function.

[0054] The mirror display (110) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The mirror display (110) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. According to an example, a touch sensor that detects a touch operation in the form of a touch film, a touch sheet, a touch pad, etc. may be disposed on the front of the mirror display (110) so as to be implemented so as to detect various types of touch inputs. For example, the mirror display (110) can detect various types of touch inputs, such as a touch input by a user's hand, a touch input by an input device such as a stylus pen, and a touch input by a specific electrostatic material. Here, the input device can be implemented as a pen-type input device that can be referred to by various terms such as an electronic pen, a stylus pen, an S-pen, etc. According to an example, the mirror display (110) can be implemented as a flat display, a curved display, a flexible display that can be folded or / and rolled, etc.

[0055] Meanwhile, the mirror display (110) can be implemented as a display that provides a mirror function and a display function.

[0056] For example, the mirror display (110) can be implemented in the form of a switchable mirror added to a conventional display panel.

[0057] As illustrated in FIGS. 3A and 3B, the mirror display (210) may be implemented in a form including a polarizer (111), an upper glass (112), a lower glass (114), and a reflective polarizer (115). In one example, a liquid crystal (LC) layer (113) may be formed between the upper glass (112) and the lower glass (114). The liquid crystal (LC) is in an intermediate state between a liquid and a crystal, and may have a structure in which rod-shaped molecules (liquid crystal molecules) are aligned in one direction similar to a solid crystal.

[0058] For example, a polarizer (111) may be implemented to transmit polarized light. For example, the upper glass (112) and the lower glass (114) may be implemented using TCO (transparent conductive oxide) glass, but the present invention is not limited thereto.

[0059] Fig. 7a illustrates a mirror display (110) when the voltage is turned off. When the voltage is turned off, the liquid crystal molecules maintain a vertical state, and the incident polarized light can be reflected along the reflection axis of the reflective polarizer (115) by passing through the LC (liquid crystal) layer (113). Accordingly, the mirror display (110) can operate in mirror mode.

[0060] Fig. 7b illustrates a mirror display (110) when the voltage is turned ON. When the voltage is turned OFF, the liquid crystal molecules collapse and maintain a twisted state, and the incident polarized light rotates 90 degrees to transmit through the reflection axis of the reflective polarizer (115). Accordingly, the mirror display (110) can operate in display mode. In addition, depending on the implementation example, a protective film that serves to protect the polarizer, a film that serves to classify light from the polarizer, etc. may be further included.

[0061] The sensor (120) can sense the presence of a user in front of the electronic device (100), the distance to the user, the user's approach speed, the user's current location, the direction (or angle) in which the user is located, changes in the user's location within a preset time range, the user's actions, etc. In this case, the sensor (120) can be implemented with various types of sensors capable of sensing the user. For example, the sensor (120) can include at least one sensor from among a ToF (Time of Flight) sensor, an ultrasonic sensor, a RADAR sensor, a photodiode sensor, a proximity sensor, a PIR (Passive Infrared Sensor) sensor, a pinhole sensor, a pinhole camera, an infrared human body detection sensor, a CMOS image sensor, a heat detection sensor, a light sensor, and a motion detection sensor. For example, when the sensor (120) is implemented as an infrared human body detection sensor (e.g., an IR (Infrared Ray) ToF (Time of Flight) sensor), the presence or absence of a user, approach speed, current location, change in location, etc. can be sensed based on the time at which the emitted infrared rays are reflected and received.

[0062] In addition, the sensor (120) may further include at least one sensor capable of sensing ambient illuminance, ambient temperature, light incidence direction, etc. In this case, the sensor (120) may be implemented as an illuminance sensor, a temperature detection sensor, a light quantity sensing layer, a camera, etc. For example, the illuminance sensor may be placed inside the glass provided in the mirror display (110), and in this case, the sensing function may be controlled to operate normally even inside the glass through an algorithm that compensates for the transmittance / reflectivity of the glass provided in the mirror display (110).

[0063] In addition, the sensor (120) may include various types of sensors such as a light sensor, a touch sensor, a proximity sensor, an acceleration sensor (or a gravity sensor), a geomagnetic sensor, a gyro sensor, a pressure sensor, a position sensor, a distance sensor, etc.

[0064] The memory (130) can store data required for various embodiments. The memory (130) may be implemented in the form of memory embedded in the electronic device (100') or in the form of memory that can be attached or detached from the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory that can be attached or detached from the electronic device (100). Meanwhile, in the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD). In addition, in the case of memory that can be attached or detached to the electronic device (100'), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented in the form of.

[0065] As an example, the memory (130) may store a computer program including at least one instruction or instructions for controlling the electronic device (100).

[0066] According to another example, the memory (130) may store an image received from an external device (e.g., a source device), an external storage medium (e.g., USB), an external server (e.g., web hard), etc., i.e., an input image. Alternatively, the memory (130) may store an image acquired through a camera provided in the electronic device (100).

[0067] According to another example, the memory (130) can store various information required for image quality processing, for example, information, algorithms, image quality parameters, etc. for performing at least one of Noise Reduction, Detail Enhancement, Tone Mapping, Contrast Enhancement, Color Enhancement, or Frame Rate Conversion.

[0068] According to one embodiment, the memory (130) may be implemented as a single memory that stores data generated from various operations according to the present disclosure. However, according to another embodiment, the memory (130) may be implemented to include multiple memories that each store different types of data or each store data generated at different stages.

[0069] The memory (130) stores the transmission characteristic information and reflection characteristic information of the mirror display (110).

[0070] Here, the transmittance characteristic information of the mirror display (110) may include information on a luminance value according to the gradation of the image signal determined based on the transmittance of the mirror display (110). For example, the luminance characteristic information of the mirror display (110) may be a luminance measurement value according to the gradation of the image signal. That is, since the mirror display (110) has a fixed transmittance, if an image signal for each gradation is input, a luminance value according to the gradation of the image signal can be obtained. That is, by inputting a 0 IRE (Institute of Radio Engineers) (black) to 100 IRE (white) image signal (for example, a 0 to 255 gradation image in the case of an 8-bit image), a luminance measurement value according to the gradation of the input image signal can be calculated and stored in the memory (130). The luminance measurement value according to the gradation of such an image signal is also called a gamma table (or gamma curve), and can be directly measured and calculated through an experiment, but it is also possible to predict and calculate it based on the transmittance of the mirror display (110).

[0071] Meanwhile, the reflection characteristic information of the mirror display (110) may include at least one of the reflectivity information of the mirror display (110) and the reflection luminance of the mirror display (110) with respect to the amount of external light. For example, the reflection characteristic information may be the reflectivity of the mirror display (110) itself. In this case, the reflection luminance according to the amount of external light according to an embodiment of the present invention may be calculated based on the amount of external light and the reflectivity measured by the sensor (120). As another example, the reflection characteristic information may be information in the form of a reflection luminance of the mirror display (110) with respect to the amount of external light that has been previously measured or calculated. Here, the reflection luminance of the mirror display (110) may be a luminance value generated as external light is reflected based on the reflectivity of the mirror display (110).

[0072] However, the luminance characteristic information and reflection characteristic information of the mirror display (110) can also be received from an external server, etc. For example, if luminance characteristic information corresponding to identification information (e.g., manufacturing number, model) of the mirror display (110) is stored in an external server, the electronic device (100) can receive the corresponding information from the external server.

[0073] FIG. 4 is a diagram showing transmission characteristic information of a mirror display according to one or more embodiments.

[0074] As illustrated in FIG. 4, a 0 IRE (black) to 100 IRE (white) image signal can be input to a mirror display (110) having a fixed transmittance to measure a luminance value, and the luminance measurement value for each grayscale, i.e., the transmittance characteristic information, can be stored in the memory (130). In addition, as described above, the luminance measurement value for each grayscale may be a value predicted by the processor (140) rather than an actually measured value, or may be received from an external source.

[0075] However, in some cases, the memory (130) may store only the basic luminance graph, and the processor (140) may produce a luminance graph corresponding to the transmittance of the mirror display (110) in real time according to a LUT or a calculation formula.

[0076] In addition, the memory (130) can store various data, programs, or applications for driving / controlling the electronic device (100). In addition, the memory (130) can include a user sensing module, a communication control module, a voice recognition module, a motion recognition module, a light receiving module, a display control module, an audio control module, an external input control module, a power control module, a voice database (DB), or a motion database (DB).

[0077] In the above-described embodiment, it has been described that various data are stored in the external memory (130) of the processor (140), but at least some of the above-described data may be stored in the internal memory of the processor (140) depending on an implementation example of at least one of the electronic device (100) or the processor (140).

[0078] One or more processors (140) control the overall operation of the electronic device (100). Specifically, one or more processors (140) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100). For example, one or more processors (140) may be electrically connected to the mirror display (110) and the memory (130) to control the overall operation of the electronic device (100). One or more processors (140) may be configured as one or more processors.

[0079] One or more processors (140) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (130).

[0080] The one or more processors (140) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (140) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (140) may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in a memory.

[0081] When a method according to one or more embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0082] One or more processors (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (140) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure.

[0083] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0084] In the embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. Hereinafter, for the convenience of description, one or more processors (140) will be referred to as a processor (140).

[0085] According to one embodiment, the processor (140) can identify spatial information corresponding to the space where the user is located and the user's gaze information based on data acquired through the sensor (120).

[0086] For example, the spatial information may be information about the space in front of the electronic device (100). For example, the information about the space reflected on the mirror display (110) may include at least one of size information of the space, information about a real object located in the space, or information about a user located in the space. For example, the information about a real object located in the space may include at least one of location information of the real object, size information, distance information from the user, or distance information between the real object and the electronic device (100). The information about the user located in the space may include at least one of location information of the user, size information, distance information between the user and the electronic device (100), and distance information between the real object and the user.

[0087] For example, the user's gaze information may include the user's gaze position information on the mirror display (110). For example, the processor (140) can detect the user's face position from a captured image acquired through a camera and track the user's face position to identify the user's gaze position. Various conventional methods can be used as a face region detection method. Specifically, a direct recognition method and a statistical method can be used. The direct recognition method creates rules using physical features such as the outline, skin color, and size of components of the face image or the distance between them, and compares, inspects, and measures according to the rules. The statistical method can detect the face region according to a pre-learned algorithm. That is, it is a method of converting unique features included in the input face into data and comparing and analyzing them with a large prepared database (shapes of faces and other objects). In particular, the face region can be detected according to a pre-learned algorithm, and methods such as a multi-layer perceptron (MLP) and a support vector machine (SVM) can be used.

[0088] The processor (140) can identify the display size and display position of a virtual object based on spatial information and the user's gaze information. For example, the processor (140) can identify the display size and display position of a virtual object based on the user's position and gaze information located in space.

[0089] The processor (140) can control the mirror display (110) to display a virtual object based on the display size and display position of the identified virtual object. For example, if the mirror display (110) is implemented as an LCE, the processor (140) can turn off the backlight of an area where a mirror function is provided based on local dimming, and turn on the backlight of an area where a virtual object is displayed.

[0090] According to one embodiment, the processor (140) can identify distance information between the user and the electronic device (100) based on data acquired through the sensor (120). The processor (140) can identify the display size and display position of the virtual object based on spatial information, the user's gaze information, and the distance information between the user and the electronic device (100).

[0091] According to one embodiment, if the processor (140) determines that at least one of the user's gaze information or the distance information between the user and the electronic device (100) has changed, the processor (140) may re-identify the display shape, display size, and display position of the virtual object based on the changed information. For example, if the position from which the user looks at the mirror display (110) changes by taking into account the user's dynamic movement, the processor (140) may re-identify the display shape, display size, and display position of the virtual object by taking into account the changed position.

[0092] According to one embodiment, the processor (140) can identify the user's virtual body data based on data acquired through the sensor (120). The processor (150) can display the user's actual body data through the mirror display (110) based on distance information between the user and the electronic device and the user's virtual body data.

[0093] According to one embodiment, the processor (140) may display a virtual object corresponding to the user through the mirror display (110) based on distance information between the user and the electronic device and the user's previous body data.

[0094] According to one embodiment, the processor (140) can identify the size and position of a real object included in a space where the user is located based on data acquired through the sensor (120). The processor (140) can identify the size and position of a real object included in a space where the user is located and the display size and display position of a virtual object based on the user's line of sight information. Accordingly, it is possible to provide a real object reflected on the mirror display (110) and a virtual object that is in focus from the user's line of sight.

[0095] According to one embodiment, the processor (140) may identify the size and direction of the shadow of the virtual object based on the context information of the electronic device (100), the display size and display position of the virtual object. According to one example, the context information of the electronic device (100) may include at least one of time information and light source information. The processor (140) may display the shadow of the virtual object through the mirror display (110) based on the size and direction of the shadow of the virtual object. For example, the shadow of the virtual object may be a UX effect for providing a three-dimensional effect to the virtual object. Accordingly, various UX effects that can provide a three-dimensional effect to the virtual object may be applied. For example, various effects such as a matte effect and a 3D effect may be applied to provide a three-dimensional effect to the virtual object.

[0096] According to one embodiment, when multiple users are identified based on data acquired through the sensor (120), the processor (140) can identify gaze information of each of the multiple users. The processor (140) can identify distance information between each of the multiple users and the electronic device (100) based on data acquired through the sensor (120). The processor (140) can identify the display size and display location of a virtual object based on spatial information, gaze information of the multiple users, and distance information between the multiple users and the electronic device (100).

[0097] According to one embodiment, the processor (140) can identify the type of a virtual object based on the type of a real object included in the space where the user is located, and identify the display size and display position of the virtual object based on the size and position of the real object.

[0098] In one embodiment, the processor (140) may identify the type of the virtual object based on the user's preference and the type of the real object. For example, if the type of the real object is a table, the processor (140) may identify the virtual object as a vase containing flowers based on the user's preference.

[0099] According to one embodiment, the processor (140) can identify the type of the virtual object based on the context of the electronic device (100) and the type of the real object. For example, if the current time is a time when the user is resting and the type of the real object is a cat, the processor (140) can identify a hunting game virtual object that can play with a cat.

[0100] According to one embodiment, the processor (140) can identify the shadow direction and ratio of a real object based on the size of the real object included in the space where the user is located and the direction of the light source, and display the shadow of a virtual object through the mirror display (110) based on the shadow direction and ratio of the real object.

[0101] FIG. 5 is a block diagram specifically illustrating a configuration of an electronic device according to one or more embodiments.

[0102] According to FIG. 5, the electronic device (100') may include a mirror display (110), a sensor (120), a memory (130), one or more processors (140), a communication interface (150), a user interface (160), and a speaker (170). Among the configurations illustrated in FIG. 5, a detailed description of configurations that overlap with those illustrated in FIG. 2 will be omitted.

[0103] It goes without saying that the communication interface (150) can be implemented as various interfaces depending on the implementation example of the electronic device (100'). For example, the communication interface (150) can communicate with an external device, an external storage medium (e.g., a USB memory), an external server (e.g., a web hard drive), etc. through a communication method such as Bluetooth, AP-based Wi-Fi (Wireless LAN network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc. In one example, the communication interface (150) can communicate with another electronic device, an external server, and / or a remote control device.

[0104] The user interface (160) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen that can also perform the display function and operation input function described above.

[0105] The speaker (170) may be configured to output various audio data as well as various notification sounds or voice messages. The processor (140) may control the speaker (170) to output feedback or various notifications in audio format according to various embodiments of the present disclosure.

[0106] In addition, the electronic device (100') may further include a microphone depending on the implementation example.

[0107] A microphone is a device configured to receive user voice or other sounds and convert them into audio data. However, according to another embodiment, the electronic device (100') may receive user voice input via an external device through a communication interface (150).

[0108] FIG. 6 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0109] According to FIG. 6, the electronic device (100) can identify spatial information corresponding to the space where the user is located and the user's gaze information based on data acquired through the sensor (120) (S610).

[0110] Next, the electronic device (100) can identify the display size and display location of the virtual object based on spatial information and the user's gaze information (S620).

[0111] Next, the electronic device (100) can control the mirror display (110) to display the virtual object based on the display size and display position of the identified virtual object (S630).

[0112] According to one embodiment, the electronic device (100) can identify the size and location of a real object included in a space where the user is located based on data acquired through the sensor (120). Subsequently, the electronic device (100) can identify the size and location of the real object included in the space where the user is located and the display size and display location of a virtual object based on the user's gaze information.

[0113] According to one embodiment, the electronic device (100) can identify the type of a virtual object based on the type of a real object contained in a space where the user is located. In addition, the electronic device (100) can identify the display size and display position of the virtual object based on the size and position of the real object.

[0114] According to one embodiment, the processor (140) may identify the display size and display position of a virtual object based on spatial information and the user's gaze information. For example, the processor (140) may identify the display size and display position of a virtual object based on the user's position and gaze information located in space. The processor (140) may control the mirror display (110) to display the virtual object based on the identified display size and display position of the virtual object.

[0115] According to one embodiment, the processor (140) may image-process a virtual object image based on the transmission characteristic information and reflection characteristic information of the mirror display (110) so that the virtual object is displayed without a sense of incongruity with the real object reflected on the mirror display (110). For example, the processor (140) may adjust pixel values, luminance values, etc. of the virtual object image so that the virtual object is displayed without a sense of incongruity with the real object reflected on the mirror display (110).

[0116] FIGS. 7A and 7B are drawings illustrating a method for displaying a virtual object based on a mirror image according to one or more embodiments.

[0117] According to one embodiment, the electronic device (100) can display a virtual object based on the mirror image principle. A mirror image refers to the image of an object reflected in a mirror. In a flat mirror, a mirror image is a reflection of the object that appears identical to each other but is opposite in the direction perpendicular to the mirror surface. This can occur due to the optical effect of reflection in a material such as a mirror or water.

[0118] According to Figure 7a, the user's eyes do not perceive the reflection from the mirror surface, but rather perceive it as if the light is coming straight from the back of the mirror. Therefore, an image must be drawn behind the mirror. In this case, the image created by the flat mirror is the same size as the real object, appears to be reversed, and the distance to the real object is the same as the distance from the mirror to the image.

[0119] According to FIG. 7b, let's assume that a table, which is a real object (720), is located in the real space where the user (710) is located. The electronic device (100) can identify the gaze position of the user (710), the distance from the user (710), the size, position, and distance of the real object (720), etc., based on the data acquired through the sensor (120). For example, as shown in FIG. 7b, if the real object is a table (720) and the virtual object is a virtual beverage, the display size and position of the virtual object (760) to be displayed on the mirror display (110) can be identified using perspective based on the mirror image (740) and the virtual object (750) of the real object (720), and can be displayed on the electronic device (100) so as to match the gaze angle of the user (710). Accordingly, the user can recognize an image that is the same as the virtual beverage (760) placed on the mirror image of the table, which is the real object (720).

[0120] FIG. 8 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0121] According to FIG. 8, the electronic device (100) can identify space information corresponding to the space where the user is located, the user's line of sight information, and distance information between the user and the electronic device (100) based on data acquired through the sensor (120) (S810).

[0122] Next, the electronic device (100) can identify the display size and display position of the virtual object based on spatial information, the user's line of sight information, and the distance information between the user and the electronic device (100) (S820).

[0123] Next, the electronic device (100) can control the mirror display (110) to display the virtual object based on the display size and display position of the identified virtual object (S830).

[0124] After this, if the electronic device (100) identifies that at least one of the user's gaze information or the distance information between the user and the electronic device (100) has changed (S840:Y), the electronic device (100) can re-identify the display shape, display size, and display position of the virtual object based on the changed information (S850).

[0125] FIGS. 9A and 9B are drawings for explaining a method of displaying a virtual object according to one or more embodiments.

[0126] According to one embodiment, when the distance between the user and the electronic device (100) changes or the direction in which the user looks at the electronic device (100) changes, the display shape, display size, and display position of the virtual object can be re-identified based on at least one of the changed distance or direction.

[0127] For example, as illustrated in FIG. 9A, when the gaze direction of the user (710) is to the right with respect to the electronic device (100) and the distance between the user and the electronic device (100) is a first distance, the display shape, display size, and display position of the virtual object (760) can be identified based on the gaze direction and distance, and the position and distance of the real object (720), and the virtual object can be displayed. For example, the electronic device (100) can identify the display shape, display size, and display position of the virtual object (760) based on the method illustrated in FIGS. 7A and 7B. Accordingly, the user can recognize an image such as a virtual beverage, which is a virtual object (760), placed on a mirror image of the table, which is a real object (720).

[0128] For example, as illustrated in FIG. 9b, when the user's gaze direction is to the left with respect to the electronic device (100) and the distance between the user and the electronic device (100) is changed to a second distance, the display shape, display size, and display position of the virtual object (760) may be re-identified based on the changed gaze direction and distance, and the position and distance of the real object (720), and the virtual object may be displayed. For example, the electronic device (100) may re-identify the display shape, display size, and display position of the virtual object (760) based on the method illustrated in FIGS. 7a and 7b. Accordingly, the user may recognize an image such as a virtual beverage, which is a virtual object (760), placed on a mirror image of the table, which is a real object (720).

[0129] FIG. 10 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0130] According to FIG. 10, the electronic device (100) can identify spatial information corresponding to the space where the user is located and the user's gaze information based on data acquired through the sensor (120) (S1010).

[0131] Next, the electronic device (100) can identify the display size and display position of the virtual object based on spatial information and the user's gaze information (S1020).

[0132] Next, the electronic device (100) can identify the size and direction of the shadow of the virtual object based on the context information and the display size and display position of the virtual object (S1030). According to an example, the context information may include at least one of time information and light source information. Here, the light source may include not only a natural light source such as the sun, but also various types of artificial light sources that emit light, such as electric lights. However, the present invention is not limited thereto and may also include virtual light source information selected by the user. Here, the time information may include not only standard time information but also virtual time information selected by the user.

[0133] Thereafter, the electronic device (100) can control the mirror display (110) to display the virtual object and the shadow of the virtual object based on the display size and display position of the virtual object and the size and direction of the shadow of the virtual object (S1040).

[0134] For example, the electronic device (100) can identify the shadow direction and ratio of a real object based on the size of the real object and the direction of a light source contained in the space where the user is located. Subsequently, the electronic device (100) can display the shadow of a virtual object based on the shadow direction and ratio of the real object.

[0135] According to one example, the electronic device (100) can identify the size and direction of the shadow of different types of virtual objects based on the type, intensity, etc. of the light source. For example, the electronic device (100) can identify the brightness, color, etc. of the shadow of the virtual object differently depending on the type of the light source. According to one example, the electronic device (100) can perform image processing so that the shadow of the virtual object is expressed without a sense of incongruity with the shadow of the real object based on the transmission characteristic information and reflection characteristic information of the mirror display (110).

[0136] FIGS. 11, 12 and 13 are drawings for explaining a method of displaying a shadow of a virtual object according to one or more embodiments.

[0137] According to FIG. 11, the electronic device (100) can identify the position and size (a) of a real object (720) in front of the electronic device (100) based on data acquired through the sensor (120), and can identify the direction, size, and ratio of a shadow (721) of the real object (720) caused by a light source.

[0138] The electronic device (100) can identify the size of the shadow (741) of the mirror image (740) of the real object, and identify the size of the shadow (751) of the virtual object (750) based on the size of the shadow (741) of the mirror image (740) of the real object.

[0139] The electronic device (100) can identify the shadow size of a virtual object (760) to be displayed on the mirror display (110) using perspective based on the shadow size (751) of the virtual object (750), and display the virtual object (760) and the corresponding shadow (761) on the mirror display (110). In one example, the electronic device (100) can identify the shadow size of the virtual object based on the principle of “size of virtual object: shadow size of virtual object = size of real object: shadow size of real object.” Here, the size of the virtual object can be identified using perspective based on the line of sight of the user (710).

[0140] According to FIG. 12, the electronic device (100) can implement various stereoscopic effects based on the environment of the virtual space.

[0141] As an example, a three-dimensional effect (3D effect) can be implemented by displaying a matte shadow together with a virtual object displayed on a mirror display (110) as shown in FIG. 12 (1210).

[0142] For example, as illustrated in FIG. 12, a three-dimensional effect can be implemented by displaying a shadow corresponding to the direction of a light source on a virtual object displayed on a mirror display (110) (1220). For example, a three-dimensional effect can be implemented by changing the direction of a shadow of a virtual object based on the direction of a light source according to time information.

[0143] For example, as shown in FIG. 12, a three-dimensional effect can be implemented by providing a video effect together with a virtual object displayed on a mirror display (110) (1230).

[0144] For example, as shown in FIG. 12, a virtual object displayed on a mirror display (110) can be displayed as a 3D shape with a transparent color to implement a three-dimensional effect (1240).

[0145] According to FIG. 13, the electronic device (100) can provide an ambient mode through a mirror display (110).

[0146] For example, as illustrated in FIG. 13, based on data acquired through a sensor (120), a direction of a shadow (711) of a user (710) may be identified, and based on the direction of the shadow (711) of the user (710), a direction of a shadow (1311, 1321, 1331) of a virtual object (1310, 1320, 1330) may be identified to provide an ambient mode. For example, as illustrated in FIG. 13, an aquarium mode including a plurality of virtual objects (1310, 1320, 1330) may be provided as an ambient mode.

[0147] FIG. 14 is a drawing for explaining a control method of an electronic device according to one or more embodiments.

[0148] According to FIG. 14, the electronic device (100) can identify space information corresponding to the space where the user is located, the user's line of sight information, and distance information between the user and the electronic device (100) based on data acquired through the sensor (120) (S1410).

[0149] Next, the electronic device (100) can identify the user's virtual body data based on the data acquired through the sensor (120) (S1420). For example, the user's virtual body data may be body data acquired through the sensor (120). For example, if an image of the user is acquired through the sensor (120), the virtual body data may be the user's body data included in the captured image.

[0150] Thereafter, the electronic device (100) can control the mirror display (110) to display the user's actual body data based on the distance information between the user and the electronic device (100) and the user's virtual body data. For example, the electronic device (100) can identify the user's actual body data based on the distance information between the user and the electronic device (100) and the user's virtual body data. For example, the distance information between the user and the electronic device (100) can be acquired through a distance sensor.

[0151] FIG. 15 is a diagram illustrating a method for displaying fitness information according to one or more embodiments.

[0152] According to FIG. 15, the electronic device (100) can display the user's body data in a size corresponding to the corresponding display position based on the distance to the user and the user's location acquired through the sensor.

[0153] For example, when the distance from the user is d1, the electronic device (100) can identify the size and position (1510) of the user reflected on the mirror display (110) and display the user's body data (1520) based on the size and position (1510) of the user.

[0154] For example, when the distance from the user is d2, the electronic device (100) can identify the size and position (1530) of the user reflected on the mirror display (110) and display the user's body data (1540) based on the size and position (1530) of the user.

[0155] FIG. 16 is a diagram illustrating a method for displaying fitness information according to one or more embodiments.

[0156] According to one embodiment, the electronic device (100) can identify spatial information corresponding to a space where the user is located, gaze information of the user, and distance information between the user and the electronic device (100) based on data acquired through the sensor (120). Then, the electronic device (100) can identify a virtual object corresponding to the user based on the distance information between the user and the electronic device (100) and the user's previous body data. Then, the electronic device (100) can control the mirror display (110) to display a virtual object corresponding to the user based on the spatial information, gaze information of the user, and distance information between the user and the electronic device (100).

[0157] As an example, according to FIG. 16, the electronic device (100) can identify the size and shadow size of the user in front of the mirror display (110) based on data acquired through the sensor (120), and display a virtual user (1620) corresponding to the user's previous body data based on the user's gaze position. In this case, the electronic device (100) can display the virtual user (1620) corresponding to the previous body data based on the distance (d1) between the electronic device (100) and the user.

[0158] For example, when the distance between the electronic device (100) and the user changes (d1 > d2), the electronic device (100) may display a virtual user (1620) corresponding to the previous body data based on the changed distance (d2) from the user. Accordingly, the user may compare the current appearance (1610) reflected on the mirror display (110) and the virtual appearance (1620) corresponding to the previous body data.

[0159] For example, when the surrounding environment changes, the electronic device (100) may change and display the background image of the mirror display (110) based on the changed surrounding environment. For example, during the day, the virtual user (1620) and a corresponding shadow may be displayed together on a bright background, and at night, the virtual user (1620) may be displayed on a dark background.

[0160] FIG. 17 is a diagram illustrating a method for displaying virtual objects for multiple users according to one or more embodiments.

[0161] According to one embodiment, when multiple users are identified based on data acquired through the sensor (120), the electronic device (100) can identify gaze information of each of the multiple users. In addition, the electronic device (100) can identify distance information between each of the multiple users and the electronic device (100) based on the data acquired through the sensor (120). Subsequently, the electronic device (100) can identify a display size and a display position of a virtual object based on spatial information, gaze information of the multiple users, and distance information between the multiple users and the electronic device (100). Thereafter, the electronic device (100) can control the mirror display (110) to display the virtual object based on the display size and display position of the identified virtual object.

[0162] According to an example, according to FIG. 17, when a plurality of users (711, 712) are identified in front of a mirror display (110), the electronic device (100) can identify the position of a mirror image (1720) of the real object (1710) based on the gaze information of the plurality of users (711, 712) and the position of the real object (1710), and display a virtual object (1730) at a position that appears as identical as possible to the plurality of users (711, 712) as a mirror image based on the position of the mirror image (1720) of the real object (1710).

[0163] FIG. 18 is a drawing for explaining a method for displaying virtual objects for an interior experience according to one or more embodiments.

[0164]

[0165] *According to FIG. 18, the electronic device (100) can display a virtual interior object for interior experience on the mirror display (110).

[0166] According to an example, the electronic device (100) may display a virtual interior object (1810, 1820) for an interior experience on the mirror display (110) based on at least one of a user command, a user preference, and a type of a real object. For example, the electronic device (100) may display a virtual interior object (1810, 1820) based on user preference information. For example, the electronic device (100) may display a virtual interior object (1810, 1820) that matches a real object based on the type of the real interior object acquired through the sensor (120). For example, the virtual interior object may include various interior props such as furniture, an aquarium, a flower pot, etc.

[0167] According to one or more embodiments, the electronic device (100) can identify the front situation of the mirror display (110) based on data acquired through the sensor (120) and recommend content corresponding to the front situation.

[0168] For example, the electronic device (100) can automatically recommend content by detecting the user's distance and then determining the user's physique. For example, if the user is determined to be a child, cartoons, children's programs, etc. can be automatically displayed. If the user is determined to be a pet, pet programs can be automatically displayed. In this case, the electronic device (100) can display recommended content at an appropriate location and size based on factors such as the distance from the real object, the location and size of the real object, etc.

[0169] As described above, according to various embodiments of the present disclosure, a virtual object that is not incongruous with the actual object reflected on the mirror display can be provided through the mirror display based on the user's gaze, location, spatial information, etc. Furthermore, an AR-based virtual object can be provided with a three-dimensional effect on the mirror display. Furthermore, a virtual object can be provided with a realistic sense of reality using 3D imaging technology.

[0170] Meanwhile, the methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0171] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0172] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0173] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0174] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0175] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, Mirror display; sensor; a memory storing at least one instruction; and comprising one or more processors connected to the mirror display, the sensor and the memory; The one or more processors, by executing the at least one instruction, Based on the data acquired through the above sensor, spatial information corresponding to the space where the user is located and the user's gaze information are identified, Identify the display size and display location of the virtual object based on the above spatial information and the user's gaze information, An electronic device that controls the mirror display to display the virtual object based on the display size and display position of the identified virtual object.

2. In paragraph 1, The one or more processors, by executing the at least one instruction, Identifying distance information between the user and the electronic device based on data acquired through the sensor; An electronic device that identifies the display size and display position of the virtual object based on the spatial information, the user's line of sight information, and the distance information between the user and the electronic device.

3. In paragraph 2, The one or more processors, by executing the at least one instruction, An electronic device that, when at least one of the user's gaze information or the distance information between the user and the electronic device is identified as having changed, re-identifies the display shape, display size, and display position of the virtual object based on the changed information.

4. In paragraph 2, The one or more processors, by executing the at least one instruction, Identifying distance information between the user and the electronic device based on data acquired through the sensor; Identifying the user's virtual body data based on the data acquired through the above sensor, An electronic device that controls the mirror display to display real body data of the user based on distance information between the user and the electronic device and virtual body data of the user.

5. In paragraph 2, The one or more processors, by executing the at least one instruction, Identifying distance information between the user and the electronic device based on data acquired through the sensor; An electronic device that controls the mirror display to display a virtual object corresponding to the user based on distance information between the user and the electronic device and previous body data of the user.

6. In paragraph 1, The one or more processors, by executing the at least one instruction, Based on the data acquired through the above sensor, the size and location of the real object contained in the space where the user is located are identified, An electronic device that identifies the size and location of a real object included in a space where the user is located, and the display size and location of a virtual object based on the user's gaze information.

7. In paragraph 1, The one or more processors, by executing the at least one instruction, Identifying the size and direction of the shadow of the virtual object based on the context information of the electronic device, the display size and display position of the virtual object, Controlling the mirror display to display the shadow of the virtual object based on the size and direction of the shadow of the virtual object; The above context information is, An electronic device comprising at least one of time information and light source information.

8. In paragraph 1, The one or more processors, by executing the at least one instruction, An electronic device that identifies the type of a virtual object based on the type of a real object included in a space where the user is located, and identifies the display size and display position of the virtual object based on the size and position of the real object.

9. In paragraph 8, The one or more processors, by executing the at least one instruction, Identify the shadow direction and ratio of the real object based on the size of the real object and the direction of the light source included in the space where the user is located, An electronic device that controls the mirror display to display a shadow of the virtual object based on the shadow direction and ratio of the real object.

10. In paragraph 1, The one or more processors, by executing the at least one instruction, When multiple users are identified based on the data acquired through the above sensor, the gaze information of each of the multiple users is identified, Identifying distance information between each of the plurality of users and the electronic device based on the data acquired through the sensor; An electronic device that identifies the display size and display position of the virtual object based on the spatial information, the line of sight information of the plurality of users, and the distance information between the plurality of users and the electronic device.

11. In a method for controlling an electronic device, A step of identifying spatial information corresponding to a space where a user is located and gaze information of the user based on data acquired through a sensor; A step of identifying the display size and display position of a virtual object based on the above spatial information and the user's gaze information; and A control method comprising: a step of displaying the virtual object through a mirror display based on the display size and display position of the identified virtual object.

12. In paragraph 11, The step of identifying the display size and display position of the above virtual object is: A step of identifying distance information between the user and the electronic device based on data acquired through the sensor; and A control method comprising: a step of identifying a display size and a display position of the virtual object based on the spatial information, the user's line of sight information, and the distance information between the user and the electronic device.

13. In paragraph 12, A control method further comprising: a step of re-identifying the display shape, display size, and display position of the virtual object based on the changed information, if at least one of the user's gaze information or the distance information between the user and the electronic device is identified as having changed; 14. In paragraph 12, A step of identifying distance information between the user and the electronic device based on data acquired through the sensor; A step of identifying the user's virtual body data based on the data acquired through the sensor; and A control method further comprising: a step of displaying real body data of the user through the mirror display based on distance information between the user and the electronic device and virtual body data of the user.

15. A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an operation, The above actions are, A step of identifying spatial information corresponding to a space where a user is located and gaze information of the user based on data acquired through a sensor; A step of identifying the display size and display position of a virtual object based on the above spatial information and the user's gaze information; and A non-transitory computer-readable medium, comprising: a step of displaying the virtual object through a mirror display based on the display size and display position of the identified virtual object.

Images