Electronic device and control method thereof

Abstract

An electronic device and a method for controlling the same are disclosed. Specifically, an electronic device according to the disclosure includes a first sensor for detecting a user, a second sensor for detecting light incident from an external light source, a mirror display, a memory, and a processor. Further, the processor: when a user in front of the mirror display is sensed by the first sensor, obtains information about a first area on the mirror display in which a first image corresponding to a mirror image of the user is displayed; determines a second area for displaying content to be provided to the user based on the information about the first area; when information about the light is obtained by the second sensor, determines an illuminance of the second area based on the information about the light; and controls the mirror display so as to display the content on the second area at the determined illuminance.

Description

Technical Field

[0001] This disclosure relates to an electronic device and a method for controlling the same, and more specifically, to an electronic device and a method for controlling the same that can improve the visibility of reflected content viewed on a mirror display. Background Technology

[0002] An electronic device equipped with a mirror display has been developed, which can display content while performing a mirror function. Specifically, the mirror display reflects light incident on it from an external light source to perform a mirror function, while simultaneously displaying content / information stored in the electronic device.

[0003] However, when an image corresponding to the user's reflection in front of the mirror display is reflected from the mirror display for the user to view according to the mirror function, there may be a problem that the visibility of the content / information displayed by the electronic device may be degraded due to light incident from external light sources and / or the user's image.

[0004] Therefore, considering the effects of light incident from external light sources and / or user images, it is necessary to improve the visibility of content / information displayed on a mirror display. Summary of the Invention

[0005] Technical issues

[0006] An electronic device and its control method are provided for improving the visibility of content displayed on a mirror display.

[0007] Technical solution

[0008] According to one aspect of this disclosure, the first electronic device may include: a first sensor configured to sense a user; a second sensor configured to sense light incident from an external light source; a mirror display; a memory; and a processor configured to: obtain information about a first area on the mirror display based on the first sensor sensing a user in front of it, wherein a first image corresponding to the user's image is located in the first area; determine a second area for displaying content to be provided to the user based on the information about the first area; determine the luminance of the second area based on the information about light obtained by the second sensor; and control the mirror display to display the content on the second area at the determined luminance.

[0009] Information about the first region may include information about its size and its location. The processor may also be configured to determine the location and size of the second region based on the information about the size and location of the first region.

[0010] Information about light may include information about the direction of light and information about the intensity of light. The processor may also be configured to: determine the position and size of a second region based on information about the first region and information about the direction of light, and determine the illuminance of the second region based on information about the intensity of light.

[0011] The first sensor may include an image sensor, and the processor may also be configured to: acquire a second image corresponding to the user via the image sensor, obtain information about the user's pose by inputting the second image into a trained neural network model, and determine the position and size of a first region based on the information about the user's pose.

[0012] The processor can also be configured to: identify the types of multiple objects within the content by inputting data about the content into a trained neural network model, and determine the illuminance of the area in the second region corresponding to each of the multiple objects based on information about light and the type of each of the multiple objects.

[0013] The processor can also be configured to determine the illuminance of the second area based on information about light, information about the type of content, and information about the brightness of the content.

[0014] The processor can also be configured to: determine the location of a user interface for controlling the content based on information about a first region and information about a second region, and control a mirror display to display the user interface at the determined location.

[0015] The processor can also be configured to: identify whether a user is a pre-registered user based on user data stored in memory; identify the content based on the user being a pre-registered user and user data corresponding to the pre-registered user; and control the mirror display to display the identified content on a second area.

[0016] The electronic device may also include a communication interface. User data may be health-related data received from an external device via the communication interface, and the content may include recommendations corresponding to the health-related data.

[0017] The processor can also be configured to: process content such that the boundary region between the first region and the second region is blended; and control the mirror display to display the processed content on the second region.

[0018] According to another aspect of this disclosure, a method for controlling an electronic device including a mirror display may include: obtaining information about a first area on the mirror display based on sensing a user in front of the mirror display, wherein a first image corresponding to the user's image is located in the first area; determining a second area for displaying content to be provided to the user based on the information about the first area; determining an illuminance in the second area based on information about light obtained by a second sensor; and displaying the content in the second area at the determined illuminance.

[0019] Information about the first region may include information about its size and its location. The steps for determining the second region may include: determining the location and size of the second region based on the information about the size and location of the first region.

[0020] Information about the light may include information about the direction of the light and information about the intensity of the light. The step of determining the second region may include: determining the location and size of the second region based on the information about the first region and the information about the direction of the light. The step of determining the illuminance of the second region includes: determining the illuminance of the second region based on the information about the light intensity.

[0021] The method may further include: obtaining a second image corresponding to the user; obtaining information about the user's pose by inputting the second image into a trained neural network model; and determining the position and size of a first region based on the information about the user's pose.

[0022] The content may include content containing multiple objects, and the step of determining the second region may include: identifying the types of the multiple objects by inputting data about the content into a trained neural network model; and determining the illuminance of the region in the second region corresponding to each of the multiple objects based on information about light and the type of each of the multiple objects.

[0023] The steps for determining the illuminance of the second area may include: determining the illuminance of the second area based on information about light, information about the type of content, and information about the brightness of the content.

[0024] The method may include: determining the location of a user interface for controlling the content based on information about a first region and information about a second region, and displaying the user interface at the determined location.

[0025] The method may further include: identifying whether a user is a pre-registered user based on user data stored in a memory; identifying the content based on user data corresponding to the pre-registered user if the user is a pre-registered user; and displaying the identified content on a second area.

[0026] User data is health-related data received from an external device via a communicator, and the content may include recommended content corresponding to the health-related data.

[0027] According to an embodiment, a non-transitory computer-readable recording medium includes a program for performing a method of controlling an electronic device, the method comprising: obtaining information about a first area on a mirror display based on sensing a user in front of the user, wherein a first image corresponding to the user is displayed in the first area; determining a second area for displaying content to be provided to the user based on the information about the first area; determining an illuminance in the second area based on information about light obtained by a second sensor; and displaying the content in the second area at the determined illuminance. Attached Figure Description

[0028] Figure 1 This is a block diagram illustrating the configuration of an electronic device according to embodiments of the present disclosure;

[0029] Figure 2 , Figure 3 and Figure 4 The operation of an electronic device according to an embodiment of the present disclosure is illustrated;

[0030] Figure 5 This is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure for determining a second region;

[0031] Figure 6 This is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure for determining a first region;

[0032] Figure 7 This is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure for determining illuminance differently for each object included in the content;

[0033] Figure 8 This is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure for displaying a user interface for controlling content;

[0034] Figure 9 This is a block diagram illustrating details of an electronic device according to embodiments of the present disclosure; and

[0035] Figure 10 This is a flowchart of a method for controlling an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0036] In the following description, embodiments of the present disclosure will be illustrated with reference to the accompanying drawings. However, it will be understood that the present disclosure is not limited to the embodiments described below, but also includes various modifications, equivalents, and / or substitutions of the embodiments of the present disclosure. Regarding the interpretation of the drawings, similar reference numerals may be used for similar constituent elements.

[0037] A detailed description of known techniques relating to this disclosure may be omitted when it is determined that such a description may unnecessarily obscure the spirit of the disclosure.

[0038] Furthermore, the following embodiments can be modified in many different forms, and the scope of the technical spirit of this disclosure is not limited to the following examples. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the technical spirit to those skilled in the art.

[0039] The terminology used herein is for describing certain embodiments and is not intended to limit the scope of the claims. Unless otherwise stated, singular expressions include plural expressions.

[0040] In this specification, the terms “have,” “may have,” “include,” “may include,” or “contain” indicate the presence of a corresponding feature (e.g., a component (such as a number, function, operation, or part)) and do not exclude the presence of additional features.

[0041] In this document, expressions such as “at least one of A and / or B” or “one or more of A and / or B” include all possible combinations of the listed items. For example, “at least one of A and B” or “at least one of A or B” includes any one of the following: (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0042] As used herein, the terms “first,” “second,” etc., may refer to various components regardless of order and / or importance, and may be used to distinguish one component from another without limiting the components.

[0043] The description of an element (e.g., a first element) being "operably or communicatively coupled" to another element (e.g., a second element) or "operably or communicatively coupled to" another element, or being "connected to" another element (e.g., a second element), should be understood to mean that the element can be directly connected to the other element or connected to the other element through yet another element (e.g., a third element).

[0044] On the other hand, the description that a certain element (e.g., the first element) is "directly coupled to" or "directly connected to" another element (e.g., the second element) can be understood to mean that there is no element (e.g., the third element) between the two elements.

[0045] Furthermore, as appropriate, the expression “configured as” as used in this disclosure may be used interchangeably with other expressions such as “suitable for,” “capable of,” “designed to,” “suitable for,” “manufactured as,” and “capable of.” At the same time, the term “configured as” does not necessarily mean that the device is “specifically designed for” in terms of hardware.

[0046] Conversely, in some cases, the phrase "a device configured to..." can mean that the device is "capable" of performing operations together with another device or component. For example, the phrase "a processor configured to perform A, B, and C" can refer to a dedicated processor (e.g., an embedded processor) for performing the respective operations, or a general-purpose processor (e.g., a central processing unit (CPU) or application processor) that can perform the respective operations by executing one or more software programs stored in a memory device.

[0047] Terms such as “module,” “unit,” and “component” are used to refer to an element that performs at least one function or operation, and such an element may be implemented as hardware or software, or a combination of hardware and software. Furthermore, except where each of multiple “modules,” “units,” and “components” needs to be provided in separate hardware, components may be integrated into at least one module or chip and provided in at least one processor.

[0048] The various elements and areas in the accompanying drawings are not shown to scale. Therefore, the scope of this disclosure is not limited by the relative dimensions or spacing drawn from the accompanying drawings.

[0049] In the following, embodiments will be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can readily make and use the embodiments.

[0050] Figure 1 This is a block diagram illustrating the configuration of an electronic device according to embodiments of the present disclosure.

[0051] like Figure 1 As shown, the electronic device 100 according to the embodiment may include a mirror display 110, a sensor 120, a memory 130, and a processor 140.

[0052] Electronic device 100 may be implemented as a mirror display 110 having a shape such as a standing mirror. However, electronic device 100 is not limited to the type and shape of electronic device 100 in the disclosed embodiments, and electronic device 100 may be implemented as a TV or a smartphone.

[0053] The mirror display 110 can be configured to display content while performing a mirror function. That is, an image corresponding to an object placed in front of the mirror display 110 can be reflected from the mirror display 110, and while displaying the image corresponding to the object placed in front of the mirror display 110, content stored in the memory 130 can be displayed like a conventional display. The mirror display 110 may include a display panel and a mirror panel disposed on the display panel, and may also include a touchscreen panel disposed on the mirror panel.

[0054] When light incident from an external light source is reflected on the mirror display 110, an image corresponding to an object located in front of the mirror display 110 can be reflected from an area of ​​the mirror display 110. In other words, if an object is located in front of the mirror display 110, an image of the object can be provided on an area of ​​the mirror display 110. When user input for displaying content is received or an event for displaying content is generated, the mirror display 110 can display content stored in the memory 130. The content stored in the memory 130 may include not only content acquired and stored in the memory 130 by the electronic device 100, but also content received from external devices and stored in the electronic device 100.

[0055] Sensor 120 can be configured to sense various information, either internal or external to electronic device 100. Sensor 120 may include one or more sensors 120, and may include a first sensor 120 for sensing a user and a second sensor 120 for sensing light incident from an external light source. The first sensor 120 may include at least one of an image sensor 120, a proximity sensor 120, and an infrared sensor 120. The second sensor 120 may include at least one of an illuminance sensor 120 and a color sensor 120. Sensor 120 may include various types of sensors 120, such as a Global Positioning System (GPS) sensor 120, a gyroscope sensor 120, an accelerometer sensor 120, a LiDAR sensor 120, an inertial measurement unit (IMU) 120, a motion sensor 120, etc.

[0056] At least one instruction relating to the electronic device 100 may be stored in the memory 130. Additionally, an operating system (O / S) for driving the electronic device 100 may be stored in the memory 130. The memory 130 may store various software programs or applications for operating the electronic device 100 according to various embodiments. The memory 130 may include semiconductor memory (such as flash memory), magnetic storage media (such as hard disks), etc.

[0057] Specifically, the memory 130 may store various software modules for operating the electronic device 100, and the processor 140 (e.g., at least one processor) may control the operation of the electronic device 100 by executing the various software modules stored in the memory 130. That is, the memory 130 may be accessed by the processor 140, and the processor 140 may perform data reading, recording, modification, deletion, updating, etc.

[0058] The term memory 130 may be used to refer to any volatile or non-volatile memory, ROM, RAM, or memory card (e.g., Micro Secure Digital (SD) card, Memory Stick) that is close to or in the processor 140 or installed in the electronic device 100.

[0059] In various embodiments, memory 130 may store various information or data as described below. Memory 130 may store information about the user sensed by the first sensor 120, information about the direction and intensity of light sensed by the second sensor 120, information about the position and size of the first region 21, information about the position and size of the second region 22, information about the illuminance of the second region 22, and other content. Memory 130 may store various types of user data. (Refer to...) Figure 9 Specific, non-limiting examples and embodiments of user data are described in detail.

[0060] Various information required for achieving the purposes of this disclosure may be stored in memory 130, and the information stored in memory 130 may be received from external devices or updated by being received by a user.

[0061] The processor 140 can control the overall operation of the electronic device 100. Specifically, the processor 140 can be connected to a configuration of the electronic device 100 including the mirror display 110, sensor 120 and memory 130 as described above, and can control the overall operation of the electronic device 100 by executing at least one instruction stored in the memory 130 as described above.

[0062] Processor 140 can be implemented in various ways. For example, processor 140 can be implemented as at least one of application-specific integrated circuit (ASIC), embedded processor, microprocessor, hardware control logic, hardware finite state machine (FSM), digital signal processor (DSP), etc. In addition, processor 140 may include at least one of central processing unit (CPU), graphics processing unit (GPU), main processing unit (MPU), etc.

[0063] According to an embodiment, the processor 140 can sense a user in front of the mirror display 110 via the first sensor 120. The processor 140 can sense the user in front of the mirror display 110 via at least one of an image sensor 120, a proximity sensor 120, and an infrared sensor 120 included in the first sensor 120.

[0064] like Figures 3 to 8 As shown, when a user is positioned in front of the mirror display 110, when light incident from an external light source is reflected on the mirror display 110, a first image 11 corresponding to the user can be reflected from a region of the mirror display 110. Hereinafter, the user image reflected from a region of the mirror display 110 when light incident from an external light source is reflected on the mirror display 110 is referred to as "first image 11".

[0065] When a user 10 is sensed in front of the mirror display 110, the processor 140 obtains information about a first region 21, on which the user's image is reflected from the mirror display 110. The information about the first region 21 may include information about its size and its position. The position of the first region 21 may be based on the user's position, such as the position of the user's eyes. That is, the first region 21 may be based on the user's viewing angle.

[0066] The processor 140 may determine a second region 22 for displaying content to be provided to the user 10 based on information about the first region 21. The processor 140 may determine the position and size of the second region 22 for displaying content based on information about the size of the first region 21 and information about the position of the first region 21. According to an embodiment, the second region 22 may be determined as a region on the mirror display 110 that does not overlap with the first region 21.

[0067] The processor 140 can obtain information about light incident from the external light source 30 via the second sensor 120. The information about the light may include information about the direction of the light and information about the intensity of the light.

[0068] When information about light is obtained through the second sensor 120, the processor 140 can determine the illuminance of the second region 22 based on the information about light and display content on the second region 22. According to an embodiment, the processor 140 can determine the illuminance of the second region 22 in accordance with the intensity of light incident on the second region 22 and display content on the second region 22 with the determined illuminance.

[0069] Reference Figures 2 to 9 Various embodiments of control based on processor 140 are further described.

[0070] Figure 2 , Figure 3 and Figure 4 The operation of an electronic device according to an embodiment of the present disclosure is illustrated.

[0071] When user 10 is not in front of mirror display 110, processor 140 can control mirror display 110 to display such as Figure 2 The standby screen 1 shown is an example. For instance, standby screen 1 may include information about the current time, such as... Figure 2 As shown, it can include various information, such as information about the current weather.

[0072] When user 10 approaches the front of mirror display 110 while the standby screen 1 is displayed, such as Figure 3 As shown, when light incident from the external light source 30 is reflected on the mirror display 110, a first image 11 corresponding to the image of the user 10 can be reflected from a region of the mirror display 110. When the first image 11 is displayed in a region of the mirror display 110, the processor 140 can stop displaying the standby screen 1, as... Figure 3 As shown in the image.

[0073] The processor 140 can sense the user 10 in front of the mirror display 110 via the first sensor 120. The processor 140 can acquire an image of the scene in front of the mirror display 110 via the image sensor 120 included in the first sensor 120, perform object recognition on the acquired image, and sense the user 10 in front of the mirror display 110. The processor 140 can sense the user 10 in front of the mirror display 110 via at least one of the proximity sensor 120 and the infrared sensor 120 included in the first sensor 120.

[0074] When a user 10 is sensed in front of the mirror display 110, the processor 140 can obtain information about a first region 21 of the mirror display 110. The processor 140 can identify pixels of the mirror display 110 corresponding to the first image 11, and identify the position and size of the first region 21 including the identified pixels. Figure 4 As shown, the first region 21 can be the central region of the mirror display 110.

[0075] Processor 140 may determine a second region 22 for displaying content to be provided to user 10 based on information about the first region 21. Processor 140 may determine the position and size of the second region 22 for displaying content based on information about the size of the first region 21 and information about the position of the first region 21. Figure 4As shown, the content to be provided to user 10 may include health-related data related to user 10's health, and the second region 22 may be defined as the upper right region of the mirror display 110 in the region that does not overlap with the first region 21.

[0076] The processor 140 can obtain information about light incident from the external light source 30 via the second sensor 120. This information may include information about the direction and intensity of the light. When the processor 140 obtains this information about the light via the second sensor 120, it can determine the illuminance of the second region 22 based on this information and display content on the second region 22 at the determined illuminance.

[0077] The visibility of user 10 relative to the content displayed on the second area 22 may vary depending on the direction and intensity of the light incident from the external light source 30. For example, when high-intensity light is incident on the mirror display 110 from the external light source 30 located in the upper right direction of the electronic device 100, the visibility of user 10 relative to the content displayed on the second area 22 at the upper right end of the mirror display 110 may be reduced.

[0078] Processor 140 can adaptively determine the illuminance of the second region 22 based on the direction and intensity of light incident from external light source 30, thereby improving the visibility of content displayed by user 10 relative to the second region 22. Processor 140 can determine the illuminance of the second region 22 to correspond to the intensity of light incident on the second region 22. According to an embodiment, processor 140 can determine the illuminance of the second region 22 as proportional to the intensity of light incident on the second region 22. According to another embodiment, data corresponding to illuminance levels according to a plurality of light intensity levels can be pre-stored in memory 130, and processor 140 can determine the illuminance of the second region 22 based on the illuminance levels corresponding to the intensity levels of light incident on the second region 22.

[0079] Based on the above reference Figures 2 to 4 In the described embodiment, the electronic device 100 may determine the area for displaying content to be provided to the user 10 based on information about the area where the user's image is displayed according to the mirror function, and adjust the illuminance of the area for displaying the content based on information about the light incident on the mirror display 110, thereby improving the visibility of the user 10 relative to the content displayed on the mirror display 110.

[0080] Figure 5 This is a diagram illustrating an embodiment for determining a second region.

[0081] Reference Figures 2 to 4The second region 22 is determined based on information about the first region 21; however, this is merely exemplary, and according to this disclosure, it is not necessary to use only information about the first region 21 when determining the second region 22. According to another embodiment of this disclosure, the processor 140 may use information about light incident from the external light source 30 and information about the first region 21 to determine the second region 22.

[0082] As described above, when user 10 approaches the front surface of mirror display 110, a first image 11 corresponding to user 10 can be reflected from a region of mirror display 110. When user 10 is sensed in front of mirror display 110, processor 140 can obtain information about the first region 21 of mirror display 110. Processor 140 can obtain information about light incident from external light source 30 via second sensor 120.

[0083] When information about the first region 21 and information about light are obtained, the processor 140 can determine the position and size of the second region 22 based on the information about the first region 21 and information about light, and determine the illuminance of the second region 22 based on the information about light.

[0084] For example, such as Figure 5 As shown, when high-intensity light from an external light source 30 positioned in the upper right direction of the electronic device 100 is incident on the mirror display 110, specifically, the visibility of the user 10 relative to the content displayed in the upper right region of the display 110 can be reduced, while the visibility of the user 10 relative to the content displayed in the lower left region of the mirror display 110 can be relatively high. Therefore, the processor 140 can select a region on the mirror display 110 that does not overlap with the first region 21, specifically the lower left side of the mirror display 110 instead of the upper right side, as the second region 22.

[0085] However, Figure 5 The embodiments described are merely embodiments, and the second region 22 may be defined as the region at the upper left or lower right end of the mirror display 110, and at least a portion of the second region 22 may be defined as overlapping with at least a portion of the first region 21. If it is determined that at least a portion of the second region 22 overlaps with at least a portion of the first region 21, the processor 140 may process content such that the boundary region between the first image 11 reflected from the first region 21 and the content displayed on the second region 22 is blended.

[0086] The processor 140 can determine the second region 22 based on information about the user 10's gaze. The processor 140 can obtain information about the user 10's gaze through the sensor 120, and determine a region including the position on the mirror display 110 corresponding to the user 10's gaze as the second region based on the information about the user 10's gaze.

[0087] Based on the above reference Figure 5 In the described embodiment, the electronic device 100 can determine the area for displaying content to be provided to the user 10 by using information about the area where the user's image is displayed according to the mirror function and information about the light incident on the mirror display 110, thereby improving the visibility of the user 10 relative to the content displayed on the mirror display 110.

[0088] Figure 6 This is a diagram illustrating an embodiment for determining a first region.

[0089] Based on the assumption that the user 10 in front of the mirror display 110 moves relatively little, a method for obtaining information about the first region 21 (where the first image 11 is reflected from the mirror display 110 in the first region 21) is described. However, if the user 10 moves significantly, it may greatly limit the area for displaying content based on the user image reflected from the mirror display 110. Therefore, the processor 140 can use information about the user 10's posture to determine a second region 22 for displaying content.

[0090] In detail, the first sensor 120 may include an image sensor 120, and the processor 140 may acquire a second image corresponding to the user 10 via the image sensor 120. The image sensor 120 may refer to a sensor 120 capable of converting light entering through the lens of a camera into an electrical image signal, and may be included as a component of a camera. In this disclosure, acquiring the second image via the image sensor 120 may be replaced by acquiring the second image via a camera.

[0091] The user image obtained by the image sensor 120 is referred to as the "second image" to distinguish it from the term "first image 11," which refers to the user image reflected from an area of ​​the mirror display 110 when light incident from the external light source 30 is reflected on the mirror display 110.

[0092] Once the second image is acquired, the processor 140 can input it into a trained first neural network model to obtain information about the pose of the user 10. The "first neural network model" refers to a model trained to obtain information about the pose of the user 10 included in the input user image. Specifically, the processor 140 can detect the user 10 included in the second image using the first neural network model and obtain image data of the region including the user 10. The processor 140 can obtain the pose information of the user 10 based on information about feature points or skeletons of the user 10 included in the image data.

[0093] If information about the user 10's posture is obtained as described above, the processor 140 can determine the position and size of the first region 21 based on the information about the user 10's posture. The processor 140 can estimate, based on the information about the user 10's posture, the area in which the first image 11 will be displayed according to the user 10's posture during a predetermined time period, and determine the estimated area as the first region 21. For example, as... Figure 6 As shown, the processor 140 can estimate, based on information about the user 10's posture, that the first image 11 may arrive at the top of the mirror display 110 during a predetermined time period, thereby displaying... Figure 6 The area 21 shown is designated as the first area 21. Here, the predetermined time period can be changed by the developer or user 10.

[0094] As described above, if a first region 21 is determined based on information about the user 10's posture, the processor 140 can determine a second region 22 for displaying content based on information about the first region 21, determine the illuminance of the second region 22 based on information about light, and display content on the second region 22 with the determined illuminance. For example, as Figure 6 As shown, the second region 22 can be defined as the upper right region of the mirror display 110, which is a region on the mirror display 110 that does not overlap with the first region 21.

[0095] Processor 140 can identify content to be provided to user 10 based on information about user 10's posture. Processor 140 can estimate user 10's exercise movements based on the information about user 10's posture and identify content corresponding to the estimated movements. For example, such as... Figure 6 As shown, if user 10 is assumed to be performing a "rush" movement based on information about user 10's posture, processor 140 can identify recommended content to guide the "rush" movement and control mirror display 110 to display the identified recommended content on second area 22.

[0096] Reference Figure 6The electronic device 100 can determine the area for displaying the user image according to the mirror function during a predetermined interval based on information about the user 10's posture, and specifically, when the user 10 moves a lot, it can prevent the area of ​​the content displayed on the mirror display 110 from being limited by the user image reflected from the mirror display 110.

[0097] Figure 7 This is a diagram illustrating an embodiment for determining illuminance differently for each object included in the content.

[0098] In the preceding text, information about the light incident from the external light source 30 can be obtained via the second sensor 120, and content can be displayed by determining the illuminance of the second region 22 in accordance with the intensity of the light incident on the second region 22. More specifically, the illuminance of the second region 22 can be determined differently depending on the objects included in the content. In the following text, in Figure 7 In the description, similar to Figure 6 As described above, the second region 22 is defined as the upper right region of the mirror display 110 that does not overlap with the first region 21.

[0099] Specifically, such as Figure 7 As shown, the content displayed in the second region 22 may include multiple objects. As an example, the multiple objects may be categorized as "people" and "background". The processor 140 may input data about the content into a trained second neural network model to identify the type of each object among the multiple objects included in the content. Here, the "second neural network model" is a neural network trained to output information about the probability of each of the multiple objects included in the content relative to a plurality of predefined categories (or classes, domains) based on data about the content.

[0100] The processor 140 can determine the illuminance of the second region 22 for each region corresponding to each of the multiple objects included in the content, based on information about light and the type of each object among the multiple objects. For example, as Figure 7 As shown, when the external light source 30 is positioned at the upper right end of the electronic device 100, the visibility of the content displayed by the user 10 relative to the content shown on the second area 22 at the upper right end of the mirror display 110 may be degraded. Figure 7 As shown, processor 140 can determine the illuminance of a “person” among multiple objects included in the content as high and the illuminance of the “background” as low.

[0101] Processor 140 can determine the illuminance of the second region 22 based on information about light, information about the type of content, and information about the brightness of the content. Memory 130 can store information about the type (or category) of content and information about the brightness of the content, and processor 140 can determine the illuminance of the second region 22 by using information about the type of content, information about the brightness of the content, and information about light. For example, if the content is a "battle movie" and the average brightness of the image frames included in the content is less than a predetermined threshold, processor 140 can determine that the illuminance of the second region 22 is relatively lower than the illuminance when the illuminance of the second region 22 is determined solely based on information about light.

[0102] Based on the above reference Figure 7 In the described embodiments, the electronic device 100 can significantly improve the visibility of the content to the user 10 by determining the illuminance of the area displaying the content differently based on the objects included in the content.

[0103] Figure 8 The operation of displaying a user interface for controlling content is shown according to an embodiment.

[0104] In the above description, a first image 11 corresponding to the user 10 in front of the mirror display 110 is reflected from the first area 21, and the content to be provided to the user 10 is displayed on the second area 22, while a user interface (UI) for controlling the content can be displayed on the mirror display 110. In the following, as... Figure 8 As shown, the first region 21 is defined as the central region of the mirror display 110, and the second region 22 is defined as the upper right region of the mirror display 110 in the region that does not overlap with the first region 21.

[0105] The processor 140 may determine the location of the user 10 interface for controlling content based on information about the first region 21 and information about the second region 22, and control the mirror display 110 to display the user 10 interface at the determined location. For example, the processor 140 may control the mirror display 110 to display the user 10 interface in an area that overlaps with a portion of the first region 21, wherein a first image 11 corresponding to the user 10 is provided to facilitate touch input by the user 10.

[0106] The user interface may include multiple UI items 23-1, 23-2, 23-3, 23-4, and 23-5 for controlling the content displayed on the second area 22.

[0107] like Figure 8As shown, the user interface may include UI item 23-1 corresponding to the "rewind" function, UI item 23-2 corresponding to the "move to previous content" function, UI item 23-3 corresponding to the "move to next content" function, UI item 23-4 corresponding to the "fast forward" function, and UI item 23-5 corresponding to the "play / pause" function. UI item 23-2 corresponding to the "move to previous content" function and UI item 23-5 corresponding to the "play / pause" function may overlap with a portion of the first area 21 displaying the first image 11. In this example, the processor 140 may process the images corresponding to UI items 23-2 and 23-5 such that the boundary area between UI items 23-2 and 23-5 is blended, thereby coordinating the UI items with the first image 11.

[0108] Processor 140 can process the images corresponding to UI items 23-2 and 23-5 such that the boundary regions between the first image 11 and UI items 23-2 and 23-5 can be blended, but can also process the content to be blended with the boundary regions between the first image 11 and the content. Processor 140 can process the content such that the boundary regions between the first region 21 and the second region 22 are blended, and controls the mirror display 110 to display the processed content on the second region 22.

[0109] Based on the above reference Figure 8 In the described embodiment, the electronic device 100 can display a user image reflected from the mirror display 110 according to the mirror function, a user interface for controlling content, and content to be provided to the user 10, thereby improving the convenience of the user 10.

[0110] Figure 9 This is a block diagram illustrating details of an electronic device according to an embodiment.

[0111] like Figure 9 As shown, the electronic device 100 according to an embodiment may further include a communication interface 150, a microphone 160, a speaker 170, a mirror display 110, a sensor 120, a memory 130, and a processor 140. (See above for reference...) Figures 1 to 8 The mirror display 110, sensor 120, memory 130, and processor 140 have been described, and repetitive descriptions will be omitted. However, before describing the communication interface 150, microphone 160, and speaker 170, the “user data” stored in memory 130 and related embodiments will be described.

[0112] like Figure 9As shown, memory 130 can store user data according to this disclosure. Here, "user data" is used to refer to various data related to user 10 of electronic device 100. Specifically, user data may include user 10's identification information, personal information, and health-related information.

[0113] "Identification information" may include facial images, fingerprint data, iris data, and account information used to identify User 10. "User personal information" may include information about User 10's personal information and schedule, etc. "Health-related information" may include information about User 10's heart rate, body temperature, weight, body composition, sleep patterns, stress levels, hormone levels, mood, and exercise history, etc.

[0114] User data can be obtained through various types of sensors 120 included in electronic device 100, or through various types of sensors 120 included in external devices, and can be received from external devices through the communication interface 150 of electronic device 100. Here, the external device can be a user terminal, such as a smartphone, smartwatch, etc., or it can be a cloud server or edge computing device configured to store / manage user data.

[0115] According to an embodiment, the processor 140 can identify whether the user 10 in front of the mirror display 110 is a pre-registered user 10 based on user data. The processor 140 can identify whether the user 10 in front of the mirror display 110 is a pre-registered user 10 by whether the identification information stored in the memory 130 matches the identification information obtained through the sensor 120 (e.g., facial image, fingerprint data, or iris data). Additionally, the processor 140 can identify whether the user 10 is a pre-registered user 10 by whether the account information stored in the memory 130 matches the account information entered by the user 10.

[0116] When it is determined that the user 10 in front of the mirror display 110 is not a pre-registered user 10, the processor 140 can control the mirror display 110 to display a basic screen including information such as the current time and current weather.

[0117] When a user 10 in front of the mirror display 110 is identified as a pre-registered user 10, the processor 140 can output a greeting message based on the user 10's personal information. For example, the processor 140 can control the mirror display 110 to display a greeting message such as "Hello, Mr. Kim", and can output a voice signal corresponding to "Hello, Mr. Kim" through the speaker 170.

[0118] If the user 10 in front of the mirror display 110 is a pre-registered user 10, the processor 140 can control the mirror display 110 to display health-related data corresponding to the pre-registered user 10. The processor 140 can control the mirror display 110 to display information about the user 10's heart rate, body temperature, weight, body composition, sleep patterns, stress levels, hormone levels, mood, exercise history, etc., where the user 10 is identified as a pre-registered user 10.

[0119] If the user 10 in front of the mirror display 110 is a pre-registered user 10, the processor 140 can identify the content to be provided to the user 10 based on health-related data corresponding to the pre-registered user 10, and control the mirror display 110 to display the identified content on the second area 22. The processor 140 can identify content such as content stored in the memory 130 that has recently been played by the user 10 identified as a pre-registered user 10, content set as preferred by the user 10, and at least some of the corresponding recommended content related to the user 10's health-related information, and display at least one of the identified contents on the second area 22.

[0120] The process of identifying recommended content to be provided to user 10 based on user data can be performed through a trained third neural network model. The "third neural network model" is trained to identify content corresponding to user 10's personal information and health-related data, and the electronic device 100 can use the third neural network model to identify recommended content to be provided to user 10.

[0121] Although the content provided to user 10 is stored in the memory 130 of electronic device 100, the content provided to user 10 may be stored in an external device, and electronic device 100 may receive content from an external device and provide the received content to user 10.

[0122] According to the embodiments described above, when the user 10 in front of the mirror display 110 is a pre-registered user 10, the electronic device 100 can provide personalized health-related data and content to the user 10 in front of the mirror display 110 based on user data, thereby helping the user 10 to manage / improve their health status.

[0123] The communication interface 150 may include circuitry and be capable of communicating with external devices. The processor 140 may receive various data or information from external devices connected via the communication interface 150 and may send various data or information to external devices.

[0124] The communication interface 150 may include at least one of a Wi-Fi module, a Bluetooth module, a wireless communication module, and a near-field communication (NFC) module. Specifically, the Wi-Fi module can communicate via Wi-Fi, and the Bluetooth module can communicate via Bluetooth. When using the Wi-Fi module or the Bluetooth module, various connection information such as a Service Set Identifier (SSID) can be sent and received to establish a communication connection, and then various information can be sent and received.

[0125] The wireless communication module can communicate according to various communication standards (such as IEEE, Zigbee, 3G, 3GPP, LTE, 5G, etc.). The NFC module can communicate via NFC using the 13.56MHz band from various RF-ID frequency bands (such as 135kHz, 13.56MHz, 433MHz, 860-960MHz, 2.45GHz, etc.).

[0126] According to various embodiments, the processor 140 can receive user data as described above from an external device via the communication interface 150, and can store the received user data in the memory 130.

[0127] Microphone 160 can receive voice signals and convert the received voice signals into electrical signals. According to an embodiment, processor 140 can receive voice signals corresponding to the voice of user 10, input the received voice signals into a trained speech recognition model to obtain text information corresponding to the voice signals, and recognize control commands corresponding to the obtained text information. For example, the user's voice may correspond to control commands for changing the position or size of the second area 22, for changing the type of content displayed on the second area 22, or for controlling the content displayed on the second area 22.

[0128] Speaker 170 can output audio data under the control of processor 140. According to an embodiment, speaker 170 can output voice signals corresponding to the content displayed on the second area 22, feedback voice signals corresponding to the voice of user 10 received through microphone 160, or greeting messages for pre-registered user 10.

[0129] Although various configurations of the electronic device 100 have been described above, these configurations are merely exemplary, and in practice, in addition to those described above, other configurations may also be used. Figure 9 In addition to the configuration shown, new configurations can be added or some configurations can be omitted.

[0130] Figure 10This is a flowchart of a method for controlling an electronic device according to an embodiment.

[0131] like Figure 10 As shown, during operation S1010, the electronic device 100 can sense a user in front of the mirror display. When no user is detected in front of the mirror display during operation S1010-N, the electronic device 100 can display a standby screen until a user is sensed in front of the mirror display.

[0132] If a user is sensed in front of the mirror display during operation S1010-Y, then during operation S1020, the electronic device 100 can obtain information about a first area, wherein a first image corresponding to the user's image is reflected from the mirror display in the first area. The information about the first area may include information about the size of the first area and information about the location of the first area.

[0133] In operation S1030, the electronic device 100 may determine a second area for displaying content to be provided to the user based on information about the first area. The electronic device 100 may determine the position and size of the second area for displaying the content based on information about the size of the first area and information about the position of the first area. According to an embodiment, the second area may be determined as one of the areas on the mirror display that does not overlap with the first area.

[0134] The electronic device 100 can obtain information about light incident from an external light source via a second sensor. This information may include information about the direction of the light and information about its intensity.

[0135] When information about light is obtained through the second sensor, in operation S1040, the electronic device 100 can determine the illuminance of the second area based on the information about light. In operation S1050, the electronic device 100 can display content on the second area with the determined illuminance. According to an embodiment, the processor can determine the illuminance of the second area to correspond to the intensity of light incident on the second area, and display content on the second area with the determined illuminance.

[0136] The control method of the electronic device 100 according to the foregoing embodiments can be implemented as a program and provided to the electronic device 100. Specifically, the program including the control method of the electronic device 100 can be stored in a non-transitory computer-readable medium and provided.

[0137] Specifically, in a computer-readable recording medium including a program for performing a control method of the electronic device 100, the control method of the electronic device 100 may include the following steps: when a user is sensed in front of a mirror display, obtaining information about a first area, wherein a first image corresponding to the user is reflected from the mirror display in the first area; determining a second area for displaying content to be provided to the user based on the information about the first area; when information about light incident from an external light source is obtained, determining the illuminance of the second area based on the information about the light; and displaying the content in the second area with the determined illuminance.

[0138] As described above, the electronic device 100 can use information about light incident from an external light source and information about a first area to determine a second area, use information about the user's posture to determine a second area for displaying content, and can determine the illuminance of the second area differently depending on the objects included in the content.

[0139] Non-transitory computer-readable media refers to media that store data semi-permanently rather than for a very short period of time, such as registers, caches, and memories, and can be read by a device (i.e., executed by at least one processor). Specifically, the various applications or programs described above can be stored in non-transitory computer-readable media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), hard disks, Blu-ray discs, universal serial bus (USB), memory cards, read-only memory (ROM), etc.) and can be provided.

[0140] A control method for electronic device 100 and a computer-readable and recordable medium including a program for executing the control method for electronic device 100 are described, but this is to omit redundant descriptions, and various embodiments of electronic device 100 can be applied to the control method for electronic device 100 and the computer-readable and recordable medium including a program for executing the control method for electronic device 100.

[0141] The functions related to the foregoing can be executed by memory and processor.

[0142] The processor may include one or more processors. In this case, one or more processors may be general-purpose processors (such as central processing unit (CPU), application processor (AP), etc.), graphics-only processing units (such as graphics processing unit (GPU)), vision processing unit (VPU) and / or AI-specific processors (such as neural processing unit (NPU)).

[0143] One or more processors control the processing of input data based on predefined operating rules or artificial intelligence (AI) models stored in non-volatile and volatile memory. These predefined operating rules or AI models are provided through training or learning.

[0144] Learning provides, for example, predefined operating rules or AI models that exhibit desired characteristics by applying learning algorithms to multiple learning datasets. Learning can be performed within the device itself that executes the AI ​​according to the embodiment, and / or can be implemented via a separate server / system.

[0145] AI models can include multiple neural network layers. Each layer has multiple weight values, and layer operations are performed by computing the previous layer and operating on the multiple weights. Examples of neural networks include, but are not limited to, Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Generative Adversarial Networks (GANs), and Deep Q-Networks.

[0146] Learning algorithms can include methods for training a predetermined target device (e.g., a robot) using multiple learning data to enable, allow, or control the target device to make determinations or predictions. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0147] Machine-readable storage media may be provided in the form of non-transitory storage media. "Non-transitory" storage media may not include signals (e.g., electromagnetic waves) and may be tangible, but does not distinguish whether data is stored permanently or temporarily in the storage medium. For example, "non-transitory storage media" may include buffers for temporarily storing data.

[0148] According to various embodiments, the methods disclosed herein may be provided in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., an optical disc ROM (CD-ROM)), or distributed online through an app store (e.g., Play Store™), or distributed online directly between two user devices (e.g., smartphones) (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be temporarily or at least temporarily stored in a storage medium (such as a manufacturer's server, a server in an app store, or memory in a relay server).

[0149] Additionally, each component (e.g., a module or program) according to various embodiments may include a single entity or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be included in various embodiments. Optionally or additionally, some components (e.g., modules or programs) may be integrated into a single entity to perform the same or similar functions performed by the respective components prior to integration.

[0150] According to various embodiments, operations performed by modules, programs or other components may be performed sequentially, in parallel, iteratively or heuristically, or at least some operations may be performed in a different order or omitted, or other operations may be added.

[0151] As used in this disclosure, the terms "unit" or "module" include a unit comprised of hardware, software, or firmware, and are used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A "unit" or "module" can be a component or the smallest unit or part thereof of an integral structure that performs one or more functions. For example, a module may be configured as an application-specific integrated circuit (ASIC).

[0152] The embodiments may be implemented as software, which includes instructions stored in a machine-readable storage medium that is machine-readable (e.g., a computer). The apparatus may invoke the instructions from the storage medium and operate according to the invoked instructions, including electronic devices (e.g., electronic device 100).

[0153] When instructions are executed by the processor, the processor may directly perform the function corresponding to the instruction or, under the processor's control, use other components to perform the function corresponding to the instruction. Instructions may include code generated or executed by a compiler or interpreter.

[0154] While embodiments of the present disclosure have been shown and described, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined at least by the appended claims and their equivalents.

Claims

1. An electronic device comprising: camera, The first sensor is configured to sense the user; The second sensor is configured to sense light incident from an external light source; Mirror display; as well as The processor is configured as follows: Based on the first sensor sensing a user in front of the mirror display, information about a first area on the mirror display is obtained, wherein a first image corresponding to the user's reflection is located in the first area. The second area is determined based on information about the first area, which is used to display the content to be provided to the user. Based on the information about light obtained through the second sensor, the illuminance of the second region is determined based on the information about light, and The mirror display is controlled to show the content in the second area at a determined illuminance. The step of obtaining information about a first area on the mirror display includes: A second image corresponding to the user is obtained through the camera. Information about the user's pose is obtained by inputting a second image into a trained neural network model. Based on information about the user's posture, the area on the mirror display to display the first image according to the user's movement during a predetermined time period is estimated. The estimated area is designated as the first region, and information about the estimated region is obtained as information about the first region.

2. The electronic device according to claim 1, wherein, Information about the first region includes information about its size and its location, and The processor is also configured to determine the position and size of the second region based on information about the size of the first region and information about the position of the first region.

3. The electronic device according to claim 1, wherein, The information about light includes information about the direction of the light and information about the intensity of the light, and The processor is also configured as follows: The location and size of the second region are determined based on information about the first region and information about the direction of light. The illuminance of the second region is determined based on information about the intensity of the light.

4. The electronic device according to claim 1, wherein, The processor is also configured as follows: By inputting data about the content into a trained neural network model, the types of multiple objects within the content are identified, and Based on the information about light and the type of each of the plurality of objects, the illuminance of the area in the second region corresponding to each of the plurality of objects is determined.

5. The electronic device according to claim 1, wherein, The processor is also configured to determine the illuminance of the second region based on the information about light, the information about the type of content, and the information about the brightness of the content.

6. The electronic device according to claim 1, wherein, The processor is also configured as follows: The location of the user interface for controlling the content is determined based on information about the first area and information about the second area. The mirror display is controlled to show the user interface at the determined location.

7. The electronic device according to claim 1, further comprising: memory, The processor is also configured as follows: Identifying whether a user is a pre-registered user is based on user data stored in memory. Based on the user being a pre-registered user, the content is identified based on user data corresponding to the pre-registered user, and The control mirror displays the identified content on the second area.

8. The electronic device according to claim 7, further comprising: Communication interface, Among them, user data is health-related data received from external devices via a communication interface, and The content includes recommended content related to health-related data.

9. The electronic device according to claim 1, wherein, The processor is also configured as follows: The content is processed such that the boundary region between the first region and the second region is blended, and The control mirror displays the processed content on the second area.

10. A method for controlling an electronic device including a mirror display, the method comprising: Based on sensing a user in front of the mirror display, information about a first area on the mirror display is obtained, wherein a first image corresponding to the user's reflection is located in the first area; A second area is determined based on information about the first area, to display the content to be provided to the user. Based on information about light obtained through a second sensor, the illuminance of the second region is determined based on said information about light; and The content is displayed in the second region at the determined illuminance. The step of obtaining information about a first area on the mirror display includes: A second image corresponding to the user is obtained through the camera. Information about the user's pose is obtained by inputting a second image into a trained neural network model. Based on information about the user's posture, the area on the mirror display to display the first image according to the user's movement during a predetermined time period is estimated. The estimated area is designated as the first region, and information about the estimated region is obtained as information about the first region.

11. The method according to claim 10, wherein, Information about the first region includes information about its size and its location, and The step of determining the second region includes: determining the location and size of the second region based on information about the size of the first region and information about the location of the first region.

12. The method according to claim 10, wherein, The information about light includes information about the direction of the light and information about the intensity of the light. The step of determining the second region includes: determining the position and size of the second region based on information about the first region and information about the direction of light, and The step of determining the illuminance of the second region includes: determining the illuminance of the second region based on information about the intensity of light.

13. A non-transitory computer-readable recording medium comprising a program for performing a method of controlling an electronic device, the method comprising: Based on sensing a user in front of the mirror display, information about a first area on the mirror display is obtained, wherein a first image corresponding to the user is displayed in the first area; A second area is determined based on information about the first area, to display the content to be provided to the user. Based on information about light obtained through a second sensor, the illuminance of the second region is determined based on the information about light; and The content is displayed in the second region at the determined illuminance. The step of obtaining information about a first area on the mirror display includes: A second image corresponding to the user is obtained through the camera. Information about the user's pose is obtained by inputting a second image into a trained neural network model. Based on information about the user's posture, the area on the mirror display to display the first image according to the user's movement during a predetermined time period is estimated. The estimated area is designated as the first region, and information about the estimated region is obtained as information about the first region.

Images