Electronic device and control method thereof

Abstract

An electronic device and a control method thereof are provided. The electronic apparatus includes: a communication interface for communicating with an external device; a display; a memory to store at least one instruction; and a processor connected to the communication interface, the display, and the memory. The processor executes at least one instruction such that if a control signal is received from an external device while a UI including a plurality of UI elements is displayed on the display, the processor controls the communication interface such that a request signal for requesting a position measurement signal is transmitted to the external device. If a position measurement signal is received from the external device, the processor obtains information about a position of the external device based on the position measurement signal. The processor identifies, based on the control signal, a first UI element in which the highlighted display is located from among the plurality of UI elements. The processor controls the display such that at least one of a size or a shape of the displayed first UI element is adjusted based on information on a position of the external device.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device and a control method thereof that provide user interface (UI) elements in an optimized size and shape. Background Art

[0002] An electronic device (e.g., a smart television (TV), etc.) provides various UIs. Specifically, the electronic device guides a user to select a UI element by highlighting one of a plurality of UI elements included in the UI.

[0003] However, in the related art, when a user is away from the electronic device, the visibility of the highlighted UI element among the plurality of UI elements included in the UI is low, so the user has to change his or her posture to move closer to the electronic device to view the highlighted UI element, which causes inconvenience to the user. Summary of the Invention

[0004] According to an aspect of the present disclosure, an electronic device includes: a communication interface configured to communicate with an external device; a display; a memory configured to store at least one instruction; and a processor connected to the communication interface, the display, and the memory, wherein the processor is configured to execute at least one instruction to: based on a control signal received by the external device when a UI including a plurality of user interface (UI) elements is displayed on the display, control the communication interface to send a request signal for requesting a position measurement signal to the external device; based on the position measurement signal received from the external device, obtain information about the position of the external device based on the position measurement signal; based on the control signal, identify a first UI element in which highlighting is located from among the plurality of UI elements; and based on the information about the position of the external device, control the display to adjust at least one of a size or a shape of the displayed first UI element.

[0005] The processor may also be configured to execute at least one instruction to: identify a size of text included in the first UI element based on a distance between the external device and the electronic device; and based on the identified size of the text, control the display to adjust a size of the displayed first UI element.

[0006] The processor may also be configured to execute at least one instruction to: identify a size of the text included in the first UI element as a first size based on identifying that the distance between the external device and the electronic device is a first distance; and identify a size of the text included in the first UI element as a second size greater than the first size based on identifying that the distance between the external device and the electronic device is a second distance greater than the first distance.

[0007] The processor may also be configured to execute at least one instruction to: receive eyesight information of a user from an external device through a communication interface; and identify a size of text included in a first UI element based on the user's eyesight information and a distance between the external device and the electronic device.

[0008] The processor may also be configured to execute at least one instruction to: predict eyesight information of a user based on a size of text displayed on a display and a distance between the electronic device and a user of the external device; and identify a size of text included in a first UI element based on the predicted eyesight information of the user and a distance between the external device and the electronic device.

[0009] The processor may also be configured to execute at least one instruction to: control the display to rotate left and display a first UI element based on identifying that the external device is located on the left side of the electronic device; and control the display to rotate right and display the first UI element based on identifying that the external device is located on the right side of the electronic device.

[0010] The processor may also be configured to execute at least one instruction to: control the display to display an image of content on a first layer and display a UI on a second layer based on the UI being displayed on an image of the content; and control the display to adjust at least one of a size or a shape of a first UI element displayed on the second layer based on information about a position of the external device while maintaining the image of the content displayed on the first layer.

[0011] The control signal may be a control signal for moving a highlight displayed on a UI element among a plurality of UI elements.

[0012] The processor may also be configured to execute at least one instruction to: obtain information about a position of the external device in real time based on a position measurement signal; and control the display to adjust at least one of a size or a shape of a first UI element displayed based on the information about the position of the external device.

[0013] According to an aspect of the present disclosure, a control method of an electronic device includes: providing a request signal for requesting a position measurement signal to an external device based on a control signal being received by the external device when a UI including a plurality of user interface (UI) elements is displayed; obtaining information about a position of the external device based on the position measurement signal received from the external device; identifying a first UI element in which a highlight is located from among the plurality of UI elements based on the control signal; adjusting at least one of a size or a shape of the first UI element displayed based on the information about the position of the external device.

[0014] The display may include: identifying a size of text included in a first UI element based on a distance between an external device and the electronic device; and adjusting a size of the displayed first UI element based on the identified size of the text.

[0015] The identification may include: identifying the size of the text included in the first UI element as a first size based on identifying that the distance between the external device and the electronic device is a first distance; and identifying the size of the text included in the first UI element as a second size greater than the first size based on identifying that the distance between the external device and the electronic device is a second distance greater than the first distance.

[0016] The control method may include: receiving vision information of a user from an external device; and wherein, the identification may include: identifying a size of text included in a first UI element based on the user's vision information and a distance between the external device and the electronic device.

[0017] The control method may include: predicting the user's vision information based on a size of text displayed on a display and a distance between the electronic device and a user using the external device; and wherein, the identification may include: identifying a size of text included in a first UI element based on the predicted user's vision information and a distance between the external device and the electronic device.

[0018] The display may include: rotating and displaying the first UI element to the left based on identifying that the external device is located on the left side of the electronic device; and rotating and displaying the first UI element to the right based on identifying that the external device is located on the right side of the electronic device.

[0019] According to an aspect of the present disclosure, an electronic device includes: a memory configured to store at least one instruction; and a processor configured to execute the at least one instruction to: provide a request signal for requesting a position measurement signal to an external device based on a control signal received by the external device when a UI including a plurality of user interface (UI) elements is displayed on a display; obtain information about a position of the external device based on the position measurement signal received from the external device; identify a first UI element in which a highlight is located from among the plurality of UI elements based on the control signal; and control the display to adjust at least one of a size or a shape of the displayed first UI element based on the information about the position of the external device.

[0020] The processor may further be configured to execute at least one instruction to: identify a size of text included in the first UI element based on a distance between the external device and the electronic device; and control the display to adjust a size of the displayed first UI element based on the identified size of the text.

[0021] The processor may also be configured to execute at least one instruction to: identify the size of the text included in the first UI element as a first size based on identifying that the distance between the external device and the electronic device is a first distance; and identify the size of the text included in the first UI element as a second size greater than the first size based on identifying that the distance between the external device and the electronic device is a second distance greater than the first distance.

[0022] The processor may also be configured to execute at least one instruction to: receive the user's vision information from the external device; and identify the size of the text included in the first UI element based on the user's vision information and the distance between the external device and the electronic device.

[0023] The processor may also be configured to execute at least one instruction to: predict the user's vision information based on the size of the text displayed on the display and the distance between the electronic device and the user of the external device; and identify the size of the text included in the first UI element based on the predicted user's vision information and the distance between the external device and the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other aspects, features, and advantages of the embodiments of the present disclosure will become more apparent from the following description in conjunction with the drawings, in which:

[0025] Figure 1 is a view showing a display system including an electronic device and an external device according to one or more embodiments;

[0026] Figure 2 is a block diagram showing the configuration of an electronic device according to one or more embodiments;

[0027] Figure 3 is a block diagram showing the configuration for adjusting a highlighted UI element according to one or more embodiments;

[0028] Figure 4 is a sequence view provided to show a control method of a system for adjusting a highlighted UI element according to the position of an external device according to one or more embodiments;

[0029] Figure 5 is a view provided to show a method for adjusting the size of a highlighted UI element according to the distance between an electronic device and an external device;

[0030] Figure 6a and Figure 6b is a view provided to show a method for adjusting the shape of a highlighted UI element according to the direction in which an external device is located according to one or more embodiments;

[0031] Figure 7aAnd Figure 7b is provided as a view showing a UI including highlighted UI elements according to various embodiments; and

[0032] Figure 8 is provided as a flowchart showing a control method of an electronic device according to one or more embodiments. Detailed Description

[0033] Hereinafter, various embodiments will be described. However, it should be understood that the scope of the present disclosure is not limited to specific embodiments, and includes various modifications, equivalents, or alternatives according to the embodiments of the present disclosure.

[0034] In the present disclosure, expressions such as "having", "may have", "including", "may include", etc. indicate the existence of corresponding features (e.g., numerical values, functions, operations, or components such as parts), and do not exclude the existence of additional features.

[0035] In the present disclosure, expressions such as "A or B", "at least one of A and / or B", "one or more of A and / or B", etc. may include all possible combinations of the items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may indicate all of the following cases: 1) a case including only A, 2) a case including only B, or 3) a case including both A and B.

[0036] Expressions such as "first", "second", etc. used in the present disclosure may indicate various components regardless of the order or importance of the components. These expressions are only used to distinguish one component from other components and do not limit the corresponding components. For example, the first user device and the second user device may represent different user devices regardless of the order or importance. For example, without departing from the scope of the rights described in the present disclosure, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0037] In the present disclosure, terms such as "module", "unit", "part", or "engine" are terms used to refer to components that perform at least one function or operation. Such components may be implemented by hardware or software, or may be implemented by a combination of hardware and software. In addition, multiple "modules", "units", "parts", or "engines" may be integrated with each other in at least one module or chip and implemented by a processor, except when each of them needs to be implemented by specific hardware.

[0038] In any case where a component (e.g., a first component) is referred to as "coupled (operatively or communicatively) to another component (e.g., a second component) / coupled (operatively or communicatively) with another component (e.g., a second component)" or "connected to" another component (e.g., a second component), it should be understood that any component can be directly coupled to another component or can be coupled to another component through yet another component (e.g., a third component). On the other hand, in any case where a component (e.g., a first component) is referred to as "directly coupled" or "directly connected" to another component (e.g., a second component), it should be understood that there is no yet another component (e.g., a third component) between any component and another component.

[0039] The expression "configured (or arranged) to" used in the present disclosure may be replaced by the expressions "suitable for", "capable of", "designed to", "adapted to", "manufactured to", or "able to" based on the circumstances. The expression "configured (or arranged) to" may not necessarily indicate "specially designed for" in hardware. Instead, in any case, the expression that a device "is configured to" may indicate that the device can "perform ~" together with another device or component. For example, "a processor configured (or arranged) to execute A, B, and C" may indicate a dedicated processor (e.g., an embedded processor) for performing the corresponding operations or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can perform the corresponding operations by executing one or more software programs stored in a memory device.

[0040] The terms used in the present disclosure are only for describing embodiments and may not be intended to limit the scope of other embodiments. Unless the context clearly indicates otherwise, singular expressions may include plural expressions. The terms used herein, including technical or scientific terms, may have the same meanings as those generally understood by those skilled in the art. Among the terms used in the present disclosure, the terms defined in a general dictionary may be interpreted as having the same or similar meanings as those in the context of the related art, and unless clearly defined in the present disclosure, they are not ideally or overly interpreted. In some cases, even the terms defined in the present disclosure may not be interpreted as excluding embodiments of the present disclosure.

[0041] Hereinafter, the present disclosure will be described in more detail with reference to the accompanying drawings. However, when describing the present disclosure, if it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted. In combination with the description of the drawings, the same reference numerals may be used for the same components.

[0042] Hereinafter, the present disclosure will be more publicly disclosed in detail with reference to the accompanying drawings.

[0043] Figure 1is a view showing a display system including an electronic device and an external device according to one or more embodiments. As Figure 1 shown, the system 10 may include an electronic device 100 and an external device 200. In this case, the electronic device 100 may be implemented as a smart TV, but this is only an example. The electronic device 100 may be implemented as a device such as a set-top box, a projector, etc. The external device 200 is a device for controlling the electronic device 100, which may be implemented as a remote control. However, this is only an example, and the external device 200 may be implemented as a portable device such as a smart phone, a tablet PC, etc.

[0044] The electronic device 100 may display content. Specifically, the electronic device 100 may display a UI including a plurality of UI elements on the content.

[0045] The external device 200 may receive a user command for controlling the electronic device 100. When a user command is input, the external device 200 may send a control signal corresponding to the user command to the electronic device 100.

[0046] In response to the control signal, the electronic device 100 may send a request signal to the external device 200 requesting a position measurement signal for identifying the current position of the external device 200.

[0047] The external device 200 may send a position measurement signal in response to the request signal, and the electronic device 100 may obtain information about the position of the external device 200 based on the received position measurement signal. In this case, the information about the position of the external device 200 may include information about the distance between the electronic device 100 and the external device 200, information about the direction in which the external device 200 is located relative to the electronic device 100, etc.

[0048] The electronic device 100 may identify the UI element on which the highlight (or focus) is located from among the plurality of UI elements included in the UI based on the signal sent by the external device 200, and may adjust and display at least one of the size or shape of the highlighted UI element based on the information about the position of the external device 200.

[0049] In this way, by providing UI elements with optimal size and shape according to the position of the user manipulating the external device 200, the effect of improving visibility is brought about.

[0050] Figure 2 is a block diagram showing the configuration of an electronic device according to one or more embodiments. As Figure 2 shown, the electronic device 100 may include a display 110, a speaker 120, a communication interface 130, an input / output interface 140, a user input unit 150, a memory 160, and a processor 170.Figure 1 The configuration of the electronic device 100 shown is merely an example, and some components may be added or removed depending on the type of the electronic device 100.

[0051] The display 110 may output various information. Specifically, the display 110 may output content provided from various sources. For example, the display 110 may output broadcast content received from the outside, output game content received through a game server, and output broadcast content or game content received from an external device (e.g., a set-top box or a game console) connected through the input / output interface 140.

[0052] In addition, the display 110 may output various UIs. In this case, the UI may include a plurality of UI elements (or items). Specifically, the UI may be a UI for changing the settings of the electronic device 100 or a UI for selecting another content that the user wishes to view, etc., but is not limited thereto.

[0053] In addition, the display 110 may generate a first layer including an image of the content and generate a second layer including the UI. Thus, when the UI is displayed on the image of the content, the display 110 may output the second layer including the UI on the first layer including the image of the content. Specifically, when the size or shape of the highlighted UI element changes, the display 110 may maintain the image of the content included in the first layer and change the size or shape of the highlighted element in the UI included in the second layer.

[0054] The display 110 may be implemented as a liquid crystal display panel, an organic light emitting diode (OLED), etc., and the display 110 may also be implemented as a flexible display, a transparent display, etc. depending on the situation. However, the display 110 according to one or more embodiments is not limited to a specific type.

[0055] The speaker 120 may output various voice messages and audio. Specifically, the speaker 120 may output the audio of various content. In this case, the speaker 120 may be provided inside the electronic device 100, but this is merely an example. The speaker 120 may be provided outside the electronic device 100 and electrically connected to the electronic device 100.

[0056] The communication interface 130 includes at least one circuit and may perform communication with various types of external devices or servers. The communication interface 130 may include at least one of an IR module, a Bluetooth low energy (BLE) module, a Wi-Fi communication module, an ultra-wideband (UWB) module, a cellular communication module, a third-generation (3G) mobile communication module, a fourth-generation (4G) mobile communication module, a fourth-generation long-term evolution (LTE) communication module, or a fifth-generation (5G) mobile communication module.

[0057] According to one or more embodiments, when the electronic device 100 communicates with the external device 200 using the first communication module (e.g., IR module) of the communication interface 130, the communication interface 130 may receive a control signal from the external device 200. The communication interface 130 may send a request signal for requesting a position measurement signal to the external device 200 and receive a position measurement signal from the external device 200.

[0058] According to other embodiments, when the electronic device 100 communicates with the external device 200 using the second communication module (e.g., BLE module, UWB module, BT module, etc.) of the communication interface 130, the communication interface 130 may receive a control signal from the external device 200. In this case, the processor 170 may obtain information about the position of the external device 200 based on the received control signal.

[0059] The input / output interface 140 is configured to input / output at least one of an audio or image signal. For example, the input / output interface 140 may be a high-definition multimedia interface (HDMI), but this is only an example. The input / output interface 140 may be one of a mobile high-definition link (MHL), a universal serial bus (USB), a display port (DP), Thunderbolt, a video graphics array (VGA) port, an RGB port, a D-subminiature (D-SUB), or a digital visual interface (DVI). Depending on the embodiment, the input / output interface 140 may separately include a port for only inputting or outputting an audio signal and a port for only inputting or outputting an image signal, or may be implemented as a single port for inputting or outputting both an audio signal and an image signal.

[0060] Specifically, the electronic device 100 may receive broadcast content or game content from an external content providing device (e.g., a set-top box, a game console, etc.) through the input / output interface 140.

[0061] The user input unit 150 includes a circuit, and the processor 170 may receive a user command for controlling the operation of the electronic device 100 through the user input unit 150. Specifically, the user input unit 150 may be composed of a touch screen, buttons, a keyboard, a mouse, etc.

[0062] In addition, the user input unit 150 may include a microphone capable of receiving a user's voice. In this case, when the user input unit 150 is implemented as a microphone, the microphone may be disposed inside the electronic device 100.

[0063] The memory 160 may store an operating system for controlling the overall operation of the components of the electronic device 100 and instructions or data related to the components of the electronic device 100. Specifically, the memory 160 may include, for example, Figure 3The signal processing engine 310, position measurement engine 320, display engine 330, and user mobile learning engine 340 shown are configured to provide a UI with an optimal size and shape based on the position of an external device. Specifically, when the electronic device 100 is executed, a module for providing UI elements with an optimal size and shape based on the position of the external device stored in the non-volatile memory may load data for performing various operations onto the volatile memory. Here, loading refers to an operation of calling and storing data stored in the non-volatile memory in the volatile memory so that the processor 170 can access the data.

[0064] In addition, the memory 160 may store the visual acuity information of the user learned by the user mobile learning engine 340.

[0065] The memory 160 may be implemented as a non-volatile memory (e.g., a hard disk, a solid state drive (SSD), a flash memory), a volatile memory (which may also include the memory in the processor 180), and the like.

[0066] The processor 170 may control the electronic device 100 according to at least one instruction stored in the memory 160. Specifically, the processor 170 may be composed of one or more processors. In this case, the one or more processors may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an integrated many-core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of the other components of the electronic device 100 and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in the memory 160. For example, the one or more processors may execute a method according to one or more embodiments by executing one or more instructions stored in the memory 160.

[0067] When a method according to one or more embodiments includes multiple operations, the multiple operations may be executed by one processor or multiple processors. For example, when performing a first operation, a second operation, and a third operation by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be executed by the first processor, or the first operation and the second operation may be executed by the first processor (e.g., a general-purpose processor), and the third operation may be executed by the second processor (e.g., an AI-only processor).

[0068] One or more processors may be implemented as a single-core processor including one core, or may be implemented as at least one multi-core processor including multiple cores (e.g., homogeneous multi-core or heterogeneous multi-core). In the case where one or more processors are implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include an internal memory of the processor, such as a cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multi-core processor. In addition, each (or some) of the multiple cores included in the multi-core processor may independently read and execute program instructions for implementing a method according to one or more embodiments, or all (or some) of the multiple cores may be linked to read and execute program instructions for implementing a method according to one or more embodiments.

[0069] In the case where a method according to one or more embodiments includes multiple operations, the multiple operations may be executed by one of the multiple cores included in the multi-core processor, or may be executed by multiple cores. For example, when a first operation, a second operation, and a third operation are executed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be executed by the first core included in the multi-core processor, or the first operation and the second operation may be executed by the first core included in the multi-core processor and the third operation may be executed by the second core included in the multi-core processor.

[0070] In an embodiment of the present disclosure, the processor 170 may be a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, and here, the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiment is not limited thereto.

[0071] Specifically, when the external device 200 receives a control signal when displaying a UI including multiple UI elements, the processor 170 controls the communication interface 130 to send a request signal for requesting a position measurement signal to the external device. When receiving a position measurement signal from the external device 200, the processor 170 obtains information about the position of the external device based on the position measurement signal. The processor 170 identifies a first UI element in which the highlight is located from among the multiple UI elements based on the control signal. The processor 170 controls the display 110 to adjust at least one of the size or shape of the first UI element based on the information about the position of the external device and display the first UI element.

[0072] Specifically, the processor 170 may identify the size of the text included in the first UI element based on the distance between the external device and the electronic device. In addition, the processor 170 may control the display 110 to adjust the size of the first UI element based on the identified size of the text and display the first UI element.

[0073] When it is determined that the distance between the external device 200 and the electronic device 100 is the first distance, the processor 170 may identify the size of the text included in the first UI element as the first size, and when it is determined that the distance between the external device 200 and the electronic device 100 is a second distance greater than the first distance, the processor 170 may identify the size of the text included in the first UI element as a second size greater than the first size.

[0074] In addition, the processor 170 may receive the vision information of the user using the external device 200 from the external device 200 through the communication interface 130. In addition, the processor 170 may identify the size of the text included in the first UI element based on the vision information of the user and the distance between the external device 200 and the electronic device 100.

[0075] In addition, when the external device 200 is used to control the electronic device 100, the processor 170 may predict the vision information of the user based on the size of the text displayed on the display 110 and the distance between the electronic device 100 and the user. The processor 170 may identify the size of the text included in the first UI element based on the predicted vision information of the user and the distance between the external device 200 and the electronic device 100.

[0076] In addition, when it is determined that the external device 200 is located on the left side of the electronic device 100, the processor 170 may control the display 110 to rotate to the left and display the first UI element. Alternatively, when it is determined that the external device 200 is located on the right side of the electronic device 100, the processor 170 may control the display 110 to rotate to the right and display the first UI element.

[0077] In addition, when a UI is displayed on the image of the content, the processor 170 may control the display 110 to display the image of the content on the first layer and display the UI on the second layer. While maintaining the image of the content displayed on the first layer, the processor 170 may control the display 110 to adjust at least one of the size or shape of the first UI element displayed on the second layer based on the information about the position of the external device 200, and display the first UI element.

[0078] The control signal according to one or more embodiments may be a control signal for moving the highlight displayed on one of the multiple UI elements, but is not limited thereto.

[0079] The processor 170 may obtain information about the position of the external device 200 in real time based on the position measurement signal, and control the display 110 to adjust at least one of the size or shape of the first UI element based on the information about the position of the external device 200 and display the first UI element in real time.

[0080] Figure 3 is a block diagram showing a configuration for adjusting a highlighted UI element according to one or more embodiments. As Figure 3 shown, the components for adjusting the highlighted UI element according to the user's position may include a signal processing engine 310, a position measurement engine 320, a display engine 330, and a user movement learning engine 340.

[0081] The signal processing engine 310 may analyze the signal received from the external device 200. The signal processing engine 310 may identify information about an event corresponding to the control signal based on the control signal received from the external device 200. For example, when a control signal for moving the highlight to the right is received from the external device 200, the signal processing engine 310 may identify information about the event of moving the highlight to the right by processing the control signal. In addition, the signal processing engine 310 may identify the position measurement signal received from the external device 200. When the position measurement signal is identified, the signal processing engine 310 may output the information included in the position measurement signal to the position measurement engine 320.

[0082] The position measurement engine 320 may calculate the position of the external device 200 based on the information included in the position measurement signal. For example, when the external device 200 includes a plurality of IR transmitters, the position measurement engine 320 may obtain information about the distance between the external device 200 and the electronic device 100 based on the time when the transmission request signal is sent and the time when the position measurement signal sent from the plurality of IR transmitters is received. In addition, the position measurement engine 320 may obtain information about the direction of the external device 200 relative to the electronic device 100 based on the time difference between the position measurement signals sent from the plurality of IR transmitters. However, as described above, obtaining information about the position of the external device 200 through a plurality of IR transmitters is only an example, and the information about the position of the external device 200 may be obtained in various ways based on the signal received through the communication module.

[0083] The display engine 330 may process and display the UI included in the display 110 based on the signal received from the external device 200.

[0084] The display engine 330 may identify the UI element in which the highlight is located from among a plurality of UI elements included in the UI based on a control signal. For example, when a control signal for moving the highlight to the right is received, the display engine 330 may identify, as the UI element in which the highlight is located, the UI element that was to the right of the UI element in which the highlight was located before the control signal was received.

[0085] In addition, the display engine 330 may change at least one of the size or shape of the highlighted UI element based on information about the position of the external device 200 obtained by the position measurement engine 320. The display engine 330 may identify the size of the text included in the highlighted UI element based on the distance between the external device 200 and the electronic device 100, adjust the size of the highlighted UI element based on the identified size of the text, and display the highlighted UI element. Specifically, when it is determined that the distance between the external device 200 and the electronic device 100 is a first distance, the display engine 330 may identify the size of the text included in the highlighted UI element as a first size. When it is determined that the distance between the external device 200 and the electronic device 100 is a second distance greater than the first distance, the display engine 330 may identify the size of the text included in the highlighted UI element as a second size greater than the first size. In other words, as the distance between the external device 200 and the electronic device 100 increases, the display engine 330 increases the size of the text included in the UI element, and thus the size of the UI element may also increase.

[0086] In addition, the display engine 330 may identify the size of the text included in the highlighted UI element based on the user's vision information and the distance between the external device 200 and the electronic device 100. In this case, the user's vision information may be sent from the external device 200 or may be predicted by the user movement learning engine 340.

[0087] In addition, when it is determined that the external device 200 is located to the left of the electronic device 100, the display engine 330 may rotate the highlighted UI element to the left and display it, and when it is determined that the external device 200 is located to the right of the electronic device 100, the display engine 330 may rotate the highlighted UI element to the right and display it.

[0088] The user mobile learning engine 340 may predict the user's vision information by learning the user's movement when receiving a control signal from the external device 200. For example, when controlling the electronic device 100 using the external device 200, the user mobile learning engine 340 may measure the distance between the electronic device 100 and the user based on the position measurement signal received from the camera included in the external device 200 or the electronic device 100 when controlling the electronic device 100 using the external device 200. In other words, when controlling the electronic device 100 using the external device 200, the user mobile learning engine 340 may measure whether the user is approaching the electronic device 100. When controlling the electronic device 100 using the external device 200, the user mobile learning engine 340 may predict the user's vision information based on the size of the text displayed on the display 110 and the user's movement (i.e., the distance between the electronic device 100 and the user). For example, when the size of the text displayed on the display is 10 and the distance between the user (external device 200) and the electronic device 100 is 3m, the user mobile learning engine 340 may predict the user's vision as 1.0, and when the size of the text displayed on the display is 10 and the distance between the user (external device 200) and the electronic device 100 is 2m, the user mobile learning engine 340 may predict the user's vision as 0.5.

[0089] Figure 4 is provided as a sequence view showing a control method of a system for adjusting a highlighted UI element according to the position of an external device according to one or more embodiments.

[0090] First, the electronic device 100 may display a UI (410). In this case, the UI may be implemented as various UIs, such as a UI for changing the settings of the electronic device 100, a UI for changing the settings of the currently displayed content, a UI for selecting other content, etc. In addition, the UI may include a plurality of UI elements, and a highlight for the user to select may be located on one of the plurality of UI elements.

[0091] The external device 200 may receive a control command (S420). In this case, the control command may be a control command for manipulating the highlight displayed on the UI, but this is only an example. The control command may be another control command. For example, when the electronic device 100 only displays content without displaying a UI, the control command may be a command for controlling the display of a UI on the content.

[0092] The external device 200 may send a control signal generated according to the control command to the electronic device 100 (S430).

[0093] When receiving a control signal, the electronic device 100 may send a signal requesting a position measurement signal (S440). In response to the request signal, the external device 200 may send a position measurement signal to the electronic device 100 (S450). When the received control signal is not a control signal for changing the current screen (e.g., a mute command, etc.), the electronic device 100 may not send a signal requesting a position measurement signal to the external device 200.

[0094] In addition, when there is a communication module in the communication module of the electronic device 100 that can measure the position through a control signal, the steps of S440 and S450 may be omitted. When the steps of S440 and S450 are omitted, the electronic device 100 may obtain information about the external device 200 based on the control signal.

[0095] The electronic device 100 may obtain information about the position of the external device 200 based on the position measurement signal (S460). In this case, the electronic device 100 may obtain information about the distance between the electronic device 100 and the external device 200 and the direction of the external device 200 relative to the location of the electronic device 100.

[0096] The electronic device 100 may identify a first UI element where the highlight is located from among a plurality of UI elements based on the control signal (S470). In other words, the electronic device 100 may identify a first UI element where the highlight is located from among a plurality of UI elements according to a control command included in the control signal. For example, when a control command for moving the highlight in a first direction is included in the control signal, the electronic device 100 may identify a first UI element located in the first direction of the UI element where the highlight was located before receiving the control signal as the UI element where the highlight is located.

[0097] The electronic device 100 may adjust at least one of the size or shape of the first UI element based on the information about the position of the external device 200 and display the first UI element (S480).

[0098] Reference Figures 5 to 7b A detailed description will be given of this.

[0099] Figure 5 is provided as a view showing a method for adjusting the size of a highlighted UI element according to the distance between an electronic device and an external device.

[0100] As Figure 5As shown at the top of FIG. , electronic device 100 may display UI 510 on the content image. When it is determined based on the position measurement signal that the distance between electronic device 100 and external device 200 is a first distance (D1), electronic device 100 may identify a first size of text corresponding to the first distance (D1), adjust the size of first UI element 520 to include the text of the first size, and display first UI element 520 on UI 510. When it is determined based on the position measurement signal that the distance between electronic device 100 and external device 200 is a second distance (D2) greater than the first distance (D1), electronic device 100 may identify a second size of text corresponding to the second distance (D2) (here, the second size is greater than the first size), adjust the size of first UI element 530 to include the text of the second size, and display first UI element 530 on UI 510. When it is determined based on the position measurement signal that the distance between electronic device 100 and external device 200 is a third distance (D3) greater than the first distance (D1) and the second distance (D2), electronic device 100 may identify a third size of text corresponding to the third distance (D3) (here, the third size is greater than the first size and the second size), adjust the size of first UI element 540 to include the text of the third size, and display first UI element 540 on UI 510.

[0101] In other words, as the distance between electronic device 100 and external device 200 increases, electronic device 100 may increase the size of the text included in the first UI element where the highlighting is located, thereby also increasing the size of the first UI element.

[0102] According to one or more embodiments, electronic device 100 linearly adjusts the size of the first UI element (or the text included in the first UI element) according to the distance between electronic device 100 and external device 200, but this is only an example. The size of the first UI element may be gradually increased according to the distance between electronic device 100 and external device 200. For example, when the distance between electronic device 100 and external device 200 is less than the first distance (D1), electronic device 100 may adjust the size of the first UI element to include text of a predetermined size. When the distance between electronic device 100 and external device 200 is equal to or greater than the first distance (D1) and less than the second distance (D2), electronic device 100 may adjust the size of the first UI element to include the text of the first size. When the distance between electronic device 100 and external device 200 is equal to or greater than the second distance (D2) and less than the third distance (D3), electronic device 100 may adjust the size of the first UI element to include the text of the second size. When the distance between electronic device 100 and external device 200 is equal to or greater than the third distance (D3), electronic device 100 may adjust the size of the first UI element to include the text of the third size.

[0103] In addition, the electronic device 100 may identify the size of the text included in the UI element based not only on the distance between the electronic device 100 and the external device 200 but also on information about the user's eyesight. For example, when the distance between the electronic device 100 and the external device 200 is D1 or the eyesight of the first user is 1.0, the electronic device 100 may adjust the size of the first UI element to include text of a first size. However, when the distance between the electronic device 100 and the external device 200 is D1 or the eyesight of the second user is 0.5, the electronic device 100 may adjust the size of the first UI element to include text of a second size that is larger than the first size. In this case, information about the user's eyesight may be received from the external device 200, but this is merely an example. It may be predicted from the user movement learning engine 340.

[0104] In the above embodiment, the text is included in the UI element, but this is merely an example. The UI element may include symbols, icons, thumbnails, etc. Accordingly, the electronic device 100 may adjust at least one of the symbols, icons, or thumbnails included in the highlighted UI element based on the distance between the electronic device 100 and the external device 200.

[0105] Figure 6a and Figure 6b is a view provided to illustrate a method for adjusting the shape of a highlighted UI element according to the direction in which an external device is located according to one or more embodiments.

[0106] As Figure 6a shown, when it is determined based on the position measurement signal that the external device 200 is located on the left side of the electronic device 100, the electronic device 100 may rotate and display the first UI element 620, in which the highlight is located on the left side within the UI 610, as Figure 6a shown. In other words, the electronic device 100 may adjust and display the shape of the first UI element 620 to provide an image effect as if the first UI element 620 rotates to the left.

[0107] As Figure 6b shown, when it is determined based on the position measurement signal that the external device 200 is located on the right side of the electronic device 100, the electronic device 100 may rotate and display the first UI element 630, in which the highlight is located on the right side within the UI 610, as Figure 6b shown. In other words, the electronic device 100 may adjust and display the shape of the first UI element 630 to provide an image effect as if the first UI element 630 rotates to the right.

[0108] In this way, by providing an image effect as if the first UI element 620 rotates to the left or the first UI element 630 rotates to the right, the user can provide a perspective effect to the first UI elements 620 and 630.

[0109] The electronic device 100 obtains information about the position of the external device 200 in real time based on the position measurement signal received in real time after receiving the control signal, and adjusts at least one of the size or shape of the first UI element based on the information about the position of the external device 200 and displays the first UI element. However, this is only an example, and when the control signal is received, the electronic device 100 may obtain information about the position of the external device 200 based on the position measurement signal after the control command, and adjust at least one of the size or shape of the first UI element based on the information about the position of the external device 200 and display the first UI element. Then, before receiving the next control signal, the electronic device 100 may maintain the size and shape of the first UI element based on the existing information about the position of the external device 200 so that the external device 200 moves.

[0110] As the distance between the electronic device 100 and the external device 200 increases, the size of the first UI element where the highlighting is located may increase. In this way, since the size of the first UI element increases, at least a part of the other UI elements may not be displayed. In this case, the user may feel inconvenient because they cannot see the part of the other UI elements blocked by the first UI element.

[0111] According to one or more embodiments, the electronic device 100 may display the first UI element 720 overlapping with other UI elements included in the UI 710 semi-transparently, as Figure 7a shown. According to other embodiments, the electronic device 100 may change and display at least one of the size or position of at least some of the UI elements 750-1 and 750-2 other than the highlighted first UI element 740 included in the UI 730, as Figure 7b shown.

[0112] Figure 8 is provided as a flowchart showing a control method of an electronic device according to one or more embodiments.

[0113] The electronic device 100 may display a UI including a plurality of UI elements (S810).

[0114] The electronic device 100 may receive a control signal from the external device 200 (S820). In this case, the control signal may be a control signal for moving the highlighting displayed on one of the plurality of UI elements.

[0115] The electronic device 100 may send a request signal for requesting a position measurement signal (S830).

[0116] The electronic device 100 may receive a position measurement signal from the external device 200 (S840).

[0117] The electronic device 100 may obtain information about the position of the external device 200 based on the position measurement signal (S850).

[0118] The electronic device 100 may identify a first UI element in which the highlight is located from among a plurality of UI elements based on a control signal (S860).

[0119] The electronic device 100 may adjust at least one of the size or shape of the first UI element based on the information about the position of the external device 200 and display the first UI element (S870).

[0120] The electronic device 100 may identify the size of the text included in the first UI element in which the highlight is located based on the distance between the external device 200 and the electronic device 100. The electronic device 100 may adjust and display the size of the first UI element based on the identified size of the text. For example, when the distance between the external device 200 and the electronic device 100 is a first distance, the electronic device 100 may identify the size of the text included in the first UI element as a first size. According to another example, when the distance between the external device 200 and the electronic device 100 is a second distance greater than the first distance, the electronic device 100 may identify the size of the text included in the first UI element as a second size greater than the first size.

[0121] In addition, the electronic device 100 may receive vision information of a user using the external device 200 from the external device 200. The electronic device 100 may identify the size of the text included in the first UI element based on the user's vision information and the distance between the external device 200 and the electronic device 100.

[0122] When controlling the electronic device 100 using the external device 200, the electronic device 100 may predict the user's vision information based on the size of the text displayed on the display 110 and the distance between the electronic device and the user. In addition, the electronic device 100 may identify the size of the text included in the first UI element based on the predicted user's vision information and the distance between the external device 200 and the electronic device 100.

[0123] In addition, when it is determined that the external device 200 is located on the left side of the electronic device 100, the electronic device 100 may rotate to the left and display the first UI element. Alternatively, when it is determined that the external device 200 is located on the right side of the electronic device 100, the electronic device 100 may rotate to the right and display the first UI element.

[0124] In addition, when a UI is displayed on an image of content, the image of the content may be displayed on a first layer, and the UI may be displayed on a second layer. In this case, the electronic device 100 may adjust at least one of the size and shape of the first UI element displayed on the second layer based on information about the position of the external device 200 while maintaining the image of the content displayed on the first layer.

[0125] In addition, the electronic device 100 may obtain information about the position of the external device 200 in real time based on a position measurement signal, and adjust at least one of the size or shape of the first UI element based on the information about the position of the external device and display the first UI element.

[0126] According to one or more embodiments, a method according to various embodiments disclosed herein may be provided when included in a computer program product. A computer program product refers to a product and may be traded between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or directly online (e.g., downloaded or uploaded) through an app store (e.g., Play Store TM ), or distributed between two user devices (e.g., smartphones). In the case of online distribution, at least a part of the computer program product (e.g., a downloadable app) may be stored at least temporarily in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an app store, and a relay server, or may be generated temporarily.

[0127] According to one or more embodiments, a method according to various embodiments may be implemented as software including instructions stored in a machine-readable storage medium readable by a machine (e.g., a computer). A machine is a device capable of calling stored instructions from a storage medium and operating according to the called instructions, and may include an electronic device according to the disclosed embodiments.

[0128] The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the "non-transitory storage medium" only means that it is a tangible device and does not include signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in a storage medium and cases where data is stored temporarily. For example, the "non-transitory storage medium" may include a buffer in which data is stored temporarily.

[0129] When an instruction is executed by a processor, the processor may directly execute, or execute using other components under the control of the processor, a function corresponding to the instruction. The instruction may include code generated or executed by a compiler or interpreter.

[0130] In the foregoing, embodiments have been illustrated and described, but the present disclosure is not limited to the above specific embodiments, and those of ordinary skill in the art to which the present disclosure pertains may make various modifications to the present disclosure without departing from the gist of the claims, and these modifications should not be understood separately from the technical spirit or aspects of the present disclosure.

Claims

1. An electronic device, comprising: a communication interface configured to communicate with an external device; a display; a memory configured to store at least one instruction; and a processor connected to the communication interface, the display, and the memory, wherein the processor is configured to execute at least one instruction to: control the communication interface to send a request signal for requesting a position measurement signal to the external device based on a control signal received by the external device when a UI including a plurality of user interface (UI) elements is displayed on the display; obtain information about the position of the external device based on the position measurement signal received from the external device; identify a first UI element in which the highlight is located from the plurality of UI elements based on the control signal; and control the display to adjust at least one of the size or shape of the displayed first UI element based on the information about the position of the external device.

2. The electronic device according to claim 1, wherein the processor is further configured to execute at least one instruction to: identify the size of the text included in the first UI element based on the distance between the external device and the electronic device; and control the display to adjust the size of the displayed first UI element based on the identified size of the text.

3. The electronic device according to claim 2, wherein the processor is further configured to execute at least one instruction to: identify the size of the text included in the first UI element as a first size based on identifying that the distance between the external device and the electronic device is a first distance; and identify the size of the text included in the first UI element as a second size greater than the first size based on identifying that the distance between the external device and the electronic device is a second distance greater than the first distance.

4. The electronic device according to claim 2, wherein the processor is further configured to execute at least one instruction to: receive the user's vision information from the external device through the communication interface; and identify the size of the text included in the first UI element based on the user's vision information and the distance between the external device and the electronic device.

5. The electronic device according to claim 4, wherein the processor is further configured to execute at least one instruction to: predict the user's vision information based on the size of the text displayed on the display and the distance between the electronic device and the user of the external device; and identify the size of the text included in the first UI element based on the predicted user's vision information and the distance between the external device and the electronic device.

6. The electronic device according to claim 1, wherein the processor is further configured to execute at least one instruction to: control the display to rotate to the left and display the first UI element based on identifying that the external device is located on the left side of the electronic device; and control the display to rotate to the right and display the first UI element based on identifying that the external device is located on the right side of the electronic device.

7. The electronic device according to claim 1, wherein the processor is further configured to execute at least one instruction to: control the display to display an image of the content on a first layer and display the UI on a second layer based on the UI being displayed on an image of the content; and While maintaining an image of content displayed on a first layer, based on information about the position of an external device, control a display to adjust at least one of a size or a shape of a first UI element displayed on a second layer.

8. The electronic device according to claim 1, wherein, the control signal is a control signal for moving a highlight displayed on a UI element among a plurality of UI elements.

9. The electronic device according to claim 1, wherein, the processor is further configured to execute at least one instruction to: obtain information about the position of the external device in real time based on a position measurement signal; and based on the information about the position of the external device, control the display to adjust at least one of a size or a shape of the first UI element displayed.

10. A control method of an electronic device, the control method comprising: sending a request signal for requesting a position measurement signal to an external device based on a control signal received by the external device when displaying a UI including a plurality of user interface (UI) elements; obtaining information about the position of the external device based on the position measurement signal based on the position measurement signal received from the external device; identifying a first UI element where a highlight is located from among a plurality of UI elements based on the control signal; and adjusting at least one of a size or a shape of the first UI element displayed based on the information about the position of the external device.

11. The control method according to claim 10, wherein, the display includes: identifying a size of text included in the first UI element based on a distance between the external device and the electronic device; adjusting a size of the first UI element displayed based on the identified size of the text.

12. The control method according to claim 11, wherein, the identification includes: identifying the size of the text included in the first UI element as a first size based on identifying that the distance between the external device and the electronic device is a first distance; and identifying the size of the text included in the first UI element as a second size greater than the first size based on identifying that the distance between the external device and the electronic device is a second distance greater than the first distance.

13. The control method according to claim 11, wherein, the control method includes: receiving vision information of a user from the external device; and wherein the identification includes: identifying the size of the text included in the first UI element based on the vision information of the user and the distance between the external device and the electronic device.

14. The control method according to claim 11, wherein, the control method includes: predicting vision information of a user based on a size of text displayed on a display and a distance between the electronic device and the user using the external device; and wherein the identification includes: identifying the size of the text included in the first UI element based on the predicted vision information of the user and the distance between the external device and the electronic device.

15. The control method according to claim 10, wherein, the display includes: rotating and displaying the first UI element to the left based on identifying that the external device is located on the left side of the electronic device; and rotating and displaying the first UI element to the right based on identifying that the external device is located on the right side of the electronic device.

Images