Electronic device for adaptively providing widget and control method thereof

Abstract

An electronic device is disclosed. The present electronic device may include: a memory in which an application is stored; a display; and one or more processors connected to the memory and the display and controlling the electronic device, wherein the processors: control the display to display information related to a function of the application through a widget; identify a widget corresponding to a user utterance voice on the basis of voice recognition for the user utterance voice when the user utterance voice is acquired; and control the display to change a display state of the identified widget on the basis of the user utterance voice.

Description

Electronic device for adaptively providing widgets and method for controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device that adaptively provides a widget and a method for controlling the same.

[0002] Advances in electronic technology have led to the development of electronic devices offering a variety of functions. In particular, recent developments have included devices that offer a variety of widgets and wallpapers, as well as convenient features based on user voice commands.

[0003] For example, electronic devices such as TVs may provide content such as movies and dramas selected by the user, but they may also provide features such as Screensaver, Ambient, and AoV (Always On Voice) when the user is not actively using the TV.

[0004] However, conventional electronic devices have caused inconvenience to users by blocking certain areas in response to user speech. For example, as illustrated in Fig. 1a, conventional electronic devices provide weather information while blocking existing widgets in response to a user's request for weather information, thereby reducing the aesthetic effect. Alternatively, as illustrated in Fig. 1b, conventional electronic devices simply display the user's speech in a preset area, thereby reducing the usability of the widgets that were previously displayed.

[0005] Additionally, there is a need to provide widgets in a more adaptive manner and provide various ways to use widgets.

[0006] According to one embodiment of the present disclosure for achieving the above purpose, an electronic device includes a memory in which an application is stored, a display, and one or more processors connected to the memory and the display to control the electronic device, wherein the processor controls the display to display information related to a function of the application as a widget, and when a user's spoken voice is acquired, identifies a widget corresponding to the user's spoken voice based on voice recognition of the user's spoken voice, and controls the display to change a display state of the identified widget based on the user's spoken voice.

[0007] In addition, the memory stores a plurality of applications, and the processor controls the display to display information related to functions of the plurality of applications as a plurality of widgets, and when the user's spoken voice is acquired, the processor controls the display to identify a widget corresponding to the user's spoken voice among the plurality of widgets based on voice recognition of the user's spoken voice, enlarge the display size of the identified widget, and dim and display the remaining widgets among the plurality of widgets.

[0008] And, if a widget corresponding to the user's spoken voice is not identified among the plurality of widgets, the processor can control the display to display an additional widget corresponding to the user's spoken voice, enlarge the display size of the additional widget, and dim the plurality of widgets and display them.

[0009] Additionally, the processor can control the display to provide information related to the user's spoken voice within the widget whose display size is enlarged.

[0010] And, it further includes a communication interface, and the memory stores at least one keyword and content corresponding to each of the at least one keyword, and the processor receives a message from a user terminal device through the communication interface, and when it is identified that the message includes a preset keyword, the processor can control the display to display content corresponding to the preset keyword based on information stored in the memory.

[0011] In addition, the processor further includes a sensor, and receives the message and information about the user corresponding to the message from the user terminal device through the communication interface, and when the user corresponding to the message is identified through the sensor, the processor can control the display to display content corresponding to the preset keyword.

[0012] And, if the processor determines that the message does not include the preset keyword, the processor can control the display to identify a widget corresponding to the message and enlarge the display size of the widget corresponding to the message.

[0013] Additionally, the processor further includes a sensor, and when at least one user is identified through the sensor, the processor can identify a widget to be displayed through the display based on the at least one user.

[0014] And, it further includes a communication interface, and the processor obtains channel state information (CSI) through the communication interface, and when an object is identified based on the channel state information, the sensor can be turned on to identify the at least one user.

[0015] Additionally, the sensor is a sound sensor, and the processor can obtain a sound generated by the at least one user through the sound sensor, and identify a widget to be displayed through the display based on at least one of the current time, the obtained sound, or the number of the at least one user.

[0016] And, it further includes a communication interface, and when a communication channel is formed with a user terminal device through the communication interface, the processor can identify a widget to be displayed through the display based on the user terminal device.

[0017] In addition, the processor further includes a communication interface, and when weather information is received from a server through the communication interface, the processor can identify at least one of a widget to be displayed through the display or a display state of the widget to be displayed based on the weather information.

[0018] Meanwhile, according to one embodiment of the present disclosure, a method for controlling an electronic device may include a step of displaying information related to a function of an application as a widget, a step of identifying a widget corresponding to a user's spoken voice based on voice recognition of the user's spoken voice when the user's spoken voice is acquired, and a step of changing a display state of the identified widget based on the user's spoken voice.

[0019] In addition, the displaying step may display information related to functions of a plurality of applications as a plurality of widgets, and the identifying step may identify a widget corresponding to the user's spoken voice among the plurality of widgets based on voice recognition of the user's spoken voice when the user's spoken voice is acquired, and the changing step may enlarge the display size of the identified widget and display the remaining widgets among the plurality of widgets by dimming them.

[0020] And, if a widget corresponding to the user's spoken voice is not identified among the plurality of widgets, the method further includes a step of displaying an additional widget corresponding to the user's spoken voice, and the changing step may enlarge the display size of the additional widget and display the plurality of widgets by dimming them.

[0021] Additionally, the changing step may provide information related to the user's spoken voice within the widget whose display size has been enlarged.

[0022] In addition, the method may further include a step of receiving a message from a user terminal device and, if the message is identified as including a preset keyword, a step of displaying content corresponding to the preset keyword.

[0023] In addition, the receiving step may receive the message and information about a user corresponding to the message from the user terminal device, and the displaying step may display content corresponding to the preset keyword when the user corresponding to the message is identified.

[0024] In addition, if it is determined that the message does not include the preset keyword, the method may further include a step of identifying a widget corresponding to the message and a step of enlarging the display size of the widget corresponding to the message.

[0025] Additionally, if at least one user is identified, the method may further include a step of identifying a widget to be displayed based on the at least one user.

[0026] In addition, the method may further include a step of obtaining channel state information (CSI) and, when an object is identified based on the channel state information, a step of turning on a sensor included in the electronic device to identify at least one user.

[0027] Additionally, the step of identifying the widget to be displayed may include acquiring a sound generated by the at least one user, and identifying the widget to be displayed based on at least one of the current time, the acquired sound, or the number of the at least one user.

[0028] And, when a communication channel is formed with the user terminal device, a step of identifying a widget to be displayed based on the user terminal device may be further included.

[0029] Additionally, when weather information is received from the server, the method may further include a step of identifying at least one of a widget to be displayed or a display state of the widget to be displayed based on the weather information.

[0030] FIG. 1A and FIG. 1B are drawings illustrating a conventional electronic device to help understand the present disclosure.

[0031] FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

[0032] FIG. 3 is a block diagram showing a detailed configuration of an electronic device according to an embodiment of the present disclosure.

[0033] FIG. 4 is a drawing for explaining a screen displaying a widget according to an embodiment of the present disclosure.

[0034] FIG. 5 is a drawing for explaining an operation according to a user's spoken voice according to one embodiment of the present disclosure.

[0035] FIG. 6 is a diagram illustrating a method for identifying a widget to be displayed according to one embodiment of the present disclosure.

[0036] FIGS. 7 to 9 are drawings for explaining operations according to user identification according to one embodiment of the present disclosure.

[0037] FIG. 10 is a diagram illustrating a method for identifying a widget according to a context according to an embodiment of the present disclosure.

[0038] FIG. 11 is a diagram illustrating a method for identifying a widget according to a user terminal device according to an embodiment of the present disclosure.

[0039] FIGS. 12 and 13 are drawings for explaining a method for identifying a widget based on sound according to one embodiment of the present disclosure.

[0040] FIG. 14 is a diagram illustrating a method for identifying a widget to be displayed according to a user pattern according to one embodiment of the present disclosure.

[0041] FIG. 15 is a diagram for explaining an operation according to a message including a preset keyword according to an embodiment of the present disclosure.

[0042] FIG. 16 is a diagram illustrating a method for providing weather information according to an embodiment of the present disclosure.

[0043] FIG. 17 is a drawing for explaining a change in screen configuration according to one embodiment of the present disclosure.

[0044] FIG. 18 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present disclosure.

[0045] The purpose of the present disclosure is to provide an electronic device and a control method thereof for adaptively displaying widgets and providing various utilization methods considering user convenience.

[0046] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0047] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0048] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0049] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0050] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0051] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0052] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0053] Various embodiments of the present disclosure will be described in more detail with reference to the attached drawings below.

[0054] FIG. 2 is a block diagram showing the configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0055] The electronic device (100) may be a device that displays at least one widget. For example, the electronic device (100) may be a device equipped with a display, such as a TV, desktop PC, laptop, smartphone, tablet PC, etc., and may display at least one widget. Here, the widget may be a graphical user interface (GUI) tool that displays a screen that includes at least one function or information provided by an application installed on the device. For example, a widget is a small window that includes at least one function or information provided by an application, and many functions provided in a browser, such as a clock, calendar, notepad, search, map, news, and real-time camera, can be provided through a widget without opening the browser.

[0056] However, the present invention is not limited thereto, and the electronic device (100) may be any device capable of displaying at least one widget. In addition, the electronic device (100) may be a device that does not have a display, such as a set-top box (STB), but provides screen information including at least one widget as a separate display device.

[0057] According to FIG. 2, the electronic device (100) includes a memory (110), a display (120), and a processor (130).

[0058] Memory (110) may refer to hardware that stores information such as data in an electrical or magnetic form so that a processor (130) or the like can access it. To this end, memory (110) may be implemented as at least one piece of hardware from among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.

[0059] The memory (110) may store at least one instruction required for the operation of the electronic device (100) or the processor (130). Here, the instruction is a unit of code that instructs the operation of the electronic device (100) or the processor (130), and may be written in machine language, which is a language that a computer can understand. Alternatively, the memory (110) may store a plurality of instructions for performing a specific task of the electronic device (100) or the processor (130) as an instruction set.

[0060] The memory (110) may store data, which is information in bit or byte units that can represent characters, numbers, images, etc. For example, at least one application may be stored in the memory (110). In addition, the memory (110) may store at least one keyword and content corresponding to each of the at least one keywords. For example, the memory (110) may store the keyword "birthday" and a cake image corresponding to the keyword "birthday."

[0061] The memory (110) is accessed by the processor (130), and reading / writing / modifying / deleting / updating instructions, instruction sets, or data can be performed by the processor (130).

[0062] The display (120) is a configuration that displays an image and can be implemented as a display of various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (120) may also include a driving circuit, a backlight unit, etc. that can be implemented as a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (120) may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

[0063] The processor (130) controls the overall operation of the electronic device (100). Specifically, the processor (130) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100). For example, the processor (130) is connected to components such as a memory (110), a display (120), a communication interface (not shown), and the like and can control the operation of the electronic device (100).

[0064] The one or more processors may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device (100) and perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in the memory (110). For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory (110).

[0065] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0066] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal memory (110) of the processor, such as a cache memory (110) or an on-chip memory (110), and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0067] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0068] In embodiments of the present disclosure, one or more processors may mean a system on a chip (SoC) in which at least one processor and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the electronic device (100) is described below using the expression processor (130).

[0069] The processor (130) can control the display (120) to display information related to the function of the application as a widget. For example, the processor (130) can control the display (120) to display weather information provided by a weather application as a widget.

[0070] When a user's spoken voice is acquired, the processor (130) can identify a widget corresponding to the user's spoken voice based on voice recognition of the user's spoken voice. For example, the electronic device (100) includes a microphone, and the processor (130) can acquire the user's spoken voice through the microphone. Alternatively, the remote control device can acquire the user's spoken voice through a microphone included in the remote control device, and the electronic device (100) can receive the user's spoken voice from the remote control device. When a user's spoken voice, such as "How's the weather today?", is acquired, the processor (130) can identify the user's intention through voice recognition, and identify a weather widget based on the user's spoken voice requesting weather information.

[0071] The processor (130) can control the display (120) to change the display status of the identified widget based on the user's spoken voice. For example, the processor (130) can enlarge the size of the identified widget or focus on its border. Alternatively, when a user's spoken voice, such as "How's the weather today?", is acquired, the processor (130) can identify a weather widget and display information about today's weather within the weather widget.

[0072] The processor (130) controls the display (120) to display information related to functions of multiple applications as multiple widgets, and when a user's spoken voice is acquired, the display (120) may be controlled to identify a widget corresponding to the user's spoken voice among the multiple widgets based on voice recognition of the user's spoken voice, enlarge the display size of the identified widget, and dim and display the remaining widgets among the multiple widgets.

[0073] For example, the processor (130) may display information related to the function of a weather application as a weather widget and display information related to the function of a memo application as a memo widget. When a user utterance, such as "How is the weather today?", is acquired, the processor (130) may identify the weather widget based on voice recognition of the user utterance, enlarge the display size of the weather widget, and control the display (120) to display the memo widget by dimming it.

[0074] However, the present invention is not limited thereto, and the processor (130) may control the display (120) to enlarge the display size of a widget corresponding to the user's spoken voice and reduce the display sizes of the remaining widgets among the plurality of widgets for display. Alternatively, the processor (130) may control the display (120) to display a widget corresponding to the user's spoken voice among the plurality of widgets on the entire screen. Alternatively, the processor (130) may change the display position of the widget corresponding to the user's spoken voice among the plurality of widgets. For example, or, the processor (130) may control the display (120) to move the widget corresponding to the user's spoken voice among the plurality of widgets to the center of the screen and display it. In this case, the processor (130) may change the positions of the remaining widgets among the plurality of widgets based on the moved position of the widget corresponding to the user's spoken voice. Alternatively, the processor (130) may move the widget corresponding to the user's spoken voice to the center of the screen and display it without moving the positions of the remaining widgets. In this case, some or all areas of the remaining widgets may be obscured by the widget corresponding to the user's spoken voice and may not be displayed.

[0075] If a widget corresponding to a user's spoken voice is not identified among a plurality of widgets, the processor (130) may control the display (120) to display an additional widget corresponding to the user's spoken voice, enlarge the display size of the additional widget, and dim and display the plurality of widgets. For example, if a user's spoken voice such as "play music" is acquired while a weather widget and a memo widget are displayed, the processor (130) may identify that a music widget corresponding to the user's spoken voice is not displayed through the display (120) and additionally display the music widget. In addition, the processor (130) may control the display (120) to enlarge the display size of the music widget and dim and display the weather widget and memo widget.

[0076] Alternatively, the processor (130) may display an additional widget and remove the display of at least one of the plurality of widgets. For example, if a music widget is added while a weather widget and a memo widget are displayed, the processor (130) may no longer display at least one of the weather widget and the memo widget.

[0077] The processor (130) can control the display (120) to provide information related to the user's spoken voice within a widget whose display size has been enlarged. For example, the processor (130) can control the display (120) to enlarge the display size of a weather widget and provide today's weather information within the enlarged weather widget.

[0078] The electronic device (100) further includes a communication interface, and the memory (110) stores at least one keyword and content corresponding to each of the at least one keywords, and the processor (130) receives a message from a user terminal device through the communication interface, and when it is identified that the message includes a preset keyword, the processor (130) controls the display (120) to display content corresponding to the preset keyword based on information stored in the memory (110).

[0079] For example, when a message saying “Today is my mother’s birthday” is received from a user terminal device, the processor (130) can identify the preset keyword “birthday” and control the display (120) to display a cake image corresponding to “birthday.”

[0080] The electronic device (100) further includes a sensor, and the processor (130) receives a message and information about a user corresponding to the message from a user terminal device through a communication interface, and when a user corresponding to the message is identified through the sensor, the processor may control the display (120) to display content corresponding to a preset keyword.

[0081] However, the present invention is not limited thereto, and the processor (130) may also obtain information about the user from the message. For example, when a message such as "Today is my mother's birthday" is received, the processor (130) may identify the preset keyword "birthday", identify "mother" as the user, and control the display (120) to display a cake image corresponding to "birthday" when "mother" is identified through the sensor. Alternatively, when information about the user corresponding to the message is not received from the user terminal device, the processor (130) may obtain information about the user from the message or transmit a signal requesting information about the user to the user terminal device.

[0082] If the processor (130) identifies that the message does not include a preset keyword, the processor (130) can control the display (120) to identify a widget corresponding to the message and enlarge the display size of the widget corresponding to the message.

[0083] The electronic device (100) further includes a sensor, and the processor (130) can identify a widget to be displayed through the display (120) based on at least one user when at least one user is identified through the sensor.

[0084] For example, the processor (130) may control the display (120) to display at least one widget frequently used by a user identified through the sensor. Alternatively, the processor (130) may control the display (120) to display at least one widget used by a user identified through the sensor at the last time of identification. Alternatively, the processor (130) may change the widgets to be displayed based on the number of users. For example, if there is only one identified user, the processor (130) may control the display (120) to display at least one widget based on the identified user, and if there are multiple identified users, to display a public widget. Here, the public widget may be a widget that does not include personal information, such as a weather widget.

[0085] The electronic device (100) further includes a communication interface, and the processor (130) obtains channel state information (CSI) through the communication interface, and when an object is identified based on the channel state information, the processor (130) can turn on the sensor to identify at least one user. For example, when at least one of the intensity or phase of the channel state information changes by a preset value or more, the processor (130) can identify that an object exists between the electronic device (100) and a device that has formed a channel with the electronic device (100).

[0086] Typically, the electronic device (100) is connected to an access point and can periodically acquire channel status information. Accordingly, the processor (130) normally keeps the sensor turned off, but when an object is identified based on the channel status information, the processor can turn the sensor on to identify at least one user, thereby reducing the power consumed by the sensor.

[0087] According to one embodiment, the sensor is a sound sensor, and the processor (130) can acquire a sound generated by at least one user through the sound sensor, and identify a widget to be displayed through the display (120) based on at least one of the current time, the acquired sound, or the number of at least one user.

[0088] Alternatively, the electronic device (100) may further include a communication interface, and the processor (130) may identify a widget to be displayed through the display (120) based on the user terminal device when a communication channel is formed with the user terminal device through the communication interface. For example, the processor (130) may identify a widget to be displayed through the display (120) based on the user of the smartphone when a Bluetooth communication channel is formed with the smartphone.

[0089] According to one embodiment, the electronic device (100) further includes a communication interface, and when weather information is received from a server through the communication interface, the processor (130) may identify at least one of a widget to be displayed through the display (120) or a display state of a widget to be displayed based on the weather information.

[0090] Meanwhile, for convenience of explanation, the processor (130) has been described as identifying a widget corresponding to a user's spoken voice, but this is not limited thereto. For example, when a user's spoken voice is acquired, the processor (130) may identify an application based on voice recognition of the acquired user's spoken voice, and control the display (120) to change the display status of a widget corresponding to the identified application.

[0091] FIG. 3 is a block diagram illustrating a detailed configuration of an electronic device (100) according to an embodiment of the present disclosure. The electronic device (100) may include a memory (110), a display (120), and a processor (130). The electronic device (100) may further include a communication interface (140), a sensor (150), a user interface (160), a camera (170), a microphone (180), and a speaker (190). For components illustrated in FIG. 3 that overlap with those illustrated in FIG. 2, a detailed description thereof will be omitted.

[0092] The communication interface (140) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, the electronic device (100) can perform communication with a user terminal device, a server, etc. through the communication interface (140).

[0093] The communication interface (140) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0094] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.

[0095] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0096] Alternatively, the communication interface (140) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0097] In addition, the communication interface (140) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.

[0098] The sensor (150) may be configured to identify a user. For example, the sensor (150) may include at least one of an image sensor or a sound sensor.

[0099] The processor (130) can acquire an image through an image sensor and identify a user from the image. Alternatively, the processor (130) can acquire sound through a sound sensor and identify a user from the sound.

[0100] However, it is not limited to this, and the sensor (150) may have any configuration as long as it is configured to identify the user.

[0101] The user interface (160) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and operation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc., formed on any area of ​​the front, side, or back of the main body of the electronic device (100).

[0102] The camera (170) is configured to capture still images or moving images. The camera (170) can capture still images at a specific point in time, but can also capture still images continuously. The camera (170) can capture images in at least one direction of the electronic device (100).

[0103] The camera (170) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time at which light reflected from a subject enters the camera (170), and the aperture mechanically increases or decreases the size of the opening through which light enters to control the amount of light incident on the lens. When the solid-state image sensor accumulates light reflected from a subject as a photocharge, the image generated by the photocharge is output as an electrical signal. The TG outputs a timing signal for reading out pixel data of the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.

[0104] The microphone (180) is configured to receive sound and convert it into an audio signal. The microphone (180) is electrically connected to the processor (130) and can receive sound under the control of the processor (130).

[0105] For example, the microphone (180) may be formed as an integrated unit integrated into the upper side, front side, side side, etc. of the electronic device (100). Alternatively, the microphone (180) may be provided in a remote control device separate from the electronic device (100). In this case, the remote control device may receive sound through the microphone (180) and provide the received sound to the electronic device (100).

[0106] The microphone (180) may include various configurations such as a microphone that collects sound in analog form, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0107] Meanwhile, the microphone (180) may be implemented in the form of a sound sensor, and any method may be used as long as it has a configuration capable of collecting sound.

[0108] The speaker (190) is a component that outputs various audio data processed by the processor (130) as well as various notification sounds and voice messages.

[0109] As described above, the electronic device (100) adaptively displays widgets based on the user, and can enhance user convenience by highlighting widgets corresponding to the user's spoken voice.

[0110] Hereinafter, the operation of the electronic device (100) will be described in more detail with reference to FIGS. 4 to 17. For convenience of explanation, individual embodiments are described in FIGS. 4 to 17. However, the individual embodiments of FIGS. 4 to 17 may be implemented in any combination.

[0111] FIG. 4 is a drawing for explaining a screen displaying a widget according to an embodiment of the present disclosure.

[0112] The processor (130) can control the display (120) to display information related to the function of an application as a widget. For example, as illustrated in FIG. 4, the processor (130) can control the display (120) to display information related to the functions of a plurality of applications as a plurality of widgets. For example, among the five widgets in FIG. 4, the first widget from the left may include information related to the weather information provision function of a weather application, the second widget from the left may include information related to the memo function of a memo application, the third widget from the left may include information related to the home control function of a smart home application, the fourth widget from the left may include information related to the music playback function of a music application, and the fifth widget from the left may include information related to the call function of a call application.

[0113] For convenience of explanation, five widgets are illustrated in Figure 4. However, this is not a limitation, and the number of displayed widgets may vary. Furthermore, the type, display location, and display size of the displayed widgets may also vary.

[0114] FIG. 5 is a drawing for explaining an operation according to a user's spoken voice according to one embodiment of the present disclosure.

[0115] The processor (130) can control the display (120) to display information related to the functions of multiple applications as multiple widgets. For example, as shown in the first drawing of FIG. 5, the processor (130) can control the display (120) to display information related to the functions of each of a weather application, a memo application, a smart home application, and a music application as a weather widget, a memo widget, a smart home widget, and a music widget.

[0116] When a user's spoken voice is acquired, the processor (130) can control the display (120) to identify a widget corresponding to the user's spoken voice among a plurality of widgets based on voice recognition of the user's spoken voice, enlarge the display size of the identified widget, and dim and display the remaining widgets among the plurality of widgets. For example, as in the second drawing of FIG. 5, when a user's spoken voice (510) such as "Turn off all the lights" is acquired, the processor (130) can identify a smart home widget (520) based on voice recognition of "Turn off all the lights," enlarge the display size of the smart home widget (520), and dim the weather widget (530-1), the memo widget (530-2), and the music widget (530-3).

[0117] The processor (130) can control the display (130) to provide information related to the user's spoken voice within the widget whose display size has been enlarged. In the example described above, the processor (130) can turn off all lights in the home based on the user's spoken voice saying "Turn off all lights" and change the icon (520-1) indicating that the lights of the smart home widget (520) are ON to an icon (520-2) indicating that the lights of the smart home widget (520) are OFF.

[0118] If no additional user speech is received for a preset period of time after changing the display status of a plurality of widgets, the processor (130) may reduce the display size of the widget corresponding to the user speech and release the dimming processing of the remaining widgets among the plurality of widgets.

[0119] FIG. 6 is a diagram illustrating a method for identifying a widget to be displayed according to one embodiment of the present disclosure.

[0120] First, the processor (130) can identify the existence of an object (S610). For example, the processor (130) can obtain channel status information through the communication interface (140) and identify the object based on the channel status information. However, the present invention is not limited thereto, and the processor (130) can also identify the existence of the object through a separate sensor. For example, the processor (130) can identify the existence of the object through an infrared sensor, but any method that can identify the existence of an object that did not exist before is acceptable.

[0121] The processor (130) maintains the current screen if the object is not identified (S623), and can identify the user if the object is identified (S620). For example, if the object is identified, the processor (130) can identify the user through the camera (170), speaker (190), etc.

[0122] The processor (130) can identify a context if the user is not identified (S621), maintain the current screen if the context is not identified (S623), and display a widget corresponding to the context if the context is identified (S622). For example, if the user is not identified but a puppy is identified, the processor (130) can display a widget corresponding to a puppy.

[0123] When a user is identified, the processor (130) can identify whether there are multiple users (S630), and if there are not multiple users, can display a private widget (S631). When there are multiple users, the processor (130) can identify whether there are public widgets (S640), and if there are no public widgets, can display a background screen (S641), and if there are public widgets, can display the public widgets (S650).

[0124] For convenience of explanation, in Fig. 6, only the widgets to be displayed are described, but this is not limited thereto. For example, when displaying a personal widget, the processor (130) may display a personal background screen, and when displaying a public widget, it may display a public background screen.

[0125] FIGS. 7 to 9 are drawings for explaining operations according to user identification according to one embodiment of the present disclosure.

[0126] The processor (130) can obtain information about the user through at least one sensor. For example, as illustrated in FIG. 7, when the user's spoken voice is identified through the microphone (180), the processor (130) can identify the user's presence, gender, age, mood, etc. based on the user's spoken voice. Alternatively, the processor (130) can identify the user's gesture through the camera (170) and identify the user's location, etc. based on the gesture. Alternatively, the processor (130) can identify the location of the user's terminal device through the communication interface and identify the user's presence based on the user's terminal device.

[0127] However, the processor (130) is not limited to the operations illustrated in FIG. 7, and may acquire information about the user and other information in a variety of ways. Furthermore, information about objects other than the user may be acquired in a manner similar to that illustrated in FIG. 7. For example, the processor (130) may acquire the sound of a dog barking.

[0128] The processor (130) can identify information about an object based on information acquired through at least one sensor. For example, as illustrated in the upper part of FIG. 8, the processor (130) can identify a child (Child) corresponding to group A based on information corresponding to group A, and can identify information about a respiratory system (Respiratory) corresponding to group B based on information corresponding to group B. In this case, the processor (130) can provide content related to a child (Kid) based on information about a child (Kid) corresponding to group A, or provide content related to a respiratory rate (Respiratory Rate) based on information about a respiratory system (Respiratory Rate) corresponding to group B, as illustrated in FIG. 9. Alternatively, the processor (130) can display a widget recommending related content.

[0129] FIG. 10 is a diagram illustrating a method for identifying a widget according to a context according to an embodiment of the present disclosure.

[0130] The processor (130) can acquire sound through the microphone (180). For example, as illustrated in FIG. 10, the processor (130) can acquire the sound of a dog barking (1010) rather than a user's spoken voice. In this case, the processor (130) can display a pet care widget (1020). Additionally, the processor (130) can enlarge the size of the pet care widget (1020) and display the remaining widgets by dimming them.

[0131] When multiple types of sounds are acquired, the processor (130) can identify priorities among the multiple types of sounds and identify a widget to be displayed based on a sound with a higher priority.

[0132] For example, when a user's spoken voice and a barking sound of a dog (1010) are acquired, the processor (130) can identify that the user's spoken voice has a higher priority than the barking sound of the dog (1010) and identify a widget to be displayed based on the user's spoken voice.

[0133] FIG. 11 is a diagram illustrating a method for identifying a widget according to a user terminal device according to an embodiment of the present disclosure.

[0134] When a communication channel is formed with a user terminal device through a communication interface (140), the processor (130) can identify a widget to be displayed through the display (120) based on the user terminal device.

[0135] For example, when a communication channel is formed with a user terminal device (1110) through a communication interface (140), the processor (130) may identify a widget to be displayed through the display (120) based on the user terminal device (1110). Alternatively, when a communication channel is formed with a user terminal device (1110) through a communication interface (140), the processor (130) may identify a user of the user terminal device (1110) and identify a widget to be displayed through the display (120) based on the identified user.

[0136] FIGS. 12 and 13 are drawings for explaining a method for identifying a widget based on sound according to one embodiment of the present disclosure.

[0137] The processor (130) may also identify a widget to be displayed based on a sound acquired through the microphone (180).

[0138] For example, if the sound acquired through the microphone (180) is a user's spoken voice, the processor (130) can identify a widget corresponding to the user's spoken voice based on voice recognition of the user's spoken voice.

[0139] Alternatively, the processor (130) may identify a widget corresponding to the type of sound based on the type of sound acquired through the microphone (180). For example, if the processor (130) performs voice recognition on the sound acquired through the microphone (180), but voice recognition is not performed, the processor (130) may identify the type of sound through a neural network model. For example, as illustrated in FIG. 12, if the type of sound identified through the neural network model is a walking sound, the processor (130) may identify a walking widget as the widget to be displayed. Alternatively, as illustrated in FIG. 13, if the type of sound identified through the neural network model is a sound of exercising, the processor (130) may identify a calorie widget as the widget to be displayed. Here, the neural network model may be a model trained to identify the type of sound based on the sound.

[0140] The functions related to artificial intelligence according to the present disclosure can be operated through a processor (130) and a memory (110).

[0141] The processor (130) may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, AP, DSP, etc., a graphics-only processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU.

[0142] One or more processors are controlled to process input data according to predefined operating rules or artificial intelligence models stored in the memory (110). Alternatively, if one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The predefined operating rules or artificial intelligence models are characterized by being created through learning.

[0143] Here, "created through learning" means that a basic artificial intelligence model is learned using a learning algorithm using a plurality of learning data, thereby creating a predefined set of operating rules or an artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0144] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations by calculating the results of previous layers and the multiple weights. The multiple weights of the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated during the learning process to reduce or minimize the loss or cost values ​​obtained by the artificial intelligence model.

[0145] Artificial neural networks may include deep neural networks (DNNs), such as, but not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), or deep Q-networks.

[0146] FIG. 14 is a diagram illustrating a method for identifying a widget to be displayed according to a user pattern according to one embodiment of the present disclosure.

[0147] The processor (130) can learn the user's widget usage pattern as a user pattern and identify a widget to be displayed based on the user pattern.

[0148] For example, the processor (130) can learn a user's widget usage pattern by time zone as a user pattern, display a screen (1410) including a weekly weather widget during a first time period based on the user pattern, and display a screen (1420) including a weather widget, a memo widget, a smart home widget, and a music widget when a second time period immediately after the first time period arrives.

[0149] FIG. 15 is a diagram for explaining an operation according to a message including a preset keyword according to an embodiment of the present disclosure.

[0150] The processor (130) receives a message from a user terminal device through a communication interface (140), and if it is identified that the message includes a preset keyword, it can control the display (120) to display content corresponding to the preset keyword.

[0151] For example, as illustrated in FIG. 15, when a message (1510) such as “Happy Birthday” is received, the processor (130) can control the display (120) to display content corresponding to “Happy Birthday” and “Happy Birthday” as text, since the preset keyword “Happy Birthday” is included in the message.

[0152] The processor (130) may receive a message and information about a user corresponding to the message from a user terminal device through a communication interface (140), and when a user corresponding to the message is identified through a sensor, the processor (130) may control the display (120) to display content corresponding to a preset keyword. In the example described above, the processor (130) may receive information about "Happy Birthday" and a son who is the subject of "Happy Birthday." In this case, when the son is identified, the processor (130) may control the display (120) to display content corresponding to "Happy Birthday" and "Happy Birthday" as text, since the preset keyword "Happy Birthday" is included in the message.

[0153] FIG. 16 is a diagram illustrating a method for providing weather information according to an embodiment of the present disclosure.

[0154] When weather information is received from the server through the communication interface (140), the processor (130) can identify at least one of a widget to be displayed through the display (130) or a display state of a widget to be displayed based on the weather information.

[0155] For example, the processor (130) may receive one of the plurality of weather information pieces shown in FIG. 16, and identify at least one of a widget to be displayed through the display (130) or a display state of a widget to be displayed based on the weather information.

[0156] The processor (130) may request weather information from the server if the user's spoken voice is not identified for a preset period of time or if the user is not identified for a preset period of time. Alternatively, the processor (130) may request weather information from the server at a preset time.

[0157] FIG. 17 is a drawing for explaining a change in screen configuration according to one embodiment of the present disclosure.

[0158] The processor (130) can display an edit button (1710), as shown in the first drawing of FIG. 17.

[0159] When the edit button (1710) is operated, the processor (130) enters the edit mode and can display an icon (1720) indicating the edit mode and an icon for editing.

[0160] The processor (130) can display an edit tab (1730) containing available widgets when one of the plurality of icons for editing (1721) is selected.

[0161] The processor (130) can display an additional widget (1750) when one of the available widgets (1740) is selected.

[0162] FIG. 18 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present disclosure.

[0163] First, information related to the application's functions is displayed as a widget (S1810). Then, when the user's spoken voice is acquired, a widget corresponding to the user's spoken voice is identified based on voice recognition of the user's spoken voice (S1820). Then, the display status of the identified widget is changed based on the user's spoken voice (S1830).

[0164] In addition, the displaying step (S1810) displays information related to functions of multiple applications as multiple widgets, the identifying step (S1820) identifies a widget corresponding to the user's spoken voice among the multiple widgets based on voice recognition of the user's spoken voice when the user's spoken voice is acquired, and the changing step (S1830) enlarges the display size of the identified widget and dims the remaining widgets among the multiple widgets for display.

[0165] And, if a widget corresponding to the user's spoken voice is not identified among the plurality of widgets, a step of displaying an additional widget corresponding to the user's spoken voice is further included, and the changing step (S1830) may enlarge the display size of the additional widget and display the plurality of widgets by dimming them.

[0166] Additionally, the changing step (S1830) can provide information related to the user's spoken voice within a widget whose display size has been enlarged.

[0167] In addition, the method may further include a step of receiving a message from a user terminal device and, if the message is identified as including a preset keyword, a step of displaying content corresponding to the preset keyword.

[0168] In addition, the receiving step may receive a message and information about a user corresponding to the message from a user terminal device, and the displaying step may display content corresponding to a preset keyword when a user corresponding to the message is identified.

[0169] In addition, if it is determined that the message does not include a preset keyword, the method may further include a step of identifying a widget corresponding to the message and a step of enlarging the display size of the widget corresponding to the message.

[0170] Additionally, if at least one user is identified, the method may further include a step of identifying a widget to be displayed based on the at least one user.

[0171] In addition, the method may further include a step of obtaining channel state information (CSI) and, when an object is identified based on the channel state information, a step of turning on a sensor included in the electronic device to identify at least one user.

[0172] Additionally, the step of identifying the widget to be displayed may include acquiring a sound generated by at least one user, and identifying the widget to be displayed based on at least one of the current time, the acquired sound, or the number of at least one user.

[0173] And, when a communication channel is formed with the user terminal device, a step of identifying a widget to be displayed based on the user terminal device may be further included.

[0174] Additionally, when weather information is received from the server, the method may further include a step of identifying at least one of a widget to be displayed or a display state of a widget to be displayed based on the weather information.

[0175] According to various embodiments of the present disclosure as described above, an electronic device can display widgets adaptively based on a user, and can enhance user convenience by highlighting a widget corresponding to a user's spoken voice.

[0176] Meanwhile, computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0177] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0178] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, Memory where applications are stored; display; and comprising one or more processors connected to the memory and the display for controlling the electronic device; The above processor, Control the display to display information related to the function of the above application as a widget, When a user's spoken voice is acquired, a widget corresponding to the user's spoken voice is identified based on voice recognition for the user's spoken voice, An electronic device that controls the display to change the display state of the identified widget based on the user's spoken voice.

2. In paragraph 1, The above memory is, Save multiple applications, The above processor, Controlling the display to display information related to the functions of the above multiple applications as multiple widgets; When the user speech voice is acquired, a widget corresponding to the user speech voice among the plurality of widgets is identified based on speech recognition of the user speech voice, An electronic device that controls the display to enlarge the display size of the identified widget and display the remaining widgets among the plurality of widgets by dimming them.

3. In paragraph 2, The above processor, If a widget corresponding to the user's spoken voice is not identified among the above plurality of widgets, an additional widget corresponding to the user's spoken voice is displayed, An electronic device that controls the display to enlarge the display size of the additional widget and display the plurality of widgets by dimming them.

4. In paragraph 2, The above processor, An electronic device that controls the display to provide information related to the user's spoken voice within the widget whose display size is enlarged.

5. In paragraph 1, further comprising a communication interface; The above memory is, Store at least one keyword and content corresponding to each of said at least one keyword, The above processor, Receives a message from a user terminal device through the above communication interface, An electronic device that controls the display to display content corresponding to the preset keyword based on information stored in the memory when the message is identified as containing a preset keyword.

6. In paragraph 5, including sensors; The above processor, Receive the message and information about the user corresponding to the message from the user terminal device through the communication interface, An electronic device that controls the display to display content corresponding to the preset keyword when a user corresponding to the message is identified through the sensor.

7. In paragraph 5, The above processor, If the above message is identified as not containing the above preset keyword, the widget corresponding to the above message is identified, An electronic device that controls the display to enlarge the display size of a widget corresponding to the above message.

8. In paragraph 1, including sensors; The above processor, An electronic device that identifies a widget to be displayed through the display based on at least one user when at least one user is identified through the sensor.

9. In paragraph 8, further comprising a communication interface; The above processor, Obtain channel state information (CSI) through the above communication interface, An electronic device that identifies at least one user by turning on the sensor when an object is identified based on the channel status information.

10. In paragraph 8, The above sensor, It's a sound sensor, The above processor, Acquire sound generated by at least one user through the above sound sensor, An electronic device that identifies a widget to be displayed through the display based on at least one of the current time, the acquired sound, or the number of the at least one user.

11. In paragraph 1, further comprising a communication interface; The above processor, An electronic device that identifies a widget to be displayed through the display based on the user terminal device when a communication channel is formed with the user terminal device through the communication interface.

12. In paragraph 1, further comprising a communication interface; The above processor, An electronic device that, when weather information is received from a server through the communication interface, identifies at least one of a widget to be displayed through the display or a display state of the widget to be displayed based on the weather information.

13. In a method for controlling an electronic device, A step for displaying information related to the application's functions as a widget; When a user's spoken voice is acquired, a step of identifying a widget corresponding to the user's spoken voice based on voice recognition of the user's spoken voice; and A control method, comprising: a step of changing the display state of the identified widget based on the user's spoken voice.

14. In paragraph 13, The above displaying steps are: Display information related to the functions of multiple applications as multiple widgets, The above identifying step is, When the user speech voice is acquired, a widget corresponding to the user speech voice among the plurality of widgets is identified based on speech recognition of the user speech voice, The above changing steps are: A control method for enlarging the display size of the identified widget and displaying the remaining widgets among the plurality of widgets by dimming them.

15. In paragraph 14, It further includes a step of displaying an additional widget corresponding to the user's spoken voice if a widget corresponding to the user's spoken voice is not identified among the plurality of widgets; The above changing steps are: A control method for enlarging the display size of the above additional widget and displaying the plurality of widgets by dimming them.

Images