Electronic device for executing function, operating method thereof, and recording medium

Abstract

According to one embodiment, an electronic device comprises: a pressure sensor disposed at one side surface portion of a housing of the electronic device; a key input unit disposed at the one side surface portion of the housing of the electronic device and disposed on the pressure sensor; a display; at least one processor; and a memory for storing instructions, wherein the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to: identify at least one piece of content included in a first screen on the basis of identifying a first swipe input in one direction to the key input unit by using the pressure sensor while the first screen is displayed via the display; identify first information about text and / or an image included in the at least one piece of content; identify second information about at least one application of the electronic device related to the at least one piece of content; on the basis of providing the first information and the second information to an artificial intelligence model, identify a plurality of functions related to the at least one piece of content; while displaying, via the display, a plurality of objects corresponding to a designated number of functions among the plurality of functions, display information about a first function related to the at least one piece of content corresponding to a first object selected from among the plurality of objects; and on the basis of identifying a first press input to the key input unit while displaying the information about the first function, execute the first function for the at least one piece of content.

Description

Electronic device performing a function, method of operation thereof, and recording medium

[0001] Embodiments of the present disclosure relate to an electronic device performing a function, a method of operating the same, and a recording medium.

[0002] Thanks to the remarkable advancements in information and communication technology and semiconductor technology, the distribution and use of various electronic devices are increasing rapidly. Electronic devices are being developed to allow users to carry them around and communicate. The term "electronic device" may refer to a device that performs specific functions according to an installed program, such as mobile communication terminals, tablet PCs, wearable electronic devices, video / audio devices, desktop / laptop computers, or vehicle navigation systems.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the above is to be claimed as prior art related to this document, nor can it be used to determine prior art.

[0004] According to one embodiment, an electronic device may include a pressure sensor disposed on one side of a housing of the electronic device, a key input unit disposed on one side of the housing of the electronic device and disposed above the pressure sensor, a display, at least one processor, and a memory for storing instructions.

[0005] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause at least one content included in the first screen to be checked based on confirming a first swipe input in one direction for the key input part using the pressure sensor while displaying the first screen through the display.

[0006] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the electronic device to identify first information regarding text and / or images included in the at least one content.

[0007] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to check second information regarding at least one application of the electronic device related to the at least one content.

[0008] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify a plurality of functions related to the at least one content based on providing the first information and the second information to an artificial intelligence model.

[0009] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to display information about a first function related to the at least one content corresponding to a first object selected among the plurality of objects, while displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display.

[0010] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first function to be executed for the at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

[0011] According to one embodiment, in a method of operating an electronic device, while displaying a first screen through a display of the electronic device, the method may include an operation of checking at least one content included in the first screen based on checking a first swipe input in one direction for a key input part of the electronic device using a pressure sensor of the electronic device.

[0012] According to one embodiment, a method of operating an electronic device may include an operation of checking first information regarding text and / or an image included in at least one content.

[0013] According to one embodiment, a method of operating an electronic device may include an operation of checking second information regarding at least one application of the electronic device related to at least one content.

[0014] According to one embodiment, a method of operating an electronic device may include an operation of identifying a plurality of functions related to at least one content based on providing the first information and the second information to an artificial intelligence model.

[0015] According to one embodiment, a method of operating an electronic device may include an operation of displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display, and displaying information about a first function related to at least one content corresponding to a first object selected among the plurality of objects.

[0016] According to one embodiment, a method of operating an electronic device may include an operation of executing the first function for at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

[0017] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include an operation of checking at least one content included in the first screen based on confirming a first swipe input in one direction to a key input part of the electronic device using a pressure sensor of the electronic device while displaying a first screen through the display of the electronic device. For example, the first swipe input may be an input for determining an area to be scanned on the first screen. For example, the electronic device (201) may scan an area designated by the first swipe input on the first screen.

[0018] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include an operation of verifying first information about text and / or an image included in the at least one content.

[0019] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include an operation of checking second information about at least one application of the electronic device related to the at least one content.

[0020] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include an operation of identifying a plurality of functions related to the at least one content based on providing the first information and the second information to an artificial intelligence model.

[0021] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include displaying information about a first function related to at least one content corresponding to a first object selected among the plurality of objects, while displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display.

[0022] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device, and the at least one operation may include an operation of performing the first function on the at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

[0023] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0024] FIG. 2a is a drawing showing an electronic device according to one embodiment.

[0025] FIG. 2b is a schematic block diagram of an electronic device according to one embodiment.

[0026] FIG. 2c is a block diagram illustrating the operation of an electronic device analyzing a screen according to one embodiment.

[0027] FIG. 3a is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a first function on at least one content included in a first screen.

[0028] FIG. 3b is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a first function on at least one content included in a first screen.

[0029] FIG. 4 is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a second function on at least one content included in a first screen.

[0030] FIG. 5a is a flowchart illustrating the operation of an electronic device checking a plurality of functions based on the priority among contents, according to one embodiment.

[0031] FIG. 5b is a flowchart illustrating the operation of an electronic device displaying a plurality of objects based on the priority among a plurality of functions, according to one embodiment.

[0032] FIG. 6 is a flowchart illustrating the operation of an electronic device, according to one embodiment, displaying a plurality of first objects corresponding to a specified number of functions and other functions among a plurality of functions.

[0033] FIG. 7 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a first function on content included in a first screen.

[0034] FIG. 8 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a second function for content included in a first screen.

[0035] FIG. 9 is a diagram illustrating the operation of an electronic device displaying a plurality of first objects instead of a plurality of objects, according to one embodiment.

[0036] FIG. 10 is a diagram illustrating an operation in which an electronic device, according to one embodiment, displays information about a plurality of objects and a first function for content included in a first screen.

[0037] FIG. 11a is a diagram illustrating the operation of an electronic device checking content according to the direction and distance of movement of a swipe input to a key assembly, according to one embodiment.

[0038] FIG. 11b is a diagram illustrating the operation of an electronic device checking content according to a tap input to a key assembly, according to one embodiment.

[0039] FIG. 12 is a diagram illustrating the operation of an electronic device checking a second screen in response to a swipe input for a first screen, according to one embodiment.

[0040] FIG. 13 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a first function regarding music-related content included in a first screen.

[0041] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0042] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0043] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0044] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0045] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0046] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0047] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0048] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0049] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0050] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0051] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0052] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0053] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0054] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0055] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0056] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0057] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0058] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0059] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0060] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0061] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0062] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0063] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0064] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as a CPU (central processing unit), AP (application processor), or DSP (digital signal processor), graphics-dedicated processors such as a GPU (graphic processing unit) or VPU (vision processing unit), or artificial intelligence-dedicated processors such as an NPU. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0065] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0066] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include a deep neural network (DNN), and examples include, but are not limited to, convolutional neural networks (CNN), deep neural networks (DNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), bidirectional recurrent deep neural networks (BRDNN), or deep Q-networks.

[0067] FIG. 2a is a drawing showing an electronic device according to one embodiment.

[0068] Referring to FIG. 2a, according to one embodiment, a key input portion may be included or disposed on one side of the housing of an electronic device (201) (e.g., the electronic device (101) of FIG. 1). For example, the key input portion may include a key assembly (200) and / or a dome switch (or dome key). The electronic device (201) may check user input for the key input portion. The electronic device (201) may perform a designated function or operation based on user input for the key input portion.

[0069] Meanwhile, the key assembly (200) of this specification is an example of a key input unit (or key input means), but for convenience of explanation, the key input unit (or key input means) will be described below as a key assembly (220). However, the key input unit (or key input means) of the present invention may be replaced with various types of input means other than the key assembly (200).

[0070] According to one embodiment, a pressure sensor (203) (e.g., the pressure sensor (203) of FIG. 2b) and / or an actuator (202) (e.g., the actuator (202) of FIG. 2b) may be disposed on one side of the housing of the electronic device (201). Additionally, a touch sensor (not shown) may be further disposed on one side of the housing of the electronic device (201). According to one embodiment, a key input unit (e.g., a key assembly (200)) may be disposed over the actuator (202) and the pressure sensor (203). Depending on the implementation, the actuator (202) and / or the pressure sensor (203) may be located at a point other than below the key input unit (e.g., the key assembly (200)). According to one embodiment, the electronic device (201) can output vibration through an actuator (202) when user input to a key input unit (e.g., key assembly (200)) is confirmed using a pressure sensor (203). Through this, the vibration output from the actuator (202) can be provided to the user as haptic feedback through the key assembly (200). According to one embodiment, the pressure sensor (203) may be replaced not only with a sensor for detecting pressure but also with other sensors that electrically confirm user input (e.g., touch input) according to various touch methods (e.g., electrostatic method) (e.g., checking how much it is pressed). For example, the pressure sensor (203) is not limited to the above-described embodiment and can be implemented with various sensors.

[0071] According to one embodiment, a pressure sensor (203) and a dome switch may be disposed on one side of the housing of the electronic device (201). According to one embodiment, a key input part (e.g., a key assembly (200)) may be disposed on the pressure sensor (203) and the dome switch.

[0072] FIG. 2b is a schematic block diagram of an electronic device according to one embodiment.

[0073] Referring to FIG. 2b, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 1, the electronic device (201) of FIG. 2a) may scan (or check) a screen displayed through a display (260) based on confirming user input (e.g., swipe input) for a key assembly (200) (e.g., the key assembly (200) of FIG. 2a) using a pressure sensor (203). According to one embodiment, the operation of scanning (or checking) the screen may include the operation of confirming content (e.g., image, text) included in the screen or the operation of acquiring an image corresponding to the screen. According to one embodiment, after scanning (or checking) the screen, the electronic device (201) may acquire information about the content included in the screen. According to one embodiment, the information about the content may include metadata of the text and / or image confirmed based on the operation of scanning (or checking) the screen. According to one embodiment, the information about the content may further include information about an application related to the content. According to one embodiment, the electronic device (201) may acquire a plurality of functions related to the content based on the information about the content.

[0074] According to one embodiment, the electronic device (201) may analyze the screen currently displayed on the display (260) using an artificial intelligence model that has been pre-trained to recognize or analyze the screen itself. The electronic device (201) may obtain information corresponding to the content included in the screen based on the information analyzed using the artificial intelligence model. For example, the electronic device (201) may scan only the content that requires analysis, excluding unnecessary content included in the screen (e.g., advertisements and / or other information) (e.g., performing a filtering operation to exclude unnecessary information) based on analyzing the screen itself using the artificial intelligence model.

[0075] By doing so, the electronic device (201) according to one embodiment can provide information to the user regarding a plurality of functions related to the content by further considering not only information about text and / or images confirmed based on the action of scanning (or confirming) the screen, but also context information not confirmed on the screen (e.g., usage history information related to the application).

[0076] According to one embodiment, the electronic device (201) may include a pressure sensor (203) (e.g., sensor module (176) of FIG. 1), an actuator (202), a key assembly (200) placed on the pressure sensor (203) and the actuator (202), a memory (210) (e.g., memory (130) of FIG. 1), a processor (220) (e.g., processor (120) of FIG. 1), a display (260) (e.g., display module (160) of FIG. 1), and a communication circuit (290) (e.g., communication module (190) of FIG. 1). According to one embodiment, the actuator (202) configuration may be omitted. Depending on the implementation, the actuator (202) may be mounted separately from the key assembly.

[0077] According to one embodiment, the electronic device (201) may include at least one key that uses a touch sensor without a pressure sensor (203). According to one embodiment, the electronic device (201) may also detect user input through a touch sensor without a pressure sensor (203).

[0078] According to one embodiment, information regarding a user gesture (e.g., swipe) input to another electronic device (e.g., wireless controller, smart ring, smartwatch, smart glass, wireless earphone, VST device, XR device, and / or VR device) connected to the electronic device (e.g., connected using wireless communication technology or wired communication technology) can be received, rather than input to at least one key included in the electronic device (201). In this case, the electronic device (201) can use the user gesture input to the other electronic device as a gesture input (e.g., swipe input) to the electronic device (201) (or at least one key of the electronic device (201).

[0079] According to one embodiment, when another electronic device is a smart glass, the user's field of view (FOV) area can be adjusted through a swipe input means built into the smart glass, and a screen capture area can be determined based on the adjusted FOV area. At this time, the other electronic device can provide analysis information about the determined screen and related information (e.g., information about the FOV area, information about the screen capture area) to the electronic device (201).

[0080] Meanwhile, for the sake of convenience of explanation, the following description will focus on input to an electronic device, but various embodiments of the present invention may not be limited thereto. For example, the technical features of the present invention may also be applied to gesture input to another electronic device connected to the electronic device (201).

[0081] According to one embodiment, the electronic device (201) may be implemented in the same or similar manner as the electronic device (201) of FIG. 1.

[0082] According to one embodiment, the processor (220) can control the overall operation of the electronic device (201).

[0083] According to one embodiment, the processor (220) may display a first screen through a display (260). According to one embodiment, the first screen may display a screen containing at least one piece of content. For example, at least one piece of content may include an image and / or text. According to one embodiment, the first screen may display an execution screen of an application of the electronic device (201). For example, the application may include a map application, a message application, a call application, a contact application, a health application, a calendar application, a web browser application, a gallery application, or a content playback application.

[0084] According to one embodiment, the processor (220) can detect a first swipe input in one direction to the key assembly (200) using a pressure sensor (203) while displaying a first screen through the display (260). For example, the processor (220) can detect the first swipe input based on a sensing value obtained using the pressure sensor (203). According to one embodiment, the processor (220) can output a vibration generated through the actuator (202) in accordance with the first swipe input.

[0085] According to one embodiment, the processor (220) can determine the distance between a first position of the key assembly (200) where the first swipe input is started and a second position of the key assembly (200) where the first swipe input is released. According to one embodiment, the processor (220) can determine at least one content corresponding to the distance on a first screen.

[0086] According to one embodiment, the processor (220) may apply visual effects over an area corresponding to at least one identified content included in the first screen. For example, the processor (220) may apply a designated color to an area corresponding to at least one identified content. For example, the processor (220) may apply a designated transparency to an area corresponding to at least one identified content. For example, the processor (220) may apply animation effects to an area corresponding to at least one identified content.

[0087] According to one embodiment, the processor (220) may identify first information regarding text and / or images included in at least one content based on identifying at least one content. For example, the first information may include at least one of the subject or context of the text included in at least one content. For example, the first information may include metadata of an image including at least one of the location, time, or object included in the image (e.g., person, object, background, plant, etc.) where the image included in at least one content was taken.

[0088] According to one embodiment, the processor (220) can check second information regarding at least one application of the electronic device (201) associated with at least one content based on checking at least one content. For example, the second information may include information regarding usage history associated with at least one application of the electronic device (201).

[0089] For example, the first screen may include a conversation screen related to a message application of the electronic device (201). The processor (220) may obtain, as first information, information regarding at least one of the subject or context of the text included in the conversation screen related to the message application and / or information regarding at least one of the location, time, or object included in the image where the image included in the conversation screen related to the message application was taken. The processor (220) may obtain, as second information, information regarding at least one of the history of a plurality of conversations related to the message application or the usage history (e.g., usage time) of the message application.

[0090] For example, the first screen may include a screen related to a content playback application of the electronic device. The processor (220) may obtain as first information information about at least one of the subject (e.g., song title, singer, movie title) or context of text included in the screen and / or information about at least one of the location, time, or object included in the image included in the screen where the image included in the screen was taken. The processor (220) may obtain as second information information about at least one of content similar to the content currently being played using the content playback application, a history of content played through the content playback application, a plot of the content currently being played using the content playback application, or a usage history of the content playback application.

[0091] For example, the first screen may include a preview screen acquired (captured) through a camera application of the processor (220). The processor (220) can use a gallery application to check an image stored in an electronic device (201) that contains an object identical to the object included in the preview screen. The processor (220) can check metadata (e.g., time of capture, location of capture) for an image stored in the electronic device (201) that contains an object identical to the object included in the preview screen as second information.

[0092] For example, the processor (220) can verify information regarding a designated name corresponding to an object (e.g., person) included in a preview screen based on a clustering technique that classifies multiple objects (e.g., person) included in multiple images stored in the electronic device (201). If the processor (220) confirms, through a schedule application, that there is a schedule for the object included in the preview screen and the user of the electronic device (201) to meet each other based on information regarding a designated name corresponding to an object included in the preview screen, the processor (220) can obtain information about the schedule (e.g., date, place, time, number of attendees) as second information.

[0093] According to one embodiment, the processor (220) can identify a plurality of functions related to at least one content based on providing the first information and the second information to an artificial intelligence (AI) model. For example, the artificial intelligence model may be stored in memory (210), or in a separate server (e.g., 108 in FIG. 1) or an external electronic device (e.g., 102, 104 in FIG. 1).

[0094] According to one embodiment, instead of providing the first information and the second information to the artificial intelligence model, the processor (220) may provide the artificial intelligence model with an image including a region corresponding to at least one identified content. According to one embodiment, the processor (220) may identify a plurality of functions related to at least one content based on providing the artificial intelligence model with an image including a region corresponding to at least one identified content.

[0095] According to one embodiment, the processor (220) may determine the priority among at least one content. According to one embodiment, the processor (220) may determine that the content displayed at the top of the first screen among at least one content is the content of high priority. According to one embodiment, the processor (220) may determine a plurality of functions related to at least one content based on providing information about the priority among at least one content and the first information to an artificial intelligence model. According to one embodiment, the processor (220) may determine a plurality of functions related to at least one content based on providing information about the priority among at least one content and an image including an area corresponding to the identified at least one content to an artificial intelligence model.

[0096] For example, a plurality of functions may include functions that can be executed using at least one application of the electronic device (201). For example, functions that can be executed using at least one application may include a function to register a schedule through a schedule application, a function to make a call through a call application, a function to register a contact through a contact application, a function to execute a map application, a function to display the screen of a specific website (e.g., an online shopping mall) through a web browser application, and a function to take an image through a camera application. For example, a plurality of functions may include functions that can be executed using a plurality of applications. For example, functions that can be executed using a plurality of applications may include a function to register a specific schedule using a schedule application and to share the specific schedule with related parties using a message application.

[0097] For example, the plurality of functions may include a function that provides recommendation information related to at least one piece of content. For example, if at least one piece of content includes content related to exercise, the function that provides recommendation information may include a function that provides information about a place suitable for exercise and a function that provides information about a time suitable for exercise. For example, if at least one piece of content includes content related to music, the function that provides recommendation information may include a function that provides information about other songs by the singer of the music and a function that provides information (e.g., song title, singer, lyrics, song release date) about other songs of the same genre (e.g., ballad) as the music.

[0098] According to one embodiment, the processor (220) can identify a specified number of functions among a plurality of functions based on the priority among a plurality of functions. For example, the processor (220) can identify (or determine) a specified number of functions among a plurality of functions in order of highest priority. For example, the specified number may be set by a user or automatically set by the processor (220).

[0099] According to one embodiment, the processor (220) can determine the priority among a plurality of functions. For example, the priority may be determined based on history information regarding how the plurality of functions were previously executed. The processor (220) may determine that the priority of a function is higher the more frequently the function has been previously executed. For example, the priority may be determined based on the preference of the user of the electronic device (201). The user's preference may be determined based on the number of times the application is executed and / or the usage time of the application. The processor (220) may determine that the priority of a function related to the application is higher the more frequently the application is executed and / or the usage time. For example, the priority of a function related to the application may be determined based on the priority assigned to each application of the electronic device (201). However, this is merely an example, and the method for determining the priority may include various methods.

[0100] According to one embodiment, the processor (220) may display a plurality of objects corresponding to a specified number of functions among a plurality of functions. For example, the plurality of objects may be displayed as a specified icon corresponding to a function, a specified widget corresponding to a function, or a specified thumbnail corresponding to a function. For example, the specified number may be set to three. For example, the specified number may be set by a user or automatically set by the processor (220). However, this is an example, and the specified number may not be limited thereto.

[0101] According to one embodiment, the processor (220) may place a function with a higher priority among a plurality of functions at the top of the first screen above other functions with a lower priority (or priority). For example, if the processor (220) determines that the first function among the plurality of functions is the function with the highest priority, it may display the first object at the top of the first screen above the remaining objects among the plurality of objects, excluding the first object. Alternatively, the processor (220) may display the function with a higher priority among the plurality of functions on a “top layer” above other functions with a lower priority. For example, if the processor (220) determines that the first function among the plurality of functions is the function with the highest priority, it may display the first object on a top layer above the remaining objects among the plurality of objects, excluding the first object. According to one embodiment, the processor (220) may display information about the first function related to at least one content corresponding to the first object selected among the plurality of objects around the first object. For example, information regarding the first function may be provided as text and / or an image. According to one embodiment, the processor (220) may not display information regarding the function corresponding to each of the remaining objects, excluding the first object among the plurality of objects.

[0102] According to one embodiment, the processor (220) may use a pressure sensor (201) to detect a first user input to the key assembly (200) while displaying information about the first function. For example, the first user input may include a press input or a tap input to the key assembly (200). Additionally, the first user input may include an input to the dome switch (or dome key) (e.g., a press input or a tap input). Depending on the implementation, the first user input may include a gesture input to the key assembly (200) or the dome switch (e.g., an input via a gesture rather than a direct touch). According to one embodiment, the processor (220) may execute the first function based on detecting the first user input.

[0103] Meanwhile, although the press input in this specification is an example of the first user input, for convenience of explanation, the first user input below will be described as the first press input. However, the first user input of the present invention may be replaced with various inputs other than the press input for the key assembly (200). Additionally, the first user input of the present invention may be replaced with various inputs other than the press input for a dome switch (or dome key) other than the key assembly (200).

[0104] According to one embodiment, the processor (220) can select a second object corresponding to the movement distance and movement direction of the second swipe input among a plurality of objects based on confirming the second swipe input to the key assembly (200) using a pressure sensor (201). According to one embodiment, the movement distance may represent the distance between the position where the second swipe input started and the position where the second swipe input was released. For example, the second object may be located at the bottom of the first screen relative to the first object. If the processor (220) confirms that the movement direction of the second swipe input is from top to bottom and the movement distance is the first distance, the processor (220) can select a second object corresponding to the first distance.

[0105] According to one embodiment, the processor (220) may display information about a second function related to at least one content corresponding to the second object through the display (260) based on selecting a second object. At this time, according to one embodiment, the processor (220) may not display information about a first function related to at least one content corresponding to the first object.

[0106] According to one embodiment, the processor (220) may detect a second user input to the key assembly (200) using a pressure sensor (201) and / or a touch sensor while displaying information about the second function. For example, the second user input may include a press input or a tap input to the key assembly (200). Additionally, the second user input may include an input to the dome switch (or dome key) (e.g., a press input or a tap input). Depending on the implementation, the second user input may include a gesture input to the key assembly (200) or the dome switch (e.g., an input via a gesture rather than a direct touch). According to one embodiment, the processor (220) may execute the second function based on detecting the second user input to the key assembly (200).

[0107] Meanwhile, although the press input in this specification is an example of a second user input, for convenience of explanation, the second user input below will be described as a second press input for the key assembly (220). However, the second user input of the present invention may be replaced with various inputs other than the press input for the key assembly (200). In addition, the second user input of the present invention may be replaced with various inputs other than the press input for a dome switch (or dome key) other than the key assembly (200). For example, the second user input may be replaced with a touch input (or hovering input) by a touch type (e.g., capacitive). For example, the second user input may be replaced with an input for a touch type touch pad that can confirm the touch input.

[0108] According to one embodiment, the processor (220) may display a plurality of first objects corresponding to a specified number (e.g., 3) of functions and other specified number (e.g., 3) of functions through the display (260), based on determining that the movement distance of a third swipe input for the key assembly (200) is greater than a specified first distance. For example, the specified first distance may represent a movement distance of a swipe input for updating and displaying a specified number of functions different from the specified number of functions among the plurality of functions, instead of a plurality of objects corresponding to the specified number of functions displayed on the first screen. For example, the specified first distance may be automatically set by the processor (220) or set by the user.

[0109] According to the implementation, in one embodiment, the processor (220) may display a plurality of first objects corresponding to a specified number (e.g., 3) of functions and other specified number (e.g., 3) of functions through the display (260), based on determining that the movement distance of a third swipe input to the key assembly (200) is greater than a specified first distance and the movement direction of the third swipe input is a specified direction. For example, the specified direction may be set to a direction of movement from bottom to top. For example, the specified direction may be set to a direction of movement from top to bottom. However, this is an example, and the specified direction may not be limited thereto.

[0110] According to one embodiment, after the processor (220) checks at least one content included in the first screen, it may change and display the first screen as a second screen based on the distance and direction of movement of the swipe input through the display (260), based on checking the swipe input for the first screen. According to one embodiment, the processor (220) may display an indicator on the second screen that can execute the function of checking the second screen. According to one embodiment, the processor (220) may check the second screen when a user input (e.g., touch input, tap input, long press input) selecting the indicator is confirmed through the display (260). For example, the processor (220) may check the entire second screen when a user input selecting the indicator is confirmed. For example, the processor (220) may check only the part of the second screen corresponding to the duration based on the duration of the user input selecting the indicator. For example, the processor (220) may obtain user input regarding the part of the second screen to be checked. According to one embodiment, the processor (220) can check third information regarding text and / or images included in at least one first content included in the second screen based on checking the second screen. According to one embodiment, the processor (220) can check fourth information regarding at least one application of an electronic device (201) related to at least one first content based on checking the second screen. According to one embodiment, the processor (220) can check a plurality of functions related to at least one first content based on providing the third information and the fourth information to an artificial intelligence model.

[0111] According to one embodiment, the processor (220) may acquire voice through a microphone (e.g., 150 in FIG. 1) of an electronic device (201). According to one embodiment, the processor (220) may identify a plurality of functions related to at least one content based on providing the acquired voice, first information, and second information to an artificial intelligence model. For example, the first screen may include an execution screen of a web browser application. For example, the voice may indicate “seasonal strawberries.” For example, the plurality of functions may include a link to a website where seasonal strawberries can be purchased, a link to a website where strawberry juice can be purchased, and a function that recommends travel destinations famous for strawberries.

[0112] According to one embodiment, the processor (220) can determine at least one of the location or altitude of the electronic device (201) through a sensor (e.g., 176 in FIG. 1) included in the electronic device (201). According to one embodiment, the processor (220) may determine a plurality of functions related to at least one content based on providing information about at least one of the location (e.g., coordinates) or altitude, first information, and second information to an artificial intelligence model. For example, the first screen may include an execution screen of a map application. Based on the location (e.g., coordinates) and altitude of the electronic device (201), if the processor (220) determines that the location of the electronic device (201) is inside a national park, it may determine a function that recommends attractions around hiking trails and a function that provides information about a website that provides at least one photo taken at a good spot for taking pictures.

[0113] According to one embodiment, the processor (220) can identify a pre-designated external electronic device (e.g., an access point (e.g., a Wi-Fi router)) located near the electronic device (201) through a communication circuit (290) included in the electronic device (201). According to one embodiment, the processor (220) may identify a plurality of functions related to at least one content based on providing information about the pre-designated external electronic device, first information, and second information to an artificial intelligence model. For example, the information about the pre-designated external electronic device may include at least one of location information or identification information of the external electronic device. For example, the first screen may include an execution screen of an application for controlling at least one external electronic device near the electronic device (201). Based on the information about the pre-designated external electronic device, the processor (220) may identify a function for controlling at least one external electronic device (e.g., a refrigerator, a television, an air conditioner) located near the pre-designated external electronic device.

[0114] According to one embodiment, when the electronic device (201) (e.g., a smart ring or wireless earphone) does not include a display (260), data of the electronic device (201) may also be included in the scan target content. For example, the data may include data acquired through a sensor or voice data (e.g., data regarding sound output from a wireless earphone). For example, when the electronic device (201) (e.g., a smart ring or wireless earphone) does not include a display (260), the electronic device (201) may output information about a recommended action in voice by outputting a recommended action (or a recommended action). Alternatively, the electronic device (201) may display information about a recommended action on a display included in another electronic device connected to the electronic device (201). For example, the recommended action (or a recommended action) may include a plurality of functions related to the content.

[0115] FIG. 2c is a block diagram illustrating the operation of an electronic device analyzing a screen according to one embodiment.

[0116] Referring to FIG. 2c, according to one embodiment, the analysis area determination unit (230) and the screen analysis unit (240) can be executed by a processor (220). For example, the analysis area determination unit (230) and the screen analysis unit (240) can be implemented as software or a program that can be executed by the processor (220).

[0117] According to one embodiment, the processor (220) can detect a first swipe input to the key assembly (200) (or key input unit).

[0118] According to one embodiment, the processor (220) can determine an area to perform a scan operation on a first screen displayed on a display (260) based on a first swipe input. For example, the processor (220) can determine the input direction and input distance of the first swipe input for the key assembly (200). For example, the processor (220) can determine the distance from the point where the first swipe input started to the point where it was released on the key assembly (200). The processor (220) can perform a scan operation on the first screen using an analysis area determination unit (230) and a screen analysis unit (240).

[0119] According to one embodiment, the analysis area determination unit (230) may determine an area on the first screen to perform a scan operation based on the distance. The screen analysis unit (240) may perform a screen analysis operation on the determined area. For example, the screen analysis unit (240) may perform a screen analysis operation on the determined area using an artificial intelligence model stored in the AI ​​DB (215) (e.g., an artificial intelligence model pre-trained to analyze the screen). For example, the AI ​​DB (215) may be part of the memory (210). Alternatively, the AI ​​DB (215) may be a database (or server) outside the electronic device (201).

[0120] According to one embodiment, the processor (220) may obtain a screen analysis result for a determined area through the screen analysis unit (240). For example, the screen analysis result may include information about content included in the determined area and functions related to said content. The processor (220) may display a plurality of objects on a first screen through the display (260) using the screen analysis result and related data stored in the memory (210). For example, the plurality of objects may represent functions related to the analyzed content.

[0121] Based on the method described above, the electronic device (201) can analyze the screen currently displayed on the display (260) and recommend functions related to the content included on the screen to the user. Through this, the electronic device (201) can automatically provide recommended functions that the user can use based on the screen analysis request, thereby increasing the usability of the electronic device (201).

[0122] The operations of the electronic device (201) described in the drawings below may be performed by a processor (220). However, for the sake of convenience of explanation, the operations performed by the processor (220) will be described as being performed by the electronic device (201).

[0123] FIG. 3a is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a first function on at least one content included in a first screen.

[0124] Referring to FIG. 3a, according to one embodiment, in operation 311, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen through a display (260) (e.g., the display (260) of FIG. 2b). According to one embodiment, the first screen may display a screen containing at least one piece of content. For example, at least one piece of content may include an image and / or text. According to one embodiment, the first screen may display an execution screen of an application of the electronic device (201). For example, the application may include a map application, a message application, a call application, a contact application, a health application, a calendar application, a web browser application, a gallery application, or a content playback application.

[0125] According to one embodiment, in operation 313, the electronic device (201) can identify at least one content included in the first screen by identifying a first swipe input in one direction to a key input unit (e.g., key assembly (200) in FIG. 2b) using a pressure sensor (203) (e.g., pressure sensor (203) in FIG. 2b) while displaying the first screen. For example, the first swipe input may be an input for determining an area to be scanned on the first screen. For example, the electronic device (201) can scan an area designated by the first swipe input on the first screen. According to one embodiment, the electronic device (201) can identify the distance between a first position of the key assembly (200) where the first swipe input started and a second position of the key assembly (200) where the first swipe input was released. According to one embodiment, the electronic device (201) can identify at least one content included in an area corresponding to said distance on the first screen. According to one embodiment, the operation of checking the screen may include the operation of checking content (e.g., image, text) included in the screen or the operation of obtaining an image corresponding to the screen.

[0126] According to one embodiment, the electronic device (201) may apply visual effects over an area corresponding to at least one identified content included in the first screen. For example, the electronic device (201) may apply a designated color to an area corresponding to at least one identified content. For example, the electronic device (201) may apply a designated transparency to an area corresponding to at least one identified content. For example, the electronic device (201) may apply an animation effect to an area corresponding to at least one identified content. For example, the electronic device (201) may apply a designated blur effect to an area corresponding to at least one identified content. Depending on the implementation, at least two or more of the above-described visual effects may be applied simultaneously.

[0127] According to one embodiment, in operation 315, the electronic device (201) can identify first information regarding text and / or an image included in at least one content. For example, the first information may include at least one of the subject or context of the text included in at least one content. For example, the first information may include metadata of an image including at least one of the location, time, or object included in the image (e.g., person, thing, background, plant, etc.) where the image included in at least one content was taken.

[0128] According to one embodiment, in operation 317, the electronic device (201) can check second information about at least one application associated with at least one content. According to one embodiment, the second information may include information about the usage history of at least one application.

[0129] For example, the first screen may include a conversation screen related to a message application of the electronic device (201). The electronic device (201) may obtain information regarding at least one of a history of multiple conversations related to the message application, or a usage history (e.g., usage time) of the message application as second information.

[0130] For example, the first screen may include a screen related to a content playback application of the electronic device (201). The electronic device (201) may obtain information as second information regarding at least one of content similar to the content currently being played using the content playback application, a history of content played through the content playback application, a plot of the content currently being played using the content playback application, or a usage history of the content playback application. For example, as one of the embodiments of the operation to verify the second information, the electronic device (201) may use an artificial intelligence model to perform content analysis and context analysis, and obtain the second information using analysis information based on the analysis results, content similar to the content currently being played, and / or a history of the content played. The electronic device (201) may analyze various data to obtain the second information and select or determine data related to the content based on the analysis results.

[0131] For example, the first screen may include a preview screen obtained (captured) through a camera application of the electronic device (201). The electronic device (201) can check an image stored in the electronic device (201) containing an object identical to the object included in the preview screen by using a gallery application. The electronic device (201) can check metadata (e.g., shooting time, shooting location) for an image stored in the electronic device (201) containing an object identical to the object included in the preview screen as second information.

[0132] For example, the electronic device (201) can verify information regarding a designated name corresponding to an object (e.g., person) included in a preview screen based on a clustering technology that classifies multiple objects (e.g., person) included in multiple images stored in the electronic device (201). If the electronic device (201) confirms through a schedule application that there is a schedule for the object included in the preview screen and the user of the electronic device (201) to meet each other based on information regarding a designated name corresponding to an object included in the preview screen, the electronic device (201) can obtain information about the schedule (e.g., date, place, time, number of attendees) as second information.

[0133] According to one embodiment, in operation 319, the electronic device (201) can identify a plurality of functions related to at least one content based on providing first information and second information to an artificial intelligence model. For example, the artificial intelligence model may be stored in memory (210) (e.g., memory (210) of FIG. 2b), or in a separate server (e.g., 108 of FIG. 1) or an external electronic device (e.g., 102, 104 of FIG. 1).

[0134] For example, a plurality of functions may include functions that can be executed using at least one application of the electronic device (201). For example, functions that can be executed using at least one application may include a function to register a schedule through a schedule application, a function to make a call through a call application, a function to register a contact through a contact application, a function to execute a map application, a function to display the screen of a specific website (e.g., an online shopping mall) through a web browser application, and a function to take an image through a camera application.

[0135] For example, the plurality of functions may include a function that provides recommendation information related to at least one piece of content. For example, if at least one piece of content includes content related to exercise, the function that provides recommendation information may include a function that provides information about a place suitable for exercise and a function that provides information about a time suitable for exercise. For example, if at least one piece of content includes content related to music, the function that provides recommendation information may include a function that provides information about other songs by the singer of the music and a function that provides information (e.g., song title, singer, lyrics, song release date) about other songs of the same genre (e.g., ballad) as the music.

[0136] According to one embodiment, in operation 321, the electronic device (201) may display information about a first function related to a selected first object among the multiple objects while displaying multiple objects corresponding to a specified number of functions among the multiple functions. For example, the multiple objects may be displayed as a specified icon corresponding to the function, a specified widget corresponding to the function, or a specified thumbnail corresponding to the function. For example, the specified number may be set to three. For example, the specified number may be set by a user or automatically set by a processor (220). However, this is an example, and the specified number may not be limited thereto. For example, information about the first function may be provided as text, video, and / or image.

[0137] According to one embodiment, an electronic device (201) can identify a specified number of functions among a plurality of functions based on the priority among a plurality of functions. For example, the electronic device (201) can identify a specified number of functions with high priority among a plurality of functions. For example, the priority may be determined based on history information regarding how a plurality of functions were previously executed. The electronic device (201) may determine that the priority of a function is higher the more frequently a function has been executed previously. For example, the priority may be determined based on the user's preference of the electronic device (201). The user's preference may be determined based on the number of times an application is executed and / or the usage time of the application. The electronic device (201) may determine that the priority of a function related to an application is higher the more frequently an application is executed and / or the usage time. However, this is merely an example, and the method for determining the priority may include various methods.

[0138] According to one embodiment, the electronic device (201) may display information regarding the first function having the highest priority among a specified number of functions. According to one embodiment, the electronic device (201) may display information regarding the first function having the highest priority among a specified number of functions, while not displaying information regarding the remaining functions excluding the first function among the specified number of functions.

[0139] According to one embodiment, in operation 323, the electronic device (201) can perform a first function for at least one content based on confirming a first user input (e.g., press input) for a key input unit (e.g., key assembly (200)).

[0140] By doing so, the electronic device (201) according to one embodiment can provide information about a plurality of functions related to the content to the user by further considering not only information about text and / or images confirmed based on the action of checking the screen, but also context information not confirmed on the screen (e.g., information about applications related to the content).

[0141] FIG. 3b is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a first function on at least one content included in a first screen.

[0142] Referring to FIG. 3b, according to one embodiment, in operation 331, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen through a display (260) (e.g., the display (260) of FIG. 2b). According to one embodiment, the first screen may display a screen containing at least one piece of content. For example, at least one piece of content may include an image and / or text. According to one embodiment, the first screen may display an execution screen of an application of the electronic device (201). For example, the application may include a map application, a message application, a call application, a contact application, a health application, a calendar application, a web browser application, a gallery application, or a content playback application.

[0143] According to one embodiment, in operation 333, the electronic device (201) can check at least one piece of content included in the first screen based on checking a first swipe input in one direction for a key input unit (e.g., key assembly (200)). For example, the electronic device (201) can check at least one piece of content included in the first screen based on a scanning operation of the first screen. For example, the electronic device (201) can perform a scanning operation of the first screen using an artificial intelligence model, and the artificial intelligence model can distinguish between content related to a function included in the first screen and content not related to said function (e.g., advertisement). The electronic device (201) can check at least one piece of content related to a function on the first screen through the scanning operation described above. According to one embodiment, the electronic device (201) can check the distance between a first position of the key assembly (200) where the first swipe input started and a second position of the key assembly (200) where the first swipe input was released. According to one embodiment, the electronic device (201) can view at least one content corresponding to the distance on a first screen. For example, the electronic device (201) can view at least one content included in an area corresponding to the distance on the first screen.

[0144] According to one embodiment, in operation 335, the electronic device (201) can identify a plurality of functions related to at least one content based on providing a first image corresponding to at least one identified content to an artificial intelligence model. For example, the artificial intelligence model may be stored in memory (210) (e.g., memory (210) of FIG. 2b), or in a separate server (e.g., 108 of FIG. 1) or an external electronic device (e.g., 102, 104 of FIG. 1). Depending on the implementation, operation 333 and operation 335 may be performed as a single operation. For example, the electronic device (201) can use the artificial intelligence model to identify at least one content included in the first screen and identify a plurality of functions related to the identified content.

[0145] According to one embodiment, in operation 337, the electronic device (201) may display information about a first function related to at least one content corresponding to a first object selected among the multiple objects, while displaying multiple objects corresponding to a specified number of functions among the multiple functions.

[0146] According to one embodiment, in operation 339, the electronic device (201) can perform a first function for at least one content based on confirming a first user input (e.g., press input) for a key input unit (e.g., key assembly (200)).

[0147] FIG. 4 is a flowchart illustrating an operation in which an electronic device, according to one embodiment, performs a second function on at least one content included in a first screen.

[0148] Referring to FIG. 4, according to one embodiment, in operation 411, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can detect a second swipe input for a key input unit (e.g., the key assembly (200) of FIG. 2b) using a pressure sensor (203) (e.g., the pressure sensor (203) of FIG. 2b) while displaying information about a plurality of objects and a first function related to the first object among the plurality of objects on a first screen through a display (260) (e.g., the display (260) of FIG. 2b). For example, the second swipe input may be an input for selecting one of the plurality of objects displayed on the first screen. The electronic device (201) can select the second object among the plurality of objects based on detecting the second swipe input while displaying information about a plurality of objects and a first function on the first screen. For example, the second object may be selected based on the movement distance and movement direction of the second swipe input. According to one embodiment, the movement distance may represent the distance between the position where the second swipe input started and the position where the second swipe input was released. For example, the second object may be located at the bottom of the first screen relative to the first object. The electronic device (201) may select the second object among a plurality of objects when the movement direction of the second swipe input is from top to bottom and the movement distance of the second swipe input corresponds to a first distance between the first object and the second object. For example, the second object may be located above the first object. The electronic device (201) may select the second object among a plurality of objects when the movement direction of the second swipe input is from bottom to top and the movement distance of the second swipe input corresponds to a first distance between the first object and the second object.

[0149] According to one embodiment, in operation 413, the electronic device (201) may display information about a second function related to at least one content corresponding to the second object through the display (260) based on selecting a second object. At this time, according to one embodiment, the electronic device (201) may not display information about a first function related to at least one content corresponding to the first object.

[0150] According to one embodiment, in operation 415, the electronic device (201) can perform a second function for at least one content based on confirming a second user input (e.g., press input) for a key input unit (e.g., key assembly (200)).

[0151] FIG. 5a is a flowchart illustrating the operation of an electronic device checking a plurality of functions based on the priority among contents, according to one embodiment.

[0152] Referring to FIG. 5a, according to one embodiment, in operation 511, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can identify a first content and a second content among the content included in a first screen based on a first swipe input. For example, the electronic device (201) can scan an area on the first screen corresponding to the direction of movement and distance of movement of the first swipe input, and identify the first content and the second content included in the scanned area.

[0153] According to one embodiment, the electronic device (201) can check information about text and / or images included in the first content and the second content based on checking the first content and the second content.

[0154] According to one embodiment, the electronic device (201) can check information about at least one application related to the first content and information about at least one application related to the second content based on checking the first content and the second content.

[0155] According to one embodiment, in operation 513, the electronic device (201) may determine the content displayed at the top of the first screen among the first content and the second content as the content of high priority. If there is no content displayed at the top of the first screen among the first content and the second content, the electronic device (201) may determine the content displayed close to the top of the first screen among the first content and the second content as the content of high priority.

[0156] According to one embodiment, in operation 515, the electronic device (201) can identify a plurality of functions based on priority. For example, the electronic device (201) can preferentially identify a plurality of functions related to the content with higher priority among the first content and the second content.

[0157] According to one embodiment, the electronic device (201) can identify a plurality of functions based on providing to an artificial intelligence model information regarding the priority between the first content and the second content, information regarding text and / or images included in the first content and the second content, information regarding at least one application related to the first content, and information regarding at least one application related to the second content. According to one embodiment, the electronic device (201) can also identify a plurality of functions based on providing to an artificial intelligence model information regarding the priority between the first content and the second content, and an image representing an area corresponding to the identified first content and the second content.

[0158] For example, the electronic device (201) can assign a weight greater than 1 to information related to the content with higher priority among the first content and the second content.

[0159] For example, the electronic device (201) can determine that the number of functions related to high-priority content among the multiple functions is greater than the number of functions related to low-priority content.

[0160] FIG. 5b is a flowchart illustrating the operation of an electronic device displaying a plurality of objects based on the priority among a plurality of functions, according to one embodiment.

[0161] Referring to FIG. 5b, according to one embodiment, in operation 531, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can determine the priority among a plurality of functions related to the identified content based on the result of scanning the first screen. For example, the priority may be determined based on history information regarding how the plurality of functions were previously executed. The electronic device (201) may determine that the priority of a function is higher the more times the function has been previously executed. For example, the priority may be determined based on the preference of the user of the electronic device (201). The user's preference may be determined based on the number of times the application is executed and / or the usage time of the application. The electronic device (201) may determine that the priority of a function related to an application is higher the more times the application is executed and / or the usage time. For example, the priority of a function related to an application may be determined based on the priority assigned to each application of the electronic device (201). However, this is just one example, and methods for determining priority may include various methods.

[0162] According to one embodiment, in operation 533, the electronic device (201) can identify a specified number (e.g., 3) of functions among a plurality of functions based on priority. For example, the electronic device (201) can identify (or determine) a specified number of functions among a plurality of functions in order of highest priority. For example, the specified number may be set by a user or automatically set by the electronic device (201).

[0163] According to one embodiment, in operation 535, the electronic device (201) may display a plurality of objects corresponding to a specified number of functions on a first screen. For example, the electronic device (201) may place or display a first object corresponding to a first function having the highest priority among the plurality of objects above the remaining objects. For example, when the specified number is 3, the plurality of objects may be three objects arranged vertically. For example, the plurality of objects may be arranged from top to bottom according to priority. For example, the first object among the plurality of objects may be placed at the very top. Additionally, the electronic device (201) may display information about the first function having the highest priority around the first object.

[0164] FIG. 6 is a flowchart illustrating the operation of an electronic device, according to one embodiment, displaying a plurality of first objects corresponding to a specified number of functions and other functions among a plurality of functions.

[0165] Referring to FIG. 6, according to one embodiment, in operation 611, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can display information about a first function while displaying a plurality of objects corresponding to a specified number (e.g., 3) of functions among a plurality of functions on a first screen.

[0166] According to one embodiment, in operation 613, the electronic device (201) can detect a third swipe input to a key input unit (e.g., the key assembly (200) of FIG. 2b).

[0167] According to one embodiment, in operation 615, the electronic device (201) can determine whether the movement distance of the third swipe input is greater than the specified first distance. For example, the specified first distance may represent the movement distance of the swipe input for updating and displaying a specified number of functions that are different from the functions among the plurality of functions, instead of a plurality of objects corresponding to a specified number of functions displayed on the first screen. For example, the specified first distance may be automatically set by the electronic device (201) or set by the user.

[0168] According to one embodiment, if the electronic device determines that the movement distance of the third swipe input is greater than the specified first distance (operation 615-e), in operation 617, it may display a plurality of first objects corresponding to a specified number of functions and other functions among a plurality of functions.

[0169] FIG. 7 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a first function on content included in a first screen.

[0170] Referring to FIG. 7(a), according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen through a display (260) (e.g., the display (260) of FIG. 2b). For example, the first screen may include a conversation screen related to a message application of the electronic device (201). For example, the first screen may include text and / or images.

[0171] According to one embodiment, the electronic device (201) can detect a swipe input for a key assembly (200) (e.g., the key assembly (200) of FIG. 2b) while displaying a first screen. According to one embodiment, the electronic device (201) can detect a portion of the first screen corresponding to the direction of movement and the distance of movement of the swipe input. According to one embodiment, when the direction of movement of the swipe input is downward relative to the electronic device (201), the electronic device (201) can designate a corresponding area by expanding the area to the lower portion based on the starting point of the swipe input. For example, in the operation of expanding the area, the designated area may be expanded proportionally to the distance of movement of the swipe input to the lower portion, and the expansion of the area may be stopped based on the point where the swipe input is ended or released. For example, an area (e.g., area (700)) may be designated based on the method described above. For example, the electronic device (201) can determine the direction in which the area is expanded based on the direction of movement of the swipe input, and can determine the size of the area based on the distance of movement of the swipe input. According to one embodiment, the electronic device (201) can determine the speed at which the size of the area is expanded or contracted based on the speed of the swipe input. For example, when the swipe input is quickly confirmed, the electronic device (201) can display (or provide) the effect of the area (700) being quickly expanded or contracted according to the corresponding speed.

[0172] According to one embodiment, the electronic device (201) may apply visual effects to an identified area (700) of a first screen. For example, the electronic device (201) may apply a color to the identified area (700) that is different from the color of the remaining area of ​​the first screen excluding the identified area (700). For example, the electronic device (201) may apply a specified transparency to the identified area (700). For example, the electronic device (201) may apply animation effects to the identified area (700).

[0173] Referring to FIG. 7(b), according to one embodiment, an electronic device (201) can identify a portion of a first screen corresponding to the direction of movement and distance of movement of a swipe input, and then identify first information regarding text and / or images included in at least one content included in the identified area (700).

[0174] According to one embodiment, an electronic device (201) may obtain as first information information regarding at least one of the subject or context of text included in a conversation screen related to a message application and / or information regarding at least one of the location, time, or object included in an image taken in a conversation screen related to a message application. For example, the electronic device (201) may determine the subject or context of the text as being related to the appointment time and appointment place. According to one embodiment, the electronic device (201) may obtain the first information through an API that includes information about the screen provided by the electronic device (201). According to one embodiment, the electronic device (201) may transmit the screen information itself as an input value to an artificial intelligence model and obtain the first information by extracting and refining (or filtering) the information through the artificial intelligence model. At this time, unnecessary information may be removed through the artificial intelligence model, and additional information may be obtained depending on the context (or situation). At this time, the electronic device (201) can generate additional information using at least one situation information stored in the electronic device (201) (e.g., past history information or information stored in a database).

[0175] According to one embodiment, the electronic device (201) may obtain information regarding at least one of the history of a plurality of conversations related to a message application, or the usage history of the message application, as second information. For example, the electronic device (201) may obtain information regarding the history of a plurality of conversations that were conversed with a current conversation partner (Lindsey Smith) and / or information regarding the history of a plurality of conversations that were conversed with a plurality of users other than the current conversation partner (Lindsey Smith) as second information. For example, the electronic device (201) may obtain information regarding the usage time of the message application as second information.

[0176] According to one embodiment, the electronic device (201) can identify a plurality of functions related to at least one content included in the identified area (700) based on providing the first information and the second information to an artificial intelligence model.

[0177] According to one embodiment, the electronic device (201) may display a first user interface (UI) (701) for an AI assistant based on providing first information and second information to an artificial intelligence model. For example, the first UI (701) may include information based on information analyzed through the artificial intelligence model (e.g., text (“What are you looking for?”). For example, the electronic device (201) may display the first UI (701) at the top of the screen.

[0178] According to one embodiment, the electronic device (201) can identify (determine) a specified number (e.g., 3) of functions among a plurality of functions based on the priority among a plurality of functions.

[0179] According to one embodiment, the electronic device (201) may display a plurality of objects (711, 720, 730) corresponding to each of a specified number of functions. According to one embodiment, the electronic device (201) may display objects corresponding to functions at the top of a first screen as the priority among the specified number of functions increases. According to one embodiment, the electronic device (201) may display objects corresponding to functions with higher priority by applying different effects to them compared to objects with lower priority. For example, the electronic device (201) may display objects with higher priority larger than other objects, or apply additional visual effects to objects with higher priority. Based on the method described above, the electronic device (201) may display objects with higher priority emphasized compared to other objects.

[0180] According to one embodiment, the electronic device (201) can apply a visual effect specified on a first screen while displaying a plurality of objects (711, 720, 730) corresponding to each of a specified number of functions.

[0181] According to one embodiment, the electronic device (201) may display a first object (711) corresponding to the highest priority first function at the top of the first screen. According to one embodiment, the electronic device (201) may display information (710) regarding the highest priority first function. For example, the first function may include a function to register a schedule using the calendar application of the electronic device (201). For example, the information (710) regarding the first function may include text. For example, the information (710) regarding the first function may further include images and / or videos.

[0182] For example, a function corresponding to object (720) may include a function to display location information (e.g., Seoul Station) for a schedule using a map application. For example, a function corresponding to object (730) may include a function to add another person other than the person currently conversing with as a conversation partner using a contact application. For example, object (740) may represent an object that displays a specified number (e.g., 3) of other functions among the plurality of functions, excluding the functions corresponding to objects (711, 720, 730).

[0183] According to one embodiment, the electronic device (201) may receive information about recommended functions from the user based on user input to the first UI (701). For example, the electronic device (201) may receive information intended by the user (e.g., text and / or voice) through user input to the first UI (701), and may further provide the received information to an artificial intelligence model to determine (or update) a specified number (e.g., 3) of functions related to the content. The electronic device (201) may provide or display information about the determined (or updated) functions as a recommendation action.

[0184] According to one embodiment, the electronic device (201) can execute a first function when a press input to the key assembly (200) is confirmed. For example, the electronic device (201) can execute a first function of executing a schedule application of the electronic device (201) when a press input to the key assembly (200) is confirmed. According to one embodiment, the press input may include touching the key assembly (200) by touch input (e.g., electrostatic method) in addition to physical pressure. For example, when the electronic device (201) confirms a press input to the key assembly (200), it may execute a first function of executing a schedule application of the electronic device (201) to automatically register a schedule (e.g., this Friday, Seoul Station, 6:00 PM, attendee Lindary Smith).

[0185] FIG. 8 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a second function for content included in a first screen.

[0186] Referring to FIG. 8, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can detect a swipe input to a key assembly (200) (e.g., the key assembly (200) of FIG. 2b).

[0187] According to one embodiment, the electronic device (201) can determine the direction of movement and the distance of movement of a swipe input while displaying a plurality of objects (e.g., 711, 720, 730, 740 of FIG. 7). For example, the electronic device (201) can determine the direction of movement and the distance of movement of the swipe input. For example, the direction of movement may be a direction from top to bottom. Based on the direction of movement and the distance of movement, the electronic device (201) can display information (831) about a second function. For example, the second function may be a function related to the object (730) of FIG. 7. At this time, the electronic device (201) may not display information about a first function (e.g., 710 of FIG. 7). For example, the distance of movement of the swipe input may correspond to the distance between the first object (711) and the object (730). Based on the swipe input, an object (730) may be selected instead of the first object (711), and the electronic device (201) may display information (831) about a second function corresponding to the object (730).

[0188] For example, the second function may include a function to add a specific user (e.g., Tom) as a conversation (chat) partner. For example, the specific user may include a user who has frequently made calls through the call application of the electronic device (201), a user who has frequently made calls through the messenger application, or a user of the electronic device (201) who has been designated in advance.

[0189] FIG. 9 is a diagram illustrating the operation of an electronic device displaying a plurality of first objects instead of a plurality of objects, according to one embodiment.

[0190] Referring to FIG. 9, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can detect a third swipe input to a key assembly (200) (e.g., the key assembly (200) of FIG. 2b) while displaying a first screen.

[0191] According to one embodiment, the electronic device (201) can detect a third swipe input with a movement distance greater than a specified first distance while displaying a plurality of objects (e.g., 711, 720, 730 of FIG. 7) corresponding to a plurality of functions related to content on a first screen. For example, the electronic device (201) can display a plurality of first objects (911, 920, 930) that are different from the plurality of objects based on detecting that the movement distance of the third swipe input is greater than the specified first distance. For example, the plurality of first objects (911, 920, 930) can each display functions different from the functions represented by the plurality of objects among the functions related to content. For example, the plurality of first objects (911, 920, 930) may be a specified number and may be arranged according to a specified rule (e.g., if arranged vertically, the higher the priority, the higher the position). That is, the electronic device (201) can change a plurality of objects into a plurality of first objects through the display (260) based on confirming that the movement distance of the third swipe input is greater than the designated first distance, and display information about a function corresponding to a selected object among the plurality of first objects. For example, the selected object among the plurality of first objects may be an object corresponding to the highest priority function among the plurality of first objects. For example, the designated first distance may be a distance for displaying objects corresponding to other functions instead of the plurality of objects displayed on the first screen. For example, if a swipe input with a movement distance exceeding the designated first distance is confirmed, the electronic device (201) may change a plurality of objects (711, 720, 730) related to content into a plurality of first objects (911, 920, 930).

[0192] According to one embodiment, the electronic device (201) may display information (910) regarding a function corresponding to the object (911) with the highest priority among a plurality of first objects. For example, the function corresponding to the object (911) may include a function that displays location information (e.g., Seoul Station) for a schedule using a map application. For example, the function corresponding to the object (920) may represent a favorites function that displays the corresponding conversation screen on the home screen of the electronic device (201). For example, the function corresponding to the object (930) may represent a sharing function that allows the first screen to be shared with other users.

[0193] According to one embodiment, the electronic device (201) may receive information about recommended functions from the user based on user input to the first UI (701). For example, the electronic device (201) may receive information intended by the user (e.g., text and / or voice) through user input to the first UI (701), and may further provide the received information to an artificial intelligence model to determine (or update) a specified number (e.g., 3) of functions related to the content. The electronic device (201) may provide or display information about the determined (or updated) functions as a recommendation action. For example, the electronic device (201) may update information (910) about a function corresponding to the highest priority object (911), and provide or display the updated information.

[0194] FIG. 10 is a diagram illustrating an operation in which an electronic device, according to one embodiment, displays information about a plurality of objects and a first function for content included in a first screen.

[0195] Referring to FIG. 10 (a), according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display information (1010) regarding a first function at the bottom of a first screen based on the selection of a first object (1011). For example, the first function may be a function related to content obtained by analyzing the context of the scanned first screen. For example, the electronic device (201) may identify and analyze information such as information about a specific area (e.g., Seoul Station) and information about an appointment (e.g., 6 PM) among the conversation content included in the first screen, and display a related function (e.g., a map application) as the first function.

[0196] According to one embodiment, the electronic device (201) may display an indicator (1011) indicating that it is displaying information about a first function among a number of functions specified in the right area (1000) of the first screen. For example, the electronic device (201) may display an indicator (1011) corresponding to the first function among the indicators (1011, 1012, 1013) by highlighting it (e.g., resizing and / or applying visual effects). According to one embodiment, the electronic device (201) may display remaining indicators (1012, 1013) indicating that it is displaying information about each of the remaining functions excluding the first function among the number of functions specified. For example, the remaining indicators (1012, 1013) may be displayed smaller than the indicator (1011) corresponding to the first function (or with other visual effects applied).

[0197] According to one embodiment, the electronic device (201) may display information (1010) for the first function on top of information (1020, 1030) for each of the remaining functions excluding the first function among a specified number of functions. However, the electronic device (201) may not display information (1020, 1030) for the other functions in detail. For example, the electronic device (201) may display information (1020, 1030) for the remaining functions briefly so as to indicate that the information (1020, 1030) for the other functions is superimposed below the information (1010) for the first function. According to one embodiment, the electronic device (201) may apply a superimposed effect with respect to the information (1020, 1030) for the remaining functions excluding the first function among a specified number of functions, with respect to the information (1010) for the first function.

[0198] Referring to FIG. 10(b), according to one embodiment, an electronic device (201) can display information (1030) for a second function corresponding to the direction and distance of movement of a swipe input at the bottom of a first screen based on confirming a swipe input for a key assembly (200) (e.g., the key assembly (200) of FIG. 2b). For example, the electronic device (201) can display information (1030) for a messenger application corresponding to the second function based on the swipe input.

[0199] According to one embodiment, the electronic device (201) may display indicators (1012, 1014, 1015) indicating that information regarding each of the remaining functions, excluding the second function, among a specified number of functions can be displayed.

[0200] According to one embodiment, the electronic device (201) may display information (1030) regarding the second function on top of information regarding each of the remaining functions excluding the second function among a specified number of functions. According to one embodiment, the electronic device (201) may not display information regarding each of the remaining functions excluding the second function among a specified number of functions. According to one embodiment, the electronic device (201) may apply an effect (1040) indicating that information regarding each of the remaining functions excluding the second function among a specified number of functions can be displayed.

[0201] FIG. 11a is a diagram illustrating the operation of an electronic device checking content according to the direction and distance of movement of a swipe input to a key assembly, according to one embodiment.

[0202] Referring to (a) and (b) of FIG. 11a, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen containing at least one piece of content. For example, at least one piece of content may include an image and / or text. For example, at least one piece of content may include a first piece of content (e.g., content 1), a second piece of content (e.g., content 2), a third piece of content (e.g., content 3), a fourth piece of content (e.g., content 4), a fifth piece of content (e.g., content 5), and a sixth piece of content (e.g., content 6).

[0203] According to one embodiment, an electronic device (201) can identify, select, or determine a plurality of contents for a recommended action based on a swipe input to a key assembly (200) (e.g., the key assembly (200) of FIG. 2b). To do this, the electronic device (201) can identify an area corresponding to the swipe input and identify a plurality of contents corresponding to the area. For example, the recommended action may include a plurality of functions related to the contents.

[0204] According to one embodiment, the electronic device (201) can detect a swipe input for a key assembly (200) (e.g., the key assembly (200) of FIG. 2b). According to one embodiment, the electronic device (201) can detect the direction of movement and the distance of movement of the swipe input. Based on the direction of movement and the distance of movement of the swipe input, the electronic device (201) can detect an area to be scanned on a first screen.

[0205] Referring to (a) of FIG. 11a, according to one embodiment, if the movement direction of the swipe input is confirmed to be from top to bottom, the electronic device (201) can determine an area (1110) extending from the top of the first screen to a portion that is a first distance (1115) corresponding to the movement distance of the swipe input. For example, the electronic device (201) can determine the distance between the first position and the second position based on the first position of the key assembly (or key input part) (200) where the swipe input started and the second position of the key input part where the swipe input was released. The electronic device (201) can determine the first distance (1115) based on the distance between the first position and the second position. For example, the first distance (1115) may be proportional to the distance between the first position and the second position of the key assembly (220). For example, the electronic device (201) can identify an area (1110) including a first content (e.g., content 1), a second content (e.g., content 2), and a third content (e.g., content 3).

[0206] According to one embodiment, when another electronic device that has formed communication with the electronic device (201) is implemented as a smart glass, the smart glass can detect a swipe input through a swipe input means (e.g., an input device capable of detecting a swipe input) built into the smart glass. The smart glass can adjust the area (1110) based on the swipe input to the user's field of view (FOV). Additionally, the smart glass can determine a screen capture area (e.g., area (1110)) based on the swipe input and the adjusted FOV area. At this time, the smart glass can provide the electronic device (201) with analysis information regarding the screen capture area (e.g., area (1110)) and information related to the first screen (e.g., information regarding the FOV area, information regarding the screen capture area). For example, the electronic device (201) can perform a scanning operation on the area (1110) identified by another electronic device. Additionally, the electronic device (201) can consider information obtained from another electronic device (e.g., smart glasses) when analyzing the area (1110).

[0207] According to one embodiment, the electronic device (201) may apply visual effects to an area (1110) so that an area to be scanned is distinguished from a first screen. For example, the electronic device (201) may apply visual effects to an area (1110) that includes a first content (e.g., content 1), a second content (e.g., content 2), and a third content (e.g., content 3). For example, the electronic device (201) may apply a specified color or a specified transparency to the area (1110).

[0208] Referring to FIG. 11a(b), according to one embodiment, if the electronic device (201) determines that the direction of movement of the swipe input for the key assembly (200) is from bottom to top, it can determine an area (1120) from the bottom of the first screen to a portion that is a second distance (1125) corresponding to the distance of movement of the swipe input. For example, the electronic device (201) can determine the distance between the first position and the second position based on the first position of the key assembly (or key input part) (200) where the swipe input started and the second position of the key input part where the swipe input was released. The electronic device (201) can determine the second distance (1125) based on the distance between the first position and the second position. For example, the second distance (1125) may be proportional to the distance between the first position and the second position of the key assembly (220). For example, the electronic device (201) can identify an area (1120) including a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6).

[0209] According to one embodiment, the electronic device (201) may apply a visual effect to an area (1120) so that the area to be scanned is distinguished from the first screen. For example, the electronic device (201) may apply a visual effect to an area (1120) that includes a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6). For example, the electronic device (201) may apply a specified color or a specified transparency to the area (1120). For example, the electronic device (201) may display an indicator (e.g., a visual effect) that is overlaid on the third content (e.g., content 3), the fourth content (e.g., content 4), the fifth content (e.g., content 5), and the sixth content (e.g., content 6). For example, the indicator may be displayed in the area (1120) and may be displayed in a visually distinguishable way, such as with a specified color, blur, and / or translucency.

[0210] According to the method described above, the electronic device (201) can visually provide information (e.g., visual effects through color, transparency adjustment) indicating which content is to be analyzed for recommended actions using an artificial intelligence (AI) model among the content displayed on the display (e.g., the display (260) of FIG. 2).

[0211] FIG. 11b is a diagram illustrating the operation of an electronic device checking content according to a tap input to a key assembly, according to one embodiment.

[0212] Referring to (a) of FIG. 11b, according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen containing at least one piece of content. For example, at least one piece of content may include an image and / or text. For example, at least one piece of content may include a first piece of content (e.g., content 1), a second piece of content (e.g., content 2), a third piece of content (e.g., content 3), a fourth piece of content (e.g., content 4), a fifth piece of content (e.g., content 5), and a sixth piece of content (e.g., content 6).

[0213] According to one embodiment, the electronic device (201) can detect a swipe input for a key assembly (200) (e.g., the key assembly (200) of FIG. 2b). According to one embodiment, the electronic device (201) can detect the direction of movement and the distance of movement of the swipe input.

[0214] According to one embodiment, the electronic device (201) can identify an area (1130) including a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6) based on the direction of movement and the distance of movement of a swipe input.

[0215] According to one embodiment, the electronic device (201) can determine whether the swipe input has reached a threshold point based on the direction and distance of movement of the swipe input. For example, when the electronic device (201) determines that the swipe input has reached a threshold point, it may display or provide a visual cue and / or vibration feedback. For example, when the threshold point of the swipe input is a point corresponding to the top of the first content or the bottom of the sixth content, the electronic device (201) may display or provide a visual cue and / or vibration feedback in response to an upward swipe input beyond the top of the first content or a downward swipe input beyond the bottom of the sixth content.

[0216] Referring to (b) of FIG. 11b, according to one embodiment, an electronic device (201) can (manually) adjust, determine, or select at least one of the multiple contents to be analyzed using an artificial intelligence (AI) model after a plurality of contents are selected by a swipe input to a key assembly (200) (e.g., the key assembly (200) of FIG. 2b).

[0217] According to one embodiment, the electronic device (201) can check for a tap input to a key assembly (200) (e.g., the key assembly (200) of FIG. 2b) after checking an area (1130) including a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6).

[0218] According to one embodiment, the electronic device (201) can identify the location where a tap input is identified in the key assembly (200). According to one embodiment, the electronic device (201) can identify the area (1140) corresponding to the location where a tap input is identified among the areas (1130) including a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6).

[0219] According to one embodiment, after the electronic device (201) identifies an area (1140) corresponding to a location where a tap input is identified, if a press input for the key assembly (200) is identified, a plurality of functions related to the third content (e.g., content 3) included in the area (1140) corresponding to the location where the tap input is identified can be identified.

[0220] According to the method described above, the electronic device (201) can visually provide information (e.g., visual effects through color, transparency adjustment) indicating which content is to be analyzed for recommended actions using an artificial intelligence (AI) model among the content displayed on the display (e.g., the display (260) of FIG. 2).

[0221] FIG. 12 is a diagram illustrating the operation of an electronic device checking a second screen in response to a swipe input for a first screen, according to one embodiment.

[0222] Referring to FIG. 12(a), according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) may display a first screen through a display (260) (e.g., the display (260) of FIG. 2b). For example, the first screen may display a screen including a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6).

[0223] According to one embodiment, as described in FIG. 11b, the electronic device (201) can identify an area (1130) corresponding to the direction of movement and distance of movement of a swipe input based on a swipe input to a key input unit (e.g., the key assembly (200) of FIG. 2b). The electronic device (201) can identify a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6) included in the area (1130). For example, the electronic device (201) can apply a visual effect indicating a swipe input to the area (1130) identified according to the swipe input to the key input unit (e.g., the key assembly (200)) on the first screen.

[0224] According to one embodiment, the electronic device (201) may start an operation to analyze the plurality of contents (e.g., contents 1 to 6) when the plurality of contents (e.g., contents 1 to 6) are identified (or after a specified time following identification). The electronic device (201) may identify first information regarding text and / or images included in the first content (e.g., content 1), the second content (e.g., content 2), the third content (e.g., content 3), the fourth content (e.g., content 4), the fifth content (e.g., content 5), and the sixth content (e.g., content 6), and / or second information regarding at least one application of the electronic device (201) related to the contents (e.g., contents 1 to 6).

[0225] According to one embodiment, the electronic device (201) can identify a plurality of functions related to a first content (e.g., content 1), a second content (e.g., content 2), a third content (e.g., content 3), a fourth content (e.g., content 4), a fifth content (e.g., content 5), and a sixth content (e.g., content 6) based on providing first information and second information to an artificial intelligence model.

[0226] According to one embodiment, the electronic device (201) may display a plurality of objects (1211, 1220, 1230) corresponding to a specified number (e.g., 3) of functions among a plurality of functions, and may display information (1210) about a first function related to at least one content corresponding to a first object (1211) selected among the plurality of objects.

[0227] For example, a plurality of objects (1211, 1220, 1230) may each represent a pre-assigned object for a specified number of functions among a plurality of functions. For example, the plurality of objects (1211, 1220, 1230) may include icons, thumbnails, or widgets. For example, object (1211) may represent a function related to a calendar application. For example, object (1220) may represent a function related to a contact application. For example, object (1230) may represent a function related to a map application.

[0228] According to one embodiment, the electronic device (201) can detect a swipe input (1240) (e.g., an upward swipe input) for the display (260) (or the first screen) after information (1210) for the first function and a plurality of objects (1211, 1220, 1230) are displayed. According to one embodiment, the electronic device (201) can scroll the first screen (e.g., scroll-down) based on detecting the swipe input (1240) for the display (260) (or the first screen). The electronic device (201) can disable the previously displayed visual effect based on the first screen being scrolled.

[0229] Referring to FIG. 12(b), according to one embodiment, an electronic device (201) may display a second screen (e.g., a second screen with the first screen scrolled down) through a display (260) based on a swipe input (1240) for a first screen. For example, the second screen may display a screen including a fifth content (e.g., content 5), a sixth content (e.g., content 6), a seventh content (e.g., content 7), an eighth content (e.g., content 8), a ninth content (e.g., content 9), and a tenth content (e.g., content 10). For example, the second screen may display at least a portion of a plurality of contents included in the first screen. Depending on the implementation, the second screen may display multiple contents (e.g., content 8, content 9, content 10) that are not included in the first screen.

[0230] According to one embodiment, the electronic device (201) may display an indicator (1250) capable of analyzing the second screen on the second screen. For example, the indicator (1250) may be an object for analyzing a plurality of contents included in the second screen.

[0231] Referring to FIG. 12(c), according to one embodiment, an electronic device (201) can detect user input (e.g., touch input, long press input, tap input) for an indicator (1250) through a display (260). In response to user input for the indicator (1250), the electronic device (201) can perform an operation to analyze content from the content where analysis of the existing first screen began (e.g., content 1) to the last content of the scrolled second screen (e.g., content 10). For example, when results of analysis for some content are obtained before the second screen is displayed, the electronic device (201) may perform analysis for the remaining content. For example, the electronic device (201) may use the analysis results for a plurality of contents included in the first screen to the second screen in providing a function related to a recommended action.

[0232] According to another embodiment, the electronic device (201) may perform an operation to analyze content from the content from the first screen that started analysis (e.g., content 1) to the last content of the second screen that scrolled from the first screen (e.g., content 10) after a specified time has elapsed since the second screen that scrolled from the first screen was displayed, even without user input to the indicator (1250). Based on the results of analyzing the content from the first screen that started analysis (e.g., content 1) to the last content of the second screen that scrolled (e.g., content 10), the electronic device (201) may provide related functions as information for recommendation actions.

[0233] According to one embodiment, the electronic device (201) can check third information about text and / or images included in the fifth content (e.g., content 5), the sixth content (e.g., content 6), the seventh content (e.g., content 7), the eighth content (e.g., content 8), the ninth content (e.g., content 9), and the tenth content (e.g., content 10), and / or fourth information about at least one application of the electronic device (201) related to the contents (e.g., content 5 to 10).

[0234] According to one embodiment, the electronic device (201) can identify a plurality of functions related to a fifth content (e.g., content 5), a sixth content (e.g., content 6), a seventh content (e.g., content 7), an eighth content (e.g., content 8), a ninth content (e.g., content 9), and a tenth content (e.g., content 10) based on providing third information and fourth information to an artificial intelligence model.

[0235] According to another embodiment, when the electronic device (201) obtains results of analysis from the first content to the sixth content through the first information and the second information, the electronic device (201) may perform an operation to analyze the content from the seventh content to the tenth content. At this time, the electronic device (201) may further provide the information analyzed regarding the content to an artificial intelligence model to provide related functions as recommendation actions.

[0236] According to one embodiment, an electronic device (201) may display a plurality of objects (1261, 1270, 1280) corresponding to a specified number (e.g., 3) of functions among a plurality of functions, and may display information (1260) regarding a second function related to at least one content corresponding to a first object (1261) selected among the plurality of objects. For example, when the first function is a function for a schedule, the information (1210) regarding the first function may include information regarding the date, time, place, and / or purpose of the schedule. When the second function is a function for a gallery, the information (1260) regarding the second function may include information regarding photos or videos (e.g., thumbnail images).

[0237] For example, a plurality of objects (1261, 1270, 1280) may each represent a pre-assigned object for a specified number of functions among a plurality of functions. For example, the plurality of objects (1261, 1270, 1280) may include icons, thumbnails, or widgets. For example, object (1261) may represent a function related to a gallery application. For example, object (1270) may represent a function related to a web browser application. For example, object (1280) may represent a function related to a camera application.

[0238] FIG. 13 is a diagram illustrating the operation of an electronic device, according to one embodiment, displaying information about a plurality of objects and a first function regarding music-related content included in a first screen.

[0239] Referring to FIG. 13(a), according to one embodiment, an electronic device (201) (e.g., the electronic device (201) of FIG. 2b) can display a first screen through a display (260) (e.g., the display (260) of FIG. 2b). For example, the first screen may display a screen containing at least one piece of content.

[0240] According to one embodiment, the first screen may include a screen in which a song playback application is executed. According to one embodiment, the first screen may include an image (1310) representing an album cover, text (1320) representing a song title and singer name, and text (1330) representing song lyrics.

[0241] According to one embodiment, when a swipe input for a key assembly (200) (e.g., the key assembly (200) of FIG. 2b) is detected, the electronic device (201) can detect only the portion of the first screen corresponding to the direction of movement and distance of movement of the swipe input. For example, the portion of the first screen corresponding to the direction of movement and distance of movement of the swipe input may display the entire first screen.

[0242] According to one embodiment, the electronic device (201) can verify, based on a verification operation, information regarding an image (1310) representing an album cover included in a first screen, text (1320) representing a song title and singer name, and text (1330) representing song lyrics as first information. According to one embodiment, the electronic device (201) can verify information regarding an object (e.g., a person) included in the image (1310), and the context or subject of the texts (1320, 1330) as first information.

[0243] According to one embodiment, the electronic device (201) can verify information related to a song playback application as second information. According to one embodiment, the electronic device (201) can obtain information as second information regarding at least one of the following: a song with a melody similar to the song currently being played using the song playback application, the genre of the song currently being played using the song playback application, other songs released by the singer of the song currently being played using the song playback application, a history of songs played through the song playback application, a storyline (content) of the song currently being played using the song playback application, or usage history of the song playback application (e.g., time of use, place of use).

[0244] Referring to FIG. 13 (b), according to one embodiment, an electronic device (201) can check a plurality of functions related to a first screen showing a screen that has executed a song playback application based on providing first information and second information to an artificial intelligence model.

[0245] According to one embodiment, the electronic device (201) can display a plurality of objects (1341, 1350) corresponding to a specified number of functions among a plurality of functions. According to one embodiment, the electronic device (201) can display information (1340) about a first function.

[0246] For example, the first function may represent a function that executes a web browser application of the electronic device (201).

[0247] For example, a function corresponding to the object (1350) may represent a function that can share information about the song currently being played (e.g., an internet address where a song playback application can be executed, the song title, the song lyrics) with an external electronic device corresponding to another user.

[0248] According to one embodiment, the electronic device (201) can execute a first function when a press input to the key assembly (200) is confirmed.

[0249] According to one embodiment, an electronic device (201) may include a pressure sensor (203) disposed on one side of a housing of the electronic device (201), a key input unit (200) disposed on one side of the housing of the electronic device (201) and disposed above the pressure sensor (203), a display (260), at least one processor (220), and a memory (210) for storing instructions.

[0250] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor (220), may cause the electronic device (201) to check at least one content included in the first screen based on checking a first swipe input in one direction for the key input part using the pressure sensor (203) while displaying the first screen through the display (260).

[0251] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause the electronic device (201) to check first information about text and / or images included in the at least one content.

[0252] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to check second information regarding at least one application of the electronic device (201) related to the at least one content.

[0253] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to identify a plurality of functions related to the at least one content based on providing the first information and the second information to an artificial intelligence model.

[0254] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to display information about a first function related to the at least one content corresponding to a first object selected among the plurality of objects, while displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display.

[0255] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause the first function to be executed for the at least one content based on confirming a first press input for the key input unit (200) while displaying information about the first function.

[0256] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to select a second object among the plurality of objects corresponding to the distance and direction of movement of the second swipe input based on confirming the second swipe input for the key input part.

[0257] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause information about a second function related to the at least one content corresponding to the second object to be displayed through the display (260).

[0258] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause the second function to be executed for the at least one content based on confirming a second press input for the key input unit while displaying information about the second function.

[0259] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to obtain, as the first information, information about at least one of the subject or context of the text included in the at least one content and / or information about at least one of the location, time, or object included in the image included in the at least one content where the image was taken, and information about at least one of the history of a plurality of conversations related to the message application or the usage history of the message application as the second information.

[0260] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to obtain, as the first information, information about at least one of the subject or context of the text included in the at least one content and / or information about at least one of the location, time, or object included in the image included in the at least one content, and as the first information, information about content similar to the content being played using the content playback application, a history of the content played through the content playback application, a plot of the content being played using the content playback application, or a usage history of the content playback application.

[0261] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to determine the distance between a first position of the key input unit (200) where the first swipe input is started and a second position of the key input unit (200) where the first swipe input is released.

[0262] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to identify the first content corresponding to the distance among the at least one content included in the first screen.

[0263] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to determine the priority among the plurality of functions.

[0264] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to display the first object at the top of the first screen above the remaining objects excluding the first object among the plurality of objects, and to display information about the first function around the first object, based on the fact that the first function among the plurality of functions is identified as the highest priority function.

[0265] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to identify a first content and a second content corresponding to the direction and distance of movement of the first swipe input among the at least one content.

[0266] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to determine the content displayed at the top of the first screen among the first content and the second content as the content of higher priority.

[0267] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause the plurality of functions to be identified based on the priority.

[0268] According to one embodiment, the instructions, when executed individually or collectively by the at least one processor (220), may cause the electronic device (201) to display information about a third function corresponding to a selected third object among the plurality of first objects, by changing the plurality of objects into a plurality of first objects different from the plurality of objects through the display (260), based on confirming that the movement distance of the third swipe input for the key input unit (200) is greater than a specified first distance. According to one embodiment, the plurality of first objects correspond to a specified number of functions among the plurality of functions, and the functions corresponding to the plurality of first objects may be different from the functions corresponding to the plurality of objects.

[0269] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to display, through the display (260), an indicator that can execute a function to check the second screen and a second screen different from the first screen according to a swipe input for the first screen, after checking the at least one content included in the first screen.

[0270] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to check the second screen when user input selecting the indicator is confirmed through the display (260).

[0271] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to check third information regarding text and / or images included in at least one first content included in the second screen based on checking the second screen.

[0272] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may be caused to check a fourth information regarding at least one application of the electronic device (201) related to the at least one first content.

[0273] According to one embodiment, the instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (220), to identify a plurality of functions related to the at least one first content based on providing the third information and the fourth information to the artificial intelligence model.

[0274] According to one embodiment, the instructions may cause the electronic device (201), when executed individually or collectively by the at least one processor (220), to identify a plurality of functions related to the at least one content based on acquiring a voice through the microphone, and providing information about the voice, the first information, and the second information to the artificial intelligence model.

[0275] According to one embodiment, when the instructions are executed individually or collectively by the at least one processor (220), the electronic device (201) may cause a visual effect to be applied over an area corresponding to the at least one identified content included in the first screen.

[0276] According to one embodiment, in a method of operating an electronic device (201), while displaying a first screen through a display (260) of the electronic device, the method may include an operation of checking at least one content included in the first screen based on checking a first swipe input in one direction for a key input unit (200) of the electronic device (201) using a pressure sensor (203) of the electronic device (201).

[0277] According to one embodiment, a method of operating an electronic device (201) may include an operation of checking first information about text and / or an image included in at least one content.

[0278] According to one embodiment, a method of operating an electronic device (201) may include an operation of checking second information regarding at least one application of the electronic device related to at least one content.

[0279] According to one embodiment, in a method of operating an electronic device (201), the method may include an operation of identifying a plurality of functions related to at least one content based on providing the first information and the second information to an artificial intelligence model.

[0280] According to one embodiment, in a method of operating an electronic device (201), the method may include displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display (260), and displaying information about a first function related to at least one content corresponding to a first object selected among the plurality of objects.

[0281] According to one embodiment, in a method of operating an electronic device (201), the method may include an operation of executing the first function for the at least one content based on confirming a first press input for the key input unit (200) while displaying information about the first function.

[0282] According to one embodiment, in a method of operating an electronic device (201), based on confirming a second swipe input for the key input unit (200), the method may include an operation of selecting a second object among the plurality of objects that corresponds to the movement distance and movement direction of the second swipe input.

[0283] According to one embodiment, a method of operating an electronic device (201) may include an operation of displaying information about a second function related to at least one content corresponding to the second object through the display (260).

[0284] According to one embodiment, in a method of operating an electronic device (201), the method may include an operation of executing the second function for at least one content based on confirming a second press input for the key input unit (200) while displaying information about the second function.

[0285] According to one embodiment, in a method of operating an electronic device (201), the first screen includes a conversation screen related to a message application of the electronic device (201), and may include an operation of obtaining as first information information about at least one of the subject or context of the text included in the at least one content and / or information about at least one of the location, time, or object included in the image included in the at least one content where the image was taken and information about at least one of the image included in the at least one content as second information, and an operation of obtaining information about at least one of the history of a plurality of conversations related to the message application or the usage history of the message application.

[0286] According to one embodiment, in a method of operating an electronic device (201), the first screen includes a screen related to a content playback application of the electronic device (201), and may include an operation of obtaining as the first information information about at least one of the subject or context of the text included in the at least one content and / or information about at least one of the location, time, or object included in the image included in the at least one content where the image was taken and the location, time, or object included in the image, and an operation of obtaining as the second information information about at least one of content similar to the content being played using the content playback application, a history of the content played through the content playback application, a plot of the content being played using the content playback application, or a usage history of the content playback application.

[0287] According to one embodiment, in a method of operating an electronic device (201), the method may include an operation of checking the distance between a first position of the key input unit (200) where the first swipe input is started and a second position of the key input unit where the first swipe input is released.

[0288] According to one embodiment, a method of operating an electronic device (201) may include an operation of checking a first content corresponding to the distance among at least one content included in the first screen.

[0289] According to one embodiment, a method of operating an electronic device (201) may include an operation of determining the priority among the plurality of functions.

[0290] According to one embodiment, in a method of operating an electronic device (201), based on the finding that the first function among the plurality of functions is the function with the highest priority, the method may include displaying the first object at the top of the first screen above the remaining objects excluding the first object among the plurality of objects, and displaying information about the first function around the first object.

[0291] According to one embodiment, in a method of operating an electronic device (201), the operation may include checking a first content and a second content corresponding to the movement direction and movement distance of the first swipe input among the at least one content.

[0292] According to one embodiment, a method of operating an electronic device (201) may include an operation of determining the content displayed at the top of the first screen among the first content and the second content as the content of high priority.

[0293] According to one embodiment, in a method of operating an electronic device (201), the operation of checking the plurality of functions based on the priority may include the operation of checking the plurality of functions.

[0294] According to one embodiment, in a method of operating an electronic device (201), based on confirming that the movement distance of a third swipe input for the key input unit (200) is greater than a designated first distance, the method may include an operation of changing the plurality of objects into a plurality of first objects different from the plurality of objects through the display (260) and displaying information about a third function corresponding to a selected third object among the plurality of first objects. According to one embodiment, the plurality of first objects correspond to a designated number of functions among the plurality of functions, and the functions corresponding to the plurality of first objects may be different from the functions corresponding to the plurality of objects.

[0295] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include an operation of checking at least one content included in the first screen based on checking a first swipe input in one direction for a key input part of the electronic device using a pressure sensor of the electronic device while displaying a first screen through the display of the electronic device.

[0296] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include an operation of verifying first information about text and / or an image included in the at least one content.

[0297] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include an operation of checking second information about at least one application of the electronic device (201) related to the at least one content.

[0298] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include an operation of identifying a plurality of functions related to the at least one content based on providing the first information and the second information to an artificial intelligence model.

[0299] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include displaying information about a first function related to a selected first object among the plurality of objects, while displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the display (260).

[0300] According to one embodiment, in a non-transient storage medium for storing computer-readable instructions, the instructions cause the electronic device (201) to perform at least one operation when executed by at least one processor (220) of the electronic device, and the at least one operation may include an operation of performing the first function on the at least one content based on confirming a first press input for the key input unit (200) while displaying information about the first function.

[0301] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0302] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationally,” it means that said component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0303] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0304] Various embodiments of this document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101, 201)). For example, a processor (e.g., processor (120, 220)) or control circuit (220) of the machine (e.g., electronic device (101, 201)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium is tangible It merely means that it is a device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently on a storage medium and cases where it is stored temporarily.

[0305] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0306] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), A pressure sensor (203) disposed on one side of the housing of the electronic device; A key input unit (200) disposed on one side of the housing of the electronic device and disposed on the pressure sensor; Display (260); At least one processor (220); and It includes a memory (210) for storing instructions, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Based on confirming a first swipe input in one direction for the key input unit using the pressure sensor while displaying a first screen through the display, and confirming at least one content included in the first screen, Identifying first information regarding text and / or images included in at least one of the above contents, and Identifying second information regarding at least one application of the electronic device related to the at least one content, and Based on providing the above first information and the above second information to an artificial intelligence model, a plurality of functions related to the above at least one content are identified, and Through the above display, a plurality of objects corresponding to a specified number of functions among the plurality of functions are displayed, and information regarding a first function related to at least one content corresponding to a first object selected among the plurality of objects is displayed. An electronic device that causes the first function to be executed for at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Based on confirming a second swipe input for the key input unit, a second object corresponding to the movement distance and movement direction of the second swipe input is selected among the plurality of objects, and Through the above display, information regarding a second function related to at least one content corresponding to the second object is displayed, and An electronic device that causes the second function to be executed for at least one content based on confirming a second press input for the key input unit while displaying information about the second function.

3. In any one of paragraphs 1 to 2, The first screen above includes a conversation screen related to a message application of the electronic device, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Information regarding at least one of the subject or context of the text included in at least one of the above-mentioned content and / or information regarding at least one of the location, time, or object included in the image included in the above-mentioned at least one of the above-mentioned image being taken is obtained as the first information, and An electronic device that causes to obtain information regarding at least one of a history of multiple conversations related to the message application or a usage history of the message application as the second information.

4. In any one of paragraphs 1 through 3, The first screen above includes a screen related to a content playback application of the electronic device, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Information regarding at least one of the subject or context of the text included in at least one of the above-mentioned content and / or information regarding at least one of the location, time, or object included in the image included in the above-mentioned at least one of the above-mentioned image being taken is obtained as the first information, and An electronic device that causes to obtain information as the second information regarding at least one of content similar to the content currently being played using the content playback application, a history of content played through the content playback application, a plot of the content currently being played using the content playback application, or a usage history of the content playback application.

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Checking the first position of the key input unit where the first swipe input is started, the second position of the key input unit where the first swipe input is released, and the distance between the first position and the second position, An electronic device that causes to check a first content corresponding to the distance among at least one content included in the first screen.

6. In any one of paragraphs 1 through 5, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Check the priority among the above multiple functions, and An electronic device that, based on the fact that the first function among the plurality of functions is identified as having the highest priority, displays the first object at the top of the first screen above the remaining objects among the plurality of objects excluding the first object, and causes information about the first function to be displayed around the first object.

7. In any one of paragraphs 1 through 6, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identifying a first content and a second content among the above at least one content that correspond to the movement direction and movement distance of the first swipe input, and Among the first content and the second content, the content displayed at the top of the first screen is determined to be the content with high priority, and An electronic device that causes the plurality of functions to be verified based on the priority between the first content and the second content.

8. In any one of paragraphs 1 through 7, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Based on confirming that the movement distance of the third swipe input for the above key input unit is greater than the designated first distance, the plurality of objects are changed into a plurality of first objects different from the plurality of objects through the display, and information regarding a third function corresponding to a selected third object among the plurality of first objects is displayed. An electronic device characterized in that the plurality of first objects correspond to the specified number of functions among the plurality of functions, and the functions corresponding to the plurality of first objects are different from the functions corresponding to the plurality of objects.

9. In any one of paragraphs 1 through 8, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, After confirming at least one content included in the first screen, the display shows, through the display, a second screen different from the first screen and an indicator capable of executing a function to confirm the second screen on the second screen according to a swipe input for the first screen, and When user input selecting the indicator is confirmed through the display, the second screen is checked, and Based on checking the second screen above, third information regarding text and / or images included in at least one first content included in the second screen, and Checking fourth information regarding at least one application of the electronic device related to at least one first content, and An electronic device that causes a plurality of functions related to at least one first content to be identified based on providing the above third information and the above fourth information to the above artificial intelligence model.

10. In any one of paragraphs 1 through 9, Includes more microphones, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Through the above microphone, voice is acquired, and An electronic device that causes a plurality of functions related to at least one content to be identified based on providing information about the voice, the first information, and the second information to the artificial intelligence model.

11. In any one of paragraphs 1 through 10, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that causes a visual effect to be applied over an area corresponding to at least one identified content included in the first screen.

12. In the method of operating the electronic device (201), An operation of checking at least one content included in the first screen based on confirming a first swipe input in one direction to the key input unit (200) of the electronic device using the pressure sensor (203) of the electronic device while displaying the first screen through the display (260) of the electronic device; An operation to verify first information regarding text and / or images included in at least one of the above contents; An operation to verify second information regarding at least one application of the electronic device related to at least one content; An operation to identify a plurality of functions related to at least one content based on providing the above first information and the above second information to an artificial intelligence model; An operation of displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the above display, and displaying information about a first function related to at least one content corresponding to a first object selected among the plurality of objects; and A method of operation of an electronic device comprising an operation of executing the first function for at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

13. In Paragraph 12, Based on confirming a second swipe input for the key input unit, an operation of selecting a second object among the plurality of objects that corresponds to the movement distance and movement direction of the second swipe input; An operation of displaying information about a second function related to at least one content corresponding to the second object through the display; and A method of operation of an electronic device further comprising an operation of executing the second function for at least one content based on confirming a second press input for the key input unit while displaying information about the second function.

14. In any one of paragraphs 12 to 13, The first screen above includes a conversation screen related to a message application of the electronic device, and The operation of obtaining as the first information information about at least one of the subject or context of the text included in at least one of the content and / or information about at least one of the location, time, or object included in the image where the image included in at least one of the content was taken; and A method of operating an electronic device further comprising the operation of obtaining information as the second information regarding at least one of a history of a plurality of conversations related to the message application or a usage history of the message application.

15. In a non-transient storage medium for storing computer-readable instructions, said instructions cause said electronic device (201) to perform at least one operation when executed by at least one processor (220) of said electronic device, said at least one operation is: An operation of checking at least one content included in the first screen based on confirming a first swipe input in one direction to the key input unit (200) of the electronic device using the pressure sensor (203) of the electronic device while displaying the first screen through the display (260) of the electronic device; An operation to verify first information regarding text and / or images included in at least one of the above contents; An operation to verify second information regarding at least one application of the electronic device related to at least one content; An operation to identify a plurality of functions related to at least one content based on providing the above first information and the above second information to an artificial intelligence model; An operation of displaying a plurality of objects corresponding to a specified number of functions among the plurality of functions through the above display, and displaying information about a first function related to at least one content corresponding to a first object selected among the plurality of objects; and A storage medium comprising an operation to execute the first function for at least one content based on confirming a first press input for the key input unit while displaying information about the first function.

Images