Electronic device and method by which electronic device determines content-based recommendation function

Abstract

An electronic device according to an embodiment of the present invention may comprise: a display; a memory for storing instructions; and at least one processor. The instructions, when executed by the at least one processor, may cause the electronic device to analyze context information about content including at least one image, a video, or text displayed on the display. The instructions, when executed by the at least one processor, may cause the electronic device to identify association information between the context information, applications in the electronic device, and functions of the electronic device. The instructions, when executed by the at least one processor, may cause the electronic device to determine at least one recommendation application or recommendation function on the basis of the context information and the association information. The instructions, when executed by the at least one processor, may cause the electronic device to display, together with the content, an icon corresponding to the at least one recommendation application or recommendation function.

Description

Method for determining electronic devices and content-based recommendation features of electronic devices

[0001] The present disclosure relates to an electronic device and a method of operating the electronic device, and to a technology for determining recommended applications and functions based on content displayed on the electronic device and on the display of the electronic device.

[0002] With the development of digital technology, various types of electronic devices such as mobile communication terminals, PDAs (personal digital assistants), electronic notebooks, smartphones, tablet PCs (personal computers), or wearable devices are widely used. To support and enhance the functionality of these electronic devices, the hardware and / or software parts of the devices are continuously being improved.

[0003] Electronic devices can provide content to users through display screens. For example, various content may include messages, files, image captures, video captures, internet search screens, and / or notes. Users can receive information through content displayed on the electronic device's screen and process that information using applications or functions present within the electronic device.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the above is to be claimed as prior art related to this document, nor can it be used to determine prior art.

[0005] An electronic device according to one embodiment of the present invention may include a display.

[0006] An electronic device according to one embodiment of the present invention may include a memory for storing instructions.

[0007] An electronic device according to one embodiment of the present invention may include at least one processor.

[0008] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may be able to analyze context information of content including at least one image, video, or text displayed on the display.

[0009] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may be able to identify association information between the context information, the application within the electronic device, and the function of the electronic device.

[0010] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may determine at least one recommendation application or recommendation function based on the context information and the association information.

[0011] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one recommendation application or recommendation function together with the content.

[0012] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may display a first notification bar including a first icon corresponding to a first application at a first position, a second icon corresponding to a second application at a second position, and information of a third application at a third position through a lock screen on the display.

[0013] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may receive a first user input with respect to the first notification bar.

[0014] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may analyze the context information of the third application to determine at least one first recommendation application or first recommendation function.

[0015] In an electronic device according to one embodiment of the present invention, when the instructions are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one first recommendation application or first recommendation function together with the first notification bar.

[0016] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments of the present invention.

[0017] FIG. 2 is a block diagram of a display module according to various embodiments of the present invention.

[0018] FIG. 3 is a block diagram of a generative artificial intelligence (AI) system according to one embodiment.

[0019] FIG. 4 is a block diagram of an AI Framework according to one embodiment.

[0020] FIG. 5 is a block diagram of an electronic device according to one embodiment of the present invention.

[0021] FIG. 6 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to determine a recommended application or recommendation function based on content displayed on a display.

[0022] FIG. 7 illustrates an electronic device that displays an icon corresponding to a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0023] FIG. 8 illustrates an electronic device that displays a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0024] FIGS. 9a and 9b illustrate an electronic device that displays an icon corresponding to a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0025] FIG. 10 illustrates an electronic device that displays a recommendation application or recommendation function along with content according to one embodiment of the present invention.

[0026] FIG. 11 illustrates an electronic device that displays a recommendation application or recommendation function along with content according to one embodiment of the present invention.

[0027] FIG. 12 illustrates examples of various content-specific recommendation applications or recommendation functions displayed on an electronic device according to an embodiment of the present invention.

[0028] FIG. 13 illustrates examples of various content-specific recommendation applications or recommendation functions displayed on an electronic device according to an embodiment of the present invention.

[0029] FIG. 14 illustrates examples of various content-specific recommendation applications or recommendation functions displayed on an electronic device according to an embodiment of the present invention.

[0030] FIG. 15 illustrates examples of various content-specific recommendation applications or recommendation functions displayed on an electronic device according to one embodiment.

[0031] FIG. 16 is a flowchart illustrating a method for determining a recommendation application or recommendation function based on content displayed on a display according to an embodiment of the present invention and executing a recommendation application.

[0032] FIG. 17 illustrates an electronic device that executes a recommendation application determined based on content displayed on a display according to an embodiment of the present invention.

[0033] FIG. 18 is a flowchart illustrating a method for determining a recommendation application or recommendation function based on content displayed on a display according to an embodiment of the present invention and executing a recommendation function.

[0034] FIG. 19 illustrates an electronic device that performs a recommendation function determined based on content displayed on a display according to one embodiment of the present invention.

[0035] FIG. 20 is a flowchart illustrating a method for determining a recommended application or recommendation function based on content displayed on a lock screen by an electronic device according to an embodiment of the present invention.

[0036] FIG. 21 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to display a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function in response to a user request.

[0037] FIG. 22 illustrates an electronic device that displays a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function according to a user request in accordance with an embodiment of the present invention.

[0038] FIG. 23 illustrates an electronic device that displays a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function according to a user request in accordance with an embodiment of the present invention.

[0039] FIGS. 24a to 24c illustrate a case where a temporary notification is received while displaying a recommendation application or recommendation function determined based on content displayed on a display according to an embodiment of the present invention.

[0040] FIGS. 25a to 25c illustrate a foldable electronic device that executes a recommendation application or recommendation function determined based on content displayed on a display according to an embodiment of the present invention.

[0041] FIGS. 26a and 26b illustrate that a recommended application or recommendation function determined based on content displayed on a display is displayed differently depending on the shape of a foldable electronic device according to an embodiment of the present invention.

[0042] An electronic device can provide content to a user through a display screen. The user can grasp information through the content displayed on the screen and use various applications or functions existing within the electronic device to record and utilize the grasped information.

[0043] In order for users to record and utilize the information they have gathered, a process of separately searching for and determining applications and functions within an electronic device may be required. Since electronic devices contain a wide variety of applications and functions, users may miss necessary applications or functions while searching for them in relation to the information they have gathered.

[0044] The technical problems to be solved in this document are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which this invention belongs from the description below.

[0045] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment of the present disclosure.

[0046] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0047] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0048] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0049] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0050] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0051] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0052] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0053] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0054] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0055] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0056] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0057] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0058] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0059] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0060] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0061] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0062] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0063] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0064] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a printed circuit board (PCB)). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0065] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0066] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0067] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0068] FIG. 2 is a block diagram of a display module (160) according to various embodiments of the present invention.

[0069] Referring to FIG. 2, the display module (160) may include a display (161) and a display driver IC (DDI) (165) for controlling the display. The DDI (165) may include an interface module (165a), a memory (165b) (e.g., buffer memory), an image processing module (165c), or a mapping module (165d). The DDI (165) may receive image information, including, for example, image data or an image control signal corresponding to a command for controlling said image data, from another component of the electronic device (101) through the interface module (165a). According to one embodiment, the image information may be received from a processor (120) (e.g., a main processor (121) (e.g., an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit) that operates independently of the function of the main processor (121). The DDI (165) can communicate with the touch circuit (168) or sensor module (176) through the interface module (165a). Additionally, the DDI (165) can store at least a portion of the received image information in memory (165b), for example, in frame units. The image processing module (165c) can perform preprocessing or postprocessing (e.g., resolution, brightness, or size adjustment) on at least a portion of the image data based at least on the characteristics of the image data or the characteristics of the display (161), for example. The mapping module (165d) can generate voltage values ​​or current values ​​corresponding to the image data preprocessed or postprocessed through the image processing module (165c). According to one embodiment, the generation of voltage values ​​or current values ​​can be performed based at least partially on the attributes of the pixels of the display (161), for example, (e.g., array of pixels (RGB stripe or pentile structure), or the size of each subpixel).At least some pixels of the display (161) are driven, for example, based on at least some of the voltage value or current value, so that visual information (e.g., text, image, or icon) corresponding to the image data can be displayed through the display (161).

[0070] According to one embodiment, the display module (160) may further include a touch circuit (168). The touch circuit (168) may include a touch sensor (168a) and a touch sensor IC (168b) for controlling the same. The touch sensor IC (168b) may control the touch sensor (168a) to detect a touch input or hovering input for a specific location on the display (161), for example. For example, the touch sensor IC (168b) may detect a touch input or hovering input by measuring a change in a signal (e.g., voltage, light intensity, resistance, or charge) for a specific location on the display (161). The touch sensor IC (168b) may provide information regarding the detected touch input or hovering input (e.g., location, area, pressure, or time) to the processor (120). According to one embodiment, at least a portion of the touch circuit (168) (e.g., touch sensor IC (168b)) may be included as part of the display driver IC (165) or the display (161), or as part of another component (e.g., auxiliary processor (123)) placed outside the display module (160).

[0071] According to one embodiment, the display module (160) may further include at least one sensor of the sensor module (176) (e.g., fingerprint sensor, iris sensor, pressure sensor, or light sensor) or a control circuit for the same. In this case, the at least one sensor or the control circuit for the same may be embedded in a part of the display module (160) (e.g., display (161) or DDI (165)) or a part of the touch circuit (168). For example, if the sensor module (176) embedded in the display module (160) includes a biometric sensor (e.g., fingerprint sensor), the biometric sensor may obtain biometric information (e.g., fingerprint image) associated with a touch input through a part of the display (161). For example, if the sensor module (176) embedded in the display module (160) includes a pressure sensor, the pressure sensor may obtain pressure information associated with a touch input through a part or the entire area of ​​the display (161). According to one embodiment, a touch sensor (168a) or a sensor module (176) may be placed between pixels of a pixel layer of a display (161), or on top of or below the pixel layer.

[0072] FIG. 3 is a generative artificial intelligence (AI) system (200) according to one embodiment. Referring to FIG. 2, the generative AI system (200) may include a User Interface (210), an AI Framework (220), a Generative AI Model (230), a Knowledge Repository (240), and an Application / Service Module (250). These components may be operated on one or more of an electronic device (101), an external electronic device (102 or 104), or a server (108). For example, the User Interface (210) and the AI ​​Framework (220) may be operated on the electronic device (101), and the Knowledge Repository (240) and the Generative AI Model (230) may be operated on the server (108).

[0073] According to one embodiment, the User Interface (210) may receive user input (e.g., user query). User input may be received in the form of text, images, voice (e.g., natural language), video, menu selection, or a combination thereof. The User Interface (210) may include various context information (e.g., running application or user location) related to the generative artificial intelligence system (200) at the time the user input is received, in addition to or instead of the user input. The User Interface (210) may provide the user input or the context information to the AI ​​Framework (220) and provide the result of processing therefrom to the user, for example, through the AI ​​Framework (220). According to one embodiment, in addition to user input, the electronic device may provide context information obtained using information included on the screen to the AI ​​Framework (220). The result may be provided in the form of text, images, voice, video, an action requested by the user (e.g., launching a specified function or app), or a combination thereof.

[0074] According to one embodiment, the AI ​​Framework (220) can identify (e.g., estimate) a user intent based on at least some user input or context information received from the User Interface (210), control each of the relevant modules (e.g., 221, 223, or 225) to perform a function or action corresponding to the identified user intent, and coordinate collaboration between two or more modules. The AI ​​Framework (220) may include a Prompt Design Module (221), an API / Plug-in Management Module (223), and an Output Modification Module (225), as illustrated in FIG. 2.

[0075] According to one embodiment, the Prompt Design Module (221) can generate a prompt to be input to the Generative AI Model (230) based at least partially on user input or context information received from the User Interface (210). For example, the Prompt Design Module (221) can generate a prompt using user preferences, a prompt library, or prompt examples stored in the Knowledge Repository (240) based at least partially on user input or context information.

[0076] According to one embodiment, the API / Plug-in Management Module (223) may communicate, for example, via an API, with various resources (e.g., Knowledge Repository (240)) that provide said additional information when there is a request for said additional information in relation to user input. Additionally or generally, when a specified action (e.g., function, app, or service) is performed in response to said user input, the API / Plug-in Management Module (223) may request the Application / Service Module (250) to perform said specified action via a corresponding API. The API / Plug-in Management Module (223) may provide information obtained from the Knowledge Repository (240), the Application / Service Module 250, or another external resource to the Prompt Design Module (221). That obtained information may be used by the Prompt Design Module (221) to generate a prompt along with the user input, or provided to a generative AI model (230).

[0077] According to one embodiment, the Output Modification Module (225) can fine-tune the results obtained through the Generative AI Model (230) as at least part of the response to user input (e.g., user query). For example, the Output Modification Module (225) can determine whether the content of the response obtained through the Generative AI Model (230) is appropriate as a response to a request made by the user input. For example, the Output Modification Module (225) can determine the degree of relevance, degree of bias (e.g., political or social bias), or degree of harmfulness (e.g., sexual or profanity) of the difference between the response obtained through the Generative AI Model (230) and the user input. Additionally or generally, the Output Modification Module (225) can request that additional AI processing be performed on the obtained response, or provide the user with a hint to avoid unwanted output. For example, additional prompts can be generated through the Prompt Design Module to obtain a response again through the Generative AI Model (230).

[0078] According to one embodiment, the Generative AI Model (230) may form at least part of an artificial intelligence neural network and may include a model that generates images or a model that generates language. The image generation model may include, for example, a generative adversarial network (GAN), a variational autoencoder (VAE), or a Diffusion-based model using a VAE and a Transformer. The language generation model may include, for example, a large language model (LLM), a large multimodal model (LMM), a large vision model (LVM), or a large action model (LAM). The LAM may automatically generate actions for an environment (e.g., a robot, a car, an electronic device (101), or a program (140)). Additionally, for at least some AI models (e.g., LLM), there may be a low-rank adaptation (LoRA) adaptor that is fine-tuned for, for example, a specific task or a specific situation.

[0079] FIG. 4 illustrates an AI Framework (220) having on-device AI processing capabilities according to one embodiment. In this case, the AI ​​Framework (220) may generate and learn a response to the user input using resources within the device, instead of sending the user input received through a User Interface (210) operating on the same device (e.g., electronic device (101)) to a Generative AI Model (230) operating on an external device (e.g., server (108)), or additionally. Referring to FIG. 3, the AI ​​Framework (220) may include a Cross-Application Action Module (310), a Personal Data Managing Module (330), an On-device AI Model (350), and an Orchestration Module (370).

[0080] According to one embodiment, the Cross-Application Action Module (310) determines one or more additional applications required for the operation of an executed application (e.g., an assistant app) and may link or suggest operations between the app and at least one additional application, or between multiple additional applications. For example, the Cross-Application Actions Module (310) may execute one or more additional applications to be used to respond to a user request through the assistant app sequentially or at least partially, simultaneously. Additionally, the Cross-Application Action Module (310) may communicate with the additional applications so that the result of the execution of one additional application (e.g., content) can be shared with other additional applications.

[0081] According to one embodiment, the Personal Data Managing Module (330) may provide personal information (e.g., schedule, contact, or message information) about a user of the application (e.g., assistant app) or the additional application running on the device (e.g., electronic device 101) or other related individuals (e.g., family or friends) to another module of the AI ​​Framework (220) or a related module (e.g., Generative AI Model 230) running on another device.

[0082] According to one embodiment, the on-device AI model (350) may include at least one model among one or more AI models (e.g., GAN, VAE, LLM, LMM, LVM, or LAM) operated on an external device (e.g., server (108)) or a corresponding lightweight AI model. Additionally, for said model or said lightweight model, there may be, for example, a LoRA adaptor.

[0083] According to one embodiment, the Orchestration Module (370) may select one or more AI models to be used to obtain a response to user input (e.g., user query). For example, the Orchestration Module (370) may select one or more AI models from among an On-Device AI Model (350), an AI model operating on an external device (e.g., server (108)) (e.g., Generative AI Model (230)), or a third AI model (not shown) operating on another external device. When multiple AI models are selected, the Orchestration Module (370) may communicate with the selected models or devices so that the operation between the selected AI models and the processing of the results thereof can be coordinated between the relevant models or devices.

[0084] According to one embodiment, two or more modules of a generative AI system (200) (e.g., Cross-Application Action Module (310) and Orchestration Module (370)) may be implemented as a single module to maintain the same functionality. Various variations are possible.

[0085] FIG. 5 is a block diagram of an electronic device according to one embodiment of the present invention.

[0086] Referring to FIG. 5, an electronic device (101) according to one embodiment of the present invention may include an input module (150), a display (161), a memory (130), and / or a processor (120). Various embodiments of this document may be implemented even if some of the illustrated configurations are omitted or replaced with other configurations. In addition to the illustrated configurations, the electronic device (101) may further include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1. At least some of each of the illustrated (or unillustrated) components of the electronic device (101) may be operatively, functionally, and / or electrically connected.

[0087] According to one embodiment, the input module (150), display (161), memory (130) and / or processor (120) may be electrically or operationally connected.

[0088] According to one embodiment, the input module (150) can receive commands or data to be used in a component (e.g., processor (120)) of the electronic device (101) from outside the electronic device (101) (e.g., user).

[0089] The input module (150) may include, for example, a display (161), a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0090] The input module (150) can receive user input from a user to receive recommendations for applications or functions associated with content displayed on the display (161). User input to receive recommendations for applications or functions associated with content displayed on the display (161) may include predetermined gesture inputs. For example, a user who wants to receive recommendations for applications or functions associated with content including images, videos, or text displayed on the display may make an input of touching the content as a predetermined gesture. The touch input for the user to select content may include inputs such as touching the content for a long period or touching it multiple times in succession.

[0091] According to one embodiment, the input module (150) may receive input from a user to generate content from at least one image, video, or text displayed on the display (161). For example, the input module (150) may receive a request from a user to generate a capture image including at least one image or video displayed on the display (161).

[0092] According to one embodiment, the input module (150) may receive user input requesting a change to the content displayed on the display (161) from the user. For example, if the content currently displayed on the display (161) is a first content (e.g., music playback content in the notification bar), the input may receive a request to change the content displayed on the display (161) from the first content to a second content (e.g., timer content in the notification bar).

[0093] According to one embodiment, a user input requesting a user to change the content displayed on the display (161) may include a predetermined gesture input. The predetermined gesture input may include a gesture input of touching the first content currently displayed on the display (161) and swiping (or dragging and stopping) in a predetermined direction (e.g., left direction). That is, when the user swipes the first content currently displayed on the display (161), the second content is displayed.

[0094] According to one embodiment, the input module (150) may receive user input for selecting an icon corresponding to a recommendation application associated with the content or an icon corresponding to a recommendation function. User input for selecting a recommendation application icon or a recommendation function icon may include inputs such as simply touching or long-term touching the content.

[0095] According to one embodiment, the input module (150) receives an input from a user selecting content present on the lock screen and can receive a user authentication input entered through a user authentication UI. The user authentication input may include obtaining personal information stored in advance from the user (e.g., fingerprint information, password information, face information, biometric information, etc.).

[0096] According to one embodiment, the memory (130) performs the function of storing a program (e.g., program (140) of FIG. 1) for processing and controlling the processor (120) of the electronic device (101), an operating system (OS) (e.g., operating system (142) of FIG. 1), various applications, and / or input / output data, and can store a program that controls the overall operation of the electronic device (101). The memory (130) can store various configuration information required for processing functions related to various embodiments of the present disclosure in the electronic device (101). The memory (130) can store executable instructions. For example, the memory (130) can store instructions that cause the electronic device (101) to perform operations when executed by the processor (120). For example, the instructions may be stored on a computer-readable recording medium. The above recording medium may be tangible and non-transitory. The memory (130) and / or the recording medium may store one or more programs containing instructions.

[0097] According to one embodiment, the memory (130) can store content displayed on the display (161) and context information related to the content.

[0098] According to one embodiment, the content may include at least one image, video, or text displayed on the display (161). For example, the content may include at least one image, video, or text related to a voice recording display, a song playback display, an alarm, or a timer displayed on the lock screen of the electronic device (101) display (161). For example, the content may include a captured image of the image, video, or text displayed on the display (161). For example, the content may include an image or text within a notification popup displayed on the display (161). For example, the content may include a message notification received by the electronic device (101), a missed call notification, a battery level notification, etc., displayed in the notification popup.

[0099] According to one embodiment, the memory (130) may store the usage history of the electronic device (101) and / or the usage history of applications existing within the electronic device (101). For example, the usage history of the electronic device (101) may include applications and functions previously or last used by the user, tasks that have not yet been terminated, etc. Additionally, the application usage history may include the types of applications frequently used by the user, search history within the applications, and whether functions within the applications have been terminated, etc. For example, applications previously used by the user may include applications running in the foreground, applications running in the background, or applications that have been suspended. An application that has been suspended may refer to an application that has been suspended from execution upon receiving user input instructing to stop (or terminate) execution or upon satisfying a predetermined suspension condition (e.g., elapsed time).

[0100] According to one embodiment, the memory (130) may store an electronic device usage model learned based on the usage history of the electronic device (101) and / or the application usage history existing within the electronic device (101). The electronic device usage model may be used to determine recommended applications or recommendation functions based on content displayed on the display (161), and may be learned to determine recommended applications or recommendation functions based on electronic device user information.

[0101] According to one embodiment, the processor (120) may be electrically or operationally connected to the input module (150), the display (161), and the memory (130). The processor (120) may control the functions and operations of the input module (150), the display (161), and the memory (130). The processor (120) may include at least one processing circuitry, and the processor (120) may include at least one processor.

[0102] According to one embodiment, the operations of the electronic device (101) or processor (120) described in FIGS. 1 to 26b may be performed individually or collectively by at least one processor included in the processor (120). The operations of the processor (120) illustrated in FIGS. 1 to 26b may be performed according to the execution of instructions included in memory (130). For example, in this document, an embodiment in which the electronic device (101) can perform any operation may be interpreted to mean that it is performed by at least one instruction stored in the processor (120) and / or memory (130).

[0103] According to one embodiment, the processor (120) can perform the function of controlling the overall operation of the electronic device (101) and the flow of signals between internal components, and processing data. The processor (120) may include, for example, a central processing unit (CPU), an application processor, and / or a communication processor. The processor (120) may include a single-core processor or a multi-core processor. The processor (120) may be composed of at least one processor. For example, the processor (120) may be an application processor (e.g., the main processor (121) of FIG. 1).

[0104] According to one embodiment, the processor (120) can determine at least one recommendation application or recommendation function based on content including at least one image, video, or text displayed on the display (161).

[0105] According to one embodiment, the processor (120) may receive user input to recommend an associated application or function based on content (162) displayed on the display (161). The user input to recommend an application or function associated with content (162) displayed on the display (161) may include a predetermined gesture input. For example, a user who wishes to receive recommendations for an application or function associated with content (162) displayed on the display (161) may make an input of touching the content as a predetermined gesture. The touch input by which the user selects the content may include inputs such as touching the content for a long period or touching it multiple times in succession. However, even in the absence of user input, the processor (120) may determine an associated application or function based on content (162) displayed on the display (161) and provide it to the user.

[0106] According to one embodiment, the processor (120) can analyze context information of content including at least one image, video, or text displayed on the display (161). Context information corresponds to information derived and extracted based on various information present in the content. For example, if the electronic device (101) is currently recording voice and this is displayed on the lock screen through the electronic device (101) display (161), the processor (120) can analyze the content currently being recorded and the language being recorded, etc., as context information based on the voice recording content. (Details regarding context information for various content will be described later in the drawings.)

[0107] According to one embodiment, context information may include context information of specific content selected by a user among content including at least one image, video, or text. For example, when a user selects specific content among an application, message, captured image, and note displayed on the display (161) of the electronic device (101), the processor (120) may analyze context information based on the content selected by the user.

[0108] According to one embodiment, the processor (120) can identify association information between the context information of the content and the function of the electronic device (101). The processor (120) can identify the association information based on at least one of the context information of the content, the user's usage history of the electronic device (101), and the usage history of the application installed in the electronic device (101). For example, if the electronic device (101) is currently performing voice recording and this is displayed through the notification bar of the lock screen on the display (161) of the electronic device (101), the processor (120) can analyze the content currently being recorded and the language being recorded, etc., as context information based on the voice recording content. The processor (120) can identify the language mainly used by the user of the electronic device (101), related information associated with the information included in the recording content, the usage history of the recording application, the number of uses, etc., as association information of the voice recording content. (Detailed information regarding association information for various content will be described later in the drawings.)

[0109] According to one embodiment, the processor (120) can determine a recommended application or recommendation function based on contextual information and association information of the content. For example, when the electronic device (101) is currently recording voice and this is displayed through the notification bar of the lock screen on the display (161) of the electronic device (101), the processor (120) can analyze the content currently being recorded and the language being recorded as contextual information, and identify the language mainly used by the user of the electronic device (101), the usage history and usage frequency of the recording application as association information. The processor (120) can determine a recommended application or recommendation function based on the contextual information and association information of the voice recording. For example, the recommended application or recommendation function may include a translation application for an automatic translation function, a note application for an automatic recording function, or a summary function.

[0110] According to one embodiment, the processor (120) may display an icon corresponding to the recommendation application or recommendation function around the content along with the content.

[0111] According to one embodiment, the processor (120) may display icons corresponding to the multiple recommendation applications along with the content around the content when the recommendation application consists of multiple recommendation applications. In response to receiving user input selecting one of the multiple recommendation application icons, the processor (120) may execute and enlarge the selected recommendation application and display it on a part of the display (161).

[0112] According to one embodiment, the processor (120) may display icons corresponding to the multiple recommendation functions along with the content around the content when the recommendation function is a plurality of recommendation functions. The processor (120) may execute the selected recommendation function and display it on a part of the display (161) in response to receiving user input selecting one of the multiple recommendation function icons.

[0113] According to one embodiment, the processor (120) may determine a recommended application or recommendation function based on content displayed on the lock screen of the electronic device (101). When the processor (120) receives user input from a user selecting a recommended application or recommendation function based on content displayed on the lock screen, the processor (120) may provide a user authentication UI on the lock screen. At this time, the information included in the recommended application or recommendation function determined based on the content (162) provided on the lock screen may include personal information requiring user authentication. When the user provides personal information for authentication (e.g., biometric information such as fingerprint information, password information, face information, etc.) through the user authentication UI, the processor (120) completes user authentication and can execute the recommended application or recommendation function selected by the user.

[0114] According to one embodiment, the processor (120) may receive user input for obtaining a capture image of at least one image, video, or text displayed on the display (161) from the user. The processor (120) may determine a recommended application or recommendation function based on the capture image displayed on the display (161) of the electronic device (101).

[0115] According to one embodiment, the processor (120) can learn an electronic device usage model specialized for the user based on the electronic device (101) user's usage history of the electronic device (101), the application usage history within the electronic device (101), etc. The processor (120) can determine a recommended application or recommendation function based on contextual information and association information of the content using the electronic device usage model learned by the electronic device (101) user.

[0116] According to one embodiment, the processor (120) may analyze context information of a first content including at least one image, video, or text displayed on a display (161). The context information of the first content may be first context information. The processor (120) may identify first association information between the first context information and the function of the electronic device (101), and determine a first recommendation application or a first recommendation function based on the first context information and the first association information. The processor (120) may display an icon corresponding to the first recommendation application or the first recommendation function around the first content together with the first content.

[0117] According to one embodiment, when the processor (120) receives a request from a user to display second content including at least one image, video, or text displayed on the display (161) from the first content, the processor (120) may analyze context information of the second content displayed on the display (161). The context information of the second content may be second context information. The processor (120) may identify second association information between the second context information and the function of the electronic device (101), and determine a second recommendation application or a second recommendation function based on the second context information and the second association information. The processor (120) may display an icon corresponding to the second recommendation application or the second recommendation function together with the second content.

[0118] According to one embodiment, the display (161) can display an initial screen, a settings screen, and / or an execution screen of various applications of the electronic device (101). The display (161) can provide a user interface (UI) capable of providing various services to the user of the electronic device (101).

[0119] According to one embodiment, the display (161) may display various content such as a home screen, a lock screen, text, images and / or videos.

[0120] According to one embodiment, the display (161) can perform display and input functions. To perform display and input functions, the display (161) may include a display (e.g., the display (161) of FIG. 2) and a touch circuit (e.g., the touch circuit (168) of FIG. 2). The display (161) can visually provide the user with a menu of the multi-electronic device (101), input data, function setting information, and various information. The display (161) may include at least one of a liquid crystal display (LCD), organic light emitting diodes (OLED), and active matrix organic light emitting diodes (AMOLED). The touch circuit (168) may include a touch sensor (e.g., the touch sensor (168a) of FIG. 2) or a pressure sensor, such as a capacitive overlay, a resistive overlay, or an infrared beam.

[0121] FIG. 6 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to determine a recommended application or recommendation function based on content displayed on a display. (See FIG. 7 and FIG. 8)

[0122] In the following embodiments, each operation of FIG. 6 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation of FIG. 6 may be changed, and at least two operations may be performed in parallel.

[0123] According to one embodiment, operations 1000 to 1300 of FIG. 6 may be understood to be performed in a processor (e.g., processor (120) of FIG. 5) of an electronic device (e.g., electronic device (101) of FIG. 5).

[0124] According to one embodiment, before performing the operation, the electronic device (101) may perform an operation to establish communication (e.g., pairing) with an external electronic device.

[0125] According to one embodiment, the processor (120) of the electronic device (101) (or the electronic device (101)) can analyze contextual information of content displayed on the display (161) (1000). Contextual information corresponds to information derived and extracted based on various information present in the content. For example, a user of the electronic device (101) can select specific content among content including at least one image, video, or text displayed on the display (161). The processor (120) of the electronic device (101) can identify contextual information based on the specific content selected by the user.

[0126] According to one embodiment, the processor (120) can identify association information between context information, an application within the electronic device, and a function of the electronic device (1100). The processor (120) can identify the association information based on at least one of the context information of the content, the user's usage history of the electronic device (101), and the usage history of an application installed within the electronic device (101).

[0127] According to one embodiment, the processor (120) can determine at least one recommendation application or recommendation function based on analyzed context information and identified association information (1200). The processor (120) can determine a recommendation application or recommendation function based on the context information and association information of the content using an electronic device usage model learned by a user of the electronic device (101).

[0128] According to one embodiment, the processor (120) may display an icon corresponding to a recommendation application or recommendation function along with the content (1300). If there are multiple recommendation applications or recommendation functions, the processor (120) may display an icon corresponding to multiple recommendation applications or recommendation functions along with the content.

[0129] FIG. 7 illustrates an electronic device that displays an icon corresponding to a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0130] FIG. 8 illustrates an electronic device that displays a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0131] Referring to FIG. 7, it can be seen that a plurality of icons corresponding to recommended applications or recommendation functions are displayed along with the content displayed on the lock screen of the display (161).

[0132] According to one embodiment, the electronic device (101) can perform the function of providing various information through a lock screen on a display (161). For example, the electronic device (101) can provide real-time information to the user by displaying information related to a running application, information related to music being played, progress information of a timer application, and real-time delivery information of a delivery application on the lock screen. This real-time information may be displayed on a notification bar located at the bottom or top of the lock screen of the electronic device display (161), but the location of the notification bar is not limited thereto.

[0133] Referring to FIG. 7-A, the notification bar (162) displayed at the bottom of the display (161) displays information that voice recording is in progress, and the user can see through this that voice recording is being performed by the electronic device (101).

[0134] According to one embodiment, a user may input user input to receive recommendations for related applications or functions based on voice recording content (162) provided in a notification bar. User input to receive recommendations for applications or functions related to voice recording content (162) provided in a notification bar may include a predetermined gesture input. For example, a user who wishes to receive recommendations for applications or functions related to voice recording content (162) provided in a notification bar may input touching the content as a predetermined gesture. The touch input by which the user selects the content may include inputs such as touching the content for a long period or touching it multiple times in succession. However, even in the absence of user input, the electronic device (101) may determine and provide related applications or functions based on voice recording content (162) provided in a notification bar to the user.

[0135] According to one embodiment, the electronic device (101) can analyze contextual information of a voice recording based on voice recording content (162) provided in a notification bar, for example. The contextual information of the voice recording may include the content of the recording, the language in which the recording is made, etc. The electronic device (101) can identify association information between the contextual information of the voice recording content (162) (e.g., content of the recording, language in which the recording is made) and applications and functions existing within the electronic device (101). The association information may include the language primarily used by the user, additional information related to the recording content (e.g., search history related to the recording content, images stored related to the recording content, etc.), and the user's application history (e.g., application history containing the recording content). The electronic device (101) can determine recommended applications or recommendation functions based on the contextual information and association information of the voice recording content (162).

[0136] According to one embodiment, the electronic device (101) may display an icon (166) corresponding to a recommendation application or recommendation function associated with the voice recording content (162) together with the voice recording content (162) (7-B).

[0137] According to one embodiment, when an icon (166) corresponding to a recommendation application or recommendation function is displayed together with voice recording content (162), the voice recording content (162) may be displayed on the display (161) in a smaller size than when the icon (166) corresponding to a recommendation application or recommendation function is displayed together (7-B).

[0138] According to one embodiment, an icon (166) corresponding to a recommended application or recommendation function may be displayed at the top of a notification bar containing content information as shown in 7-B, and an icon (7-B-1) corresponding to a phone application and an icon (7-B-2) corresponding to a camera application located on the side of the notification bar may be changed and displayed as an icon corresponding to a recommended application or recommendation function.

[0139] According to one embodiment, a user can drag an icon corresponding to a recommended application or recommended function displayed at the top of the notification bar to display a phone function icon and a camera function icon located on the side of the notification bar.

[0140] According to one embodiment, when a user inputs that a user selects one of the icons (166) corresponding to a recommended application or a recommended function, the electronic device (101) may display a card with greatly enlarged content-related information, etc., included in the selected application or recommended function on a part of the display (161).

[0141] According to one embodiment, for example, if the recording language included in the voice recording content (162) is Spanish and the language primarily used by the user of the electronic device (101) is English, a function that automatically translates the voice recording content currently being recorded in Spanish into English may be recommended as a recommended function, and a translation function icon may be displayed around the voice recording content (162) along with an icon (166) corresponding to a recommended application or recommended function. When the user selects the translation function icon, the electronic device (101) may display a card (7-C-1) that greatly enlarges the relevant information included in the translation function on a part of the display (7-C).

[0142] According to one embodiment, a card displayed around the content (162) may include a card containing information obtained from an external electronic device or an external network, rather than a card related to a recommendation application or recommendation function determined based on the content (162). For example, a card containing advertising information obtained from an external electronic device may be included.

[0143] According to one embodiment, the electronic device (101) can execute the selected recommended application or recommended function when the user selects a card related to a recommended application or recommended function that is enlarged and displayed on a portion of the display (161). The action of the user selecting the enlarged card related to the recommended application or recommended function may include long-touching or swiping the card. However, it is not limited thereto.

[0144] Referring to FIG. 8, it can be seen that a plurality of recommended applications or a plurality of recommended functions are displayed along with the content displayed on the lock screen of the display (161).

[0145] According to one embodiment, the current electronic device (101) may indicate that voice recording is in progress through a notification bar located at the bottom of the display (161) (8-A). The user may input user input to receive recommendations for related applications or functions based on the voice recording content (162) provided in the notification bar. However, even in the absence of user input, related applications or functions may be determined and provided to the user based on the voice recording content (162) provided in the notification bar.

[0146] According to one embodiment, an electronic device (101) can analyze contextual information of a voice recording based on voice recording content (162) provided in a notification bar. For example, contextual information of a voice recording may include the content of the recording, the language in which the recording is made, etc. The electronic device (101) can identify association information between the contextual information of the voice recording content (162) (e.g., content of the recording, the language in which the recording is made) and applications and functions existing within the electronic device (101). Association information may include the language primarily used by the user, additional information related to the recording content (e.g., search history related to the recording content, images stored related to the recording content, etc.), and the user's application history (e.g., application history containing the recording content). The electronic device (101) can determine recommended applications or recommended functions based on the contextual information and association information of the voice recording content (162).

[0147] According to one embodiment, the electronic device (101) may display an icon (166) corresponding to a plurality of recommendation applications or a plurality of recommendation functions associated with the voice recording content (162) together with the voice recording content (162) (8-B).

[0148] According to one embodiment, a user may make a touch input on an all-view icon (169) to view multiple recommendation applications or content-related information included in recommendation functions at once. When the electronic device (101) receives user input selecting the all-view icon (169) from the user, it may greatly enlarge and list and display multiple recommendation applications and content-related information included in multiple recommendation functions related to the content (162) on a part of the display (161).

[0149] For example, if the recording language included in the voice recording content (162) is Spanish and the language primarily used by the user of the electronic device (101) is English, a function to automatically translate the voice recording content currently being recorded in Spanish into English may be recommended as a recommended function, and a translation function icon may be displayed together with the voice recording content (162). An automatic recording function icon for automatically recording the voice recording content may be recommended as a recommended function, and a recording function icon may be displayed together with the voice recording content (162). In addition, if an image related to the voice recording content is stored in the electronic device (101), an application that displays the image may be determined as a recommended application and displayed together with the voice recording content (162). When the electronic device (101) receives user input from the user to select the full view icon (169), the card related to the recording function (8-C-1), the card related to the translation function (8-C-2), and the card related to the image display application (8-C-3) may be greatly enlarged and displayed in a list on a part of the display (161) (8-C).

[0150] FIGS. 9a and 9b illustrate an electronic device that displays an icon corresponding to a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0151] Referring to FIG. 9a, it can be seen that a plurality of icons corresponding to recommended applications or recommendation functions are displayed along with the content displayed on the lock screen of the display (161).

[0152] According to one embodiment, an electronic device (101) can provide real-time information to a user by displaying content (162) (e.g., information related to music being played) through a notification bar displayed on a lock screen on a display (161). The electronic device (101) can analyze contextual information based on the music playback content (162) provided on the notification bar and identify association information between applications and functions existing within the electronic device (101). The electronic device (101) can determine recommended applications or recommended functions based on the analyzed contextual information and association information. According to one embodiment, the electronic device (101) can display an icon (166) corresponding to the recommended application or recommended function related to the music playback content (162) together with the music playback content (162) (9a-A).

[0153] According to one embodiment, when a user selects at least one of the icons (166) corresponding to a recommended application or a recommended function, the electronic device (101) can display and execute the selected recommended application or recommended function on the display (161). The electronic device (101) can execute the recommended application selected by the user and, at the same time, display icons (166') corresponding to the recommended application or recommended function, excluding the icon corresponding to the selected recommended application or recommended function, around the application being executed (9a-B).

[0154] Referring to FIG. 9b, it can be seen that a plurality of icons corresponding to recommended applications or recommendation functions are displayed along with the content displayed on the lock screen of the display (161).

[0155] According to one embodiment, the electronic device (101) displays content (162) through a notification bar displayed on a lock screen on a display (161) (e.g., information related to music being played), and can determine a recommended application or recommendation function based on the content (162).

[0156] According to one embodiment, the electronic device (101) does not display an icon corresponding to a recommendation application or recommendation function associated with the music playback content (162) together with the music playback content (162) (9b-A), and when a user inputs to select the content (162), the recommendation function associated with the content is displayed on the display (161), and at the same time, an icon (166) corresponding to a recommendation application or recommendation function, excluding the icon corresponding to the recommendation function that is executed, can be displayed around the function that is executed (9b-B).

[0157] FIGS. 10 and 11 illustrate an electronic device that displays a recommendation application or recommendation function along with content according to an embodiment of the present invention.

[0158] Referring to FIGS. 10 and 11, it can be seen that a card related to a recommendation application or a plurality of recommendation functions is displayed along with the content displayed on the display (161) of the electronic device (101).

[0159] For example, if the content (162) displayed on the display (161) of the electronic device (101) is a notification regarding a job posting for a specific company (10-A), the electronic device (101) may determine a recommended application or recommendation function based on the job posting for the specific company provided through the notification. The electronic device (101) may display a card (10-B-1) related to the determined recommended application or recommendation function around the content (162) containing the job posting for the specific company (10-B).

[0160] As another example, if the content (162) displayed on the display (161) of the electronic device (101) is a message received from a specific person (11-A), the electronic device (101) can determine a recommended application or recommendation function by analyzing the contextual information included in the message notification. The electronic device (101) can display a card (11-B-1) related to the determined recommended application or recommendation function around the content (162) containing the message notification (11-B).

[0161] FIGS. 12 to 15 illustrate examples of various content-specific recommendation applications or recommendation functions displayed on an electronic device according to an embodiment of the present invention.

[0162] Referring to FIG. 12, examples of recommended applications or recommendation features can be seen when the content displayed on the display (161) within the electronic device (101) is the content displayed on the notification bar on the lock screen.

[0163] According to one embodiment, the content currently provided to the electronic device (101) includes voice recording content (162) provided in a notification bar displayed on the lock screen. The user may input user input to receive recommendations for associated applications or functions based on the voice recording content (162) provided in the notification bar. However, even in the absence of user input, associated applications or functions may be determined and provided to the user based on the voice recording content (162) provided in the notification bar.

[0164] According to one embodiment, an electronic device (101) can analyze contextual information of voice recording content (162) displayed on a notification bar and identify association information between applications and functions existing within the electronic device (101). For example, the contextual information of the voice recording may include the content of the recording, the language in which the recording is made, etc. The electronic device (101) can identify association information between the contextual information of the voice recording content (162) (e.g., content of the recording, the language in which the recording is made) and applications and functions existing within the electronic device (101). The association information may include the language primarily used by the user, additional information related to the recording content (e.g., search history related to the recording content, images stored related to the recording content, etc.), and the user's application history (e.g., application history containing the recording content).

[0165] According to one embodiment, the electronic device (101) determines a recommended application or recommendation function based on context information and association information of voice recording content (162) of a notification bar, and can display an icon (166) corresponding to the recommended application or recommendation function (164) together with the content (162) (12-A).

[0166] For example, if the content being recorded is included as an example of contextual information of the voice recording content, and the electronic device function primarily used by the user is a summary function as an example of related information, a function to automatically summarize the recorded content in English may be included in the recommendation function (12-B), and an icon (12-A-1) corresponding to the automatic summary function may be displayed together with the content (162).

[0167] For example, as an example of contextual information of voice recording content, the recording language is Spanish, and as an example of related information, the language primarily used by the user is English, a feature that automatically translates content recorded in Spanish into English may be included in the recommendation feature (12-C), and an icon (12-A-2) corresponding to the automatic translation feature may be displayed together with the content (162).

[0168] For example, if the content being recorded is included as an example of context information for voice recording content, and an image related to the recording content is stored in the user's electronic device (101) as an example of related information, an image app that displays the image related to the recording content may be included in the recommendation application (12-D), and an icon (12-A-3) corresponding to the image app may be displayed together with the content (162).

[0169] According to one embodiment, the content provided to the electronic device (101) may include music playback content displayed in a notification bar. For example, a user may input user input to receive recommendations for related applications or functions based on the music playback content displayed in the notification bar. The electronic device (101) may analyze contextual information of the music playback content displayed in the notification bar and identify association information between applications and functions existing within the electronic device (101). The contextual information of the music playback content may include song titles, song lyrics, singers, etc. The electronic device (101) may identify association information between the contextual information of the music playback content (e.g., song titles, song lyrics, singers) and applications and functions existing within the electronic device (101). For example, the association information may include the language primarily used by the user, additional information related to the song (e.g., records related to song lyrics, records related to song titles, etc.), and the user's application history (e.g., singer search history, song playlists, etc.). The electronic device (101) can determine a recommendation application or recommendation function, such as an automatic song lyric translation function, a song-related information search result and a saved playlist recommendation function, based on context information and association information of music playback content displayed on a notification bar, and can display an icon corresponding to the recommendation application or an icon corresponding to the recommendation function along with the music playback content.

[0170] The content provided to the electronic device (101) may include alarm and timer content displayed in a notification bar. The user may input user input to receive recommendations for related applications or functions based on the alarm and timer content displayed in the notification bar. The electronic device (101) may analyze contextual information of the alarm and timer content displayed in the notification bar and identify association information between applications and functions existing within the electronic device (101). For example, the contextual information of the alarm and timer content may include the alarm setting time, alarm memo content, timer remaining time, etc. The electronic device (101) may identify association information between the contextual information of the alarm and timer content (e.g., alarm setting time, alarm memo content, timer remaining time) and applications and functions existing within the electronic device (101). The association information may include applications containing alarm memos, additional information related to alarm memos (e.g., schedule list containing alarm memos, images related to alarm memos, notes containing alarm memos, etc.), and the user's application history (e.g., past alarm setting records, etc.). The electronic device (101) can determine recommended applications or recommended functions, such as modifying calendar schedules, modifying note content, or providing internet search results, based on contextual information and association information of alarm and timer content, and can display an icon corresponding to the recommended application or an icon corresponding to the recommended function along with music playback content.

[0171] Referring to FIG. 13, examples of recommended applications or recommendation functions can be seen when the content displayed on the display (161) within the electronic device (101) is a captured image of an image displayed on the display (161).

[0172] According to one embodiment, the content currently provided to the electronic device (101) includes a captured image of an image displayed on the display (161). The user may input user input to receive recommendations for associated applications or functions based on the content (162) which includes the captured image of an image displayed on the display (161). However, even in the absence of user input, the associated applications or functions may be determined and provided to the user based on the captured image of an image displayed on the display (161).

[0173] According to one embodiment, an electronic device (101) can analyze contextual information of a captured image (e.g., some images, text, logos, etc. included in the image) and identify association information between applications and functions existing within the electronic device (101). For example, if the overall information included in the image is about a new employee recruitment notice, the contextual information of the captured image may include information about the recruiting company, the number of recruits, and the recruitment interview. The electronic device (101) can identify association information between the contextual information of the captured image (e.g., information about the number of recruits and the recruitment interview) and applications and functions existing within the electronic device (101). The association information may include additional information related to the content of the recruitment notice (e.g., a file containing content related to the recruiting company, a note containing information related to the recruiting company, etc.), the user's application history (e.g., a record of messages related to the content of the recruitment notice, an app search record related to the recruiting company), etc.

[0174] According to one embodiment, the electronic device (101) determines a recommended application or recommendation function based on context information and association information of a captured image, and can display an icon corresponding to the recommended application or recommendation function together with content (162) including the captured image (13-A).

[0175] For example, if job posting information is included as an example of contextual information of a cap image and content related to the job posting is included in the content of a message recently sent by the user as an example of related information, the message app may be included in the recommendation application (13-B), and a card (13-A-1) corresponding to the message app may be displayed along with the content (162). Additionally, if there is a job posting-related outgoing message that the user has not completed in the message conversation window related to the job posting, a function to send that message may be included in the recommendation function.

[0176] For example, if a job interview date is included as an example of contextual information of a captured image, and the job interview date of the company is included in the user's calendar app as an example of related information, the calendar app may be included in the recommended application (13-C), and a card (13-A-2) corresponding to the calendar app may be displayed along with content (162) containing the captured image. Additionally, if the job interview date stored in the calendar app differs from the job interview date identified from the captured image, a function to change the stored job interview date may be recommended.

[0177] For example, if job posting information is included as an example of contextual information of a captured image and interview materials, past interview questions, and a portfolio related to the company's recruitment are included as an example of related information, a file application for executing a file containing such related information may be included in the recommendation application (13-D), and a card (13-A-3) corresponding to the file application may be displayed together with content (162) containing the captured image.

[0178] For example, if job posting information is included as an example of contextual information of a captured image and materials related to a company's recruitment briefing and recruitment list information are included as an example of related information, a note application (13-E) for executing a note containing such information may be included in the recommendation application, and an icon corresponding to the note application may be displayed together with the content containing the captured image.

[0179] Referring to FIG. 14, examples of recommendation applications or recommendation functions can be seen when the content (162) displayed on the display (161) within the electronic device (101) is a specific document displayed on the display (161).

[0180] According to one embodiment, if the content (162) provided to the electronic device (101) includes a specific document displayed on the display (161), the user may input user input to receive recommendations for associated applications or functions based on the specific document displayed on the display (161). However, even in the absence of user input, an associated application or function may be determined and provided to the user based on the specific document displayed on the display (161).

[0181] According to one embodiment, an electronic device (101) can analyze contextual information of a specific document and identify association information between applications and functions existing within the electronic device (101). For example, if the document is an annual report of a specific company, the contextual information of the specific document may include the language in which the report was written, the specific content of the report, etc. The electronic device (101) can identify association information between the contextual information of the specific company's annual report (e.g., the language in which the report was written, the specific content of the report) and applications and functions existing within the electronic device (101). The association information may include additional information related to the content of the associated report, the user's application history (e.g., the language the user primarily uses, the applications and functions the user primarily uses), etc.

[0182] According to one embodiment, the electronic device (101) can determine a recommended application or recommendation function based on contextual information and association information of a specific company's annual report, and display an icon corresponding to the recommended application or recommendation function along with a specific document corresponding to the content (162) (14-A).

[0183] For example, as an example of related information, if the function primarily used by the user is the summarizing function, the summarizing function (14-B) can be included in the recommendation function. That is, by analyzing the contents of a specific company's annual report, a function that automatically summarizes the document content can be provided, and a card (14-A-1) corresponding to the summarizing function can be displayed together with content (162) containing a specific document.

[0184] For example, if contextual information of a specific company's annual report includes the fact that the language used is English, and related information includes the fact that the language primarily used by the user is Korean, then a translation function (14-C) may be included in the recommendation function. That is, a translation function that automatically translates a specific company's annual report written in English into Korean may be provided as a recommendation function, and a card (14-A-2) corresponding to the translation function may be displayed together with content (162) containing a specific document.

[0185] For example, as an example of related information, if the function primarily used by the user is the correction function, the correction function may be included in the recommendation function (14-D), and a card (14-A-3) corresponding to the correction function may be displayed together with content (162) containing a specific document. That is, by analyzing the writing style of a specific company's annual report, a function to change to the writing style primarily used by the user may be provided as a recommendation function. Additionally, by analyzing typos present in a specific company's annual report, a function to correct typos may be provided as a recommendation function.

[0186] Referring to FIG. 15, examples of recommended applications or recommended features can be seen when the content (162) displayed on the display (161) within the electronic device (101) is a notification window displayed on the display (161). The notification window content (162) may include notifications about the remaining battery and various notification alarms.

[0187] According to one embodiment, the content (162) currently provided to the electronic device (101) refers to a notification display including a notification about the remaining battery displayed on the display (161). The user may input user input to receive recommendations for associated applications or functions based on the notification (162) about the remaining battery. An associated application or function may be determined and provided to the user.

[0188] According to one embodiment, the electronic device (101) can analyze context information of a notification regarding the battery and identify association information between applications and functions existing within the electronic device (101). The context information of the notification regarding the battery may include information such as the remaining battery amount and the time expected until the electronic device (101) is turned off. The electronic device (101) can identify association information between the context information of the notification regarding the battery (e.g., remaining battery amount and the time expected until the electronic device (101) is turned off) and applications and functions existing within the electronic device (101). The association information may include the user's application history (e.g., information on the last viewed application, incomplete outgoing messages, and incomplete posts).

[0189] According to one embodiment, the electronic device (101) determines a recommended application or recommended function based on context information and association information of a notification about a battery, and can display an icon corresponding to the recommended application or recommended function along with the notification about the battery (15-A).

[0190] For example, if the remaining usage time of the electronic device (101) is included as an example of context information for a battery notification, a power saving mode function may be included in the recommendation function (15-B), and a card (15-A-1) corresponding to the power saving mode function may be displayed together with content (162) containing a battery notification. That is, the user can check the remaining usage time based on the remaining battery amount of the electronic device (101) and switch the electronic device (101) to power saving mode.

[0191] For example, if an example of related information includes the creation of an incomplete post, an app (15-C) for creating an incomplete post may be included in the recommended applications, and a card (15-A-2) corresponding to the app for creating an incomplete post may be displayed along with content (162) containing a notification about the battery. Additionally, a post creation function for creating an incomplete post may be provided as a recommendation function.

[0192] For example, if the remaining usage time of the electronic device (101) is included as an example of contextual information for a battery notification, a recording function (15-D) for recording the content included in the last accessed application or post may be included in the recommendation function, and a card (15-A-3) corresponding to the recording function may be displayed together with content (162) containing the battery notification. That is, the user can check the remaining usage time based on the remaining battery amount of the electronic device (101) and execute a recording function to record the content included in the last accessed application or post.

[0193] For example, if an incomplete outgoing message is included as an example of associated information, a message app (15-E) for sending the incomplete outgoing message may be included in the recommended application, and a card (15-A-4) corresponding to the message app may be displayed along with content (162) containing a notification about the battery. Additionally, a message sending function for sending the incomplete outgoing message may be provided as a recommendation function.

[0194] FIG. 16 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to determine a recommendation application or recommendation function based on content displayed on a display and to execute a recommendation application. (See FIG. 17)

[0195] In the following embodiments, each operation of FIG. 17 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation of FIG. 17 may be changed, and at least two operations may be performed in parallel.

[0196] According to one embodiment, the 2000 to 2500 operations of FIG. 17 may be understood to be performed in a processor (e.g., processor (120) of FIG. 5) of an electronic device (e.g., electronic device (101) of FIG. 5).

[0197] According to one embodiment, before performing the 2000 operation, the electronic device (101) may perform an operation to establish communication (e.g., pairing) with an external electronic device.

[0198] According to one embodiment, the processor (120) of the electronic device (101) (or the electronic device (101)) can analyze contextual information of content displayed on the display (161) (2000). Contextual information corresponds to information derived and extracted based on various information present in the content. For example, a user of the electronic device (101) can select specific content among content including at least one image, video, or text displayed on the display (161). The processor (120) of the electronic device (101) can identify contextual information based on the specific content selected by the user.

[0199] According to one embodiment, the processor (120) can identify association information between context information, an application within the electronic device, and a function of the electronic device (2100). The processor (120) can identify the association information based on at least one of the context information of the content, the user's usage history of the electronic device (101), and the usage history of an application installed within the electronic device (101).

[0200] According to one embodiment, the processor (120) can determine at least one recommendation application or recommendation function based on analyzed context information and identified association information (2200). The processor (120) can determine a recommendation application or recommendation function based on the context information and association information of the content using an electronic device usage model learned by a user of the electronic device (101).

[0201] According to one embodiment, the processor (120) may display an icon corresponding to at least one recommendation application or recommendation function along with the content (2300). If there are multiple recommendation applications or recommendation functions, the processor (120) may display icons corresponding to multiple recommendation applications or recommendation functions along with the content.

[0202] According to one embodiment, the processor (120) can detect whether it receives user input from a user selecting an icon corresponding to at least one recommendation application (2400).

[0203] According to one embodiment, when the processor (120) receives user input selecting an icon corresponding to at least one recommended application from the user (e.g., 2400), the recommended application selected by the user can be enlarged and displayed on a part of the display (161) (2500).

[0204] FIG. 17 illustrates an electronic device that executes a recommendation application determined based on content displayed on a display according to an embodiment of the present invention.

[0205] Referring to FIG. 17, examples of recommended applications or recommendation functions can be seen when the content (162) displayed on the display (161) within the electronic device (101) is a captured image of an image displayed on the display (161).

[0206] According to one embodiment, the content (162) currently provided to the electronic device (101) includes a captured image of an image displayed on the display (161) (17-A).

[0207] According to one embodiment, an electronic device (101) can analyze contextual information of a captured image (e.g., some images, text, logos, etc. included in the image) and identify association information between applications and functions existing within the electronic device (101). For example, if the overall information included in the image is about a new employee recruitment announcement, the contextual information of the captured image may include information about the recruiting company, the number of recruits, and recruitment interviews. The association information may include additional information related to the content of the recruitment announcement (e.g., a file containing content related to the recruiting company, a note containing information related to the recruiting company, etc.), and the user's application history (e.g., a record of messages related to the content of the recruitment announcement, a search record related to the recruiting company).

[0208] According to one embodiment, the electronic device (101) determines recommended applications and recommended functions based on context information of a captured image and identified association information, and can display an icon or card (17-B-1) corresponding to the recommended applications and recommended functions around the captured image (17-B). When a user selects an icon or card corresponding to a recommended application, the electronic device (101) can enlarge and display the selected recommended application on a part of the display.

[0209] FIG. 18 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to determine a recommended application or a recommendation function based on content displayed on a display and to execute the recommendation function. (See FIG. 19)

[0210] In the following embodiments, each operation of FIG. 18 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation of FIG. 18 may be changed, and at least two operations may be performed in parallel.

[0211] According to one embodiment, the 3000 to 3500 operations of FIG. 18 may be understood to be performed in a processor (e.g., processor (120) of FIG. 5) of an electronic device (e.g., electronic device (101) of FIG. 5).

[0212] According to one embodiment, before performing the 3000 operation, the electronic device (101) may perform an operation to establish communication (e.g., pairing) with an external electronic device.

[0213] According to one embodiment, the processor (120) of the electronic device (101) (or the electronic device (101)) can analyze contextual information of content displayed on the display (161) (3000). Contextual information corresponds to information derived and extracted based on various information present in the content. For example, a user of the electronic device (101) can select specific content among content including at least one image, video, or text displayed on the display (161). The processor (120) of the electronic device (101) can identify contextual information based on the specific content selected by the user.

[0214] According to one embodiment, the processor (120) can identify association information between context information, an application within the electronic device, and a function of the electronic device (3100). The processor (120) can identify the association information based on at least one of the context information of the content, the user's usage history of the electronic device (101), and the usage history of an application installed within the electronic device (101).

[0215] According to one embodiment, the processor (120) can determine at least one recommendation application or recommendation function based on analyzed contextual information and identified association information (3200). The processor (120) can determine a recommendation application or recommendation function based on the contextual information and association information of the content using an electronic device usage model learned by a user of the electronic device (101).

[0216] According to one embodiment, the processor (120) may display an icon corresponding to a recommendation application recommendation function along with the content (3300). If there are multiple recommendation applications or recommendation functions, the processor (120) may display an icon corresponding to multiple recommendation applications or recommendation functions along with the content.

[0217] According to one embodiment, the processor (120) can detect whether it receives user input from a user selecting an icon corresponding to at least one recommendation function (3400).

[0218] According to one embodiment, when the processor (120) receives user input selecting an icon corresponding to at least one recommended function from the user (e.g., 3400), the function selected by the user can be displayed on the display (161) and executed (3500).

[0219] FIG. 19 illustrates an electronic device that performs a recommendation function determined based on content displayed on a display according to one embodiment of the present invention.

[0220] Referring to FIG. 19, examples of recommendation applications or recommendation functions can be seen when the content (162) displayed on the display (161) within the electronic device (101) is a specific document displayed on the display (161).

[0221] According to one embodiment, the content (162) currently provided to the electronic device (101) includes a specific document displayed on the display (161) (19-A).

[0222] According to one embodiment, the electronic device (101) can analyze contextual information of a specific document and identify association information between applications and functions existing within the electronic device (101). For example, if the specific document is an annual report of a specific company, the contextual information of the annual report (162) may include the language in which the report was written, information about the specific content of the report, etc. The association information may include additional information related to the content of the annual report (162), the user's application history (e.g., the language the user primarily uses, the applications and functions the user primarily uses), etc.

[0223] According to one embodiment, the electronic device (101) determines a recommended application and a recommended function based on context information of a specific document and identified association information, and can display an icon or card (19-B-1) corresponding to the recommended application and the recommended function around content including the specific document (19-B). When a user selects an icon or card corresponding to the recommended function, the electronic device (101) can enlarge and display the selected recommended function on a part of the display.

[0224] FIG. 20 is a flowchart illustrating a method for determining a recommended application or recommendation function based on content displayed on a lock screen by an electronic device according to an embodiment of the present invention.

[0225] In the following embodiments, each operation of FIG. 20 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation of FIG. 20 may be changed, and at least two operations may be performed in parallel.

[0226] According to one embodiment, operations 4000 to 4700 of FIG. 20 can be understood as being performed in a processor (e.g., processor (120) of FIG. 5) of an electronic device (e.g., electronic device (101) of FIG. 5).

[0227] According to one embodiment, before performing the 4000 operation, the electronic device (101) may perform an operation to establish communication (e.g., pairing) with an external electronic device.

[0228] According to one embodiment, the processor (120) of the electronic device (101) (or the electronic device (101)) can analyze contextual information of content displayed on the display (161) (4000). Contextual information corresponds to information derived and extracted based on various information present in the content. For example, a user of the electronic device (101) can select specific content among content including at least one image, video, or text displayed on the display (161). The processor (120) of the electronic device (101) can identify contextual information based on the specific content selected by the user.

[0229] According to one embodiment, the processor (120) can identify association information between context information, an application within the electronic device, and a function of the electronic device (4100). The processor (120) can identify association information based on at least one of the context information of the content, the user's usage history of the electronic device (101), and the usage history of an application installed within the electronic device (101).

[0230] According to one embodiment, the processor (120) can determine at least one recommendation application or recommendation function based on analyzed context information and identified association information (4200). The processor (120) can determine a recommendation application or recommendation function based on the context information and association information of the content using an electronic device usage model learned by a user of the electronic device (101).

[0231] According to one embodiment, the processor (120) can display an icon corresponding to a recommendation application recommendation function along with the content (4300). If there are multiple recommendation applications or recommendation functions, the processor (120) can display an icon corresponding to multiple recommendation applications or recommendation functions along with the content.

[0232] Content displayed on the lock screen may include content displayed on the notification bar (e.g., timer, alarm, song playback bar, etc.), lock screen capture images, and notification windows (e.g., battery level notification, various application notifications).

[0233] According to one embodiment, the processor (120) can detect whether it receives user input from a user selecting an icon corresponding to at least one recommendation application or recommendation function (4400).

[0234] According to one embodiment, when the processor (120) receives user input from the user on an icon corresponding to a recommendation application or recommendation function displayed along with content on the lock screen (e.g., 4400), it may provide a UI (user interface) that requires user authentication on the lock screen. If the information included in the recommendation application or recommendation function includes information requiring user security, the processor (120) may provide a user authentication UI to obtain user authentication from the user before executing the recommendation application or recommendation function (4500).

[0235] According to one embodiment, the processor (120) can detect whether user authentication is completed through the user authentication UI (4600).

[0236] According to one embodiment, when user authentication is completed through the user authentication UI (e.g., 4600), the processor (120) can enlarge and display a recommended application selected by the user on a part of the display (161) or display and execute a recommended function selected by the user on a part of the display (161) (4700). The user may provide personal information (e.g., fingerprint information, password information, face information, biometric information) to the user authentication UI for user authentication.

[0237] FIG. 21 is a flowchart illustrating a method for an electronic device according to an embodiment of the present invention to display a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function in response to a user request. (See FIG. 22 and FIG. 23)

[0238] In the following embodiments, each operation of FIG. 21 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation of FIG. 21 may be changed, and at least two operations may be performed in parallel.

[0239] According to one embodiment, operations 5000 to 5800 of FIG. 21 may be understood to be performed in a processor (e.g., processor (120) of FIG. 5) of an electronic device (e.g., electronic device (101) of FIG. 5).

[0240] According to one embodiment, before performing the 5000 operation, the electronic device (101) may perform an operation to establish communication (e.g., pairing) with an external electronic device.

[0241] According to one embodiment, the processor (120) of the electronic device (101) (or the electronic device (101)) can analyze first context information of first content displayed on the display (161) (5000). The first context information corresponds to information derived and extracted based on various information present in the first content. For example, a user of the electronic device (101) can select a specific content (e.g., first content) among content including at least one image, video, or text displayed on the display (161). The processor (120) of the electronic device (101) can identify the first context information based on the specific content (e.g., first content) selected by the user.

[0242] According to one embodiment, the processor (120) can identify first association information between the first context information, the application within the electronic device, and the function of the electronic device (5100). The processor (120) can identify first association information based on at least one of the first context information of the content, the user's usage history of the electronic device (101), and the usage history of the application installed within the electronic device (101).

[0243] According to one embodiment, the processor (120) can determine at least one first recommendation application or first recommendation function based on analyzed first context information and identified first association information (5200). The processor (120) can determine the first recommendation application or first recommendation function based on the first context information and first association information of the content using an electronic device usage model learned by a user of the electronic device (101).

[0244] According to one embodiment, the processor (120) may display an icon corresponding to a first recommendation application or a first recommendation function together with the first content (5300). If there are multiple first recommendation applications or first recommendation functions, the processor (120) may display icons corresponding to multiple first recommendation applications or first recommendation functions together with the first content.

[0245] According to one embodiment, the processor (120) can detect whether a request to display the second content from the first content is received from the user (5400).

[0246] According to one embodiment, the processor (120) may, in response to receiving a request from a user to display a second content from a first content (e.g., 5400), change the first content displayed on the display (161) to the second content and analyze the second context information of the second content (5500). The second context information corresponds to information derived and extracted based on various information present in the second content.

[0247] According to one embodiment, the processor (120) can identify second association information between second context information, an application within the electronic device, and a function of the electronic device (5600). The processor (120) can identify second association information based on at least one of second context information of the second content, the user's usage history of the electronic device (101), and the usage history of an application installed within the electronic device (101).

[0248] According to one embodiment, the processor (120) can determine at least one second recommendation application or second recommendation function based on the analyzed second context information and the identified second association information (5700). The processor (120) can determine the second recommendation application or second recommendation function based on the second context information and second association information of the second content using an electronic device usage model learned by the user of the electronic device (101).

[0249] According to one embodiment, the processor (120) may display an icon corresponding to a second recommendation application or a second recommendation function along with the second content (5800). If there are multiple second recommendation applications or second recommendation functions, the processor (120) may display icons corresponding to multiple second recommendation applications or second recommendation functions along with the second content.

[0250] FIG. 22 illustrates an electronic device that displays a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function according to a user request in accordance with an embodiment of the present invention.

[0251] Referring to 22, it can be seen that a first recommended application (e.g., YouTube app, internet app, Instagram app, etc.) or recommendation function is determined based on a first content (e.g., music playback content in a notification bar) (162) displayed within an electronic device (101), and an icon (166) corresponding to the first recommended application or recommendation function is displayed together with the first content (162) (22-A).

[0252] According to one embodiment, a user may provide a user input (167) that changes from the first content to the second content so that the display (161) displays the first content to the second content (e.g., music playback information in the notification bar) (162'). The user input for changing the content may include touch input (e.g., providing a long touch input, swiping up and down or left and right, etc.).

[0253] According to one embodiment, in response to a user input that changes from the first content to the second content, the second content (162') may be displayed on the display (161). Based on the second content (e.g., a timer in the notification bar) (162') displayed on the display (161), a second recommended application (e.g., a calendar app, a gallery app, etc.) or a second recommended function may be determined, and an icon (166') corresponding to the second recommended application or second recommended function may be displayed together with the second content (162') (22-B).

[0254] FIG. 23 illustrates an electronic device that displays a second recommendation application or a second recommendation function in a first recommendation application or a first recommendation function according to a user request in accordance with an embodiment of the present invention.

[0255] Referring to FIG. 23, it can be seen that a first recommended application (e.g., YouTube app, internet app, Instagram app, etc.) or a first recommended function (e.g., automatic lyrics translation, automatic lyrics recording, etc.) is determined based on a first content (e.g., music playback content in a notification bar) (162) displayed in an electronic device (101), and a plurality of cards (23-A-1, 23-A-2, 23-A-3) corresponding to the first recommended application or recommended function are displayed together with the first content (162) (23-A).

[0256] According to one embodiment, a user may provide user input to change from the first content to the second content so that the display (161) displays the first content (162) to the second content (e.g., voice recording in the notification bar) (162'). User input to change the content may include touch input (e.g., providing a long touch input, swiping up and down or left and right, etc.).

[0257] According to one embodiment, in response to a user providing input to change the first content (162) to the second content (162'), the second content (162') may be displayed on the display (161). Based on the second content (e.g., voice recording in the notification bar) (162') displayed on the display (161), a second recommended application (e.g., gallery app, etc.) or recommended function (recording summary function, automatic translation function, etc.) is determined, and a plurality of cards (23-C-1, 23-C-2, 23-C-3) corresponding to the second recommended application or recommended function may be displayed together with the second content (162') (23-C).

[0258] According to one embodiment, in response to a user providing user input to display a second recommended application or a second recommended function on a plurality of cards (23-A-1, 23-A-2, 23-A-3) corresponding to a first recommended application or a first recommended function, a second content (162') may be displayed on a display (161). Based on the second content (e.g., voice recording in a notification bar) (162') displayed on the display (161), a second recommended application (e.g., a gallery app, etc.) or a recommended function (e.g., a summary function during recording, an automatic translation function, etc.) is determined, and a plurality of cards (23-C-1, 23-C-2, 23-C-3) corresponding to the second recommended application or recommended function may be displayed together with the second content (162') (23-B, 23-C).

[0259] FIGS. 24a to 24c illustrate a case where a temporary notification is received while displaying a recommendation application or recommendation function determined based on content displayed on a display according to an embodiment of the present invention.

[0260] According to one embodiment, the content displayed on the lock screen of the display (161) may include music playback, timer progress, voice recording, etc., displayed via a notification bar. Such content corresponds to information about an application or function currently in progress on the electronic device (101). When a temporary notification, including a message or phone call, is received by the electronic device (101), a notification window containing the temporary notification may be displayed on the lock screen of the display (161). The temporary notification may be displayed on the notification bar present on the lock screen. However, it is not limited thereto.

[0261] Referring to FIG. 24a, according to one embodiment, voice recording content (162) is displayed through a notification bar displayed on a display (161) of an electronic device (101). The electronic device (101) may display a card (24a-1) corresponding to a recommended application or recommended function determined based on the content (162) including voice recording, together with the content (162).

[0262] Referring to FIG. 24b, according to one embodiment, when content (162) including voice recording is displayed on the notification bar, it can be seen that the electronic device (101) receives a temporary notification.

[0263] For example, referring to 24b-A, when the electronic device (101) receives a temporary notification while displaying content through the notification bar, information about the temporary notification (162' of 24b-A) may be displayed on the notification bar instead of the content. At this time, a card (24b-1) corresponding to a recommended application or recommendation function determined based on the content displayed through the notification bar may be displayed in the same way around the information about the temporary notification (162' of 24b-A).

[0264] For example, referring to 24b-B, when the electronic device (101) receives a temporary notification while displaying content through the notification bar, information about the temporary notification (162' of 24b-B) may be displayed on the notification bar instead of the content. At this time, a card (24b-1 of 24b-A) corresponding to a recommended application or recommendation function determined based on the content displayed through the notification bar may not be displayed while the information about the temporary notification (162' of 24b-B) is displayed.

[0265] For example, referring to 24b-C, when the electronic device (101) receives a temporary notification while displaying content through a notification bar, information about the temporary notification (162' in 24b-C) may be displayed at a location different from the notification bar. At this time, a card (24b-3) corresponding to a recommended application or recommendation function determined based on the content displayed through the notification bar may be displayed together with the content (162 in 24b-C).

[0266] Referring to FIG. 24c, according to one embodiment, when the information display for a temporary notification disappears, a card (24c-1) corresponding to a recommended application or recommendation function determined based on the content displayed through the notification bar may be displayed in the same way around the content (162) displayed on the notification bar.

[0267] FIGS. 25a to 25c illustrate a foldable electronic device that executes a recommendation application or recommendation function determined based on content displayed on a display according to an embodiment of the present invention.

[0268] Referring to FIGS. 25a through 25c, according to one embodiment, an electronic device (101) may include a foldable electronic device having a first housing, a second housing, and a third housing. The first housing may include a first display (161-1). The second housing may include a second display (161-2). The third housing may include a third display (161-3).

[0269] Referring to FIG. 25a, according to one embodiment, an electronic device (101) may determine a recommended application or recommendation function based on content (162) displayed on at least one of a first display (161-1), a second display (161-2), or a third display (161-3). The electronic device (101) may display an icon (166) corresponding to the recommended application or recommendation function determined based on the notification window content (162) displayed on at least one of a first display (161-1), a second display (161-2), or a third display (161-3) together with the notification window content (162).

[0270] Referring to FIG. 25b, according to one embodiment, a user may select an all-view icon (166 in FIG. 25a) to view recommended applications or recommended functions displayed together with notification window content (162) at once, and a plurality of cards (25b-1) corresponding to all recommended applications or recommended functions may be displayed together with the notification window content (162) according to user input. According to one embodiment, in the case of a foldable electronic device, a plurality of cards (25b-1) corresponding to recommended applications or recommended functions may be displayed without overlapping depending on the size of at least one display among the first display (161-1), the second display (161-2), or the third display (161-3) included in the electronic device.

[0271] Referring to FIG. 25c, according to one embodiment, a user may input an icon or a plurality of cards corresponding to a desired application or function among a plurality of recommended applications or recommended functions determined based on the content of a notification window. After the user inputs a touch to select an icon or a plurality of cards corresponding to an application or function, the user may execute the recommended application or recommended function on at least one of the first display (161-1), the second display (161-2), or the third display (161-3) by dragging it to the first display, the second display, or the third display.

[0272] FIGS. 26a and 26b illustrate that a recommended application or recommendation function determined based on content displayed on a display is displayed differently depending on the shape of a foldable electronic device according to an embodiment of the present invention.

[0273] Referring to FIG. 26a and 26b, according to one embodiment, an electronic device (101) may include a foldable electronic device having a first housing, a second housing, and a third housing. The first housing may include a first display (161-1), and the second housing may include a second display (161-2). The third housing may include a third display (161-3) and a fourth display (161-4). The third display (161-3) and the fourth display (161-4) may each be located on both sides of the third housing.

[0274] Referring to FIG. 26a, according to one embodiment, a plurality of cards (26a-1) corresponding to a recommendation application or recommendation function associated with content (162) may be displayed together with the content (162). According to one embodiment, in the case of a multi-foldable electronic device, a plurality of cards corresponding to a recommendation application or recommendation function may be displayed without overlapping (26a-1) depending on the size of at least one display among the first display (161-1), the second display (161-2), or the third display (161-3) included in the electronic device. However, it is not limited thereto.

[0275] Referring to FIG. 26b, the multi-foldable electronic device may include a folded form in which at least one of the first housing or the third housing is folded. For example, when the third housing is folded toward the second housing, the third display (161-3) contained in the third housing overlaps with the second display (161-2) contained in the second housing, and the fourth display (161-4) contained in the third housing and located on the opposite side of the third display (161-3) is shown to the user.

[0276] According to one embodiment, a plurality of cards corresponding to a recommendation application or recommendation function related to the content (162) may be displayed together with the content (162) on a display of a foldable electronic device in which at least one of the first housing or the third housing is folded. For example, when a plurality of cards (26b-1) corresponding to a recommendation application or recommendation function related to the content (162) are displayed on a fourth display (161-4) of a foldable electronic device in which the third housing is folded, the plurality of cards may be displayed in an overlapping form depending on the size of the fourth display (161-4). However, it is not limited thereto.

[0277] An electronic device according to one embodiment of the present invention may include a display, a memory for storing instructions, and at least one processor. When the instructions are executed by the at least one processor, the electronic device may analyze context information of content including at least one image, video, or text displayed on the display. When the instructions are executed by the at least one processor, the electronic device may identify association information between the context information, an application within the electronic device, and a function of the electronic device. When the instructions are executed by the at least one processor, the electronic device may determine at least one recommended application or recommended function based on the context information and the association information. When the instructions are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one recommended application or recommended function along with the content.

[0278] According to one embodiment of the present invention, the at least one recommendation application is a plurality of recommendation applications, and when the commands are executed by the at least one processor, the electronic device may receive user input selecting an icon corresponding to at least one recommendation application among the icons corresponding to the plurality of recommendation applications. When the commands are executed by the at least one processor, the electronic device may enlarge the selected recommendation application and display it on the display in response to the user input.

[0279] According to one embodiment of the present invention, the at least one recommendation function is a plurality of recommendation functions, and when the commands are executed by the at least one processor, the electronic device may receive user input selecting an icon corresponding to at least one recommendation function among the icons corresponding to the plurality of recommendation functions. When the commands are executed by the at least one processor, the electronic device may enlarge the selected recommendation function and display it on a part of the display in response to the user input.

[0280] The context information according to one embodiment of the present invention may be context information of content including at least one image, video, or text displayed on the lock screen when the electronic device displays the lock screen through the display.

[0281] When the commands according to one embodiment of the present invention are executed by the at least one processor, the electronic device may provide a user authentication UI on the lock screen in response to the reception of user input selecting an icon corresponding to the at least one recommendation application or recommendation function. When the commands are executed by the at least one processor, the selected recommendation application or recommendation function may be enlarged and displayed on a part of the display based on the completion of user authentication through the user authentication UI.

[0282] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device may receive a request from a user to generate a capture image including at least one image or video displayed on the display. The context information according to one embodiment of the present invention may be context information of content including the capture image.

[0283] The context information according to one embodiment of the present invention may be context information of content including at least one image or text included in a notification popup displayed on the display.

[0284] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device can identify the associated information based on at least one of the context information, the user's usage history of the electronic device, and the usage history of the application within the electronic device.

[0285] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device may determine the at least one recommendation application or recommendation function based on the electronic device usage model stored in the memory. The electronic device usage model according to one embodiment of the present invention may be learned to determine the at least one recommendation application or recommendation function based on the electronic device user information.

[0286] According to one embodiment of the present invention, the content is a first content, the context information is a first context information, the association information is a first association information, the recommendation application and the recommendation function are a first recommendation application and a first recommendation function, and the instructions, when executed by the at least one processor, enable the electronic device to analyze second context information of the second content in response to receiving a request from a user to display second content including at least one image, video, or text displayed on the display in the first content. When executed by the at least one processor, the instructions enable the electronic device to identify second association information between the second context information and an application within the electronic device and a function of the electronic device. When executed by the at least one processor, the instructions enable the electronic device to determine at least one second recommendation application and a second recommendation function based on the second context information and the second association information. When the above commands are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one second recommendation application or second recommendation function together with the second content.

[0287] An electronic device according to one embodiment of the present invention may include a display, a memory for storing instructions, and at least one processor. When the instructions are executed by the at least one processor, the electronic device may display a first notification bar containing information of a first application at a first position, a second icon corresponding to a second application at a second position, and a third application at a third position, through a lock screen on the display. When the instructions are executed by the at least one processor, the electronic device may receive a first user input regarding the first notification bar. When the instructions are executed by the at least one processor, the electronic device may determine at least one first recommendation application or first recommendation function by analyzing context information of the third application. When the instructions are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one first recommendation application or first recommendation function together with the first notification bar.

[0288] When the commands according to one embodiment of the present invention are executed by the at least one processor, the electronic device may receive a first swipe user input for the first notification bar. When the commands are executed by the at least one processor, the electronic device may display a second notification bar containing information of a fourth application at the third location in response to the first swipe user input. When the commands are executed by the at least one processor, the electronic device may determine at least one second recommendation application or second recommendation function by analyzing context information of the fourth application. When the commands are executed by the at least one processor, the electronic device may display an icon corresponding to the at least one second recommendation application or second recommendation function together with the second notification bar.

[0289] When the commands according to one embodiment of the present invention are executed by the at least one processor, the electronic device may receive a second user input regarding an icon corresponding to the at least one first recommendation application or first recommendation function. When the commands are executed by the at least one processor, the electronic device may delete an icon corresponding to the at least one first recommendation application or first recommendation function from around the first notification bar in response to the second user input. When the commands are executed by the at least one processor, the electronic device may enlarge and display the at least one first recommendation application or first recommendation function on a part of the display.

[0290] According to one embodiment of the present invention, the at least one first recommendation application or first recommendation function includes a first-1 recommendation application or first-1 recommendation function and a first-2 recommendation application or first-2 recommendation function, and when the instructions are executed by the at least one processor, the electronic device may display an icon corresponding to the first-2 recommendation application or first-2 recommendation function and an icon corresponding to the first-1 recommendation application or first-1 recommendation function together with the first notification bar.

[0291] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device may receive a second user swipe input on an icon corresponding to the at least one first recommendation application or first recommendation function. When the instructions are executed by the at least one processor, in response to the second user swipe input, the icon corresponding to the at least one second recommendation application or second recommendation function may be displayed at the location of the icon corresponding to the at least one first recommendation application or first recommendation function. When the instructions are executed by the at least one processor, a second notification bar containing information of a fourth application may be displayed at the third location through the display.

[0292] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device may display the first icon, the second icon, and the first notification bar in a reduced manner when displaying an icon corresponding to the at least one first recommendation application or first recommendation function.

[0293] When the commands according to one embodiment of the present invention are executed by the at least one processor, the electronic device can receive a notification from a fifth application. When the commands are executed by the at least one processor, an icon corresponding to the at least one first recommendation application or first recommendation function can be deleted. When the commands are executed by the at least one processor, a third notification bar including notification information received from the fifth application can be displayed through the display.

[0294] When the instructions according to one embodiment of the present invention are executed by the at least one processor, the electronic device can determine the at least one first recommendation application or first recommendation function based on the electronic device usage model stored in the memory, and the electronic device usage model can be learned to determine the at least one recommendation application or recommendation function based on the electronic device user information.

[0295] Electronic devices can provide various content, such as messages, files, video captures, and notes, through a display. By analyzing the context of content present on the display and determining and providing recommended applications or features based on applications, functions, or device history within the device, the electronic device can enhance the efficiency of tasks required by the user.

[0296] An electronic device can provide a rich and convenient user experience to the user by analyzing the context of content displayed on the display, determining related recommended applications and functions, and displaying them together with the content.

[0297] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0298] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any possible combination of items listed together in the corresponding phrase. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0299] As used in this document, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0300] Various embodiments of the present document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., memory (120)) readable by a machine (e.g., an electronic device (100)). For example, a processor (e.g., a processor (110)) of the machine (e.g., an electronic device (100)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-transient' merely means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0301] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0302] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the components of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

In an electronic device (100), Display (161); Memory (130) for storing instructions; and It includes at least one processor (120); and When the above instructions are executed by the at least one processor, the electronic device, Analyzing context information of content including at least one image, video, or text displayed on the above display, and Identifying association information between the above context information, the application within the electronic device, and the function of the electronic device, and Based on the above context information and the above association information, at least one recommendation application or recommendation function is determined, and An electronic device that displays an icon corresponding to at least one recommendation application or recommendation function along with the above content. In Article 1, The above at least one recommendation application is a plurality of recommendation applications, and When the above instructions are executed by the at least one processor, the electronic device, Receiving user input to select an icon corresponding to at least one of the icons corresponding to the plurality of recommended applications, and An electronic device that enlarges and displays the selected recommended application on the display in response to the above user input. In Article 1, The above at least one recommendation function is a plurality of recommendation functions, and When the above instructions are executed by the at least one processor, the electronic device, Receiving user input to select at least one icon corresponding to a recommendation function among the icons corresponding to the plurality of recommendation functions above, and An electronic device that, in response to the above user input, expands the selected recommendation function and displays it on a part of the display. In Article 1, The above context information is, When the electronic device displays a lock screen through the display, the content includes at least one image, video, or text displayed on the lock screen, and is context information of the content. When the above instructions are executed by the at least one processor, the electronic device, In response to receiving input from a user selecting an icon corresponding to at least one recommendation application or recommendation function, a user authentication UI is provided on the lock screen, and An electronic device that expands and displays the selected recommended application or recommendation function on a part of the display based on the completion of user authentication through the user authentication UI. In Article 1, When the above instructions are executed by the at least one processor, the electronic device, Receiving a request from a user to generate a capture image including at least one image or video displayed on the display, and The above context information is, An electronic device that is context information of content including the above-mentioned captured image. In Article 1, The above context information is, An electronic device that is context information of content including at least one image or text included in a notification popup displayed on the above display. In Article 1, When the above instructions are executed by the at least one processor, the electronic device, An electronic device that identifies the associated information based on at least one of the above context information, the user's usage history of the electronic device, and the usage history of an application within the electronic device. In Article 1, When the above instructions are executed by the at least one processor, the electronic device, Determining at least one recommendation application or recommendation function based on an electronic device usage model stored in the memory, The above electronic device usage model is an electronic device learned to determine at least one recommended application or recommendation function based on the electronic device user information. In Article 1, The above content is the first content, and The above context information is the first context information, and The above association information is the first association information, and The above recommendation application and the above recommendation function are the first recommendation application and the first recommendation function, and When the above instructions are executed by the at least one processor, the electronic device, In response to receiving a request from a user to display second content including at least one image, video, or text displayed on the display in the first content, and analyzing second context information of the second content, Identifying second association information between the second context information, the application within the electronic device, and the function of the electronic device, and Based on the second context information and the second association information, at least one second recommendation application and a second recommendation function are determined, and An electronic device that displays an icon corresponding to at least one second recommendation application or second recommendation function together with the second content. In an electronic device (100), Display (161); Memory (130) for storing instructions; and It includes at least one processor (120); and When the above instructions are executed by the at least one processor, the electronic device, A first notification bar is displayed through the lock screen on the above display, comprising a first icon corresponding to a first application at a first position, a second icon corresponding to a second application at a second position, and information of a third application at a third position. Receives a first user input for the above first notification bar, and By analyzing the context information of the above-mentioned third application, at least one first recommendation application or first recommendation function is determined, and An electronic device that displays an icon corresponding to at least one first recommendation application or first recommendation function together with the first notification bar. In Paragraph 10, When the above instructions are executed by the at least one processor, the electronic device, Receives a first swipe user input for the first notification bar, and In response to the above first swipe user input, a second notification bar containing information of the fourth application is displayed at the above third location, and By analyzing the context information of the above-mentioned fourth application, at least one second recommendation application or second recommendation function is determined, and An electronic device that displays an icon corresponding to at least one second recommendation application or second recommendation function together with the second notification bar. In Article 10, When the above instructions are executed by the at least one processor, the electronic device, Receiving a second user input for an icon corresponding to at least one first recommendation application or first recommendation function, and In response to the above second user input, an icon corresponding to at least one first recommendation application or first recommendation function is deleted from around the first notification bar, and An electronic device that enlarges and displays the at least one first recommendation application or first recommendation function on a portion of the above display. In Paragraph 10, When the above instructions are executed by the at least one processor, the electronic device, An electronic device that displays the first icon, the second icon, and the first notification bar in a reduced manner when displaying an icon corresponding to at least one first recommendation application or first recommendation function. In Paragraph 10, When the above instructions are executed by the at least one processor, the electronic device, Receive a notification from the 5th application, and Delete the icon corresponding to at least one first recommendation application or first recommendation function, and An electronic device that displays a third notification bar containing notification information received from the fifth application through the display. In Paragraph 10, When the above instructions are executed by the at least one processor, the electronic device, Based on the electronic device usage model stored in the memory, the at least one first recommendation application or first recommendation function is determined, and The above electronic device usage model is an electronic device learned to determine at least one recommended application or recommendation function based on the electronic device user information.

Images