Method and electronic device for displaying recommendation information

Abstract

This electronic device may comprise: a memory which stores instructions and which includes at least one storage medium; and at least one processor including processing circuitry. When executed individually or collectively by the at least one processor, the instructions can control such that the electronic device: identifies whether a specified condition is satisfied, the specified condition including at least one from among operations, in which a specific application is executed, connection to an external electronic device is established, a measurement value of a sensor satisfies a preset level, and the electronic device is located at a specified place; collects behavior data of a user of the electronic device for a preset time on the basis that the specified condition is satisfied; determines, on the basis of the behavior data of the user collected for the preset time, information related to at least one operation performed using an input of the user when the specified condition is satisfied; determines an operation that has accumulated more than a specified number of times among pieces of information related to at least one operation performed using the input of the user when the specified condition is satisfied; provides a user interface asking whether to execute, together, the operation accumulated more than the specified number of times when the specified condition is detected; and executes the operation accumulated more than the specified number of times when the specified condition is detected on the basis of a user input to the user interface.

Description

Methods and electronic devices for displaying recommended information

[0001] This article relates to electronic devices, and more specifically, to methods for collecting user behavioral data to display recommended information.

[0002] Recently released electronic devices, such as smartphones, wearable electronic devices (e.g., wearable watches, and / or wearable glasses), and / or tablet devices, can operate online by connecting to a wireless communication server (e.g., an internet server) and can wirelessly transmit and receive data with external electronic devices. The electronic devices and external electronic devices can transmit and receive various types of data (e.g., messages, images, files).

[0003] With the recent increase in electronic devices and the Internet usage via them, automated recommendations (e.g., recommended content) are now being provided to users in various fields (e.g., news, education, history, entertainment, and / or fashion). For example, recommendation servers provide personalized content recommendation services based on the user's content history regarding the content they have consumed using their electronic devices.

[0004] The content history a user leaves online using an electronic device can be divided into records of intentional content usage and records of unintentional content usage (e.g., advertisements, malfunctions, viewing content unrelated to the user's interests). Therefore, current content recommendation services unilaterally provide content that reflects the user's unintentional content usage history (e.g., content outside the user's interests) through recommendation servers. Consequently, current content recommendation services fail to fully reflect users' interests, potentially reducing the reliability of recommended content.

[0005] The method of determining recommended features based on all collected user behavior also involves collecting user behaviors unrelated to contextual changes, reducing the validity of recommended results. Furthermore, recording all user behavior and analyzing patterns increases memory and power consumption.

[0006] And when recommending preset automated features that are not based on the user's behavior, it is difficult for the user to understand why the recommendation occurred, and therefore, there is a problem in that the user does not feel that the recommendation is accurate or useful.

[0007] An electronic device may include at least one processor, including processing circuitry, and memory for storing instructions and including one or more storage media. The instructions, when individually or collectively executed by the at least one processor, may be configured to: determine whether a specified condition is satisfied, including at least one of an operation in which a specific application is executed, an external electronic device is connected, a measurement value of a sensor satisfies a preset level, or a location of the electronic device is located at a preset location; collect user behavior data of the electronic device for a preset period of time based on the satisfaction of the specified condition; determine information related to at least one operation performed by a user input when the specified condition is satisfied based on the user behavior data collected for the preset period of time; determine an operation that has accumulated a specified number of times or more among the information related to at least one operation performed by the user input when the specified condition is satisfied; provide a user interface that asks whether to execute the operations that have accumulated a specified number of times or more together when the specified condition is detected; and control to execute the operations that have accumulated a specified number of times or more when the specified condition is detected based on a user input to the user interface.

[0008] An electronic device according to this document can provide personalized recommendations by recording and analyzing patterns of actions (e.g., application execution, setting change) performed using the electronic device within a valid time limit after a change in the user's context occurs, in order to improve the accuracy of recommendations and increase the satisfaction felt by the user that the electronic device is useful.

[0009] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0010] FIG. 2 is a block diagram illustrating an integrated intelligence system according to various embodiments.

[0011] FIG. 3 is a diagram showing the form in which relationship information between concepts and actions according to various embodiments is stored in a database.

[0012] FIG. 4 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0013] FIG. 5 is a block diagram illustrating a data collection and recommendation method of an electronic device according to one embodiment.

[0014] FIG. 6A illustrates an interface for a personalized routine recommendation provision function of an electronic device according to one embodiment.

[0015] FIG. 6b illustrates an interface for a personalized routine recommendation provision function of an electronic device according to one embodiment.

[0016] FIG. 7 illustrates data collected and data not collected in an electronic device according to one embodiment.

[0017] FIG. 8 illustrates a process of collecting data and generating a routine according to context of an electronic device according to one embodiment.

[0018] FIG. 9 illustrates an interface for a process of collecting data and generating a routine according to context of an electronic device according to one embodiment.

[0019] FIG. 10 is a flowchart illustrating a method for displaying recommendation information of an electronic device according to one embodiment.

[0020] FIG. 11 is a flowchart illustrating a method for displaying recommendation information of an electronic device according to one embodiment.

[0021] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0022] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0023] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0024] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0025] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0026] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0027] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0028] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0029] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0030] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0031] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0032] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0033] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0034] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0035] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0036] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0037] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0038] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0039] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0040] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0041] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0042] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0043] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0044] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0045] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0046] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0047] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0048] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0049] FIG. 2 is a block diagram illustrating an integrated intelligence system according to one embodiment.

[0050] Referring to FIG. 2, an integrated intelligence system according to one embodiment may include an electronic device (210) (e.g., electronic device (101) of FIG. 1), an intelligent server (230) (e.g., server (108) of FIG. 1), and a service server (250) (e.g., server (108) of FIG. 1).

[0051] According to one embodiment, the electronic device (210) may be a terminal device (or electronic device) that can connect to the Internet, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a laptop computer, a TV, white goods, a wearable device, a head mounted display (HMD), or a smart speaker.

[0052] According to the illustrated embodiment, the electronic device (210) may include a communication interface (213) (e.g., the interface (177) of FIG. 1), a microphone (212) (e.g., the input module (150) of FIG. 1), a speaker (216) (e.g., the audio output module (155) of FIG. 1), a display module (211) (e.g., the display module (160) of FIG. 1), a memory (215) (e.g., the memory (130) of FIG. 1), or a processor (214) (e.g., the processor (120) of FIG. 1). The components listed above may be operatively or electrically connected to each other. The electronic device (210) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1.

[0053] In one embodiment, the communication interface (213) may be configured to connect to an external device and transmit and receive data. In one embodiment, the microphone (212) may receive sound (e.g., user speech) and convert it into an electrical signal. In one embodiment, the speaker (216) may output the electrical signal as sound (e.g., voice).

[0054] In one embodiment, the display module (211) may be configured to display an image or video. In one embodiment, the display module (211) may also display a graphical user interface (GUI) of a running app (or application program). In one embodiment, the display module (211) may receive a touch input via a touch sensor. For example, the display module (211) may receive a text input via a touch sensor in an on-screen keyboard area displayed within the display module (211).

[0055] According to one embodiment, the memory (215) can store a client module (218), a software development kit (SDK) (217), and a plurality of apps (e.g., a first app (219a), a second app (219b)). The client module (218) and the SDK (217) can form a framework (or solution program) for performing general functions. In addition, the client module (218) or the SDK (217) can form a framework for processing user input (e.g., voice input, text input, touch input).

[0056] According to one embodiment, the plurality of apps (219a, 219b) stored in the memory (215) may be programs for performing a specified function. According to one embodiment, the plurality of apps may include a first app (219a) and a second app (219b). According to one embodiment, each of the plurality of apps (219a, 219b) may include a plurality of operations for performing a specified function. For example, the apps (219a, 219b) may include an alarm app, a message app, and / or a schedule app. According to one embodiment, the plurality of apps (219a, 219b) may be executed by the processor (214) to sequentially execute at least some of the plurality of operations.

[0057] According to one embodiment, the processor (214) can control the overall operation of the electronic device (210). For example, the processor (214) can be electrically connected to a communication interface (213), a microphone (212), a speaker (216), and a display module (211) to perform designated operations.

[0058] According to one embodiment, the processor (214) may also execute a program stored in the memory (215) to perform a designated function. For example, the processor (214) may execute at least one of the client module (218) or the SDK (217) to perform the following operations for processing user input. The processor (214) may control the operations of a plurality of apps (219a, 219b), for example, through the SDK (217). The following operations described as operations of the client module (218) or the SDK (217) may be operations executed by the processor (214).

[0059] According to one embodiment, the client module (218) can receive user input. For example, the client module (218) can receive a voice signal corresponding to a user utterance detected through the microphone (212). Alternatively, the client module (218) can receive a touch input detected through the display module (211). Alternatively, the client module (218) can receive a text input detected through a keyboard or a visual keyboard. In addition, the client module (218) can receive various forms of user input detected through an input module included in the electronic device (210) or an input module connected to the electronic device (210). The client module (218) can transmit the received user input to the intelligent server (230). The client module (218) can transmit status information of the electronic device (210) together with the received user input to the intelligent server (230). The status information can be, for example, execution status information of an app.

[0060] In one embodiment, the client module (218) may receive a result corresponding to the received user input. For example, the client module (218) may receive a result corresponding to the received user input if the intelligent server (230) can produce a result corresponding to the received user input. The client module (218) may display the received result on the display module (211). Additionally, the client module (218) may output the received result as audio through the speaker (216).

[0061] According to one embodiment, the client module (218) can receive a plan corresponding to the received user input. The client module (218) can display the results of executing multiple operations of the app according to the plan on the display module (211). For example, the client module (218) can sequentially display the results of executing multiple operations on the display module (211) and output audio through the speaker (216). The electronic device (210) can, for another example, display only some results of executing multiple operations (e.g., the result of the last operation) on the display module (211) and output audio through the speaker (216).

[0062] In one embodiment, the client module (218) may receive a request from the intelligent server (230) to obtain information necessary to produce a result corresponding to a user input (e.g., voice input or text input). In one embodiment, the client module (218) may transmit the necessary information to the intelligent server (230) in response to the request.

[0063] According to one embodiment, the client module (218) may transmit result information of executing multiple operations according to a plan to the intelligent server (230). The intelligent server (230) may use the result information to confirm that the received user input has been processed correctly.

[0064] In one embodiment, the client module (218) may include a voice recognition module. In one embodiment, the client module (218) may recognize user input to perform limited functions through the voice recognition module. For example, the client module (218) may execute an intelligent app to process user input to perform organic actions through a specified input (e.g., "Wake up!").

[0065] According to one embodiment, the intelligent server (230) can receive information related to user input from the electronic device (210) via a communication network. According to one embodiment, the intelligent server (230) can convert data related to the received user input into text data. According to one embodiment, the intelligent server (230) can generate a plan for performing a task corresponding to the user input based on the text data.

[0066] In one embodiment, the plan may be generated by an artificial intelligence (AI) system. The AI ​​system may be a rule-based system, a neural network-based system (NNS) (e.g., a feedforward neural network (FNN), a recurrent neural network (RNN)), or a combination of the aforementioned or another AI system. In one embodiment, the plan may be selected from a set of predefined plans or generated in real time in response to a user request. For example, the AI ​​system may select at least one plan from a plurality of predefined plans.

[0067] According to one embodiment, the intelligent server (230) may transmit the results according to the generated plan to the electronic device (210), or transmit the generated plan to the electronic device (210). According to one embodiment, the electronic device (210) may display the results according to the plan on the display module (211). According to one embodiment, the electronic device (210) may display the results of executing an operation according to the plan on the display module (211).

[0068] According to one embodiment, the intelligent server (230) may include a front end (231), a natural language platform (232), a capsule database (238), an execution engine (233), an end user interface (234), a management platform (235), a big data platform (236), and / or an analytic platform (237).

[0069] According to one embodiment, the front end (231) can receive user input from the electronic device (210). The front end (231) can transmit a response corresponding to the user input.

[0070] According to one embodiment, the natural language platform (232) may include an automatic speech recognition module (ASR module) (232a), a natural language understanding module (NLU module) (232b), a planner module (232c), a natural language generator module (NLG module) (232d), and / or a text to speech module (TTS module) (232e).

[0071] According to one embodiment, the automatic speech recognition module (232a) can convert user input received from the electronic device (210) into text data. According to one embodiment, the natural language understanding module (232b) can use the text data of the user input to determine the user's intent. For example, the natural language understanding module (232b) can perform syntactic analysis and / or semantic analysis on user input in the form of text data to determine the user's intent. According to one embodiment, the natural language understanding module (232b) can use linguistic features (e.g., grammatical elements) of morphemes or phrases to determine the meaning of words extracted from the user input, and can match the meaning of the determined words to the intent to determine the user's intent. The natural language understanding module (223b) can obtain intent information corresponding to the user's utterance. The intent information can be information indicating the user's intent determined by interpreting the text data. The intent information can include information indicating an action or function that the user intends to execute using the device.

[0072] According to one embodiment, the planner module (232c) can generate a plan using the intent and parameters determined by the natural language understanding module (232b). According to one embodiment, the planner module (232c) can determine a plurality of domains necessary to perform a task based on the determined intent. The planner module (232c) can determine a plurality of operations included in each of the plurality of domains determined based on the intent. According to one embodiment, the planner module (232c) can determine parameters necessary to execute the plurality of determined operations or result values ​​output by the execution of the plurality of operations. The parameters and the result values ​​can be defined as concepts of a specified format (or class). Accordingly, the plan can include a plurality of operations and a plurality of concepts determined by the user's intent.

[0073] According to one embodiment, the planner module (232c) can determine the relationship between the plurality of operations and the plurality of concepts in a stepwise (or hierarchical) manner. For example, the planner module (232c) can determine the execution order of the plurality of operations determined based on the user's intention based on the plurality of concepts. In other words, the planner module (232c) can determine the execution order of the plurality of operations based on the parameters required for the execution of the plurality of operations and the results output by the execution of the plurality of operations. Accordingly, the planner module (232c) can generate a plan including association information (e.g., ontology) between the plurality of operations and the plurality of concepts. The planner module (232c) can generate the plan using information stored in a capsule database (238) in which a set of relationships between concepts and operations is stored.

[0074] According to one embodiment, the natural language generation module (232d) can convert specified information into text format. The information converted into text format may be in the form of natural language speech. According to one embodiment, the text-to-speech conversion module (232e) can convert text-to-speech information into speech information.

[0075] According to one embodiment, some or all of the functions of the natural language platform (232) may also be implemented in the electronic device (210).

[0076] The capsule database (238) may store information on the relationships between multiple concepts and actions corresponding to multiple domains. According to one embodiment, a capsule may include multiple action objects (or action information) and concept objects (or concept information) included in a plan. According to one embodiment, the capsule database (238) may store multiple capsules in the form of a concept action network (CAN). According to one embodiment, the multiple capsules may be stored in a function registry included in the capsule database (238).

[0077] The capsule database (238) may include a strategy registry that stores strategy information necessary for determining a plan corresponding to a user input. The strategy information may include reference information for determining a single plan when there are multiple plans corresponding to a user input. According to one embodiment, the capsule database (238) may include a follow-up registry that stores information on follow-up actions for suggesting follow-up actions to a user in a given situation. The follow-up actions may include, for example, follow-up utterances. According to one embodiment, the capsule database (238) may include a layout registry that stores layout information of information output through the electronic device (210). According to one embodiment, the capsule database (238) may include a vocabulary registry that stores vocabulary information included in capsule information. According to one embodiment, the capsule database (238) may include a dialog registry that stores information on a dialogue (or interaction) with a user. The capsule database (238) can update stored objects through a developer tool. The developer tool may include, for example, a function editor for updating action objects or concept objects. The developer tool may include a vocabulary editor for updating vocabulary. The developer tool may include a strategy editor for creating and registering strategies that determine plans. The developer tool may include a dialog editor for creating conversations with users.The developer tool may include a follow-up editor that activates follow-up goals and allows editing of follow-up utterances that provide hints. The follow-up goals may be determined based on the currently set goals, user preferences, or environmental conditions. In one embodiment, the capsule database (238) may also be implemented within the electronic device (210).

[0078] In one embodiment, the execution engine (233) can use the generated plan to produce a result. The end user interface (234) can transmit the produced result to the electronic device (210). Accordingly, the electronic device (210) can receive the result and provide the received result to the user. In one embodiment, the management platform (235) can manage information used in the intelligent server (230). In one embodiment, the big data platform (236) can collect user data. In one embodiment, the analysis platform (237) can manage the quality of service (QoS) of the intelligent server (230). For example, the analysis platform (237) can manage the components and processing speed (or efficiency) of the intelligent server (230).

[0079] According to one embodiment, the service server (250) may provide a service (e.g., food ordering or hotel reservation) specified to the electronic device (210). According to one embodiment, the service server (250) may be a server operated by a third party. According to one embodiment, the service server (250) may provide information for generating a plan corresponding to the received user input to the intelligent server (230). The provided information may be stored in the capsule database (238). In addition, the service server (250) may provide result information according to the plan to the intelligent server (230). The service server (250) may include a plurality of service providers (e.g., CP service A (251), CP service B (252), CP service C (253)), and each of the service providers (251, 252, 253) may provide a function for a domain associated with each capsule stored in the capsule database (238) of the intelligent server (230).

[0080] In the integrated intelligence system described above, the electronic device (210) can provide various intelligent services to the user in response to user input. The user input may include, for example, input via a physical button, touch input, or user input.

[0081] According to one embodiment, the electronic device (210) may provide a voice recognition service through an intelligent app (or voice recognition app) stored within the device. In this case, for example, the electronic device (210) may recognize a user utterance or voice input received through the microphone (212) and provide the user with a service corresponding to the recognized user input.

[0082] According to one embodiment, the electronic device (210) may perform a designated operation based on received user input, either alone or together with the intelligent server (230) and / or the service server (250). For example, the electronic device (210) may execute an app corresponding to the received user input and perform a designated operation through the executed app.

[0083] According to one embodiment, when an electronic device (210) provides a service together with an intelligent server (230) and / or a service server (250), the electronic device (210) may detect a user's speech using the microphone (212) and generate a signal (or voice data) corresponding to the detected user's speech. The electronic device (210) may transmit the voice data to the intelligent server (230) via a network (240) using a communication interface (213).

[0084] In one embodiment, an intelligent server (230) may generate a plan for performing a task corresponding to a user input received from an electronic device (210), or a result of performing an operation according to the plan, in response to a user input. The plan may include, for example, a plurality of operations for performing a task corresponding to the user input, and a plurality of concepts related to the plurality of operations. The concept may define parameters input to the execution of the plurality of operations, or result values ​​output by the execution of the plurality of operations. The plan may include association information between the plurality of operations and the plurality of concepts.

[0085] According to one embodiment, the electronic device (210) can receive the response using the communication interface (213). The electronic device (210) can output a voice signal generated within the electronic device (210) to the outside using the speaker (216), or can output an image generated within the electronic device (210) to the outside using the display module (211).

[0086] Although FIG. 2 illustrates an example in which voice recognition, natural language understanding and generation, and plan-based output operations of user input received from an electronic device (210) are performed on an intelligent server (230), the various embodiments of the present document are not limited thereto. For example, at least some components of the intelligent server (230) (e.g., natural language platform (232), execution engine (233), capsule database (238)) may be embedded in the electronic device (210) (or the electronic device (101) of FIG. 1), such that the operations may be performed by the electronic device (210).

[0087] FIG. 3 is a diagram showing a form in which relationship information between concepts and actions is stored in a database according to various embodiments.

[0088] According to one embodiment, a capsule database (e.g., capsule database (238) of FIG. 2) of an intelligent server (e.g., intelligent server (230) of FIG. 2) may store capsules in the form of a CAN (concept action network) (300). The capsule database may store operations for processing tasks corresponding to user inputs of a user, and parameters necessary for the operations, in the form of a CAN (concept action network).

[0089] According to one embodiment, the capsule database may store a plurality of capsules (capsule (A) (310), capsule (B) (320)) corresponding to each of a plurality of domains (e.g., applications). According to one embodiment, one capsule (e.g., capsule (A) (310)) may correspond to one domain (e.g., location (geo), application). In addition, one capsule may correspond to at least one service provider (e.g., CP 1 (331) or CP 2 (332)) for performing a function for a domain related to the capsule. According to one embodiment, one capsule may include at least one operation (350) and at least one concept (360) for performing a specified function.

[0090] According to one embodiment, a natural language platform (e.g., the natural language platform (232) of FIG. 2) can generate a plan for performing a task corresponding to a received natural language input (e.g., a voice input or a text input) using capsules stored in a capsule database. For example, a planner module of the natural language platform (e.g., the planner module (232c) of FIG. 2) can generate a plan using capsules stored in a capsule database. For example, a plan can be generated using actions (311, 313) and concepts (312, 314) of capsule A (310) and actions (321) and concepts (322) of capsule B (320).

[0091] FIG. 4 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0092] Referring to FIG. 4, an electronic device (101) according to one embodiment of the present disclosure may include a display (490) (e.g., the display module (160, 211) of FIG. 1 or 2), a memory (130) (e.g., the memory (130) of FIG. 1), a communication circuit (495) (e.g., the communication module (190) or the communication interface (213) of FIG. 1), and / or a processor (120) (e.g., the processor (120, 214) of FIG. 1 or 2). According to one embodiment, the electronic device (101) may include all or at least a part of the components of the electronic device (101, 210) described in the description with reference to FIG. 1 or 2. For example, in various embodiments of the present document, some of the illustrated components may be omitted or replaced. The electronic device (101) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1 and / or the electronic device (210) of FIG. 2. At least some of the respective configurations of the illustrated (or not illustrated) electronic device (101) may be operatively, functionally, and / or electrically connected to each other.

[0093] According to one embodiment, the display (490) may include a configuration identical or similar to the display module (160, 211) of FIG. 1 or FIG. 2. According to one embodiment, the display (490) may display various images provided from the processor (120). According to one embodiment, the display (490) may visually provide, under the control of the processor (120), an application being executed (e.g., the application (146) of FIG. 1) and various screens related to its use (e.g., a contents screen, an application execution screen, a menu screen, and / or a function execution screen).

[0094] According to one embodiment, the display (490) may be combined with a touch sensor, a pressure sensor capable of measuring the intensity of a touch, and / or a touch panel (e.g., a digitizer) that detects a magnetic stylus pen. According to one embodiment, the display (490) may detect a touch input, an air gesture input, and / or a hovering input (or a proximity input) by measuring a change in a signal (e.g., voltage, light intensity, resistance, electromagnetic signal, and / or charge) for a specific location of the display (490) based on the touch sensor, the pressure sensor, and / or the touch panel. For example, the display (490) may include a touchscreen that detects a touch and / or a proximity touch (or a hovering) input using a part of a user's body (e.g., a finger) or an input device (e.g., a stylus pen). The display (490) may include at least some of the configuration and / or functions of the display module (160) of FIG. 1 and / or the display module (211) of FIG. 2.

[0095] In one embodiment, the display (490) may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED) display, and / or an active matrix OLED (AMOLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. In one embodiment, the display (490) may include a flexible display.

[0096] According to one embodiment, the memory (130) includes at least a portion of the configuration and / or function of the memory (130) of FIG. 1 and / or the memory (215) of FIG. 2, and may store software (e.g., the program (140) of FIG. 1). The memory (101) may store various applications (e.g., the application (146) of FIG. 1 or the plurality of apps (219a, 219b) of FIG. 2), and program modules supporting intelligent services (e.g., the client module (218) of FIG. 2).

[0097] According to one embodiment, the memory (130) may store various data used by at least one component (e.g., processor (120)) of the electronic device (101). In one embodiment, the data may include, for example, software (e.g., program (140) of FIG. 1), and input data or output data for commands related to the software.

[0098] According to one embodiment, the memory (130) may include volatile memory (e.g., volatile memory (132) of FIG. 1) or non-volatile memory (134) (e.g., non-volatile memory (134) of FIG. 1). According to one embodiment, the memory (130) may store instructions or data received from the processor (120) in the volatile memory (1320), and may store result data of instructions or data stored in the volatile memory (132) being processed by the processor (120) in the non-volatile memory (134).

[0099] In one embodiment, the data may include various data (e.g., learning data, prompt data, contextual awareness data, and / or urgency (or priority) determination data) to assist in controlling the operation of the electronic device (101) to provide artificial intelligence-based notification information. In one embodiment, the data may include information regarding various settings to assist in controlling the operation of the electronic device (101) to provide artificial intelligence-based notification information.

[0100] In one embodiment, the data may include various learning data and / or parameters acquired based on the user's learning through interaction with the user. In one embodiment, the data may include various schemas (or algorithms, models, networks, or functions) for supporting the operation of providing AI-based notification information.

[0101] For example, a scheme for supporting an operation of providing artificial intelligence-based notification information in an electronic device (101) may include a neural network. In one embodiment, the neural network may include a neural network model based on at least one of an artificial neural network (ANN), a convolution neural network (CNN), a region with convolution neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a deconvolution network, a deep belief network (DBN), a restricted Boltzman machine (RBM), a long short-term memory (LSTM) network, a classification network, a plain residual network, a dense network, a hierarchical pyramid network, and / or a fully convolutional network. According to one embodiment, the type of the neural network model is not limited to the examples described above.

[0102] According to one embodiment, the memory (130) may store instructions that, when executed, cause the processor (120) to operate. For example, the instructions may be stored as software (e.g., program (140) of FIG. 1) on the memory (130) and executable by the processor (120). For example, the instructions may include control commands such as arithmetic and logical operations, data movement, and / or input / output that may be recognized by the processor (120). According to one embodiment, the software may include various applications (e.g., application (146) of FIG. 1) that may provide various functions (or services) (e.g., call function, message function, messenger function, e-mail function, SNS (social networking service) function, search function, media (e.g., video and / or music) playback function, game function, and / or wireless communication function) in the electronic device (101).

[0103] In one embodiment, the communication circuit (495) may support wireless communication with an external device using cellular wireless communication (e.g., 4G LTE, 5G NR) and / or short-range wireless communication (e.g., Wi-Fi). For example, the electronic device (101) may use the communication circuit (495) to communicate with an external server providing a voice assistant function over a network. In one embodiment, the communication circuit (495) may include at least a portion of the configuration and / or functions of the communication module (190) of FIG. 1 and / or the communication interface (213) of FIG. 2.

[0104] According to one embodiment, the processor (120) may perform an application layer processing function requested by a user of the electronic device (101). According to one embodiment, the processor (120) may provide control and commands of functions for various blocks of the electronic device (101). According to one embodiment, the processor (120) may perform operations or data processing related to control and / or communication of each component of the electronic device (101). For example, the processor (120) may include at least some of the configurations and / or functions of the processor (120) of FIG. 1. According to one embodiment, the processor (120) may be operatively connected to the components of the electronic device (101). According to one embodiment, the processor (120) may load commands or data received from other components of the electronic device (101) into the memory (130), process the commands or data stored in the memory (130), and store result data.

[0105] According to one embodiment, the processor (120) may include one or more processors including processing circuitry and / or executable program elements. According to one embodiment, the processor (120) may control (or process) the overall operation related to generating and providing context-sensitive notification information based on artificial intelligence (AI) based on the processing circuitry and / or executable program elements.

[0106] According to one embodiment, the processor (120) may process an operation for performing a task related to an application. According to one embodiment, the processor (120) may perform an operation for detecting a notification while performing the task. According to one embodiment, the processor (120) may perform an operation for performing context awareness related to the task based on detecting the notification. According to one embodiment, the processor (120) may perform an operation for extracting data for a notification interface based on a situation resulting from the context awareness. According to one embodiment, the processor (120) may perform an operation for generating a notification interface corresponding to the situation based on the extracted data. According to one embodiment, the processor (120) may perform an operation for providing a notification interface in association with a task.

[0107] According to one embodiment, the detailed operation of the processor (120) of the electronic device (101) is described with reference to the drawings described below.

[0108] According to one embodiment, the processor (120) may be an application processor (AP). According to one embodiment, the processor (120) may be a system semiconductor that is responsible for the operation and multimedia driving functions of the electronic device (101). According to one embodiment, the processor (120) may be configured in the form of a system-on-chip (SoC), and may include a technology-intensive semiconductor chip that integrates multiple semiconductor technologies into one and implements system blocks into a single chip.

[0109] According to one embodiment, the system blocks of the processor (120) may include a graphics processing unit (GPU) (410), an image signal processor (ISP) (420), a central processing unit (CPU) (430), a neural processing unit (NPU) (440), a digital signal processor (DSP) (450), a modem (460), a connectivity (470), and / or a security (480) block, as illustrated in FIG. 4.

[0110] In one embodiment, the GPU (410) may be responsible for graphics processing. In one embodiment, the GPU (410) may receive commands from the CPU (430) and perform graphics processing to express the shape, position, color, shading, movement, and / or texture of objects (or objects) on the display.

[0111] In one embodiment, the ISP (420) may be responsible for image processing and correction of images and videos. In one embodiment, the ISP (420) may correct raw data (e.g., raw data) transmitted from an image sensor of a camera (e.g., the camera module (180) of FIG. 1) to generate an image in a form more preferred by the user. In one embodiment, the ISP (420) may perform post-processing, such as adjusting partial brightness of an image and emphasizing detailed parts. For example, the ISP (420) may independently perform a process of tuning and correcting the image quality of an image acquired through a camera to generate a result preferred by the user.

[0112] According to one embodiment, the ISP (420) may support artificial intelligence (AI)-based image processing technology. According to one embodiment, the ISP (420) may support scene segmentation (e.g., image segmentation) technology that recognizes and / or classifies parts of a scene being captured in conjunction with the NPU (440). For example, the ISP (420) may include a function that applies different parameters to objects such as the sky, bushes, and / or skin and processes them. According to one embodiment, the ISP (420) may detect and display a human face during image capture using the AI ​​function, or adjust the brightness, focus, and / or color of the image using the coordinates and information of the face.

[0113] According to one embodiment, the CPU (430) may perform a role corresponding to the processor (120). According to one embodiment, the CPU (430) may decode a user's command, perform arithmetic and logical operations, and / or data processing. For example, the CPU (430) may be responsible for functions such as memory, interpretation, calculation, and control. According to one embodiment, the CPU (430) may control the overall function of the electronic device (101). For example, the CPU (430) may execute all software of the electronic device (101) (e.g., the application (146) of FIG. 1 and / or the plurality of apps (219a, 219b) of FIG. 2) on an operating system (OS) and control the hardware device. According to one embodiment, the CPU (430) may execute an application and control the overall operation of the processor (120) to perform neural network-based tasks required according to the execution of the application.

[0114] According to one embodiment, the CPU (430) may store instructions or data in volatile memory (e.g., volatile memory (132) of FIG. 1) of the memory (130), process the instructions or data stored in the volatile memory, and store resultant data in nonvolatile memory (e.g., nonvolatile memory (134) of FIG. 1) of the memory (130) as at least part of data processing or calculation.

[0115] According to one embodiment, the CPU (430) may include a single processor core or multiple processor cores (multi-core). According to one embodiment, the CPU (430) may be a programmable processor that stores executable instructions (e.g., instructions capable of performing operations of the CPU (430)) and executes the instructions.

[0116] According to one embodiment, the CPU (430) can operate in a multi-domain environment. According to one embodiment, the CPU (430) can operate in a multi-domain environment of a normal world (e.g., a non-secure world, a framework, or a non-secure environment) and a secure world (e.g., a secure framework or a secure environment). In one embodiment, a domain of the secure world can include one or more domains (e.g., a trusted OS, a trust zone, and / or a virtualization framework).

[0117] In one embodiment, the NPU (440) may be responsible for processing optimized for artificial intelligence deep-learning algorithms. In one embodiment, the NPU (440) is a processor optimized for deep-learning algorithm operations (e.g., artificial intelligence operations) and can process big data quickly and efficiently like a human neural network. For example, the NPU (440) may be mainly used for artificial intelligence operations. In one embodiment, the NPU (440) may be responsible for automatically adjusting the focus by recognizing objects, environments, and / or people in the background when taking a video through a camera, automatically switching the shooting mode of the camera module (180) to food mode when taking a picture of food, and / or deleting only unnecessary subjects from the captured results. In one embodiment, the NPU (440) may be responsible for processing that analyzes various situations based on a running application based on situational awareness and provides notifications tailored to the situation corresponding to the analyzed results.

[0118] According to one embodiment, the electronic device (101) can support integrated machine learning processing by interacting with all processors such as the GPU (410), the ISP (420), the CPU (430), and the NPU (440).

[0119] In one embodiment, the DSP (450) may represent an integrated circuit that facilitates rapid processing of digital signals. In one embodiment, the DSP (450) may perform the function of converting analog signals into digital signals and performing high-speed processing.

[0120] According to one embodiment, the modem (460) may perform a role that enables the use of various communication functions in the electronic device (101). For example, the modem (460) may support communications such as phone calls and data transmission and reception by exchanging signals with a base station. According to one embodiment, the modem (460) may include an integrated modem (e.g., a cellular modem, an LTE modem, a 5G modem, a 5G-Advanced modem, and a 6G modem) that supports communication technologies such as long term evolution (LTE) and 2G to 5G. According to one embodiment, the modem (460) may include an artificial intelligence modem that applies an artificial intelligence algorithm.

[0121] In one embodiment, the connectivity (470) may support wireless data transmission based on IEEE 802.11. In one embodiment, the connectivity (470) may support communication services based on IEEE 802.11 (e.g., Wi-Fi) and / or 802.15 (e.g., Bluetooth, ZigBee, UWB). For example, the connectivity (470) may support communication services targeting an unspecified number of people in a localized area, such as indoors, using an unlicensed band.

[0122] According to one embodiment, security (480) may provide an independent security execution environment between data or services stored in the electronic device (101). According to one embodiment, security (480) may play a role in preventing external hacking through software and hardware security during the process of user authentication when providing services such as biometrics, mobile identification, and / or payment of the electronic device (101). For example, security (480) may provide an independent security execution environment for device security for reinforcing the security of the electronic device (101) itself and for security services based on user information such as mobile identification, payment, and car keys in the electronic device (101).

[0123] According to one embodiment, the operations performed by the processor (120) may be implemented as a recording medium (or a computer program product). For example, the recording medium may include a non-transitory computer-readable recording medium having recorded thereon a program for executing various operations performed by the processor (120).

[0124] The embodiments described in the present disclosure may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. In a hardware implementation, the operations described in one embodiment may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0125] In one embodiment, the recording medium (or computer program product) may include a computer-readable recording medium having recorded thereon a program that causes the computer to perform operations such as performing a task related to an application, detecting a notification while performing the task, performing context awareness related to the task based on detecting the notification, extracting data for a notification interface based on a context according to a result of the context awareness, generating a notification interface corresponding to the context based on the extracted data, and providing the notification interface in association with the task.

[0126] An electronic device (101) according to one embodiment of the present disclosure may include a display (160), a memory (130) storing instructions, and a processor (120) operatively connected to the display and the memory.

[0127] FIG. 5 is a block diagram illustrating a data collection and recommendation method of an electronic device according to one embodiment.

[0128] In Figure 510, the first point in time (512) may refer to a point in time when a change in context that is perceivable to the user occurs. The change in context may refer to a situation in which a specified condition is satisfied. The specified condition may include, for example, at least one of the following actions: a specific application is executed, an external electronic device is connected, a sensor measurement value meets a preset level, or the electronic device is located at a specified location.

[0129] A specific application may include, for example, an application that plays a video. For example, an electronic device (e.g., the electronic device (101) of FIG. 1) may determine that a context has changed when an application that plays a video is executed. The electronic device (101) may collect user behavioral data from a first point in time (512) to a second point in time (514) after a specified time (515). The user behavioral data may include, for example, changing settings, executing an application, or controlling the electronic device (101).

[0130] In Figure 520, the electronic device (101) can infer repetitive user behavior based on collected user behavior data. The electronic device (101) can determine an action that has been performed a specified number of times (e.g., six times) or more over a certain period (e.g., one month) from among the collected user behavior data. The specified period (e.g., one month) and the specified number of times (e.g., six times) are merely examples and may vary depending on the settings.

[0131] In Figure 530, the electronic device (101) can measure similarity with a pre-configured template. According to one embodiment, the electronic device (101) can pre-configure templates of actions expected to be effective when a context changes. The templates are as shown in Table 1 below.

[0132] Category Type Action Definition Data Collection Time Temporal Context Before Bed Time The time 30 minutes before falling asleep based on sleep patterns 30 minutes Place Context Home The place you spend the most time and are present during the time you sleep 5 minutes from the moment you arrive at home Office The place you spend the most time outside of home on weekdays 5 minutes from the moment you arrive at work Car The place you connect to your electronic device via Bluetooth 3 minutes after connecting Frequently Visited The place you spend more than 30 minutes at least twice a week 5 minutes from the moment you arrive at your frequently visited location Device Context Low Battery Battery level is 15% or less Within 10 minutes Action Context Exercising Determined by sensor readings or the launch of a health-related application From the start to the end of exercise Wake Up When the electronic device is turned on from off or detects user input after a few hours 30 minutes Application Launch Processor detects application launch 5 minutes Charging Detected by battery and sensors 10 minutes External device Connect using Bluetooth or Wi-Fi for detection in 5 minutes

[0133] The categories, types, action definitions, and data collection times described in [Table 1] are merely examples and are not intended to be limiting. According to one embodiment, the electronic device (101) may predetermine actions corresponding to context changes and collect user behavioral data for a specified period of time for each corresponding action.

[0134] In Figure 540, the electronic device (101) can recommend a generated routine. A routine may refer to a series of processes for executing repeated user actions from collected user behavior data when a context change is detected. The electronic device (101) can create a routine that executes a recommended action once or repeatedly based on user input. Alternatively, the electronic device (101) can receive feedback rejecting the pattern recommendation and control the routine from being generated.

[0135] FIG. 6A illustrates an interface for a personalized routine recommendation provision function of an electronic device according to one embodiment.

[0136] In Figure 610, an electronic device (e.g., electronic device (101) of Figure 1) can detect a change in context. The change in context may mean, for example, a situation in which a communication connection is established with an external electronic device (e.g., a vehicle). The change in context is not limited to this.

[0137] The electronic device (101) can collect user behavioral data for a specified period of time (e.g., 3 minutes) based on the time when a communication connection is established with an external electronic device (e.g., a vehicle). The electronic device (101) can determine, among the collected user behavioral data, a behavior (e.g., launching a navigation application) whose repetition frequency exceeds a specified level (e.g., 5 times in a week). Alternatively, the electronic device (101) can determine whether the number of pattern occurrences or the number of times a context is satisfied exceeds a specified value. The criteria for selecting specific data among the user behavioral data is not fixed to counting the number of times and may vary depending on the settings. The electronic device (101) can then display an interface asking whether to execute the determined behavior (e.g., launching a navigation application) when a communication connection is established with an external electronic device (e.g., a vehicle). The behaviors whose repetition frequency exceeds a specified level among the collected user behavioral data are not limited to this and may further include various actions, such as switching to sound mode or activating Wi-Fi.

[0138] According to one embodiment, the electronic device (101) can determine whether a specified condition is satisfied, including at least one of the following operations: a specific application is executed, a connection is made to an external electronic device, a sensor measurement value satisfies a preset level, or the electronic device is located at a specified location. The electronic device (101) can determine a corresponding time based on the type of condition satisfied, and collect behavioral data of the user of the electronic device (101) during the determined time. The electronic device (101) can extract a user behavioral pattern from information related to at least one operation performed by a user input when the specified condition is satisfied, and determine a user-customized action based on the frequency of the pattern, the consistency of the pattern, and the duration of the pattern. The electronic device (101) can provide a user interface that asks whether to execute the user-customized action together when the specified condition is detected, and can execute the user-customized action when the specified condition is detected based on a user input to the user interface.

[0139] FIG. 6b illustrates an interface for a personalized routine recommendation provision function of an electronic device according to one embodiment.

[0140] An electronic device (e.g., the electronic device (101) of FIG. 1) may perform a recommendation function in response to a situation in which a communication connection is established with an external electronic device (e.g., a vehicle), as well as a situation in which a user arrives at a designated location (e.g., home). The electronic device (101) may determine whether the user has arrived at the designated location (e.g., home) based on the location of the electronic device (101), and may collect the user's behavioral data for a designated period of time (e.g., 10 minutes) based on the moment of arrival at the designated location (e.g., home). The electronic device (101) may determine, among the collected user's behavioral data, a behavior (e.g., Wi-Fi activation) whose execution frequency exceeds a designated level. Thereafter, the electronic device (101) may display an interface that prompts the user to perform the behavior (e.g., Wi-Fi activation) whose execution frequency exceeds the designated level, based on detecting that the user has arrived at the designated location (e.g., home). The behavior (e.g., Wi-Fi activation) whose execution frequency exceeds the designated level is an example and is not limited thereto. The designated location (e.g., home) is an example and is not limited to this.

[0141] FIG. 7 illustrates data collected and data not collected in an electronic device according to one embodiment.

[0142] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) may collect user behavioral data during a specified time (715) between a first time (712) and a second time (714). The user behavioral data may include, for example, actions such as changing settings, executing applications, or controlling the electronic device (101).

[0143] The first point in time (712) may refer to a point in time when a change in context that is perceivable to the user occurs. The change in context may refer to a situation in which a specified condition is satisfied. The specified condition may include, for example, at least one of the following actions: a specific application is executed, an external electronic device is connected, a sensor measurement value meets a preset level, or the electronic device is located at a specified location.

[0144] On the other hand, the electronic device (101) may not collect the user's behavioral data after a specified time (715) has passed. In FIG. 7, the electronic device (101) may collect the first data (721), the second data (723), and the third data (725) between the first time point (712) and the second time point (714) among the user's behavioral data between the specified time point (715). The electronic device (101) may not collect the fourth data (727), the fifth data (729), and the sixth data (731) among the user's behavioral data. The electronic device (101) may determine to collect data before the specified time point based on the dotted line (726) between the third data (725) and the fourth data (727) and not to collect data after the specified time point.

[0145] In one embodiment, the fourth data (727), the fifth data (729), and the sixth data (731) may have relatively low relevance to the context since they are behavioral data that occurred after a specified time period from the time when the context change occurred. The first data (721), the second data (723), and the third data (725) may have relatively high relevance to the context since they are behavioral data that occurred within a specified time period from the time when the context change occurred.

[0146] According to one embodiment, the electronic device (101) may collect data having a relatively high relevance to the context, such as the first data (721), the second data (723), and the third data (725), and may not collect data having a relatively low relevance to the context, such as the fourth data (727), the fifth data (729), and the sixth data (731). The electronic device (101) may increase the validity of the recommendation result by collecting data having a high relevance to the context change. The electronic device (101) may save the capacity of the memory (e.g., the memory (130) of FIG. 1) and reduce power consumption by not collecting data having a low relevance to the context change.

[0147] FIG. 8 illustrates a process of collecting data and generating a routine according to context of an electronic device according to one embodiment.

[0148] In operation 810, an electronic device (e.g., electronic device (101) of FIG. 1) may collect user behavior data and data about the state of the electronic device (101).

[0149] In operation 820, the electronic device (101) can train a machine learning model using the collected user data. The electronic device (101) can determine a usage pattern when the context changes using the machine learning model.

[0150] In operation 830, the electronic device (101) may generate a routine based on a pattern determined to be used when a context changes, and verify its validity with pre-stored context-specific actions. The pre-stored context-specific actions may refer to actions determined in advance to be useful for each context. The pre-stored context-specific actions may have the form shown in [Table 2] below.

[0151] Recognizable conditionsTime limitSupported actionsArriving home30 minutesRunning a video application, connecting to Wi-Fi, sound mode, and read-aloud modeGetting in the car5 minutesRunning a video application (e.g., maps, music), invoking voice assistant, and keeping the phone unlockedConnecting an external electronic device3 minutesRunning a specific application (e.g., music)When waking up30 minutesRunning an application, invoking voice assistant to provide weather informationEntering sleep mode30 minutesRunning dark mode, invoking power saving mode, switching to silent or vibration modeRunning a video applicationWithin 5 minutesAuto-rotate screen, do not disturb, increase screen-on timeDuring exerciseUntil the end of exerciseDo not disturb mode, launching a music application, increasing media volume

[0152] The conditions, times, and actions in [Table 2] are only examples and are not limited thereto and may vary depending on the settings. In operation 840, the electronic device (101) may generate a routine and generate a phrase to guide the generated routine using a large language model (LLM).

[0153] The entered user query or action can be sent to a prompt design component (not shown). The prompt design component (251) can be used to generate prompts suitable for input into a large language model (LLM) or a large multimodal model. The prompt design component can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component can access a knowledge component containing user preference data, a prompt library, and prompt examples to generate prompts and pass them to the large language model (LLM) or the large multimodal model (LMM).

[0154] In one embodiment, the electronic device (101) may provide a prompt to a generative AI to provide result data (e.g., a response to a user query) based on the prompt. In one embodiment, the prompt may be provided to an on-device generative AI and / or a server-side generative AI (e.g., an LLM server). In one embodiment, the result data may be provided based on the on-device and / or server.

[0155] FIG. 9 illustrates an interface for a process of collecting data and generating a routine according to context of an electronic device according to one embodiment.

[0156] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) can collect condition information (910). The condition information (910) can include, for example, at least one of a situation (e.g., sleeping, driving), a time (e.g., from 1:00 to 2:00), a location where the electronic device (101) is located (e.g., home, work), or a connection status with an external electronic device (e.g., vehicle). The situation, time, location, and type of external electronic device are merely examples and are not limited thereto.

[0157] The electronic device (101) can collect information (920) about user actions. The information (920) about user actions can include, for example, at least one of information about changing settings of the electronic device (101), changing settings of a connected external electronic device, executing a specific mode, or executing a specific application.

[0158] The electronic device (101) can collect user behavior patterns (930). The user behavior patterns (930) may include, for example, information about bedtime or the time when the user sets the Do Not Disturb mode. The user behavior patterns (930) are merely examples and are not limited thereto.

[0159] According to one embodiment, the electronic device (101) may generate information (940) about a routine based on status information (910), information about user actions (920), and information about the user's behavioral patterns (930). For example, a routine may refer to a pattern of activating a do-not-disturb mode and a sleep mode during a time period from 11 PM to 7 AM when the user's bedtime is set to that time. The routine may be determined based on the user's behavioral data and the status of the electronic device (101) to provide convenience to the user. The electronic device (101) may provide feedback based on the user's evaluation of the recommended routine and may maintain or change the routine.

[0160] FIG. 10 is a flowchart illustrating a method for displaying recommendation information of an electronic device according to one embodiment.

[0161] The operations described through FIG. 10 may be implemented based on instructions that may be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method (1000) may be executed by an electronic device (e.g., electronic device (101) of FIG. 1) described above through FIGS. 1 to 9, and the technical features described above will be omitted below. The order of each operation of FIG. 10 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0162] In operation 1010, the electronic device (101) may detect a change in user context under the control of a processor (e.g., the processor (120) of FIG. 1). A change in context may refer to a situation in which a specified condition is satisfied. The specified condition may include, for example, at least one of: a specific application being executed, a connection to an external electronic device, a sensor measurement value satisfying a preset level, or the electronic device (101) being located at a specified location. In addition, the specified condition may also include a specific time (e.g., before going to work, before going to bed, upon waking up, etc.) or a state of the electronic device (101) (e.g., data usage, lighting, whether a specified application is running, whether a specified application is terminated, whether a connection to an external electronic device is made, or the location of the electronic device (101). The specified condition is merely an example and may vary depending on the setting and is not limited thereto.

[0163] In operation 1020, the electronic device (101) may collect user behavior data for a set period of time. The electronic device (101) may collect user behavior data of the electronic device for a set period of time based on the satisfaction of a set condition. The set period of time may vary depending on the context.

[0164] In operation 1030, the electronic device (101) can extract a user's behavioral pattern based on behavioral data collected from the moment a change in context is recognized. The electronic device (101) can determine information related to at least one action performed by the user's input when a specified condition is met based on the user's behavioral data collected over a specified period of time.

[0165] In operation 1040, the electronic device (101) may recommend to the user a behavioral pattern that is repeated a set number of times or more when a context changes, and may generate and display an automated object (routine). The electronic device (101) may determine an action that has been accumulated and executed a set number of times or more from among information related to at least one action performed by the user's input when a specified condition is met. When a specified condition is detected, the electronic device (101) may provide a user interface that asks whether to execute the actions accumulated a set number of times or more together.

[0166] In operation 1050, the electronic device (101) may receive feedback from the user to modify or maintain the automated object (routine). The electronic device (101) may execute the accumulated operation a specified number of times or more when the specified condition is detected based on user input to the user interface.

[0167] According to one embodiment, the electronic device (101) can detect a change in the user's context by receiving an event generated by an application. The electronic device (101) can store the change in context in memory (130) along with a timestamp.

[0168] For example, when an electronic device (101) runs a video application (e.g., YouTube), it can store information about the type of application run and the time the application was run in the memory (130).

[0169] For example, the electronic device (101) can determine whether the user has arrived at a specific location based on the location of the electronic device (101). The electronic device (101) can store information about the location the user entered or left along with time information in the memory (130).

[0170] According to one embodiment, the electronic device (101) can collect user behavioral data using a time limit. To improve the accuracy of behavioral patterns, user behavior can be collected by applying separate time limits for each context. A context may include, for example, at least one of the following situations: arriving at a specific location, establishing a communication connection with an external electronic device, or changing the mode of the electronic device (101). This is described in [Table 2] of FIG. 8.

[0171] According to one embodiment, the electronic device (101) can extract a user's behavioral pattern based on behavioral change data that occurred during a time limit from the moment a change in context is recognized. For example, the electronic device (101) can extract a behavioral pattern based on a change in context and behavioral change data collected during the time limit. The electronic device (101) can confirm (confidence) whether the behavioral pattern is repeated a certain number of times (e.g., 5 times) or more during a specified period (e.g., 1 month). The electronic device (101) can collect and analyze contexts and behavioral patterns until the number of repetitions of the extracted behavioral pattern exceeds a certain level.

[0172] According to one embodiment, the electronic device (101) may display a user interface (e.g., interfaces (620, 630) of FIG. 6B) recommending an automated behavior pattern when the next context change is detected if the behavior pattern is repeated a certain number of times (e.g., 5 times) or more over a specified period (e.g., 1 month).

[0173] For example, the electronic device (101) may determine to recommend only actions that are executed 5 or more times during a specified period. The electronic device (101) may collect user behavior data indicating that screen auto-rotation was performed 6 times within 5 minutes from the time a video application (e.g., YouTube app) was executed. In addition, the electronic device (101) may collect user behavior data indicating that an action of setting the screen-on time to 10 minutes was performed 4 times within 5 minutes from the time the video application was executed. The electronic device (101) may then display an interface asking whether to execute screen auto-rotation in a situation where the video application is executed. The electronic device (101) may then detect one more action of setting the screen-on time to 10 minutes within 5 minutes from the time the video application was executed. In this case, the electronic device (101) may then display an interface asking whether to extend the screen-on time to 10 minutes in a situation where the video application is executed.

[0174] In one embodiment, the electronic device (101) may separately set a period for accumulating the number of executions of a user behavior pattern. For example, the electronic device (101) may accumulate the number of executions of a user behavior pattern for one month and reset the number of executions of the user behavior pattern after one month. One month is merely an example, and the period may vary depending on the setting.

[0175] In one embodiment, the electronic device (101) may provide the user with an option to perform an automated object (e.g., a routine that executes a recommendation function) only once. Additionally, the electronic device (101) may provide an option to control the automated object to be automatically executed each time.

[0176] According to one embodiment, the electronic device (101) may execute an operation that has been cumulatively executed a specified number of times or more based on a detected change in context. The electronic device (101) may provide a notification asking whether the user is satisfied with the operation executed along with the change in context, and may generate a routine for executing a recommendation function based on a user's response that they are satisfied with the operation executed along with the change in context. The electronic device (101) may control not to generate a routine for executing the recommendation function based on a user's response that they are not satisfied. The electronic device (101) may collect user feedback and determine whether to execute an automated object (e.g., a routine for executing a recommendation function).

[0177] FIG. 11 is a flowchart illustrating a method for displaying recommendation information of an electronic device according to one embodiment.

[0178] The operations described through FIG. 11 may be implemented based on instructions that may be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method (1100) may be executed by an electronic device (e.g., electronic device (101) of FIG. 1) described above through FIGS. 1 to 9, and the technical features described above will be omitted below. The order of each operation of FIG. 11 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0179] In operation 1102, the electronic device (101) may determine a function to recommend based on user behavior under the control of a processor (e.g., processor (120) of FIG. 1).

[0180] According to one embodiment, the electronic device (101) can extract a user's behavioral pattern based on behavioral change data that occurred during a time limit from the moment a change in context is recognized. For example, the electronic device (101) can extract a behavioral pattern based on a change in context and behavioral change data collected during the time limit. The electronic device (101) can confirm (confidence) whether the behavioral pattern is repeated a certain number of times (e.g., 5 times) or more during a specified period (e.g., 1 month). The electronic device (101) can collect and analyze contexts and behavioral patterns until the number of repetitions of the extracted behavioral pattern exceeds a certain level.

[0181] In operation 1110, the electronic device (101) may display an interface asking whether to choose to execute the recommended function. The electronic device (101) may terminate the operation based on the choice not to execute the recommended function.

[0182] According to one embodiment, the electronic device (101) may display a user interface (e.g., interfaces (620, 630) of FIG. 6B) recommending an automated behavior pattern when the next context change is detected if the behavior pattern is repeated a certain number of times (e.g., 5 times) or more over a specified period (e.g., 1 month).

[0183] In operation 1112, the electronic device (101) may execute a recommendation function based on a selection by the user to execute the recommendation function.

[0184] In operation 1120, the electronic device (101) may display an interface asking for satisfaction with the execution of the recommended function.

[0185] In operation 1122, the electronic device (101) may generate a routine to execute a recommended function upon a context change based on receiving a response that satisfies the recommended function.

[0186] According to one embodiment, the electronic device (101) may execute actions that have been cumulatively executed a specified number of times or more based on a detected change in context. The electronic device (101) may provide a notification asking whether the user is satisfied with the action executed along with the change in context, and may generate a routine that executes a recommendation function based on the user's response that they are satisfied with the action executed along with the change in context.

[0187] In operation 1124, the electronic device (101) may not generate a routine to execute the recommended function upon a context change based on receiving a response indicating that the recommended function is not satisfactory.

[0188] In one embodiment, the electronic device (101) may control not to generate a routine for executing a recommendation function based on a user's response indicating dissatisfaction. The electronic device (101) may collect user feedback to determine whether to execute an automated object (e.g., a routine for executing a recommendation function).

[0189] According to one embodiment, the electronic device (101) collects information related to at least one operation performed by a user input when a specified condition is satisfied for a specified period of time based on the time at which an application for executing a video is executed, and analyzes the collected information to determine an operation performed a specified number of times or more when the application for executing the video is executed. Thereafter, the electronic device (101) displays a user interface asking whether to execute the operation performed a specified number of times or more together based on the execution of the application for executing the video, and when the application for executing the video is executed based on the user input for the user interface, the electronic device can control to execute the operation performed a specified number of times or more together.

[0190] According to one embodiment, the electronic device (101) can detect a connection to an external electronic device using Bluetooth or Wi-Fi, and collect user behavioral data for a specified period of time based on the time of connection with the external electronic device. The electronic device (101) can identify, among the collected behavioral data, actions that have been cumulatively performed a specified number of times or more over a certain period of time, and display an interface that asks whether to execute the actions that have been cumulatively performed a specified number of times or more when connected to the external electronic device.

[0191] According to one embodiment, the electronic device (101) may execute an operation that has been cumulatively executed a specified number of times or more when connected to the external electronic device based on a user input to the interface. The electronic device (101) may provide a notification asking whether the user is satisfied with the operation that has been executed together, generate a routine for executing a recommendation function based on a user's response that the user is satisfied with the operation that has been executed together, and control not to generate a routine for executing the recommendation function based on a user's response that the user is not satisfied.

[0192] According to one embodiment, the electronic device (101) can detect whether the user arrives at or leaves a designated location using GPS, and collect behavioral data of the user of the electronic device for a designated period of time based on the time of arrival or departure at the designated location. The electronic device (101) can identify, among the collected behavioral data, actions that have been cumulatively performed a specified number of times or more over a certain period of time, and display an interface that asks whether to perform the actions that have been cumulatively performed a specified number of times or more when the electronic device (101) arrives at or leaves the designated location.

[0193] According to one embodiment, the electronic device (101) may execute a motion that has been cumulatively executed a specified number of times or more based on a user input to the interface when the electronic device detects that the electronic device has arrived at or left a designated location. The electronic device (101) may provide a notification asking whether the user is satisfied with the executed motion, generate a routine for executing a recommendation function based on a user's response that the user is satisfied with the executed motion, and control not to generate a routine for executing the recommendation function based on a user's response that the user is not satisfied.

[0194] According to one embodiment, the electronic device (101) may determine whether the battery charge of the electronic device is below a specified level based on whether a measurement value of a sensor satisfies a preset level, determine whether the user of the electronic device (101) is exercising, determine whether the user of the electronic device (101) is waking up from sleep, or determine whether the battery of the electronic device (101) is being charged. If the electronic device (101) or the state of the user is determined to be in a preset state, the electronic device (101) may collect behavioral data of the user of the electronic device (101) for a specified period of time. The electronic device (101) may check, among the collected behavioral data, actions that have been cumulatively executed a specified number of times or more for a certain period of time, and if the state of the electronic device or the user is determined to be in a preset state, display an interface that asks whether to execute the actions that have been cumulatively executed a specified number of times or more together.

[0195] According to one embodiment, the electronic device (101) may execute an operation that has been cumulatively executed a specified number of times or more when the state of the electronic device or the user is determined to be a preset state based on a user input to the interface. The electronic device (101) may provide a notification asking whether the user is satisfied with the operation that has been executed together, generate a routine for executing a recommendation function based on a user's response that the user is satisfied with the operation that has been executed together, and control not to generate a routine for executing the recommendation function based on a user's response that the user is not satisfied.

[0196] According to one embodiment, the electronic device (101) may provide a user interface that asks whether to execute an operation accumulated a specified number of times or more when the specified condition is detected, and may execute the operation accumulated a specified number of times or more when the specified condition is detected based on a user input to the user interface. The electronic device (101) may generate a routine that executes a recommendation function based on a user's response that they are satisfied with the execution of the operation accumulated a specified number of times or more, and may automatically execute the operation accumulated a specified number of times or more when the specified condition is detected, and may control the operation accumulated a specified number of times or more to be terminated when the specified condition is not detected.

[0197] According to one embodiment, the designated time for collecting user behavioral data of an electronic device may vary depending on the type of designated condition. At least one action performed by the user's input when the designated condition is met may also vary depending on the type of designated condition. The designated number of actions used to determine whether an action has accumulated more than a designated number of times may vary depending on the type of designated condition.

[0198] The embodiments of this document disclosed in this specification and drawings are merely specific examples to easily explain the technical contents according to the embodiments of this document and to help understand the embodiments of this document, and are not intended to limit the scope of the embodiments of this document. Therefore, the scope of one embodiment of this document should be interpreted to include all changes or modified forms derived based on the technical idea of ​​one embodiment of this document, in addition to the embodiments disclosed herein.

Claims

1. In electronic devices, A memory that stores instructions and includes one or more storage media; At least one processor comprising processing circuitry, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Verifying that a specified condition is satisfied, including at least one of the following actions: a specific application is executed, an external electronic device is connected, a sensor measurement value meets a preset level, or the electronic device is located at a specified location; Determine the response time based on the type of condition that is satisfied, and collect the user's behavior data of the electronic device during the determined time, Determine information related to at least one action performed by the user's input when a specified condition is met based on the user's behavior data collected during the above-determined time, When the above specified condition is satisfied, extract the user's behavior pattern from among the information related to at least one action performed by the user's input, and determine the user-customized action based on the frequency of the pattern's appearance, the consistency of the pattern, and the duration of the pattern. Provide a user interface that asks whether to execute the above-mentioned customized action together when the above-mentioned specified condition is detected; An electronic device that controls the execution of the user-customized action when the specified condition is detected based on user input to the user interface.

2. In paragraph 1, The above specific application is Includes an application that runs the video, Information related to at least one action performed by user input when the above specified condition is met Contains information about at least one of the following: the sound volume of the above video, the screen size, whether the screen automatically rotates, whether the do not disturb function is running, or the screen-on time. The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Collect information related to at least one action performed by user input when the specified condition is met for a specified period of time based on the time at which the application executing the above video is executed, By analyzing the collected information, the application that runs the video determines whether an action has been performed more than a specified number of times. Display a user interface asking whether to perform an action that has been performed a specified number of times or more based on the application executing the above video; An electronic device that controls the execution of an operation performed a specified number of times or more when an application that executes the image is executed based on a user input to the user interface.

3. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Detects connection to external electronic devices using Bluetooth or Wi-Fi, Collects user behavior data for a specified period of time based on the time of connection with the external electronic device. Among the collected behavioral data, check the actions that have been executed more than a specified number of times over a certain period of time. Controls the display of an interface asking whether to execute an operation that has been executed more than a specified number of times when connected to the external electronic device; The above specified time An electronic device characterized in that it changes based on the satisfaction of any of the specified conditions including at least one of: a specific application being executed, a connection to an external electronic device, a sensor measurement value meeting a preset level, a specific time zone, the location of the electronic device being located in a specified location, data usage, light level, whether a specified application is running, whether a specified application is terminated, or whether a connection to an external electronic device is made.

4. In paragraph 3, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to When connected to the external electronic device based on user input to the above interface, the operation is executed together with the accumulated number of times specified, Create a routine that provides a notification asking whether the user is satisfied with the action being performed together, and executes a recommendation function based on the user's response that they are satisfied with the action being performed together. An electronic device that controls the generation of a routine that executes a recommendation function based on a user's response that he or she is not satisfied with.

5. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Detecting whether you arrive at or leave a designated location using either GPS, Wi-Fi AP, or network-based location positioning (NLP), Collecting user behavioral data of the electronic device for a specified period of time based on the time of arrival at or departure from a specified location; Among the collected behavioral data, check the actions that have been executed more than a specified number of times over a certain period of time. An electronic device that controls an interface to ask whether to perform an action that has been performed cumulatively a specified number of times when the electronic device arrives at or leaves a specified location.

6. In paragraph 5, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to When the electronic device is detected to arrive at or leave a designated location based on user input to the above interface, the actions that have been executed cumulatively a specified number of times are executed together, Create a routine that provides a notification asking whether the user is satisfied with the action being performed together, and executes a recommendation function based on the user's response that they are satisfied with the action being performed together. An electronic device that controls the generation of a routine that executes a recommendation function based on a user's response that he or she is not satisfied with.

7. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Based on whether the sensor's measurement value satisfies a preset level. Determine whether the battery charge of the electronic device is below a specified level, or Determine whether the user of the above electronic device is in a state of exercise, or Determine whether the user of the above electronic device is awake or asleep; or Determine whether the battery of the above electronic device is charging, When the state of the electronic device or the user is confirmed to be in a preset state, the user's behavioral data of the electronic device is collected for a specified period of time, Among the collected behavioral data, check the actions that have been executed more than a specified number of times over a certain period of time. An electronic device that controls an interface to ask whether to execute an action that has been executed cumulatively a specified number of times or more when the state of the electronic device or the user is determined to be a preset state.

8. In paragraph 7, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to When the state of the electronic device or the user is determined to be a preset state based on user input to the interface, the actions that have been cumulatively executed a specified number of times or more are executed together. Create a routine that provides a notification asking whether the user is satisfied with the action being performed together, and executes a recommendation function based on the user's response that they are satisfied with the action being performed together. An electronic device that controls the generation of a routine that executes a recommendation function based on a user's response that he or she is not satisfied with.

9. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to Provide a user interface that asks whether to execute the above-mentioned customized action together when the above-mentioned specified condition is detected; Executes the user-customized action when the above-specified condition is detected based on user input to the above-mentioned user interface; Generate a routine that executes a recommendation function based on the user's response that he or she is satisfied with the execution of the above-mentioned customized action, If the above specified conditions are detected, the above user-customized action is automatically executed. An electronic device that controls the termination of the above user-customized action in situations where a specified condition is not detected.

10. In paragraph 1, The time designated to collect the user's behavioral data of the above electronic device is determined differently depending on the type of specified condition, At least one action performed by the user's input when the above specified condition is met is also determined differently depending on the type of the specified condition. An electronic device in which the specified number of times used to determine whether an action has accumulated more than a specified number of times is determined differently depending on the type of specified condition.

11. In the method of operating an electronic device, An action to determine whether a specified condition is satisfied, including at least one of the following actions: a specific application is executed, an external electronic device is connected, a sensor measurement value meets a preset level, or the electronic device is located at a specified location; An action of determining a response time based on the type of condition being satisfied and collecting behavioral data of a user of the electronic device during the determined time; An action of determining information related to at least one action performed by a user's input when a specified condition is met based on the user's behavioral data collected during the determined time; An action of extracting a user's behavior pattern from among information related to at least one action performed by a user's input when the above-mentioned condition is met, and determining a user-tailored action based on the frequency with which the pattern appears, the consistency of the pattern, and the duration of the pattern; Actions that provide a user interface asking whether to execute the above-mentioned customized action together when the above-mentioned specified condition is detected; and A method comprising an action for executing said user-customized action when said specified condition is detected based on user input to said user interface.

12. In paragraph 11, The above specific application is Includes an application that runs the video, Information related to at least one action performed by user input when the above specified condition is met Contains information about at least one of the following: the sound volume of the above video, the screen size, whether the screen automatically rotates, whether the do not disturb function is running, or the screen-on time. The method of operation of the above electronic device is An action of collecting information related to at least one action performed by a user's input when a specified condition is met for a specified period of time based on the time at which the application executing the video is executed; An action of analyzing the collected information and determining an action that has been performed more than a specified number of times when the application executing the video is executed; An action that displays a user interface asking whether to perform an action that has been performed a specified number of times or more based on the application executing the above video; and A method further comprising executing an action that has been performed a specified number of times or more when an application that executes the image is executed based on a user input to the user interface.

13. In paragraph 11, An action of detecting a connection to an external electronic device using Bluetooth or Wi-Fi and collecting user behavior data for a specified period of time based on the time of connection to the external electronic device; An action to check the actions that have been executed more than a specified number of times over a certain period of time among the collected action data; and Further comprising an action of displaying an interface asking whether to execute an action that has been executed cumulatively a specified number of times or more when connected to the external electronic device, The above specified time A method characterized in that the change is based on which of the specified conditions is satisfied, including at least one of: a specific application is running, an external electronic device is connected, a sensor measurement value meets a preset level, a specific time zone, the electronic device is located at a specified location, data usage, light level, whether a specified application is running, whether a specified application is terminated, or whether a connection to an external electronic device is made.

14. In paragraph 13, An action of executing an action that has been cumulatively executed a specified number of times or more when connected to the external electronic device based on a user input to the interface; An action that provides a notification asking whether the user is satisfied with the action being performed together, and creates a routine that executes a recommendation function based on the user's response that he or she is satisfied with the action being performed together; and A method further comprising an action to control not to create a routine that executes a recommendation function based on a user's response that he or she is not satisfied.

15. In paragraph 11, An action that detects whether a user has arrived at or left a designated location using either GPS, Wi-Fi AP, or network-based location positioning (NLP); An action to collect behavioral data of a user of the electronic device for a specified period of time based on the time of arrival at or leaving a specified location; An action to check the actions that have been executed more than a specified number of times over a certain period of time among the collected action data; and A method further comprising the action of displaying an interface asking whether to perform an action that has been performed cumulatively a specified number of times when the electronic device arrives at or leaves a designated location.

Images