Method for providing routine and electronic device supporting the routine

Abstract

A method for providing a routine and an electronic device supporting the routine. The electronic device includes: a user interface; at least one processor operably connected to the user interface; and a memory operably connected to the at least one processor, wherein the memory is configured to store instructions that, when executed, cause the at least one processor to perform the following operations: when the electronic device is in a first state, determine whether a first condition is satisfied; if the first condition is satisfied, perform a first action to change the state of the electronic device from the first state to the second state; through the user interface, receive user input to change the state of the electronic device from the second state to the third state; based at least in part on the received user input, change the state of the electronic device from the second state to the third state; and maintain the third state even if the first condition is not satisfied in the third state.

Description

Technical Field

[0001] The present disclosure relates to a method for providing a routine and an electronic device supporting the routine. Background Art

[0002] Recently, technologies related to artificial intelligence (AI) systems have been developed. AI systems may refer to computer systems for achieving human-like intelligence, in which the recognition rate increases in proportion to the degree of autonomous learning, judgment, and use performed by the machine.

[0003] Electronic devices can use such AI systems to determine the user's pattern (e.g., life pattern, electronic device usage pattern) and provide personalized services based on the user's pattern. For example, a user can configure desired conditions and desired operations through an electronic device so that if the user's desired conditions are met, the electronic device performs the user's desired operations. Electronic devices can use AI systems so that when the electronic device has confirmed that the user-configured conditions are met, the electronic device performs the configured operations in response to the configured conditions.

[0004] In the case of a conventional electronic device, if the configured condition is no longer satisfied (or if the condition is lifted) after the electronic device has performed an operation corresponding to the condition configured by the user, the electronic device does not perform any operation for changing (or restoring) the state of the electronic device to the state of the electronic device before performing the operation. Summary of the invention

[0005] One aspect of the present disclosure provides a method for providing a routine and an electronic device supporting the method, wherein, if a condition configured by a user is no longer satisfied after the electronic device has performed an operation corresponding to the configured condition, the state of the electronic device can be changed to the state of the electronic device before the operation was performed.

[0006] According to one aspect of the present disclosure, an electronic device is provided. The electronic device includes: a user interface; at least one processor operably connected to the user interface; and a memory operably connected to the at least one processor, wherein the memory is configured to store instructions that, when executed, cause the at least one processor to perform the following operations: when the electronic device is in a first state, determine whether a first condition is satisfied; if the first condition is satisfied, perform a first action that changes the state of the electronic device from the first state to the second state; through the user interface, receive a user input that changes the state of the electronic device from the second state to the third state; at least partially based on the received user input, change the state of the electronic device from the second state to the third state; and maintain the third state even if the first condition is not satisfied in the third state.

[0007] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes: a user interface; at least one processor operably connected to the user interface; and a memory operably connected to the at least one processor, wherein the memory is configured to store instructions that, when executed, cause the at least one processor to perform the following operations: when the electronic device is in a first state, determine whether a first condition is satisfied; if the first condition is satisfied, perform a first action to change the state of the electronic device from the first state to the second state; when the electronic device is in a second state, determine whether a second condition is satisfied; if the second condition is satisfied, perform a second action to change the state of the electronic device from the second state to the third state; when the electronic device is in a third state, if the second condition is satisfied but the first condition is no longer satisfied, maintain the state of the electronic device in the third state; and when the electronic device is in a third state, if the first condition and the second condition are no longer satisfied, restore the state of the electronic device from the third state to the first state.

[0008] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes: a communication interface configured to perform wireless communication; at least one processor operably connected to the communication interface; and a memory operably connected to the at least one processor, wherein the memory is configured to store instructions that, when executed, cause the at least one processor to perform the following operations: receiving information related to a first routine including a first action to be performed when a first condition is satisfied from an external electronic device by using the communication interface; determining, based at least in part on the received information related to the first routine, whether the first action can be performed when the electronic device is in a first state when the first condition is satisfied; if the first action can be performed, storing the first routine and the first state in the memory; after storing the first routine and the first state, executing a first action for changing the first state to a second state; and after the first action is performed, returning the electronic device to the first state. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is a block diagram of an electronic device in a network environment according to an embodiment;

[0011] Figure 2 is a block diagram of an integrated intelligent system according to an embodiment;

[0012] Figure 3is a diagram showing a type of storing information on a relationship between concepts and actions in a database according to an embodiment;

[0013] Figure 4 is a diagram of an electronic device showing a screen for processing a voice input received through a smart application (app) according to an embodiment;

[0014] Figure 5 is a block diagram of a system for providing routines according to an embodiment;

[0015] Figure 6 is a block diagram of a method for executing a routine of an electronic device according to an embodiment;

[0016] Figure 7 is a flowchart of a method for restoring a state of an electronic device after executing a routine according to an embodiment;

[0017] Figure 8 is a flowchart of a method for restoring a state of an electronic device after executing a plurality of routines according to an embodiment;

[0018] Fig. 9 is a diagram of a method for restoring a state of an electronic device after executing a plurality of routines according to an embodiment;

[0019] Fig.10 is a flowchart of a method for providing, by an electronic device, a routine based on information related to the routine received from another electronic device according to an embodiment;

[0020] Fig.11 is a flowchart of a method for providing, by an electronic device, a routine based on information related to the routine received from another electronic device according to an embodiment;

[0021] Fig.12 is a flowchart of a method for recommending a routine for an electronic device according to an embodiment;

[0022] Fig.13 is a flowchart of a method for determining a recommendation priority of an electronic device according to an embodiment;

[0023] Fig.14 is a flow chart of a method for recommending an action based at least in part on an already generated routine of an electronic device according to an embodiment;

[0024] Fig.15 is a flowchart of a method for displaying an icon in a locked state of an electronic device according to an embodiment;

[0025] Fig.16 is an illustration of a routine associated with a hardware key of an electronic device according to an embodiment;

[0026] Fig.17 is an illustration of a method for providing a routine based on information related to the routine received from another electronic device according to an embodiment;

[0027] Fig.18 is an illustration of a method for providing a routine based on information related to the routine received from another electronic device according to an embodiment;

[0028] Fig.19 is an illustration of a method for providing a routine related to a location of an electronic device according to an embodiment;

[0029] Fig. 20 is an illustration of a method for providing a routine related to a location of an electronic device according to an embodiment;

[0030] Fig.21 is an illustration of a method for configuring a routine of an electronic device according to an embodiment;

[0031] Fig. 22 is an illustration of a method for configuring a routine of an electronic device according to an embodiment;

[0032] Fig.23 is an illustration of a method for configuring a routine of an electronic device according to an embodiment;

[0033] Fig.24 is a diagram of a method for displaying an icon in a locked state of an electronic device according to an embodiment;

[0034] Fig.25 is an illustration of a method for displaying information related to a routine in a locked state of an electronic device according to an embodiment;

[0035] Fig.26 is an illustration of a method for generating a routine related to a lock state of an electronic device according to an embodiment; and

[0036] Fig. 27 is an illustration of a method for recommending a routine of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0037] Figure 1 is a block diagram of an electronic device 101 in a network environment 100 according to an embodiment.

[0038] The electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). The electronic device 101 may communicate with the electronic device 104 via the server 108. The electronic device 401 may include a processor 120, a memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. At least one of the components (e.g., the display device 160 or the camera module 180) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. Some of the components may be implemented as a single integrated circuit.For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented to be embedded in the display device 160 (eg, a display).

[0039] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. As at least part of the data processing or calculation, the processor 120 may load commands or data received from another component (e.g., sensor module 176 or communication module 190) into the volatile memory 132, process the commands or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. The processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operationally independent or combined with the main processor 121. Additionally or alternatively, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121 , or as part of the main processor 121 .

[0040] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) among the components of the electronic device 101, or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) together with the main processor 121. The auxiliary processor 123 (e.g., ISP or CP) may be implemented as part of another component (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor 123.

[0041] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0042] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0043] The input device 150 may receive commands or data from outside the electronic device 101 (e.g., a user) to be used by other components of the electronic device 101 (e.g., the processor 120). The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0044] The sound output device 155 can output a sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records, and the receiver can be used for incoming calls. The receiver can be implemented as a separate part from the speaker, or as a part of the speaker.

[0045] The display device 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. The display device 160 may include a touch circuit adapted to detect a touch or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0046] The audio module 170 can convert sound into an electrical signal, or vice versa. The audio module 170 can obtain sound via the input device 150, or output sound via the sound output device 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0047] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. The sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0048] The interface 177 may support one or more designated protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly with an external electronic device (e.g., the electronic device 102). The interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0049] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. The connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0050] The haptic module 179 may convert the electric signal into mechanical stimulation (eg, vibration or motion) or electric stimulation that can be recognized by the user via tactile or kinesthetic sense. The haptic module 179 may include, for example, a motor, a piezoelectric element, or an electric stimulator.

[0051] The camera module 180 may capture still images or moving images. The camera module 180 may include one or more lenses, image sensors, ISPs, or flashes.

[0052] The power management module 188 may manage power supply to the electronic device 101. The power management module 188 may be implemented as, for example, at least part of a power management integrated circuit (PMIC).

[0053] The battery 189 may power at least one component of the electronic device 101. The battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0054] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and the electronic device 102, the electronic device 104, or the server 108, and perform communication via the established communication channel. The communication module 190 may include one or more CPs that can operate independently with the processor 120 (e.g., AP) and support direct (e.g., wired) communication or wireless communication. The communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate via a first network 198 (e.g., a short-range communication network such as Bluetooth TM , Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA) standards) or a second network 199 (e.g., a long-distance communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN)))) to communicate with an external electronic device. These various types of communication modules may be implemented as a single component (e.g., a single integrated circuit or chip), or may be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 may use user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the SIM 196 to identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199).

[0055] The antenna module 197 may transmit a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). The antenna module 197 may include an antenna including a radiating element composed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). The antenna module 197 may include a plurality of antennas. In such a case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. Another component (e.g., a radio frequency integrated circuit (RFIC)) other than the above-mentioned radiating element may be additionally formed as a part of the antenna module 197.

[0056] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0057] A command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 and the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. All or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the external server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service without executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0058] The electronic device may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. However, the electronic device is not limited to those electronic devices described above.

[0059] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the present disclosure to specific embodiments, but should be understood to include various changes, equivalents or alternative forms for the corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the noun corresponding to the project in the singular form may include one or more things, unless the relevant context has other clear instructions. As used herein, each phrase in the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st", "2nd", "first" and "second" may only be used to simply distinguish the corresponding parts from another part, but do not limit the parts in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it indicates that the element may be directly (e.g., wired) combined with the other element, wirelessly combined with the other element, or combined with the other element via a third element.

[0060] As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or the smallest unit or portion of the single integrated component. For example, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0061] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Wherein, the term "non-transitory" only indicates that the storage medium is a tangible device, but does not include signals (e.g., electromagnetic waves), and here, the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0062] The method according to the embodiment of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., PlayStore). TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0063] Each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities. One or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may still perform the one or more functions in the same or similar manner as one or more functions of each component in the multiple components performed before integration with a corresponding component in the multiple components. The operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0064] Figure 2 is a block diagram of an integrated intelligent system 200 according to an embodiment.

[0065] Reference Figure 2 , the integrated intelligent system 200 may include an electronic device 201 , an intelligent server 208 and a service server 300 .

[0066] The electronic device 201 may be a user terminal device (or electronic device) that can connect to the Internet, and may be, for example, a portable phone, a smart phone, a personal digital assistant (PDA), a netbook computer, a television (TV), a home appliance, a wearable device, a head mounted device (HMD), or a smart speaker.

[0067] The electronic device 201 may include a communication interface 210, a microphone 220, a speaker 230, a display 240, a memory 250, or a processor 260. The enumerated constituent elements may be operably connected or electrically connected to each other.

[0068] The communication interface 210 may be configured to connect to an external device and send and / or receive data to and / or from an external device. The microphone 220 may receive sound (e.g., user speech) and convert the sound into an electrical signal. The speaker 230 may output the electrical signal as sound (e.g., voice). The display 240 may be configured to display an image or video. The display 240 may also display a graphical user interface (GUI) of an executed app (or application).

[0069] The memory 250 may store a client module 251, a software development kit (SDK) 253, and a plurality of apps 255. The client module 251 and the SDK 253 may configure a framework (or solution program) for performing general functions. In addition, the client module 251 or the SDK 253 may configure a framework for processing voice input.

[0070] The plurality of apps 255 stored in the memory 250 may be programs for executing a specified function. The plurality of apps 255 may include a first app 255_1 and a second app 255_2. The plurality of apps 255 may each include a plurality of actions for executing a specified function. For example, the apps may include an alarm app, a message app, and / or a schedule app. The plurality of apps 255 may be executed by the processor 260 and at least some of the plurality of actions may be sequentially executed.

[0071] The processor 260 may control general operations of the electronic device 201. For example, the processor 260 may be electrically connected to the communication interface 210, the microphone 220, the speaker 230, and the display 240, and perform designated operations.

[0072] The processor 260 may also execute the program stored in the memory 250 and perform the designated function. For example, the processor 260 may execute at least one of the client module 251 or the SDK 253 and perform subsequent operations for processing the voice input. The processor 260 may control the operation of the plurality of apps 255, for example, through the SDK 253. The operations of the client module 251 or the SDK 253 described below may be operations performed by the processor 260.

[0073] The client module 251 may receive a voice input. For example, the client module 251 may receive a voice signal corresponding to a user's speech sensed by the microphone 220. The client module 251 may send the received voice input to the smart server 208. The client module 251 may send the state information of the electronic device 201 together with the received voice input to the smart server 208. The state information may be, for example, app execution state information.

[0074] The client module 251 may receive a result corresponding to the received voice input. For example, in response to the intelligent server 208 calculating a result corresponding to the received voice input, the client module 251 may receive a result corresponding to the received voice input from the intelligent server 208. The client module 251 may display the received result on the display 240.

[0075] The client module 251 may receive a plan corresponding to the received voice input. The client module 251 may display the results of executing multiple actions of the app according to the plan on the display 240. For example, the client module 251 may sequentially display the execution results of the multiple actions on the display. For example, the electronic device 201 may display only part of the results of executing the multiple actions (e.g., the result of the last operation) on the display.

[0076] The client module 251 may receive a request for obtaining information required for calculating a result corresponding to the voice input from the intelligent server 208. In response to the request, the client module 251 may send the required information to the intelligent server 208.

[0077] The client module 251 can send the result information of executing the multiple actions according to the plan to the intelligent server 208. By using the result information, the intelligent server 208 can recognize that the received voice input is processed correctly.

[0078] The client module 251 may include a voice recognition module. The client module 251 may recognize a voice input for executing a limited function through the voice recognition module. For example, the client module 251 may execute a smart app for processing voice input to execute the operation of the system through a specified input (e.g., "wake up!").

[0079] The intelligent server 208 may receive information related to the user voice input from the electronic device 201 through the communication network. The intelligent server 208 may convert the data related to the received voice input into text data. The intelligent server 208 may generate a plan for executing a task corresponding to the user voice input based on the text data.

[0080] The plan can be generated by an AI system. The AI ​​system can also be a rule-based system, and can also be a neural network-based system (e.g., a feedforward neural network (FNN)) and / or a recursive neural network (RNN). In addition, the AI ​​system can also be a combination of the aforementioned systems, or the AI ​​system can also be an AI system different from the aforementioned systems. The plan can be selected from a set of predefined plans, or can be generated in real time in response to a user request. For example, the AI ​​system can select at least one of a plurality of predefined plans.

[0081] The intelligent server 208 may transmit the result of the generated plan to the electronic device 201, or transmit the generated plan to the electronic device 201. The electronic device 201 may display the result of the plan on the display 240. The electronic device 201 may display the result of the action of executing the plan on the display 240.

[0082] The intelligent server 208 may include a front end 210 - 1 , a natural language platform 220 - 1 , a capsule database (DB) 230 - 1 , an execution engine 240 - 1 , an end user interface 250 - 1 , a management platform 260 - 1 , a big data platform 270 - 1 , or an analysis platform 280 - 1 .

[0083] The front end 210-1 may receive a voice input received from the electronic device 201. The front end 210-1 may send a response corresponding to the voice input.

[0084] The natural language platform 220 - 1 may include an automatic speech recognition module (ASR module) 221 - 1 , a natural language understanding module (NLU module) 223 - 1 , a planner module 225 - 1 , a natural language generator module (NLG) module 227 , or a text to speech (TTS) module 229 - 1 .

[0085] The automatic speech recognition module 221-1 can convert the speech input received from the electronic device 201 into text data. By using the text data of the speech input, the natural language understanding module 223-1 can understand the user's intention. For example, by performing grammatical analysis or semantic analysis, the natural language understanding module 223-1 can understand the user's intention. By using the language features (e.g., grammatical factors) of morphemes or phrases, the natural language understanding module 223-1 can understand the meaning of the words extracted from the speech input, and match the meaning of the understood words with the user's intention to identify the user's intention.

[0086] By using the intention and parameters identified by the natural language understanding module 223-1, the planner module 225-1 can generate a plan. Based on the identified intention, the planner module 225-1 can identify multiple domains required for performing the task. The planner module 225-1 can identify multiple actions included in each of the multiple domains identified based on the intention. The planner module 225-1 can identify the parameters required for performing the identified multiple actions, or identify the result values ​​output by the execution of multiple actions. The parameters and result values ​​can be defined with the concept of a specified form (or class). The plan can include multiple concepts and multiple actions identified by the user's intention. The planner module 225-1 can gradually (or hierarchically) identify the relationship between multiple actions and multiple concepts. For example, based on multiple concepts, the planner module 225-1 can identify the execution order of multiple actions identified based on the user's intention. In other words, the planner module 225-1 can identify the execution order of multiple actions based on the parameters required for performing multiple actions and the results output by performing multiple actions. The planner module 225-1 may generate a plan including association information (eg, ontology) between a plurality of actions and a plurality of concepts. The planner module 225-1 may generate a plan by using information stored in the capsule database 230-1, wherein the capsule database 230-1 stores a set of relationships between concepts and actions.

[0087] The natural language generator module 227-1 can convert the specified information into text form. The information converted into text form can be in the form of natural language speech. The text-to-speech module 229-1 can convert the information in text form into information in speech form.

[0088] Part or all of the functions of the natural language platform 220 - 1 may also be implemented in the electronic device 201 .

[0089] The capsule database 230-1 may store information about the relationship between a plurality of concepts and actions corresponding to a plurality of domains. The capsule may include a plurality of action objects (or action information) and concept objects (or concept information) included in the plan. The capsule database 230-1 may store a plurality of capsules in the form of a concept action network (CAN). A plurality of capsules may be stored in a function registry included in the capsule database 230-1.

[0090] The capsule database 230-1 may include a policy registry storing policy information necessary for identifying a plan corresponding to the voice input. The policy information may include reference information for identifying a plan in response to the presence of multiple plans corresponding to the voice input. The capsule database 230-1 may include a subsequent registry storing subsequent operation information for proposing subsequent operations to the user under specified conditions. The subsequent operation may include, for example, a subsequent utterance. The capsule database 230-1 may include a layout registry storing layout information of information output by the electronic device 201. The capsule database 230-1 may include a vocabulary storing vocabulary information included in the capsule information. The capsule database 230-1 may include a dialogue registry storing user dialogue (or interaction) information. The capsule database 230-1 may update objects stored by the developer tool. The developer tool may include, for example, a function editor for updating an action object or a concept object. The developer tool may include a vocabulary editor for updating a vocabulary. The developer tool may include a policy editor for generating and registering a policy for identifying a plan. The developer tool may include a dialogue editor for generating a dialogue with the user. The developer tool may include a subsequent editor that may edit a subsequent voice that activates a subsequent target and provides a prompt. The subsequent target may be identified based on the currently set target, the user's preference, or environmental conditions. The capsule database 230 - 1 may also be implemented in the electronic device 201 .

[0091] The execution engine 240-1 can calculate the result by using the generated plan. The end user interface 250-1 can send the calculated result to the electronic device 201. The electronic device 201 can receive the result and provide the received result to the user. The management platform 260-1 can manage the information used in the intelligent server 208. The big data platform 270-1 can collect the user's data. The analysis platform 280-1 can manage the quality of service (QoS) of the intelligent server 208. For example, the analysis platform 280-1 can manage the elements and processing speed (or efficiency) of the intelligent server 208.

[0092] The service server 300 may provide a specified service (e.g., food order or hotel reservation via capsule (CP) service A 301, CP service B 302, and CP service C 303) to the electronic device 201. The service server 300 may be a server managed by a third party. The service server 300 may provide information for generating a plan corresponding to the received voice input to the smart server 208. The provided information may be stored in the capsule database 230-1. In addition, the service server 300 may provide the result information of the plan to the smart server 208.

[0093] In the above integrated intelligent system 200, in response to user input, the electronic device 201 can provide various intelligent services to the user. The user input may include, for example, input through a physical button, touch input, or voice input.

[0094] The electronic device 201 can provide a voice recognition service through a smart app (or voice recognition application) stored therein. In this case, for example, the electronic device 201 can recognize a user's speech or voice input received through the microphone 220, and provide the user with a service corresponding to the recognized voice input.

[0095] The electronic device 201 may perform a designated operation based on the received voice input, alone or together with the smart server 208 and / or the service server 300. For example, the electronic device 201 may execute an app corresponding to the received voice input and perform the designated operation through the executed app.

[0096] In response to the electronic device 201 providing a service together with the smart server 208 and / or the service server 300, the electronic device 201 can sense the user's speech by using the microphone 220 and generate a signal (or voice data) corresponding to the sensed user's speech. The electronic device 201 can send the voice data to the smart server 208 by using the communication interface 210.

[0097] In response to the voice input received from the electronic device 201, the intelligent server 208 can generate a plan for performing a task corresponding to the voice input, or generate a result of performing an action according to the plan. The plan can include, for example, a plurality of actions for performing a task corresponding to the user's voice input and a plurality of concepts related to the plurality of actions. The concept can be a definition of a parameter input by performing the plurality of actions or a result value output by performing the plurality of actions. The plan can include association information between the plurality of actions and the plurality of concepts.

[0098] The electronic device 201 may receive the response by using the communication interface 210. The electronic device 201 may output a voice signal generated by the electronic device 201 to the outside by using the speaker 230, or output an image generated by the electronic device 201 to the outside by using the display 240.

[0099] Figure 3 is a diagram illustrating a type of information on a relationship between concepts and actions stored in a database according to an embodiment.

[0100] Reference Figure 3 The capsule database 230-1 of the intelligent server 208 may store capsules in the form of CAN. The capsule database may store actions for processing tasks corresponding to the user's voice input and parameters required for the actions in the form of CAN.

[0101] The capsule database may store a plurality of capsules (i.e., capsule A 401 and capsule B 404) corresponding to each of a plurality of domains (e.g., applications). One capsule (e.g., capsule A 401) may correspond to one domain (e.g., location (geographic location) and / or application). In addition, one capsule may correspond to at least one service provider (e.g., CP 1 402, CP 2 403, CP 4 405, or CP 3 406) for performing a function of the domain related to the capsule. One capsule may include at least one or more actions 410 and at least one or more concepts 420 for performing a specified function.

[0102] By using capsules stored in the capsule database, the natural language platform 220-1 can generate a plan for performing a task corresponding to the received speech input. For example, by using capsules stored in the capsule database, the planner module 225-1 of the natural language platform 220-1 can generate a plan. For example, the planner module 225-1 can generate a plan 407 by using actions 4011 and 4013 and concepts 4012 and 4014 of capsule A 401, and action 4041 and concept 4042 of capsule B 404.

[0103] Figure 4 is a diagram 400 of the electronic device 201 displaying a screen for processing a voice input received through a smart app according to an embodiment.

[0104] Reference Figure 4 In order to process user input through the smart server 208, the electronic device 201 can execute a smart app.

[0105] In screen 431, in response to recognizing a specified voice input (e.g., "Wake up!") or receiving an input through a hardware key (e.g., a dedicated hardware key), the electronic device 201 can execute a smart app for processing the voice input. The electronic device 201 can execute the smart app, for example, while executing a schedule app. The electronic device 201 can display an object (e.g., icon) 435 corresponding to the smart app on the display 240. The electronic device 201 can receive user input through the user's voice. For example, the electronic device 201 can receive a voice input "Let me know the schedule for this week!". The electronic device 201 can display a user interface (UI) 437 (e.g., an input window) of the smart app on the display 240, wherein the text data of the received voice input is displayed in the UI 437.

[0106] In screen 433, the electronic device 201 may display a result corresponding to the received voice input on the display. For example, the electronic device 201 may receive a plan corresponding to the received user input and display "this week's schedule" on the display 240 according to the plan.

[0107] Figure 5 is a block diagram of a system 500 for providing routines, according to an embodiment.

[0108] Hereinafter, a “condition” may refer to a state (or context) of the electronic device 101 (or a user of the electronic device 101 ) configured to cause the electronic device 101 to perform an operation.

[0109] Hereinafter, an "action" may refer to an operation of the electronic device 101 performed when the electronic device 101 satisfies a condition. The action may be configured to correspond to (or be interconnected with) a condition.

[0110] The term "routine" may refer to a program including definitions of conditions and actions corresponding to the conditions.

[0111] The routine may be configured by a user of the electronic device 101. For example, at least one of the condition or the action may be configured by the user of the electronic device 101. However, this example is not limited in any way, and the routine may be configured by a designer of the electronic device 101 or a developer of an application.

[0112] Reference Figure 5 , a system 500 for providing routines may include an electronic device 101, a second electronic device 103 (or electronic devices 102 and 104), and an intelligent server 108-1 and a recommendation server 108-2.

[0113] The electronic device 101 can be connected to Figure 1 The electronic device 101 or Figure 2The electronic device 201 in is at least partially identical or similar.

[0114] The electronic device 101 may include a multi-action manager 510 , a big data client 520 , an application 530 , and an OS framework 580 .

[0115] The multi-action manager 510 may control the overall operation for providing the routine.

[0116] The multi-action manager 510 may include a service management module 540 , an analysis module 550 (or an intelligent module), a log / context information management module 560 , a plurality of application programming interfaces APIs 571 , 572 , and 753 , and a recommendation client 574 .

[0117] The service management module 540 may include a user / device management manager 541 , a condition management manager 542 , and an action management manager 543 .

[0118] The user / device management manager 541 may manage device context information of the electronic device 101. The device context information of the electronic device 101 may include information about the battery 189 of the electronic device 101, information about the processor 120 (e.g., CPU) of the electronic device 101, or information about devices / networks connected to the electronic device 101, or a combination thereof.

[0119] The condition management manager 542 may manage information on situations, locations, events, times, devices, and operation manuals.

[0120] The action management manager 543 may manage information about the action. For example, the action management manager 543 may manage information related to the configuration of the action.

[0121] The analysis module 550 may include a general profile engine 551 , a feedback-based user profile engine 552 , a domain-specific profile engine 553 , and a priority platform 554 .

[0122] The general profile engine 551 may generate a general profile based on information received from the big data client 520 .

[0123] The feedback-based user profile engine 552 may generate a feedback-based profile based on information about feedback received from the big data client 520. For example, the feedback-based user profile engine 552 may generate the feedback-based profile by using a reinforcement learning scheme.

[0124] The domain specific profile engine 553 may generate domain specific profiles by differentiating, for each domain, information received from the big data client 520. The domains may include health and intent extraction.

[0125] The priority platform 554 may generate priority conditions based on time, location, and / or occasion for determining recommendations or prompts to be provided to the electronic device 101. The priority platform 554 may be a time / location / occasion (TPO) based analyzer.

[0126] The log / context information management module 560 may include a context information manager 561, an inference engine 562, and a log collection manager 563. The context information manager 561 may store context information when logs of the electronic device 101 are collected. The inference engine 562 may generate information for recommending routines. The log collection manager 563 may manage log collection operations.

[0127] The API 571 may be configured to enable information to be sent / received between the multi-action manager 510 and the application 530 .

[0128] The API 572 may be configured to enable information to be sent / received between the multi-action manager 510 and the big data client 520 .

[0129] The API 573 may be configured to enable information to be sent / received between the multi-action manager 510 and the electronic device 103 .

[0130] The recommendation client 574 may receive information related to routines recommended by the recommendation server 108 - 2 (eg, a routine recommendation list) from the recommendation server 108 - 2 .

[0131] The big data client 520 may include a log collector 521 , an inference engine 522 , a context information manager 523 , a personal manager 524 , a proposal manager 525 , and an API 526 .

[0132] The log collector 521 may collect logs (or log data) of the electronic device 101. The log collector 521 may collect logs related to the state (or state change) of the electronic device 101 due to user input of the electronic device 101. The user input may include voice input, touch input, button (or key) input, and / or input through a remote controller. However, the user input is not limited to the above examples.

[0133] The inference engine 522 may infer context information of the electronic device 101 (or the user of the electronic device 101) from user input and information collected by using the sensor module of the electronic device 101. The inference engine 522 may infer context information of the electronic device 101 (or the user of the electronic device 101) based at least in part on the logs of the electronic device 101 collected by the log collector 521. For example, the inference engine 522 may infer the location of the user's home or company or context information related to the location (e.g., during going to / leaving the office, traveling, or driving).

[0134] The context information manager 523 may collect context information of the electronic device 101. For example, the context information manager 523 may collect context information of the electronic device 101 inferred by the inference engine 522.

[0135] The context information manager 523 may transmit the collected context information to the offer manager 525 .

[0136] The personal manager 524 may manage personal information of the user of the electronic device 101. For example, the personal manager 524 may collect the user's personal information based at least in part on context information of the electronic device 101 inferred by using the inference engine 522. The personal information of the user of the electronic device 101 may include information about a result of an operation of an application executed by the electronic device 101 or information about the current state of the electronic device 101.

[0137] The personal manager 524 may transmit personal information of the user of the electronic device 101 to the offer manager 525 .

[0138] The personal manager 524 may send at least a portion of the user's personal information to the smart server 108-1. The personal manager 524 may send at least a portion of the user's personal information accumulated within a specified time period (eg, one month) to the smart server 108-1.

[0139] The suggestion manager 525 can predict the user's intention and can transmit the prompt or information required for the electronic device 101 to perform the operation for the user to the multi-action manager 510. For example, the suggestion manager 525 can provide information to the multi-action manager 510 in view of the user's current state (e.g., time, place, occasion, application being used, etc.).

[0140] The proposal manager 525 can transmit the prompts or information required for the electronic device 101 to perform operations to the multi-action manager 510 based on the context information of the electronic device 101 collected by the context information manager 523, the personal information of the user of the electronic device 101 received by the personal manager 524, and the context information of the user inferred by the inference engine 522.

[0141] The API 526 may be configured to enable information to be sent / received between the big data client 520 and the smart server 108 - 1 .

[0142] The application 530 may include a plurality of applications 531 , 532 , and 533 . The application may include preference settings 534 .

[0143] Each of the multiple applications 531, 532, and 533 may include an SDK. The SDK included in each of the multiple applications 531, 532, and 533 may define information about conditions and actions that each of the multiple applications 531, 532, and 533 can support. For example, the SDK may include information about functions that can be configured as conditions of the application and about functions that the application can be configured as actions. The SDK may serve as an interface between the application 530 and the multi-action manager 510. For example, the SDK may include information about a communication method for sending / receiving information between the application 530 and the multi-action manager 510.

[0144] The OS framework 580 may include an event manager 581. The event manager 581 may monitor the state of the system of the electronic device 101. The event manager 581 may transmit information about a change in the system state of the electronic device 101 (or a system event) to the multi-action manager 510. The event manager 581 may detect a change in the state of the electronic device 101 configured as a condition, and may transmit information about the detected change in the state of the electronic device 101 to the multi-action manager 510.

[0145] Figure 5 The multi-action manager 510, the big data client 520, and the OS framework 580 in the embodiment may be at least partially included in Figure 1 In the processor 120.

[0146] The second electronic device 103 may transmit / receive information related to the routine with the electronic device 101. When the second electronic device 103 has received the information related to the routine from the electronic device 101, the second electronic device 103 may perform an operation for executing the routine. The operation of the second electronic device 103 performing the operation for executing the routine based at least in part on the information about the routine received from the electronic device 101 is described in more detail below.

[0147] The smart server 108-1 may manage at least a portion of the personal information of the user of the electronic device 101 periodically received from the personal manager 524. The smart server 108-1 may execute Figure 2 The functions performed by the smart server 208 in are at least partially the same or similar functions.

[0148] The recommendation server 108-2 may generate a recommended routine, and may send information about the generated recommended routine (e.g., a routine recommendation list) to the recommendation client 574. The recommendation server 108-2 may generate a routine to be recommended to the electronic device 101 based on information about another electronic device (e.g., the second electronic device 103) or about a user of another electronic device. The recommendation server 108-2 may send the recommended routine generated based on information about another electronic device or about a user of another electronic device 101 to the recommendation client 574.

[0149] Figure 6 is a block diagram 600 of a method for executing a routine of an electronic device according to an embodiment.

[0150] Reference Figure 6 , showing a situation where conditions are configured for a first application 531 and actions are configured for a second application 532.

[0151] The multi-action manager 510 may configure conditions related to the first application 531 through the SDK 531-1 included in the first application 531. The SDK 531-1 may define information about conditions that the first application 531 can support. For example, the SDK 531-1 may include information about functions that can be configured as conditions of the first application 531. When an input for configuring a condition has been input from a user, the multi-action manager 510 may acquire information about functions that can be configured as conditions of the first application 531 from the first application 531 through the SDK 531-1.

[0152] The multi-action manager 510 may configure an action related to the second application 532 through the SDK 532-1 included in the second application 532. The SDK 532-1 may define information about actions that the second application 532 can support. For example, the SDK 532-1 may include information about functions that can be configured as actions of the second application 532. When receiving an input for configuring an action from a user, the multi-action manager 510 may obtain information about functions that can be configured as actions of the second application 532 from the second application 532 through the SDK 532-1.

[0153] The multi-action manager 510 may configure at least one of the conditions or actions based on the information obtained from the OS framework 580. For example, the multi-action manager 510 may configure at least one of the conditions or actions related to the system based at least in part on the information about the system obtained from the OS framework 580. For example, the multi-action manager 510 may configure at least one of the conditions or actions related to the context based at least in part on the information about the context obtained from the big data client 520.

[0154] However, the method in which the multi-action manager 510 configures at least one of the conditions or actions is not limited to the above-mentioned example.

[0155] If the routine is generated (or configured), the multiple action manager 510 may store information related to the generated routine (or at least one of the condition or action) in a database of the multiple action manager 510 .

[0156] If conditions related to contextual information are configured, for example, the multi-action manager 510 can register information for confirming whether the configured conditions are met in the big data client 520, or register events indicating the satisfaction of the configured conditions when the configured conditions are met.

[0157] The multi-action manager 510 may receive an event related to a configured condition (e.g., an event indicating that a condition is satisfied) from the first application 531 through the SDK 531-1. The multi-action manager 510 may receive an event related to a condition configured for the system from the event manager 581. The multi-action manager 510 may receive an event related to a condition configured for context information from the big data client 520.

[0158] The multi-action manager 510 can receive events related to conditions and then confirm actions corresponding to the conditions (or interconnected with the conditions). The multi-action manager 510 can perform operations for performing actions corresponding to the conditions. For example, the multi-action manager 510 can transmit information necessary for performing actions to the second application 532 through SDK532-1. For example, the multi-action manager 510 can transmit information for performing actions (e.g., actions related to displays, sensors, view interfaces, or batteries) to the OS architecture 580 (e.g., display manager 584, sensor manager 585, view system 586, or power manager 587). However, the operations of the multi-action manager 510 that performs actions corresponding to the conditions and the constituent elements included in the OS architecture 580 are not limited to the above examples.

[0159] Refer to below Figures 7 to 27 The method of providing a routine for the electronic device 101 is described in more detail.

[0160] According to an embodiment, an electronic device includes: a user interface; at least one processor operably connected to the user interface; and a memory operably connected to the at least one processor. The memory may be configured to store instructions that, when executed, cause the at least one processor to perform the following operations: when the electronic device is in a first state, determine whether a first condition is satisfied; if the first condition is satisfied, perform a first action that changes the state of the electronic device from the first state to the second state; receive, through the user interface, a user input that changes the state of the electronic device from the second state to the third state; based at least in part on the received user input, change the state of the electronic device from the second state to the third state; and if the first condition is not satisfied in the third state, maintain the third state.

[0161] The electronic device may further include at least one of a communication interface or a sensor module, and the instructions may be configured to cause the at least one processor to determine whether the first condition is met based at least in part on data acquired by at least one of the communication interface or the sensor module.

[0162] The instructions may be configured to cause at least one processor to generate a first routine including a first condition and a first action.

[0163] The instructions may be configured to cause the at least one processor to perform the following operations: in a locked state of the electronic device, displaying, through a user interface, information indicating that the first routine is being executed if the first condition is satisfied.

[0164] The instructions may be configured to cause at least one processor to display detailed information about the first routine based at least in part on user input regarding information indicating that the first routine is being executed.

[0165] The instructions may be configured to cause at least one processor to perform the following operations: generating an executable first routine in a locked state of the electronic device.

[0166] The instructions can be configured to cause at least one processor to perform the following operations: if a first condition is met, obtain a log related to a changed state in an electronic device within a specified time period; determine a second action related to a changed state in the electronic device based at least in part on the log; and recommend a second action.

[0167] The instructions can be configured to cause at least one processor to perform the following operations: obtain information about a user mode of the electronic device within a specified time period; generate a second routine based at least in part on the information about the user mode of the electronic device; recommend the second routine if a second condition included in the second routine is different from the first condition; and output a notification through a user interface so that the first routine also includes the second action when the second condition is the same as the first condition and the second action included in the second routine is different from the first action.

[0168] The instructions may be configured to cause the at least one processor to perform the following operations: displaying information related to the first routine via the user interface by using a quick panel.

[0169] According to an embodiment, an electronic device includes: a user interface; at least one processor operably connected to the user interface; and a memory operably connected to the at least one processor. The memory may be configured to store instructions that, when executed, cause the at least one processor to perform the following operations: when the electronic device is in a first state, determine whether a first condition is satisfied; if the first condition is satisfied, perform a first action to change the state of the electronic device from the first state to the second state; when the electronic device is in a second state, determine whether a second condition is satisfied; if the second condition is satisfied, perform a second action to change the state of the electronic device from the second state to a third state; when the electronic device is in a third state, if the second condition is satisfied but the first condition is no longer satisfied, maintain the state of the electronic device in the third state; when the electronic device is in a third state, if the first condition and the second condition are no longer satisfied, restore the state of the electronic device from the third state to the first state.

[0170] The electronic device may further include at least one of a communication interface or a sensor module, and the instructions may be configured to cause the at least one processor to determine whether the first condition and the second condition are met based at least in part on data acquired by at least one of the communication interface or the sensor module.

[0171] The instructions can be configured to cause at least one processor to perform the following operations: when the electronic device remains in the third state because the second condition is satisfied but the first condition is no longer satisfied, if the second condition is not satisfied, restore the state of the electronic device from the third state to the first state just before the first condition is satisfied.

[0172] The instructions may be configured to cause at least one processor to perform the following operations: when the electronic device is in the third state, if the first condition is satisfied but the second condition is no longer satisfied, restoring the state of the electronic device from the third state to the second state.

[0173] According to an embodiment, an electronic device includes: a communication interface configured to perform wireless communication; at least one processor operably connected to the communication interface; and a memory operably connected to the at least one processor. The memory may be configured to store instructions that, when executed, cause the at least one processor to perform the following operations: receiving information related to a first routine including a first action to be performed when a first condition is satisfied from an external electronic device by using the communication interface; when the electronic device is in a first state, if the first condition is satisfied, determining whether the first action can be performed based at least in part on the received information related to the first routine; if the first action can be performed, storing the first routine and the first state in the memory; after storing the first routine and the first state, performing the first action to change the first state to the second state; and returning to the first state after performing the first action.

[0174] The information related to the first routine may include at least one of: package information related to determining whether the first condition is satisfied, tag information including a key value designated for distinguishing the first condition, or parameter information including a detailed configuration value regarding the tag information.

[0175] The first action may use at least one of: package information related to the first action, tag information including a key value designated for distinguishing the first action, or parameter information including a detailed configuration value regarding the tag information.

[0176] The instructions may be configured to cause the at least one processor to perform the following operations: if the first action cannot be performed, receiving information necessary for performing the first action from a different electronic device through a communication interface.

[0177] The instructions may be configured to cause at least one processor to: change a portion of information related to the first routine based at least in part on information related to a user of the electronic device; and generate a second routine based at least in part on the changed portion of the information.

[0178] The instructions may be configured to cause at least one processor to perform the following operations: when performing an operation for configuring a first routine, obtain information about a first location input from a user; and receive information recommending a routine related to the first location from an external electronic device through a communication interface.

[0179] The instructions can be configured to cause at least one processor to perform the following operations: determine a second location as a point of interest for a user of the electronic device based at least in part on log information of the electronic device; and when the electronic device is located at the second location, receive a routine related to the second location from an external electronic device related to the second location through a communication interface.

[0180] Figure 7 is a flowchart 700 of a method for restoring a state of the electronic device 101 after executing a routine according to an embodiment.

[0181] Reference Figure 7 In step 701, the processor 120 (eg, the multi-action manager 510) may determine whether a first condition is satisfied when the electronic device 101 is in a first state.

[0182] The processor 120 may confirm whether the first condition configured for the first routine is satisfied while the first routine is activated. If the first condition is satisfied, the first routine may be generated (or configured) to perform a first action that changes the state of the electronic device 101 to the second state.

[0183] The processor 120 may determine whether a first condition configured for the first routine is satisfied based at least in part on an event received from an SDK of the application, an event received from the big data client 520 , or an event received from the event manager 581 .

[0184] The processor 120 may store information about the first state in the memory 130. For example, if the first condition is "return home" and if the first action is "set the volume level to 100", the processor 120 may store the state value of the electronic device 101 (e.g., volume level 50) in the memory 130 before performing the first action to correspond to the first condition.

[0185] In step 703 , the processor 120 may perform a first action of changing the state of the electronic device 101 from the first state to the second state (eg, volume level 100 ) when it has been determined that the first condition is satisfied.

[0186] The processor 120 may cause the application configured with the first action to execute the first action through the SDK. The processor 120 may cause the system operation configured with the first action to be executed.

[0187] In step 705, the processor 120 may receive a user input for changing the state of the electronic device 101 from the second state to the third state through the user interface of the electronic device 101. For example, the processor 120 may receive a user input for changing the state of the electronic device 101 from the second state (e.g., volume level 100) which is the state after performing the first action to the third state (e.g., volume level 75) (e.g., user input for changing the volume level to 75).

[0188] In step 707 , in an embodiment, the processor 120 may change the state of the electronic device 101 from the second state to the third state based at least in part on the user input.

[0189] The processor 120 may store the third state in the memory 130 .

[0190] In step 709 , if the first condition is released (or if the first condition is no longer satisfied), the processor 120 may maintain the third state.

[0191] If the first condition is released (for example, if "return home" is configured as the first condition and if the electronic device 101 is located outside the home), the processor 120 may compare the second state (volume level 100) just after the first action is performed with the current state (or third state) (for example, volume level 75). If the second state and the current state are not the same, the processor 120 may maintain the current state (or third state). If the second state and the current state are the same (for example, if the second state is the current state, or if a user input for changing to the third state is not received), the processor 120 may change (or restore) the state of the electronic device 101 to the first state (for example, volume level 50) just before the first action is performed.

[0192] If an action configured for a routine is performed because a condition configured for the routine is satisfied, if the state of electronic device 101 is changed by user input, and if the condition configured for the routine is subsequently released, electronic device 101 may maintain the state of electronic device 101 changed by user input, rather than the state just before the action was performed, thereby reflecting the user's intention.

[0193] Figure 8 is a flowchart 800 of a method for restoring a state of the electronic device 101 after executing multiple routines according to an embodiment.

[0194] Fig. 9 is a diagram of a method for restoring a state of an electronic device after executing a plurality of routines according to an embodiment.

[0195] Reference Figure 8 and Fig. 9 In step 801, when the electronic device 101 is in a first state, the processor 120 (eg, the multi-action manager 510) may determine whether a first condition is satisfied.

[0196] The processor 120 may confirm whether the first condition configured for the first routine is satisfied while the first routine is activated. If the first condition is satisfied, the first routine may be generated (or configured) to perform a first action that changes the state of the electronic device 101 to the second state.

[0197] The processor 120 may confirm whether the first condition configured for the first routine is satisfied based at least in part on an event received from the SDK of the application, an event received from the big data client 520 , or an event received from the event manager 581 .

[0198] The processor 120 may store information about the first state in the memory 130. For example, if the first condition is "return home" and if the first action is "set the volume level to 100", the processor 120 may store the state value of the electronic device 101 (e.g., volume level 50) in the memory 130 before performing the first action to match the first condition (e.g., Fig. 9 Corresponding to A1) in .

[0199] In step 803 , the processor 120 may perform a first action of changing the state of the electronic device 101 from the first state to the second state (eg, volume level 100 ) when it has been determined that the first condition is satisfied.

[0200] The processor 120 may cause the application configured with the first action to execute the first action through the SDK. The processor 120 may cause the system operation configured with the first action to be executed.

[0201] In step 805 , when the electronic device 101 is in the second state, the processor 120 (eg, the multi-action manager 510 ) may determine whether a second condition is satisfied.

[0202] The processor 120 may confirm whether the second condition configured for the second routine is satisfied when the second routine is activated. In step 807, if the second condition is satisfied, the second routine may be generated (or configured) to perform a second action that changes the state of the electronic device 101 to the third state. For example, the second condition (e.g., Fig. 9 A2) in the example may be “if the time is any time from 12 am to 6 pm”, and the second action may be “set the volume level to 0”. The third state may be a state of “volume level 0”.

[0203] If the second condition is satisfied, the processor 120 may store information about a state (eg, a second state) just before performing the second action in the memory 130 .

[0204] In step 809 , the processor 120 may confirm whether the first condition is released (or whether the first condition is no longer satisfied).

[0205] In step 811, if the first condition that was previously satisfied before the second condition is resolved before the second condition (e.g., if Fig. 9If the first condition with a smaller timestamp (or satisfied at an earlier time) in 910 is released before the second condition with a larger timestamp, the processor 120 may maintain the third state (or current state) (eg, volume level 0).

[0206] As in Fig. 9 In 920, if the first condition is released before the second condition, the processor 120 may delete the information about the state related to the first action of the first condition. Fig. 9 As shown in 920 of , the processor 120 may change (or update) the state value just before the second action of the second condition is performed (or the state value before the second action) (e.g., volume level 100) to the state value just before the first action of the first condition is performed (or the value about the first state) (e.g., volume level 50). The state value just before the second action of the second condition is performed may be the state value of the electronic device 101 to be restored when the second condition is released.

[0207] In step 813, the processor 120 may confirm whether the second condition is released (or whether the second condition is no longer satisfied). For example, if the second condition is "if the time is any time from 12 am to 6 pm", the processor 120 may confirm whether the current time is over 6 pm.

[0208] In step 815, if the second condition is removed, the processor 120 may restore the state of the electronic device 101 from the third state to the first state. Fig. 9 As shown in 920 , the processor 120 may restore the state of the electronic device 101 to the state value just before executing the first action of the first condition, which state value (eg, the volume level 50 ) has been changed in step 813 .

[0209] Figure 8 and Fig. 9 The case where the first condition that is satisfied before the second condition is released before the second condition is shown, but the present disclosure is not limited thereto. For example, if the second condition is satisfied after the first condition is satisfied, and if the second condition that is satisfied at a later time than the first condition is released first, the state of the electronic device 101 can be restored (or changed) from the third state (e.g., volume level 0) to the second state (e.g., volume level 100). If the first condition is released after the second condition is released, the state of the electronic device 101 can be restored from the second state to the first state (e.g., volume level 50).

[0210] Figure 8 and Fig. 9A case is shown where two conditions (e.g., a first condition and a second condition) are assumed, but the present disclosure is not limited thereto. For example, multiple routines may be configured such that there are N conditions different from each other, where the N different conditions respectively correspond to actions in different states that are changed to the same function (e.g., a volume adjustment function). In this case, if the first condition to the Nth condition are continuously satisfied, and if the Mth (M < N) condition is subsequently released first, the processor 120 may delete information about the state related to the action of the Mth condition. The processor 120 may change (or update) the state value just before the action of the (M + 1)th condition (immediately after the Mth condition) to the state value just before the action of the Mth condition. The state value just before the action of the (M + 1)th condition may be the state value of the electronic device 101 that will be restored when the (M + 1)th condition is released.

[0211] After executing multiple routines, the state of the electronic device 101 can be restored based on the method described with reference to Figure 8 and Fig. 9 so that the state of the electronic device 101 desired by the user is maintained.

[0212] Fig.10 is a flowchart 1000 of a method for the electronic device 101 to provide a routine based on information about the routine received from an external electronic device.

[0213] Referring to Fig.10 , in step 1001, the processor 120 may receive, from an external electronic device through the communication module 190, information related to a first routine including a first action to be executed when a first condition is satisfied.

[0214] The information related to the first routine may include at least one of package information, tag information, or parameter information including detailed configuration values about the tag information.

[0215] The package information may include information about determining whether the first condition related to the first routine is satisfied. The package information may include the identification (ID) of an application configured with the first condition related to the first routine. The package information may include the ID of an application configured with the first action corresponding to the first condition.

[0216] The tag information may include a key value designated to distinguish the first condition related to the first routine from other conditions. The tag information may include a key value designated to distinguish the first action corresponding to the first condition from other actions.

[0217] The parameter information may be information necessary for performing the first action. The parameter information may be detailed information value about the tag information. If the parameter information corresponds to personal information of the user of the electronic device 101 or corresponds to information that must be configured separately for each user, the parameter information may not be received from the external electronic device.

[0218] In step 1003 , the processor 120 may determine, based at least in part on the received information related to the first routine, whether a first action can be performed when the electronic device 101 is in the first state and a first condition is satisfied.

[0219] The processor 120 may determine whether the electronic device 101 is capable of executing the first routine. For example, the processor 120 may determine whether an application configured with the first condition of the first routine is installed in the electronic device 101. If it is confirmed that the application configured with the first condition of the first routine is not installed in the electronic device 101, the processor 120 may receive information for installing (or downloading) the application configured with the first condition from the server based at least in part on information related to the first routine, thereby installing the application configured with the first condition. For example, the processor 120 may determine whether an application configured with the first action corresponding to the first condition of the first routine is installed in the electronic device 101. If it is confirmed that the application configured with the first action corresponding to the first condition of the first routine is not installed in the electronic device 101, the processor 120 may receive information for installing the application configured with the first action from the server 108 based at least in part on information related to the first routine, thereby installing the application configured with the first action.

[0220] In step 1005 , if it is determined that the first action can be performed, the processor 120 may store the first routine and the first state in the memory 130 .

[0221] The processor 120 may change the first routine so that the first routine corresponds to information related to the user of the electronic device 101 (or so that the first routine is appropriate, personalized, or customized). For example, the processor 120 may change a portion of the received information about the first routine to correspond to information related to the user of the electronic device 101.

[0222] The processor 120 may change the information related to the first condition. For example, if the first condition is "return home", the processor 120 may change (or configure) the home location coordinates as the context information of the user of the electronic device 101 to the home location coordinates of the user of the electronic device 101.

[0223] The processor 120 may change the information related to the first action. For example, if the first action is "connecting the electronic device 101 to the vehicle via Bluetooth", the processor 120 may change the information about the Bluetooth device of the vehicle to the information about the Bluetooth device of the user of the electronic device 101.

[0224] The processor 120 may change information about at least one of the first condition or the first action based at least in part on historical information (or user pattern information) of the user of the electronic device 101. For example, the processor 120 may check information about an application (or a type of application) (e.g., a music application) related to the first action (or an application configured with the first action) based at least in part on information about the routine. The processor 120 may change the first action so that the specific music application is executed if it is confirmed that the user of the electronic device 101 frequently uses the specific music application and the first condition is satisfied.

[0225] If the information related to the first routine is changed, the processor 120 may store the changed first routine in the memory 130 based at least in part on the changed information related to the first routine.

[0226] If it is determined that the first routine does not need to be changed, the processor 120 may store the first routine in the memory 130 without changing the first routine.

[0227] In step 1007, in an embodiment, the processor 120 may store the first routine and the first state, and then may perform a first action so that the first state is changed to the second state. Operation 1007 is at least partially related to Figure 7 The same or similar step 703 in the above description will be omitted here.

[0228] In step 1009, the processor 120 may perform an operation of restoring from the second state to the first state. Step 1009 is at least partially related to Figure 7 The same or similar step 709 in the above description will be omitted here.

[0229] Fig.11 is a flowchart 1100 of a method for providing a routine by the electronic device 101 according to an embodiment based on information about the routine received from another electronic device.

[0230] Reference Fig.11 In step 1101 , the processor 120 may receive information related to the first routine from an external electronic device by using the communication module 190 .

[0231] The processor 120 may receive information related to the first routine from the external electronic device based at least in part on the user's input. For example, the processor 120 may receive information related to the first routine based at least in part on a user input (e.g., a user input for confirming (or allowing) the reception of information related to the first routine) about a message including information about the first routine (e.g., information related to the first routine or a uniform resource locator (URL) for receiving the first routine), which is received when the electronic device 101 uses a messenger application (e.g., a chat application) to send / receive messages with other external electronic devices. However, the method of receiving information related to the first routine from the external electronic device based at least in part on the user's input is not limited to the above example.

[0232] The processor 120 may receive information related to the first routine from the external electronic device 101 based on the context information. For example, if the electronic device 101 is located at a specific place (e.g., a place included in the search history of the user of the electronic device 101), the processor 120 may receive information related to the first routine (e.g., a routine configured so that the condition of being located at the specific place and the action related to the service provided at the specific place are executed) from the external electronic device 101 arranged at the specific place (e.g., a communication device arranged at a coffee shop) or from an external electronic device related to the specific place (e.g., a server providing services related to an amusement park). However, the method for receiving information related to the first routine from the external electronic device based on the context information is not limited to the above examples.

[0233] The information related to the first routine may include at least one of package information, tag information, or parameter information including detailed configuration values ​​about the tag information. The description of the information related to the first routine is at least partially related to Fig.10 The description of step 1001 in is the same or similar, and its detailed description will be omitted here.

[0234] In step 1103, the processor 120 may determine whether the first routine can be executed based at least in part on the received information related to the first routine. For example, the processor 120 may determine whether an application configured with the first condition of the first routine is installed in the electronic device 101. If it is confirmed that no application configured with the first condition of the first routine is installed in the electronic device 110, the processor 120 may receive information for installing the application configured with the first condition from the server based at least in part on the information related to the first routine, thereby installing the application configured with the first condition. For example, the processor 120 may determine whether an application configured with the first action corresponding to the first condition of the first routine is installed in the electronic device 101. If it is confirmed that no application configured with the first action corresponding to the first condition of the first routine is installed in the electronic device 101, the processor 120 may receive information for installing the application configured with the first action from the server based at least in part on the information related to the first routine, thereby installing the application configured with the first action.

[0235] In step 1105 , if it is determined that the first routine can be executed, the processor 120 may store the first routine.

[0236] The processor 120 may change the first routine so that the first routine corresponds to the electronic device 101 (or the user of the electronic device 101) (or so that the first routine is appropriate, personalized or customized). The processor 120 may change a portion of the received information about the first routine so as to correspond to the electronic device 101.

[0237] The processor 120 may change the information related to the first condition. The processor 120 may change the information about at least one of the first condition or the first action based at least in part on history information (or user pattern information) of the user of the electronic device 101.

[0238] If the information related to the first routine is changed, the processor 120 may store the changed first routine in the memory 130 based at least in part on the changed information related to the first routine.

[0239] If it is determined that the first routine does not need to be changed, the processor 120 may store the first routine in the memory 130 without changing the first routine.

[0240] Fig.12 is a flowchart 1200 of a method for recommending a routine for an electronic device according to an embodiment.

[0241] Reference Fig.12 In step 1201 , the processor 120 (eg, the big data client 520 or the multi-action manager 510 ) may obtain log information of the electronic device 101 .

[0242] The log information may include information about at least one of TPO context, time, place, occasion, content, reminder, user profile, TPO preference, or wearable data synchronization.

[0243] TPO situations may include finishing work, working, working, near home, at home, commuting to work, before waking time, likely to fall asleep, before bedtime, and in a frequently visited place.

[0244] Time may include sleep statistics, commuting statistics, and exercise statistics. Sleep statistics may be information necessary to analyze a user's sleep pattern, and may include daytime, sleep time, and wake-up time. For example, sleep statistics may include date (e.g., December 19), day of the week (e.g., Wednesday), sleep time (e.g., 00:55:06), wake-up time (e.g., 06:41:05). Commuting statistics may include daytime, confidence, leaving home, arriving at work, leaving work, and arriving home.

[0245] Locations may include information related to visits, location analysis, vehicle analysis, travel information analysis, living space, country information, and Wi-Fi analysis. For example, information related to visits may include visit information and geo-fence information. Visit information may include time, category, and location type. Geofence information may include location ID, category, location type, geo-fence ID, and status related information. Information related to location analysis may include information about ID, category, latitude, longitude, and confidence. Information related to vehicle analysis may include information about name, device address, and confidence. Information related to travel analysis may include information about status, start time, end time, confidence, and whether it has ended. Information related to living space may include location information about locations frequently visited by users. Country information may include information about location type, current location, home, and time. Information related to Wi-Fi analysis may include information about service set ID (SSID), basic SSID (BASSID), category, location type, and confidence.

[0246] The occasion may include information related to driving log, commuting analysis, wake-up time, refresh log, work log, learning log, and going home / leaving home. Information related to driving log may include information about global positioning system (GPS) information, time, car, and status. Information related to commuting analysis may include information about time, status, and confidence. Information related to wake-up time may include information related to wake-up time, reference time, and confidence. Information related to refresh log may include information about start time, end time, status, and confidence. Information related to work log may include information about start time, end time, status, and confidence. Information related to learning log may include information about start time, end time, status, and confidence. Information related to going home / leaving home may include information about status, location ID, location type, additional information, and time.

[0247] Content may include information about recent things of interest, preferred contacts (integrated / recommended), preferred apps (integrated / recommended), preferred settings, preferred places, preferred music, preferred music genres, preferred music artists, preferred websites, preferred domains, preferred data, preferred devices, and preferred charging. Information about recent things of interest may include information about short-term preferred apps, short-term preferred contacts, short-term preferred music, and short-term preferred music artists. Information about preferred contacts may include information about time-specific, location-specific names and confidences. Information about preferred apps may include information about time-specific, location-specific names and confidences, and information about preferred applications based on ranking differences and app groups. Information about preferred settings may include information about time-specific, location-specific names and confidences, as well as sleep time, commuting, TPO, names, and confidences. Information about preferred places may include information about time-specific names and confidences. Information about preferred music may include information about time-specific, location-specific tracks, albums, and confidences. Information related to preferred music genres may include information about time-specific, location-specific genres and confidence. Information related to preferred music artists may include information about time-specific, location-specific artists and confidence. Information related to preferred websites may include information about time-specific preferred websites, location-specific preferred websites, URLs, and confidence. Information related to preferred domains may include information about time-specific preferred domains, location-specific preferred domains, and confidence. Information related to preferred data may include information about text messages, phone calls, emails, names, and confidence. Information related to preferred devices may include information about time-specific preferred devices, location-specific preferred devices, device names, and confidence. Information related to preferred charging may include information about charging levels, time-specific preferred charging types, location-specific preferred charging types, names, and confidence.

[0248] The user profile may include information related to analyzed user profiles, associated user profiles, feedback demographics, and feedback interests.

[0249] The TPO preferences may include information related to app TPO preferences, setting TPO preferences, music TPO preferences, and exercise TPO preferences.

[0250] Wearable data synchronization can include synchronization with a gear (e.g., by Samsung Electronics TMGear data synchronization may include information about the analyzed location, commuting pattern information, and sleep pattern information. The analyzed location of gear data synchronization may include information about ID, category, latitude, longitude, and confidence. The commuting pattern information of gear data synchronization may include information about week type, confidence, leaving home, at work, work completed, and returning home. The sleep pattern information of gear data synchronization may include information about week type, confidence, bedtime, and waking up. The gear service status may include information about basic information, sleep time status, and TPO context history. The basic information of the gear service may include project and data information. The sleep time status of the gear service may include information about date, day of the week, bedtime, and waking up. The TPO context history of the gear service may include information about time, TPO context, and confidence.

[0251] In step 1203, the processor 120 may determine the recommendation priority. For example, the processor 120 may determine the recommendation priority in order to generate a recommendation list for at least one of the conditions or actions of the recommendation routine.

[0252] Refer to the following Fig.13 The method for determining the recommendation priority in step 1203 is described in more detail.

[0253] Fig.13 is a flowchart 1300 of a method for determining a recommendation priority of an electronic device according to an embodiment.

[0254] Reference Fig.13 In step 1301, the processor 120 may generate a profile based at least in part on the acquired log information.

[0255] Processor 120 may generate a profile based at least in part on the table of available functions, the table based on user usage, and the table of contextual information.

[0256] The available function table may be a table defining functions that the electronic device 101 can define as routines.

[0257] An example of a possible function table is given in Table 1 below:

[0258] Table 1

[0259]

[0260]

[0261]

[0262] The user-based usage table may include information about recent interests, preferred contacts (integrated / recommended), preferred apps (integrated / recommended), preferred settings, preferred locations, preferred music, preferred music genres, preferred music artists, preferred websites, preferred domains, preferred data, preferred devices, and preferred charging.

[0263] The context information table may include finishing work, working, working, near home, at home, commuting to work, before waking time, likely to fall asleep, before bedtime, and frequently visited places.

[0264] Profiles may include location profiles, behavioral profiles, and user profiles. For example, a location profile may include information about home, work, car, frequently visited locations, and user-entered locations. A behavioral profile may include information about sleep time and commuting time. A user profile may include categories and keywords that interest the user as well as service-specific user profiles.

[0265] In step 1303, the processor 120 may generate context information. The context information may be context information of the electronic device 101. The processor 120 may combine the generated profile with the log information of the electronic device 101, thereby generating the context information of the electronic device 101.

[0266] Context information may include location context information and occasion context information. For example, location context information may include information about the interior of a location, the exterior of a location, the periphery of a location, special locations, whether a location is domestic, and overseas travel. Context information may include information about the context before bedtime, after bedtime, before commuting, after commuting, during commuting, before driving, after driving, during driving, and weather.

[0267] In step 1305, the processor 120 may determine a recommendation priority. The recommendation priority may include a normal priority, a service-specific priority, and a proposal. For example, the normal priority may include application preference, contract preference, setting preference, location preference, music preference, URL, and search keyword. The service-specific priority may include an Internet of Things (IOT) preference and a service preference.

[0268] Reference Fig.12 In step 1205, processor 120 may generate a recommendation table based at least in part on the recommendation priority.

[0269] Table 2 below gives an example of a recommendation table including recommendation priorities:

[0270] Table 2

[0271]

[0272] However, the recommendation table including the recommendation priorities is not limited to Table 2 above.

[0273] In step 1207, the processor 120 may confirm whether a routine related to the recommendation table is pre-stored (or pre-generated) in the electronic device 101. For example, the processor 120 may confirm whether a routine related to the recommendation table (e.g., a routine including conditions included in the recommendation table) has been pre-generated (or configured) by a user of the electronic device 101.

[0274] In step 1209, if it is confirmed that the routine related to the recommendation table is pre-stored in the electronic device 101, the processor 120 may recommend an action. For example, if it is confirmed that the routine including the condition included in the recommendation table is pre-stored in the electronic device 101, the processor 120 may output a notification to indicate that an additional action may be configured to correspond to the condition included in the recommendation table (or, if the condition is satisfied, to be executed).

[0275] In step 1211, if it is confirmed that no routine related to the recommendation table is pre-stored in the electronic device 101, the processor 120 may recommend a routine. For example, if it is confirmed that no routine related to the recommendation table is pre-stored in the electronic device 101, the processor 120 may output a notification indicating that a routine may be generated.

[0276] Fig.14 is a flow chart 1400 of a method for recommending an action based at least in part on a pre-generated routine of an electronic device, according to an embodiment.

[0277] Reference Fig.14 In step 1401 , the processor 120 may obtain log information of the electronic device 101 related to a first condition of a pre-generated routine.

[0278] If the first condition is satisfied, the processor 120 may cause the first action corresponding to the first condition to be executed. The processor 120 may acquire information about the state of the electronic device 101 that is changed when the first action is executed as the first condition is satisfied (or information about a function that is executed together when the first action is executed).

[0279] The state of the electronic device 101 that is changed when the first action is performed as the first condition is satisfied may be related to a function that is frequently used by the user of the electronic device 101 when the first condition is satisfied.

[0280] In addition to the information about the state of the electronic device 101 that is changed when the first action is performed as the first condition is satisfied, the processor 120 may also additionally acquire information about the state of the electronic device 101 that is changed if the first condition is released after the first condition is satisfied. The processor 120 may consider both the information about the state of the electronic device 101 that is changed when the first action is performed as the first condition is satisfied and the information about the state of the electronic device 101 that is changed if the first condition is released after the first condition is satisfied, thereby more accurately determining the function frequently used by the user of the electronic device 101 when the first condition is satisfied.

[0281] The log information of the electronic device 101 may include the above combination Fig.12 At least a portion of the log information of the electronic device 101 described in step 1201.

[0282] In step 1403 , the processor 120 may determine a recommendation priority based at least in part on log information of the electronic device 101 related to the first condition.

[0283] In step 1405 , processor 120 may generate a recommendation table based at least in part on the recommendation priorities.

[0284] Step 1403 and step 1405 are respectively Fig.12 Step 1203 and step 1205 in are at least partially identical or similar, and their detailed description is omitted here.

[0285] In step 1407, the processor 120 may recommend an action. The processor 120 may output a notification based at least in part on the generated recommendation table to indicate that an additional action (e.g., a function frequently used by the user of the electronic device 101 when the first condition is met) may be configured to correspond to the condition included in the recommendation table (or, to be performed if the condition is met).

[0286] Fig.15 is a flowchart 1500 of a method for displaying an icon in a locked state of the electronic device 101 according to an embodiment.

[0287] Reference Fig.15 In step 1501, when the electronic device 101 is in the first state, the processor 120 may display a first icon related to a first application through the display device 160 in a locked (or locked) state of the electronic device 101. For example, when the electronic device 101 is in the first state, the processor 120 may display a lock screen user interface including a first icon related to a first application through the display device 160 in a locked state of the electronic device 101.

[0288] The processor 120 may determine a first icon to be displayed when the electronic device 101 is in a locked state. The processor 120 may determine the first application to be displayed during the locked state from the applications that the electronic device 101 is capable of executing based at least in part on the frequency of use of applications executed in the first state of the electronic device 101 (e.g., when the electronic device 101 is located in the user's home). For example, the processor 120 may determine two applications with a high frequency of use in the first state of the electronic device 101 as first applications. The processor 120 may determine at least one application that was most recently used in the first state of the electronic device 101 among the applications that the electronic device 101 is capable of executing as the first application. After determining the first application, the processor 120 may display a lock screen user interface including a first icon corresponding to the first application through the display device 160.

[0289] The processor 120 may display the first icon in a designated area of ​​the display device 160. For example, the processor 120 may display the first icon in the bottom area of ​​the display device 160. However, the area of ​​the display device 160 where the first icon is displayed is not limited to the bottom area of ​​the display device 160.

[0290] The processor 120 may display a lock screen user interface including a first icon related to a first application in a locked state of the electronic device 101 based at least in part on a routine. For example, if the first condition of the routine is "if the state of the electronic device 101 changes to the first state", and if the first action corresponding to the first condition is "in the locked state, display a first icon corresponding to the first application and a second icon corresponding to the second application", so that if the state of the electronic device 101 changes to the first state (or if the first condition is satisfied), the processor 120 may display a lock screen user interface including a first icon corresponding to the first application and a second icon corresponding to the second application through the display device 160. For example, if the first condition of the pre-generated routine is "if the state of the electronic device 101 changes to the first state", and if the first action corresponding to the first condition is "in the locked state, display icons corresponding to the two most frequently used applications from the first to the Nth applications", so that if the electronic device 101 changes to the first state (or, if the first condition is satisfied), the processor 120 may display a lock screen user interface including icons corresponding to the two most frequently used applications from the first to the Nth applications through the display device 160. However, the method of displaying icons in the locked state is not limited to the above example. For example, the processor 120 can configure the lock screen user interface display function so that at least one application icon selected by the user is displayed in the locked state.

[0291] The processor 120 may display information related to the currently executed routine (e.g., a routine configured so that the first action corresponding to the first condition is currently executed when the first condition is satisfied) through the display device 160 in the locked state. For example, the processor 120 may display information indicating that the first routine is currently executed in the locked state through the display device 160. For example, if multiple routines are currently executed in the locked state, the processor 120 may display information about the most recently (or last) executed routine among the multiple routines (e.g., information indicating that the most recently executed routine is currently executed) through the display device 160.

[0292] In step 1503, if the state of the electronic device 101 changes from the first state to the second state, the processor 120 may display a second icon related to the second application instead of the first icon through the display device 160. For example, when the electronic device 101 is in the second state, the processor 120 may display a lock screen user interface including a second icon related to the second application through the display device 160 in a locked state of the electronic device 101.

[0293] The operation of the processor 120 displaying the second icon in the second state changed from the first state is at least partially the same or similar to the operation of the processor 120 displaying the first icon in the first state in step 1501, and its detailed description is omitted here.

[0294] Fig.16 is an illustration 1600 of a routine associated with hardware keys of an electronic device, according to an embodiment.

[0295] Reference Fig.16 , the processor 120 may configure the input related to the hardware key as a condition of the routine. For example, the electronic device 101 may include a hardware key 1610 related to the volume adjustment function, a hardware key 1620 related to the power function, and a hardware key 1630 related to the voice command (or recognition) function (e.g., Bixby TM The processor 120 may configure the input regarding the hardware key 1630 related to the voice command function as a condition based at least in part on the user input, and may generate (or configure) a routine that causes at least one action (e.g., flash operation, music playback, camera execution, emergency call, or calling the electronic device 101 of a designated user) to be executed to correspond to the condition (or, the condition is satisfied).

[0296] In an embodiment, the processor 120 may generate a routine by combining a condition related to an input on a hardware key with another condition. For example, the processor 120 may generate a routine in which an input on a hardware key 1630 related to a voice command function is configured as a first condition, a phone call function is configured as a second condition, and an operation of calling a specific contact is configured as an action if the first condition and the second condition are satisfied. However, generating a routine based on a combination of a condition related to an input on a hardware key and another condition is not limited to the above example.

[0297] Fig.17 is an illustration 1700 of a method of providing, by the electronic device 101 , a routine based on information about the routine received from another electronic device according to an embodiment.

[0298] Reference Fig.17 As shown in panel 1710, the processor 120 may start sending / receiving messages with another external electronic device by using a messenger application (e.g., a chat application). The processor 120 may display a screen 1711 including messages sent by the electronic device 101 and messages received from another electronic device through the display device 160. The processor 120 may display information 1713 related to a travel routine received from another electronic device. The information 1713 related to the travel routine may include information required for the electronic device 101 to receive the travel routine (e.g., a URL for downloading the travel routine).

[0299] If an input for receiving a travel routine is received from the user of the electronic device 101 , the processor 120 may display information 1721 through the display device 160 as shown in a panel 1720 to indicate whether the travel routine is to be received.

[0300] If an input allowing (or accepting) receiving the travel routine is received from the user of the electronic device 101 , the processor 120 may receive the travel routine through the communication module 190 .

[0301] Fig.18 1800 is a diagram of a method for providing a routine by the electronic device 101 based on information about the routine received from another electronic device according to an embodiment. For example, Fig.18 It may be an illustration of a method for providing routines related to digital pen functionality.

[0302] Reference Fig.18, the electronic device 101 may store a routine related to the digital pen function. For example, as shown in panel 1810, the electronic device 101 may store a routine having a configuration condition "if the digital pen related to the electronic device 101 is attached to the electronic device 101 and then separated therefrom, and if the note function is executed", and having an action configured to correspond to the condition "execute a function related to the digital pen configured by the user of the electronic device 101 (e.g., the thickness or color of a line to be displayed during digital pen input)".

[0303] The electronic device 101 may transmit information about the routine related to the digital pen function to the second electronic device 103 through the communication module 190. As shown in panel 1820, after receiving the routine related to the digital pen function from the electronic device 101, the second electronic device 103 may display information 1821 through the display device 160 to indicate whether the routine related to the digital pen function is received.

[0304] If an input of allowing (or accepting) the routine related to the digital pen function is received from the user of the second electronic device 103 , the second electronic device 103 may receive the routine related to the digital pen function through the communication module.

[0305] Fig.19 is an illustration 1900 of a method for providing a routine related to a location of an electronic device 101 according to an embodiment.

[0306] Reference Fig.19 When configuring a routine, the electronic device 101 may receive a recommendation for a routine generated by another electronic device.

[0307] The processor 120 may receive an input for searching for a specific place from the user on the screen for configuring the conditions of the routine. As shown in panel 1910, if an input for searching for a specific place (e.g., Samsung Cafe) is received from the user, the processor 120 may display a map 1913 including an indicator 1915 indicating the location of the specific place in area 1911 through the display device 160.

[0308] If the specific place is found, the processor 120 may receive a routine related to the service provided at the specific place (e.g., a routine including a condition “the electronic device 101 is located at the specific place” and an action “displaying information related to the service provided at the specific place”) from a device related to the specific place (e.g., a server related to the service provided at the specific place). For example, as shown in panel 1920, if the specific place is found, the processor 120 may display information 1921 through the display device 160 to indicate whether to receive a routine related to the service provided at the specific place from a device related to the specific place. If an input allowing a routine related to the service provided at the specific place is received from the user of the electronic device 101, the processor 120 may receive a routine related to the service provided at the specific place.

[0309] As shown in panel 1930, if a routine related to services provided at the specific place is received, and if the electronic device 101 located at (or entering) the specific place satisfies the condition "the electronic device 101 is located at the specific place", the processor 120 can display information 1931 related to the services provided at the specific place (for example, a menu or coupon related to the services provided at the specific place) through the display device 160.

[0310] If the electronic device 101 is located at the specific place, the electronic device 101 may receive a recommendation of a routine generated by another electronic device from another electronic device. For example, if the electronic device 101 is located at (or enters) a specific place, the processor 120 may receive a routine related to a service provided at the specific place from a device related to the specific place (e.g., a server related to the service provided at the specific place).

[0311] Fig. 20 is an illustration 2000 of a method for providing a routine related to a location of an electronic device according to an embodiment.

[0312] Reference Fig. 20 , the electronic device 101 may receive a recommendation of a routine generated by another electronic device from the other electronic device based at least in part on information about the interest points of the user of the electronic device 101 .

[0313] The processor 120 may determine a point of interest of a user of the electronic device 101 based on the log information of the electronic device 101. For example, as shown on screen 2011 of panel 2010, if the user of the electronic device 101 has searched for a specific place at least a specified number of times within a specified time period, the processor 120 may determine the specific place as a point of interest.

[0314] If a specific place is determined as a point of interest, the processor 120 may send information indicating that the specific place is determined (or registered) as a point of interest to an electronic device related to the specific place (or a server managing services related to the specific place) through the communication module 190.

[0315] If the electronic device 101 is located at a specific place determined as a point of interest (or a location adjacent to the specific place), the processor 120 may receive a recommendation of a routine related to the specific place (or a routine related to a service provided at the specific place) from another electronic device related to the specific place. For example, as shown in panel 2020, if the electronic device 101 is located at a specific place determined as a point of interest, the processor 120 may display information 2021 through the display device 160 to indicate whether a routine related to the specific place is received. If an input allowing a routine related to a service provided at the specific place is received from the user of the electronic device 101, the processor 120 may receive a routine related to the specific place.

[0316] The processor 120 may execute the received routine related to the specific place. For example, as shown in panel 2030, the processor 120 may display information 2031 (e.g., information about menus, coupons, promotions, or amenities) related to services provided at the specific place through the display device 160.

[0317] Fig.21 is an illustration 2100 of a method for configuring a routine of an electronic device 101 according to an embodiment.

[0318] Reference Fig.21 , as shown in panel 2110 , when a function for configuring a routine is executed, the processor 120 may display a screen for inputting a routine name 2111 (eg, Routine 2 ) through the display device 160 .

[0319] As shown in panel 2120 , after inputting a routine name 2111 , processor 120 may display, through display device 160 , a screen for configuring conditions (eg, at home, traveling abroad, time, location, headphones, charging status, battery level, and Wi-Fi network).

[0320] In response to user input (e.g., a slide or drag input) to a screen for configuring conditions, as shown in panels 2130 and 2140, processor 120 may display a screen 2141 for configuring an action (e.g., Wi-Fi device activation 2121 in panel 2120) through display device 160.

[0321] The processor 120 may receive inputs related to a routine name 2111 , conditions, and actions 2141 , thereby generating (or configuring) a routine.

[0322] However, the routine configuration method is not limited to the above reference Fig.21 Describe the method.

[0323] Fig. 22 is an illustration 2200 of a method for configuring a routine of an electronic device 101 according to an embodiment.

[0324] Reference Fig. 22 , the processor 120 may display information about a preset routine that may be configured as a routine through the display device 160. The preset routine may be a primary routine generated by a designer of the electronic device 101 or a developer of an application to serve as a basis for generating a routine. As shown in the panel 2210, the processor 120 may display preset conditions (e.g., good morning 2211, work 2213, at home 2215, and save battery at night 2217) (or a list of preset conditions) through the display device 160.

[0325] The processor 120 may recommend at least one of the preset routines based at least in part on the log information of the electronic device 101 or the usage pattern of the user of the electronic device 101. For example, the processor 120 may determine the location information of the home of the user of the electronic device 101 based at least in part on the location information (e.g., coordinate information) of the electronic device 101 collected within a specified time period (e.g., one month) and the time information of the electronic device 101. The processor 120 may map the determined location information of the home of the user of the electronic device 101 to a preset condition related to the home (e.g., at home 2215). If the processor 120 maps the location information of the home of the user of the electronic device 101 to the preset condition related to the home, the processor 120 may indicate through the display device 160 that the preset condition related to the home can be configured as a condition of the routine. For example, as shown in the panel 2220, the processor 120 may display the preset condition related to the home (e.g., at home 2215) above the other preset conditions through the display device 160, and may display information 2215-1 indicating that the preset condition related to the home can be configured as a condition of the routine.

[0326] As shown in panel 2230 , if the preset routine is configured as a routine, the processor 120 may display a notification 2231 indicating the configuration of the routine through the display device 160 .

[0327] Fig.23 is an illustration 2300 of a method for configuring a routine of an electronic device 101 according to an embodiment.

[0328] Reference Fig.23 , the processor 120 may display the generated routine in various types through the display device 160.

[0329] As shown in the panel 2310, the processor 120 may display the generated routine. For example, the processor 120 may display a quick panel 2311 including the generated routine 2313 based at least in part on a drag input from a portion of the housing of the electronic device 101 to the display device 160.

[0330] However, the method for displaying the generated routine is not limited to the panel 2310. For example, the processor 120 may display the generated routine in a designated area of ​​the currently displayed screen (e.g., the upper end of the screen). For example, the processor 120 may display the generated routine through another screen different from the currently displayed screen.

[0331] If a user input related to the generated routine 2313 is received, the processor 120 may display at least one generated routine through the display device 160. For example, as shown in a panel 2320, the processor 120 may display a screen including information 2321 for generating an additional routine and information 2322, 2323, and 2324 about the generated routine through the display device 160.

[0332] Fig.24 is an illustration 2400 of a method for displaying an icon in a locked state of the electronic device 101 according to an embodiment.

[0333] Reference Fig.24 , in the locked (locked) state of the electronic device 101, and when the electronic device 101 is in a specified state, the processor 120 can display icons related to the application through the display device 160.

[0334] If the electronic device 101 is in a specified state (for example, if the electronic device 101 is located at the home of the user of the electronic device 101), the processor 120 may display icons related to applications through the display device 160 in a locked state of the electronic device 101 at least partially based on the frequency of use of the applications in the specified state. For example, if the electronic device 101 is in a specified state, the processor 120 may display icons 2420 and 2430 corresponding to applications most frequently used in a specified time period in the bottom area of ​​the display device 160 in the locked state of the electronic device 101. However, the method for displaying the icons 2420 and 2430 is not limited to the method of displaying the icons 2420 and 2430 in the bottom area of ​​the display device 160.

[0335] If the state of the electronic device 101 is changed (for example, if the state of the electronic device 101 is changed from a designated first state to a designated second state), the processor 120 may display icons related to the application based at least in part on the frequency of use of the application in the changed designated state.

[0336] Fig.25 is an illustration 2500 of a method for displaying information related to a routine in a locked state of the electronic device 101 according to an embodiment.

[0337] Reference Fig.25 , the processor 120 may display information related to the generated routine through the display device 160 when the electronic device 101 is in a locked (or locked) state.

[0338] The processor 120 may display information about the currently executed routine or information indicating the existence of the currently executed routine together with information 2511 indicating the time through the display device 160 in the locked state of the electronic device 101. As shown in the panel 2510, the processor 120 may display information 2512 about a representative routine among a plurality of routines currently being executed (e.g., information about the most recently (or last) executed routine) through the display device 160 in the locked state of the electronic device 101.

[0339] As shown in panel 2520 , based at least in part on user input regarding information 2512 related to representative routines, processor 120 may display, via display device 160 , information 2521 indicative of at least one currently executing routine.

[0340] As shown in panel 2530 , based at least in part on user input regarding information 2521 indicating at least one currently executed routine, processor 120 may display, through display device 160 , a list of at least one routine 2531 and 2533 .

[0341] As shown in panel 2540, based at least in part on a user input regarding routine 2531 of at least one currently executed routine 2531 and 2533, processor 120 may display, through display device 160, one or more actions 2541 of at least one action corresponding to routine 2531. Processor 120 may not display information regarding an action related to personal information of a user of electronic device 101 of at least one action corresponding to routine 2531. As shown in panel 2540, processor 120 may display, through display device 160, information 2543 indicating that user authentication is required to display an action related to personal information of a user of electronic device 101 of at least one action corresponding to routine 2531.

[0342] As shown in panel 2550 , based on user input regarding information 2543 indicating that user authentication is required to display an action related to personal information of the user of the electronic device 101 , a screen 2551 for authenticating the user may be displayed through the display device 160 .

[0343] As shown in panel 2560 , if the user authentication is successful, the processor 120 may display information 2561 (eg, predetermined information configured by the user) about an action related to personal information of the user of the electronic device 101 through the display device 160 .

[0344] Fig.26 is an illustration 2600 of a method for generating a routine related to a lock state of an electronic device 101 according to an embodiment.

[0345] Reference Fig.26 , the processor 120 may configure a routine that may be executed in a locked state of the electronic device 101 .

[0346] The processor 120 may configure a condition 2610 “if at home” and an action 2620 “display a phone icon and a camera icon in a locked state of the electronic device 101”. The processor 120 may display an object 2630 for selecting whether to display a notification about a routine related to the locked state of the electronic device 101 through the display device 160 .

[0347] However, the method for generating the routine related to the lock state of the electronic device 101 is not limited to the above referenced Fig.26 Describe the method.

[0348] Fig. 27 is an illustration 2700 of a method for recommending a routine of an electronic device 101 according to an embodiment.

[0349] Reference Fig. 27 , the processor 120 may generate (or configure) a routine. For example, as shown in panel 2710, the processor 120 may generate a routine including a routine name 2711 (e.g., Routine 1), a condition 2712 (e.g., "if entering charging state"), and an action 2713 (e.g., "execute blue light filtering function").

[0350] The processor 120 may recommend a routine (e.g., an action) based at least in part on the usage pattern of the user of the electronic device 101. For example, if the condition 2712 (e.g., “if entering the charging state”) is satisfied, the processor 120 may display, through the display device 160, information 2714 for recommending an application related to a function frequently performed by the electronic device 101.

[0351] The processor 120 may display, through the display device 160 , an object 2715 for selecting whether to display a notification related to the routine.

[0352] After the routine is generated, the processor 120 may display information indicating that the routine is being executed through the display device 160. Based at least in part on the user input regarding the information indicating that the routine is being executed, the processor 120 may display information recommending applications related to functions most frequently executed by the electronic device 101 when the routine (e.g., the conditions of the routine) are satisfied through the display device 160. For example, as shown in the panel 2720, the processor 120 may display information 2723 recommending applications related to functions most frequently executed by the electronic device 101 when the routine is satisfied together with the routine name 2721 and the action 2722 through the display device 160.

[0353] However, the routine recommendation method is not limited to the above reference Fig. 27 An example of description.

[0354] In addition, the structure of the data used in the above-mentioned embodiments can be recorded in a non-transitory computer-readable recording medium in various ways. Non-transitory computer-readable recording media include storage media such as magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.) and optically readable media (e.g., CD-ROM, digital versatile disk (DVD), etc.).

[0355] The present disclosure is disclosed above in conjunction with the embodiments of the present disclosure. Those skilled in the art to which the present disclosure belongs will understand that the present disclosure can be implemented in a modified form without departing from the scope of the present disclosure. Therefore, the present disclosure should be considered from an illustrative perspective rather than a restrictive perspective. The scope of the present disclosure is defined by the attached claims and their equivalents.

Claims

1. An electronic device, comprising: user interface; at least one processor comprising processing circuitry; and Memory, storage instructions, The instruction is executed by the at least one processor so that the electronic device performs the following operations: When the electronic device is in a first state, determining whether a first condition set in a first routine is satisfied; In response to a first condition being satisfied, executing a first action to change the state of the electronic device from a first state to a second state according to a first routine and storing a first state value of the electronic device before the electronic device executes the first action; identifying whether a second condition set in the second routine is satisfied; In response to a second condition being satisfied, executing a second action of changing the turntable of the electronic device from the second state to the third state according to a second routine and storing a second state value of the electronic device before the electronic device executes the second action; When the state of the electronic device is a third state, identifying whether at least one of the first condition and the second condition is not satisfied; In response to the first condition being satisfied and the second condition not being satisfied, changing the state of the electronic device from the third state to the second state; In response to the first condition not being met, the second state value is replaced with the first state value and the stored first state value is deleted from the memory, and if the second condition is met, the state of the electronic device is maintained in the third state, and if the second condition is not met, the state of the electronic device is changed from the third state to the first state by replacing the state value of the electronic device with the changed second state value.

2. The electronic device of claim 1, further comprising at least one of a communication interface or a sensor module, in, The instructions are executed by the at least one processor to cause the electronic device to perform the following operations: determine whether a first condition is met based at least in part on data acquired by the at least one of the communication interface or the sensor module.

3. The electronic device as claimed in claim 2, wherein: The instructions are executed by the at least one processor to cause the electronic device to perform the following operations: generate a first routine including a first condition and a first action.

4. The electronic device as claimed in claim 3, wherein: The instructions are executed by the at least one processor to cause the electronic device to perform the following operation: in a locked state of the electronic device, if a first condition is satisfied, displaying information indicating that a first routine is being executed through the user interface.

5. The electronic device as claimed in claim 4, wherein: Execution of the instructions by the at least one processor causes the electronic device to display detailed information about the first routine based at least in part on user input regarding information indicating that the first routine is being performed.

6. The electronic device as claimed in claim 4, wherein: The instructions are executed by the at least one processor to cause the electronic device to perform the following operations: generating a first routine that can be executed in a locked state of the electronic device.

7. The electronic device as claimed in claim 3, wherein: The instructions are executed by the at least one processor so that the electronic device performs the following operations: acquiring a log related to a changed state in the electronic device when a first condition is satisfied within a specified time period; Based at least in part on the log, determining a second action related to a changed state in the electronic device; and Recommended second action.

8. The electronic device as claimed in claim 3, wherein: The instructions are executed by the at least one processor so that the electronic device performs the following operations: obtaining information about a user pattern of the electronic device within a specified time period; generating a second routine based at least in part on the information regarding the user pattern; If a second condition included in the second routine is different from the first condition, recommending the second routine; as well as The notification is output through the user interface such that if the second condition is the same as the first condition and if the second action included in the second routine is different from the first action, the first routine further includes the second action.

9. The electronic device as claimed in claim 3, wherein: The instructions are executed by the at least one processor to cause the electronic device to perform the following operation: displaying information related to the first routine through the user interface.

Images