Electronic device and control method therefor

Abstract

An electronic device is disclosed. The electronic device comprises: an input interface; a memory for storing action information including detailed operations of each of a plurality of actions, condition information including trigger information of each of a plurality of conditions, and at least one instruction; and at least one processor for executing the at least one instruction, wherein the at least one processor, when a user input is received, obtains at least one candidate action from among the plurality of actions and at least one candidate condition from among the plurality of conditions on the basis of a keyword included in the received user input, and generates a user routine by using the detailed operation corresponding to the obtained at least one candidate action and the trigger information corresponding to the obtained at least one candidate condition, and sets the user routine in the electronic device.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device for creating and setting user routines and a method for controlling the same.

[0002] Thanks to advancements in electronic technology, various types of electronic devices are being used in daily life. Among these electronic devices may be user terminal devices that create and configure user routines.

[0003] User routines refer to automation functions that configure electronic devices to automatically perform specific actions based on specific conditions.

[0004] According to at least one embodiment of the present disclosure, an electronic device comprises an input interface, action information including detailed operation for each of a plurality of actions, condition information including trigger information for each of a plurality of conditions, a memory storing at least one instruction, and at least one processor executing said at least one instruction. When user input is received, said at least one processor may obtain at least one candidate action among the plurality of actions and at least one candidate condition among the plurality of conditions based on keywords included in the received user input, and may generate a user routine using a detailed operation corresponding to the obtained at least one candidate action and trigger information corresponding to the obtained at least one candidate condition and set the user routine in the electronic device.

[0005] Additionally, according to at least one embodiment of the present disclosure, a control method for an electronic device generating a user routine may include the steps of, upon receiving a user input, obtaining at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on a keyword included in the received user input, and generating the user routine and setting it on the electronic device using a detailed operation corresponding to the obtained at least one candidate action and trigger information corresponding to the obtained at least one candidate condition.

[0006] Additionally, a non-transient readable recording medium according to at least one embodiment of the present disclosure stores a program for performing a method of controlling an electronic device, comprising the steps of: when a user input is received, obtaining at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on a keyword included in the received user input; and generating a user routine using a detailed operation corresponding to the obtained at least one candidate action and trigger information corresponding to the obtained at least one candidate condition and setting it on the electronic device.

[0007] FIG. 1 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0008] FIG. 2 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0009] FIG. 3 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0010] FIG. 4 is a drawing for illustrating an example in which an electronic device according to at least one embodiment of the present disclosure generates a user routine.

[0011] FIG. 5 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0012] FIG. 6 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0013] FIG. 7 is a drawing for illustrating an example in which an electronic device according to at least one embodiment of the present disclosure generates a user routine.

[0014] FIG. 8 is a flowchart illustrating an example of an electronic device setting a user routine according to at least one embodiment of the present disclosure.

[0015] FIG. 9 is a flowchart illustrating a method for an electronic device to generate a user routine according to at least one embodiment of the present disclosure.

[0016] FIG. 10 is a flowchart illustrating a method for an electronic device to generate a user routine according to at least one embodiment of the present disclosure.

[0017] The terms used in the various embodiments of this Disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0018] The various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0019] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0020] The singular form of the noun corresponding to the item may include one or multiple items, unless the relevant context clearly indicates otherwise.

[0021] In the present disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0022] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0023] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0024] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0025] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0026] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0027]

[0028] The term "and / or" includes a combination of multiple related described components or any of the multiple related described components.

[0029] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0030] Meanwhile, various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present disclosure is not limited by the relative sizes or spacing depicted in the attached drawings.

[0031] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0032] Hereinafter, with reference to the attached drawings, the operation of the sperm device (100) will be described in detail along with specific configuration examples.

[0033] FIG. 1 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0034] Referring to FIG. 1, the electronic device (100) includes an input interface (110), a memory (120), and at least one processor (130).

[0035] The electronic device (100) of the present disclosure may be implemented having more components than the components shown in FIG. 1, or having fewer components.

[0036] For example, if the electronic device (100) according to an embodiment of the present disclosure supports a function of recognizing a user's voice through a microphone to set a user routine, a configuration of a microphone (not shown) may be further included. Additionally, if the electronic device (100) according to an embodiment of the present disclosure performs the operation of the present disclosure by communicating with an external device, a configuration of a communication interface (not shown) may be further included.

[0037] The input interface (110) can receive various feedback from the user. For example, the user can receive feedback from the user setting a user routine through the input interface (110).

[0038] Here, "user routine" may refer to an automation function that sets the electronic device (100) to automatically perform a specific action according to specific conditions.

[0039] The input interface (110) may include multiple input interfaces such as a microphone and a touch screen.

[0040] For example, the electronic device (100) can recognize the user's voice through a microphone and set a user routine corresponding to the user's request. However, it is not limited to this, and the electronic device (100) may also receive user feedback through a touch screen.

[0041] The memory (120) can store data necessary for various embodiments of the present disclosure. For example, the memory (120) can store action information including detailed operation of each of a plurality of actions, condition information including trigger information for each of a plurality of conditions, etc.

[0042] Depending on the purpose of data storage, the memory (120) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100).

[0043] For example, data for driving the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that is detachable from the electronic device (100).

[0044] In the case of memory embedded in an electronic device (100), it may be implemented in the form of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD).

[0045] In the case of a memory that can be attached to and detached from an electronic device (100), it can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory).

[0046] The memory (120) may include various instructions required for the operation of at least one processor (130). Here, the instructions may include an instruction for obtaining at least one candidate action and candidate condition, an instruction for creating and setting a user routine based on the obtained at least one candidate action and condition, an instruction for user voice recognition or voice processing, an instruction for receiving user input, and an instruction for performing an action corresponding to the detailed operation of a candidate action according to the occurrence of an event.

[0047] The memory (120) can store user pattern information. The user pattern information may include at least one of user routine information set by the user or history information of actions executed by the user.

[0048] Here, "user routine information set by the user" may include information regarding a condition and an action selected by the user among at least one candidate condition and at least one candidate action proposed by the electronic device (100). The operation of creating and setting a user routine by the user is described in detail in FIGS. 3 to 5.

[0049] Here, "history information on actions executed by the user" may include action information containing detailed operations of the action executed by the user and condition information corresponding to the executed action.

[0050] For example, if a user boards a vehicle during the evening commute and launches a specific app, the detailed operation of the launched specific app is included in the action information, and information regarding the condition of boarding the vehicle during the evening commute may be included in the condition information corresponding to the launched action. The operation of setting a user routine based on the history information of the user launching an action is explained in detail in FIG. 5.

[0051] According to an embodiment, at least one processor (130) controls the overall operation of the electronic device (100). Specifically, at least one processor (130) is connected to each component of the electronic device (100) to control the overall operation of the electronic device (100).

[0052] According to an embodiment, at least one processor (130) may control various external devices, such as smart home appliances. For example, the electronic device (100) may perform operations such as adjusting the brightness of a smart light or receiving operation history information from a smart light. An embodiment in which the operation of the present disclosure is performed by communicating with an external device is described in detail in FIG. 7.

[0053] At least one processor (130) can perform the operation of an electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0054] For example, when at least one processor (130) receives user input, it can obtain at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input.

[0055] Here, user input may be received through an input interface (110) or through a communication unit (not shown). User input may include at least one of user voice input, touch input, or button input. For example, the user may input feedback to the electronic device (100) in various forms, such as spoken voice or text.

[0056] At least one processor (130) may obtain text corresponding to candidate conditions and candidate actions according to the user's intent based on the user's voice input, if the user input is user voice input.

[0057] For example, at least one processor (130) can input natural language-based commands entered by a user into a Large Language Model (LLM) to extract keywords for user routine settings in the form of text. A specific example of this is described in detail in FIG. 4.

[0058] At least one processor (130) may obtain candidate actions or candidate conditions that consider the user's intent by utilizing a model or database learned based on user pattern information, such as user routine information set by the user or history information of actions executed by the user. The operation of obtaining candidate actions or candidate conditions that consider the user's intent is described in detail in FIG. 4.

[0059] Here, "multiple actions" may include various operations that the electronic device (100) can perform. For example, the operation of running a video application to play a video of a specific channel may be an action that includes detailed operations.

[0060] However, this is not limited to this, and actions such as launching a video application to adjust volume or screen brightness may also be included in the detailed actions of the action.

[0061] Here, "multiple conditions" may include trigger information for the electronic device (100) to perform a specific action. For example, if a user has set a user routine to "run a navigation app when the vehicle is in the vehicle and the vehicle and the electronic device (100) are connected via Bluetooth," the trigger information for "connecting via Bluetooth to the vehicle" may correspond to the conditions.

[0062] However, it is not limited to this, and the condition may include multiple trigger information. Trigger information may refer to information about conditions that cause a specific action to be performed.

[0063] At least one processor (130) can create a user routine and set it in the electronic device (100) using detailed operation corresponding to at least one acquired candidate action and trigger information corresponding to at least one acquired candidate condition.

[0064] Here, detailed actions corresponding to candidate actions and trigger information corresponding to candidate conditions may be information selected by the user. When the electronic device (100) provides information about candidate actions and candidate conditions to the user based on user input, a user routine may be generated based on the action, condition, detailed actions corresponding to the action, or trigger information corresponding to the condition selected by the user.

[0065] At least one processor (130) can monitor the occurrence of an event corresponding to trigger information when a user routine is set, and when an event occurs, perform an action corresponding to the detailed action of a candidate action.

[0066] For example, if a user sets a user routine in the electronic device (100) to “unlock the screen orientation when watching a video,” at least one processor (130) monitors whether an event called “video playback” occurs, and if it detects the occurrence of an event called “video playback,” it can execute a function to “unlock the screen orientation” among the functions of the settings application of the electronic device (100).

[0067] According to an embodiment, at least one processor (130) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a TCON (Timing controller). However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an AI (Artificial Intelligence) processor. Additionally, at least one processor (130) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). At least one processor (130) can perform various functions by executing computer executable instructions stored in memory.

[0068] At least one processor (130) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator.

[0069] At least one processor (130) can control one or any combination of other components of the electronic device and can perform operations or data processing related to communication. At least one processor (130) can execute one or more programs or instructions stored in memory. For example, at least one processor (130) can perform the method according to an embodiment of the present disclosure by executing one or more instructions stored in memory.

[0070] If the method according to the embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors.

[0071] For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence dedicated processor).

[0072] At least one processor (130) may be implemented as a single core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore).

[0073] When at least one processor (130) is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multi-core processor.

[0074] Each of the multiple cores (or some of the multiple cores) included in the multi-core processor may independently read and execute program instructions for implementing the method according to the embodiment of the present disclosure, or all (or some of) of the multiple cores may be linked together to read and execute program instructions for implementing the method according to the embodiment of the present disclosure.

[0075] FIG. 2 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0076] Referring to FIG. 2, an electronic device (100) according to one embodiment of the present disclosure may include an input interface (110), a memory (120), at least one processor (130), and at least one sensor (140). Parts that overlap with the description above will be omitted or abbreviated below.

[0077] The electronic device (100) can sense user input, user feedback, the state of the electronic device (100), the surrounding environment of the electronic device (100), etc. by using at least one sensor (140), such as a microphone sensor.

[0078] At least one sensor (140) may include at least one of a position sensor for detecting a position (e.g., a GPS sensor, etc.), a microphone sensor for detecting voice, an accelerometer sensor for detecting rotational movement of an electronic device, an illuminance sensor for detecting ambient illuminance, a temperature sensor for measuring the internal temperature of an electronic device, or a pressure sensor for detecting touch input.

[0079] For example, the electronic device (100) can detect a voice utterance requesting a user routine setting through a microphone sensor. When the electronic device (100) receives user input in the form of audio through the microphone sensor, it may also obtain user input in the form of text by utilizing various models for processing natural language.

[0080] If the trigger information corresponding to at least one candidate condition includes voice information, the electronic device (100) can identify whether an event has occurred by monitoring the occurrence of an event corresponding to the trigger information through a sensor that detects voice.

[0081] For example, if a user sets a user routine so that a "sound mode" is executed in the electronic device (100) when ambient noise exceeds a preset level, the electronic device (100) can execute the sound mode by detecting the noise exceeding the preset level through a microphone sensor as an event corresponding to trigger information.

[0082] However, this is not limited to this, and the sensor performing the sensing operation may differ depending on the trigger information corresponding to the candidate condition. Additionally, depending on the trigger information corresponding to the candidate condition, not only a single sensor but also multiple sensors may perform the sensing operation.

[0083] FIG. 3 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0084] Referring to FIG. 3, when a request for the creation of a user routine is received as user input, the electronic device (100) may suggest candidate actions or candidate conditions and create a user routine based on the user's selection.

[0085] Specifically, the electronic device (100) can detect that a user requests the electronic device (100) to create a routine through user input (S310). For example, the user may request the electronic device (100) to create a user routine by performing text input "create routine" and inputting "delete screenshots I haven't seen for a month".

[0086] When the electronic device (100) receives user input, it may suggest candidate actions or candidate conditions to be set as user routines based on user input (S320).

[0087] Specifically, the received user input can be extracted in the form of keywords, and based on the keywords, at least one candidate action among multiple actions and at least one candidate condition among multiple conditions can be obtained.

[0088] For example, the electronic device (100) can obtain a candidate action to “delete a screenshot” by running a gallery application based on the keyword “delete screenshot”, and obtain a candidate condition with “a screenshot that has been clicked for more than a month” as trigger information based on the keyword “screenshot not viewed for a month”.

[0089] When a user requests the creation of a routine to generate candidate actions and candidate conditions as a user routine, the electronic device (100) can create a user routine including action and condition information based on user input (S330).

[0090] Referring to FIG. 3, the electronic device (100) can create and set a screenshot deletion routine based on user input called routine creation.

[0091] FIG. 4 is a drawing for illustrating an example in which an electronic device according to at least one embodiment of the present disclosure generates a user routine.

[0092] Referring to FIG. 4, when a request for the creation of a user routine is received as user input, the electronic device (100) may suggest a plurality of candidate actions or candidate conditions and create a user routine based on the user's selection.

[0093] Specifically, the electronic device (100) can detect that a user requests the electronic device (100) to create a routine through user input (S410). For example, the user may perform text input "create routine" and request the electronic device (100) to create a user routine through input "unlock landscape / portrait when watching video".

[0094] When the electronic device (100) receives user input, it may suggest a plurality of candidate actions or candidate conditions to be set as user routines based on the user input (S420).

[0095] Specifically, the received user input can be extracted in the form of keywords, and based on the keywords, at least one candidate action among multiple actions and at least one candidate condition among multiple conditions can be obtained.

[0096] For example, the electronic device (100) can obtain candidate actions such as “screen auto-rotation” or “screen landscape lock” by running a settings application based on the keyword “unlock landscape portrait”, and obtain candidate conditions such as “play video with N-Player app” or “play video with D-Player app” as trigger information based on the keyword “when watching video”.

[0097] The electronic device (100) may consider the user's intent when obtaining and proposing candidate actions or candidate conditions. For example, a video application that the user has not run on the electronic device (100) for more than one year may be excluded from candidate actions. That is, candidate conditions may be proposed by considering the user's pattern information, etc.

[0098] If the user suggests a user routine based on a plurality of candidate actions and candidate conditions obtained by the electronic device (100), some of the proposed plurality of candidate actions and candidate conditions can be selected as the user routine (S430).

[0099] When a user requests the creation of a routine that generates actions and conditions selected by the user as a user routine, the electronic device (100) can create a user routine that includes action and condition information based on user input (S430).

[0100] FIG. 5 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0101] Referring to FIG. 5, when a specific action is executed based on user input and a request for the creation of a user routine is received, the electronic device (100) may suggest candidate actions or candidate conditions and create a user routine by reflecting the user's selection and the user's detailed action request.

[0102] Specifically, the electronic device (100) can detect when a user gets into a vehicle and runs a specific app (S510). For example, the electronic device (100) can detect a situation where a user gets into a vehicle during the evening commute, connects the electronic device (100) to the vehicle via Bluetooth, runs a navigation app and a music app, and switches the screen to landscape mode.

[0103] When an action is executed based on user input, the electronic device (100) may identify a condition corresponding to the action based on a sensing value from at least one sensor (140).

[0104] The electronic device (100) can receive user input requesting the creation of a user routine after the above-described situation is detected (S520).

[0105] When the electronic device (100) receives input from a user, it may suggest candidate actions or candidate conditions to be set as user routines based on the detected situation (S530).

[0106] For example, the electronic device (100) may suggest a detailed action as a candidate action for the user to get into a vehicle and "run a navigation app and set the starting point to Building A and the destination to Building B." Additionally, the electronic device (100) may suggest a setting as a candidate action for running a music app and switching the screen to landscape mode.

[0107] Additionally, the electronic device (100) may suggest conditions such as "Bluetooth connection" or "time zone between 6 and 7 PM" as candidate conditions to activate the selected action.

[0108] The user can select some or all of the proposed candidate conditions and candidate actions, and request detailed operation of the selected action.

[0109] For example, the electronic device (100) may select all proposed candidate actions and candidate conditions, and when user input requesting a detailed action such as “navigation volume at highest, music at medium” is received, it may generate a user routine reflecting the user’s selection and the user’s request for a detailed action. The user routine may include actions based on user input and conditions corresponding to the actions.

[0110] Meanwhile, in FIGS. 3 to 5, the electronic device (100) is shown in the form of a mobile phone, but is not limited thereto, and the electronic device (100) can be implemented as various types of electronic devices such as a TV, laptop, monitor, kiosk, tablet PC, smartphone, user terminal, electronic picture frame, LFD (large format display), Digital Signage, DID (Digital Information Display), video wall, mobile projector, display device, server device, etc.

[0111] FIG. 6 is a diagram illustrating an example of an electronic device generating a user routine according to at least one embodiment of the present disclosure.

[0112] Referring to FIG. 6, when a specific action is executed based on user input and a request for the creation of a user routine is received, the electronic device (100) may suggest candidate actions or candidate conditions or create a user routine based on user input.

[0113] Specifically, the electronic device (100) can detect a situation in which the user watches media before falling asleep (S610). For example, the electronic device (100) can detect a situation in which the user watches media before falling asleep through at least one sensor (140) as an action and condition of watching media and content while lying down during the evening.

[0114] The electronic device (100) can receive input that adjusts actions while the user is watching media during the evening (S620). For example, it can receive input from the user that lowers the brightness and sets the screen settings to comfortable viewing while the user is watching media during the evening.

[0115] In this case, the electronic device (100) may generate a routine (S630-1) based on user input from the user, such as “switch to an eye-friendly mode when watching video content after 9 PM,” but may also suggest candidate actions and candidate conditions by detecting actions that adjust the illumination and screen settings.

[0116] The operation of generating a user routine by selecting some or all of the proposed candidate actions and conditions has been described above, so a redundant explanation is omitted.

[0117] FIG. 7 is a drawing for illustrating an example in which an electronic device according to at least one embodiment of the present disclosure generates a user routine.

[0118] Referring to FIG. 7, the electronic device (100) detects user input that executes a specific action during sleep time and receives user setting information of an external electronic device, such as a smart device, and may generate routines for the electronic device (100) according to the present disclosure as well as for the external electronic device.

[0119] For example, the user can adjust the settings of the smart light and electronic device (100) during sleep time (S710). For example, if the illuminance of the smart light is adjusted to 30% and the screen settings of the electronic device are adjusted to dark mode, the smart light and electronic device (100) can suggest candidate routines and candidate actions together based on the setting information received from the smart light and the sensing value received from at least one sensor (140) of the electronic device (S730).

[0120] When a user requests the creation of a routine by selecting actions and conditions for the smart light, the electronic device (100) may transmit the created routine information to the smart light to control the operation of the smart light.

[0121] However, it is not limited thereto, and the electronic device (100) may generate and propose a modified user routine by comparing an action set as a user routine based on user input with an action based on user input.

[0122] For example, if an action based on user input does not match an action set as a user routine, a modified user routine based on the action based on user input can be generated and suggested.

[0123] FIG. 8 is a flowchart illustrating an example of an electronic device setting a user routine according to at least one embodiment of the present disclosure.

[0124] Referring to FIG. 8, the electronic device (100) can recognize user input (S810) and obtain a candidate action or candidate condition (S820). The electronic device (100) determines whether a candidate action or candidate condition has been obtained (S830), and if a candidate action or candidate condition has not been obtained, it may request re-input.

[0125] That is, if the electronic device (100) does not obtain at least one candidate condition or at least one candidate action corresponding to the user input, it may request the user input again.

[0126] If there is a lack of information regarding candidate actions or candidate conditions, the electronic device (100) may search for candidate actions or candidate conditions based on user pattern information or a sensing value from at least one sensor (140) (S821). The electronic device (100) may also supplement the input for candidate actions or candidate conditions by extracting final candidate action or candidate condition information based on the searched information (S822).

[0127] The electronic device (100) can display information about candidate actions or candidate conditions and request a selection (S840).

[0128] The electronic device (100) may request the user to re-enter additional details even if there are additional details required (S850).

[0129] The electronic device (100) can perform user routine generation and setting based on user input (S860).

[0130] FIG. 9 is a flowchart illustrating a method for an electronic device to generate a user routine according to at least one embodiment of the present disclosure.

[0131] When the electronic device receives user input, it can obtain at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input (S910).

[0132] For example, when user input is received through a user's voice utterance, keywords included in the user's voice utterance can be analyzed to obtain at least one candidate action and at least one candidate condition.

[0133] The electronic device can create a user routine and set it on the electronic device using detailed operation corresponding to at least one candidate action obtained and trigger information corresponding to at least one candidate condition obtained (S920).

[0134] For example, when an electronic device acquires trigger information corresponding to a candidate condition, a user routine can be created and configured to perform a detailed action corresponding to a candidate action.

[0135] FIG. 10 is a flowchart illustrating a method for an electronic device to generate a user routine according to at least one embodiment of the present disclosure.

[0136] When an electronic device receives user input, it can obtain at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input (S1010).

[0137] The electronic device can create a user routine and set it on the electronic device using detailed operation corresponding to at least one acquired candidate action and trigger information corresponding to at least one acquired candidate condition (S1020).

[0138] When a user routine is set, the electronic device monitors the occurrence of an event corresponding to the trigger information, and when the event occurs, it can perform an action corresponding to the detailed operation of the candidate action (S1030).

[0139] Therefore, when an event corresponding to the trigger information included in the condition set by the user routine occurs, the electronic device can perform an action corresponding to the detailed operation of the candidate action without separate user operation.

[0140] The user routine generation method described in FIGS. 9 and 10 can be performed by a device having various configurations such as FIGS. 1 and 2 described above, but is not necessarily limited thereto and can be performed by a device having various configurations.

[0141] The various embodiments described above may be implemented as individual embodiments, or at least one embodiment may be combined with one another, either wholly or partially, to be implemented together in a single device.

[0142] According to the various embodiments described above, the user can conveniently set up user routines by having the electronic device suggest candidate actions or candidate conditions based on user input. Ultimately, the user experience can be improved.

[0143] Various embodiments of the present disclosure may be implemented as software stored on a machine-readable storage media that can be mounted on or connected to a smartphone, a user terminal device, and various other electronic devices (e.g., a computer).

[0144] Specifically, a non-transient readable storage medium may be provided that stores software for sequentially performing the steps of: obtaining at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input when user input is received; and generating a user routine and setting it in an electronic device using detailed operation corresponding to the obtained at least one candidate action and trigger information corresponding to the obtained at least one candidate condition.

[0145] A device equipped with such a non-transient readable medium can perform various operations such as tag identification corresponding to user actions described in the various embodiments above, verification of importance per multiple frames, and generation of edited images.

[0146] In non-transitory readable storage media, 'non-transitory' simply means that the storage medium does not contain a signal and is tangible; it does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

[0147] Alternatively, a program for performing the methods according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0148] Each component (e.g., module or program) according to various embodiments may consist of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration.

[0149] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0150] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Input interface; A memory storing action information including detailed operation for each of a plurality of actions, condition information including trigger information for each of a plurality of conditions, and at least one instruction; and It includes at least one processor that executes the above at least one instruction; and The above-mentioned at least one processor is, When user input is received, at least one candidate action among the plurality of actions and at least one candidate condition among the plurality of conditions are obtained based on keywords included in the received user input, and An electronic device that generates a user routine and sets it in the electronic device using a detailed operation corresponding to at least one candidate action obtained above and trigger information corresponding to at least one candidate condition obtained above.

2. In Paragraph 1, The above-mentioned at least one processor is, An electronic device that, when the above user routine is set, monitors the occurrence of an event corresponding to the above trigger information, and when the above event occurs, performs an action corresponding to the detailed operation of the above candidate action.

3. In Paragraph 1, The above user input is, Includes at least one of user voice input, touch input, or button input, and The above-mentioned at least one processor is, An electronic device that, if the above user input is user voice input, obtains text corresponding to candidate conditions and candidate actions according to the user's intention.

4. In Paragraph 1, The above-mentioned at least one processor is, An electronic device that requests user input again if at least one candidate condition or at least one candidate action corresponding to the above user input is not obtained.

5. In Paragraph 1, Includes at least one sensor; and The above-mentioned at least one processor is, If information regarding at least one candidate condition or at least one candidate action corresponding to the above user input is not obtained, at least one candidate condition or at least one candidate action is obtained based on user pattern information or a sensing value from the at least one sensor, and The above user pattern information is, An electronic device comprising at least one of user routine information set by the user or history information in which the user executed an action.

6. In Paragraph 1, Includes at least one sensor; and The above-mentioned at least one sensor is, An electronic device comprising at least one of a sensor for detecting position, a sensor for detecting voice, a sensor for detecting rotational movement of the electronic device, a sensor for detecting ambient light, or a sensor for detecting touch input.

7. In Paragraph 6, The above-mentioned at least one processor is, An electronic device that identifies whether an event has occurred by monitoring the occurrence of an event corresponding to the trigger information through a sensor that detects the voice, wherein the trigger information corresponding to at least one candidate condition includes voice information.

8. In Paragraph 1, Includes at least one sensor; and The above-mentioned at least one processor is, When a condition set as a user routine is identified based on a sensing value from at least one of the above sensors, and an action is executed based on user input, Compare actions based on user input with actions set as user routines, and An electronic device that, if the action based on the above user input and the action set as the above user routine do not match, generates and proposes a modified user routine based on the action based on the above user input.

9. In Paragraph 1, Includes at least one sensor; and The above-mentioned at least one processor is, When an action is executed based on user input, a condition corresponding to the action is identified based on a sensing value from at least one sensor, and An electronic device that generates and proposes a user routine including an action based on the above user input and a condition corresponding to the above action.

10. A method for controlling an electronic device that generates a user routine, When user input is received, a step of obtaining at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input; and A control method comprising the step of generating a user routine and setting it on an electronic device using a detailed action corresponding to at least one candidate action obtained above and trigger information corresponding to at least one candidate condition obtained above.

11. In Paragraph 10, A control method further comprising the step of, when the above user routine is set, monitoring the occurrence of an event corresponding to the above trigger information, and when the above event occurs, performing an action corresponding to the detailed operation of the above candidate action.

12. In Paragraph 10, The above user input is, Includes at least one of user voice input, touch input, or button input, and A control method further comprising the step of obtaining text corresponding to candidate conditions and candidate actions according to the user's intent when the above user input is user voice input.

13. In Paragraph 10, A control method further comprising the step of requesting user input again if at least one candidate condition or at least one candidate action corresponding to the above user input is not obtained.

14. In Paragraph 10, If information regarding at least one candidate condition or at least one candidate action corresponding to the above user input is not obtained, the method further includes the step of obtaining at least one candidate condition or at least one candidate action based on user pattern information or a value sensed by the electronic device. The above user pattern information is, A control method comprising information including at least one of user routine information set by the user or history information of actions executed by the user.

15. A non-transient readable recording medium comprising a program for executing a method of controlling an electronic device, The control method of the above electronic device is, When user input is received, a step of obtaining at least one candidate action among a plurality of actions and at least one candidate condition among a plurality of conditions based on keywords included in the received user input; and A non-transient readable recording medium comprising: a step of generating a user routine and setting it on the electronic device using detailed operation corresponding to at least one candidate action obtained above and trigger information corresponding to at least one candidate condition obtained above.

Images