Method and apparatus for configuring and recommending device action using user context

[0030] With reference to the accompanying drawings, the following description is provided to facilitate a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. The following description includes various specific details to facilitate understanding, but these specific details are only regarded as exemplary. Therefore, those of ordinary skill in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, for clarity and brevity, descriptions of well-known functions and configurations may be omitted.

[0031] The terms and words used in the following description are not limited to the literal meaning, but are only used by the inventor to provide a clear and consistent understanding of the present invention. Therefore, it should be clear to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only, and is not intended to limit the present disclosure as defined by the appended claims and their equivalents.

[0032] It should be understood that, unless the context clearly dictates otherwise, the singular forms "a", "an" and "the" include plural forms. Thus, for example, reference to "the surface of a component" includes one or more of such surfaces.

[0033] In this specification and claims, the term "comprising" should not mean excluding other components or operations. Unless otherwise indicated, singular nouns shall encompass plural nouns. For example, "user action" or "state" may indicate multiple user actions or multiple states. The suffix "unit" used to indicate components in the following description is only introduced or used interchangeably for the purpose of describing the present disclosure, and therefore does not provide a unique meaning or function to distinguish each other. In this specification and claims, the terms "first", "second", "third", etc. are used to distinguish similar elements, and are not necessarily used to describe the elements in order or chronological order. In this specification and claims, "first", "second" and "third" are not used as mutually exclusive meanings. For example, the devices included in the "first device group" may be included in the "third device group".

[0034] In this specification and claims, a rule may refer to the relationship between a condition and an action triggered by the condition, and is determined in consideration of the history of state information. Rules can be interchanged with pattern services. In this specification and claims, a condition may refer to a specific state or state change of one or more devices included in the smart home system. In this specification and claims, actions may refer to user operations of one or more devices included in the smart home system.

[0035] The rules can be expressed as shown in Table 1. As shown in Table 1, the condition may include the state of one or more devices, and the action may include the action of one or more devices.

[0036] 【Table 1】

[0037]

[0038] The rules may have unique names and icons defined by the user or recommended by the smart home system. When the rule includes conditions and actions, each of the conditions and actions can have a unique name and icon. The rule name may correspond to, for example, sales, rainy, away, or sleep. The condition name may correspond to, for example, Near Field Communication (NFC), door, date and time, weather, temperature, humidity, or lamps in the living room. The action name may correspond to, for example, one of lamps, safety, and air conditioning. The equipment can be classified into, for example, household appliances/lighting equipment and other equipment according to equipment types. The rules can also include device type information.

[0039] The following sentences express the rules that include actions (if clauses) and actions (then words).

[0040] If the current time is 10:00-21:00 and it is raining, the lights in the living room are turned off. If Tom leaves and no movement is detected in Tom's room within 5 minutes, the light in Tom's room is turned off. If the temperature at Jenny’s location is 30 degrees Celsius or higher, the air conditioner is turned on. If someone leaves after 9 o'clock, the camera and sensor are activated. If the washing has been completed by the washing machine, then this status is notified using pop-up messages and light-emitting diode (LED) lamps. If the proximity of the NFC tag belonging to Alice is detected, the computer in Alice's room is turned on and a word processing program is executed. If the power consumed yesterday is twice the power consumed two days ago, a warning message is displayed on Maggie's terminal.

[0041] The detection of conditions and actions through the gateway may refer to detection through a sensor or input device included in the gateway, or may refer to receiving sensing information or input information of another device through the gateway.

[0042] Specifically, the condition may indicate the external environment information measured by the sensor of the smart home system or the state or state change of one or more devices included in the smart home system. In this specification and claims, "condition" can be used interchangeably with "user context" or "device status". Table 2 shows examples of conditions.

[0043] 【Table 2】

[0044]

[0045] With reference to the external conditions in Table 2, the time may correspond to one of dawn, morning, noon, afternoon, evening, and night. In the household appliances/lighting equipment of Table 2, a television (TV) may correspond to one of on, off, channel 1 display, and channel 2 display. In other devices in Table 2, the thermostat can correspond to one of high temperature, normal temperature, and low temperature. Information related to the conditions detected by each device can be transmitted to the gateway device.

[0046] Specifically, the action indicates the action of one or more devices included in the smart home system. In this specification and claims, "action" can be used interchangeably with "user operation". Table 3 shows examples of actions.

[0047] 【table 3】

[0048]

[0049] In household appliances/lighting equipment, the LED can detect an action corresponding to one of the actions of turning on the power, turning off the power, lighting up the screen, and darkening the screen. In other devices, the motion detector can detect an action corresponding to one of motion detection and no motion detection. Information related to the actions detected by each device can be transmitted to the gateway device. Table 3 only shows examples of actions, and the user can edit the actions to determine whether to repeat the action, the number of repetitions, and the repetition time interval. Information related to the actions detected by each device can be transmitted to the gateway device.

[0050] Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0051] figure 1 The configuration of a smart home system according to an embodiment of the present disclosure is shown.

[0052] reference figure 1 The smart home system may include a gateway 110, a user equipment (UE) 120, an associated device 130, a smart appliance 140, and a remote server 150, but it is not limited thereto.

[0053] The gateway 110 may also be referred to as a gateway device, and may communicate with the UE 120, the associated device 130, the smart appliance 140, and/or the remote server 150. The gateway 110 may store data received from the UE 120, the associated device 130, the smart appliance 140, and/or the remote server 150 or data required to control the UE 120, the associated device 130, the smart appliance 140, and/or the remote server 150. The gateway 110 may include one or more displays (touch screen), which may display a user interface (UI). According to an embodiment of the present disclosure, the gateway 110 may perform all or some of the functions of the UE 120, the associated device 130, or the smart appliance 140. In other words, smart phones, wearable devices, tablet personal computers (PCs), vacuum cleaners, refrigerators, TVs, thermostats, etc. can all be used as the gateway 110.

[0054] The UE 120 is an electronic device configured to execute applications driven in an operating system (OS) or display content. The UE 120 may include a display (touch screen) for displaying the UI, a communication unit for communicating with external devices, a storage unit for storing data, an input unit for receiving user input, or a device for controlling the execution of each action. Controller. The UE 120 may include, for example, a tablet PC, a portable multimedia player (PMP), a personal digital assistant (PDA), a smart phone, a mobile phone, a digital photo frame, a wearable device, and so on. Wearable devices are electronic devices that are worn on the user's body, and may take the form of glasses, watches, necklaces, clothing, sports equipment, medical machines, and so on. The wearable device may have a form attached to the skin or transplanted into the human body.

[0055] The associated equipment 130 may include a photovoltaic device (PV), an energy storage system (ESS), a door lock, an internet protocol (IP) camera, a camera on the door, and so on. The associated device 130 may include sensors that sense geomagnetism, atmospheric pressure, user motion, user proximity, user location, window, temperature, humidity, illuminance, gravity, and so on.

[0056] The smart appliance 140 may include a TV, a refrigerator, a washing machine, an air conditioner, an oven, a dishwasher, or a robot vacuum cleaner.

[0057] reference figure 1 The UE 120, the associated device 130, and the smart appliance 140 may be collectively referred to as "devices". In other words, in this specification and claims, a device can be defined as a device whose status can be monitored or whose actions can be controlled by a user through a gateway (GW), terminal, and/or server. Meanwhile, other devices not mentioned above may correspond to devices other than the devices listed above (such as the UE 120, the associated device 130, and the smart appliance 140).

[0058] The remote server 150 can control device actions via remote access. Although it is described in this specification that a gateway or a device controls device actions, the remote server 150 may also control device actions. The remote server 150 may include a display (touch screen) for displaying UI, a communication unit for communicating with external devices, a storage unit for storing data, an input unit for receiving user input, or a device for controlling the performance of other units. Controller.

[0059] figure 2 The configuration of a smart home system according to another embodiment of the present disclosure is shown.

[0060] reference figure 2 , The smart home platform 210 may include a rule database (DB) 211. The smart home platform 210 may include a context checker module 212 configured to execute appropriate rules by comparing the status information of the devices monitored by the smart home platform 210 with user context information defined in the rule DB 211. The smart home platform 210 can correspond to figure 1 The gateway 110. The smart home platform 210 can be connected to other devices 220 and 230, the manufacturer's server 240 or the social network 250. The manufacturer server 240 or the social network 250 may include figure 1 The remote server 150.

[0061] image 3 It is a flowchart showing a process of receiving conditions and actions and generating rules according to an embodiment of the present disclosure.

[0062] reference image 3 , In operation 310, when the rule generation process starts, the gateway of the smart home system determines whether the user selects the conditions provided by default. When the user does not select the condition provided by default, in operation 315, the gateway generates a new condition. The gateway can receive the information needed to generate the condition from the user. The process of receiving required information may include, for example, a process of receiving a selection of a person to be sensed in operation 320, a process of receiving a selection of a sensing device in operation 325, and a process of receiving a selection of a sensing item in operation 330. A process, and a process of receiving a condition name that triggers sensing or generation and/or adding an icon in operation 335. The process of receiving the selection of the person to be sensed may include a process of selecting the NFC tag corresponding to the person to be sensed. According to an embodiment of the present disclosure, when the user selects a condition provided by default, the condition input can be completed only by an input for selecting a person to be sensed. When the condition input is completed, in operation 340, it is determined whether there is an input for adding a new condition.

[0063] When there is no input for adding a new condition, the action corresponding to the condition selected or generated by the user is input by the user. The process of receiving an action corresponding to the condition may include a process of selecting a device that performs an action corresponding to the condition in operation 345 and a process of selecting an action of the selected device in operation 350. When it is determined in operation 355 that there is no input for adding a new action after the action input is completed, the condition and the action input by the user match to generate a rule, and the generated rule is stored in operation 360. In addition, in operation 355, when it is determined that there is an input, return to operation 345. The process of storing the generated rule may include a process of receiving the rule name and icon from the user and storing the received rule name and icon.

[0064] Figure 4 It is a flowchart showing the process of editing rules and executing actions based on the application of the rules according to an embodiment of the present disclosure.

[0065] reference Figure 4 In operation 410, in a state where the smart home service is activated, the gateway (GW) 110 may determine whether the device is added to the smart home system or linked with the smart home system. The gateway can record a rule list, in which the device and the rule are matched in the form of a mapping table. The mapping table may also include device type information. When the gateway determines to add a device in operation 410, in operation 420, the gateway may query whether the mapping table contains a device matching the added device or a rule including device type information. The mapping table can be stored in the gateway, another device or the server. In operation 430, when the device or device type that is mapped between the added or linked device and the device included in the specific rule in the list is determined through a response to the query, the gateway may display to the user information about the corresponding specific rule. recommend. When the user selects the displayed specific rule, the selected rule is registered as a new rule corresponding to the added device.

[0066] In operation 440, in a state where the rule is registered, the gateway (GW) may determine whether the user context included in the registered rule has been generated. The generation of the user context may be detected when one or more states shown in Table 2 or a state change in one or more states shown in Table 2 is detected. When it is determined that the user context has been generated, the gateway can query the information included in the rule list stored in the rule DB. For example, the context checker module 212 may compare the device status information with the condition information defined in the rule DB 211. The condition information may include a specific time or date defined by the user.

[0067] When it is determined that the user context has been generated, in operation 450, the gateway may control one or more devices in the smart home system to perform actions corresponding to the user context.

[0068] The user can query the rule list stored in the rule DB. In the state where the rule list is displayed, the user can activate or deactivate (turn on/off) each rule. Even if the user does not make an inactive input, rules that have problems with the device link can be deactivated during the query of the rule list. The display content of the activated or deactivated (on/off) rule during the query rule list can be distinguished. The distinguished display content may correspond to, for example, different colors. In the process of querying the rule list, users can add, edit or delete conditions (user context), actions (user operations), names or icons for each rule.

[0069] The user can generate a new rule by inputting conditions and actions via a tool included in the gateway or a device connected to the gateway. Unlike the above, the gateway can analyze the relationship between the user's use of the device and the context mode, and extract conditions and actions through the analysis results to automatically generate rules.

[0070] Figure 5 It is a flowchart showing a process of generating rules and applying the generated rules based on the user's use of the device and the context mode according to an embodiment of the present disclosure.

[0071] reference Figure 5 , The event can be defined as the user context detected by the device and the user operation on the device. The smart home system may include an event history repository 505 for storing events. The information about the user context and user operations stored in the event history repository 505 can be automatically discarded from the storage device after a predetermined time.

[0072] When the user's action on the device is detected in operation 510, in operation 515, the smart home system may store the information monitored by the smart home system in a predetermined period before or after the time point when the action is detected in the event history storage. In the library 505. The device action may correspond to, for example, a movement of the user opening the front door and entering from the outside. The predetermined period before or after the detected time point may correspond to, for example, 2 minutes from the detected movement. The information monitored by the smart home system is the information obtained from the sensors/devices included in the smart home system. The information monitored by the smart home system may include, for example, user operations to turn on lamps, user operations to turn on TV, and user contexts in the kitchen.

[0073] In operation 520, the smart home system may periodically retrieve sensor/device information (event information) stored in the event history repository 505 through user input in order to generate rules or generate rules whenever an event is detected, and in operation 525 , It can be determined whether there are repeated events in each period based on the detected event information. When there is a repetitive event, in operation 530, the smart home system may extract the user context and user operation included in the event, and store the extracted user context and user operation in the event history storage 505. Through storage, in operation 535, a rule generation process including a specific user operation triggered by a specific user context can be completed.

[0074] According to another embodiment of the present disclosure, when there is a condition that is repeated for a specific action a predetermined number of times in the process of storing repeated event information, the event history repository 505 may generate a rule including the repeated condition and the specific action and recommend the generated rule to the user. the rule of.

[0075] When the smart home system detects a sensor/device status change in operation 540, in operation 545, the smart home system may retrieve the sensor/device information (event information) stored in the event history repository 505 before detecting the sensor/device status change. ). In operation 550, the smart home system determines whether the user context included in the event information obtained through the search matches the detected state change, and when the user context matches the detected state, performs the matching with the user context in operation 555. The corresponding action.

[0076] Image 6 An action period according to an embodiment of the present disclosure is shown.

[0077] reference Image 6 , Shows the action period when k7 (condition) is detected and the event (user context and user operation) detected in the action period. The action period can have the same meaning as the recording period. by Image 6 , It can be noted that the action period corresponding to k7 (condition) starts before the detection time point of k7, and ends after the detection time point of k7. Alternatively, the action period may start or end at the detection time point of the condition. The action period is a preset time period, and may correspond to, for example, one of 30 seconds, 2.5 minutes, 2 hours, and 48 hours. The action period (recording period) can take into account the time that the user generally takes when performing the associated action before or after the user-specific action (for example, the action of opening the house entrance door from the outside) (for example, when the user moves to the farthest distance). The time spent in the room at the entrance of the house) is determined.

[0078] K2, k3, and k7 refer to device status changes (user context) and a, b, c, and d refer to user operations on the device. At the timeline 601, there is an event E1={a, b, c, d, m, n, u, w, k2, k3, k7} detected in the action period. At the time line 603, there is an event E2={a, b, c, d, e, m, r, t, y, k3, k7} detected in the action period. The expression of E1 and E2 does not need to consider the user context and the time point when the user operation is generated. Alternatively, the user context and user operation can be specified in consideration of the generated sequence.

[0079] Figure 7 It is a flowchart showing a process of extracting an event in an action period according to an embodiment of the present disclosure.

[0080] reference Figure 7 When the smart home system detects a change in the state of the device (for example, a sensor) in operation 710, the smart home system may store the information monitored by the smart home system in a predetermined period before and after the detection time point of the state change in the operation event In the historical repository 720. Based on user operation detection and stored in Figure 5 And the storage based on state change detection is in Figure 7 Shown in. The device state change may correspond to, for example, a change in the current time from dawn (05:00-08:00) to morning (08:00-12:00). The specific event corresponding to dawn or morning can be changed by the user. The information monitored by the smart home system may include, for example, the detection of the movement of the user in the bedroom, the detection of the movement of the user in the living room, and the user operation of turning on the lamps in the living room.

[0081] The smart home system may periodically retrieve the event information stored in the event history storage 720 to generate rules through user input, or retrieve the event information stored in the event history storage 720 to generate rules whenever an event is detected. In operation 730, the smart home system may determine whether there is an event based on the retrieved event information, the pattern of which is repeated in each action period. When there is no repeat mode at operation 730, the smart home system enters the standby state 705 to wait for the state change 710.

[0082] Based on the retrieved event information, it will be referenced to determine whether there is an event with a pattern that repeats in each action period Image 6 A and Image 6 B is described. Such as Image 6 A and Image 6 As shown in B, past events can be divided into action periods based on events.

[0083] In the process of extracting user operations in each action period, it is assumed that Image 6 The user operation in A is S1={a,b,c,d,m,n,u,w} and in Image 6 The user operation in B is S2={a,b,c,d,e,m,r,t,y}. In this case, the intersection S={a, b, c, d, m} between S1 and S2 can be determined as the extracted user operation. The extracted user operations can be referred to as public user operations.

[0084] In the process of extracting the user context (device status) in each action period, it is assumed that Image 6 The user context in A is K1={k2,k3,K7} and in Image 6 The user context in B is K2={k3,k2}. In this case, the intersection K={k2, k3} between K1 and K2 corresponds to the extracted user context. The extracted user content can be called a public user context or a common device state.

[0085] Image 6 A and Image 6 B only shows the comparison between two action periods, but does not limit the present disclosure, and common events (user operation and user context) can be extracted through the comparison of two or more action periods.

[0086] In operation 730, the smart home system determines whether there is a repeated pattern event. When the smart home system determines that there is a repetitive pattern event, in operation 740, the smart home system may extract the user context and user operation included in the event, and store the extracted user context and user operation in the event history storage 720.

[0087] In the above various embodiments, the conditions and actions are distinguished according to the user context detected by the device or the user operation of the device. However, the distinction between conditions and actions is not limited to this. For example, when a specific user context or a specific user operation is detected, another user context or user operation detected before the specific user context or a specific user operation is detected can be limited to the condition, and when the specific user context or a specific user operation is detected Another user context or user operation detected after the operation can be defined as an action.

[0088] Figure 8 The name and icon according to an embodiment of the present disclosure are shown.

[0089] reference Figure 8 , The user can query the rule list through the device or gateway of the smart home system. At this time, when the rule includes a name or icon, the name or icon may be displayed on the device and/or gateway. For example, a name 810 and an icon 820 are shown. The conditions and actions included in the rules can also have names and icons. Examples of the name 830 of the action, the icon 840 of the action, the name 850 of the condition, and the icon 860 of the condition are shown.

[0090] Picture 9 It is a block diagram of a gateway device according to an embodiment of the present disclosure.

[0091] reference Picture 9 The gateway device (gateway) may include a detection unit 910, an input unit 920, a storage unit 930, a communication unit 940, and a display unit 950, but is not limited thereto.

[0092] The detection unit 910 may include one or more sensors for detecting user context or user operations. For example, sensors may include geomagnetic sensors, temperature sensors, atmospheric pressure sensors, proximity sensors, illuminance sensors, global positioning systems (GPS), acceleration sensors, angular velocity sensors, speed sensors, gravity sensors, tilt sensors, gyroscope sensors, gas sensors , Camera, microphone, etc.

[0093] The input unit 920 may transmit a signal input by the user to the controller 960, the signal being related to gateway function control and various function configurations. In addition, the input unit 920 may be combined with the display unit 950 to implement a touch screen or may be formed by a normal keyboard. In addition, the input unit 920 may include function keys, soft keys, and so on.

[0094] The storage unit 930 may store monitoring information collected by the gateway or information input by the user. The monitoring information may include information related to the user context detected by the device (including the gateway) of the smart home system or information related to the user operation of the device. According to another embodiment of the present disclosure, the storage unit 930 may be included in a separate server.

[0095] The communication unit 940 may transmit/receive monitoring information or information required to control another device by communicating with another device or a server. The display unit 950 may display information about the rule list or recommended rules through the user interface.

[0096] The controller 960 controls the general status and actions of the components included in the gateway. The controller 960 may control the monitoring information received through the detection unit 910 or the communication unit 940 to be stored in the storage unit 930. The controller 960 controls the display unit 950 to display information and controls another unit to perform a function corresponding to a user input received through the input unit 920 in order to perform processes according to various embodiments of the present disclosure.

[0097] Although in Picture 9 , The detection unit 910, the input unit 920, the storage unit 930, the communication unit 940, the display unit 950, and the controller 960 are shown as separate blocks and these blocks perform different functions, but this is only for technical convenience, and these functions are not necessary. separate.

[0098] Picture 10 It is a flowchart showing a process of generating a rule and executing the generated rule according to an embodiment of the present disclosure.

[0099] reference Picture 10 In operation 1005, the gateway device for the smart home system may detect a user operation customized for the first device group that is allocated a recording period by the user among the devices included in the smart home system. The device group may include only one device. Customized user operations can include one or more actions shown in Table 3. The customized user operation may correspond to actions such as turning on the TV and the air conditioner. The recording period is a period during which monitoring information is recorded to automatically generate rules. In operation 1010, the gateway device may record a state change of the device included in the smart home system in the recording period.

[0100] In operation 1015, when the gateway device periodically detects user input, or whenever the gateway device detects an event to automatically generate a rule, the gateway device retrieves the monitoring information (status change) recorded so far. The periodic retrieval of monitoring information may be performed, for example, every 12 hours. Event detection may correspond to no motion being detected in the residence for 30 minutes or longer.

[0101] In operation 1020, when the state change has a repeating pattern for the retrieved monitoring information (state change), the gateway device determines the corresponding state change as a candidate state change and displays the candidate state change to allow the user to view the candidate state change. The candidate status change may include, for example, arriving at 10 AM on Saturday.

[0102] The public state change can be a state change that triggers a customized user operation after the rule is registered. In operation 1025, when there is a user input for the configuration candidate state change to be a public state change, the gateway device configuration candidate state change is regarded as the public state change. According to another embodiment of the present disclosure, even when there is no user input for a predetermined time, the gateway may configure the candidate state change as a public state change. Unlike the above, according to another embodiment of the present disclosure, when there is no user input for a predetermined time, the gateway stops the display of the candidate state change without configuring the common state change.

[0103] In operation 1030, the gateway device may map and store the configured public state changes and customized user operations. The rules are generated through the above process.

[0104] After the rule is generated, in operation 1035, the gateway device may determine whether a state change that matches the configured public state change is detected. In operation 1040, when the gateway device determines that a state change that matches the configured public state change is detected, the gateway device may make a request for a customized user operation to the first device group without user input.

[0105] by Picture 10 In the process, the gateway collects contextual information when the action periodically entered by the user is executed, and when the context is the same as the collected contextual information, it will execute the action performed by the user input without user input. For example, a gateway device that learns that the user turns on the TV and the air conditioner at 10 am on Saturday can control the TV and the air conditioner to be turned on automatically without user input.

[0106] Picture 11 It is a flowchart showing a process of generating a rule and executing the generated rule according to another embodiment of the present disclosure.

[0107] reference Picture 11 In operation 1105, the gateway device of the smart home system may detect the user operation or device status of the devices included in the smart home system. User operations can include one or more actions shown in Table 3. The state can include one or more states shown in Table 2. In operation 1110, the gateway device may record the detected user operation or status. The recording status may correspond to recording the status of one or more predetermined devices, rather than recording the status of all the devices included in the smart home system.

[0108] In operation 1115, when the gateway device periodically detects user input, or whenever the gateway device detects an event to automatically generate a rule, the gateway device retrieves the recorded user operations and status so far.

[0109] In operation 1120, the gateway device may extract a user operation for the first device group, and extract the state of the first device group from among the retrieved user operations and states. The first device group may refer to one or more devices that are allocated a recording period by the user among the devices included in the smart home system. The recording period is a period allocated for each first device group to group user operations and states. The recording period may start at or before the time point when the first device group detects the user operation or state, and end at or after the time point when the first device group detects the user operation or state.

[0110] In operation 1125, the gateway device may group each of the user operations and states for each recording period with respect to each of the extracted user operations or the extracted states. In operation 1130, when there is a grouped user operation or a grouped state having a repeating pattern, the gateway device may determine the corresponding user operation or state as a common user operation and a common state. The repetition pattern may be limited to a predetermined number of repetitions (for example, 5 times). The predetermined number of repetitions may be defined in different ways according to grouped user operations or grouped states. According to another embodiment of the present disclosure, similar to Picture 10 In operation 1020, in order to determine the user operation and state as a public user operation and public state, candidate user operations and candidate states are displayed to receive user selection input.

[0111] In operation 1135, the gateway device may map and store the determined public user operation and public state. The rules are generated through the above process.

[0112] After the rule is generated, in operation 1140, the gateway device may determine whether the current state of one or more predetermined devices among the devices included in the smart home system matches the public state. The one or more predetermined devices may be the same as the one or more devices that detect the user operation or state in operation 1105, or may be included in the one or more devices. When the gateway device determines that it has detected that the current state of one or more predetermined devices matches the stored public state, in operation 1145, the gateway device may make a request to the first device group to the public user without user input. Operation request.

[0113] by Picture 11 In the process, the gateway device collects context information when the user performs periodic repetitive actions, and when it detects the same context as the collected context information, performs repetitive actions without user input.

[0114] Picture 12 It is a flowchart showing a process in which a gateway controls another device according to an embodiment of the present disclosure.

[0115] reference Picture 12 In operation 1210, if the gateway device of the smart home system receives user operation information for the operation of the device from the user device, the gateway device records the device status information received from the device group in the recording period. The device group may include one or more devices. The user device (for example, a smart phone) can detect a request for device operation.

[0116] The user operation information may include one or more operations customized by the user. The customized user operation may correspond to, for example, an action of the user pressing a laundry mode button configuring the washing machine.

[0117] The device status information may include context information detected by one or more devices included in the device group or status information of one or more devices included in the device group. The device status information may include time information or proximity information of a specific person. The time information may include a specific date, day of the week, time of day, or a combination thereof. The proximity information of the specific person may include information related to the proximity notification of the NFC tag belonging to the specific person. The device state information may include state changes of one or more devices included in the device group. The change of the device state refers to the detection of the change of the device (for example, a thermostat) from one state (for example, high temperature) to another state (for example, normal temperature). That is, when the device state information includes a change in the device state, the device state information may include both the past state and the current state of the device.

[0118] The recording period can be determined based on device and user operation information. For example, when the device is a refrigerator and the user operation information is high temperature, the recording period may be determined to be 3 minutes. When the device is a refrigerator and the user operation information is to lower the temperature, the recording period can be determined to be 4 minutes. In another example, when the device is a TV and an air conditioner, the recording period may be determined to be 30 minutes, regardless of user operation information. When the device is a TV, the recording period can be determined to be 10 minutes, regardless of user operation information. The above examples are only illustrative and can be implemented in different forms.

[0119] The recording period starts at or before the time point when the user operation information is received from the user equipment, and ends at or after the time point when the user operation information is received from the user equipment. For example, when the user operation information is received at 10:00, the recording period may correspond to one of 09:50 to 10:00, 09:55 to 10:05, and 10:00 to 10:10.

[0120] In operation 1220, the gateway device may determine the device status information recorded a predetermined number of times (for example, 5 times) within a predetermined period of time (for example, 7 days) as the public device status information. The determination may be performed periodically or when a predetermined event is generated. The predetermined event may include re-detection of user input or reception of specific information. The predetermined period may correspond to a time that is three times or more longer than the recording period. For example, the display unit may display device status information recorded 7 times (a predetermined number of times) in 2 months (within a predetermined time period). The displayed device status information can be displayed in the form of a name or an icon. When the input unit detects a predetermined user input (for example, a selection input to the displayed device status information), the displayed device status information may be determined as the public device status information.

[0121] In operation 1230, the gateway device may match and store the public device status information and user operation information. When a request input for editing information is detected, the gateway device can display a user interface, and the stored public device status information and user operation information can be edited through the user interface. When a device selection input through the user interface is detected, the stored public device status information and user operation information corresponding to the selected device may be displayed in the form of a name or an icon. The user can select the information to be edited from the displayed public equipment status information and user operation information.

[0122] In operation 1240, when the device status information received from the device group matches the public device status information, the gateway device may generate a request to the device to perform an operation corresponding to the user operation information. Therefore, the device performs the requested operation.

[0123] Figure 13 It is a flowchart showing a process of a user equipment controlling another device according to an embodiment of the present disclosure.

[0124] reference Figure 13 In operation 1310, when the user equipment of the smart home system receives a request for device operation, the user equipment records the device status information received from the device group during the recording period. A device group can include one or more devices.

[0125] Operations may include user operations on the device. The operation information may include one or more operations customized by the user. The user-customized operation may correspond to, for example, an action of the user pressing a laundry mode button configuring the washing machine.

[0126] The device status information may include context information detected by one or more devices included in the device group or status information of one or more devices included in the device group. The device status information may include time information or proximity information of a specific person. The time information may include a specific date, day of the week, time of day, or a combination thereof. The proximity information of the specific person may include information related to the proximity notification of the NFC tag belonging to the specific person. The device state information may include state changes of one or more devices included in the device group. The change of the device state refers to the detection of the change of the device (for example, a thermostat) from one state (for example, high temperature) to another state (for example, normal temperature). That is, when the device state information includes a device state change, the device state information may include both the past state and the current state of the device.

[0127] The recording period can be determined based on the request and the device. For example, when the user device is a smart phone, the device is a refrigerator, and the request is to increase the temperature, the recording period may be determined to be 5 minutes. On the other hand, when the request is to lower the temperature, the recording period can be determined to be 6 minutes. In another example, when the user equipment is a TV and the equipment is an air conditioner, the recording period may be fixed at 30 minutes, regardless of the request. When the device is a TV, the recording period can be determined to be 10 minutes, regardless of the request. The above examples are only illustrative and can be implemented in different forms.

[0128] The recording period starts at or before the time point when the request is detected, and ends at or after the time point when the request is detected. For example, when a request is detected at 10:00, the recording period may correspond to one of 09:50 to 10:00, 09:55 to 10:05, and 10:00 to 10:10.

[0129] In operation 1320, the user equipment may display device status information recorded a predetermined number of times (for example, 5 times) within a predetermined period of time (for example, 7 days). The predetermined period may correspond to a time that is three times or more longer than the recording period. In operation 1330, when the input unit detects a predetermined user input (for example, a selection input to the displayed device status information), the displayed device status information may be determined as the public device status information. The displayed device status information can be displayed in the form of a name or an icon.

[0130] In operation 1340, the user equipment may match and store the public device status information and the request. When a request input for editing information is detected, the user equipment can display a user interface, and the stored public equipment status information and request can be edited through the user interface. When a device selection input through the user interface is detected, the stored public device status information and request corresponding to the selected device may be displayed in the form of a name or an icon. In operation 1350, the user may select information to be edited from the displayed public device status information and request.

[0131] In operation 1360, when the device status information received from the device group matches the public device status information, the user equipment may send a request to the device. Therefore, the device performs the requested operation.

[0132] Figure 14 It is a flowchart showing device actions of using a context according to another embodiment of the present disclosure.

[0133] reference Figure 14 For device actions that use context, the gateway may divide past actions into action periods (recording periods) based on time in operation 1410, extract actions in each action period in operation 1420, and perform each action in operation 1430 Extraction conditions.

[0134] Various aspects of the present disclosure can also be implemented as computer readable codes on a non-transitory computer readable recording medium. The non-transitory computer readable recording medium is any data storage device that can store data that can be thereafter read by a computer system. Non-transitory computer readable recording media include read only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device. The non-transitory computer-readable recording medium can also be distributed on networked computer systems, so that computer codes are stored and executed in a distributed manner. In addition, the functional programs, codes, and code segments used to complete the present disclosure can be easily understood by programmers in the field to which the present disclosure belongs.

[0135] In this regard, it should be noted that the various embodiments of the present disclosure as described above generally involve input data processing and output data generation to a certain extent. Such input data processing and output data generation can be implemented in hardware or a combination of software and hardware. For example, specific electronic components may be used in mobile devices or similar or related circuit systems to implement the functions associated with the various embodiments of the present disclosure as described above. Alternatively, one or more processors operating in accordance with stored instructions may implement the functions associated with the various embodiments of the present disclosure as described above. If this happens, it is within the scope of the present disclosure that such instructions may be stored on one or more non-transitory processor-readable media. Examples of processor-readable media include read only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device. The processor-readable medium can also be distributed on networked computer systems so that instructions are stored and executed in a distributed manner. In addition, the functional computer programs, instructions, and instruction segments used to complete the present disclosure can be easily understood by programmers in the field to which the present disclosure belongs.

[0136] Although the present disclosure has been shown and described with reference to the various embodiments of the present disclosure, those skilled in the art will understand that without departing from the spirit and scope of the present invention as defined by the appended claims and their equivalents Next, various changes can be made to the form and details.