Electronic device and operating method thereof

Abstract

An electronic device according to one embodiment of the present disclosure may comprise a communication interface, and one or more processors, which: acquire location data of a user; acquire an activity space and a main occupied space of the user on the basis of the acquired location data; generate, on the basis of the acquired activity space and main occupied space, a control signal for controlling an operation of a home appliance; and transmit the generated control signal to the home appliance through the communication interface.

Description

Electronic device and method of operation thereof

[0001] The present disclosure relates to an electronic device, and more particularly, to an electronic device that estimates a user's activity space based on the user's location data.

[0002] Conventional techniques for obtaining user behavioral or location information rely on the use of sensors installed in space.

[0003] Passive infrared intrusion detection sensors or ultrasonic sensors, which are inexpensive sensors installed in large spaces, can be used to collect data by recording with cameras, or tomographic imaging can be performed using wireless transmitters and receivers. This method can only identify a user's location within the immediate vicinity of the sensor installation.

[0004] Control of home appliances can be performed based on the user's location.

[0005] However, according to the prior art, the home appliance can be controlled to detect a moving user and aim towards the area where the user is present, but it does not take into account the space blocked by walls and the user's activity radius within the space.

[0006] That is, in the case of conventional technology, control of home appliances is limited to the user's location, so there is a limit to the efficient control of home appliances.

[0007] The purpose of the present disclosure may be to estimate the shape of a space and the user's activity radius using user location data acquired through a sensor.

[0008] The purpose of the present disclosure may be to optimally control home appliances based on the shape of the space and the user's activity radius.

[0009] An object of the present disclosure may be to identify the relative locations of home appliances and the places where users mainly stay by using a heat map that accumulates user location information.

[0010] An object of the present disclosure may be to provide users with convenient information based on their location and interactions with home appliances.

[0011] An electronic device according to one embodiment of the present disclosure may include a communication interface and one or more processors that acquire location data of a user, acquire an activity space and a major occupied space of the user based on the acquired location data, generate a control signal for controlling an operation of a home appliance based on the acquired activity space and major occupied space, and transmit the generated control signal to the home appliance through the communication interface.

[0012] A method of operating an electronic device according to one embodiment of the present disclosure may include a step of acquiring location data of a user, a step of acquiring an activity space and a major occupied space of the user based on the acquired location data, a step of generating a control signal for controlling an operation of a home appliance based on the acquired activity space and major occupied space, and a step of transmitting the generated control signal to the home appliance.

[0013] According to an embodiment of the present disclosure, the energy efficiency of home appliances can be improved through optimal control of home appliances according to the shape of a user's living space.

[0014] According to an embodiment of the present disclosure, the user's convenience can be greatly improved by checking the user's activity radius and performing the operation of the home appliance in advance in the space that the user mainly uses.

[0015] According to an embodiment of the present disclosure, the relative positions of a place where a user mainly stays and a home appliance are identified, so that the control of the home appliance can be controlled in a more user-friendly manner.

[0016] According to an embodiment of the present disclosure, the location between a home appliance and a user can be determined through the user's location, so that information that facilitates interaction with a nearby home appliance can be provided to the user.

[0017] FIG. 1 is a block diagram illustrating components of an artificial intelligence device according to one embodiment of the present disclosure.

[0018] FIG. 2 is a diagram for explaining the configuration of an artificial intelligence server according to one embodiment of the present disclosure.

[0019] FIG. 3 is a flowchart illustrating an operation method of an artificial intelligence device according to an embodiment of the present disclosure.

[0020] FIG. 4 is a flowchart illustrating a process of acquiring a user's activity space based on a cumulative location data set according to one embodiment of the present disclosure.

[0021] FIG. 5 is a diagram showing a heat map based on a cumulative location data set according to one embodiment of the present disclosure.

[0022] FIG. 6 is a diagram showing a clustering map generated based on a heat map according to an embodiment of the present disclosure.

[0023] FIG. 7A and FIG. 7B are diagrams illustrating a process of obtaining the shape of a user's activity space from a clustering map according to one embodiment of the present disclosure.

[0024] Figure 8 is a drawing for explaining activity space information.

[0025] FIGS. 9A and 9B are diagrams illustrating examples of identifying a major occupied space within a user's activity space according to one embodiment of the present disclosure.

[0026] FIG. 10 is a diagram illustrating the configuration of a spatial understanding system according to one embodiment of the present disclosure.

[0027] FIG. 11 is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0028] Figures 12a and 12b are diagrams showing the linkage of a user's location data and an event of a home appliance.

[0029] FIG. 13 is a drawing illustrating an example of identifying the location of an estimated home appliance according to one embodiment of the present disclosure.

[0030] FIG. 14 is a diagram illustrating a position estimation system according to one embodiment of the present disclosure.

[0031] FIG. 15 is a flowchart illustrating an operation method of an artificial intelligence device according to another embodiment of the present disclosure.

[0032] FIG. 16 is a drawing illustrating a screen that provides a user's activity space, a main occupied space, and the location of home appliances according to an embodiment of the present disclosure.

[0033] FIG. 17 is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0034] FIGS. 18 and 19 are drawings illustrating examples of differently controlling the cleaning path of a robot cleaner based on the main occupied space of each of a first user and a second user according to an embodiment of the present disclosure.

[0035] FIGS. 20A and 20B are drawings illustrating an example of extracting a major occupied space for each time interval and controlling cooling of the extracted major occupied space according to an embodiment of the present disclosure.

[0036] FIG. 21 is a diagram illustrating the configuration of an artificial intelligence cloud device according to another embodiment of the present disclosure.

[0037] FIG. 22 is a sequence diagram illustrating an operation method of a system according to an embodiment of the present disclosure.

[0038] Artificial intelligence refers to a field that studies artificial intelligence or the methodologies for creating it, and machine learning refers to a field that defines various problems in the field of artificial intelligence and studies the methodologies for solving them.

[0039] Machine learning is sometimes defined as an algorithm that improves its performance on a task through continuous experience.

[0040] An artificial neural network (ANN) is a model used in machine learning. It can refer to a model with problem-solving capabilities that is composed of artificial neurons (nodes) that form a network through the combination of synapses.

[0041] An artificial neural network can be defined by the connection patterns between neurons in different layers, the learning process that updates model parameters, and the activation function that generates the output values.

[0042] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer contains one or more neurons, and the artificial neural network may include synapses connecting neurons. In an artificial neural network, each neuron can output a function value of an activation function based on input signals, weights, and biases received through the synapses.

[0043] Model parameters are parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters are parameters that must be set before learning in machine learning algorithms, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0044] The goal of artificial neural network training can be seen as determining model parameters that minimize a loss function. The loss function can be used as an indicator for determining optimal model parameters during the artificial neural network training process.

[0045] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0046] Supervised learning refers to a method of training an artificial neural network given labels for training data. The labels can refer to the correct answer (or result value) that the artificial neural network must infer when training data is input to the artificial neural network.

[0047] Unsupervised learning can refer to a method of training an artificial neural network without being given labels for the training data.

[0048] Reinforcement learning can refer to a learning method that teaches an agent defined in an environment to select an action or action sequence that maximizes the cumulative reward in each state.

[0049] Among artificial neural networks, machine learning implemented with a deep neural network (DNN) that includes multiple hidden layers is also called deep learning, and deep learning is a part of machine learning.

[0050] Hereinafter, machine learning is used to mean deep learning.

[0051] FIG. 1 is a block diagram illustrating components of an artificial intelligence device according to one embodiment of the present disclosure.

[0052] The artificial intelligence device (100) can be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, digital signage, a robot, a vehicle, etc.

[0053] Referring to FIG. 1, an artificial intelligence device (100) may include a communication interface (110), an input interface (120), a learning processor (130), a sensor (140), an output interface (150), a memory (170), and a processor (180).

[0054] The communication interface (110) can transmit and receive data with external devices such as other artificial intelligence devices or AI servers (200) using wired or wireless communication technology. For example, the communication interface (110) can transmit and receive sensor information, user input, learning models, control signals, etc. with external devices.

[0055] The communication technologies used by the communication interface (110) include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth (Bluetooth), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0056] The input interface (120) can obtain various types of data.

[0057] The input interface (120) may include a camera (121) for capturing images, a microphone (122) for receiving audio signals, and a user input interface (123) for receiving information from a user.

[0058] By treating the camera (121) or microphone (122) as a sensor, the signal obtained from the camera (121) or microphone (122) can be called sensing data or sensor information.

[0059] The input interface (120) can acquire input data to be used when obtaining output using learning data and a learning model for model learning. The input interface (120) can also acquire raw input data, in which case the processor (180) or learning processor (130) can extract input features as preprocessing for the input data.

[0060] The camera (121) processes image frames, such as still images or moving images, obtained by the image sensor in video call mode or shooting mode. The processed image frames can be displayed on the display (151) or stored in the memory (170).

[0061] The microphone (122) processes external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function (or application program) being performed by the artificial intelligence device (100). Meanwhile, various noise removal algorithms can be applied to the microphone (122) to remove noise generated during the process of receiving external acoustic signals.

[0062] The user input interface (123) is for receiving information from a user. When information is input through the user input interface (123), the processor (180) can control the operation of the artificial intelligence device (100) to correspond to the input information.

[0063] The user input interface (123) may include a mechanical input means (or a mechanical key, for example, a button located on the front / rear or side of the artificial intelligence device (100), a dome switch, a jog wheel, a jog switch, etc.) and a touch input means.

[0064] As an example, the touch input means may be composed of virtual keys, soft keys, or visual keys displayed on a touch screen through software processing, or may be composed of touch keys placed on a part other than the touch screen.

[0065] The learning processor (130) can train a model composed of an artificial neural network using learning data. The trained artificial neural network can be referred to as a learning model. The learning model can be used to infer result values ​​for new input data other than the learning data, and the inferred values ​​can be used as a basis for judgment to perform a certain action.

[0066] The running processor (130) can perform AI processing together with the running processor (240) of the AI ​​server (200).

[0067] The running processor (130) may include a memory integrated or implemented in the artificial intelligence device (100). The running processor (130) may also be implemented using a memory (170), an external memory directly coupled to the artificial intelligence device (100), or a memory maintained in an external device.

[0068] The sensor (140) can obtain at least one of internal information of the artificial intelligence device (100), information about the surrounding environment of the artificial intelligence device (100), and user information by using various sensors.

[0069] The sensor (140) may include one or more of a proximity sensor, a light sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a lidar sensor, and a radar sensor.

[0070] The output interface (150) can generate output related to visual, auditory, or tactile sensations.

[0071] The output interface (150) may include a display (151) that outputs images, an audio output interface (152) that outputs audio, a haptic device (153) that outputs tactile information, and a light output interface (154) that outputs light.

[0072] The display (151) displays (outputs) information processed in the artificial intelligence device (100). For example, the display (151) may display execution screen information of an application program running in the artificial intelligence device (100), or UI (User Interface) or GUI (Graphical User Interface) information according to such execution screen information.

[0073] The display (151) can be implemented as a touch screen by forming a mutual layer structure with the touch sensor or forming an integral structure. The touch screen can function as a user input interface (123) that provides an input interface between the artificial intelligence device (100) and the user, and at the same time, provide an output interface between the artificial intelligence device (100) and the user.

[0074] The audio output interface (152) can output audio data received from the communication interface (110) or stored in the memory (170) in a call signal reception mode, call mode, recording mode, voice recognition mode, broadcast reception mode, etc.

[0075] The audio output interface (152) may include at least one of a receiver, a speaker, and a buzzer.

[0076] The haptic device (153) generates various tactile effects that can be felt by the user. A representative example of the tactile effect generated by the haptic device (153) may be vibration.

[0077] The light output interface (154) outputs a signal to notify the occurrence of an event using light from a light source of the artificial intelligence device (100). Examples of events occurring in the artificial intelligence device (100) may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, and information reception through an application.

[0078] The memory (170) can store data that supports various functions of the artificial intelligence device (100). For example, the memory (170) can store input data, learning data, learning models, learning history, etc. obtained from the input interface (120).

[0079] The processor (180) can determine at least one executable operation of the artificial intelligence device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm.

[0080] The processor (180) can control components of the artificial intelligence device (100) to perform determined operations.

[0081] To this end, the processor (180) can request, retrieve, receive or utilize data from the running processor (130) or memory (170), and control components of the artificial intelligence device (100) to execute at least one of the executable operations, a predicted operation or an operation determined to be desirable.

[0082] When the processor (180) requires connection to an external device to perform a determined operation, it can generate a control signal for controlling the external device and transmit the generated control signal to the external device.

[0083] The processor (180) can obtain intent information for user input and determine the user's requirements based on the obtained intent information.

[0084] The processor (180) can obtain intent information corresponding to the user input by using at least one of a STT (Speech To Text) engine for converting voice input into a string or a natural language processing (NLP) engine for obtaining intent information of natural language.

[0085] At least one of the STT engine or the NLP engine may be configured with an artificial neural network, at least in part, trained according to a machine learning algorithm. Furthermore, at least one of the STT engine or the NLP engine may be trained by the learning processor (130), the learning processor (240) of the AI ​​server (200), or through distributed processing thereof.

[0086] The processor (180) can collect history information including the operation details of the artificial intelligence device (100) or the user's feedback on the operation, and store the information in the memory (170) or the learning processor (130), or transmit the information to an external device such as an AI server (200). The collected history information can be used to update the learning model.

[0087] The processor (180) can control at least some of the components of the artificial intelligence device (100) to run an application program stored in the memory (170).

[0088] The processor (180) can operate two or more of the components included in the artificial intelligence device (100) in combination to drive the application program.

[0089] FIG. 2 is a diagram for explaining the configuration of an artificial intelligence server according to one embodiment of the present disclosure.

[0090] Referring to FIG. 2, the AI ​​server (200) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0091] The AI ​​server (200) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network. The AI ​​server (200) may be included as part of the artificial intelligence device (100) and may perform at least a portion of the AI ​​processing.

[0092] The AI ​​server (200) may include a communication interface (210), memory (230), a learning processor (240), and a processor (260).

[0093] The communication interface (210) can transmit and receive data with an external device such as an artificial intelligence device (100).

[0094] The memory (230) may include a model memory (231). The model memory (231) may store a model (or artificial neural network, 231a) being learned or learned through the learning processor (240).

[0095] A learning processor (240) can train an artificial neural network (231a) using learning data. The learning model can be used while mounted on the AI ​​server (200) of the artificial neural network, or can be mounted on an external device such as an artificial intelligence device (100).

[0096] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (230).

[0097] The processor (260) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0098] Hereinafter, the artificial intelligence device (100) or AI server (200) may be referred to as an electronic device.

[0099] FIG. 3 is a flowchart illustrating an operation method of an artificial intelligence device according to an embodiment of the present disclosure.

[0100] Hereinafter, one or more processors may be provided.

[0101] Referring to FIG. 3, the processor (180) of the artificial intelligence device (100) can obtain the user's location data (301).

[0102] In one embodiment, the processor (180) may obtain the user's location data through either a sensor (140) provided in the artificial intelligence device (100) or a sensor provided separately from the artificial intelligence device (100).

[0103] The sensor used to obtain the user's location data may be a millimeter wave (mmWave) sensor. A mmWave sensor may be a sensor that detects objects using electromagnetic waves with very short wavelengths. A mmWave sensor may be placed in a fixed location.

[0104] A millimeter wave sensor may include a transmitting antenna and a receiving antenna.

[0105] The transmitting antenna of the millimeter wave sensor can transmit electromagnetic waves operating in the frequency range of 30 GHz to 300 GHz. The receiving antenna of the millimeter wave sensor can receive electromagnetic waves reflected when the transmitted electromagnetic waves strike an object (e.g., a user).

[0106] Millimeter wave sensors can measure the distance to an object based on the time it takes for the transmitted electromagnetic waves to reflect off the object and return. A detection area can be defined where the millimeter wave sensor can detect an object installed in a fixed location. The processor (180) can identify the user's location within the detection area using coordinates.

[0107] The processor (180) can convert the distance between the millimeter wave sensor and the user, received from the millimeter wave sensor, into coordinate information, and obtain the converted coordinate information as the user's location data. The processor (180) can obtain the user's location data in real time.

[0108] The processor (180) can detect movement of an object by recognizing a change in distance between the millimeter wave sensor and the object.

[0109] The processor (180) can remove location data that moves within a short period of time from the acquired location data. This is to remove noise or data regarding objects other than people. The short period of time may be 0.1 seconds, but this is merely an example.

[0110] The processor (180) can identify a user based on sensing information received from the millimeter wave sensor. The sensing information may include one or more of the following: the distance between the millimeter wave sensor and the object, and the shape or size of the object based on the phase change between the transmitted and reflected electromagnetic waves.

[0111] The processor (180) may also identify a user based on the shape or size of an object. That is, the processor (180) may identify each of a plurality of users having different shapes or sizes of objects.

[0112] The processor (180) can obtain an accumulated location data set based on the obtained user location data (S303).

[0113] A cumulative location data set may be a data set that accumulates the number of times a user is detected at each location based on location data. The user's location may be expressed as a spatial coordinate. Each cumulative location data included in the cumulative location data set may include spatial coordinates and a frequency.

[0114] The cumulative position data set is a data set that considers the maximum detection width, maximum detection length, and cumulative frequency of each location of the millimeter wave sensor, and can be stored in the memory (140). Accordingly, the capacity of the cumulative data set is fixed at a maximum size, and thus has the advantage of not taking up a large amount of memory (140).

[0115] The processor (180) can obtain an accumulated location data set using location data accumulated over a certain period of time. The processor (180) can obtain an accumulated location data set by updating the location data acquired over a certain period of time.

[0116] The processor (180) can obtain the user's activity space based on the accumulated location data set (S305).

[0117] In one embodiment, the processor (180) can estimate the user's activity space within a detection area based on a set of accumulated location data. The detection area may be an area where objects can be detected using a millimeter wave sensor. The detection area may be formed based on the angle of the electromagnetic waves transmitted by the millimeter wave sensor and the transmission distance of the electromagnetic waves.

[0118] The processor (180) can cluster the accumulated location data set to generate clustered data. The processor (180) can estimate the user's activity space from the clustered data using a polygon algorithm.

[0119] The processor (180) can obtain activity space information corresponding to the estimated user's activity space. The process of obtaining the user's activity space based on the accumulated location data set is described in detail.

[0120] FIG. 4 is a flowchart illustrating a process of acquiring a user's activity space based on a cumulative location data set according to one embodiment of the present disclosure.

[0121] FIG. 4 may be a drawing that embodies step S305 of FIG. 3.

[0122] The processor (180) of the artificial intelligence device (100) can cluster the accumulated location data set to create a clustering map (S401).

[0123] In one embodiment, the processor (180) may generate clustering data using a density-based clustering technique. The clustering data may be referred to as a clustering map.

[0124] The processor (180) can generate a heat map representing the distribution of the accumulated location data using the accumulated location data set, and can generate clustering data based on the generated heat map.

[0125] The processor (180) can extract a plurality of cluster areas from a cumulative location data set using a density-based clustering technique, and can generate clustering data using the extracted plurality of cluster areas.

[0126] A density-based clustering technique could be the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique.

[0127] In the DBSCAN technique, the minimum number of data points required to form a clustered area can be set. The processor (180) can then calculate the number of other data points within a radius of a data point in the accumulated location data set. The processor (180) can then identify high-density areas with a high concentration of other data points as clustered areas, and consider low-density areas with a low concentration of other data points as noise.

[0128] The processor (180) can identify high-density areas to obtain clustering data (or a final clustering map).

[0129] This is explained in Figs. 5 and 6.

[0130] FIG. 5 is a diagram showing a heat map based on a cumulative location data set according to an embodiment of the present disclosure, and FIG. 6 is a diagram showing a clustering map generated based on the heat map according to an embodiment of the present disclosure.

[0131] In Fig. 5, the arrangement of actual furniture and walls is projected onto a heat map (500) for reference.

[0132] The processor (180) can accumulate user location data acquired through a millimeter wave sensor to generate an accumulated location data set. Using the accumulated location data set, the processor (180) can generate a heat map (500) as illustrated in FIG. 5.

[0133] The horizontal axis of the heat map (500) may represent the detection width of the millimeter wave sensor, and the vertical axis may represent the detection length of the millimeter wave sensor. The detection width may have a positive value to the right of the location of the millimeter wave sensor, and may have a negative value to the left of the location of the millimeter wave sensor.

[0134] Each point in the heat map (500) may represent a cumulative number (or cumulative frequency) of location data.

[0135] The cumulative location data set (500) can be expressed in the form of a heat map. A heat map can be a graphical tool that visually represents the cumulative distribution of a user's location data. A heat map can express the density or frequency of location data using colors on a two-dimensional grid. A darker color can indicate a higher frequency, while a lighter color can indicate a lower frequency.

[0136] Referring to Figure 5, the frequency of location data can be expressed as 0 to 140, and a higher frequency can be expressed in red, and a lower frequency can be expressed in blue.

[0137] In one embodiment, the processor (180) may generate a first type of heat map based on a set of accumulated location data acquired over a preset period of time. The preset period may be any one of a lunch hour, dinner hour, or a specific time period.

[0138] In another embodiment, the processor (180) may identify a user and generate a second type of heat map based on the cumulative location data set of the identified user. That is, the processor (180) may generate a heat map corresponding to each of a plurality of users. The heat map corresponding to each user may be used to perform personalized control of home appliances.

[0139] In another embodiment, the processor (180) may generate a third type of heat map using a cumulative location data set of users identified over a preset period of time.

[0140] The processor (180) can obtain a clustering map (600) as illustrated in FIG. 6 based on the heat map (500). The clustering map (600) may be referred to as a clustering graph or a clustering space.

[0141] The processor (180) can obtain a clustering map (600) using the DBSCAN technique.

[0142] The horizontal axis of the clustering map (600) may represent the detection width of the millimeter wave sensor, and the vertical axis may represent the detection length of the millimeter wave sensor. The detection width may have a positive value to the right of the location of the millimeter wave sensor, and may have a negative value to the left of the location of the millimeter wave sensor.

[0143] The processor (180) can identify high-density areas and low-density areas based on each data point included in the clustering map (600) using the DBSCAN technique.

[0144] The processor (180) can identify a clustering area (610) including high-density areas from the clustering map (600).

[0145] Again, Figure 4 is explained.

[0146] The processor (180) can obtain the shape of the user's activity space from a clustering map generated using a polygon approximation algorithm (S403).

[0147] In one embodiment, the polygon approximation algorithm may be either the Ramer-Douglas-Peucker algorithm or the convex hull algorithm.

[0148] The processor (180) can obtain the shape of the user's activity space matching the clustering area (610) using a polygon approximation algorithm.

[0149] The Ramer-Douglas-Peucker algorithm may be an algorithm that creates a polygonal area by simplifying the outline of the clustering area (610).

[0150] The Ramer-Douglas-Peucker algorithm can simplify the clustering area (610) in the following way.

[0151] 1. Draw a straight line connecting the start and end points.

[0152] 2. Find the point furthest from the line. If the distance between the line and this point is greater than a threshold, include the point and create two new line segments.

[0153] 3. This process is repeated recursively to simplify all straight line segments so that they have a distance less than the threshold.

[0154] The convex hull algorithm is an algorithm that, when dividing multiple data points into two groups, obtains the convex hull of each group separately and combines the convex hulls of the groups to obtain the entire convex hull as a polygonal region.

[0155] The processor (180) can obtain activity space information based on the shape of the acquired user's activity space (S405).

[0156] In one embodiment, the activity space information may include one or more of the coordinates of vertices of a polygon representing the activity space, the shape of the activity space, the area of ​​the activity space, the length of the activity space, or an effective angle relative to a millimeter wave sensor.

[0157] The processor (180) can obtain activity space information including at least one of a millimeter wave sensor that collects the area of ​​the activity space or the user's location data based on the shape of the user's activity space and an angle formed with the activity space.

[0158] The processor (180) can control the operation of the home appliance based on activity space information. This will be described later.

[0159] FIG. 7a and FIG. 7b are diagrams illustrating a process of obtaining the shape of a user's activity space from a clustering map according to one embodiment of the present disclosure, and FIG. 8 is a diagram for explaining activity space information.

[0160] Referring to FIGS. 7a and 7b, a user's activity space (710, 730) obtained from the clustering map (600) of FIG. 6 using a polygonal approximation algorithm is illustrated.

[0161] In Fig. 7a, the arrangement of actual furniture and walls is projected onto the activity space (710) for reference.

[0162] The processor (180) can generate a user's activity space (710, 730) from a clustering area (610) of a clustering map (600) using the polygon algorithm described above.

[0163] The processor (180) can obtain activity space information from the activity space (710, 730). The activity space information can include one or more of the coordinates of the vertices of the polygon represented by the activity space (710, 730), the area of ​​the activity space (710, 730), or one or more effective angles of the activity space (710, 730) based on a millimeter wave sensor.

[0164] FIG. 8 is a diagram illustrating activity space information obtained with reference to the clustering area (710) of FIG. 7a. The activity space information may include one or more of the effective angles (58 degrees, 27 degrees, 95 degrees) of the activity space (710) measured based on the position (P) of the millimeter wave sensor or the area of ​​the activity space (710).

[0165] The first effective angle (58 degrees) of the activity space (710) may be the angle formed by one side (711) of the wall surface where the position (P) of the millimeter wave sensor is placed and the first side (713) of the activity space (710).

[0166] The second effective angle (27 degrees) of the active space (710) may be the angle formed by one side (711) of the wall surface where the position (P) of the millimeter wave sensor is placed and the second side (715) of the active space (710). The first side (713) and the second side (715) are adjacent, and the extension line of the second side (715) may intersect the extension line of the first side (713).

[0167] The third effective angle (95 degrees) of the activity space (710) may be the angle formed by the extension of the first side (713) and the extension of the second side (715) based on the position (P) of the millimeter wave sensor.

[0168] The processor (180) can calculate the area of ​​the activity space (710) using the coordinates of the vertices of the polygon representing the activity space (710). The processor (180) can calculate the area of ​​the activity space (710) using the known Shoelace formula.

[0169] FIGS. 9A and 9B are diagrams illustrating examples of identifying a major occupied space within a user's activity space according to one embodiment of the present disclosure.

[0170] Figures 9a and 9b illustrate the user's activity space and major occupied space identified within the detection area detectable by the millimeter wave sensor.

[0171] A primary occupied space may be a space representing an area where the accumulated frequency of the user's location data is greater than a preset frequency.

[0172] Referring to FIG. 9A, major occupied spaces (901, 903) and the locations (P) of the millimeter wave sensor can be identified in the activity space (710) of the millimeter wave sensor user. Each of the major occupied spaces (901, 903) may be a space where the frequency of accumulated location data is greater than a preset frequency.

[0173] Referring to FIG. 9b, major occupied spaces (911, 913) and the positions (P) of millimeter wave sensors can be identified in the user's activity space (910). Each of the major occupied spaces (911, 913) may be a space where the frequency of accumulated location data is greater than a preset frequency.

[0174] The user's activity space (910) of Fig. 9b may be a space obtained based on location data collected from 3 PM to 9 PM.

[0175] The artificial intelligence device (100) can transmit information about the user's activity space (710, 910) and major occupied spaces (901, 903, 911, 913) to the user device through the communication interface (110).

[0176] Information about the user's activity space (710, 910) may include activity space information. Information about the main occupied space (901, 903, 911, 913) may include one or more of the location information, area, or shape of the main occupied space (901, 903, 911, 913).

[0177] The user device may be any device such as a smartphone, a smart pad, a PC, or a laptop. The user device may include all of the components of the artificial intelligence device (100) of FIG. 1. The user device may also be an artificial intelligence device (100).

[0178] The user device may be equipped with a home appliance management application that provides information about the user's activity area based on the user's location data. Through the installed home appliance management application, the user device may display an activity radius screen (900-1, 900-2) including the user's activity area (710, 910) and major occupied areas (901, 903, 911, 913), as illustrated in FIG. 9A or FIG. 9B .

[0179] In another embodiment, the processor (180) of the artificial intelligence device (100) may display an activity radius screen (900-1, 900-2) including the user's activity space (710, 910) and main occupied space (901, 903, 911, 913) on the display (151).

[0180] According to embodiments of the present disclosure, a user's active space and main occupied space can be obtained using a millimeter wave sensor without the need for a photographing device such as a camera. The user's active space and main occupied space can then be used to efficiently control home appliances.

[0181] Additionally, the user's activity space and main occupied space can be estimated without using a camera, so the user's privacy can be protected.

[0182] The processor (180) can control the operation of the home appliance based on the main occupied space (901, 903, 911, 913). This will be described later.

[0183] Meanwhile, in FIGS. 9a and 9b, furniture or electronic devices such as TVs can be identified and displayed within the user's activity space (710, 910).

[0184] Meanwhile, referring to FIG. 9b, the activity radius screen (900-2) may further include a progress bar (920). The progress bar (920) may be a bar for providing the user's activity space and main occupied space within a specific time interval. The progress bar (920) may include multiple time interval items corresponding to multiple time intervals.

[0185] The processor (180) of the artificial intelligence device (100) can identify the user's activity space (910) and main occupied space (911, 913) on the activity radius screen (900-2) when a time interval item (921) on the progress bar (920) is selected.

[0186] FIG. 10 is a diagram illustrating the configuration of a spatial understanding system according to one embodiment of the present disclosure.

[0187] Referring to FIG. 10, the spatial understanding system (1000) may include a millimeter wave sensor (1001), a cloud server (1030), and an AI server (200). The spatial understanding system (1000) may include an AI device (100) instead of the AI ​​server (200).

[0188] A millimeter wave sensor (1001) can transmit electromagnetic waves within a detection area (1010) and detect electromagnetic waves reflected from a user to obtain the user's location (1011).

[0189] The millimeter wave sensor (1001) can transmit location data corresponding to the acquired user's location (1011) to a cloud server (1030). The location data can be expressed as coordinates such as (x, y).

[0190] The cloud server (1030) can transmit the user's location data to the AI ​​server (200) or AI device (100).

[0191] The cloud server (1030) may be a server for managing one or more home appliances within the detection area (1010). The cloud server (1030) may be included in the AI ​​server (200) or the AI ​​device (100).

[0192] The AI ​​device (100) or AI server (200) can obtain an accumulated location data set based on the received user location data. The AI ​​device (100) or AI server (200) can store the user's location data.

[0193] An AI device (100) or an AI server (200) can acquire a user's activity space based on a cumulative location data set using a spatial understanding engine. The spatial understanding engine may be an engine that estimates the user's activity space based on a clustering or polygon approximation algorithm of the cumulative location data set. The spatial understanding engine may be included in the processor (180) of the AI ​​device (100) or the processor (260) of the AI ​​server (200).

[0194] An AI device (100) or an AI server (200) can cluster a cumulative location data set to create a clustering map.

[0195] The AI ​​device (100) or AI server (200) can obtain the shape of the user's activity space from a clustering map generated using a polygonal approximation algorithm.

[0196] The AI ​​device (100) or AI server (200) can obtain activity space information based on the shape of the acquired user's activity space.

[0197] The AI ​​device (100) or AI server (200) can store the acquired activity space information.

[0198] The AI ​​device (100) or AI server (200) can transmit the acquired activity space information to the cloud server (1030). The acquired activity space information can be used for efficient control of home appliances within the detection area (1010).

[0199] FIG. 11 is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0200] The processor (180) of the artificial intelligence device (100) can obtain the user's location data and events of the home appliance (S1101).

[0201] The processor (180) can receive the user's location data from the millimeter wave sensor.

[0202] The processor (180) may receive an event of a home appliance from the home appliance or the cloud server (1030). The event of the home appliance may indicate a change in the operational status of the home appliance. For example, the event of the home appliance may be any one of an open event indicating that the door of the home appliance is opened, a close event indicating that the door of the home appliance is closed, an on event indicating that the home appliance is turned on, or an off event indicating that the home appliance is turned off.

[0203] The time when an event occurs on a home appliance may be the time when the event is acquired on the home appliance.

[0204] The processor (180) can obtain the occurrence of an event of a home appliance and the time of occurrence of the event.

[0205] The processor (180) can determine whether there is an intention to use the home appliance based on location data prior to the occurrence of an event of the home appliance (S1103).

[0206] When an event of a home appliance is acquired, the processor (180) can determine whether there is an intention to use the home appliance based on the user's location data collected over a certain period of time prior to the acquisition of the event. The certain period of time may be 3 seconds, but this is merely an example.

[0207] The processor (180) can track the user's location based on the user's location data collected over a certain period of time prior to the acquisition of an event of the home appliance. The processor (180) can obtain the user's movement path based on the user's location tracking.

[0208] In one embodiment, the processor (180) may determine that the user intends to use the home appliance if the tracked user's movement path matches a preset pattern. The preset pattern may be a straight line pattern, but this is merely an example.

[0209] In another embodiment, the processor (180) may determine that there is an intention to use the home appliance if the distance of the tracked user's movement path is greater than a certain distance.

[0210] If it is determined that there is an intention to use the home appliance, the processor (180) can calculate an average of locations based on the user's location data corresponding to the time of occurrence of the event of the home appliance, and can obtain the calculated average as the first center coordinate (S1105).

[0211] The processor (180) can calculate an average coordinate value of the user's location data corresponding to the time of occurrence of an event of the home appliance. The processor (180) can obtain the average coordinate value as the first center coordinate of the home appliance.

[0212] The processor (180) can obtain the second center coordinates (S1107).

[0213] The processor (180) of the artificial intelligence device (100) can calculate an average distance between the location of the millimeter wave sensor and the user's location. The processor (180) can remove, from among the locations used to calculate the first center coordinates, the user's location whose distance from the location of the millimeter wave sensor is greater than the average distance. After removal, the processor (180) can recalculate the average coordinate values ​​of the remaining locations to obtain the second center coordinates.

[0214] The processor (180) can estimate the acquired second center coordinates as the location of the home appliance (S1109).

[0215] The processor (180) can obtain the locations of multiple home appliances in the above manner.

[0216] In one embodiment, the processor (180) can place the location of each acquired home appliance within the detection area of ​​the millimeter wave sensor.

[0217] Figures 12a and 12b are diagrams showing the linkage of a user's location data and an event of a home appliance.

[0218] Referring to FIG. 12A, the artificial intelligence device (100) can identify a first location distribution (1210) within a detection area (1200) based on the user's location data collected in real time. The first location distribution (1210) may be a heat map based on the user's accumulated location data.

[0219] The artificial intelligence device (100) can obtain a first location distribution (1210) based on location data collected when the air purifier is powered on. The artificial intelligence device (100) can estimate the location of the air purifier using the first location distribution (1210). The first location distribution (1210) can include a user location data set collected when the air purifier is powered on.

[0220] The artificial intelligence device (100) can sequentially calculate the first center coordinate and the second center coordinate through the first location distribution (1210), and can obtain the second center coordinate as the location of the air purifier.

[0221] Referring to FIG. 12b, the artificial intelligence device (100) can identify a second location distribution (1230) within a detection area (1200) based on the user's location data collected in real time. The second location distribution (1230) may be a heat map based on the user's accumulated location data.

[0222] The artificial intelligence device (100) can obtain a second location distribution (1230) based on location data collected when the refrigerator door is opened. The artificial intelligence device (100) can estimate the location of the refrigerator using the second location distribution (1230). The second location distribution (1230) can include a user location data set collected when the refrigerator door is opened.

[0223] The artificial intelligence device (100) can sequentially calculate the first center coordinate and the second center coordinate through the second location distribution (1230), and obtain the second center coordinate as the location of the refrigerator.

[0224] The artificial intelligence device (100) can estimate the location of the air purifier and then update the relative location of the refrigerator within the detection area.

[0225] FIG. 13 is a drawing illustrating an example of identifying the location of an estimated home appliance according to one embodiment of the present disclosure.

[0226] The processor (180) of the artificial intelligence device (100) can display a location estimation screen (1300) of the home appliance on the display (151).

[0227] The home appliance location estimation screen (1300) may include the location of the millimeter wave sensor (1301), the detection area of ​​the millimeter wave sensor (1310), the location of the first home appliance (1311), and the location of the second home appliance (1313).

[0228] The detection area (1310) may correspond to the detection area (1200) of FIGS. 12a and 12b.

[0229] The location of the first home appliance (1311) may represent the location of the air purifier of Fig. 12a, and the location of the second home appliance (1313) may represent the location of the refrigerator of Fig. 12b.

[0230] According to an embodiment of the present disclosure, the relative positions of home appliances can be identified using millimeter wave sensors and events of the home appliances.

[0231] Thus, according to embodiments of the present disclosure, the position and relative positions of a home appliance can be estimated using millimeter wave sensors and home appliance events. The estimated positions and relative positions of the home appliances can be useful for efficient placement and control of the home appliances.

[0232] Additionally, since there is no need for a separate camera or other filming device, the user's privacy can be protected.

[0233] FIG. 14 is a diagram illustrating a position estimation system according to one embodiment of the present disclosure.

[0234] Fig. 14 may be a location estimation system (1400) that estimates the location of a home appliance.

[0235] The location estimation system (1400) may include a millimeter wave sensor (1101), a home appliance (1401), a cloud server (1030), and an AI server (200). The location estimation system (1400) may further include an AI device (100).

[0236] The millimeter wave sensor (1101) can collect the user's location data and transmit the collected location data to a cloud server (1030).

[0237] The home appliance (1401) can detect the occurrence of an event and transmit information about the detected event to the cloud server (1030). The information about the event may include one or more of the type of operating state of the home appliance (1401) or the time of occurrence of the event.

[0238] The cloud server (1030) can transmit the user's location data and events of the home appliance (1401) to the AI ​​server (200) or AI device (100).

[0239] The AI ​​server (200) or AI device (100) can determine whether there is an intention to use the home appliance based on location data prior to the occurrence of an event of the home appliance.

[0240] If the AI ​​server (200) or AI device (100) determines that there is an intention to use the home appliance, the AI ​​server (200) or AI device (100) can calculate an average of locations based on the user's location data corresponding to the time of occurrence of the event of the home appliance, and obtain the calculated average as the first center coordinate.

[0241] The AI ​​server (200) or AI device (100) can calculate the average distance between the location of the millimeter wave sensor and the user's location. The AI ​​server (200) or AI device (100) can remove the user's location whose distance from the millimeter wave sensor is greater than the average distance from the locations used to calculate the first center coordinate. After the removal, the AI ​​server (200) or AI device (100) can recalculate the average coordinate value of the remaining locations to obtain the second center coordinate.

[0242] The AI ​​server (200) or AI device (100) can estimate the second center coordinates as the location of the home appliance.

[0243] FIG. 15 is a flowchart illustrating an operation method of an artificial intelligence device according to another embodiment of the present disclosure.

[0244] FIG. 15 may be an embodiment of identifying a user's activity space and a location of a home appliance within a detection area based on the user's location data and an event of a home appliance.

[0245] Referring to FIG. 15, the processor (180) of the artificial intelligence device (100) can obtain the user's location data and the events of the home appliance (S1501).

[0246] The processor (180) can receive the user's location data from the millimeter wave sensor.

[0247] The processor (180) can receive events of a home appliance from a home appliance or a cloud server (1030).

[0248] The processor (180) can obtain the user's activity space based on the user's location data (S1503).

[0249] The processor (180) can acquire the user's activity space within the detection area of ​​the millimeter wave sensor. The process of acquiring the user's activity space based on the user's location data is replaced with the description of the embodiments of FIGS. 3 and 4.

[0250] The processor (180) can obtain the location of the home appliance based on the user's location data and the event of the home appliance (S1505).

[0251] The processor (180) can acquire the location of the home appliance within the detection area based on the user's location data and the home appliance's events. The process of acquiring the location of the home appliance based on the user's location data and the home appliance's events is described in the embodiment of FIG. 11.

[0252] The processor (180) can identify the user's activity space and the location of the home appliance within the detection area (S1507).

[0253] The processor (180) can display a detection area where the user's activity space and the location of the home appliance are identified on the display (151). The processor (180) can display the detection area according to the execution of the home appliance management application.

[0254] FIG. 16 is a drawing illustrating a screen that provides a user's activity space, a main occupied space, and the location of home appliances according to an embodiment of the present disclosure.

[0255] Referring to FIG. 16, the processor (180) of the artificial intelligence device (100) can display a service screen (1600) on the display (151) upon receipt of a command.

[0256] The service screen (1600) may include an activity space (1630) of a user identified on a detection area (1610) of a millimeter wave sensor, a main occupied space (1631, 1633) included within the activity space (1630), a location (1651) of a first home appliance, and a location (1653) of a second home appliance.

[0257] The service screen (1600) may further include the position (P) of the millimeter wave sensor.

[0258] The service screen (1600) may further include the location (1671) of one or more pieces of furniture.

[0259] The service screen (1600) may be provided differently for each user. This is because location data may be collected differently for each user. The artificial intelligence device (100) may display a first service screen corresponding to the first user on the display (151) in response to a request from the first user, and may display a second service screen corresponding to the second user on the display (151) in response to a request from the second user.

[0260] Accordingly, control of home appliances optimized for each user can be performed.

[0261] The processor (180) can provide a recommended location of the home appliance based on the user's activity space (1630), main occupied space (1631, 1633), and location of the home appliance (1651, 1653).

[0262] For example, the processor (180) may display a placement guide on the display (151) to position the air purifier within the main occupied space (1631, 1633).

[0263] FIG. 17 is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0264] The embodiment of FIG. 17 can be performed after step S305 of FIG. 3.

[0265] The processor (180) of the artificial intelligence device (100) can control the operation of the home appliance based on the acquired user's activity space (S1701).

[0266] The appliance could be a robot vacuum cleaner, an air conditioner, or an air purifier, but these are just examples.

[0267] The processor (180) can control the operation of the home appliance based on the user's activity space and one or more major occupied spaces included in the activity space.

[0268] First, when the home appliance is a robot vacuum cleaner, an embodiment of controlling the robot vacuum cleaner based on the user's activity space or main occupied space is described.

[0269] In one embodiment, the processor (180) may control the operation of the robot cleaner to first clean the user's activity area. The processor (180) may transmit information about the detection area, the activity area identified within the detection area, the major occupied space, and the arrangement of furniture to the robot cleaner.

[0270] In another embodiment, the processor (180) may set the cleaning path of the robot cleaner to first clean major occupied spaces within the activity space.

[0271] The processor (180) can transmit a cleaning control signal including coordinate information of major occupied spaces to the robot cleaner via the communication interface (110). The robot cleaner can clean major occupied spaces based on the cleaning control signal received from the artificial intelligence device (100).

[0272] The processor (180) can transmit a cleaning control signal to the robot cleaner after detecting the last location of the user within the detection area. This is to ensure that cleaning is performed after the user leaves the detection area.

[0273] In one embodiment, the processor (180) may set a cleaning path by prioritizing major occupied spaces. The processor (180) may set the cleaning path of the robot cleaner so that cleaning is performed in order from spaces with a high frequency of location data to spaces with a low frequency among major occupied spaces. The processor (180) may transmit a cleaning control signal including the set cleaning path to the robot cleaner via the communication interface (110).

[0274] In another embodiment, the processor (180) may set different cleaning modes for each of the major occupied spaces. The cleaning modes may include a strong cleaning mode and a normal cleaning mode. The strong cleaning mode may be a mode that has a greater cleaning intensity and longer cleaning time than the normal cleaning mode.

[0275] Cleaning modes can vary based on multiple cleaning factors. These cleaning factors may include one or more of the following: cleaning time, motor suction power, brush rotation speed, pressure applied to the cleaning mop, steam output, water output, or the number of times a specific section is repeatedly cleaned.

[0276] Specifically, the strong cleaning mode may be a mode in which at least one of the plurality of cleaning elements is more than the normal cleaning mode.

[0277] Cleaning modes can be further subdivided into more modes than just the normal and strong cleaning modes. Multiple cleaning modes may have different sizes or strengths among one or more of the cleaning elements.

[0278] The processor (180) can determine the size or intensity of a plurality of cleaning elements that determine the cleaning mode differently depending on the frequency of the location data.

[0279] For example, the processor (180) can control the robot cleaner so that the size or strength of the plurality of cleaning elements that determine the cleaning mode increases as the frequency of the location data increases.

[0280] The processor (180) can control the robot cleaner so that the smaller the frequency of the location data, the smaller the size or intensity of the plurality of cleaning elements that determine the cleaning mode.

[0281] The processor (180) may set the cleaning mode for the main occupied space to a strong cleaning mode when the frequency of the location data is greater than or equal to a preset frequency. The processor (180) may set the cleaning mode for the main occupied space to a normal cleaning mode when the frequency of the location data is less than or equal to a preset frequency.

[0282] The processor (180) can transmit a cleaning control signal including a cleaning mode set for each major occupied space to the robot cleaner via the communication interface (110). Accordingly, cleaning of the space mainly occupied by the user can be intensively performed.

[0283] FIGS. 18 and 19 are drawings illustrating examples of differently controlling the cleaning path of a robot cleaner based on the main occupied space of each of a first user and a second user according to an embodiment of the present disclosure.

[0284] FIG. 18 may be a diagram illustrating a first user service screen (1800) including a detection area (1810) detected based on the position (P) of a millimeter wave sensor. The first user service screen (1800) may be referred to as a first user map. The artificial intelligence device (100) may display the first user service screen (1800) corresponding to the first user on the display (151).

[0285] The artificial intelligence device (100) can identify the first user's first activity space (1830) and the major occupied spaces (1831, 1833) within the first activity space (1830) based on location data corresponding to the first user. The first activity space (1830) and the major occupied spaces (1831, 1833) within the first activity space (1830) can be expressed in the form of a heat map.

[0286] The artificial intelligence device (100) can transmit a first cleaning control signal to the robot cleaner to perform cleaning along a first cleaning path (Path1) starting from the first main occupied space (1831) and moving to the second main occupied space (1833).

[0287] The robot vacuum cleaner can perform cleaning along a first cleaning path (Path1) according to a first cleaning control signal.

[0288] FIG. 19 may be a second user service screen (1900) including a detection area (1810) detected based on the position (P) of the millimeter wave sensor located at the same location as FIG. 18. The second user service screen (1900) may be referred to as a second user map. The artificial intelligence device (100) may display the second user service screen (1900) corresponding to the second user on the display (151).

[0289] The artificial intelligence device (100) can identify the second user's second activity space (1930) and the major occupied spaces (1931, 1933) within the second activity space (1930) based on location data corresponding to the second user. The second activity space (1930) and the major occupied spaces (1931, 1933) within the second activity space (1930) can be expressed in the form of a heat map.

[0290] The shape and size of the first activity space (1830) of FIG. 18 may be different from the shape and size of the second activity space (1930) of FIG. 19.

[0291] The shape, size and location of each of the main occupied spaces (1831, 1833) of FIG. 18 may be different from those of the main occupied spaces (1931, 1933) of FIG. 19.

[0292] The artificial intelligence device (100) can transmit a second cleaning control signal to the robot cleaner to perform cleaning along a second cleaning path (Path2) starting from the third main occupied space (1931) and moving to the fourth main occupied space (1933).

[0293] The robot cleaner can perform cleaning along a second cleaning path (Path2) according to a second cleaning control signal.

[0294] In this way, according to embodiments of the present disclosure, the operation of home appliances can be controlled differently based on each user's primary occupied space. Accordingly, home appliance control can be performed in a personalized manner, thereby enhancing user convenience.

[0295] Next, when the home appliance is an air conditioner, an embodiment of controlling the air conditioner based on the user's activity space or main occupied space is described.

[0296] In one embodiment, the processor (180) can obtain the user's main occupied space by time interval and control the operation of the air conditioner differently by time interval.

[0297] For example, the processor (180) may extract a first major occupied space, which represents an area with the highest frequency of location data during the lunch hour, and a second major occupied space, which represents an area with the highest frequency of location data during the dinner hour. The first and second major occupied spaces may be obtained based on location data collected over a two-week period, but the two-week period is merely an example. Location data may be collected over a period from when a user enters the detection area until the user leaves the detection area.

[0298] The processor (180) can control the air conditioner to lower the temperature of the first main occupied space to a preset temperature before the lunch time period arrives.

[0299] Additionally, the processor (180) can control the air conditioner to lower the temperature of the second main occupied space to a preset temperature before the evening time period arrives.

[0300] In another embodiment, the processor (180) can recognize a main occupied space matching each user and control the air conditioner to adjust the temperature of the main occupied space matching the user.

[0301] For example, if the processor (180) detects that a first user will enter the detection area after a certain period of time, it can control the air conditioner to lower the temperature of the first main occupied space matching the first user by a preset temperature in advance.

[0302] When the processor (180) detects that a second user will enter the detection area after a certain period of time, it can control the air conditioner to lower the temperature of the second main occupied space matching the second user by a preset temperature.

[0303] FIGS. 20A and 20B are drawings illustrating an example of extracting a major occupied space for each time interval and controlling cooling of the extracted major occupied space according to an embodiment of the present disclosure.

[0304] FIG. 20A may be a drawing illustrating a user service screen (2000) including a detection area (2010) detected based on the position (P) of a millimeter wave sensor. The user service screen (2000) may be referred to as a user map. The artificial intelligence device (100) may display the user service screen (2000) on a display (151).

[0305] Referring to FIG. 20A, the artificial intelligence device (100) may identify the user's activity space (2030) and a first major occupied space (2031) within the activity space (2030) based on the user's location data. The first major occupied space (2031) may be a space based on location data obtained in a first time interval over a two-week period. The first major occupied space (2031) may be a set of unit areas in which the frequency of the user's location data is greater than or equal to a preset frequency. The first time interval may be a period from 12:00 to 14:00.

[0306] The activity space (2030) and the first major occupied space (2031) within the activity space (2030) can be represented in the form of a heat map.

[0307] The artificial intelligence device (100) may transmit a first cooling control signal to the air conditioner to lower the temperature of the first main occupied space to a preset temperature before the arrival of the first time period. The first cooling control signal may be a signal that controls the air volume and air speed of the air conditioner.

[0308] Accordingly, before the arrival of the first time period, the temperature of the first main occupied space (2031) where the user mainly stays is lowered in advance, so that the user does not feel the heat.

[0309] Referring to FIG. 20b, the artificial intelligence device (100) may identify the user's activity space (2030) and a second major occupied space (2033) within the activity space (2030) based on the user's location data. The second major occupied space (2033) may be a space based on location data obtained in a second time period over a two-week period. The second major occupied space (2033) may be a set of unit areas in which the frequency of the user's location data is greater than or equal to a preset frequency. The second time period may be from 7:00 PM to 9:00 PM.

[0310] The activity space (2030) and the second major occupied space (2033) within the activity space (2030) can be represented in the form of a heat map.

[0311] The artificial intelligence device (100) may transmit a second cooling control signal to the air conditioner to lower the temperature of the second main occupied space to a preset temperature before the arrival of the second time period. The second cooling control signal may be a signal that controls the air volume and air speed of the air conditioner.

[0312] Accordingly, before the second time period arrives, the temperature of the second main occupied space (2033) where users mainly stay is lowered in advance, so that users do not feel the heat.

[0313] The artificial intelligence device (100) can control the wind direction and wind speed of the air conditioner so that cooling is adjusted according to the user's movement path by time interval.

[0314] FIG. 21 is a diagram illustrating the configuration of an artificial intelligence cloud device according to another embodiment of the present disclosure.

[0315] The artificial intelligence cloud device (2100) may include a location database (2110), a heat map engine (2120), a result database (2130), and an engine processor (2150).

[0316] The location database (2110) can store user location data collected by the millimeter wave sensor (1101). One or more millimeter wave sensors (1101) may be provided. In this case, each millimeter wave sensor can transmit location data (coordinate information) along with an ID identifying the sensor to the artificial intelligence device (2100).

[0317] The location database (2110) can store an accumulated location data set.

[0318] The heat map engine (2120) can generate a heat map representing the location distribution of users based on a location data set. The heat map engine (2120) can generate multiple heat maps for each time interval for a single user.

[0319] The heat map engine (2120) can generate a user's activity space and major occupied space based on the heat map. The process of generating a user's activity space and major occupied space based on the heat map is the same as the embodiments of FIGS. 3 and 4.

[0320] The heat map engine (2120) can periodically generate heat maps, activity areas, and major occupied areas. The heat map engine (2120) can generate heat maps, activity areas, and major occupied areas when the accumulated capacity of location data exceeds a certain capacity.

[0321] The heat map engine (2120) can obtain a heat map, user activity space, and major occupied space for each location by using millimeter wave sensors installed in each of a plurality of locations.

[0322] The result database (2130) can store the generated heat map, user activity space, and major occupied space.

[0323] The engine processor (2150) can control the overall operation of the artificial intelligence device (2100). The engine processor (2150) can control the operation of the heat map engine (2120) and control home appliances such as a robot vacuum cleaner (2101) and an air conditioner (2103).

[0324] The engine processor (2150) can transmit information about the heat map, user's activity space, and major occupied space stored in the result database (2130) to the robot cleaner (2101) and the air conditioner (2103).

[0325] The engine processor (2150) can transmit a control signal to the robot cleaner (2101) or the air conditioner (2103) based on one or more of the heat map stored in the result database (2130), information about the user's activity space, or information about the main occupied space.

[0326] The artificial intelligence cloud device (2100) may be an example of the AI ​​device (100) of FIG. 1 or the AI ​​server (200) of FIG. 2.

[0327] If the artificial intelligence cloud device (2100) is an example of the AI ​​device (100) of FIG. 1, the location database (2110) and the result database (2130) may be included in the memory (170), and the heat map engine (2120) and the engine processor (2150) may be included in the processor (180).

[0328] If the artificial intelligence cloud device (2100) is an example of the AI ​​server (200) of FIG. 2, the location database (2110) and the result database (2130) may be included in the memory (230), and the heat map engine (2120) and the engine processor (2150) may be included in the processor (260).

[0329] FIG. 22 is a sequence diagram illustrating an operation method of a system according to an embodiment of the present disclosure.

[0330] The engine processor (2150) can transmit a request to the millimeter wave sensor (1101) to collect the user's location data (S2201).

[0331] The engine processor (2150) can transmit a request for collection of the user's location data and information about the collection cycle of the location data to the millimeter wave sensor (1101).

[0332] The engine processor (2150) can communicate with the millimeter wave sensor (1101) via a communication interface.

[0333] The millimeter wave sensor (1101) can collect the user's location data in response to a request and transmit the collected location data to the location database (2110) (S2203).

[0334] The millimeter wave sensor (1101) can collect its own identifier and location data of the user.

[0335] The location data base (2110) can accumulate location data received from the millimeter wave sensor (1101) to obtain a location data set, and transmit the obtained location data set to the heat map engine (2120) (S2205).

[0336] The heat map engine (2120) can request a location data set collected by time interval from the location database (2110).

[0337] The heat map engine (2120) can generate one or more of a heat map, a user's activity space, and a major occupied space based on a location data set (S2207), and can transmit result information including the heat map, the user's activity space, and the major occupied space to a result database (2130) (S2209).

[0338] The heat map engine (2120) can receive a control command indicating a heat map generation cycle received from the engine processor (2150), and can generate a heat map in a cycle according to the received control command.

[0339] The result database (2130) can store heat maps, user activity spaces, and major occupied spaces. The result database (2130) can store heat maps, user activity spaces, and major occupied spaces for each user. The result database (2130) can store heat maps, user activity spaces, and major occupied spaces for each time period.

[0340] The engine processor (2150) can transmit a result information request to the result database (2130) (S2211), and can receive result information from the result database (2130) in response to the result information request (S2213).

[0341] The engine processor (2150) can transmit result information and a control signal for controlling the operation of the home appliance (2200) to the home appliance (2200) (S2215).

[0342] The control signal may be a signal generated based on the result information.

[0343] The home appliance (2200) can perform an operation according to a control signal using the result information (S2217).

[0344] The home appliance (2200) may be either a robot vacuum cleaner (2101) or an air conditioner (2103) of FIG. 21.

[0345] An electronic device (100) according to an embodiment of the present disclosure may include a communication interface (110) and one or more processors (180) that acquire user location data, acquire user activity space and major occupied space based on the acquired location data, generate a control signal for controlling the operation of a home appliance based on the acquired activity space and major occupied space, and transmit the generated control signal to the home appliance through the communication interface.

[0346] The one or more processors (180) may acquire a first activity space and a first main occupied space of a first user, transmit a first control signal to the home appliance based on the first activity space and the first main occupied space, and acquire a second activity space and a second main occupied space of a second user, and transmit a second control signal to the home appliance based on the second activity space and the second main occupied space.

[0347] The above one or more processors (180) can acquire a main occupied space corresponding to each of a plurality of time intervals and control the operation of the home appliance differently for each time interval.

[0348] The above one or more processors (180) can control the operation of the air conditioner to lower the temperature of the first main occupied space corresponding to the first time period by a preset temperature before the first time period arrives, if the home appliance is an air conditioner, and can control the operation of the air conditioner to lower the temperature of the second main occupied space corresponding to the second time period by a preset temperature before the second time period arrives.

[0349] The above home appliance is a robot vacuum cleaner, and the one or more processors (180) can acquire a plurality of major occupied spaces, set a cleaning path connecting the plurality of major occupied spaces, and control the robot vacuum cleaner to clean according to the set cleaning path.

[0350] The one or more processors (180) can set priorities among a plurality of major occupied spaces and control the robot cleaner to sequentially clean the plurality of major occupied spaces based on the priorities.

[0351] The one or more processors (180) may receive the position data from the millimeter wave sensor (1101).

[0352] The one or more processors (180) may obtain a cumulative location data set based on the location data, generate a heat map representing the location distribution of the user based on the cumulative location data set, and obtain the user's activity space and the main occupied space based on the generated heat map.

[0353] The above-described present disclosure can be implemented as computer-readable code on a program-recorded medium. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer may include a processor (180) of an artificial intelligence device.

Claims

1. In electronic devices, communication interface; and One or more processors that obtain the user's location data, obtain the user's activity space and main occupied space based on the obtained location data, generate a control signal for controlling the operation of the home appliance based on the obtained activity space and main occupied space, and transmit the generated control signal to the home appliance through the communication interface. Electronic devices.

2. In paragraph 1, One or more of the above processors Acquire the first user's first activity space and the first main occupied space, Transmitting a first control signal to the home appliance based on the first activity space and the first main occupied space, Acquire the second user's second activity space and the second main occupied space, Transmitting a second control signal to the home appliance based on the second activity space and the second main occupied space. Electronic devices.

3. In paragraph 1, One or more of the above processors Obtain a main occupied space corresponding to each of multiple time intervals, and control the operation of the home appliance differently for each time interval. Electronic devices.

4. In paragraph 3, One or more of the above processors If the above home appliance is an air conditioner, before the first time period arrives, the operation of the air conditioner is controlled to lower the temperature of the first main occupied space corresponding to the first time period by a preset temperature, Before the second time period arrives, the operation of the air conditioner is controlled to lower the temperature of the second main occupied space corresponding to the second time period by a preset temperature. Electronic devices.

5. In paragraph 1, The above home appliance is a robot vacuum cleaner, One or more of the above processors Acquire multiple major occupied spaces, set a cleaning path connecting the multiple major occupied spaces, and control the robot cleaner to clean along the set cleaning path. Electronic devices.

6. In paragraph 5, One or more of the above processors Setting priorities among multiple major occupied spaces and controlling the robot cleaner to sequentially clean the multiple major occupied spaces based on the priorities. Electronic devices.

7. In paragraph 1, One or more of the above processors Receiving the above location data from a millimeter wave sensor Electronic devices.

8. In paragraph 1, One or more of the above processors Obtaining a cumulative location data set based on the above location data, generating a heat map representing the location distribution of the user based on the cumulative location data set, and obtaining the user's activity space and the main occupied space based on the generated heat map. Electronic devices.

9. In the method of operating an electronic device, Step of obtaining user location data; A step of acquiring the user's activity space and main occupied space based on the acquired location data; A step of generating a control signal for controlling the operation of the home appliance based on the acquired activity space and main occupied space; and A step of transmitting the generated control signal to the home appliance. How electronic devices work.

10. In paragraph 9, The steps for acquiring the user's activity space and main occupied space are as follows: Steps for acquiring the first activity space and the first main occupied space of the first user and Including a step of acquiring a second activity space and a second main occupied space of a second user, The above transmitting steps are A step of transmitting a first control signal to the home appliance based on the first activity space and the first main occupied space, and A step of transmitting a second control signal to the home appliance based on the second activity space and the second main occupied space. How electronic devices work.

11. In paragraph 9, The steps to acquire the above major occupied space are Comprising a step of obtaining a major occupied space corresponding to each of a plurality of time intervals, The above transmitting steps are A step of controlling the operation of the home appliance differently for each time period, How electronic devices work.

12. In paragraph 11, The above controlling steps are If the above home appliance is an air conditioner, a step of controlling the operation of the air conditioner to lower the temperature of the first main occupied space corresponding to the first time period by a preset temperature before the first time period arrives; and A step of controlling the operation of the air conditioner to lower the temperature of the second main occupied space corresponding to the second time period by a preset temperature before the second time period arrives. How electronic devices work.

13. In paragraph 9, The above home appliance is a robot vacuum cleaner, The steps to acquire the above major occupied space are comprising the step of acquiring multiple major occupied spaces; The above transmitting steps are A step of setting a cleaning path connecting the above plurality of major occupied spaces, and A step of controlling the robot cleaner to clean along a set cleaning path is included. How electronic devices work.

14. In paragraph 13, The above controlling steps are The step of setting priorities between multiple major occupied spaces is A step of controlling the robot cleaner to sequentially clean a plurality of major occupied spaces based on priorities. How electronic devices work.

15. In paragraph 9, Further comprising the step of receiving the position data from a millimeter wave sensor. How electronic devices work.

Images