Electronic device and operating method thereof

Abstract

The electronic device may include: a memory configured to store location data of a user; and at least one processor configured to: obtain a cumulative location data set based on the location data, the cumulative location data set including information about a number of times a user is detected at one or more locations within the space; generating a thermodynamic map representing a location distribution of the user based on the cumulative location data set; and obtaining an activity space of the user based on the thermodynamic map, wherein the activity space is an area in the space.

Description

Technical Field

[0001] This disclosure relates to electronic devices, and more specifically, to electronic devices for estimating a user's activity space based on a user's location data. Background Technology

[0002] Traditional techniques for identifying user behavior or location information rely on sensors that depend on infrastructure installed in space.

[0003] Inexpensive sensors, such as passive infrared intrusion detection sensors or ultrasonic sensors, are installed in large spaces. Data is collected by recording video with a camera and tomographic imaging is performed using a wireless transceiver. In this method, the user's location can only be determined at locations where sensors are directly installed.

[0004] Based on the user's location, control can be exercised over home appliances within the home.

[0005] However, according to existing technology, it is possible to control home appliances to detect moving users and face the area where the user is located, but without taking into account the space blocked by walls and the user's activity radius within the space.

[0006] In other words, according to existing technology, the control of household appliances is limited to the user's location, thus limiting the effective control of household appliances. Summary of the Invention

[0007] The purpose of this disclosure is to estimate the shape of a space and the user's activity radius using user location data obtained through sensors.

[0008] The purpose of this disclosure is to optimize the control of home appliances based on the shape of the space and the user's activity radius.

[0009] The purpose of this disclosure is to identify the relative locations of users' primary residences and home appliances by using heat maps that accumulate user location information.

[0010] The purpose of this disclosure is to provide users with easy access to information based on their location and interactions with home appliances.

[0011] An electronic device according to an embodiment of the present disclosure may include: a memory configured to store user location data; and at least one processor configured to: obtain an accumulated location dataset based on the location data, generate a heat map representing the user's location distribution based on the accumulated location dataset, and obtain the user's activity space based on the generated heat map.

[0012] The operation method according to embodiments of the present disclosure may include: storing user location data; obtaining a cumulative location dataset based on the location data; generating a heat map representing the user's location distribution based on the cumulative location dataset; and obtaining the user's activity space based on the generated heat map.

[0013] According to embodiments of this disclosure, the energy efficiency of home appliances can be improved by optimally controlling them according to the type of user's living space.

[0014] According to embodiments of this disclosure, user convenience can be greatly improved by checking the user's activity radius and pre-executing the operation of home appliances in the main use space.

[0015] According to embodiments of this disclosure, the relative positions of the user's primary residence and home appliances can be identified, thereby enabling the control of home appliances in a more user-friendly manner.

[0016] According to embodiments of this disclosure, the location of a home appliance relative to the user can be determined by the user's location, thereby providing the user with information that facilitates interaction with nearby home appliances. Attached Figure Description

[0017] The above and other objects, features and advantages of this disclosure will become more apparent to those skilled in the art by referring to the exemplary embodiments of the present disclosure which are described in detail below with reference to the accompanying drawings, which are briefly described in detail.

[0018] Figure 1 This is a block diagram illustrating the elements of an artificial intelligence device according to embodiments of the present disclosure.

[0019] Figure 2 This is a diagram illustrating the configuration of an artificial intelligence server according to an embodiment of the present disclosure.

[0020] Figure 3 This is a flowchart illustrating a method of operating an artificial intelligence device according to embodiments of the present disclosure.

[0021] Figure 4 This is a flowchart illustrating the process of obtaining a user activity space based on a cumulative location dataset according to an embodiment of the present disclosure.

[0022] Figure 5 This is a diagram illustrating a heat map based on a cumulative location dataset according to an embodiment of the present disclosure.

[0023] Figure 6 This is a diagram illustrating a clustering map generated based on a heat map according to an embodiment of the present disclosure.

[0024] Figure 7A and Figure 7BThis is a diagram illustrating the process of obtaining the shape of a user activity space from a clustering map according to an embodiment of the present disclosure.

[0025] Figure 8 This is a diagram used to illustrate activity space information according to embodiments of the present disclosure.

[0026] Figure 9A and Figure 9B This is a diagram illustrating an example of identifying the main occupied space within the user activity space according to an embodiment of the present disclosure.

[0027] Figure 10 This is a diagram illustrating the configuration of a spatial understanding system according to an embodiment of the present disclosure.

[0028] Figure 11 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0029] Figure 12A and Figure 12B This is a diagram illustrating the link between user location data and events of home appliances according to embodiments of the present disclosure.

[0030] Figure 13 This is a diagram illustrating an example of identifying the estimated location of a household appliance according to an embodiment of the present disclosure.

[0031] Figure 14 This is a diagram illustrating a position estimation system according to an embodiment of the present disclosure.

[0032] Figure 15 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0033] Figure 16 This is a diagram illustrating a screen that provides a user's activity space, main occupied space, and the location of home appliances according to embodiments of the present disclosure.

[0034] Figure 17 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0035] Figure 18 and Figure 19 This is a diagram illustrating an example of controlling the cleaning path of a robotic vacuum cleaner differently based on the primary occupancy space of each of the first user and the second user, according to embodiments of the present disclosure.

[0036] Figure 20A and Figure 20B This is a diagram illustrating an example of extracting the main occupied space for each time period and controlling the cooling of the extracted main occupied space according to an embodiment of the present disclosure.

[0037] Figure 21 This is a diagram illustrating the configuration of an artificial intelligence cloud device according to another embodiment of the present disclosure.

[0038] Figure 22 This is a timing diagram illustrating a method of operating a system according to an embodiment of the present disclosure. Detailed Implementation

[0039] Artificial intelligence refers to the field of studying artificial intelligence or the methodology of creating artificial intelligence, while machine learning refers to the field of defining the various problems dealt with in the field of artificial intelligence and studying the methodology of solving these problems.

[0040] Machine learning is also defined as algorithms that improve the performance of a task through consistent experience.

[0041] Artificial Neural Networks (ANNs) are models used in machine learning. They can refer to an overall model with problem-solving capabilities, composed of artificial neurons (nodes) that form a network through the combination of synapses.

[0042] Artificial neural networks can be defined by the connection patterns between neurons in different layers, the learning process for updating model parameters, and the activation functions that generate output values.

[0043] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may include one or more neurons, and the artificial neural network may include synapses connecting the neurons. In an artificial neural network, each neuron may output the input signal received through the synapse, weights, and activation function values ​​used for bias.

[0044] Model parameters are parameters determined through learning, including the weights of synaptic connections and the biases of neurons. Hyperparameters are parameters set before learning in a machine learning algorithm, including the learning rate, number of repetitions, mini-batch size, and initialization function.

[0045] The goal of learning an artificial neural network can be viewed as determining the model parameters that minimize the loss function. The loss function can be used as an indicator of the optimal model parameters during the learning process of the artificial neural network.

[0046] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning based on the learning method.

[0047] Supervised learning refers to the method of training an artificial neural network using labels for given learning data. The labels can represent the correct answer (or result value) inferred by the artificial neural network when the learning data is input into the artificial neural network.

[0048] Unsupervised learning refers to the method of training artificial neural networks without providing labels for training data.

[0049] Reinforcement learning can refer to a learning method in which an agent defined in an environment learns to select actions or sequences of actions that maximize the cumulative reward in each state.

[0050] In artificial neural networks, machine learning implemented using deep neural networks (DNNs) that include multiple hidden layers is also called deep learning, and deep learning is a part of machine learning.

[0051] In the following text, machine learning will be used to include deep learning.

[0052] Features of the various embodiments of this disclosure may be coupled or combined with each other in part or in whole, and may be interlocked and operated in various technical ways, and embodiments may be performed independently of each other or in association with each other. Furthermore, the term "can" as used herein includes all the meanings and definitions of the term "may".

[0053] Figure 1 This is a block diagram illustrating the elements of an artificial intelligence device according to embodiments of the present disclosure.

[0054] Artificial intelligence device 100 can be implemented as a fixed or mobile device, such as a TV, projector, mobile phone, smartphone, desktop computer, laptop, digital broadcasting terminal, PDA (personal digital assistant), PMP (portable multimedia player), navigation, tablet PC, wearable device, set-top box (STB), DMB receiver, radio, washing machine, refrigerator, desktop computer, digital signage, robot, vehicle, etc.

[0055] refer to Figure 1 The artificial intelligence device 100 may include a communication interface 110, an input interface 120, a learning processor 130, a sensor 140, an output interface 150, a memory 170, and a processor 180.

[0056] The communication interface 110 can use wired or wireless communication technologies to send and receive data with external devices such as other artificial intelligence devices or AI server 200. For example, the communication interface 110 can send and receive sensor information, user input, learning models, and control signals with external devices.

[0057] The communication technologies used by the communication interface 110 include Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), 5G, Wireless LAN (WLAN), Wi-Fi, Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0058] Input interface 120 can obtain various types of data.

[0059] The input interface 120 may include a camera 121 for capturing images, a microphone 122 for receiving audio signals, and a user input interface 123 for receiving information from a user.

[0060] The camera 121 or microphone 122 is regarded as a sensor, and the signal obtained from the camera 121 or microphone 122 can be referred to as sensing data or sensor information.

[0061] Input interface 120 can obtain training data for model learning and input data to be used when obtaining output using the learned model. Input interface 120 can obtain unprocessed input data, and in this case, processor 180 or learning processor 130 can extract input features by preprocessing the input data.

[0062] Camera 121 processes image frames, such as still or moving images, acquired by the image sensor in video call mode or shooting mode. The processed image frames can be displayed on display 151 or stored in memory 170.

[0063] Microphone 122 processes external acoustic signals into electronic speech data. The processed speech data can be utilized in various ways depending on the function (or application) being performed by the artificial intelligence device 100. Simultaneously, various noise removal algorithms can be applied to microphone 122 to remove noise generated during the reception of external acoustic signals.

[0064] User input interface 123 is used to receive information from the user. When information is input through user input interface 123, processor 180 can control the operation of artificial intelligence device 100 to correspond to the input information.

[0065] User input interface 123 is a mechanical input device (or mechanical key, such as a button, dome switch, scroll wheel or micro switch, etc. located on the front / rear or side of the artificial intelligence device 100) and a touch input device.

[0066] As an example, touch input can consist of virtual keys, soft keys, or visual keys displayed on the touchscreen via software processing, or it can consist of touch keys placed in a portion outside the touchscreen.

[0067] The learning processor 130 can use training data to train a model composed of an artificial neural network. The learned artificial neural network can be called a learning model. The learning model can be used to infer the result value of new input data other than the training data, and the inferred value can be used as the basis for decisions to perform operations.

[0068] The learning processor 130 can perform AI processing together with the learning processor 240 of the AI ​​server 200.

[0069] The learning processor 130 may include memory integrated or implemented in the artificial intelligence device 100. The learning processor 130 may be implemented using memory 170, external memory directly coupled to the artificial intelligence device 100, or memory maintained in an external device.

[0070] Sensor 140 can use various sensors to acquire at least one of the following: internal information of artificial intelligence device 100, information about the surrounding environment of artificial intelligence device 100, or user information.

[0071] Sensor 140 may include at least one of a proximity sensor, a lighting sensor, an acceleration sensor, a magnetic sensor, a gyroscope sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint sensor, an ultrasonic sensor, an optical sensor, a microphone, a lidar sensor, or a radar sensor.

[0072] Output interface 150 can generate outputs related to vision, hearing, or touch.

[0073] The output interface 150 may include a display 151 for outputting images, an audio output interface 152 for outputting audio, a tactile device 153 for outputting tactile information, and an optical output interface 154 for outputting light.

[0074] Display 151 displays (outputs) information processed by artificial intelligence device 100. For example, display 151 may display implementation screen information of an application running on artificial intelligence device 100, or user interface (UI) and graphical user interface (GUI) information based on the implementation screen information.

[0075] The display 151 can be implemented as a touchscreen by forming an interlayer structure or by integrating it with a touch sensor. The touchscreen serves as a user input interface 123, providing an input interface between the artificial intelligence device 100 and the user, and can simultaneously provide an output interface between the artificial intelligence device 100 and the user.

[0076] The audio output interface 152 can output audio data received from the communication interface 110 or audio data stored in the memory 170 in call signal receiving, call mode or recording mode, voice recognition mode, broadcast receiving mode, etc.

[0077] The audio output interface 152 may include at least one of a receiver, a speaker, or a buzzer.

[0078] The haptic device 153 generates various haptic effects that a user can perceive. A representative example of a haptic effect generated by the haptic device 153 could be vibration.

[0079] The optical output interface 154 uses light from a light source in the artificial intelligence device 100 to output a signal to notify of an event that has occurred. Examples of events occurring in the artificial intelligence device 100 may include receiving a message, receiving a call signal, a missed call, an alarm, a calendar notification, receiving an email, receiving information via an application, etc.

[0080] The memory 170 can store data that supports various functions of the artificial intelligence device 100. For example, the memory 170 can store input data, learning data, learning models, learning history, etc., obtained from the input interface 120.

[0081] The processor 180 can determine at least one operable operation of the artificial intelligence device 100 based on information determined or generated using data analysis algorithms or machine learning algorithms.

[0082] The processor 180 can control the components of the artificial intelligence device 100 to perform the determined operations.

[0083] For this purpose, processor 180 can request, search, receive, or utilize data from learning processor 130 or memory 170, and can control the elements of artificial intelligence device 100 to perform at least one predicted operation or determined as a desired operation in an operable operation.

[0084] If it is necessary to link with an external device to perform a specific operation, the processor 180 can generate control signals to control the external device and send the generated control signals to the external device.

[0085] The processor 180 can obtain intent information for user input and determine the user's request based on the obtained intent information.

[0086] The processor 180 may use at least one of an STT (Speech to Text) engine for converting speech input into a string or a Natural Language Processing (NLP) engine for obtaining intent information corresponding to the user input.

[0087] At least one of the STT engine and the NLP engine may consist of at least a portion of an artificial neural network learned according to a machine learning algorithm. Furthermore, at least one of the STT engine or the NLP engine may be learned by the learning processor 130, by the learning processor 240 of the AI ​​server 200, or through its distributed processing.

[0088] Processor 180 collects historical information, including user feedback on the operation of AI device 100, and stores it in memory 170, learning processor 130, or AI server 200, etc. This historical information can be sent to external devices. The collected historical information can be used to update the learning model.

[0089] The processor 180 can control at least some of the components of the artificial intelligence device 100 to run applications stored in the memory 170.

[0090] The processor 180 can operate two or more components included in the artificial intelligence device 100 in combination to run an application.

[0091] Figure 2 This is a diagram illustrating the configuration of an artificial intelligence server according to an embodiment of the present disclosure.

[0092] refer to Figure 2 AI server 200 can refer to a device that uses machine learning algorithms to train artificial neural networks or uses learned artificial neural networks.

[0093] AI server 200 may consist of multiple servers to perform distributed processing and may be defined as a 5G network. AI server 200 may be included as part of artificial intelligence device 100 and may perform at least part of the AI ​​processing.

[0094] AI server 200 may include communication interface 210, memory 230, learning processor 240 and processor 260.

[0095] The communication interface 210 can send and receive data with external devices such as artificial intelligence device 100.

[0096] The memory 230 may include a model memory 231. The model memory 231 may store models (or artificial neural networks, 231a) that have been learned or are being trained by the learning processor 240.

[0097] The learning processor 240 can use training data to train the artificial neural network 231a. The learning model can be installed on the AI ​​server 200 of the artificial neural network for use, or it can be installed and used on an external device such as the artificial intelligence device 100.

[0098] The learning model can be implemented using hardware, software, or a combination of hardware and software. When the learning model is implemented, either partially or entirely, as software, one or more instructions constituting the learning model can be stored in memory 230.

[0099] Processor 260 can use the learning model to infer the result value of new input data and generate a response or control command based on the inferred result value.

[0100] In the following text, artificial intelligence device 100 or AI server 200 may be referred to as electronic device.

[0101] Figure 3 This is a flowchart illustrating a method of operating an artificial intelligence device according to embodiments of the present disclosure.

[0102] One or more processors may be provided below.

[0103] refer to Figure 3 The processor 180 of the artificial intelligence device 100 can obtain the user's location data (S301).

[0104] In one embodiment, the processor 180 may obtain user location data through either the sensor 140 disposed in the artificial intelligence device 100 or a sensor disposed separately from the artificial intelligence device 100.

[0105] The sensor used to obtain user location data can be a millimeter-wave (mmWave) sensor. A millimeter-wave sensor is a sensor that uses electromagnetic waves with very short wavelengths to detect objects. Millimeter-wave sensors can be placed in a fixed location.

[0106] Millimeter-wave sensors can include a transmitting antenna and a receiving antenna.

[0107] The transmitting antenna of a millimeter-wave sensor can emit electromagnetic waves operating in the frequency range between 30 GHz and 300 GHz. The receiving antenna of a millimeter-wave sensor can receive electromagnetic waves reflected when the emitted electromagnetic waves strike an object (e.g., a user).

[0108] Millimeter-wave sensors can measure the distance to an object based on the time it takes for emitted electromagnetic waves to reflect and return to the object. A detection area can be defined within which the millimeter-wave sensor can detect objects mounted in a fixed position. Processor 180 can identify the user's location using the coordinates within the detection area.

[0109] The processor 180 can convert the distance between the user and the millimeter-wave sensor received from the millimeter-wave sensor into coordinate information, and obtain the converted coordinate information as user location data. The processor 180 can obtain user location data in real time.

[0110] Processor 180 can detect the movement of an object by recognizing changes in the distance between the millimeter-wave sensor and the object.

[0111] Processor 180 can remove location data that has moved within a short period of time from the acquired location data. This is to remove noise or data about objects other than people. A short period could be 0.1 seconds, but this is just an example.

[0112] The processor 180 can identify a user based on sensing information received from a millimeter-wave sensor. The sensing information may include at least one of the following: the distance between the millimeter-wave sensor and the object, and the shape or size of the object based on the phase change between emitted and reflected electromagnetic waves.

[0113] Processor 180 can identify users based on the shape or size of an object. That is, processor 180 can identify each of multiple users whose objects have different shapes or sizes.

[0114] The processor 180 can obtain a cumulative location dataset based on the obtained user location data (S303).

[0115] A cumulative location dataset can be a dataset that accumulates the number of times a user is detected at each location based on location data. A user's location can be represented as coordinates in space. Each cumulative location data point included in the cumulative location dataset can include both spatial coordinates and frequency.

[0116] The cumulative location dataset is a dataset that takes into account the maximum detection width, maximum detection length, and cumulative frequency of the millimeter-wave sensor, and can be stored in memory 140. Therefore, the cumulative dataset has a maximum capacity, thus having the advantage of not occupying a large amount of capacity in memory 140.

[0117] Processor 180 can obtain a cumulative location dataset using location data accumulated over a certain period of time. Processor 180 can obtain the cumulative location dataset by updating the location data obtained during that period of time.

[0118] The processor 180 can obtain the user's activity space based on the accumulated location dataset (S305).

[0119] In one embodiment, processor 180 can estimate the activity space of a user within the sensing area based on an accumulated location dataset. The detection area can be an area capable of detecting objects using a millimeter-wave sensor. The detection area can be formed based on the angle and transmission distance of the electromagnetic waves emitted by the millimeter-wave sensor.

[0120] Processor 180 can generate clustered data by clustering the accumulated location dataset. Processor 180 can then use a polygon algorithm to estimate the user activity space from the clustered data.

[0121] Processor 180 can obtain activity space information corresponding to the estimated user activity space. The process of obtaining the user activity space based on the cumulative location dataset is described in detail.

[0122] Figure 4 This is a flowchart illustrating the process of obtaining a user activity space based on a cumulative location dataset according to an embodiment of the present disclosure.

[0123] Figure 4 It can be Figure 3 The detailed diagram for step S305.

[0124] The processor 180 of the artificial intelligence device 100 can generate a clustering map by clustering the accumulated location dataset (S401).

[0125] In one embodiment, processor 180 may use density-based clustering techniques to generate clustered data. The clustered data may be referred to as a cluster map.

[0126] The processor 180 can use the cumulative location dataset to generate a heat map representing the distribution of the cumulative location data, and can generate clustered data based on the generated heat map.

[0127] The processor 180 can use density-based clustering techniques to extract multiple cluster regions from the cumulative location dataset and use the extracted multiple cluster regions to generate clustered data.

[0128] Density-based clustering techniques can be density-based spatial clustering of applications with noise (DBSCAN).

[0129] In DBSCAN technology, a minimum number of data pointers can be set to form clustered regions. The processor 180 can then calculate the number of other data points within the radius of a data point in the cumulative location dataset. Subsequently, the processor 180 can identify high-density regions with a dense number of other data points as clustered regions, and treat low-density regions with a sparse number of other data points as noise.

[0130] Processor 180 can identify high-density areas to obtain clustering data (or the final clustering map).

[0131] Reference Figure 5 and Figure 6 This will be described.

[0132] Figure 5 This is a diagram illustrating a heat map based on a cumulative location dataset according to an embodiment of the present disclosure. Figure 6 This is a diagram illustrating a clustering map generated based on a heat map according to an embodiment of the present disclosure.

[0133] exist Figure 5 In the process, the actual furniture and wall arrangements are projected onto the heat map 500 for reference.

[0134] Processor 180 can generate an accumulated location dataset by accumulating user location data obtained through millimeter-wave sensors. Processor 180 can then use the accumulated location dataset to generate data such as... Figure 5 The heat map shown is 500.

[0135] The horizontal axis of the heat map 500 represents the detection width of the millimeter-wave sensor, and the vertical axis represents the detection length. The detection width can have a positive value to the right centered on the position of the millimeter-wave sensor, and a negative value to the left centered on the position of the millimeter-wave sensor.

[0136] Each point in a heatmap 500 can represent the cumulative number (or cumulative frequency) of location data.

[0137] The cumulative location dataset 500 can be represented as a heatmap. A heatmap is a graphical tool that visually represents the cumulative distribution of user location data. Heatmaps can use colors on a two-dimensional grid to represent the density or frequency of location data. Darker colors indicate higher frequencies, and lighter colors indicate lower frequencies.

[0138] refer to Figure 5 The frequency of location data can be represented as 0 to 140, with higher frequencies shown in red and lower frequencies shown in blue.

[0139] In one embodiment, processor 180 may generate a first type of heat map based on a cumulative location dataset obtained over a preset time period. The preset time period may be any one of a lunch break, a dinner break, or a specific time period.

[0140] In another embodiment, processor 180 can identify users and generate a second type of heat map based on a cumulative location dataset of the identified users. That is, processor 180 can generate a heat map corresponding to each of a plurality of users. Personalized control of home appliances can be performed using the heat map corresponding to each user.

[0141] In another embodiment, processor 180 may use a cumulative location dataset of users identified during a preset time period to generate a third type of heat map.

[0142] Processor 180 can obtain data based on heatmap 500, such as... Figure 6 The clustering map 600 shown is also called a clustering graph or clustering space.

[0143] Processor 180 can use DBSCAN technology to obtain cluster map 600.

[0144] The horizontal axis of the cluster map 600 represents the detection width of the millimeter-wave sensor, and the vertical axis represents the detection length. The detection width can have a positive value to the right centered on the position of the millimeter-wave sensor, and a negative value to the left centered on the position of the millimeter-wave sensor.

[0145] The processor 180 can use DBSCAN technology to identify high-density and low-density regions based on each data point included in the clustering map 600.

[0146] The processor 180 can identify cluster regions 610 containing high-density areas from the cluster map 600.

[0147] Next, the description Figure 4 .

[0148] The processor 180 can obtain the shape of the user activity space from the clustered map generated using a polygon approximation algorithm (S403).

[0149] In one embodiment, the polygon approximation algorithm may be the Ramer-Douglas-Peucker algorithm or the convex hull algorithm.

[0150] The processor 180 can use a polygon approximation algorithm to obtain the shape of the user activity space that matches the clustering region 610.

[0151] The Ramer-Douglas-Peucker algorithm can be an algorithm that simplifies the outline of clustered regions 610 to generate polygonal regions.

[0152] The Ramer-Douglas-Peucker algorithm can simplify clustering regions 610 in the following way.

[0153] 1. Draw a straight line connecting the starting point and the ending point.

[0154] 2. Find the point farthest from the line. If the distance between the line and that point is greater than a threshold, include that point and generate two new line segments.

[0155] 3. Recursively repeat this process to simplify all line segments so that they have a distance less than a threshold.

[0156] The convex hull algorithm can be an algorithm that obtains the convex hull of each group when multiple data points are divided into two groups, and combines the convex hulls of these groups to obtain the entire convex hull as a polygonal region.

[0157] The processor 180 can obtain activity space information based on the shape obtained from the user's activity space (S405).

[0158] In one embodiment, the activity space information may include at least one of the coordinates of the vertices of the polygon representing the activity space, the shape of the activity space, the area of ​​the activity space, the length of the activity space, or the effective angle based on the millimeter-wave sensor.

[0159] The processor 180 can obtain activity space information, which includes at least one of the area of ​​the activity space or the angle formed by a millimeter-wave sensor that collects user location data using the activity space and based on the shape of the user's activity space.

[0160] The processor 180 can control the operation of home appliances based on activity space information. This will be described later.

[0161] Figure 7A and Figure 7B This is a diagram illustrating the process of obtaining the shape of a user activity space from a clustering map according to embodiments of the present disclosure, and Figure 8 It is a diagram illustrating information about the activity space.

[0162] refer to Figure 7A and Figure 7B This demonstrates the use of polygon approximation algorithms from Figure 6 The clustering map 600 yielded user activity spaces 710 and 730.

[0163] exist Figure 7AIn the activity space 710, the actual furniture and wall arrangement is projected onto the activity space for reference.

[0164] The processor 180 can use the aforementioned polygon algorithm to generate user activity spaces 710 and 730 from cluster regions 610 of the cluster map 600.

[0165] The processor 180 can obtain activity space information from activity spaces 710 and 730. The activity space information may include at least one of the following: the coordinates of the vertices of the polygon represented by activity spaces 710 and 730, the area of ​​activity spaces 710 and 730, or one or more effective angles of activity spaces 710 and 730 based on millimeter-wave sensors.

[0166] Figure 8 It is a reference. Figure 7A A graph of the activity space information obtained from the clustered region 710. The activity space information may include at least one of the effective angles (58 degrees, 27 degrees, 95 degrees) of the activity space 710 measured based on the position (P) of the millimeter-wave sensor or the area of ​​the activity space 710.

[0167] The first effective angle (58 degrees) of the activity space 710 can be the angle formed between one side 711 of the wall at the location (P) where the millimeter-wave sensor is placed and the first side 713 of the activity space 710.

[0168] The second effective angle (27 degrees) of the activity space 710 can be the angle formed between one side 711 of the wall at the location (P) where the millimeter-wave sensor is placed and the second side 715 of the activity space 710. The first side 713 and the second side 715 are adjacent, and the extension line of the second side 715 can intersect the extension line of the first side 713.

[0169] The third effective angle (95 degrees) of the activity space 710 can be the angle formed by the extension of the first side 713 and the extension of the second side 715 based on the position (P) of the millimeter-wave sensor.

[0170] Processor 180 can use the coordinates of the vertices of the polygon representing the activity space 710 to calculate the area of ​​the activity space 710. Processor 180 can use the known shoelace formula to calculate the area of ​​the activity space 710.

[0171] Figure 9A and Figure 9B This is a diagram illustrating an example of identifying the main occupied space within the user activity space according to an embodiment of the present disclosure.

[0172] Figure 9A and Figure 9BThis describes the user activity space and main occupied space that can be identified within the detection area that can be detected by millimeter-wave sensors.

[0173] The main space occupied can be the space representing the area where the cumulative frequency of user location data is greater than a preset frequency.

[0174] refer to Figure 9A It can identify the main occupied spaces 901 and 903 and the position P of the millimeter-wave sensor within the user's activity space 710. Each of the main occupied spaces 901 and 903 can be a space where the frequency of accumulated position data is higher than a preset frequency.

[0175] refer to Figure 9B The location P of the main occupied spaces 911 and 913 and the millimeter-wave sensor can be identified within the user activity space 910. Each of the main occupied spaces 911 and 913 can be a space in which the frequency of accumulated location data is higher than a preset frequency.

[0176] Figure 9B The user activity space 910 in the data can be obtained based on location data collected from 3 p.m. to 9 p.m.

[0177] Artificial intelligence device 100 can send information about user activity spaces 710, 910 and main occupied spaces 901, 903, 911, 913 to user equipment via communication interface 110.

[0178] Information about user activity spaces 710 and 910 may include activity space information. Information about primary occupied spaces 901, 903, 911, and 913 may include at least one of the location, area, or shape of primary occupied spaces 901, 903, 911, and 913.

[0179] User equipment can be any of a device such as a smartphone, tablet, PC, or laptop computer. User equipment may include... Figure 1 All components of the artificial intelligence device 100. The user device may be the artificial intelligence device 100.

[0180] Home appliance management applications that provide information about user activity spaces based on user location data can be installed on user devices. For example... Figure 9A or Figure 9B As shown, through the installed home appliance management application, the user device can display activity radius screens 900-1 and 900-2, including user activity spaces 710 and 910 and main occupied spaces 901, 903, 911, and 913.

[0181] In another embodiment, the processor 180 of the artificial intelligence device 100 can display on the display 151 activity radius screens 900-1 and 900-2, including user activity spaces 710 and 910 and main occupied spaces 901, 903, 911, and 913.

[0182] Thus, according to embodiments of this disclosure, the user activity space and main occupied space can be obtained using millimeter-wave sensors, eliminating the need for imaging equipment such as cameras. The user activity space and main occupied space can be used to effectively control home appliances in the future.

[0183] Additionally, it is possible to estimate the user's activity space and main occupied space without using a camera, thus protecting the user's privacy.

[0184] The processor 180 can control the operation of household appliances based on the main footprints 901, 903, 911, and 913. This will be described later.

[0185] At the same time, Figure 9A and Figure 9B Electronic devices such as furniture or TVs can be identified and displayed within the user's activity space 710 and 910.

[0186] At the same time, refer to Figure 9B The activity radius screen 900-2 may also include a progress bar 920. The progress bar 920 may be a bar used to provide the user's activity space and main occupied space within a specific time period. The progress bar 920 may include multiple time period items corresponding to multiple time periods.

[0187] When the time period item 921 is selected on the progress bar 920, the processor 180 of the artificial intelligence device 100 can identify the user's activity space 910 and the main occupied space 911, 913 on the activity radius screen 900-2.

[0188] Figure 10 This is a diagram illustrating the configuration of a spatial understanding system according to an embodiment of the present disclosure.

[0189] refer to Figure 10 The spatial understanding system 1000 may include a millimeter-wave sensor 1001, a cloud server 1030, and an AI server 200. The spatial understanding system 1000 may include an AI device 100 instead of an AI server 200.

[0190] The millimeter-wave sensor 1001 can emit electromagnetic waves within the detection area 1010 and obtain the user's position 1011 by detecting the electromagnetic waves reflected from the user.

[0191] The millimeter-wave sensor 1001 can send location data corresponding to the obtained user location 1011 to the cloud server 1030. The location data can be represented as coordinates, such as (x, y).

[0192] Cloud server 1030 can send user location data to AI server 200 or AI device 100.

[0193] Cloud server 1030 can be a server used to manage one or more home appliances within sensing area 1010. Cloud server 1030 can be included in AI server 200 or AI device 100.

[0194] AI device 100 or AI server 200 can obtain an accumulated location dataset based on the received user location data. AI device 100 or AI server 200 can store the user's location data.

[0195] AI device 100 or AI server 200 can use a spatial understanding engine to obtain the user's activity space based on a cumulative location dataset. The spatial understanding engine can be an engine that estimates the user's activity space based on polygon approximation algorithms and clustering of the cumulative location dataset. The spatial understanding engine can be included in the processor 180 of AI device 100 or the processor 260 of AI server 200.

[0196] AI device 100 or AI server 200 can generate clustered maps by clustering the accumulated location dataset.

[0197] AI device 100 or AI server 200 can obtain the shape of the user's activity space from a clustered map generated using a polygon approximation algorithm.

[0198] AI device 100 or AI server 200 can obtain activity space information based on the shape of the obtained user activity space.

[0199] AI device 100 or AI server 200 can store the acquired activity space information.

[0200] AI device 100 or AI server 200 can send the obtained activity space information to cloud server 1030. The obtained activity space information can be used to effectively control household appliances within sensing area 1010.

[0201] Figure 11 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0202] The processor 180 of the artificial intelligence device 100 can acquire the user's location data and events of home appliances (S1101).

[0203] The processor 180 can receive user location data from millimeter-wave sensors.

[0204] The processor 180 can receive events from the home appliance or the cloud server 1030. These events can indicate changes in the operational state of the home appliance. For example, an event could be any one of the following: an open event indicating the opening of a door, a close event indicating the closing of a door, an open event indicating the turning on of a door, or a close event indicating the turning off of a door.

[0205] The event occurrence point of a household appliance can be the event acquisition point of the household appliance.

[0206] The processor 180 can obtain the occurrence of events related to household appliances and the time when the events occurred.

[0207] The processor 180 can determine whether there is an intention to use the home appliance based on the location data prior to the occurrence of the event (S1103).

[0208] When an event is received from a home appliance, the processor 180 can determine whether there is an intention to use the home appliance based on user location data collected over a certain period of time prior to the receipt of the event. This certain period of time could be 3 seconds, but this is just an example.

[0209] Processor 180 can track user location based on user location data collected over a period of time prior to the occurrence of an event involving a home appliance. Processor 180 can then determine the user's movement path based on the user's location tracking.

[0210] In one embodiment, if the tracked user movement path matches a preset pattern, the processor 180 can determine that there is an intention to use a household appliance. The preset pattern could be a straight-line pattern, but this is just an example.

[0211] In another embodiment, if the tracking distance of the user's movement path is greater than a certain distance, the processor 180 can determine that there is an intention to use the household appliance.

[0212] When it is determined that there is an intention to use a home appliance, the processor 180 can calculate the average location based on the user location data corresponding to the time of occurrence of the home appliance event, and obtain the calculated average as the first center coordinate (S1105).

[0213] Processor 180 can calculate the average coordinates of the user location data corresponding to the time of occurrence of the event involving the home appliance. Processor 180 can obtain the average coordinates as the first center coordinates of the home appliance.

[0214] Processor 180 can obtain the second center coordinates (S1107).

[0215] The processor 180 of the artificial intelligence device 100 can calculate the average distance between the position of the millimeter-wave sensor and the user's position. The processor 180 can remove user positions whose distance from the millimeter-wave sensor position is greater than the average distance used to calculate the first center coordinates. After removal, the processor 180 can recalculate the average coordinate values ​​of the remaining positions to obtain a second center coordinate.

[0216] The processor 180 can estimate the location of the household appliance from the obtained second center coordinates (S1109).

[0217] The processor 180 can obtain the location of multiple household appliances in the same manner as described above.

[0218] In one embodiment, the processor 180 can place the acquired position of each household appliance within the detection area of ​​the millimeter-wave sensor.

[0219] Figure 12A and Figure 12B This is a diagram showing the link between user location data and events related to home appliances.

[0220] refer to Figure 12A The artificial intelligence device 100 can identify a first location distribution 1210 within the sensing area 1200 based on real-time collected user location data. The first location distribution 1210 can be a heat map based on accumulated user location data.

[0221] The artificial intelligence device 100 can obtain a first location distribution 1210 based on location data collected when the air purifier is turned on. The artificial intelligence device 100 can use the first location distribution 1210 to estimate the location of the air purifier. The first location distribution 1210 may include a set of user location data collected when the air purifier is turned on.

[0222] The artificial intelligence device 100 can sequentially calculate the first center coordinates and the second center coordinates through the first position distribution 1210, and obtain the second center coordinates as the position of the air purifier.

[0223] refer to Figure 12B The artificial intelligence device 100 can identify a second location distribution 1230 within the sensing area 1200 based on real-time collected user location data. The second location distribution 1230 can be a heat map based on accumulated user location data.

[0224] Artificial intelligence device 100 can obtain a second location distribution 1230 based on location data collected when the refrigerator door is opened. Artificial intelligence device 100 can use the second location distribution 1230 to estimate the location of the refrigerator. The second location distribution 1230 may include a set of user location data collected when the refrigerator door is opened.

[0225] Artificial intelligence device 100 can sequentially calculate the first center coordinates and the second center coordinates through the second position distribution 1230, and obtain the second center coordinates as the position of the refrigerator.

[0226] After estimating the location of the air purifier, the AI ​​device 100 can update the relative position of the refrigerator within the detection area.

[0227] Figure 13 This is a diagram illustrating an example of identifying the estimated location of a household appliance according to an embodiment of the present disclosure.

[0228] The processor 180 of the artificial intelligence device 100 can display the location estimation screen 1300 of home appliances on the display 151.

[0229] The home appliance position estimation screen 1300 may include the position 1301 of the millimeter-wave sensor, the detection area 1310 of the millimeter-wave sensor, the position 1311 of the first home appliance, and the position 1313 of the second home appliance.

[0230] Sensing area 1310 can correspond to Figure 12A and Figure 12B The sensing area is 1200.

[0231] The location of the first household appliance, 1311, can be represented as... Figure 12A The location of the air purifier and the location of the second household appliance 1313 can be indicated. Figure 12B The location of the refrigerator in the picture.

[0232] According to embodiments of this disclosure, the relative position of a household appliance can be identified using millimeter-wave sensors and events from the appliance.

[0233] Thus, according to embodiments of this disclosure, the position and relative position of a household appliance can be estimated using millimeter-wave sensors and events from the appliance. The estimated position and relative position of the appliance can be used for effective placement and control.

[0234] Additionally, since no separate shooting equipment such as a camera is required, user privacy can be protected.

[0235] Figure 14 This is a diagram illustrating a position estimation system according to an embodiment of the present disclosure.

[0236] Figure 14 It can be a location estimation system 1400 that estimates the location of household appliances.

[0237] The location estimation system 1400 may include a millimeter-wave sensor 1101, a home appliance 1401, a cloud server 1030, and an AI server 200. The location estimation system 1400 may also include an AI device 100.

[0238] The millimeter-wave sensor 1101 can collect user location data and send the collected location data to the cloud server 1030.

[0239] Home appliance 1401 can detect the occurrence of an event and send information about the detected event to cloud server 1030. The information about the event may include at least one of the type of operating state of home appliance 1401 or the time when the event occurred.

[0240] The cloud server 1030 can send the user's location data and events from the home appliance 1401 to the AI ​​server 200 or the AI ​​device 100.

[0241] AI server 200 or AI device 100 can determine whether there is an intention to use the home appliance based on location data prior to the event occurring at the home appliance.

[0242] If it is determined that there is an intention to use a home appliance, the AI ​​server 200 or AI device 100 can calculate the average location based on the user location data corresponding to the time of the event of the home appliance, and obtain the calculated average as the first center coordinate.

[0243] AI server 200 or AI device 100 can calculate the average distance between the position of the millimeter-wave sensor and the user's position. AI server 200 or AI device 100 can remove user positions that are farther from the millimeter-wave sensor than the average distance used to calculate the first center coordinates. After removal, AI server 200 or AI device 100 can recalculate the average coordinates of the remaining positions to obtain the second center coordinates.

[0244] AI server 200 or AI device 100 can estimate the second center coordinates as the location of the home appliance.

[0245] Figure 15 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0246] Figure 15 This could be an implementation that identifies the location of home appliances and the user's activity space within a detection area based on user location data and events from home appliances.

[0247] refer to Figure 15 The processor 180 of the artificial intelligence device 100 can obtain the user's location data and events of home appliances (S1501).

[0248] The processor 180 can receive user location data from millimeter-wave sensors.

[0249] The processor 180 can receive events from home appliances or cloud servers 1030.

[0250] The processor 180 can obtain the user's activity space based on the user's location data (S1503).

[0251] The processor 180 can obtain the user's activity space within the detection area of ​​the millimeter-wave sensor. The process of obtaining the user's activity space based on the user's location data is as follows: Figure 3 and Figure 4 The description of the embodiments is replaced.

[0252] The processor 180 can obtain the location of the home appliance based on the user's location data and events of the home appliance (S1505).

[0253] The processor 180 can determine the location of home appliances within the detection area based on user location data and events from the home appliances. The process of determining the location of home appliances based on user location data and events from the home appliances will be handled by… Figure 11 The description of the embodiments is replaced.

[0254] The processor 180 can identify the user's activity space and the location of household appliances within the detection area (S1507).

[0255] The processor 180 can display a detection area on the display 151 that identifies the user's activity space and the location of home appliances. The processor 180 can display the detection area based on the execution of a home appliance management application.

[0256] Figure 16 This is a diagram illustrating a screen that provides a user's activity space, main occupied space, and the location of home appliances according to embodiments of the present disclosure.

[0257] refer to Figure 16 The processor 180 of the artificial intelligence device 100 can display a service screen 1600 on the display 151 according to the received command.

[0258] The service screen 1600 may include a user’s activity space 1630 identified on the detection area 1610 of the millimeter-wave sensor, main occupancy spaces 1631 and 1633 included in the activity space 1630, and the location of the first household appliance 1651 and the location of the second household appliance 1653.

[0259] The service screen 1600 may also include the location (P) of the millimeter-wave sensor.

[0260] The service screen 1600 may also include the location of one or more pieces of furniture 1671.

[0261] A different service screen 1600 can be provided for each user. This is because location data can be collected differently for each user. The artificial intelligence device 100 can display a first service screen corresponding to the first user on the display 151 in response to a request from the first user, and a second service screen corresponding to the second user on the display 151 in response to a request from the second user.

[0262] Therefore, it is possible to perform home appliance controls optimized for each user.

[0263] The processor 180 can provide recommended locations for home appliances based on the user's activity space 1630, main occupied space 1631, 1633 and the location of home appliances 1651, 1653.

[0264] For example, processor 180 can display placement guidelines on display 151, which allow the air purifier to be placed within the main occupancy spaces 1631, 1633.

[0265] Figure 17 This is a flowchart illustrating a method of operating an artificial intelligence device according to another embodiment of the present disclosure.

[0266] It is possible Figure 3 Execute after step S305 Figure 17 Examples of implementations.

[0267] The processor 180 of the artificial intelligence device 100 can control the operation of home appliances based on the obtained user activity space (S1701).

[0268] The home appliance could be any of a robotic vacuum cleaner, an air conditioner, or an air purifier, but this is just an example.

[0269] The processor 180 can control the operation of home appliances based on the user's activity space and one or more main occupancy spaces included in the activity space.

[0270] First, when the household appliance is a robotic vacuum cleaner, embodiments of controlling the robotic vacuum cleaner based on the user's activity space or main occupied space will be described.

[0271] In one embodiment, processor 180 can control the operation of a robotic vacuum cleaner to first clean the user's living space. Processor 180 can send information about the detection area, the living space identified within the detection area, the main occupancy space, and the arrangement of furniture to the robotic vacuum cleaner.

[0272] In another embodiment, the processor 180 can set the cleaning path of the robotic vacuum cleaner to first clean the main occupied space within the activity space.

[0273] The processor 180 can send cleaning control signals, including coordinate information of the main occupied space, to the robotic vacuum cleaner via the communication interface 110. The robotic vacuum cleaner can then clean the main occupied space based on the cleaning control signals received from the artificial intelligence device 100.

[0274] The processor 180 can send a cleaning control signal to the robotic vacuum cleaner after detecting the user's last position within the detection area. This is to ensure that cleaning is performed after the user leaves the detection area.

[0275] In one embodiment, processor 180 can set a cleaning path by prioritizing the primary occupancy space. Processor 180 can configure the cleaning path of the robotic vacuum cleaner to first clean the space with the highest location data frequency within the primary occupancy space, and then clean the spaces with lower frequency. Processor 180 can send cleaning control signals, including the configured cleaning path, to the robotic vacuum cleaner via communication interface 110.

[0276] In another embodiment, the processor 180 can set different cleaning modes for each major occupied space. The cleaning modes can include a powerful cleaning mode and a normal cleaning mode. The powerful cleaning mode can require more cleaning intensity and time than the normal cleaning mode.

[0277] The cleaning mode can vary based on several cleaning factors. These factors may include at least one of the following: cleaning time, motor suction power, brush rotation speed, pressure applied to the cleaning mop, steam jet volume, water jet volume, or the number of times cleaning is repeated in a specific section.

[0278] Specifically, a powerful cleaning mode can be a mode in which at least one of the multiple cleaning factors is greater than that of a normal cleaning mode.

[0279] In addition to the normal and intensive cleaning modes, cleaning modes can be further subdivided into more modes. Each of these multiple cleaning modes can have a different size or intensity of at least one of several cleaning factors.

[0280] The processor 180 can determine the size or intensity of each of the multiple cleaning factors that determine the cleaning mode differently, based on the frequency of the location data.

[0281] For example, processor 180 can control a robotic vacuum cleaner so that, as the frequency of location data increases, the magnitude or intensity of multiple cleaning factors that determine the cleaning pattern increases.

[0282] The processor 180 can control the robotic vacuum cleaner so that as the frequency of location data decreases, the magnitude or intensity of multiple cleaning factors that determine the cleaning pattern decreases.

[0283] If the frequency of the location data is greater than or equal to a preset frequency, the processor 180 can set the cleaning mode for the main occupied space to a powerful cleaning mode. If the frequency of the location data is less than the preset frequency, the processor 180 can set the cleaning mode for the main occupied space to a normal cleaning mode.

[0284] The processor 180 can send cleaning control signals to the robotic vacuum cleaner via the communication interface 110. These cleaning control signals include cleaning modes set for each major occupied space. Therefore, cleaning of the spaces mainly occupied by the user can be performed intensively.

[0285] Figure 18 and Figure 19 This is a diagram illustrating an example of controlling the cleaning path of a robotic vacuum cleaner differently based on the primary occupancy space of each of the first user and the second user, according to embodiments of the present disclosure.

[0286] Figure 18 This may be a diagram illustrating a first user service screen 1800 including a detection area 1810 based on position (P) detection by a millimeter-wave sensor. The first user service screen 1800 may be referred to as a first user map. The artificial intelligence device 100 may display the first user service screen 1800 corresponding to the first user on a display 151.

[0287] The artificial intelligence device 100 can identify the first user's first activity space 1830 and the main occupied spaces 1831 and 1833 within the first activity space 1830 based on the location data corresponding to the first user. The first activity space 1830 and the main occupied spaces 1831 and 1833 within the first activity space 1830 can be represented in the form of a heat map.

[0288] Artificial intelligence device 100 can send a first cleaning control signal to a robotic vacuum cleaner to perform cleaning along a first cleaning path (Path1) that starts from a first main occupied space 1831 and moves to a second main occupied space 1833.

[0289] The robotic vacuum cleaner can perform cleaning along a first cleaning path (Path1) based on a first cleaning control signal.

[0290] Figure 19 It can be a second user service screen 1900, which includes a screen based on the location of the user service screen. Figure 18 The detection area 1810 is detected by the position (P) of the millimeter-wave sensor at the same location. The second user service screen 1900 can be referred to as the second user map. The artificial intelligence device 100 can display the second user service screen 1900 corresponding to the second user on the display 151.

[0291] The artificial intelligence device 100 can identify the second user's second activity space 1930 and the main occupied spaces 1931 and 1933 within the second activity space 1930 based on location data corresponding to the second user. The second activity space 1930 and the main occupied spaces 1931 and 1933 within the second activity space 1930 can be represented in the form of a heat map.

[0292] Figure 18 The shape and size of the first activity space 1830 can be compared with Figure 19 The second activity space of 1930 differed in shape and size.

[0293] Figure 18 The shape, size, and position of the main occupiers 1831 and 1833 can be compared with... Figure 19 The main space occupied in 1931 and 1933 are different.

[0294] Artificial intelligence device 100 can send a second cleaning control signal to a robotic vacuum cleaner to perform cleaning along a second cleaning path (Path2) that starts from the third main occupancy space 1931 and moves to the fourth main occupancy space 1933.

[0295] The robotic vacuum cleaner can perform cleaning along the second cleaning path (Path2) according to the second cleaning control signal.

[0296] Thus, according to embodiments of this disclosure, the operation of home appliances can be controlled differently based on each user's primary occupied space. Therefore, home appliance control can be performed in a personalized manner, thereby improving user convenience.

[0297] Next, an embodiment of controlling the air conditioner based on the user's activity space or main occupied space will be described.

[0298] In one embodiment, the processor 180 can obtain the user's main occupied space for each time period and control the operation of the air conditioner differently for each time period.

[0299] For example, processor 180 can extract a first primary occupancy space representing the region with the highest frequency of location data during the lunch break and a second primary occupancy space representing the region with the highest frequency of location data during the evening break. The first and second primary occupancy spaces can be obtained based on location data collected over a two-week period, but two weeks is only an example period. Location data can be collected from the time a user enters the detection area until the user leaves the detection area.

[0300] The processor 180 can control the air conditioner to lower the temperature of the primary occupied space to a preset temperature before lunchtime.

[0301] Additionally, the processor 180 can control the air conditioner to lower the temperature of the second main occupied space to a preset temperature before the evening hours arrive.

[0302] In another embodiment, the processor 180 can identify the primary occupancy space matched with each user and control the air conditioner to adjust the temperature of the primary occupancy space matched with that user.

[0303] For example, when the processor 180 detects that the first user will enter the detection area after a certain period of time, the processor 180 can control the air conditioner to lower the temperature of the first main occupied space matched with the first user to a preset temperature.

[0304] When the processor 180 detects that a second user will enter the detection area after a certain period of time, the processor 180 can control the air conditioner to lower the temperature of the second main occupied space matched with the second user to a preset temperature.

[0305] Figure 20A and Figure 20B This is a diagram illustrating an example of extracting the main occupied space for each time period and controlling the cooling of the extracted main occupied space according to an embodiment of the present disclosure.

[0306] Figure 20A This can be a diagram illustrating a detection area 2010 including position P detection based on a millimeter-wave sensor, of a user service screen 2000. The user service screen 2000 can be referred to as a user map. The artificial intelligence device 100 can display the user service screen 2000 on a display 151.

[0307] refer to Figure 20AThe artificial intelligence device 100 can identify the user's activity space 2030 and a first primary occupied space 2031 within the activity space 2030 based on the user's location data. The first primary occupied space 2031 can be a space based on location data obtained within a first time period of two weeks. The first primary occupied space 2031 can be a set of unit areas where the frequency of user location data is greater than or equal to a preset frequency. The first time period can be from 12:00 to 14:00.

[0308] The activity space 2030 and the first primary occupied space 2031 within the activity space 2030 can be represented in the form of a heat map.

[0309] The artificial intelligence device 100 can send a first cooling control signal to the air conditioner to reduce the temperature of the first main occupied space to a preset temperature before the arrival of a first time period. The first cooling control signal can be a signal that controls the air volume and fan speed of the air conditioner.

[0310] Therefore, before the first time period arrives, the temperature of the first main occupied space 2031 where the user mainly stays is reduced in advance, so that the user cannot feel the heat (for example, a pre-cooling operation can be performed).

[0311] refer to Figure 20B The artificial intelligence device 100 can identify the user's activity space 2030 and a second primary occupied space 2033 within the activity space 2030 based on the user's location data. The second primary occupied space 2033 can be a space based on location data obtained over a second time period of two weeks. The second primary occupied space 2033 can be a set of unit areas where the frequency of user location data is greater than or equal to a preset frequency. The second time period can be from 19:00 to 21:00.

[0312] The activity space 2030 and the second main occupied space 2033 within the activity space 2030 can be represented in the form of a heat map.

[0313] The artificial intelligence device 100 can send a second cooling control signal to the air conditioner to reduce the temperature of the second main occupied space to a preset temperature before the arrival of the second time period. The second cooling control signal can be a signal that controls the air volume and fan speed of the air conditioner.

[0314] Therefore, before the second time period arrives, the temperature of the second main occupied space 2033 where the user mainly stays is lowered in advance, so that the user cannot feel the heat.

[0315] Figure 21 This is a diagram illustrating the configuration of an artificial intelligence cloud device according to another embodiment of the present disclosure.

[0316] The artificial intelligence cloud device 2100 may include a location database 2110, a heat map engine 2120, a results database 2130, and an engine processor 2150.

[0317] Location database 2110 can store user location data collected by millimeter-wave sensor 1101. One or more millimeter-wave sensors 1101 can be provided. In this case, each millimeter-wave sensor can send its location data (coordinate information) along with an ID that identifies it to the artificial intelligence cloud device 2100.

[0318] Location database 2110 can store cumulative location datasets.

[0319] The Heatmap Engine 2120 can generate heatmaps representing the location distribution of users based on accumulated location datasets. The Heatmap Engine 2120 can generate multiple heatmaps for each time period for a single user.

[0320] The Heatmap Engine 2120 can generate a user's activity space and main occupied space based on a heatmap. The process of generating a user's activity space and main occupied space based on a heatmap is as follows... Figure 3 and Figure 4 The embodiments are the same.

[0321] The heatmap engine 2120 can periodically generate heatmaps, activity spaces, and main occupied spaces. When the accumulated volume of location data exceeds a certain amount, the heatmap engine 2120 can generate heatmaps, activity spaces, and main occupied spaces.

[0322] The Heatmap Engine 2120 can use millimeter-wave sensors installed in each of multiple spaces to obtain heatmaps, user activity spaces, and main occupied spaces for each space.

[0323] The resulting database 2130 can store the generated heat map, the user's activity space, and the main space occupied.

[0324] Engine processor 2150 typically controls the operation of AI cloud device 2100. Engine processor 2150 can also control the operation of heat map engine 2120 and household appliances such as robotic vacuum cleaner 2101 and air conditioner 2103.

[0325] The engine processor 2150 can send information about the heat map, the user's activity space, and the main occupied space stored in the results database 2130 to the robot vacuum cleaner 2101 and the air conditioner 2103.

[0326] The engine processor 2150 can send control signals based on at least one of the information stored in the results database 2130 about a heat map, the user’s activity space, or the main occupied space to the robot vacuum cleaner 2101 or the air conditioner 2103.

[0327] AI cloud device 2100 can be Figure 1 100 or more AI devices Figure 2 An example of an AI server 200.

[0328] When AI cloud device 2100 is Figure 1 In the case of an AI device 100, a location database 2110 and a results database 2130 may be included in a memory 170, and a heat map engine 2120 and an engine processor 2150 may be included in a processor 180.

[0329] When the AI ​​cloud device 2100 is Figure 2 In the case of an AI server 200, a location database 2110 and a results database 2130 may be included in a memory 230, and a heat map engine 2120 and an engine processor 2150 may be included in a processor 260.

[0330] Figure 22 This is a timing diagram illustrating a method of operating a system according to an embodiment of the present disclosure.

[0331] The engine processor 2150 can send a request to collect user location data to the millimeter-wave sensor 1101 (S2201).

[0332] The engine processor 2150 can send requests for collecting user location data and information about the collection cycle of location data to the millimeter-wave sensor 1101.

[0333] The engine processor 2150 can communicate with the millimeter-wave sensor 1101 via a communication interface.

[0334] The millimeter-wave sensor 1101 can collect the user's location data in response to a request and send the collected location data to the location database 2110 (S2203).

[0335] The millimeter-wave sensor 1101 can collect its own identifier and the user's location data.

[0336] The location database 2110 can accumulate location data received from the millimeter-wave sensor 1101, obtain a location dataset, and send the obtained location dataset to the heat map engine 2120 (S2205).

[0337] The heat map engine 2120 can request location datasets collected for each time period from the location database 2110.

[0338] The heat map engine 2120 can generate at least one of a heat map, a user's activity space, and a main occupied space based on a location dataset (S2207), and send the result information including the heat map, the user's activity space, and the main occupied space to the result database 2130 (S2209).

[0339] The heat map engine 2120 can receive control commands from the engine processor 2150 indicating the heat map generation cycle, and generate heat maps periodically according to the received control commands.

[0340] The results database 2130 can store heat maps, user activity spaces, and main occupied spaces. The results database 2130 can store heat maps, user activity spaces, and main occupied spaces for each user. The results database 2130 can store heat maps, user activity spaces, and main occupied spaces for each time period.

[0341] The engine processor 2150 can send a result information request to the result database 2130 (S2211) and receive result information from the result database 2130 in response to the result information request (S2213).

[0342] The engine processor 2150 can send the result information and control signals for controlling the operation of the home appliance 2200 to the home appliance 2200 (S2215).

[0343] Control signals can be signals generated based on result information.

[0344] The household appliance 2200 can use the result information to perform operations based on the control signal (S2217).

[0345] Home appliances 2200 can be Figure 21 Robotic vacuum cleaner 2101 or air conditioner 2103.

[0346] An electronic device 100 according to an embodiment of the present disclosure may include: a memory 170 configured to store user location data; and at least one processor 180 configured to: obtain a cumulative location dataset based on the location data, generate a heat map representing the user's location distribution based on the cumulative location dataset, and obtain the user's activity space based on the generated heat map.

[0347] The activity space may include one or more primary occupied spaces, and the one or more primary occupied spaces represent areas where the cumulative frequency of location data is greater than a preset frequency.

[0348] One or more primary footprints can be different for each of the multiple users.

[0349] One or more primary occupancy spaces can be different for each time period.

[0350] Heat maps can be personalized maps based on location data.

[0351] The electronic device 100 may also include a display 151, and at least one processor 180 is configured to display on the display 151 the detection area and activity space of the sensor that has obtained the position data.

[0352] The activity space may include one or more primary occupied spaces, and the one or more primary occupied spaces represent areas where the cumulative frequency of location data is greater than a preset frequency.

[0353] One or more primary storage spaces can be displayed differently for each user or time period.

[0354] At least one processor 180 can cluster the accumulated location dataset to generate a cluster map, and use a polygon approximation algorithm to obtain the shape of the activity space from the cluster map.

[0355] At least one processor 180 can obtain activity space information based on the shape of the activity space, the activity space information including at least one of the area of ​​the activity space or the angle formed between the activity space and the sensor that collects the location data.

[0356] At least one processor 180 can use the DBSCAN (Density-based Noise Applied Spatial Clustering) technique to extract multiple cluster regions from a cumulative location dataset, identify high-density regions within the multiple cluster regions, and generate a cluster map based on the identified high-density regions.

[0357] At least one processor 180 can obtain events from home appliances and determine the location of home appliances based on user location data collected when the events are obtained.

[0358] Events involving household appliances can indicate changes in their operating status.

[0359] The electronic device 100 may also include a display 151, on which at least one processor 180 can display the location of the activity space and household appliances.

[0360] The electronic device 100 may also include a communication interface 110, through which at least one processor 180 can receive position data from a millimeter-wave sensor.

[0361] The above disclosure can be implemented as computer-readable code on a program recording medium. Computer-readable media include all types of recording devices that store data readable by a computer system. Examples of computer-readable media are HDDs (hard disk drives), SSDs (solid-state drives), SDDs (silicon disk drives), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. Additionally, the computer may include a processor 180 for an artificial intelligence device.

Claims

1. An electronic device, comprising: The memory is configured to store the user's location data; as well as At least one processor is configured as follows: A cumulative location dataset is obtained based on the location data. The cumulative location dataset includes information about the number of times the user was detected at one or more locations within space. A heat map representing the location distribution of the user is generated based on the accumulated location dataset, and The user's activity space is obtained based on the heat map, and the activity space is the area within the space.

2. The electronic device of claim 1, further comprising a display configured to display an image. in, The at least one processor is further configured to display on the display the detection area of ​​the sensor configured to obtain the location data and the activity space.

3. The electronic device according to claim 2, wherein, The activity space includes one or more main occupied spaces, and each of the one or more main occupied spaces represents a region where the cumulative frequency of the location data is greater than a preset frequency.

4. The electronic device according to claim 3, wherein, The one or more main occupied spaces are displayed differently for multiple users or multiple time periods.

5. The electronic device according to claim 1, wherein, The at least one processor is further configured to: Cluster the accumulated location dataset to generate a clustering map, and The shape of the activity space is obtained from the clustering map based on a polygon approximation algorithm.

6. The electronic device according to claim 5, wherein, The at least one processor is further configured to: The activity space information is obtained, including at least one of the area of ​​the activity space or the angle formed between the activity space and the sensor, wherein the sensor is configured to collect the position data based on the shape of the activity space.

7. The electronic device according to claim 6, wherein, The at least one processor is configured to: Based on density-based noise spatial clustering (DBSCAN) technology, multiple clustering regions are extracted from the accumulated location dataset. Identify high-density regions within the multiple clustering regions, and The cluster map is generated based on the high-density regions.

8. The electronic device of claim 1, further comprising a communication interface configured to receive sensor data, in, The at least one processor is also configured to receive the location data from the millimeter-wave sensor via the communication interface.

9. A method for controlling an electronic device, the method comprising: The user's location data is stored in the memory of the electronic device; A cumulative location dataset is obtained based on the location data via a processor in the electronic device. The cumulative location dataset includes information about the number of times the user was detected at one or more locations in space. The processor generates a heat map representing the user's location distribution based on the accumulated location dataset. as well as The processor obtains the user's activity space based on the heat map, where the activity space is a region within the space.

10. A method for controlling an electronic device, the method comprising: The user's location data in space is obtained via a millimeter-wave sensor in the electronic device; The location of the multiple home appliances in the space is determined by the processor in the electronic device based on the user's location data and events corresponding to the multiple home appliances, each of the events corresponding to a change in the operating state of one of the multiple home appliances; as well as Commands are sent via the processor to at least one of the plurality of home appliances to perform functions based on the user's location data.

Images