Method for managing integrated map for indoor space and electronic device performing same

The method allows electronic devices to exchange and update integrated maps within IoT environments, enabling efficient task coordination and data sharing through communication and sensor units, leveraging neural networks for improved performance in indoor spaces.

WO2024237455A1PCT designated stage expired Publication Date: 2024-11-21SAMSUNG ELECTRONICS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2024/003451
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-12
Filing Date
2024-03-19
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

In IoT environments, there is a challenge in managing integrated maps of indoor spaces effectively, particularly in coordinating tasks among various electronic devices without direct access to comprehensive information, leading to inefficiencies in task performance and data exchange.

Method used

A method where electronic devices exchange and update integrated maps using a communication unit, sensor unit, and processor, allowing them to adjust tasks based on received information and share surrounding data to create a unified map of the indoor space, leveraging artificial neural networks for data processing and integration.

Benefits of technology

This approach enables efficient task coordination and data sharing among devices, enhancing their performance by utilizing information from other devices, thus overcoming the limitations of fixed locations and improving task execution in IoT environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024003451_21112024_PF_FP_ABST
    Figure KR2024003451_21112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided is a method performed by a first electronic device, the method comprising the steps of: receiving, from a third electronic device, an integrated map for an indoor space including the first electronic device and at least one second electronic device; on the basis of the integrated map, adjusting a task set to be performed by the first electronic device; collecting first surrounding information related to the indoor space while performing the adjusted task; and transmitting the collected first surrounding information to the third electronic device (300) in order to update the integrated map.
Need to check novelty before this filing date? Find Prior Art

Description

Method for managing an integrated map of an indoor space and an electronic device for performing the same

[0001] The present disclosure relates to a method for managing an integrated map for an indoor space and an electronic device for performing the same.

[0002] The Internet is evolving from a human-centric network where humans create and consume information to the Internet of Things (IoT), where information is exchanged and processed among distributed components such as objects. The Internet of Everything (IoE), which combines IoT technologies with big data processing technologies such as cloud servers, is also emerging. IoT can be applied to areas such as smart appliances, smart homes, smart buildings, and smart cities through the convergence and integration of existing IT (information technology) technologies with various industries.

[0003] In an IoT environment, interconnected electronic devices collect, generate, analyze, or process data, and share this data with each other to utilize it for their respective tasks. With recent rapid advancements in artificial intelligence, various types of electronic devices utilizing artificial neural networks are being developed. Consequently, interest in connecting various types of electronic devices in the IoT environment is growing.

[0004] A method performed by a first electronic device according to one embodiment of the present disclosure may include receiving, from a third electronic device, an integrated map of an indoor space including the first electronic device and at least one second electronic device.

[0005] In one embodiment, the method may include adjusting a task set to be performed by the first electronic device based on the integrated map.

[0006] In one embodiment, the method may include collecting first ambient information about the indoor space while performing the adjusted task.

[0007] In one embodiment, the method may include a step of transmitting the collected first surrounding information to the third electronic device (300) for updating the integrated map.

[0008] A method performed by a third electronic device according to one embodiment of the present disclosure may include a step of receiving, from a first electronic device, first surrounding information about an indoor space collected by the first electronic device.

[0009] In one embodiment, the method may include updating an integrated map for the indoor space including the first electronic device and at least one second electronic device based on the received first surrounding information.

[0010] In one embodiment, the integrated map may include second surrounding information about the indoor space obtained by the third electronic device from the at least one second electronic device.

[0011] A first electronic device according to one embodiment of the present disclosure may include a communication unit, a sensor unit, a memory storing one or more instructions, and at least one processor executing the one or more instructions stored in the memory.

[0012] In one embodiment, the at least one processor may execute the one or more instructions to cause the first electronic device to receive, through the communication unit, an integrated map of an indoor space including the first electronic device and at least one second electronic device from a third electronic device.

[0013] In one embodiment, the at least one processor may cause the first electronic device to coordinate a task that the first electronic device is set to perform based on the integrated map by executing the one or more instructions.

[0014] In one embodiment, the at least one processor may cause the first electronic device to collect first ambient information about the indoor space while performing the adjusted task through the sensor unit by executing the one or more instructions.

[0015] In one embodiment, the at least one processor may cause the first electronic device to transmit the collected first surrounding information to the third electronic device through the communication unit for updating the integrated map by executing the one or more instructions.

[0016] A third electronic device according to one embodiment of the present disclosure may include a communication unit, a memory storing one or more instructions, and at least one processor executing the one or more instructions stored in the memory.

[0017] In one embodiment, the at least one processor may execute the one or more instructions to cause the third electronic device to receive, through the communication unit, first ambient information about the indoor space collected by the first electronic device from the first electronic device.

[0018] In one embodiment, the at least one processor may cause the third electronic device to update an integrated map of the indoor space including the first electronic device and at least one second electronic device based on the received first surrounding information by executing the one or more instructions.

[0019] In one embodiment, the integrated map may include second spatial information about the indoor space obtained by the third electronic device from the at least one second electronic device.

[0020] Figure 1 is a drawing for explaining an IoT system in an indoor space in which multiple electronic devices are connected to each other.

[0021] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure.

[0022] FIG. 3 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure.

[0023] FIG. 4 is a diagram illustrating a method for electronic devices according to one embodiment of the present disclosure to manage an integrated map for an indoor space.

[0024] FIG. 5 illustrates an example of an integrated map according to one embodiment of the present disclosure.

[0025] FIG. 6 is a diagram illustrating a method for a first electronic device to coordinate a task according to an embodiment of the present disclosure.

[0026] FIG. 7 is a diagram illustrating a method for a third electronic device to update an integrated map according to an embodiment of the present disclosure.

[0027] FIG. 8 is a diagram illustrating a method for electronic devices to manage an integrated map by reflecting a user request according to an embodiment of the present disclosure.

[0028] FIG. 9 is a diagram illustrating a method for a third electronic device to update an integrated map according to an embodiment of the present disclosure.

[0029] FIG. 10A is a diagram illustrating an example of electronic devices reflecting and utilizing a user request in an integrated map according to one embodiment of the present disclosure.

[0030] FIG. 10b is a diagram illustrating an example of electronic devices reflecting and utilizing a user request in an integrated map according to one embodiment of the present disclosure.

[0031] In this disclosure, the expression “at least one of a, b, and c” may refer to “a,” “b,” “c,” “a and b,” “a and c,” “b and c,” “all of a, b, and c,” or variations thereof.

[0032] The terms used in this disclosure are selected from widely used, current terms, taking into account the functions of the embodiments. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be understood simply as names, but rather based on their inherent meanings and the overall content of this disclosure.

[0033] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0034] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components in between.

[0035] Unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" are to be understood to include plural referents. Thus, for example, the description "a component surface" may also include reference to one or more such surfaces.

[0036] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art described herein.

[0037] In this disclosure, it should be understood that terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this disclosure, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0038] In this disclosure, components expressed as "units", "modules", etc. may be two or more components combined into one component, or one component may be divided into two or more components with more detailed functions. In addition, each component described below may additionally perform some or all of the functions performed by other components in addition to its own main function, and of course, some of the main functions performed by each component may be exclusively performed by other components.

[0039] All functions or operations described in this disclosure may be processed by a single processor or a combination of processors. A single processor or a combination of processors may be a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphics Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0040] In the present disclosure, functions related to artificial intelligence can be operated through a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors may process input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0041] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning may mean that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rule or artificial intelligence model set to perform a specific desired task (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0042] An AI model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the previous layer's computational results and the multiple weights. The multiple weights included in the multiple neural network layers may be optimized based on the AI ​​model's learning results. For example, the multiple weights may be updated during the learning process to reduce or minimize the loss or cost values ​​obtained from the AI ​​model. The artificial neural network may include a deep neural network (DNN: Deep Neural Network), for example, a graphic neural network (GNN), a graph convolution network (GCN), a graph attention network (GAT), a gated graph sequence neural network (GGS-NN), a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks (DQN), but is not limited to the examples described above.

[0043] In the present disclosure, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0044] In the present disclosure, it should be understood that the blocks and combinations of the flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory, or may be divided and stored in multiple different memories.

[0045] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0046] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0047] Figure 1 is a drawing for explaining an IoT system in an indoor space in which multiple electronic devices are connected to each other.

[0048] Various electronic devices included in an indoor space can perform their own tasks. For example, a robot vacuum cleaner (10), an air purifier (21), an air conditioner (22), a night light (23), a refrigerator (24), etc. can be included in the indoor space. The robot vacuum cleaner (10) can move to a living room, a bedroom, a kitchen, etc., and suck up dust from the floor. The air purifier (21) can be located in the living room and absorb pollutants contained in the surrounding air. The air conditioner (22) can be located in the living room and operate to maintain the temperature of the space at a temperature set by the user. The night light (23) can be located in the bedroom and turn the light on or off when the user is in the bedroom at night. The refrigerator (24) can be located in the kitchen and control the internal temperature to prevent food stored therein from spoiling.

[0049] Electronic devices may include various sensors to obtain information necessary for performing tasks. For example, a robot vacuum cleaner (10) may include a sensor for detecting objects in the surroundings. An air purifier (21) may include a sensor for detecting the level of pollution in the surrounding air, and an air conditioner (22) may include a sensor for detecting the temperature of an indoor space, a sensor for measuring the level of pollution in the surrounding air, etc. A night light (23) may include a sensor for detecting the presence of people in the surroundings and / or a sensor for detecting the brightness of the surroundings, and a refrigerator (24) may include a sensor for detecting the temperature of the internal space.

[0050] Electronic devices can exchange data with the server (30) via wired or wireless communication. The data exchanged between the electronic devices and the server (30) may include physical information such as the type, location, and size of each electronic device, information regarding the task of each electronic device, and surrounding information collected by each electronic device using sensors. For example, a robot cleaner (10) may exchange physical information such as its current location, task-related information such as cleaning time, cleaning area, and cleaning order, and surrounding information collected by sensors such as the user's location and the position of a table with the server (30). An air purifier (21) may exchange physical information such as its current location, task-related information such as operating time and operating mode, and surrounding information collected by sensors such as the pollution level of the surrounding air with the server (30). An air conditioner (22) may exchange physical information such as its current location, task-related information such as operating time, set temperature, and operating mode, and surrounding information collected by sensors such as the temperature of an indoor space with the server (30). The night light (23) can exchange physical information such as its current location, task-related information such as the user's bedtime, and surrounding information collected using sensors such as the user's location with the server (30). The server (30) can create an integrated map of the indoor space based on data received from electronic devices.

[0051] The integrated map may be implemented as a graph including nodes representing electronic devices and / or objects included in an indoor space, and edges connecting the nodes. Each edge may include a weight vector determined based on the relationship between electronic devices and / or objects corresponding to two connected nodes. Here, the relationship between the electronic devices and / or objects may include at least one of physical information of each electronic device and / or object (e.g., type, location, size, etc.), information regarding tasks of the electronic devices (e.g., operation mode, operation time, etc.), surrounding information collected by the electronic devices using sensors (e.g., pollution level of the surrounding air, etc.), or information regarding the interrelationships between the electronic devices and / or objects. The weight vector may be composed of values ​​or vectors corresponding to each of the above-described pieces of information.

[0052] For example, the correlation between electronic devices and / or objects may refer to the degree to which the physical distance between the electronic devices and / or objects is close within an indoor space, and / or the degree to which the electronic devices are functionally similar, etc. As an example, the closer two electronic devices are physically, the greater the value corresponding to the correlation between the two electronic devices may be determined. As another example, the value corresponding to the correlation between two electronic devices may be determined as a value between 0 and 1 depending on the degree to which the physical distance between the two electronic devices is close and / or the degree to which the two electronic devices are functionally similar. The correlation between electronic devices and / or objects may be determined based on physical information of each electronic device and / or object, information about the tasks of the electronic devices, and surrounding information measured by the electronic devices using sensors.

[0053] The server (30) can transmit an integrated map to electronic devices so that the electronic devices can utilize it when performing tasks. For example, a robot vacuum cleaner (10) can receive an integrated map from the server (30) and determine a cleaning path based on information contained in the integrated map.

[0054] The server (30) can receive information about indoor spaces from electronic devices and update an integrated map using the received information. Here, the information about the indoor space may include information about the tasks of each electronic device, surrounding information collected by each electronic device using sensors, etc. For example, the surrounding information collected by the electronic device using sensors may include information about the structure of the indoor space, such as the location, area, shape, and distance of a living room, bedroom, or kitchen. In addition, the information collected by the electronic device using sensors may include information about objects (e.g., people, tables, etc.) located in the indoor space, such as the presence of a person in the bedroom or the presence of a table in the living room. In addition, the information collected by the electronic device using sensors may include information about the environment of the indoor space, such as changes in brightness in the bedroom due to day-night changes or lighting, air pollution in the living room, and the presence of a lot of cat hair on the living room floor.

[0055] For example, the server (30) can receive surrounding information (e.g., a lot of cat hair on the living room floor, etc.) measured by the robot cleaner (10) while moving through the indoor space, and update the integrated map based on the information. The server (30) can receive indoor pollution information measured by the air purifier (21) and the air conditioner (22), and update the integrated map based on the information. The server (30) can receive information about the time the TV is turned on from the TV (not shown), and update the integrated map based on the information. At this time, updating the integrated map can include deleting an existing node or adding a new node, modifying a weight vector included in an existing edge, and / or calculating a weight vector included in a new edge.

[0056] The server (30) can update the integrated map using an artificial neural network. For example, GNN (Graphic Neural Network), GCN (Graph Convolution Network), GAT (Graph Attention Network), GGS-NN (Gated Graph Sequence Neural Network), etc., which are easy to process graph-type data, can be used as the artificial neural network for updating the integrated map, but are not limited thereto, and CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or DQN (Deep Q-Network) can also be used.

[0057] Electronic devices can receive an updated integrated map from the server (30) and adjust tasks based on the updated integrated map. The electronic devices can extract information related to their tasks from the weight vectors of edges connected to nodes representing themselves in the updated integrated map, and can change or delete preset tasks or add new tasks based on the extracted information. Rules regarding how to adjust tasks based on information extracted from the updated integrated map may be predefined for each electronic device. For example, an air purifier (21) can receive an updated integrated map from the server (30) that includes information collected by a robot cleaner (10) while cleaning an indoor space (e.g., a lot of cat hair on the living room floor) and activate a pet care function. The robot cleaner (10) can receive an updated integrated map that includes information on the indoor pollution level measured by the air purifier (21) and the air conditioner (22), and determine a cleaning path or adjust the suction strength of dust. The robot vacuum cleaner (10) receives an updated integrated map that includes information about the time the TV is on, and can determine a cleaning path or adjust the cleaning time.

[0058] In this way, the server (30) can create and manage an integrated map while exchanging various information with electronic devices, and the electronic devices can receive the integrated map from the server (30), obtain information collected by other electronic devices from the integrated map, and use this to coordinate tasks. Since the integrated map containing various information is created and managed by the server, each electronic device can utilize information obtained by other electronic devices equipped with sensors that it does not have when performing its own tasks. For example, static electronic devices such as an air purifier (21), an air conditioner (22), a night light (23), and a washing machine (24) can obtain and utilize information collected by a dynamic electronic device such as a robot cleaner (10) while moving through an indoor space through the integrated map, thereby complementing the problem of difficulty in obtaining various information due to a fixed location within an indoor space. In addition, dynamic electronic devices can obtain and utilize information collected by static electronic devices located in different locations through the integrated map without having to collect it directly at that location, thereby performing tasks more quickly or omitting a separate movement operation for information collection.

[0059] FIG. 2 is a block diagram showing the configuration of an electronic device (200) according to one embodiment of the present disclosure.

[0060] An electronic device (200) according to one embodiment of the present disclosure can perform a preset task in an indoor space. For example, the electronic device (200) may be any one of a robot vacuum cleaner (10), an air purifier (21), an air conditioner (22), a night light (23), or a refrigerator (24) illustrated in FIG. 1, but is not limited thereto. As another example, the electronic device (200) may be various types of assistant robots or mobile devices driven for the convenience of a user, an augmented reality (AR) device, a virtual reality (VR) device, or a device that senses the surrounding environment and provides a predetermined service at a specific location or space.

[0061] Referring to FIG. 2, an electronic device (200) according to one embodiment of the present disclosure may include a driving unit (210), a sensor unit (220), a processor (230), a communication unit (240), a user interface (250), and a memory (260).

[0062] In one embodiment, the driving unit (210) may include components that the electronic device (200) utilizes to perform a task.

[0063] For example, if the electronic device (200) is a robot vacuum cleaner, the driving unit (210) may include a suction unit, a driving unit, etc., but is not limited thereto.

[0064] The suction unit has the function of collecting dust on the floor while sucking in air, and may include, but is not limited to, a rotating brush or broom, a rotating brush motor, an air intake, a filter, a dust collection chamber, and an air exhaust. The suction unit may additionally be equipped with a rotating brush structure capable of sweeping away dust from corners.

[0065] The driving unit may include, but is not limited to, a motor that drives each wheel installed in the electronic device (200) to rotate, and a timing belt installed to transmit power generated from the wheel.

[0066] In one embodiment, the sensor unit (220) may include a plurality of sensors configured to detect information about the surrounding environment of the electronic device (200). For example, the sensor unit (220) may include, but is not limited to, an image sensor (221), a LiDAR (light detection and ranging) sensor (222), an infrared sensor (223), an ultrasonic sensor (224), a ToF (time of flight) sensor (225), and a gyro sensor (226).

[0067] The image sensor (221) may include a stereo camera, a mono camera, a wide-angle camera, an around-view camera, or a 3D vision sensor.

[0068] A lidar sensor (222) can detect the distance to an object and various physical properties by shining a laser on the target. The lidar sensor (222) can be used to detect surrounding objects, terrain features, etc., and model them into 3D images.

[0069] The infrared sensor (223) may be either an active infrared sensor that detects changes by emitting infrared rays and blocking the light, or a passive infrared sensor that does not have a light emitter and only detects changes in infrared rays received from the outside. For example, an infrared proximity sensor may be installed around the wheels of an electronic device (200) and used as a fall prevention sensor by irradiating and then receiving infrared rays to the floor.

[0070] An ultrasonic sensor (224) can measure the distance to an object using ultrasonic waves, and can emit and detect ultrasonic pulses that convey information about the proximity of the object. The ultrasonic sensor (224) can be used to detect nearby objects and transparent objects.

[0071] The ToF sensor (225) can obtain the three-dimensionality and movement of an object by calculating the distance it takes for light emitted from an object to bounce back. The ToF sensor (225) enables high-level object recognition in complex spaces, dark places, and even obstacles in front of the eyes, thereby enabling the electronic device (200) to avoid obstacles.

[0072] The gyro sensor (226) can detect angular velocity. The gyro sensor (226) can be used to measure the position and set the direction of the electronic device (200).

[0073] In one embodiment, the processor (230) may control operations or functions so that the electronic device (200) can perform tasks by executing instructions or programmed software modules stored in the memory (260). The processor (230) may be configured with hardware components that perform arithmetic, logic, input / output operations, and signal processing. The processor (230) may control overall operations of the electronic device (200) for managing an integrated map by executing one or more instructions stored in the memory (260). The processor (230) may control the driving unit (210), the sensor unit (220), the communication unit (240), and the user interface (250) by executing programs stored in the memory (260).

[0074] In one embodiment, the processor (230) is a component that controls a series of processes to allow the electronic device (200) to operate, and may be composed of one or more processors. The one or more processors included in the processor (230) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The one or more processors included in the processor (230) may be a general-purpose processor such as a Central Processing Unit (CPU), an Application Processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a Graphics Processing Unit (GPU), a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as a Neural Processing Unit (NPU). For example, when one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0075] For example, the processor (230) may be configured as at least one of a central processing unit, a microprocessor, a graphic processing unit, application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), an application processor, a neural processing unit, or an artificial intelligence processor designed as a hardware structure specialized for processing artificial intelligence models, but is not limited thereto. Each processor constituting the processor (230) may be a dedicated processor for performing a predetermined function.

[0076] In one embodiment, the artificial intelligence processor may perform calculations and control to process tasks set for the electronic device (200) to perform using an artificial intelligence model. The artificial intelligence processor may be manufactured in the form of a dedicated hardware chip for artificial intelligence, or may be manufactured as part of a general-purpose processor (e.g., CPU or application processor) or a graphics-only processor (e.g., GPU) and mounted on the electronic device (200).

[0077] In one embodiment, the communication unit (240) may include one or more components that enable the electronic device (200) to communicate with an external device. For example, the communication unit (240) may include, but is not limited to, a short-range communication unit (241), a long-range communication unit (242), etc.

[0078] The short-range communication unit (241) may include, but is not limited to, a Bluetooth communication unit, a BLE (Bluetooth low energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an Ant+ communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultra wideband) communication unit, an infrared (IrDa, infrared data association) communication unit, a microwave (uWave) communication unit, etc.

[0079] In one embodiment, the user interface (250) may include an input interface (251) and an output interface (252). The user interface (250) may be a separate input interface (251) and an output interface (252), or may be an integrated unit, such as a touch screen. The user interface (250) may receive input information from a user and provide output information to the user.

[0080] The input interface (251) may refer to a means for a user to input data for controlling the electronic device (200). For example, the input interface (251) may be a key pad, a touch pad (contact electrostatic capacitance type, pressure resistive film type, infrared detection type, surface ultrasonic conduction type, integral tension measurement type, piezo effect type, etc.), etc. In addition, the input interface (251) may include a jog wheel, a jog switch, etc., but is not limited thereto.

[0081] The output interface (252) can output an audio signal, a video signal, or a vibration signal, and the output interface (252) can include a display unit, an audio output unit, and a vibration motor.

[0082] The display unit can display information processed in the electronic device (200). For example, the display unit can display an interface for receiving input from a user's operation. When the display unit and the touchpad are configured as a touch screen in a layered structure, the display unit can be used as an input device in addition to an output device. The display unit can include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, an organic light emitting diode, a flexible display, and a 3D display. Depending on the implementation form of the electronic device (200), the electronic device (200) can include two or more display units.

[0083] The audio output unit can output audio data stored in the memory (260). The audio output unit can output audio signals related to functions performed in the electronic device (200). The audio output unit may include a speaker, a buzzer, etc.

[0084] A vibration motor can output a vibration signal. For example, a vibration motor can output a vibration signal corresponding to the output of audio data or video data. A vibration motor can output a vibration signal when a touch is input to a touch screen.

[0085] In one embodiment, the memory (260) may store a program for processing and controlling the processor (230), and may store data (e.g., an integrated map, spatial information, a user request, etc.) input to or output from the electronic device (200). The memory (260) may store instructions, data structures, and program codes that can be read by the processor (230). In the disclosed embodiments, operations performed by the processor (230) may be implemented by executing instructions or codes of a program stored in the memory (260).

[0086] For example, the memory (260) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0087] In one embodiment, the memory (260) may store one or more instructions and / or programs that control the electronic device (200) to perform a task.

[0088] FIG. 3 is a block diagram showing the configuration of an electronic device (300) according to one embodiment of the present disclosure.

[0089] An electronic device (300) according to one embodiment of the present disclosure can create and manage an integrated map. For example, the electronic device (300) may correspond to the server (30) illustrated in FIG. 1, but is not limited thereto.

[0090] Referring to FIG. 3, an electronic device (300) according to one embodiment of the present disclosure may include a memory (310), a processor (320), and a communication unit (330). The memory (310), the processor (320), and the communication unit (330) correspond to the memory (260), the processor (230), and the communication unit (240) illustrated in FIG. 2, respectively, and therefore, a detailed description of the functions and operations of each will be omitted.

[0091] FIG. 4 is a diagram illustrating a method for electronic devices to manage an integrated map for an indoor space according to an embodiment of the present disclosure. In FIG. 4 , the electronic device (200B) may be a different electronic device from the electronic device (200A), and the electronic device (300) may be a server, but is not limited thereto. For example, the electronic device (200A) may be a dynamic electronic device, and the electronic device (200B) may be a static electronic device. In FIG. 4 and the following description, operations performed by the electronic device (200A) in relation to the integrated map may also be performed by the electronic device (200B). For convenience of explanation, the electronic devices will be referred to as a first electronic device (200A), a second electronic device (200B), and a third electronic device (300), respectively.

[0092] In one embodiment, the first electronic device (200A) can receive an integrated map of an indoor space including the first electronic device (200A) and at least one second electronic device (200B) from the third electronic device (300) (410).

[0093] In one embodiment, the integrated map may include information regarding the relationship between the first electronic device (200A) and the second electronic device (200B). For example, the information regarding the relationship between the first electronic device (200A) and the second electronic device (200B) may include at least one of physical information of the first electronic device (200A) and the second electronic device (200B), information regarding tasks of the first electronic device (200A) and the second electronic device (200B), surrounding information collected by the first electronic device (200A) and the second electronic device (200B) using sensors, and information regarding the mutual correlation between the first electronic device (200A) and the second electronic device (200B).

[0094] In one embodiment, the integrated map may be implemented in the form of a graph including a plurality of nodes representing a first electronic device (200A) and at least one second electronic device (200B) and a plurality of edges connecting the plurality of nodes. In this case, each edge may include a weight vector determined based on a relationship between electronic devices corresponding to two connected nodes. For example, an edge connecting a node representing the first electronic device (200A) and a node representing the second electronic device (200B) may include a weight vector determined based on information regarding the relationship between the first electronic device (200A) and the second electronic device (200B). For example, the weight vector may be composed of weight values ​​each corresponding to physical information of the first electronic device (200A) and the second electronic device (200B), information about tasks of the first electronic device (200A) and the second electronic device (200B), surrounding information collected by the first electronic device (200A) and the second electronic device (200B) using sensors, and information about the mutual correlation between the first electronic device (200A) and the second electronic device (200B).

[0095] The integrated map will be described in more detail later with reference to Fig. 5.

[0096] In one embodiment, the first electronic device (200A) can adjust tasks set to be performed by the first electronic device (200A) based on the integrated map (420). The first electronic device (200A) can extract information related to its tasks from the integrated map and, based on the extracted information, modify or delete preset tasks or add new tasks. Rules regarding how to adjust tasks based on the information extracted from the integrated map may be predefined in the memory of the first electronic device (200A).

[0097] For example, if the first electronic device (200A) is a robot vacuum cleaner and the second electronic device (200B) is an air purifier, the first electronic device (200A) can extract the contamination level data collected by the second electronic device (200B) from the received integrated map and determine a cleaning path or adjust the dust suction strength based on the extracted contamination level data. The detailed process of operation 420 will be described later with reference to FIG. 6.

[0098] In one embodiment, the first electronic device (200A) may collect ambient information about an indoor space while performing a coordinated task (430). The information collected by the electronic device using sensors may include information about the structure of the indoor space, information about objects located in the indoor space, information about the environment of the indoor space, etc. For example, if the first electronic device (200A) is a robot vacuum cleaner, the first electronic device (200A) may collect ambient information such as "a lot of cat hair is on the living room floor" or "a person is in the bedroom" while cleaning the indoor space.

[0099] In one embodiment, the first electronic device (200A) may transmit the collected surrounding information to the third electronic device (300) for updating the integrated map (440). Additionally, the first electronic device (200A) may transmit information regarding the task of the first electronic device (200A) to the third electronic device (300). For example, if the first electronic device (200A) is a robot vacuum cleaner, the first electronic device (200A) may transmit the surrounding information collected in operation 430 and / or information regarding the task adjusted in operation 420 to the third electronic device (300).

[0100] In one embodiment, the third electronic device (300) may receive ambient information about an indoor space collected by the first electronic device (200A) from the first electronic device (200A) (440). Additionally, the third electronic device (300) may receive information about a task of the first electronic device (200A) from the first electronic device.

[0101] In one embodiment, the third electronic device (300) may update an integrated map of an indoor space including the first electronic device (200A) and at least one second electronic device (200B) based on the received surrounding information (450). Updating the integrated map may include deleting existing nodes, adding new nodes, and / or modifying weight vectors included in edges. For example, if a person who has been out returns home and is located in a bedroom, the first electronic device (200A) may collect information indicating that "the person is in the bedroom" and transmit it to the third electronic device (300). Based on the information, the third electronic device (300) may add a new node representing the person and edges connecting the node representing the person to other existing nodes to the integrated map. As another example, if the first electronic device (200A) is a robot vacuum cleaner, the third electronic device (300) can modify the weight vector included in the edge connecting the node representing the living room and other nodes based on the information “there is a lot of cat hair on the living room floor” received from the first electronic device (200A).

[0102] To update the integrated map, the third electronic device (300) may utilize an artificial neural network. For example, a GNN (Graphic Neural Network), a GCN (Graph Convolution Network), a GAT (Graph Attention Network), a GGS-NN (Gated Graph Sequence Neural Network), etc., which are easily capable of processing graph-type data, may be used as an artificial neural network for updating the integrated map, but the present invention is not limited thereto. The updating of the integrated map (500) will be described later with reference to FIG. 7.

[0103] In one embodiment, the third electronic device (300) can transmit the updated integrated map to the first electronic device (200A) and the second electronic device (200B) (460, 470). The first electronic device (200A) and the second electronic device (200B) can perform operations 420, 430, and 440 on the updated integrated map, and the third electronic device (300) can re-update the updated integrated map based on the spatial information received from the first electronic device (200A) and the second electronic device (200B). For example, if the first electronic device (200A) is a robot vacuum cleaner and the second electronic device (200B) is an air purifier, the second electronic device (200B) can extract information that “there is a lot of cat hair on the living room floor” based on the updated integrated map received from the third electronic device (300) and activate the pet care function based on the information.

[0104] FIG. 5 illustrates an example of an integrated map (500) according to one embodiment of the present disclosure.

[0105] In one embodiment, the integrated map (500) may be implemented as a graph including a plurality of nodes (510) representing a first electronic device (200A) and at least one second electronic device, and a plurality of edges (520) connecting the plurality of nodes. For example, as illustrated in FIG. 5 , the integrated map (500) may include a plurality of nodes representing a robot vacuum cleaner, a night light, an air conditioner, an air purifier, and a refrigerator, respectively, and may include a plurality of edges connecting the plurality of nodes. However, the method of implementing the integrated map (500) is not limited to a graph, and the integrated map (500) may be implemented in any form that may include various pieces of information to be described later.

[0106] In one embodiment, each edge (520) may include a weight vector determined based on the relationship between electronic devices corresponding to two connected nodes. Here, the relationship between electronic devices may include at least one of physical information of each electronic device (e.g., type, location, size, etc.), information about tasks of the electronic devices (e.g., operation mode, operation time, etc.), surrounding information collected by the electronic devices using sensors, and information about interrelationships between the electronic devices. In one embodiment, the weight vector may be composed of values ​​or vectors corresponding to each of the above-described pieces of information.

[0107] For example, ambient information collected by electronic devices using sensors may include information about the structure of an indoor space, such as the location, area, shape, and distance of a living room, bedroom, or kitchen. Furthermore, ambient information collected by electronic devices using sensors may include information about objects (e.g., people, tables, etc.) located in the indoor space, such as whether there is a person in the bedroom or whether there is a table in the living room. Furthermore, ambient information collected by electronic devices using sensors may include information about the environment of the indoor space, such as changes in brightness in the bedroom due to day-night changes or lighting, air pollution in the living room, and whether there is a lot of cat hair on the living room floor.

[0108] For example, the correlation between electronic devices may refer to the degree of physical proximity between the electronic devices, the degree to which the electronic devices are functionally similar, the degree to which the peripheral information collected by the electronic devices is related to each other's tasks, and / or the frequency with which the electronic devices exchange data with each other. Information regarding the correlation between electronic devices may be determined based on physical information of each electronic device, information regarding the tasks of the electronic devices, and / or peripheral information collected by the electronic devices using sensors. As an example, the closer the first electronic device (200A) and the second electronic device (200B) are physically, the larger the value corresponding to the correlation between the two electronic devices may be determined. As another example, the value corresponding to the correlation between the two electronic devices may be determined as a value between 0 and 1 depending on the degree to which the first electronic device (200A) and the second electronic device (200B) are functionally similar.

[0109] In one embodiment, the integrated map (500) may be configured to include information about a first electronic device (200A), information about a second electronic device (200B), and information indicating a correlation between the first electronic device (200A) and the second electronic device (200B).

[0110] In one embodiment, information about the first electronic device (200A) may include physical information such as the type, location, and size of the first electronic device (200A), information about a task of the first electronic device (200A), or surrounding information collected by the first electronic device (200A).

[0111] Information about the task of the first electronic device (200A) may vary depending on the type of the first electronic device (200A). For example, if the first electronic device (200A) is a robot vacuum cleaner, information about the task of the first electronic device (200A) may include cleaning time, cleaning area, and / or cleaning path.

[0112] The surrounding information collected by the first electronic device (200A) may vary depending on the sensor equipped with the first electronic device (200A). For example, if the first electronic device (200A) is a robot vacuum cleaner equipped with an image sensor, the spatial information collected by the first electronic device (200A) may include the presence or absence or location of objects (e.g., people, tables, etc.) within an indoor space.

[0113] In one embodiment, information about the second electronic device (200B) may include physical information such as the type, location, and size of the second electronic device (200B), information about a task of the second electronic device (200B), or spatial information collected by the second electronic device (200B).

[0114] Information about the task of the second electronic device (200B) may vary depending on the type of the second electronic device (200B). For example, if the second electronic device (200B) is an air purifier, information about the task of the second electronic device (200B) may include operation time, operation mode, etc.

[0115] The surrounding information collected by the second electronic device (200B) may vary depending on the sensor equipped with the second electronic device (200B). For example, if the second electronic device (200B) is an air purifier equipped with a sensor for measuring pollution levels, the surrounding information collected by the second electronic device (200B) may include the pollution level, fine dust level, presence or concentration of harmful gases, etc., around the second electronic device (200B).

[0116] In one embodiment, information indicating a mutual correlation between the first electronic device (200A) and the second electronic device (200B) may include, but is not limited to, the degree to which the physical distance between the first electronic device (200A) and the second electronic device (200B) is close, the degree to which the first electronic device (200A) and the second electronic device (200B) are functionally similar, the degree to which data collected by the first electronic device (200A) and the second electronic device (200B) is related to each other's tasks, or the frequency with which the first electronic device (200A) and the second electronic device (200B) exchange data.

[0117] In one embodiment, information indicating a relationship between the first electronic device (200A) and the second electronic device (200B) may be determined based on information about the first electronic device (200A) and information about the second electronic device (200B).

[0118] In one embodiment, the integrated map (500) may further include at least one node representing at least one object (e.g., a person, a table, etc.) within an indoor space, and at least one edge connecting the at least one node to other nodes. The at least one edge may include a weight vector determined based on a relationship between an object and an electronic device corresponding to two connected nodes. Here, the relationship between the object and the electronic device may include at least one of physical information of the object and the electronic device (e.g., type, location, size, etc.), information about a task of the electronic device (e.g., operation mode, operation time, etc.), surrounding information collected by the electronic device using a sensor, and information about a correlation between the object and the electronic device. In one embodiment, the weight vector may be composed of values ​​or vectors corresponding to each of the above-described pieces of information.

[0119] For example, the correlation between an object and an electronic device may refer to the degree of physical proximity between the object and the electronic device, the degree to which the electronic device's task is related to the object, etc. Information regarding the correlation between an object and an electronic device may be determined based on physical information of the object and the electronic device, information regarding the task of the electronic device, and / or surrounding information collected by the electronic device using sensors.

[0120] In one embodiment, nodes and edges associated with at least one object may be added to the integrated map (500) by an operation (450) of updating the integrated map (500). For example, if a robot vacuum cleaner collects information indicating that a person is in a bedroom as spatial information while performing cleaning and transmits this to a server, the server may update the integrated map (500) by adding a node representing the person and edges representing the relationship between the person and other electronic devices.

[0121] In one embodiment, the integrated map (500) may be configured by combining multiple local maps (531, 532, 533). Here, the local map may be understood as a subgraph including at least one node representing electronic devices and objects included in a portion of an indoor space (e.g., a living room, a bedroom, a kitchen, etc.) and at least one edge connecting the at least one node to other nodes. For example, the integrated map (500) may be configured by combining a local map (531) corresponding to the living room, a local map (532) corresponding to the bedroom, and a local map (533) corresponding to the kitchen. The local map (531) corresponding to the living room may include nodes representing an air conditioner and an air purifier, respectively, the local map (532) corresponding to the bedroom may include nodes representing nighttime, etc., and the local map (533) corresponding to the kitchen may include nodes representing a refrigerator. The server's method of constructing multiple local maps (531, 532, 533) and then combining them to construct an integrated map (500) can reduce the server's computational burden. For example, if the server receives information from a robot vacuum cleaner indicating that there is a person in the bedroom, the server can first create a local map (532) representing the bedroom, rather than updating the entire integrated map (500) from scratch, and then update the integrated map (500) based on this local map.

[0122] FIG. 6 is a diagram for explaining a method for a first electronic device (200A) to coordinate a task according to one embodiment of the present disclosure.

[0123] In one embodiment, the first electronic device (200A) can extract task-related information from the integrated map (500) (431). The first electronic device (200A) can extract task-related information from the weight vector of the edge connected to the node representing the first electronic device (200A) in the integrated map (500). For example, if the first electronic device (200A) is a robot vacuum cleaner, the first electronic device (200A) can extract information such as the level of contamination in an indoor space, the location of objects (e.g., people, tables, etc.), and whether the TV is on from the integrated map (500). The first electronic device (200A) can consider the extracted information when adjusting the cleaning path, cleaning time, dust suction strength, etc.

[0124] In one embodiment, the first electronic device (200A) may apply the extracted information to an individual map for a task (432). Here, the individual map may be data about an indoor space, separate from the integrated map (500), that the first electronic device (200A) creates and manages to perform its own task. For example, the individual map may be a spatial map having a hierarchical structure composed of a base layer that provides information about the basic structure of the indoor space, such as walls, pillars, and passageways, a semantic map layer that provides semantic information on top of the base layer, and a real-time layer that provides information about at least one object within the indoor space.

[0125] If the first electronic device (200A) does not use a separate individual map to perform its task, operation 432 may be omitted. In this case, the first electronic device (200A) may directly adjust the task based on information extracted from the integrated map (500).

[0126] In one embodiment, the first electronic device (200A) can coordinate tasks based on individual maps (433). Rules regarding how to coordinate tasks based on information extracted from the integrated map may be predefined in the memory of the first electronic device (200A). Alternatively, rules regarding how to coordinate tasks based on information extracted from the integrated map may be set by a user, and the user may set the task coordination rules through the user interface of the first electronic device (200A) or a user terminal (e.g., a smartphone, smartwatch, tablet, personal computer, etc.) connected to the first electronic device (200A) via a wired or wireless network.

[0127] For example, if the first electronic device (200A) is a robot vacuum cleaner, the first electronic device (200A) can adjust tasks to increase the suction strength of dust in a highly contaminated space, lower noise near a person, or exclude a space with a TV from the cleaning path while the TV is on and perform cleaning when the TV is off, according to predefined rules.

[0128] FIG. 7 is a diagram illustrating a method for a third electronic device (300) to update an integrated map according to one embodiment of the present disclosure.

[0129] In one embodiment, the third electronic device (300) may preprocess the received peripheral information (451). Operation 451 may be understood as a process of converting the raw data collected by the first electronic device (200A) into a format suitable for constructing an integrated map (500). Therefore, the specific preprocessing method may vary depending on the format of the raw data.

[0130] In one embodiment, operation 451 may be performed by the first electronic device (200A). In other words, depending on the implementation method, the first electronic device (200A) may transmit raw data regarding the collected surrounding information to the third electronic device (300) as is, or may preprocess the raw data and then transmit it to the third electronic device (300). For example, in the implementation, the entity performing operation 451 may be determined based on the computational capabilities of the first electronic device (200A) and the third electronic device (300).

[0131] In one embodiment, the third electronic device (300) may generate a local map using the preprocessed surrounding information (452). Here, the local map may be understood as a subgraph including at least one node representing electronic devices and objects included in a portion of an indoor space (e.g., a living room, a bedroom, a kitchen, etc.) and at least one edge connecting the at least one node to other nodes. A weight vector to be included in an edge within the local map may be calculated based on the surrounding information received from the first electronic device (200A). The calculated weight vector may indicate a similarity between two connected nodes. Here, the similarity between nodes may indicate, for example, a degree to which objects and / or electronic devices represented by two nodes influence each other, and a higher similarity (i.e., a larger corresponding value within the weight vector) may indicate a closer distance between two nodes.

[0132] For example, if a first electronic device (200A) transmits peripheral information indicating that “a person is in a bedroom” to a third electronic device (300), the third electronic device (300) can generate a local map (532) corresponding to the bedroom, and the local map (532) corresponding to the bedroom can include a node representing the person, a node representing other electronic devices included in the bedroom (e.g., a night light), and edges connecting the nodes. In addition, the third electronic device (300) can calculate a weight vector based on physical information (e.g., distance) between the person and the other electronic device.

[0133] In one embodiment, the third electronic device (300) may update the integrated map (500) based on the local map (453). Here, updating the integrated map (500) may include deleting existing nodes or adding new nodes, modifying weight vectors included in existing edges, and / or calculating weight vectors to be included in new edges. As an example, if a node not present in the integrated map (500) is included in the local map, the third electronic device (300) may add the new node to the integrated map (500). As another example, the third electronic device (300) may calculate weight vectors to be included in edges connecting the new node and other existing nodes, and at this time, the weight vectors of the edges within the local map may be taken into account.

[0134] In one embodiment, the third electronic device (300) can store the updated integrated map (500) in memory (310) and transmit it to the first electronic device (200A) and the second electronic device (200B) when necessary.

[0135] FIG. 8 is a diagram illustrating a method for electronic devices according to one embodiment of the present disclosure to manage an integrated map (500) by reflecting a user request.

[0136] Electronic devices according to one embodiment of the present disclosure can perform a task to meet the user request by obtaining a user request related to a task of at least one electronic device and updating an integrated map (500) based on the user request.

[0137] For example, when a user request to minimize noise near a person is obtained (810), the electronic devices can update the integrated map (500) based on the user request (830). Thereafter, when information that a person is in a bedroom is collected by one of the electronic devices, the integrated map (500) can be updated based on the information (i.e., operations 430, 440, and 450 of FIG. 4). The robot cleaner that receives the updated integrated map (500) (i.e., operation 410 of FIG. 4) can obtain information that a person is in the bedroom and adjust the suction strength of the dust to the lowest level to minimize noise in the bedroom (i.e., operation 420 of FIG. 4).

[0138] For example, the process of electronic devices obtaining user requests and updating the integrated map (500) may be performed as follows.

[0139] In one embodiment, a first electronic device (200A) may obtain a user request related to a task of at least one electronic device included in an indoor space from a user (810). The first electronic device (200A) may obtain the user request in various ways, and the first electronic device (200A) may include a suitable sensor or interface to obtain the user request. For example, if the first electronic device (200A) includes a microphone as an input interface, the first electronic device (200A) may obtain the user request in the form of voice. As another example, if the first electronic device (200A) is connected to a user terminal via a wired or wireless network, the first electronic device (200A) may obtain a user request input by the user via the user terminal.

[0140] In one embodiment, the first electronic device (200A) may transmit a user request related to a task of at least one electronic device included in an indoor space to the third electronic device (300) (820).

[0141] In one embodiment, the third electronic device (300) may receive a user request related to a task of at least one electronic device included in an indoor space from the first electronic device (200A) (820).

[0142] In one embodiment, the third electronic device (300) may update the integrated map (500) based on a received user request (830). Updating the integrated map (500) may include preprocessing the received user request and updating edges associated with the user request within the integrated map based on the preprocessed user request. Updating the integrated map will be described later with reference to FIG. 9.

[0143] In one embodiment, the third electronic device (300) may transmit the updated integrated map (500) to the first electronic device (200A) and the second electronic device (200B) (840, 850). The first electronic device (200A) and the second electronic device (200B) may perform the operations of FIG. 4 on the updated integrated map reflecting the user request.

[0144] FIG. 9 is a diagram for explaining a method for a third electronic device (300) to update an integrated map according to one embodiment of the present disclosure.

[0145] In one embodiment, the third electronic device (300) may preprocess the received user request (831). Operation 831 may be understood as a process in which the third electronic device (300) extracts information necessary to update the integrated map (500) from the received user request and converts it into a form suitable for configuring the integrated map (500). Accordingly, the specific preprocessing method may vary depending on the form of the received user request.

[0146] For example, if the received user request is in the form of a voice, the third electronic device (300) may perform natural language processing on the voice to extract information associated with the task of the first electronic device (200A), such as nodes, edges, attributes, and / or actions associated with the user request. Here, an attribute may be understood as an object that needs to be adjusted in relation to the user request and the task of the first electronic device (200A). For example, the attribute may be determined based on the content of the user request, such as a physical quantity such as sound or time, the task of the electronic device itself, or a part of the task of the electronic device. An action may be understood as the content of a necessary adjustment with respect to an attribute.

[0147] For example, if a user request in the form of a voice such as "I would like you to lower your voice in front of people" is received, the third electronic device (300) can perform natural language processing on the voice to extract information such as 'node representing a person', 'all edges connected to the node representing a person', 'sound', and 'lower' as nodes, edges, properties, and actions.

[0148] In one embodiment, the third electronic device (300) may update an edge associated with a user request within the integrated map based on the preprocessed user request (832). The edge associated with the user request may be determined in operation 831, and updating the edge may be understood as updating a weight vector included in the edge based on the information extracted in operation 831.

[0149] In one embodiment, the weight vector included in an edge within the integrated map (500) may include values ​​or vectors representing information about a user request. For example, the weight vector may include values ​​or vectors representing information such as 'a node representing a person,' 'all edges connected to the node representing a person,' 'sound,' and 'lower.'

[0150] FIGS. 10A and 10B are diagrams illustrating examples of electronic devices reflecting and utilizing user requests in an integrated map (500) according to one embodiment of the present disclosure. FIGS. 10A and 10B illustrate an example where the received user request is a voice message stating, "Please lower your voice in front of people."

[0151] When receiving a user request in voice form from a first electronic device (200A), the third electronic device (300) can perform natural language processing on the voice according to operation 831 to extract information such as a 'node representing a person' as a node associated with the user request, 'all edges connected to the node representing a person' as an edge associated with the user request, 'sound' as an attribute associated with the user request, and 'lower' as an action associated with the user request.

[0152] The third electronic device (300) may update the weight vectors of all edges connected to the node representing the person based on the extracted information, according to operation 832. At this time, the degree to which the weight vector of each edge changes may vary depending on the location of the person. For example, if the third electronic device (300) obtains information from the first electronic device (200A) that the person is in the bedroom, according to operations 420 and 430 of FIG. 4, the edges between the node representing the person and the nodes representing electronic devices located in the bedroom may be strengthened and have a larger value than the edges between the node representing the person and the nodes representing other electronic devices.

[0153] According to operations 840 and 850, the integrated map (500) updated based on a user request can be transmitted to electronic devices included in an indoor space, and each electronic device that receives the updated integrated map (500) can adjust its own task, such as changing its operating mode to reduce noise generated or operating only when no one is present, according to operation 420. For example, a robot vacuum cleaner can minimize dust suction intensity in a bedroom to reduce noise or adjust its cleaning time to clean the bedroom when a person leaves the bedroom based on the updated integrated map (500).

[0154] Meanwhile, according to the operations of FIG. 4, the integrated map (500) reflecting the user request may also be updated. For example, if information indicating that there is no person in an indoor space is collected by the first electronic device (200A), the third electronic device (300) may delete nodes representing people and edges connected to the nodes representing people from the integrated map (500) based on the information. Thereafter, when the integrated map (500) with the nodes and edges related to people deleted is transmitted to the first electronic device (200A) and the second electronic device (200B), the first electronic device (200A) and the second electronic device (200B) may increase the noise generated based on the received integrated map (500) to its original level or perform a task that was not performed when a person was present. For example, a robot vacuum cleaner may perform bedroom cleaning based on the integrated map (500) with the nodes and edges related to people deleted.

[0155] A method performed by a first electronic device according to one embodiment of the present disclosure may include receiving, from a third electronic device, an integrated map of an indoor space including the first electronic device and at least one second electronic device.

[0156] In one embodiment, the method may include a step (420) of adjusting a task set to be performed by the first electronic device (200A) based on the integrated map (500).

[0157] In one embodiment, the method may include collecting first ambient information about the indoor space while performing the adjusted task.

[0158] In one embodiment, the method may include a step (440) of transmitting the collected first surrounding information to the third electronic device (300) for updating the integrated map (500).

[0159] In one embodiment, the integrated map (500) may include second surrounding information about the indoor space obtained by the third electronic device (300) from the at least one second electronic device (200B).

[0160] In one embodiment, the integrated map (500) is a graph including a plurality of nodes representing the first electronic device (200A) and the at least one second electronic device (200B), and a plurality of edges connecting the plurality of nodes, and each edge may include a weight vector determined based on a relationship between electronic devices corresponding to two connected nodes.

[0161] In one embodiment, the relationship between the electronic devices corresponding to the two connected nodes may include at least one of physical information of the electronic devices, information about tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information about mutual correlations between the electronic devices.

[0162] In one embodiment, the step of coordinating the task (430) may include the steps of extracting information associated with the task from the integrated map (500) (431), applying the extracted information to an individual map for the task (432), and coordinating the task based on the individual map (433).

[0163] In one embodiment, the method may further include a step (810) of obtaining a user request related to a task of at least one electronic device included in the indoor space from a user, and a step (820) of transmitting the user request to the third electronic device (300).

[0164] In one embodiment, the first electronic device (200A) may be a dynamic electronic device and the second electronic device (200B) may be a static electronic device.

[0165] A method performed by a third electronic device according to one embodiment of the present disclosure may include a step (440) of receiving, from a first electronic device (200A), first surrounding information about an indoor space collected by the first electronic device (200A).

[0166] In one embodiment, the method may include a step (450) of updating an integrated map (500) for the indoor space including the first electronic device (200A) and at least one second electronic device (200B), based on the received first surrounding information.

[0167] In one embodiment, the integrated map (500) may include second surrounding information about the indoor space obtained by the third electronic device (300) from the at least one second electronic device (200B).

[0168] In one embodiment, the integrated map (500) is a graph including a plurality of nodes representing the first electronic device (200A) and the at least one second electronic device (200B), and a plurality of edges connecting the plurality of nodes, and each edge may include a weight vector determined based on a relationship between electronic devices corresponding to two connected nodes.

[0169] In one embodiment, the relationship between the electronic devices corresponding to the two connected nodes may include at least one of physical information of the electronic devices, information about tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information about mutual correlations between the electronic devices.

[0170] In one embodiment, the step (450) of updating the integrated map (500) may include the step (451) of preprocessing the received first surrounding information, the step (452) of generating a local map using the preprocessed first surrounding information, and the step (453) of updating the integrated map (500) based on the local map.

[0171] In one embodiment, the method may further include a step (820) of receiving, from the first electronic device (200A), a user request related to a task of at least one electronic device included in the indoor space, and a step (830) of updating the integrated map (500) based on the received user request.

[0172] In one embodiment, the step (830) of updating the integrated map (500) based on the received user request may include the step (831) of preprocessing the received user request, and the step (832) of updating an edge associated with the user request in the integrated map (500) based on the preprocessed user request.

[0173] In one embodiment, the received user request may be in the form of voice, and the step (831) of preprocessing the received user request may include a step of performing natural language processing on the received user request to extract a node associated with the user request, an edge associated with the user request, an attribute associated with the user request, and an action associated with the user request.

[0174] A first electronic device according to one embodiment of the present disclosure may include a communication unit (240), a sensor unit (220), a memory (260) storing one or more instructions, and at least one processor (230) executing the one or more instructions stored in the memory.

[0175] In one embodiment, the at least one processor (230) may execute the one or more instructions to cause the first electronic device to receive, through the communication unit (240), an integrated map (500) of an indoor space including the first electronic device (200A) and at least one second electronic device (200B), from the third electronic device (300).

[0176] In one embodiment, the at least one processor (230) may cause the first electronic device (200A) to coordinate a task that the first electronic device (200A) is set to perform based on the integrated map (500) by executing the one or more instructions.

[0177] In one embodiment, the at least one processor (230) may execute the one or more instructions to cause the first electronic device to collect first ambient information about the indoor space while performing the adjusted task through the sensor unit (220).

[0178] In one embodiment, the at least one processor (230) may execute the one or more instructions to cause the first electronic device to transmit the collected first surrounding information to the third electronic device (300) through the communication unit (240) for updating the integrated map (500).

[0179] In one embodiment, the integrated map (500) may include second surrounding information about the indoor space obtained by the third electronic device (300) from the at least one second electronic device (200B).

[0180] In one embodiment, the integrated map (500) is a graph including a plurality of nodes representing the first electronic device (200A) and the at least one second electronic device (200B), and a plurality of edges connecting the plurality of nodes, and each edge may include a weight vector determined based on a relationship between electronic devices corresponding to two connected nodes.

[0181] In one embodiment, the relationship between the electronic devices corresponding to the two connected nodes may include at least one of physical information of the electronic devices, information about tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information about mutual correlations between the electronic devices.

[0182] In one embodiment, the at least one processor (230) may execute the one or more instructions to cause the first electronic device to extract information associated with the task from the integrated map (500), apply the extracted information to an individual map for the task, and adjust the task based on the individual map.

[0183] In one embodiment, the at least one processor (230) may execute the one or more instructions to cause the first electronic device to obtain a user request related to a task of at least one electronic device included in the indoor space from a user through the user interface (250), and to transmit the user request to the third electronic device (300) through the communication unit (240).

[0184] In one embodiment, the first electronic device (200A) may be a dynamic electronic device and the second electronic device (200B) may be a static electronic device.

[0185] A third electronic device according to one embodiment of the present disclosure may include a communication unit (330), a memory (310) storing one or more instructions, and at least one processor (320) executing the one or more instructions stored in the memory (330).

[0186] In one embodiment, the at least one processor (320) may execute the one or more instructions to cause the third electronic device to receive, through the communication unit (330), first surrounding information about the indoor space collected by the first electronic device (200A) from the first electronic device (200A).

[0187] In one embodiment, the at least one processor (320) may cause the third electronic device to update an integrated map (500) for the indoor space including the first electronic device (200A) and at least one second electronic device (200B) based on the received first surrounding information by executing the one or more instructions.

[0188] In one embodiment, the integrated map (500) may include second surrounding information about the indoor space obtained by the third electronic device (300) from the at least one second electronic device (200B).

[0189] In one embodiment, the integrated map (500) is a graph including a plurality of nodes representing the first electronic device (200A) and the at least one second electronic device (200B), and a plurality of edges connecting the plurality of nodes, and each edge may include a weight vector determined based on a relationship between electronic devices corresponding to two connected nodes.

[0190] In one embodiment, the relationship between the electronic devices corresponding to the two connected nodes may include at least one of physical information of the electronic devices, information about tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information about mutual correlations between the electronic devices.

[0191] In one embodiment, the at least one processor (320) may execute the one or more instructions to cause the third electronic device to preprocess the received first surrounding information, generate a local map using the preprocessed first surrounding information, and update the integrated map (500) based on the local map.

[0192] In one embodiment, the at least one processor (320) may execute the one or more instructions to cause the third electronic device to receive, from the first electronic device (200A) through the communication unit (330), a user request related to a task of at least one electronic device included in the indoor space, and update the integrated map (500) based on the received user request.

[0193] In one embodiment, the at least one processor (320) may execute the one or more instructions to cause the third electronic device to preprocess the received user request and update an edge associated with the user request in the integrated map (500) based on the preprocessed user request.

[0194] In one embodiment, the received user request may be in the form of a voice, and the at least one processor (320) may perform natural language processing on the received user request by executing the one or more instructions to extract a node associated with the user request, an edge associated with the user request, an attribute associated with the user request, and an action associated with the user request.

[0195] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0196] The scope of the present disclosure is indicated by the claims set forth below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. In the method performed by the first electronic device (200A), A step (410) of receiving an integrated map (500) of an indoor space including the first electronic device (200A) and at least one second electronic device (200B) from a third electronic device (300); A step (420) of adjusting a task set to be performed by the first electronic device (200A) based on the integrated map (500); A step (430) of collecting first surrounding information about the indoor space while performing the above-mentioned adjusted task; and Step (440) of transmitting the collected first surrounding information to the third electronic device (300) for updating the integrated map (500); A method comprising:

2. In paragraph 1, A method in which the integrated map (500) includes second surrounding information about the indoor space obtained by the third electronic device (300) from at least one second electronic device (200B).

3. In paragraph 1 or 2, The above integrated map (500) A plurality of nodes representing the first electronic device (200A) and at least one second electronic device (200B), and A graph including multiple edges connecting the multiple nodes, A method wherein each edge includes a weight vector determined based on the relationship between electronic devices corresponding to two connected nodes.

4. In any one of paragraphs 1 to 3, The relationship between the electronic devices corresponding to the two connected nodes above is: A method comprising at least one of physical information of the electronic devices, information regarding tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information regarding interconnections between the electronic devices.

5. In any one of paragraphs 1 to 4, The step (430) of adjusting the above task is: Step (431) of extracting information related to the task from the above integrated map (500); Step (432) of applying the extracted information to an individual map for the task, and A method comprising a step (433) of adjusting the task based on the individual map.

6. In any one of paragraphs 1 to 5, Step (810) of obtaining a user request related to a task of at least one electronic device included in the indoor space from a user; and Step (820) of transmitting the user request to the third electronic device (300); A method further comprising:

7. In any one of paragraphs 1 to 6, A method wherein the first electronic device (200A) is a dynamic electronic device and the second electronic device (200B) is a static electronic device.

8. In the method performed by the third electronic device (300), A step (440) of receiving first surrounding information about an indoor space collected by the first electronic device (200A) from the first electronic device (200A); and A step (450) of updating an integrated map (500) for the indoor space including the first electronic device (200A) and at least one second electronic device (200B), based on the received first surrounding information; Including, A method in which the integrated map (500) includes second surrounding information about the indoor space obtained by the third electronic device (300) from at least one second electronic device (200B).

9. In paragraph 8, The above integrated map (500) A plurality of nodes representing the first electronic device (200A) and at least one second electronic device (200B), and A graph including multiple edges connecting the multiple nodes, A method wherein each edge includes a weight vector determined based on the relationship between electronic devices corresponding to two connected nodes.

10. In clause 8 or 9, The relationship between the electronic devices corresponding to the two connected nodes above is: A method comprising at least one of physical information of the electronic devices, information regarding tasks of the electronic devices, surrounding information collected by the electronic devices using sensors, and information regarding interconnections between the electronic devices.

11. In any one of paragraphs 8 to 10, The step (450) of updating the above integrated map (500) is as follows: Step (451) of preprocessing the first peripheral information received above; Step (452) of generating a local map using the above preprocessed first surrounding information; Step (453) of updating the integrated map (500) based on the local map; A method comprising:

12. In any one of paragraphs 8 to 11, A step (820) of receiving a user request related to a task of at least one electronic device included in the indoor space from the first electronic device (200A); and A step (830) of updating the integrated map (500) based on the received user request; A method further comprising:

13. In any one of paragraphs 8 to 12, The step (830) of updating the integrated map (500) based on the received user request is as follows: Step (831) of preprocessing the received user request, and A method comprising the step (832) of updating an edge associated with the user request in the integrated map (500) based on the preprocessed user request.

14. In any one of paragraphs 8 to 13, The above received user request is in the form of voice, The step (831) of preprocessing the received user request is as follows: A method comprising the step of performing natural language processing on the received user request to extract a node associated with the user request, an edge associated with the user request, an attribute associated with the user request, and an action associated with the user request.

15. A computer-readable recording medium storing a computer program for performing any one of the methods of clauses 1 to 14.

Citation Information

Patent Citations

  • Information processing apparatus, information terminal, and information processing method

    JP2017068589A

  • Method and apparatus for providing indoor eco-map

    KR1020090010367A

  • Information processing device, map update method, program, and information processing system

    KR1020110102174A

  • Apparatus and method for generating indoor map using duplication grid analysis

    KR1020180078736A

  • Robot management system, robot management method, information processing device, information processing method and information processing program

    WO2019171916A1