Method for controlling electronic device by using space information and electronic device using space information

Abstract

A method of controlling an electronic device by using spatial information and an electronic device using spatial information are provided. The method includes selecting, based on space information on a space including at least one object and a task that the electronic device is set to execute, an object that hinders the task from among objects located in the space corresponding to the task, providing object movement guide information corresponding to attribute information on the selected object to a user of the electronic device, and determining a movement path for performing the task based on a response of the user corresponding to the object movement guide information, and driving the electronic device according to the determined movement path.

Description

Technical Field

[0001] The present disclosure relates to a method of controlling an electronic device by using spatial information and an electronic device using the spatial information. Background Art

[0002] The Internet has evolved from a human-centered connected network where humans create and consume information to an Internet of Things (IoT) network where decentralized components such as objects exchange information with each other to process information. The Internet of Everything (IoE) technology has emerged, in which IoT technology is combined with technology for processing big data, such as through connection with a cloud server. Through the fusion and integration between existing information technology (IT) and various industries, IoT can be applied to various fields such as smart home appliances, smart homes, smart buildings, smart cities, etc.

[0003] Electronic devices interconnected in an IoT environment can each collect, generate, analyze or process data, and share data with each other so that the data can be used to complete tasks on each device. Recently, with the rapid development of the field of computer vision, various types of electronic devices that use neural network models to perform visual tasks have been developed. Therefore, there is an increasing interest in the connection between various types of electronic devices in an IoT environment.

[0004] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with respect to the present disclosure. Summary of the invention

[0005] Technical Solution Embodiments of the present disclosure solve at least the above problems and / or disadvantages and provide at least the advantages described below.Accordingly, embodiments of the present disclosure provide a method of controlling an electronic device by using spatial information and an electronic device using spatial information.

[0006] Embodiments of the present disclosure will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0007] According to an embodiment of the present disclosure, a method for controlling an electronic device by using spatial information is provided. The method includes: based on spatial information about a space including at least one object and a task that the electronic device is set to perform, selecting an object that hinders the task from objects located in the space corresponding to the task. In addition, the method for controlling an electronic device by using spatial information includes: providing object movement guide information corresponding to attribute information about the selected object to a user of the electronic device. In addition, the method for controlling an electronic device by using spatial information includes: determining a moving path for performing the task based on a user's response corresponding to the object movement guide information. In addition, the method for controlling an electronic device by using spatial information may include: driving the electronic device according to the determined moving path.

[0008] According to an embodiment of the present disclosure, there is provided a computer-readable recording medium having recorded thereon a program for executing the above-mentioned method.

[0009] According to an embodiment of the present disclosure, an electronic device using spatial information is provided. The electronic device includes a memory storing one or more instructions, a processor configured to execute the one or more instructions stored in the memory, and a sensor unit. The processor is configured to execute the one or more instructions to select an object that hinders the task from objects located in the space corresponding to the task based on spatial information about a space including at least one object obtained via the sensor unit and the task that the electronic device is set to perform. The processor is configured to execute the one or more instructions to provide the user of the electronic device with object movement guidance information corresponding to the attribute information about the selected object. The processor is configured to execute the one or more instructions to determine a moving path for performing the task based on a user's response corresponding to the object movement guidance information. The processor is configured to execute the one or more instructions to drive the electronic device according to the determined moving path.

[0010] The aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects, features and advantages of the embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which: Figure 1 is a diagram for describing an internal Internet of Things (IoT) environment in which an electronic device is interconnected with an external device according to an embodiment of the present disclosure; Figure 2a and Figure 2b is a flowchart for describing a spatial map according to an embodiment of the present disclosure; Figure 3a , Figure 3b , Figure 3c and Figure 3d is a diagram for describing a method of using layers constituting a spatial map according to an embodiment of the present disclosure; Figure 4 is a flowchart of a method for obtaining a spatial map according to an embodiment of the present disclosure; Figure 5 is a flowchart of a method for controlling an electronic device by using spatial information according to an embodiment of the present disclosure; Figure 6 is a detailed flowchart illustrating an operation of selecting an object that obstructs a task from objects located in a space corresponding to a task according to an embodiment of the present disclosure; Figure 7 is a detailed flowchart illustrating an operation of providing object movement guide information corresponding to attribute information about an object selected as an object that obstructs a task according to an embodiment of the present disclosure; Figure 8 is a diagram for describing a first move request process according to an embodiment of the present disclosure; Fig. 9 is a diagram for describing an example of providing object movement guide information to a user according to an embodiment of the present disclosure; Fig.10 is a diagram for describing another example of providing object movement guide information to a user according to an embodiment of the present disclosure; Fig.11 is a diagram for describing a second move request process according to an embodiment of the present disclosure; Fig.12 is a diagram for describing a process of selecting a candidate position to which a selected object is to be moved according to an embodiment of the present disclosure; Fig.13 is a diagram for describing an example of providing object movement guide information to a user according to an image evaluation result according to an embodiment of the present disclosure; Fig.14 is a detailed flowchart illustrating an operation of determining a moving path for performing a task according to an embodiment of the present disclosure; and Fig.15 and Fig.16 is a block diagram illustrating a configuration of an electronic device using spatial information according to an embodiment of the present disclosure.

[0012] The same reference numerals are used throughout the drawings to denote the same elements. DETAILED DESCRIPTION

[0013] The following description in conjunction with the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but these details are considered to be exemplary only. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications of the embodiments described herein may be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0014] The terms and words used in the following description and claims are not limited to the bibliographic meanings, but are used only by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of the embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0015] It will be understood that singular forms include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0016] Throughout the disclosure, the expression "at least one of a, b, or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0017] The terms used in this disclosure are general terms currently widely used in the art based on the functions described in this disclosure, but may be changed according to the intention of technicians engaged in this field, precedents, the emergence of new technologies, etc. In addition, specific terms may be arbitrarily selected by the applicant, and in this case, the meaning of the selected terms will be described in detail in the specific embodiments of the present disclosure. Therefore, the terms used herein should not be defined by their simple names, but should be defined based on the meaning of the terms and the overall description of the present disclosure.

[0018] All terms used herein, including technical or scientific terms, may have the same meaning as those generally understood by those of ordinary skill in the art. In addition, although terms including ordinal numbers (such as "first", "second", etc.) may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another element or component.

[0019] Throughout the specification, when a component "includes" or "comprises" an element, unless there is a specific description to the contrary, it will be understood that the component may also include other elements, but other elements are not excluded. In addition, terms such as "part", "module", etc. described in the specification refer to a unit for processing at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software.

[0020] Functions associated with artificial intelligence (AI) according to the present disclosure are executed via a processor and a memory. The processor may be configured as one or more processors. In this case, the one or more processors may be a general-purpose processor (such as a central processing unit (CPU), an application processor (AP), a digital signal processor (DSP), etc.), a dedicated graphics processor (such as a graphics processing unit (GPU), a visual processing unit (VPU), etc.), or a dedicated AI processor (such as a neural processing unit (NPU)). One or more processors control the input data to be processed according to predefined operating rules or AI models stored in the memory. Optionally, when one or more processors are dedicated AI processors, the dedicated AI processor may be designed with a hardware structure dedicated to processing a specific AI model.

[0021] Predefined operating rules or AI models are created through a training process. In this case, creation through a training process means creating predefined operating rules or AI models that are set to perform desired characteristics (or uses) by training a basic AI model based on a large amount of training data via a learning algorithm. The training process can be performed by the device executing AI itself, or via a separate server and / or system. Examples of learning algorithms may include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

[0022] The AI ​​model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and the neural network calculation may be performed via calculations between the calculation results in the previous layer and the multiple weight values. The multiple weight values ​​assigned to each of the multiple neural network layers may be optimized by training the results of the AI ​​model. For example, multiple weight values ​​may be updated to reduce or minimize the loss or cost value obtained in the AI ​​model during the training process. The artificial neural network may include a deep neural network (DNN), and may be, for example, a convolutional neural network (CNN), a DNN, a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recursive DNN (BRDNN), or a deep Q network (DQN), but is not limited thereto.

[0023] Embodiments of the present disclosure will now be described more fully below with reference to the accompanying drawings so that those skilled in the art can easily implement the embodiments. However, the present disclosure may be implemented in different forms and should not be construed as limited to the embodiments set forth herein.

[0024] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0025] Figure 1is a diagram for describing an internal IoT environment in which the electronic device 100 is interconnected with external devices according to an embodiment of the present disclosure.

[0026] In the present disclosure, the electronic device 100 is described as a robot cleaner, but it can be any type of auxiliary robot or mobile device driven for user convenience, an augmented reality (AR) device, a virtual reality (VR) device, or the device can be a device that senses the surrounding environment and provides a certain service in a specific location or space. The electronic device 100 may be equipped with various types of sensors and neural network models for scanning the space and detecting objects in the space. For example, the electronic device 100 may include at least one of an image sensor (such as a camera), a light detection and ranging (LiDAR) sensor (such as a laser distance sensor (LDS)), or a time of flight (ToF) sensor. The electronic device 100 may include at least one model (such as a DNN, a CNN, a RNN, and a BRDNN, or any combination thereof).

[0027] The external device interconnected with the electronic device 100 may be a cloud server 200 and various types of Internet of Things (IoT) devices (300-1, 300-2, and 300-3). Figure 1 , the IoT device may be, but is not limited to, a housekeeping robot 300-1, a pet robot 300-2, a smart home camera 300-3, etc., and may be a device of the same type as the electronic device 100. The housekeeping robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may each use various types of sensors formed therein to scan a space and detect an object in the space.

[0028] According to an embodiment of the present disclosure, the electronic device 100, the housekeeper robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may each use the space scanning information or object information collected therefrom to generate and store a space map as space information about a space including at least one object. The electronic device 100, the housekeeper robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may send or receive multiple pieces of space scanning information or object information or space maps to each other and store them so that they can be shared with each other.

[0029] Because even devices in the same space scan the space and detect objects from different perspectives at different times depending on the location of each device, the performance or sensing range of each device, whether each device is stationary or moving, the behavior of each device, etc., the sensing information including images or audio obtained from one device can be usefully used to train AI models loaded in other devices.

[0030] According to an embodiment of the present disclosure, any one of the electronic device 100, the housekeeper robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may be a master device or a server device, and the remaining devices may be slave devices or client devices. The device corresponding to the master device or the server device may receive multiple pieces of spatial scanning information or object information or spatial maps from other IoT devices, and store and manage them. The device corresponding to the master device or the server device may classify, store and manage the received multiple pieces of information by location. For example, the device corresponding to the master device or the server device may classify, collect and manage the received multiple pieces of spatial scanning information or object information or spatial maps according to whether the received multiple pieces of information are related to the same space, the same zone or the same area. The device corresponding to the master device or the server device may update the stored first information with the second information corresponding to the same location, thereby maintaining the latest and accuracy of the information related to the corresponding location.

[0031] According to an embodiment of the present disclosure, the electronic device 100, the housekeeper robot 300-1, the pet robot 300-2, and the smart home camera 300-3 may send their multiple pieces of space scanning information or object information or space maps to the cloud server 200, so that the multiple pieces of space scanning information or object information or space maps may be stored and managed via the cloud server 200. For example, when the IoT device cannot send the space scanning information or object information or space map to the electronic device 100 because the IoT device is powered off or is performing a specific function, the electronic device 100 may request and receive the space scanning information or object information or space map from the cloud server 200.

[0032] Reference Figure 1 , the cloud server 200 can manage multiple pieces of spatial scanning information or object information or spatial maps received from the electronic device 100, the housekeeper robot 300-1, the pet robot 300-2 and the smart home camera 300-3, respectively, and monitor the space in the house. The cloud server 200 can store and manage multiple pieces of spatial scanning information or object information or spatial maps collected from multiple IoT devices for each registered user account or registered location. For example, the cloud server 200 can classify, collect and manage multiple pieces of spatial scanning information or object information or spatial maps from IoT devices based on whether they are in the same space or the same zone. The cloud server 200 can send information about the space in the house, such as a spatial map, in response to requests from the electronic device 100, the housekeeper robot 300-1, the pet robot 300-2 and the smart home camera 300-3 located in the house.

[0033] According to an embodiment of the present disclosure, an AI hub (e.g., an AI speaker) located in a house may receive multiple pieces of space scanning information or object information or space maps from IoT devices in the house, and store and manage them, instead of the cloud server 200. The AI ​​hub may store and manage multiple pieces of space scanning information or object information or space maps collected from multiple IoT devices by space or zone in the house.

[0034] According to an embodiment of the present disclosure, the AI ​​hub located in the house may store and manage multiple pieces of space scanning information or object information or space maps in conjunction with the cloud server 200. For example, the AI ​​hub may process multiple pieces of space scanning information or object information to generate or manage a space map, or convert data to protect personal information and send the data to the cloud server 200. The cloud server 200 may process information received from the AI ​​hub to store and manage multiple pieces of space scanning information or object information or space maps, and send them to the AI ​​hub.

[0035] According to an embodiment of the present disclosure, an electronic device 100 (such as a robot cleaner) can use a space map to perform tasks (such as cleaning). To accomplish this, the electronic device 100 can scan the space by using various types of sensors and update the space map with the latest space scanning information. The electronic device 100 can update the space map stored in the electronic device 100 by using not only directly sensed information but also some or all of the space maps received from the cloud server 200, the housekeeper robot 300-1, the pet robot 300-2, and the smart home camera 300-3 interconnected in the internal IoT environment.

[0036] For example, in order to clean a space in a house, when charging is completed at a charging station, a robot cleaner can perform cleaning by using a space map stored in the robot cleaner. The robot cleaner can use the same space map as that used most recently in order to clean the same space. However, since the state of the space at the time of previous cleaning is different from the current state of the space, it is desirable to reflect the latest information about objects located in the space in the space map in order to perform effective cleaning. To this end, starting from the charging station, the robot cleaner can travel along the main route in advance to directly collect information about objects in the space. However, when the robot cleaner travels in advance, the advance travel may take more time, and the battery may be further consumed due to such advance travel. In this case, the robot cleaner can update the space map stored in the robot cleaner by receiving the latest space map from another robot cleaner or at least one external device located in the same space.

[0037] The robot cleaner may utilize part or all of a spatial map received from an external device. The robot cleaner may update the spatial map using a spatial map received from a robot cleaner of the same type as itself, or using information about an object whose position is expected to change frequently. Even when a spatial map is received from a device of a different type from the robot cleaner, the robot cleaner may update its spatial map using part or all of a spatial map for the same space.

[0038] Figure 2a and Figure 2b is a flowchart for describing a spatial map according to an embodiment of the present disclosure.

[0039] Figure 2a A space map stored in the electronic device 100 as a robot cleaner and a hierarchical structure between a plurality of layers constituting the space map are shown.

[0040] Reference Figure 2a ,The spatial map may include but is not limited to a base layer, a semantic map layer, and a real-time layer, and layers may be added or omitted depending on the characteristics of the task.

[0041] The base layer provides information about the basic structure of the entire space, such as walls, columns, and passages. By processing three-dimensional (3D) point cloud data to match the 3D point cloud data with a coordinate system and store the position, the base layer can provide 3D information about the space, position information about objects, movement trajectory information, etc. The base layer is used as a base map and a geometric map.

[0042] The semantic map layer is a layer that provides semantic information on top of the base layer. The user of the electronic device 100 can assign semantic information (such as "room 1", "room 2", "prohibited zone", etc.) to the basic structure of the entire space in the base layer, and use the semantic information to perform tasks on the electronic device 100. For example, when the electronic device 100 is a robot cleaner, the user can set semantic information in the semantic map layer so that the robot cleaner can only clean "room 2" or not clean the "prohibited zone".

[0043] The real-time layer is a layer that provides information about at least one object in the space. The object may include static and dynamic objects. In the present disclosure, the real-time layer may include a plurality of layers based on the attribute information about the object, and have a hierarchical structure between the layers. Figure 2a , the real-time layer may include but is not limited to the first layer, the second layer and the third layer, and the number of layers may be increased or decreased according to the classification criteria for the attribute information about the object. Figure 2a As shown, the first layer may include system wardrobes and built-in cabinets, the second layer may include tables and sofas, and the third layer may include chairs.

[0044] Figure 2bVarious examples of a real-time layer including a plurality of layers based on attribute information about an object are shown.

[0045] The attribute information about the object may be information that can be an objective standard of the object (such as type, shape, size, height, etc.), or information that can be classified by combining multiple standards. In addition, since the attribute information about the object may vary depending on the user and the environment, the attribute information may be input by marking each object.

[0046] According to an embodiment of the present disclosure, when the attribute information about the object is the mobility level (ML) of the object, the first layer may include objects corresponding to ML 1, the second layer may include objects corresponding to ML 2 and ML 3, and the third layer may include objects corresponding to ML 4. The ML of the object may be determined by applying objective characteristics of the object to a predetermined classification standard for evaluating mobility. For example, ML 1 corresponds to an immovable object, ML 2 corresponds to an object that is movable but mostly remains stationary, ML 3 corresponds to an object that is movable but is occasionally moved, and ML 4 corresponds to an object that is movable and frequently moved.

[0047] According to an embodiment of the present disclosure, when the attribute information about an object is the position movement cycle of the object, the first layer may include objects whose positions have not been moved within a month, the second layer may include objects whose positions have been moved within a month, and the third layer may include objects whose positions have been moved within a week. Unlike ML based on objective characteristic classification of objects, even for the same object, the position movement cycle may be different depending on the user using the object or the environment in which the object is located. For example, object "A" may be an object frequently used by a first user but rarely used by a second user. Object "B" may be an object frequently used at a first location but rarely used at a second location.

[0048] According to an embodiment of the present disclosure, when the attribute information about an object is the height at which the object is located, the first layer may include objects corresponding to a height of 1m or less, the second layer may include objects corresponding to a height greater than or equal to 1m but less than or equal to 2m, and the third layer may include objects corresponding to a height exceeding 2m.

[0049] According to an embodiment of the present disclosure, classification criteria for multiple layers included in the real-time layer can be defined by the user. For example, the user can create a spatial map that reflects the characteristics of the task by setting a combination of multiple types of attribute information about the object for the classification criteria. For example, for a robot cleaner, since it usually moves below a height of 50 cm, there is no need to consider objects located above 1 m (such as lamps or photo frames hanging on the wall). Therefore, the user can directly set the classification criteria for classifying each layer, so that the first layer includes objects with ML 1 and located at 1 m or lower, the second layer includes objects with ML 2 or ML 3 and located at 1 m or lower, and the third layer includes objects with ML 4 and located at 1 m or lower.

[0050] Figure 3a , Figure 3b , Figure 3c and Figure 3d is a diagram for describing a method of using layers constituting a space map according to an embodiment of the present disclosure.

[0051] The spatial map used in each device may differ depending on the type of electronic device 100 and IoT device or the characteristics of the task. The electronic device 100 may utilize an existing spatial map stored in the electronic device 100, but when a change occurs in the space where the task is to be performed, the existing spatial map may be updated to reflect the corresponding change. The electronic device 100 may update the existing spatial map by receiving a spatial map that already reflects the change in the space from at least one external device. The electronic device 100 may generate a new spatial map based on the existing spatial map.

[0052] Reference Figure 3a , the electronic device 100 may load a previously stored spatial map (hereinafter, a first spatial map). The first spatial map is composed of a base layer, a first layer, a second layer, and a third layer. In the following, for ease of description, it is assumed that the first layer to the third layer include Figure 2b When the first spatial map was generated only a few minutes ago or there is no change in the space since the first spatial map was used, the electronic device 100 may generate a new spatial map (hereinafter, referred to as a second spatial map) using the first spatial map as it is and use the second spatial map to perform a new task.

[0053] Reference Figure 3b, the electronic device 100 may load the stored first spatial map. When performing a task, the electronic device 100 does not need information about objects with ML 4 that are frequently moved or uses only information about objects that have not been moved for a week or more, and the electronic device 100 may obtain a second spatial map by selecting a base layer, a first layer, and a second layer from the layers constituting the first spatial map or removing the third layer from the first spatial map.

[0054] Reference Figure 3c , the electronic device 100 may load the stored first spatial map. When performing a new task, the electronic device 100 only needs information about objects having ML 1 or uses only information about objects that have not been moved for a month or more, and the electronic device 100 may obtain a second spatial map by selecting a base layer and a first layer from layers constituting the first spatial map or removing a second layer and a third layer from the first spatial map.

[0055] Reference Figure 3d , the electronic device 100 may load the stored first spatial map. When the electronic device 100 performs a new task and needs to reflect the latest information about movable objects corresponding to ML 2, ML 3, and ML 4, the electronic device 100 may obtain a second spatial map by selecting a base layer and a first layer from the layers constituting the first spatial map or removing the second layer and the third layer from the first spatial map. Thereafter, the electronic device 100 may obtain a third spatial map by extracting the second layer and the third layer from a spatial map received from an external device and reflecting them in the second spatial map. Alternatively, at least one sensor provided in the electronic device 100 may be used to detect objects corresponding to ML 2, ML 3, and ML 4 and reflect them in the second spatial map to obtain a third spatial map.

[0056] Figure 4 is a flowchart of a method for obtaining a spatial map according to an embodiment of the present disclosure.

[0057] In operation S410, the electronic device 100 may obtain a first spatial map. The first spatial map may be composed of a plurality of layers based on attribute information about an object. The first spatial map may be generated by the electronic device 100 or received from a device outside the electronic device 100.

[0058] In operation S420, the electronic device 100 may determine whether the first space map needs to be updated. For example, the electronic device 100 may determine whether the first space map needs to be updated based on the characteristics of the task. The task refers to a task that the electronic device 100 is set to perform through a unique purpose of the electronic device 100 or a function that can be performed by the electronic device 100. Setting information associated with performing the task may be directly input to the electronic device 100 by the user, or may be sent to the electronic device 100 via a terminal (such as a mobile device or a dedicated remote controller). For example, when the electronic device 100 is a robot cleaner, the task of the robot cleaner may be cleaning a house or a zone set by the user, scheduled cleaning based on a schedule function, quiet mode cleaning, etc. When the information for performing the task is insufficient, the electronic device 100 may determine that the first space map needs to be updated. When the latest information about the objects in the space where the task is to be performed is required, the electronic device 100 may determine that the first space map needs to be updated. Alternatively, the electronic device 100 may determine whether the first space map needs to be updated according to the time elapsed since the first space map was obtained or the set update cycle. When there is no need to update the first spatial map, the electronic device 100 may utilize the first spatial map as the second spatial map for performing the task.

[0059] When the first spatial map needs to be updated, the electronic device 100 may obtain object information in operation S430. The electronic device 100 may directly collect spatial scanning information or object information by using at least one sensor. The electronic device 100 may receive part or all of the spatial map, or the spatial scanning information or object information from an external device.

[0060] In operation S440, the electronic device 100 may update the spatial scanning information or object information in the first spatial map by using the obtained spatial scanning information or object information. For example, for an object that moves frequently, the electronic device 100 may newly obtain information about the object and spatial scanning information about the location of the corresponding object, and update the newly obtained information about the object or spatial scanning information in the first spatial map so that the latest location information is reflected in the first spatial map.

[0061] In operation S450, the electronic device 100 may obtain the second spatial map. The electronic device 100 may obtain the second spatial map by using the first spatial map as it is, using the first spatial map in a modified form with some object information or some layers modified, or by updating the first spatial map.

[0062] In addition, the second spatial map for performing the task may be modified or generated as a map in an appropriate form and utilized depending on the function or characteristics of the task of the electronic device 100. For example, when the electronic device 100 is a robot cleaner, the robot cleaner may generate a navigation map based on the spatial map and perform cleaning along a moving path provided by the navigation map.

[0063] Figure 5 is a flowchart of a method of controlling the electronic device 100 by using spatial information according to an embodiment of the present disclosure.

[0064] In operation S510, the electronic device 100 may select an object that obstructs the task from objects located in the space corresponding to the task based on spatial information about the space including at least one object and the task that the electronic device 100 is set to perform. The spatial information may be a spatial map of the space including at least one object. The task that the electronic device 100 is set to perform may be determined by a user inputting a job to be processed to the electronic device 100 or by setting the electronic device 100 to complete the job through the purpose of the electronic device 100 or a function that can be performed by the electronic device 100. The user of the electronic device 100 may directly input task-related settings to the electronic device 100, or send task-related control commands to the electronic device 100 via a user terminal. For example, when the electronic device 100 is a robot cleaner, the task that the robot cleaner is set to perform may be cleaning the space designated by the user as the location where the task is to be performed at a specified time when the task is to be performed according to a specified operating mode. For example, the user of the electronic device 100 may set the robot cleaner to perform cleaning of a specific zone, scheduled cleaning according to a schedule function, or quiet mode cleaning according to an operating mode. In the following, reference is made to Figure 6 Provide a description.

[0065] Figure 6 is a detailed flowchart illustrating an operation of selecting an object that obstructs a task from objects located in a space corresponding to a task according to an embodiment of the present disclosure.

[0066] In operation S610, the electronic device 100 may obtain a spatial map of the space as spatial information about the space including at least one object. The electronic device 100 may obtain the spatial map based on at least one of a spatial map stored in the electronic device 100 or a spatial map received from an external device capable of communicating with the electronic device 100. For example, the electronic device 100 may obtain the spatial map based on the spatial map stored in the electronic device 100 or a spatial map received from an external device capable of communicating with the electronic device 100. Figure 4 The method for obtaining a spatial map is described to obtain a spatial map.

[0067] In operation S620, the electronic device 100 may analyze a prediction about a process of a task that the electronic device 100 is set to perform by using a spatial map. The electronic device 100 may identify the task that the electronic device is set to perform, and obtain a spatial map corresponding to a location where the task is to be performed. The electronic device 100 may predict and analyze various situations in which the task is performed on the spatial map. The electronic device 100 may compare and analyze the results of the prediction about the process of the task for a plurality of situations that are distinguished from each other based on the location of at least one branch point and at least one object on a virtual moving path of the electronic device 100 for performing the task.

[0068] According to an embodiment of the present disclosure, the electronic device 100 may generate a task processing model that considers whether its direction changes at each branch point or whether the position of each object moves. For example, the task processing model may take the position of the electronic device 100 as an input, and based on whether the direction at the corresponding position changes or whether the object is moved, each layer in the task processing model may correspond to the position of the branch point or object on the virtual moving path, and each node included in each layer may be the position of the electronic device 100 on the virtual moving path. The task processing model may be designed so that when moving from each node in each layer constituting the task processing model to a node in the next layer, a higher weight is applied to each node due to a lower degree of position overlap on the virtual moving path. When the task processing model reaches the last layer through at least one node in each layer included in all layers constituting the task processing model, the processing of the task can be determined to be completed. The virtual moving path for processing the task can be tracked based on the position corresponding to each node in the task processing model. The time required to perform the task, the amount of battery required to perform the task, and the degree of completion of the task can be analyzed for each tracked virtual moving path.

[0069] According to an embodiment of the present disclosure, the electronic device 100 can create various scenarios for processing tasks by considering whether its direction changes at each branch point or whether the position of each object is moved. For each scenario, the electronic device 100 can perform a simulation of the task on the obtained spatial map for analysis. For each scenario, the electronic device 100 can analyze the time required to perform the task, the amount of battery required to perform the task, the degree of completion of the task, etc.

[0070] For example, when the electronic device 100 is a robot cleaner, the robot cleaner may compare the time required for cleaning, the amount of battery required to perform cleaning, the degree of completion of cleaning, etc. for each virtual movement path tracked using the task processing model or for each simulation scenario.

[0071] In operation S630, the electronic device 100 may determine at least one object that obstructs the task based on an analysis result obtained by analyzing a prediction about the processing of the task. The electronic device 100 may select an optimal situation based on at least one criterion (i.e., at least one of the time required to process the task, the amount of battery required, or the degree of completion of the task). For example, the situation with the least time or the least amount of battery required to process the task, the situation with the highest degree of completion of the task, or the situation with the highest weighted average of weights assigned to each criterion may be selected as the optimal situation. The electronic device 100 may trace back the virtual moving path corresponding to the optimal situation to determine whether the position of the object on the virtual moving path has changed. At this time, at least one object whose position has been moved may be determined as an object that obstructs the task.

[0072] For example, when the electronic device 100 is a robot cleaner, the best case for the robot cleaner to perform cleaning may be selected based on at least one criterion, such as the time required to complete cleaning, the amount of battery required, the area cleaned out of the total area, etc. The robot cleaner may track back the virtual moving path corresponding to the best case to determine whether the position of the object on the virtual moving path has changed, and when the position of the object has changed, the robot cleaner may determine at least one object that contributes to the selection of the best case as an object that hinders the task.

[0073] Return to reference Figure 5 In operation S520, the electronic device 100 may provide the user of the electronic device 100 with object movement guide information corresponding to the attributes of the selected object, as described below with reference to Figures 7 to 13 described.

[0074] Figure 7 is a detailed flowchart illustrating an operation of providing object movement guide information corresponding to attribute information about an object selected as an object that obstructs a task according to an embodiment of the present disclosure.

[0075] In operation S710, the electronic device 100 may identify attribute information about an object selected as an object that needs to be moved in a space where a task is to be performed. For example, the electronic device 100 may identify attribute information about the object based on at least one of the type of the selected object, information about a layer to which the selected object belongs, or a tag of the selected object. The electronic device 100 may identify at least one piece of attribute information (such as ML, position movement period, height, size, etc.) of the selected object.

[0076] For all selected objects, the electronic device 100 may perform a move request process corresponding to the identified attribute information about the object. The move request process may include a process for providing the user with a result of performing a move request algorithm corresponding to the attribute information about the object and confirming a response from the user.

[0077] Reference Figure 7 , considering an example where the attribute information about the object identified is the ML of the object, in operation S720, the electronic device 100 may identify the ML of the object selected as the object that obstructs the task. For ease of description, it is assumed that the attribute information about the object is the ML of the object, but is not limited thereto, and may be other types of attribute information (such as the position movement period, height, size, etc. of the object). In addition, the classification of the identified attribute information is not limited to the following. Figure 7 There are three cases shown, and there may be an appropriate number of classifications according to each attribute information.

[0078] When the ML of the selected object is 4, in operation S730, the electronic device 100 may perform a first move request process as described below with reference to Figure 8 described.

[0079] Figure 8 A diagram for describing a first move request process according to an embodiment of the present disclosure.

[0080] In operation S810, the electronic device 100 may provide the user with an analysis result obtained by analyzing a prediction about the processing of a task and object movement guidance information. For example, the electronic device 100 may send an analysis result and object movement guidance information requesting the movement of an object that hinders the task to a user terminal, the analysis result being obtained by analyzing the difference between the result when the object that hinders the task is moved and the result when the object is not moved in the prediction result about the processing of the task. When the electronic device 100 is a robot cleaner, the electronic device 100 may provide the user with an analysis result obtained by analyzing the difference in how much the cleaning time is reduced or how much the cleanable area is increased when the movement of the selected object is reflected. The electronic device 100 may request the user to move the selected object, and provide the user with information about a suitable location to which the object can be moved.

[0081] In operation S820, the electronic device 100 may receive a user's response after providing an analysis result obtained by analyzing a prediction of the process regarding the task and object movement guide information. When the electronic device 100 confirms the response from the user, the first movement request process may be terminated. When the electronic device 100 receives a response regarding the movement of the object from the user terminal, or when a predetermined time has passed after providing a virtual simulation analysis result to the user terminal and requesting the movement of the object, the electronic device 100 may process the response regarding the movement of the object as having been received.

[0082] Fig. 9 is a diagram for describing an example of providing object movement guide information to a user according to an embodiment of the present disclosure.

[0083] The electronic device 100 may transmit the analysis result obtained by analyzing the prediction about the processing of the task and the object movement guide information to the user terminal 400. For example, the electronic device 100 may transmit the analysis result obtained by analyzing the difference between the result when the object hindering the task is moved and the result when the object is not moved in the prediction result about the processing of the task and the object movement guide information requesting the movement of the object hindering the task to the user terminal 400.

[0084] Fig. 9 A case is shown in which the electronic device 100 is a robot cleaner, and as a result of analyzing the prediction regarding the processing of the task, a bag on the living room floor is selected as an object that obstructs the task.

[0085] Reference Fig. 9 , the robot cleaner may send a message requesting the user to move the bag on the living room floor and an analysis result by analyzing the difference in how much the cleaning time is reduced when the movement of the bag selected as an object obstructing the task is reflected. The message sent to the user terminal 400 may also include information about a suitable location to which the bag may be moved.

[0086] The robot cleaner may receive a response from the user after transmitting the analysis result obtained by analyzing the prediction of the process on the task and the object movement guide information to the user terminal 400. The robot cleaner may determine that the package has been moved by receiving a response indicating that the request to move the package has been confirmed from the user terminal 400, or by processing the response as having been received from the user when a predetermined time has passed after requesting to move the package.

[0087] Fig.10 is a diagram for describing another example of providing object movement guide information to a user according to an embodiment of the present disclosure.

[0088] and Fig. 9 Unlike the embodiments of the present invention, the electronic device 100 may output the analysis result and the object movement guidance information obtained by analyzing the prediction of the processing of the task in the form of voice. For example, the electronic device 100 may send the analysis result and the object movement guidance information requesting the movement of the object that hinders the task to the user in the form of voice, and the analysis result is obtained by analyzing the difference between the result when the object that hinders the task is moved and the result when the object is not moved in the prediction result of the processing of the task.

[0089] Refer to above Fig. 9 Describing the situation, such as Fig.10As shown, when the movement of the bag selected as the object of the obstacle task is reflected, the robot cleaner can send a request to move the bag on the living room floor to the user in the form of voice, and an analysis result of the difference by analyzing how much the cleaning time is reduced. The robot cleaner can also send information about a suitable position to which the bag can be moved in the form of voice.

[0090] The robot cleaner may receive a user's response after outputting the analysis result obtained by analyzing the prediction of the process on the task and the object movement guide information in the form of voice. The robot cleaner may determine that the package has been moved by receiving a response from the user indicating that the request to move the package has been confirmed, or by processing the response as having been received from the user when a predetermined time has passed after the request to move the package.

[0091] Return to reference Figure 7 , when the ML of the selected object is 2 or 3, in operation S740, the electronic device 100 may perform a second move request process as described below with reference to Fig.11 described.

[0092] Fig.11 is a diagram for describing a second move request process according to an embodiment of the present disclosure.

[0093] When the second move request process starts, in operation S1110, the electronic device 100 may generate a 3D space map of the area where the selected object is located by using the space map. While generating the 3D space map, the electronic device 100 may reflect the size of the object selected as the object that obstructs the task, thereby ensuring an area where the selected object can be moved and a space in which the electronic device 100 will move.

[0094] In operation S1120, the electronic device 100 may select a candidate position on the generated 3D space map to which the object selected as the object obstructing the task will be moved. For example, since the candidate position is closer to the current position of the selected object and does not overlap with the main movement line of the user, the position may be determined to have a higher priority.

[0095] Fig.12 is a diagram for describing a process of selecting a candidate position to which a selected object is to be moved according to an embodiment of the present disclosure.

[0096] Reference Fig.12, when the electronic device 100 is a robot cleaner, a process of generating a 3D space map corresponding to the location where the robot cleaner is located and selecting candidate locations is shown, and the table selected as an object that obstructs the task will be moved to the candidate location. When generating the 3D space map, the robot cleaner can identify the area where the table can be moved based on the size of the table and the space where the robot cleaner will be placed. The robot cleaner can identify the area where the table can be moved by ensuring in advance the space that the robot cleaner needs to pass through to perform cleaning. In addition to locations used as passages through which the robot cleaner enters or leaves, the robot cleaner can select candidate locations for the table in the area where the table can be moved. According to a predetermined formula, the closer the candidate location of the table is to the current location of the table, the higher the score assigned to it. Refer to Fig.12 , it can be seen that the robot cleaner has selected three positions as candidate positions to which the table will be moved, and assigned scores of "0.9", "0.4" and "0.3" to them respectively. The number of candidate positions can be preset, and the criteria for the minimum score as a candidate position can be adjusted.

[0097] Return to reference Fig.11 In operation S1130, the electronic device 100 may obtain, for each candidate position, an image showing a state when the selected object is moved to the corresponding candidate position through image synthesis between the 3D space map and the selected object. By using the image synthesis technology, the electronic device 100 may generate an image showing a state when the object selected as the object obstructing the task is moved to the space corresponding to the candidate position on the 3D space map.

[0098] In operation S1140, the electronic device 100 may obtain an image evaluation result via an image evaluation model by inputting an image obtained when the object is moved into the image evaluation model. The image evaluation model may be a model that performs a certain evaluation based on the use of the location where the electronic device 100 is located or the user's settings. According to an embodiment of the present disclosure, the image evaluation model may be, but is not limited to, a model that takes the obtained composite image of the moved object as input and outputs a result value obtained by scoring the internal aesthetic value. According to an embodiment of the present disclosure, the image evaluation model may be a model that takes the obtained composite image of the moved object as input and outputs a result value obtained by scoring the security level of the corresponding space.

[0099] In operation S1150, the electronic device 100 may provide object movement guidance information according to the image evaluation result. For example, when an object selected as an object that hinders a task is moved to a candidate position, the electronic device 100 may provide a recommended position to the user according to the image evaluation result (such as how much internal aesthetic value the corresponding space has or how safe the space is). Based on the image evaluation result, the electronic device 100 may determine a candidate position with a high evaluation score among the candidate positions as a recommended position. Alternatively, the electronic device 100 may provide a certain number of candidate positions with high evaluation scores for moving the selected object to the user as recommended positions. The electronic device 100 may receive a user's response (in operation S1160) after providing a certain number of candidate positions with high evaluation scores. When the electronic device 100 confirms the response from the user, the second movement request process may be terminated. When the electronic device 100 receives a response about the movement of the object from the user terminal, or when a predetermined time has passed after providing a certain number of candidate positions with high evaluation scores to the user terminal and requesting the movement of the object, the electronic device 100 may process the response about the movement of the object as having been received.

[0100] Fig.13 is a diagram for describing an example of providing object movement guide information to a user according to an image evaluation result according to an embodiment of the present disclosure.

[0101] Reference Fig.13 , when the electronic device 100 is a robot cleaner, the robot cleaner can obtain images showing states where a table selected as an object obstructing the task is moved to a first candidate position, a second candidate position, and a third candidate position, respectively. By synthesizing an image representing a 3D space map with an image of the table selected as an object obstructing the task, the robot cleaner can generate an image showing the table located at each candidate position.

[0102] The robot cleaner may input the image for each candidate position obtained by image synthesis into the image evaluation model, thereby obtaining an image evaluation result through the image evaluation model. The robot cleaner may provide object movement guidance information according to the image evaluation result. Fig.13 , the robot cleaner may send a recommended position based on the image evaluation result to the user terminal 400. The robot cleaner may send an image corresponding to the recommended position among the images for each candidate position to the user terminal 400, and indicate a moving direction and distance from the current position to the recommended position on the corresponding image.

[0103] Return to reference Fig.11, in operation S1160, the electronic device 100 may receive a response from the user after providing the object movement guide information according to the image evaluation result. When the electronic device 100 confirms the response from the user, the second movement request process may be terminated. When the electronic device 100 receives a response regarding the movement of the object from the user terminal 400, or when a predetermined time has passed after providing the object movement guide information according to the image evaluation result to the user terminal 400 and requesting the movement of the object, the electronic device 100 may process the response regarding the movement of the object as having been received.

[0104] Return to reference Figure 7 , when the ML of an object is 1, the object is not movable, so even when a certain move request process is not executed, the move request process corresponding to the object can be processed as having been executed.

[0105] In operation S750, the electronic device 100 may check whether the move request process has been performed on all selected objects. The electronic device 100 may repeat the corresponding operation until it recognizes the attribute information about each object and the corresponding move request process is performed on all selected objects.

[0106] In addition, the electronic device 100 may generate a 3D spatial map of the area where the object selected as the object that obstructs the task is located to identify the area where the selected object can be moved and the space in which the electronic device 100 will move. The electronic device 100 may select at least one candidate position on the generated 3D spatial map, and the object selected as the object that obstructs the task will be moved to the at least one candidate position. The electronic device 100 may move the object selected as the object that obstructs the task to one of the candidate positions. For example, when the user has previously set the electronic device 100 not to receive object movement guide information, or when the object is sufficiently movable by the electronic device 100, the electronic device 100 may move the object selected as the object that obstructs the task to the candidate position. When moving the object in this manner, the electronic device 100 can obtain a moving path for performing the task by using the position to which the object is moved as the starting point of the moving path, or re-perform the process by reselecting the object that obstructs the task at the position where the object has been moved.

[0107] Return to reference Figure 5 , in operation S530, the electronic device 100 may obtain a moving path for performing the task based on a user's response corresponding to the object movement guide information.

[0108] Fig.14 is a detailed flowchart illustrating an operation of determining a moving path for performing a task according to an embodiment of the present disclosure.

[0109] In operation S1410, the electronic device 100 may identify a moving object among the selected objects based on a user's response corresponding to the object movement guide information. In addition to the actually moved object, when confirmation of the movement of the object is received from the user, the electronic device 100 may process the selected object as having been moved. When the electronic device 100 receives a response from the user indicating rejection of the movement of the selected object, or when there is no response from the user after a predetermined time has passed after requesting the movement of the selected object from the user terminal, the electronic device 100 may process the selected object as not having been moved.

[0110] In operation S1420, the electronic device 100 may determine a moving path of an object reflecting movement in a space corresponding to the task. For example, the electronic device 100 may generate a navigation map. The navigation map may provide a moving path used by the electronic device 100 to perform the task.

[0111] Return to reference Figure 5 , in operation S540, the electronic device 100 may drive the electronic device 100 according to the moving path. The electronic device 100 may perform a task while moving along the moving path provided by the navigation map.

[0112] Fig.15 and Fig.16 is a block diagram illustrating a configuration of an electronic device 100 using spatial information according to an embodiment of the present disclosure.

[0113] Reference Fig.15 According to an embodiment of the present disclosure, the electronic device 100 may include a memory 110, a processor 120, and a sensor unit 130, but is not limited thereto, and more general components may be added. For example, referring to Fig.16 In addition to the memory 110, the processor 120, and the sensor unit 130, the electronic device 100 may further include an input / output (I / O) interface 140, a communication interface 150, and a driver 160. Fig.15 and Fig.16 Describe the component.

[0114] According to an embodiment of the present disclosure, the memory 110 may store programs required for processing or control by the processor 120, and store data (e.g., spatial information, object information, spatial map, moving path, etc.) input to or output from the electronic device 100. The memory 110 may store instructions, data structures, and program codes readable by the processor 120. In an embodiment of the present disclosure, the operations performed by the processor 120 may be implemented by executing the code of the instructions or programs stored in the memory 110.

[0115] According to an embodiment of the present disclosure, the memory 110 may include a flash memory, a hard disk memory, a multimedia card micro memory, and a card-type memory (e.g., an SD card or an XD memory), and include a non-volatile memory and a volatile memory (such as a random access memory (RAM) or a static RAM (SRAM)), the non-volatile memory including at least one of a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a PROM, a magnetic memory, a magnetic disk, or an optical disk.

[0116] According to an embodiment of the present disclosure, the memory 110 may store one or more instructions and / or programs for controlling the electronic device 100 to use spatial information to perform tasks. For example, a spatial information management module, a task processing module, a driving module, etc. may be stored in the memory 110.

[0117] According to an embodiment of the present disclosure, the processor 120 may execute instructions stored in the memory 110 or a programmed software module to control operations or functions so that the electronic device 100 can perform tasks. The processor 120 may be composed of hardware components for performing arithmetic, logic and I / O operations and signal processing. The processor 120 may execute one or more instructions stored in the memory 110 to control all operations of the electronic device 100 to perform tasks using spatial information. The processor 120 may execute a program stored in the memory 110 to control the sensor unit 130, the I / O interface 140, the communication interface 150, and the driver 160.

[0118] For example, according to an embodiment of the present disclosure, the processor 120 may include, but is not limited to, at least one of a CPU, a microprocessor, a GPU, an application specific integrated circuit (ASIC), a DSP, a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), an AP, an NPU, or a dedicated AI processor designed with a hardware structure dedicated to processing an AI model. Each processor constituting the processor 120 may be a dedicated processor for performing a predetermined function.

[0119] According to an embodiment of the present disclosure, the AI ​​processor may perform calculation and control using an AI model in order to perform a task that the electronic device 100 is set to perform. The AI ​​processor may be manufactured in the form of a dedicated hardware chip for AI, or it may be manufactured as part of a general-purpose processor (e.g., CPU or AP) or a dedicated graphics processor (e.g., GPU) and installed in the electronic device 100.

[0120] According to an embodiment of the present disclosure, the sensor unit 130 may include a plurality of sensors configured to detect information about the environment around the electronic device 100. For example, the sensor unit 130 may include an image sensor 131, a LiDAR sensor 132, an infrared sensor 133, an ultrasonic sensor 134, a ToF sensor 135, a gyro sensor 136, etc., but is not limited thereto.

[0121] According to an embodiment of the present disclosure, the image sensor 131 may include a stereo camera, a mono camera, a wide-angle camera, a panoramic camera, or a 3D vision sensor.

[0122] The LiDAR sensor 132 may emit laser light onto a target to detect the distance and various physical properties of the object. The LiDAR sensor 132 may be used to detect surrounding objects, terrain features, etc., and model them into a 3D image.

[0123] The infrared sensor 133 may be an active infrared sensor that emits infrared light and detects changes by blocking light, or a passive infrared sensor that has no light emitter and detects only changes in infrared light received from an external source. For example, an infrared proximity sensor may be installed around the wheels of the electronic device 100 and may be used as a fall prevention sensor by emitting infrared light to the floor and receiving reflected infrared light.

[0124] The ultrasonic sensor 134 may measure the distance to an object using ultrasonic waves, and emit and detect ultrasonic pulses that convey information about the proximity of an object. The ultrasonic sensor 134 may be used to detect nearby objects and to detect transparent objects.

[0125] The ToF sensor 135 can obtain 3D effects, movement, and spatial information of an object by calculating the distance based on the time it takes for light emitted toward the object to bounce off the object. The ToF sensor 135 can provide advanced object recognition in complex spaces, dark places, and even obstacles in front of the eyes, allowing the electronic device 100 to avoid obstacles.

[0126] The gyro sensor 136 may detect an angular velocity. The gyro sensor 136 may be used to measure the position of the electronic device 100 or set its direction.

[0127] According to an embodiment of the present disclosure, the sensor unit 130 may be used to generate spatial information about a space including at least one object by using at least one sensor. For example, the electronic device 100 may obtain spatial information about a space including at least one object by obtaining spatial scanning information or object information using multiple sensors of the same or different types among the image sensor 131, the LiDAR sensor 132, the infrared sensor 133, the ultrasonic sensor 134, the ToF sensor 135, and the gyro sensor 136.

[0128] Reference Fig.16 , the electronic device 100 may further include an I / O interface 140, a communication interface 150, and a driver 160, and although Fig.16 Although not shown in the figure, the electronic device 100 may further include components such as a power supply.

[0129] The I / O interface 140 may include an input interface 141 and an output interface 143. The I / O interface 140 may have the input interface 141 and the output interface 143 separated from each other, or may be a single integrated component such as a touch screen. The I / O interface 140 may receive input information from a user and provide output information to the user.

[0130] The input interface 141 may refer to a device through which a user inputs data for controlling the electronic device 100. The input interface may include, for example, a keypad, a touch panel (capacitive overlay type, resistive overlay type, infrared beam type, surface acoustic wave type, integrated strain gauge type, piezoelectric type, etc.), etc. In addition, the input interface 141 may include a scroll wheel, a scroll wheel switch, etc., but is not limited thereto.

[0131] The output interface 143 may output an audio signal or a video signal or a vibration signal, and may include a display, an audio output interface, and a vibration motor.

[0132] The display may display information processed by the electronic device 100. For example, the display may display a user interface for receiving a user's manipulation. When the display and the touch panel form a layer structure to construct a touch screen, the display may be used as an input device as well as an output device. The display may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT LCD), an organic light emitting diode (OLED) display, a flexible display, or a 3D display. The electronic device 100 may include two or more displays according to the configuration implemented thereof.

[0133] The audio output interface may output audio data stored in the memory 110. The audio output interface may also output a sound signal related to a function performed by the electronic device 100. The audio output interface may include a speaker, a buzzer, and the like.

[0134] The vibration motor may output a vibration signal. For example, the vibration motor may output a vibration signal corresponding to the output of audio data or video data. When a touch is input to the touch screen, the vibration motor may output a vibration signal.

[0135] The communication interface 150 may include one or more components that enable the electronic device 100 to communicate with external devices such as the cloud server 200, IoT devices (e.g., 300-1, 300-2, and 300-3), and the user terminal 400. For example, the communication interface 150 may include a short-range wireless communication unit 151, a mobile communication unit 153, etc., but is not limited thereto.

[0136] The short-range wireless communication unit 151 may include, but is not limited to, a Bluetooth communication module, a Bluetooth low energy (BLE) communication module, a near field communication (NFC) module, a wireless local area network (WLAN) (or Wi-Fi) communication module, a ZigBee communication module, an Ant+ communication module, a Wi-Fi Direct (WFD) communication module, an ultra-wideband (UWB) communication module, an infrared data association (IrDA) communication module, a microwave (uWave) communication module, and the like.

[0137] The mobile communication unit 153 transmits or receives wireless signals to or from at least one of a base station, an external terminal or a server on a mobile communication network. In this case, the wireless signals may include voice call signals, video call signals or various forms of data according to the transmission and reception of text / multimedia messages.

[0138] The driver 160 may include components for driving (traveling) the electronic device 100 and operating devices inside the electronic device 100. When the electronic device 100 is a robot cleaner, the driver 160 may include a suction unit, a traveling unit, etc., but is not limited thereto.

[0139] The suction unit is used to collect dust on the floor while sucking air, and may include a rotating brush or broom, a rotating brush motor, an air suction port, a filter, a dust collection chamber, an air discharge port, etc., but is not limited thereto. The suction unit may be further equipped with a structure that rotates a brush that can sweep dust in corners.

[0140] The traveling unit may include a motor for rotating and driving wheels installed in the electronic device 100 and a timing belt installed to transmit power generated by the wheels, but is not limited thereto.

[0141] According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to select an object that hinders a task from objects located in the space corresponding to the task based on spatial information about a space including at least one object obtained via the sensor unit 130 and a task that the electronic device 100 is set to perform.

[0142] The processor 120 may execute one or more instructions stored in the memory 110 to obtain a spatial map as spatial information. The processor 120 may execute one or more instructions stored in the memory 110 to obtain a spatial map based on at least one of a first spatial map stored in the electronic device 100 or a second spatial map received from an external device communicating with the electronic device 100. The spatial map may include a plurality of layers based on attribute information about an object.

[0143] The processor 120 may execute one or more instructions stored in the memory 110 to analyze the predicted result of the processing of the task by using the obtained spatial map. According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to compare and analyze the predicted result of the processing of the task for a plurality of moving paths that are distinguished from each other based on at least one branch point and the position of at least one object on the virtual moving path of the electronic device 100 for performing the task. The processor 120 may execute one or more instructions stored in the memory 110 to determine at least one object that obstructs the task based on the analysis result.

[0144] According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to provide the user of the electronic device 100 with object movement guide information corresponding to the attribute information about the object selected as the object that obstructs the task. The processor 120 may provide the user with the object movement guide information by identifying the attribute information about the object selected as the object that obstructs the task and performing a movement request process corresponding to the identified attribute information about the selected object.

[0145] According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to send the analysis result and the object movement guidance information obtained by analyzing the prediction of the processing of the task to the user terminal 400 of the user via the communication interface 150. According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to select the candidate position to which the selected object will be moved on the 3D space map of the area where the object selected as the object that hinders the task is located. The processor 120 may obtain an image showing the state when the selected object is moved to the corresponding candidate position for each candidate position, and evaluate the obtained image via the image evaluation model by inputting the image into the image evaluation model. The processor 120 may send the object movement guidance information according to the image evaluation result to the user terminal 400 via the communication interface 150.

[0146] According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to determine a moving path for performing a task based on a user's response corresponding to the object movement guide information. According to an embodiment of the present disclosure, the processor 120 may identify a moving object among objects selected as objects that obstruct the task based on the user's response, and obtain a moving path that reflects the moving object in a space corresponding to the task.

[0147] According to an embodiment of the present disclosure, the processor 120 may execute one or more instructions stored in the memory 110 to drive the electronic device 100 according to the determined moving path. When an unexpected object or an object determined to have moved is detected while the electronic device 100 is traveling along the moving path, the processor 120 may bypass the object and then drive the electronic device 100 along the moving path again, or notify the user of the presence of the corresponding object.

[0148] In addition, the embodiments of the present disclosure may be implemented in the form of a recording medium including instructions executable by a computer (such as a program module executable by a computer). Computer-readable recording media may be any available media accessible by a computer, and include volatile and non-volatile media and removable and non-removable media. In addition, computer-readable recording media may include computer storage media and communication media. Computer storage media include volatile and non-volatile media and removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Communication media may typically include computer-readable instructions, data structures, program modules, or other data in a modulated data signal.

[0149] The computer-readable storage medium may be provided in the form of a non-transitory storage medium. In this regard, the term "non-transitory storage medium" simply means that the storage medium does not include a signal (e.g., an electromagnetic wave) and is a tangible device, and the term does not distinguish between a location where data is semi-permanently stored in the storage medium and a location where data is temporarily stored in the storage medium. For example, a "non-transitory storage medium" may include a buffer for temporarily storing data.

[0150] According to an embodiment of the present disclosure, the method according to an embodiment of the present disclosure (if provided) may be included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a computer-readable storage medium (e.g., a compact disc ROM (CD-ROM)), or distributed online (e.g., downloaded or uploaded) via an application store, or distributed directly between two user devices (e.g., smart phones). For online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be at least temporarily stored or temporarily generated in a computer-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0151] According to an embodiment of the present disclosure, a method for controlling an electronic device 100 by using spatial information is provided. The method for controlling an electronic device 100 by using spatial information may include: in operation S510, based on spatial information about a space including at least one object and a task that the electronic device 100 is set to perform, selecting an object that obstructs the task from objects located in the space corresponding to the task. In addition, the method for controlling an electronic device 100 by using spatial information may include: in operation S520, providing a user of the electronic device 100 with object movement guide information corresponding to attribute information about the selected object. In addition, the method for controlling an electronic device 100 by using spatial information may include: in operation S530, determining a moving path for performing the task based on a response of a user corresponding to the object movement guide information. In addition, the method for controlling an electronic device 100 by using spatial information may include: in operation S540, driving the electronic device 100 according to the determined moving path.

[0152] In addition, according to an embodiment of the present disclosure, selecting an object that hinders a task in operation S510 includes: in operation S610, obtaining a spatial map as spatial information. In addition, selecting an object that hinders a task in operation S510 includes: in operation S620, analyzing a prediction about a process of a task by using the obtained spatial map. In addition, selecting an object that hinders a task in operation S510 includes: in operation S630, determining at least one object that hinders a task based on an analysis result obtained by the analysis.

[0153] In addition, obtaining the space map in operation S610 includes obtaining the space map based on at least one of a first space map stored in the electronic device 100 or a second space map received from an external device communicating with the electronic device 100 .

[0154] Furthermore, analyzing the prediction in operation S620 includes comparing and analyzing results of the prediction regarding the process of the task with respect to a plurality of moving paths distinguished from each other based on at least one branch point and a position of at least one object on a virtual moving path of the electronic device 100 for performing the task.

[0155] In addition, according to an embodiment of the present disclosure, providing the object movement guide information to the user of the electronic device 100 at operation S520 includes: identifying the attribute information about the selected object at operation S710. In addition, providing the object movement guide information to the user of the electronic device 100 at operation S520 includes: providing the object movement guide information to the user by performing a movement request process corresponding to the identified attribute information about the selected object at operations S720, S730, S740, and S750.

[0156] Furthermore, providing to the user in operation S730 includes transmitting, in operation S810 , a result of analyzing the prediction regarding the process of the task and the object movement guide information to a user terminal of the user.

[0157] In addition, in operation S740, the user is provided with: in operations S1110 and S1120, a candidate position to which the selected object will be moved is selected on the 3D spatial map of the area where the selected object is located. In addition, in operation S740, the user is provided with: in operation S1130, an image showing the state when the selected object is moved to the corresponding candidate position is obtained for each candidate position. In addition, in operation S740, the user is provided with: in operation S1140, the image obtained is evaluated by the image evaluation model by inputting the image into the image evaluation model. In addition, in operation S740, the user is provided with: in operation S1150, object movement guidance information is sent to the user terminal of the user according to the result of the evaluation of the obtained image.

[0158] In addition, according to an embodiment of the present disclosure, determining the moving path in operation S530 includes: in operation S1410, identifying a moving object among the selected objects based on a user's response. In addition, determining the moving path in operation S530 includes: in operation S1420, obtaining a moving path reflecting the moving object in a space corresponding to the task.

[0159] Furthermore, according to an embodiment of the present disclosure, the electronic device 100 is a robot cleaner.

[0160] According to an embodiment of the present disclosure, there may be provided a computer-readable recording medium having recorded thereon a program for executing the above-described method.

[0161] According to an embodiment of the present disclosure, an electronic device 100 using spatial information is provided. The electronic device 100 using spatial information includes a memory 110, a processor 120 configured to execute one or more instructions stored in the memory 110, and a sensor unit 130. The processor 120 executes one or more instructions to select an object that hinders a task from objects located in a space corresponding to a task based on spatial information about a space including at least one object obtained via the sensor unit 130 and a task that the electronic device 100 is set to perform. In addition, the processor 120 executes one or more instructions to provide a user of the electronic device 100 with object movement guide information corresponding to attribute information about the selected object. In addition, the processor 120 executes one or more instructions to determine a moving path for performing the task based on a response of a user corresponding to the object movement guide information. In addition, the processor 120 executes one or more instructions to drive the electronic device 100 according to the determined moving path.

[0162] In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to obtain a space map as space information. In addition, the processor 120 executes one or more instructions to analyze a prediction of processing of a task by using the obtained space map. In addition, the processor 120 executes one or more instructions to determine at least one object that obstructs the task based on an analysis result obtained by the analysis.

[0163] In addition, the processor 120 executes one or more instructions to obtain a spatial map based on at least one of a first spatial map stored in the electronic device 100 or a second spatial map received from an external device capable of communicating with the electronic device 100 .

[0164] In addition, the processor 120 executes one or more instructions to compare and analyze predicted results regarding processing of a task for a plurality of movement paths distinguished from each other based on at least one branch point and the position of at least one object on a virtual movement path of the electronic device 100 for performing the task.

[0165] Furthermore, the spatial map includes a plurality of layers based on attribute information about objects.

[0166] In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to identify attribute information about the selected object. In addition, the processor 120 executes one or more instructions to provide the user with object movement guidance information by executing a movement request process corresponding to the identified attribute information about the object.

[0167] In addition, the electronic device 100 may further include a communication interface 150. In addition, the processor 120 executes one or more instructions to transmit the analysis result and the object movement guide information about the prediction of the process of the task to the user terminal 400 of the user via the communication interface 150.

[0168] In addition, the electronic device 100 may further include a communication interface 150. In addition, the processor 120 executes one or more instructions to select a candidate position to which the selected object will be moved on the 3D spatial map of the area where the selected object is located. In addition, the processor 120 executes one or more instructions to obtain an image showing the state when the selected object is moved for each candidate position. In addition, the processor 120 executes one or more instructions to evaluate the obtained image via the image evaluation model by inputting the image into the image evaluation model. In addition, the processor 120 executes one or more instructions to send the object movement guide information to the user terminal 400 of the user via the communication interface 150 according to the result of the evaluation of the image.

[0169] In addition, according to an embodiment of the present disclosure, the processor 120 executes one or more instructions to identify a moving object among the selected objects based on the user's response. In addition, the processor 120 executes one or more instructions to obtain a moving path reflecting the moving object in the space corresponding to the task.

[0170] Furthermore, according to an embodiment of the present disclosure, the electronic device 100 is a robot cleaner.

[0171] The above description of the present disclosure is provided for illustration, and it will be understood by those skilled in the art that changes in form or detail may be easily made therein without departing from the technical ideas or basic features of the present disclosure. Therefore, it should be understood that the above-described embodiments of the present disclosure and all aspects thereof are merely exemplary and non-restrictive. For example, each component defined as an integrated component may be implemented in a distributed manner, and similarly, components defined as separate components may be implemented in an integrated form.

[0172] While the present disclosure has been shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. A method for controlling an electronic device (100) by using spatial information, the method comprising: selecting an object that obstructs the task from objects located in the space corresponding to the task based on space information about a space including at least one object and a task that the electronic device (100) is set to perform (S510); providing object movement guide information corresponding to attribute information about a selected object to a user of the electronic device (100) (S520); determining a movement path for performing the task based on a user's response corresponding to the object movement guide information (S530); and The electronic device (100) is driven according to the determined moving path (S540).

2. The method according to claim 1, wherein: Selecting an object that hinders the task (S510) includes: Obtaining a spatial map as spatial information (S610); analyzing predictions regarding the processing of the task by using the obtained spatial map (S620); and At least one object that hinders the task is determined based on the result of the analysis ( S630 ).

3. The method according to claim 1 or claim 2, wherein: Obtaining a space map (S610) includes obtaining a space map based on at least one of a first space map stored in the electronic device (100) or a second space map received from an external device communicating with the electronic device (100).

4. The method according to any one of claims 1 to 3, wherein: Analyzing the prediction (S620), including: comparing and analyzing the results of the prediction on the processing of the task for a plurality of movement paths distinguished from each other based on at least one branch point and the position of at least one object on a virtual movement path of the electronic device (100) for performing the task.

5. The method according to any one of claims 1 to 4, wherein: Providing object movement guidance information (S520) includes: identifying attribute information about a selected object (S710); and The object movement guide information is provided to the user by performing a movement request process corresponding to the recognized attribute information about the selected object (S720, S730, S740, S750).

6. The method according to any one of claims 1 to 5, wherein: Providing object movement guide information ( S730 ) includes: transmitting an analysis result obtained by analyzing a prediction regarding a process of the task and the object movement guide information to a user terminal of a user ( S810 ).

7. The method according to any one of claims 1 to 6, wherein: Providing object movement guidance information (S740) includes: Selecting a candidate position to which the selected object is to be moved on a three-dimensional 3D space map of the area where the selected object is located ( S1110 , S1120 ); For each candidate position, obtaining an image showing a state when the selected object is moved to the corresponding candidate position (S1130); evaluating the obtained image via the image evaluation model by inputting the image to the image evaluation model ( S1140 ); and The object movement guide information is transmitted to the user terminal of the user according to the result of the evaluation of the obtained image (S1150).

8. The method according to any one of claims 1 to 7, wherein: Determining a moving path (S530) includes: Based on the user's response, identifying a moving object among the selected objects (S1410); and A movement path of an object reflecting movement in a space corresponding to the task is obtained (S1420).

9. The method according to any one of claims 1 to 8, wherein: The electronic device (100) is a robot cleaner.

10. A computer-readable recording medium having a program recorded thereon, the program causing at least one processor to control the at least one processor to perform the following operations when executed by the at least one processor: selecting, based on spatial information about a space including at least one object and a task that the electronic device is set to perform, an object that obstructs the task from objects located in the space corresponding to the task; providing a user of the electronic device with object movement guide information corresponding to attribute information about a selected object; determining a movement path for performing the task based on a user's response corresponding to the object movement guidance information; as well as The electronic device is driven according to the determined moving path.

11. An electronic device (100) using spatial information, the electronic device (100) comprising: A memory (110) storing one or more instructions; A processor (120) configured to execute the one or more instructions stored in the memory (110); as well as A sensor unit (130), The processor (120) is configured to execute the one or more instructions to perform the following operations: Based on spatial information about a space including at least one object obtained via a sensor unit (130) and a task that the electronic device (100) is set to perform, selecting an object that obstructs the task from objects located in the space corresponding to the task, providing a user of the electronic device (100) with object movement guide information corresponding to attribute information about a selected object, determining a movement path for performing the task based on a user's response corresponding to the object movement guidance information, and The electronic device (100) is driven according to the determined moving path.

12. The electronic device (100) according to claim 11, wherein: The processor (120) is further configured to: obtaining a spatial map as spatial information, analyzing a prediction about the processing of the task by using the obtained spatial map, and At least one object that obstructs the task is determined based on the results of the analysis.

13. The electronic device (100) according to claim 11 or claim 12, wherein: The processor (120) is further configured to: identifying attribute information about the selected object, and The object movement guide information is provided to the user by performing a movement request process corresponding to the recognized attribute information about the selected object.

14. The electronic device (100) according to any one of claims 11 to 13, further comprising: Communication interface (150), The processor (120) is further configured to send the analysis result and the object movement guidance information obtained by analyzing the prediction of the process of the task to the user terminal (400) of the user via the communication interface (150).

15. The electronic device (100) according to any one of claims 11 to 14, wherein: The processor (120) is further configured to: identifying a moving object among the selected objects based on the user's response, and A movement path of an object reflecting movement in a space corresponding to the task is obtained.

Images