Electronic device for generating map and control method thereof

Abstract

An electronic device and a control method thereof are provided. The electronic device according to an embodiment of the present disclosure comprises: a communication interface; a memory that stores at least one computer program; and one or more processors communicatively connected to the communication interface and the memory, wherein the one or more computer programs comprise computer-executable instructions that, when executed by the one or more processors, control the electronic device: to acquire a two-dimensional map for an indoor space, which has been generated on the basis of sensing data and driving data acquired by at least one external device; simplify the acquired two-dimensional map on the basis of the driving data; and generate a three-dimensional map on the basis of the simplified two-dimensional map.

Description

Electronic device for generating a map and method for controlling the same

[0001] The present disclosure relates to an electronic device for generating a map and a method for controlling the same, and more particularly, to an electronic device for generating a map for an indoor space in which a robot moves and a method for controlling the same.

[0002] Recently, technologies have been developed to create maps of indoor spaces so that various types of robots (e.g., robot vacuum cleaners, serving robots, guide robots, etc.) can navigate indoor spaces.

[0003] Previously, a 2D map generated by a robot was defined as the floor, and a 3D map was created by erecting a wall vertically on the line identified as a wall. However, the electronic device did not perform any separate simplification of the map, resulting in complex and inaccurate 3D maps. For example, walls were often displayed as complex, with at least one uneven curve or short, disconnected line, non-wall areas were displayed as walls, and diagonal walls were displayed as steps.

[0004] Therefore, a method is required to create a more accurate three-dimensional map closer to the indoor space based on the two-dimensional map detected and created by the robot.

[0005] The above information is provided solely as background information to aid in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether the above content constitutes prior art in connection with the present disclosure.

[0006] Aspects of the present disclosure address at least the problems and / or drawbacks mentioned above, and provide at least the advantages described below. Accordingly, one aspect of the present invention provides an electronic device for generating a map of an indoor space in which a robot navigates, and a control method thereof.

[0007] Additional aspects will be partly explained in the following description, partly will be obvious from the description, or may be learned by practice of the presented embodiments.

[0008] According to one embodiment of the present disclosure, an electronic device is provided. The electronic device includes: a communication interface; a memory storing at least one computer program; and one or more processors communicatively connected to the communication interface and the memory; wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors, control the electronic device to obtain a two-dimensional map of the indoor space generated based on sensing data and driving data acquired by at least one external device, simplify the obtained two-dimensional map based on the driving data, and generate a three-dimensional map based on the simplified two-dimensional map.

[0009] According to one embodiment of the present disclosure, a control method performed by an electronic device is provided. The method includes the steps of: obtaining a two-dimensional map of an indoor space generated based on sensing data and driving data acquired by at least one external device; simplifying the obtained two-dimensional map based on the driving data; and generating a three-dimensional map based on the simplified two-dimensional map.

[0010] According to one embodiment of the present disclosure, a non-transitory computer-readable medium is provided that stores computer instructions that, when executed by one or more processors included in an electronic device, cause the electronic device to perform operations. The operations include: obtaining a two-dimensional map of the indoor space generated based on sensing data and driving data acquired by at least one external device; simplifying the obtained two-dimensional map based on the driving data; and generating a three-dimensional map based on the simplified two-dimensional map.

[0011] Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the present disclosure taken in conjunction with the accompanying drawings.

[0012] The above and other aspects, features and advantages of specific embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0013] FIG. 1 is a diagram illustrating a system for generating a map for an indoor space according to one embodiment of the present disclosure;

[0014] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure;

[0015] FIG. 3 is a flowchart illustrating a method for generating a three-dimensional map by simplifying a two-dimensional map according to one embodiment of the present disclosure;

[0016] FIG. 4 is a diagram illustrating a two-dimensional grid map according to one embodiment of the present disclosure;

[0017] FIG. 5 is a flowchart illustrating a method for converting a two-dimensional grid map into a two-dimensional linear vector map according to one embodiment of the present disclosure;

[0018] FIG. 6a, FIG. 6b, FIG. 6c and FIG. 6d are drawings for explaining a method of converting a two-dimensional grid map into a two-dimensional linear vector map according to one embodiment of the present disclosure.

[0019] FIG. 7 is a flowchart illustrating a method for simplifying a two-dimensional linear vector map according to one embodiment of the present disclosure;

[0020] FIGS. 8a, 8b, 9a, 9b, 9c and 9d are drawings illustrating a method for simplifying a two-dimensional linear vector map according to one embodiment of the present disclosure.

[0021] FIG. 10 is a drawing for explaining a method for correcting a room area of ​​a two-dimensional map according to one embodiment of the present disclosure;

[0022] FIG. 11 is a sequence diagram for modifying a method for modifying a two-dimensional map or a three-dimensional map according to one embodiment of the present disclosure;

[0023] FIG. 12 is a sequence diagram illustrating a method for a system including a server for generating a map for an indoor space according to one embodiment of the present disclosure to generate a map for an indoor space;

[0024] FIG. 13 is a diagram illustrating a UI for selecting a degree of simplification of a two-dimensional map according to one embodiment of the present disclosure;

[0025] FIGS. 14A, 14B and 14C are diagrams illustrating a UI and a 3D map for selecting a type of obstacle included in a 3D map according to one embodiment of the present disclosure;

[0026] FIGS. 15A, 15B and 15C are diagrams illustrating a UI for modifying a two-dimensional map or a three-dimensional map according to one embodiment of the present disclosure.

[0027] FIG. 16 is a diagram illustrating a UI for inquiring whether to perform a simplification operation on some areas among a plurality of areas included in a two-dimensional map according to one embodiment of the present disclosure, and

[0028] FIG. 17 is a control method of an electronic device for generating a map according to one embodiment of the present disclosure.

[0029] It should be noted that throughout the drawings, similar reference numbers are used to describe identical or similar elements, features and structures.

[0030] The following description, with reference to the accompanying drawings, is provided to assist in a comprehensive understanding of various embodiments of the disclosure as defined by the claims and their equivalents. While it includes numerous specific details to aid understanding, these are to be considered merely exemplary. Accordingly, those skilled in the art will recognize that various modifications and variations can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, descriptions of known functions and configurations may be omitted for clarity and brevity.

[0031] The terms and words used in the following description and claims are not to be construed as limited by their bibliographic meanings, but rather have been used by the inventors solely to facilitate a clear and consistent understanding of the present invention. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present invention is provided for illustrative purposes only and is not intended to limit the present invention as defined by the appended claims and their equivalents.

[0032] The singular forms "a," "an," and "the" should be understood to include plural referents unless the context clearly dictates otherwise. Thus, for example, a reference to "a surface of a part" includes a reference to one or more of those surfaces.

[0033]

[0034] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0035] The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0036] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0037] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0038] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0039] The terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0040] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0041] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0042] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0043] It should be understood that each block of the flowchart and the combination of flowcharts can be performed by one or more computer programs containing instructions. The entirety of one or more computer programs may be stored in a single memory device, or the one or more computer programs may be divided into different parts stored in multiple different memory devices.

[0044] Any function or operation described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing, and may include circuits such as an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphics processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a Wi-Fi chip, a Bluetooth® chip, a global positioning system (GPS) chip, a near field communication (NFC) chip, a connection chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display drive integrated circuit (IC), an audio codec chip, a universal serial bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system on a chip (SoC), an integrated circuit (IC), etc. Hereinafter, the operating principle and embodiments of the present invention will be described with reference to the accompanying drawings.

[0045] FIG. 1 is a diagram illustrating a system for generating a map of an indoor space according to one embodiment of the present disclosure. As illustrated in FIG. 1, the system for generating a map of an indoor space may include an electronic device (100) and a robot vacuum cleaner (200).

[0046] At this time, the electronic device (100) can be carried by the user or placed in the user's home or office, etc. The electronic device (100) may be a smart phone as illustrated in FIG. 1, but is not limited thereto, and may be implemented as a personal computer, a terminal, a portable telephone, a handheld device, a wearable device, etc. Alternatively, the electronic device (100) may of course be implemented as a separate server device.

[0047] In addition, the robot cleaner (200) may be implemented as a device capable of performing a cleaning operation while driving in an indoor space, but this is only one embodiment, and it is obvious that the robot cleaner (200) may be implemented as a robot capable of driving in an indoor space (e.g., a serving robot, a guide robot, etc.). The robot cleaner (200) may include not only the robot cleaner (200) but also a station for charging or placing the robot cleaner (200).

[0048] The robot cleaner (200) can acquire sensing data and driving data while driving in an indoor space. Specifically, the robot cleaner (200) can acquire sensing data containing information about the indoor space in which the robot cleaner (200) drives through various sensors (e.g., lidar sensors, ToF sensors, cameras, etc.), and can acquire driving data containing driving information about the robot cleaner (200) driving in the indoor space.

[0049] The robot vacuum cleaner (200) can transmit acquired sensing data and driving data to an electronic device (100).

[0050] The electronic device (100) can obtain a two-dimensional map of an indoor space based on sensing data acquired from the robot cleaner (200). At this time, the two-dimensional map is a two-dimensional map representing information about the indoor space, and may include a two-dimensional grid map, a two-dimensional linear vector map, a map combining a two-dimensional grid map and a two-dimensional linear vector, or a two-dimensional map newly generated using a two-dimensional map and a two-dimensional linear vector and displayed on a display. Specifically, the electronic device (100) can obtain a two-dimensional grid map of an indoor space based on sensing data. At this time, the two-dimensional grid map may be a two-dimensional map that represents the surrounding space as a grid of the same size and indicates the presence or absence of an object (e.g., a wall, an obstacle, etc.) in each grid. In addition, the electronic device (100) can convert the two-dimensional grid map into a two-dimensional linear vector map in a multi-line form. At this time, the two-dimensional linear vector map may be a two-dimensional map including a linear vector in the form of a multiline connecting linear vector candidates extracted from a binary image.

[0051] The electronic device (100) simplifies a two-dimensional map acquired based on driving data. Specifically, the electronic device (100) can remove noise included in the two-dimensional linear vector map. In addition, the electronic device (100) can simplify a polyline included in the two-dimensional linear vector map from which noise has been removed.

[0052] In addition, the electronic device (100) can correct a two-dimensional map based on a two-dimensional linear vector map that simplifies the multiple lines. At this time, the electronic device (100) can correct a room area included in the two-dimensional map based on the multiple lines included in the two-dimensional linear vector map.

[0053] In addition, the electronic device (100) can generate a three-dimensional map of an indoor space based on a simplified two-dimensional linear vector map. In addition, the electronic device (100) can display the three-dimensional map of the indoor space on a display.

[0054] Additionally, the electronic device (100) can display a UI on the display for modifying a two-dimensional map or a three-dimensional map according to a user input, and can modify the two-dimensional map or the three-dimensional map according to a user command input through the UI.

[0055] Meanwhile, in one embodiment of the present disclosure, it has been described that the electronic device (100) performs a two-dimensional map creation operation, a two-dimensional map simplification operation, and a three-dimensional map creation operation, but this is only one embodiment, and at least some of these operations may be implemented in a robot cleaner (200) or another device (e.g., a server).

[0056] As described above, by simplifying a two-dimensional map using the robot's driving data and generating a three-dimensional map, a map with high visibility can be provided to the user.

[0057] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure. As illustrated in FIG. 2, the electronic device (100) may include a communication interface (110), a display (120), a memory (130), a user input unit (140), and at least one processor (150). However, this is merely an example, and it is understood that some components may be removed or added depending on the type of the electronic device (100). For example, if the electronic device (100) is implemented as a set-top box, the electronic device (100) may not include a display (120).

[0058] The communication interface (110) includes at least one circuit and can communicate with various types of external devices or servers. The communication interface (110) can include at least one of a BLE (Bluetooth Low Energy) module, a Wi-Fi communication module, a cellular communication module, a 3G (third generation) mobile communication module, a 4G (fourth generation) mobile communication module, a 4th generation LTE (Long Term Evolution) communication module, and a 5G (fifth generation) mobile communication module.

[0059] In particular, the communication interface (110) can receive sensing data and driving data from an external robot cleaner (200). Alternatively, the communication interface (110) can receive a two-dimensional grid map generated based on the sensing data and driving data from the external cleaner (200).

[0060] In addition, the communication interface (110) can transmit a control command to perform a cleaning operation to the robot cleaner (200). In addition, the communication interface (110) can receive information about the cleaning operation from the robot cleaner (200).

[0061] In one or more embodiments, the communication interface (110) may communicate with an external server. At this time, when information regarding a two-dimensional grid map is received from the robot cleaner (200) and a user input for generating a three-dimensional map is received, the communication interface (110) may transmit a signal to the server for requesting the generation of a three-dimensional map. In addition, the communication interface (110) may receive a three-dimensional map from the server (1200).

[0062] In one or more embodiments, the communication interface (110) may have different modules for communicating with the robot cleaner (200) and for communicating with an external server.

[0063] The display (120) may include various types of display panels such as, but not limited to, an LCD (Liquid Crystal Display) panel, an OLED (Organic Light Emitting Diodes) panel, an AM-OLED (Active-Matrix Organic Light-Emitting Diode), an LcoS (Liquid Crystal on Silicon), a QLED (Quantum dot Light-Emitting Diode), a DLP (Digital Light Processing), a PDP (Plasma Display Panel) panel, an inorganic LED panel, and a micro LED panel. Meanwhile, the display (120) may also configure a touch screen together with a touch panel, and may also be formed of a flexible panel.

[0064] In particular, the display (120) can display a two-dimensional map (e.g., a two-dimensional grid map) or a three-dimensional map. Alternatively, the display (120) can display various UIs for simplifying or modifying the two-dimensional map or the three-dimensional map. This will be described in detail later with reference to the drawings.

[0065] The memory (130) may store an operating system (OS) for controlling the overall operation of the components of the electronic device (100) and instructions or data related to the components of the electronic device (100). In particular, the memory (130) may include various modules for creating a three-dimensional map by simplifying a two-dimensional map. In particular, when a function for creating a three-dimensional map by simplifying a two-dimensional map is executed, the electronic device (100) may load data for performing various operations by the various modules for creating a three-dimensional map by simplifying a two-dimensional map stored in a non-volatile memory into a volatile memory. Here, loading means an operation of loading and storing data stored in a non-volatile memory into a volatile memory so that at least one processor (150) can access it.

[0066] Additionally, the memory (130) can store information on various neural network models for creating a three-dimensional map by simplifying a two-dimensional map.

[0067] Meanwhile, the memory (130) may be implemented as a non-volatile memory (e.g., hard disk, SSD (Solid state drive), flash memory), volatile memory (which may also include memory within at least one processor (150)), etc.

[0068] The user input unit (140) may include a button, a lever, a switch, a touch interface, etc. In this case, the touch interface may be implemented in a manner of receiving input by the user's touch on the display (120) screen of the electronic device (100).

[0069] In particular, the user input unit (140) can obtain (or receive, input, etc.) user input for generating a three-dimensional map on the screen. Alternatively, the user input unit (140) can obtain user input for modifying a two-dimensional map or a three-dimensional map. Alternatively, the user input unit (140) can obtain user input for determining the degree of simplification of a two-dimensional map.

[0070] At least one processor (150) can control the electronic device (100) according to at least one instruction stored in the memory (130).

[0071] In particular, at least one processor (150) may include one or more processors. Specifically, the one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0072] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0073] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0074] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among a plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0075] In embodiments of the present disclosure, at least one processor (150) may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0076] In particular, at least one processor (150) executes at least one instruction so that the electronic device (100) obtains a two-dimensional map of an indoor space generated based on sensing data and driving data acquired by at least one external device. At least one processor (150) simplifies the obtained two-dimensional map based on the driving data. At least one processor (150) generates a three-dimensional map based on the simplified two-dimensional map.

[0077] In one or more embodiments, at least one processor (150) may obtain a two-dimensional grid map generated based on sensing data. At this time, at least one processor (150) may obtain a binary image based on information about obstacles included in the two-dimensional grid map. At least one processor (150) may perform a thinning operation that converts the thickness of lines included in the binary image to a constant value. At least one processor (150) may extract straight linear vector candidates from the binary image. At least one processor (150) may connect the extracted straight linear vector candidates and convert them into linear vectors to obtain a two-dimensional linear vector map.

[0078] In one or more embodiments, at least one processor (150) can remove noise included in a two-dimensional linear vector map. At least one processor (150) can simplify polylines included in the two-dimensional linear vector map from which noise has been removed based on driving data. At least one processor (150) can correct a room area included in an indoor space based on the simplified two-dimensional linear vector map.

[0079] In one or more embodiments, at least one processor (150) may determine a line having a length less than or equal to a threshold value or a line having an irregularity greater than or equal to a threshold value among a plurality of lines included in a two-dimensional linear vector map as noise.

[0080] In one or more embodiments, at least one processor (150) may simplify the polylines included in the two-dimensional linear vector map from which noise has been removed so that the area in which the robot cleaner has driven is included as an indoor area based on the driving data.

[0081] In one or more embodiments, at least one processor (150) may delete an area outside a boundary line included in a simplified two-dimensional linear vector map among room areas included in an indoor space from an existing room area, and may add an area inside a boundary line included in the simplified two-dimensional linear vector map but not included as a room area as a new room area.

[0082] In one or more embodiments, at least one processor (150) may display a UI on the display (120) that prompts the user for a simplification task during the simplification task.

[0083] In one or more embodiments, at least one processor (150) may adjust the degree of simplification of the two-dimensional map based on obstacle information obtained from user input or sensing data.

[0084] In one or more embodiments, at least one processor (150) may determine at least one of a shape, height, and thickness of a wall included in a three-dimensional map based on obstacle information obtained from sensing data.

[0085] In one or more embodiments, at least one processor (150) may display an editing UI on the display (120) for editing a two-dimensional map or a three-dimensional map based on user input.

[0086] Below, we will describe in more detail a method of creating a 3D map by simplifying a 2D map with reference to a drawing.

[0087] FIG. 3 is a flowchart illustrating a method for generating a three-dimensional map by simplifying a two-dimensional map according to one embodiment of the present disclosure.

[0088] FIG. 4 is a diagram illustrating a 2D grid map according to one embodiment of the present disclosure.

[0089] The electronic device (100) can obtain a two-dimensional grid map (step S310). Specifically, the robot cleaner (200) can obtain sensing data through various sensors while driving in an indoor space. Specifically, the robot cleaner (200) can obtain sensing data for the indoor space through various sensors such as a camera, a lidar sensor, etc., in order to obtain information about obstacles (e.g., objects, walls, etc.). In addition, the robot cleaner (200) can obtain driving data of driving in the indoor space through a driving unit. At this time, the driving data can include a driving path, driving speed, driving history, etc. of the robot cleaner (200) driving in the indoor space. In addition, the robot cleaner (200) can generate a two-dimensional grid map based on the sensing data and the driving data. At this time, the grid can include a plurality of cells, and each cell can be configured as 4 cm x 4 cm. At this time, each cell can include information on the presence or absence of an obstacle. Additionally, the two-dimensional grid map can be displayed by dividing the areas that divide the indoor space into different colors. For example, the robot cleaner (200) can create a two-dimensional grid map as illustrated in FIG. 4. Furthermore, the robot cleaner (200) can transmit information about the two-dimensional grid map to the electronic device (100), thereby allowing the electronic device (100) to obtain the two-dimensional grid map.

[0090] Meanwhile, in the above-described embodiment, it has been described that the robot cleaner (200) collects sensing data and driving data, but this is only one embodiment, and the sensing data and driving data may be collected by at least one external device (e.g., a camera installed externally) other than the robot cleaner (200).

[0091] Meanwhile, although the robot cleaner (200) is described as generating a two-dimensional grid map, this is only one embodiment, and the electronic device (100) can receive sensing data and driving data from at least one external device and generate a two-dimensional grid map based on the received sensing data and driving data.

[0092] Meanwhile, the robot cleaner (200) or electronic device (100) can create a two-dimensional grid map using SLAM (Simultaneous Localization and Mapping) technology.

[0093] FIGS. 6a, 6b, 6c and 6d are diagrams illustrating a method for converting a two-dimensional grid map into a two-dimensional linear vector map according to various embodiments of the present invention.

[0094] The electronic device (100) can convert a two-dimensional grid map into a two-dimensional linear vector map in the form of multiple lines (step S320). Specifically, the electronic device (100) can extract lines representing the boundaries of obstacles included in the two-dimensional grid map to obtain a two-dimensional linear vector map. A method by which the electronic device (100) converts a two-dimensional grid map into a two-dimensional linear vector map will be described with reference to FIG. 5 and FIGS. 6A to 6D.

[0095] FIG. 5 is a flowchart illustrating a method for converting a two-dimensional grid map into a two-dimensional linear vector map according to one embodiment of the present disclosure.

[0096] The electronic device (100) can generate a binary image based on information about obstacles included in a two-dimensional grid map (step S510). At this time, the binary image is an image indicating the presence or absence of an obstacle at each pixel location, as illustrated in FIG. 6a.

[0097] The electronic device (100) may perform a thinning operation on a binary image (step S520). At this time, the thinning operation may be a operation of converting the thickness of all lines included in the binary image to a constant thickness. Specifically, if some of the lines representing obstacles included in the binary image have a width greater than a certain standard, the electronic device (100) may remove the outer pixels of some of the lines so that the width of some of the lines becomes the standard width, as illustrated in FIG. 6b.

[0098] The electronic device (100) can extract a straight linear vector candidate from a binary image on which a thinning operation has been performed (step S530). Specifically, the electronic device (100) can extract a straight linear vector candidate, as illustrated in FIG. 6c, by tracking the nearest neighboring pixel where an obstacle is located, connecting the tracked pixels, and determining whether the connected neighboring pixels fit a straight line of a certain length or longer.

[0099] The electronic device (100) can connect the extracted straight linear vector candidates and convert them into linear vectors in the form of multiple lines (step S540). Specifically, the electronic device (100) can connect the extracted linear vector candidates to obtain linear vectors in the form of multiple lines, as illustrated in FIG. 6D. As a result, the electronic device (100) can obtain a two-dimensional linear vector map including linear vectors in the form of multiple lines.

[0100] Referring to FIG. 3, the electronic device (100) can obtain a simplified two-dimensional linear vector map (step S330). Specifically, the electronic device (100) can obtain a simplified two-dimensional linear vector map by performing a simplification operation on the two-dimensional linear vector map. At this time, the simplification operation may include, but is not limited to, a task of removing noise included in the map, a task of removing protrusions of lines included in the map or connecting broken lines, and a task of flattening curved portions of lines.

[0101] FIGS. 8a, 8b, 9a, 9b, 9c, and 9d are drawings illustrating a method for simplifying a two-dimensional linear vector map according to one embodiment of the present disclosure.

[0102] A method for simplifying a two-dimensional linear vector map of an electronic device (100) will be described with reference to FIGS. 7, 8a, 8b, and 9a to 9d.

[0103] FIG. 7 is a flowchart illustrating a method for simplifying a two-dimensional linear vector map according to one embodiment of the present disclosure.

[0104] First, the electronic device (100) can remove noise included in a two-dimensional linear vector map (step S710). Specifically, the electronic device (100) can identify a line determined to be noise among a plurality of lines included in the two-dimensional linear vector map. In particular, the electronic device (100) can determine a line whose length is less than or equal to a threshold value among the plurality of lines included in the two-dimensional linear vector map as noise. In addition, the electronic device (100) can determine irregularity of the plurality of lines included in the two-dimensional linear vector map. In this case, the irregularity refers to the irregularity of the curve of the line, and may be a numerical value of the frequency with which the line irregularly changes direction per unit length. In addition, the electronic device (100) can determine a line whose irregularity is greater than or equal to a threshold value among the plurality of lines included in the two-dimensional linear vector map as noise. In addition, the electronic device (100) can remove the line determined to be noise to obtain a two-dimensional linear vector map from which noise has been removed, as illustrated in FIG. 8A.

[0105] In addition, the electronic device (100) can simplify the multiple lines included in the two-dimensional linear vector map from which noise has been removed (step S720). Specifically, the electronic device (100) can simplify the two-dimensional linear vector map by removing protrusions of lines, connecting broken lines, or flattening curves of lines. In particular, the electronic device (100) can simplify the multiple lines based on the driving path among the driving data received from the robot cleaner (200).

[0106] Specifically, the electronic device (100) removes a portion with a high degree of irregularity in the curvature between straight lines or in the intersection area of ​​lines within a multiline included in a two-dimensional linear vector map. For example, as illustrated in 910 of FIG. 9A , if a highly irregular curvature exists between straight lines, the electronic device (100) may remove the highly irregular curvature and connect the two straight lines, as illustrated in 915 of FIG. 9A . As another example, as illustrated in 920 of FIG. 9B , if a highly irregular curvature exists between the intersection area of ​​two straight lines, the electronic device (100) may remove the highly irregular curvature and intersect the two straight lines, as illustrated in 925 of FIG. 9B .

[0107] In addition, if there is a bend in the form of a diagonal line within a multiline included in a two-dimensional linear vector map, the electronic device (100) can linearize the bend in the form of a diagonal line using a line fitting or curve fitting method. For example, as shown in 930 of FIG. 9c, if there is a bend with high irregularity in the form of a diagonal line, the electronic device (100) can simplify it into a diagonal line using a curve fitting method, as shown in 935 of FIG. 9c.

[0108] Additionally, if the irregularity of the bending at both ends of a multiline included in a two-dimensional linear vector map is greater than a threshold value, the electronic device (100) can remove the end portion having the irregularity greater than the threshold value.

[0109] Additionally, if both ends of a multiline included in a two-dimensional linear vector map are close to each other and perpendicular to each other, the electronic device (100) can determine that it is a closed loop and connect both ends.

[0110] Additionally, the electronic device (100) can connect two multiple lines included in a two-dimensional linear vector map by determining them as one multiple line if the distance between them is less than a threshold value and they are on the same line or perpendicular to each other.

[0111] Additionally, if a boundary box consisting of a single multiline is below a threshold value and is located within a room area, the electronic device (100) can determine that the single multiline is an object within the room and remove it.

[0112] In addition, the electronic device (100) can simplify the multiple lines included in the two-dimensional linear vector map based on the driving data. Specifically, the electronic device (100) can simplify the multiple lines included in the two-dimensional linear vector map from which noise has been removed so that the area in which the robot cleaner (200) has driven is included as an indoor area based on the driving data. For example, part (a) of FIG. 9D is a diagram illustrating a two-dimensional linear vector map before simplification, part (b) of FIG. 9D is a diagram illustrating a two-dimensional linear vector map simplified without considering the driving data, and part (c) of FIG. 9D is a diagram illustrating a two-dimensional linear vector map simplified by considering the driving data. For reference, the straight line illustrated in FIG. 9D is a multiple line included in the two-dimensional linear vector map, and the dotted line is an execution path of the robot cleaner (200).

[0113] As illustrated in part (a) of FIG. 9d, a two-dimensional linear vector map that is not simplified includes many curves within a single polyline, so when a three-dimensional map is created using a two-dimensional linear vector map, there is a limitation that it cannot reflect complex or actual indoor spaces. In addition, as illustrated in part (b) of FIG. 9d, a two-dimensional linear vector map that does not consider driving data has a limitation that it cannot reflect actual indoor spaces. Therefore, as illustrated in part (c) of FIG. 9d, a two-dimensional linear vector map that considers driving data can have the advantage of reflecting actual indoor spaces while improving visibility by simplifying the two-dimensional linear vector map so that the area where the robot cleaner (200) drove is included as an indoor area.

[0114] The electronic device (100) can obtain a two-dimensional linear vector map with a simplified multiple line, as shown in FIG. 8b, through the simplification operation of the two-dimensional linear vector map as described above.

[0115] Referring back to FIG. 3, the electronic device (100) can correct the room area of ​​the two-dimensional map based on the simplified two-dimensional linear vector map (step S340).

[0116] FIG. 10 is a diagram for explaining a method for correcting a room area of ​​a two-dimensional map according to one embodiment of the present disclosure.

[0117] Specifically, the electronic device (100) may delete an area outside the boundary line included in the simplified two-dimensional linear vector map among the room areas included in the two-dimensional map as illustrated in 1010 of FIG. 10 (wherein the two-dimensional map may be a two-dimensional map generated by a two-dimensional grid map and a two-dimensional linear vector map), from the existing room area, and may add an area inside the boundary line included in the simplified two-dimensional linear vector map but not included in the room area as a new room area. Accordingly, the electronic device (100) may obtain a two-dimensional map including a corrected room area as illustrated in 1020 of FIG. 10.

[0118] And, the electronic device (100) can generate a three-dimensional map based on the simplified two-dimensional linear vector map (step S350). Specifically, the electronic device (100) can generate walls of the three-dimensional map based on information about obstacles included in the simplified two-dimensional linear vector map. In addition, the electronic device (100) can generate a floor based on the two-dimensional map in which the room area is corrected. And, the electronic device (100) can generate a three-dimensional map based on the information about the walls and floor described above.

[0119] The electronic device (100) can display the generated three-dimensional map on the display (120) (step S360). Specifically, the electronic device (100) can render the three-dimensional map and display it on the display (120). In addition, the electronic device (100) can receive user input for controlling the robot cleaner (200) or other home appliances using the three-dimensional map, similar to a two-dimensional map (in particular, the two-dimensional map can be generated by a two-dimensional grid map and a two-dimensional linear vector map), and can provide information on the cleaning result.

[0120] FIG. 11 is a sequence diagram for modifying a method for modifying a two-dimensional map or a three-dimensional map according to one embodiment of the present disclosure.

[0121] First, the robot cleaner (200) can collect sensing data and driving data while driving in an indoor space (step S1105).

[0122] The robot cleaner (200) can generate a two-dimensional grid map based on the collected sensing data and driving data (step S1110). At this time, the robot cleaner (200) can generate the two-dimensional grid map using SLAM technology.

[0123] And, the robot cleaner (200) can transmit information about the two-dimensional grid map to the electronic device (100) (step S1115).

[0124] The electronic device (100) can convert a two-dimensional grid map into a two-dimensional linear vector map (step S1120). At this time, the electronic device (100) can convert the two-dimensional grid map into a two-dimensional linear vector map including linear vectors in the form of multiple lines using the method described in FIGS. 5 to 6d.

[0125] The electronic device (100) can simplify a two-dimensional linear vector map (step S1125). At this time, the electronic device (100) can obtain a two-dimensional linear vector map including a simplified multiple line through a method as described in FIGS. 7, 8A, 8B, and 9A to 9D. At this time, the electronic device (100) can correct a room area of ​​the two-dimensional map based on the simplified two-dimensional linear vector map.

[0126] The electronic device (100) can display a corrected two-dimensional map through a simplified two-dimensional linear vector map (step S1130). At this time, the two-dimensional map may be a map generated based on a two-dimensional grid map and a two-dimensional linear vector map.

[0127] Meanwhile, the electronic device (100) may receive a user input for a two-dimensional map modification request (step S1135). Upon receiving the user input for the two-dimensional map modification request, the electronic device (100) may simplify the two-dimensional linear vector map again based on the map modification information.

[0128] The electronic device (100) can receive user input for a 3D map request (step S1140).

[0129] The electronic device (100) can generate a three-dimensional map using a simplified two-dimensional linear vector map (step S1145).

[0130] The electronic device (100) can render a three-dimensional map and display it on the display (120) (step S1150). The electronic device (100) can control the robot cleaner (200) or display cleaning results according to user input entered on the displayed three-dimensional map.

[0131] Meanwhile, the electronic device (100) may receive a user input for a 3D map modification request (step S1155). When the user input for the 3D map modification request is received, the electronic device (100) may convert the user input for the 3D map modification request into map modification information for the 2D map in step S1160. Then, in step S1160, the electronic device (100) may simplify the 2D linear vector map again based on the map modification information for the 2D map. The electronic device (100) may regenerate the 3D map based on the simplified 2D linear vector map. However, the above-described embodiment is merely one embodiment, and the electronic device (100) may acquire map modification information for the 3D map and, of course, directly modify the 3D map based on the acquired map modification information.

[0132] Meanwhile, the robot cleaner (200) can perform cleaning operations based on user input or a preset cycle (step S1165). At this time, the robot cleaner (200) can perform cleaning operations based on the generated two-dimensional grid map, and can update the two-dimensional grid map based on the cleaning operation results.

[0133] The robot cleaner (200) can transmit information about its cleaning operation to the electronic device (100) (step S1170). At this time, the information about the cleaning operation may include the area where the robot cleaner (200) performed the cleaning operation, the travel path, the cleaning time, obstacle information, etc. while performing the cleaning operation.

[0134] The electronic device (100) can display the cleaning results on a three-dimensional map (step S1175). That is, the electronic device (100) can display information about the area where the cleaning operation was performed, information about the driving path, information about the cleaning time, obstacle information, etc. on the three-dimensional map.

[0135] Meanwhile, in the above-described embodiment, the electronic device (100) is described as generating a three-dimensional map by simplifying a two-dimensional map. However, this is merely an example, and a three-dimensional map can be generated by simplifying a two-dimensional map through a separate server. This will be described with reference to FIG. 12.

[0136] FIG. 12 is a sequence diagram illustrating a method for a system including a server for generating a map for an indoor space to generate a map for an indoor space according to one embodiment of the present disclosure.

[0137] First, the robot cleaner (200) can collect sensing data and driving data while driving in an indoor space (step S1205).

[0138] The robot cleaner (200) can generate a two-dimensional grid map based on the collected sensing data and driving data (step S1210). At this time, the robot cleaner (200) can generate the two-dimensional grid map using SLAM technology.

[0139] In addition, the robot cleaner (200) can transmit information about the two-dimensional grid map to the server (1200) (step S1215), and the server (1200) can transmit information about the two-dimensional grid map to the electronic device (100) (step S1220). Meanwhile, the robot cleaner (200) can transmit information about the two-dimensional grid map to the electronic device (100) via the server (1200), but this is only one embodiment, and the robot cleaner (200) can directly transmit information about the two-dimensional grid map to the electronic device (100).

[0140] The electronic device (100) can display a two-dimensional map (step S1225). At this time, the displayed two-dimensional map may be a two-dimensional map generated by a two-dimensional grid map and a two-dimensional linear vector map. In addition, the electronic device (100) can receive a user input for a three-dimensional map request (step S1230).

[0141] The electronic device (100) can transmit a signal to the server (1200) to request generation of a 3D map in response to user input (step S1235).

[0142] The server (1200) can convert a two-dimensional grid map into a two-dimensional linear vector map (step S1240). At this time, the server (1200) can convert the two-dimensional grid map into a two-dimensional linear vector map including linear vectors in the form of multiple lines using the method described in FIGS. 5 and 6A to 6D.

[0143] The server (1200) can simplify a two-dimensional linear vector map (step S1245). At this time, the server (1200) can obtain a two-dimensional linear vector map including a simplified multiple line through the method described in FIGS. 7, 8A, 8B, and 9A to 9D. At this time, the server (1200) can correct the room area of ​​the two-dimensional map through the simplified two-dimensional linear vector map.

[0144] The server (1200) can create a three-dimensional map using a simplified two-dimensional linear vector map (step S1250).

[0145] The server (1200) can transmit a three-dimensional map generated using a simplified two-dimensional grid map to the electronic device (100) (step S1255).

[0146] The electronic device (100) can render a three-dimensional map and display it on the display (120) (step S1260). The electronic device (100) can control the robot cleaner (200) or display cleaning results according to user input entered on the displayed three-dimensional map.

[0147] The robot cleaner (200) can perform cleaning operations based on user input or preset cycles (step S1265). At this time, the robot cleaner (200) can perform cleaning operations based on the generated two-dimensional grid map, and can update the two-dimensional grid map based on the cleaning operation results.

[0148] The robot cleaner (200) can transmit information about its cleaning operation to the server (1200) (step S1270), and the server (1200) can transmit information about its cleaning operation to the electronic device (100) (step S1275). At this time, the robot cleaner (200) can transmit information about its cleaning operation to the electronic device (100) via the server (1200), but this is only one embodiment, and information about its cleaning operation can be transmitted directly to the electronic device (100).

[0149] The electronic device (100) can display cleaning results on a three-dimensional map (step S1280). For example, the electronic device (100) can display information about the area where cleaning was performed, information about the driving path, information about the cleaning time, obstacle information, etc. on the three-dimensional map.

[0150] Meanwhile, in FIG. 12, it is described that a 3D map is created by simplifying a 2D map through a server (1200), but this is only one example, and it is of course possible for the robot cleaner (200) itself to create a 3D map by simplifying a 2D map. In this case, a 3D map can be created by simplifying a 2D map not only through the robot cleaner (200) body but also through a station included in the robot cleaner (200).

[0151] Meanwhile, in one or more embodiments, the electronic device (100) may adjust the degree of simplification of the two-dimensional map based on obstacle information obtained from user input or sensing data. Specifically, when the degree of simplification is set to a high value, the electronic device (100) may perform a simplification operation to remove noise or to increase the portion where the bends within the multiline are flattened. When the degree of simplification is set to a low value, the electronic device (100) may perform a simplification operation to remove noise or to decrease the portion where the bends within the multiline are flattened. Accordingly, when the degree of simplification is set to a high value, the electronic device (100) may perform a simplification operation to make the two-dimensional map or the three-dimensional map relatively simpler, and when the degree of simplification is set to a low value, the electronic device (100) may perform a simplification operation to make the two-dimensional map or the three-dimensional map relatively more complex. At this time, the electronic device (100) may adjust the degree of simplification by adjusting a threshold value for removing noise and a threshold value for determining irregularity.

[0152] FIG. 13 is a diagram illustrating a UI for selecting a degree of simplification of a two-dimensional map according to one embodiment of the present disclosure.

[0153] In one or more embodiments, the electronic device (100) may adjust the degree of simplification of a two-dimensional map based on a user input input through a UI (1310) as illustrated in FIG. 13. For example, the electronic device (100) may adjust the degree of simplification of a two-dimensional map based on a user input for adjusting a progress bar included in the UI (1310). At this time, when a user input setting the degree of simplification to a high value is received, the electronic device (100) may perform a simplification operation to remove noise or increase the portion where the bends within the multiline are flattened. When a user input setting the degree of simplification to a low value is received, the electronic device (100) may perform a simplification operation to remove noise or decrease the portion where the bends within the multiline are flattened. Meanwhile, FIG. 13 illustrates a UI (1310) for adjusting the degree of simplification using a progress bar, but this is only one embodiment, and the degree of simplification can be adjusted through a UI including other UI elements (e.g., icons for selecting the degree of simplification (e.g., high, medium, low, etc.)).

[0154] In one or more embodiments, the electronic device (100) may adjust the degree of simplification of a two-dimensional map based on obstacle information acquired from sensing data. Specifically, the electronic device (100) may identify whether an obstacle exceeds a threshold value or the size of the obstacle in the indoor space based on the sensing data. If the number of obstacles exceeds the threshold value or if the number of obstacles smaller than the threshold size exceeds the threshold value, the electronic device (100) may set the degree of simplification to a low value to increase the precision of the map. Accordingly, the electronic device (100) may prevent obstacles from being simplified and omitted and may display the outlines of the obstacles in greater detail. If the number of obstacles exceeds the threshold value or if the number of obstacles smaller than the threshold size exceeds the threshold value in the indoor space, the electronic device (100) may set the degree of simplification to a high value.

[0155] In one or more embodiments, the degree of simplification may be set differently for each room area defined by the two-dimensional map, rather than for the entire indoor space. For example, a room area with many obstacles (e.g., a storage area) may have a lower degree of simplification, while a room area with fewer obstacles (e.g., a living room area) may have a higher degree of simplification.

[0156] In one or more embodiments, when generating a three-dimensional map through a two-dimensional map, instead of uniformly generating walls of the same height and width, the electronic device (100) may identify the shape, height, and width of the wall to be generated by considering obstacle information acquired when the robot cleaner (200) moves through an indoor space. For example, the two-dimensional grid map may be generated to include information (e.g., shape, width, or height) about the obstacle at the corresponding location, rather than indicating the presence or absence of an obstacle as binary data. Accordingly, when generating a wall in the three-dimensional map, the electronic device (100) may identify the shape, height, and width of the wall included in the three-dimensional map based on the information about the obstacle included in the two-dimensional grid map. Alternatively, the electronic device (100) may additionally generate a separate grid map including information about the shape, width, or height of the obstacle, together with the two-dimensional grid map indicating the presence or absence of the obstacle. Additionally, the electronic device (100) can identify the shape, height, and width of a wall included in a three-dimensional map using a two-dimensional grid map and a separate grid map.

[0157] In one or more embodiments, when the robot cleaner (200) recognizes the type of obstacle through a camera or other sensor, the electronic device (100) can use the recognized obstacle information to determine the shape of the wall / obstacle to be generated in the 3D map. For example, the type of obstacle can be recognized by inputting the sensing data collected by the robot cleaner (200) into a trained neural network model. Then, the electronic device (100) can insert the recognized obstacle into the 3D map based on the recognized obstacle type.

[0158] FIGS. 14A, 14B, and 14C are diagrams illustrating a UI and a 3D map for selecting a type of obstacle included in a 3D map according to one embodiment of the present disclosure.

[0159] At this time, the electronic device (100) may display a UI for selecting the type of obstacle included in the 3D map on the display (120). For example, the electronic device (100) may display a UI (1410) for determining the type or location of an obstacle included in the 3D map on the display (120), as illustrated in FIG. 14a. If the type of obstacle is not selected through the UI (1410), the electronic device (100) may generate a 3D map (1420) so as not to include obstacles other than walls, as illustrated in FIG. 14b. However, if a home appliance is selected as the type of obstacle through the UI (1410), the electronic device (100) may generate a 3D map (1430) so as to include the home appliance, as illustrated in FIG. 14c. As another example, when a floor is selected through the UI (1410), the electronic device (100) can generate a three-dimensional map to include obstacles located on the floor.

[0160] Accordingly, the electronic device (100) can provide a variety of services to the user by providing a different 3D map that includes only the types of obstacles desired by the user. For example, the electronic device (100) can receive user commands to control home appliances through a 3D map that includes the home appliances. As another example, the electronic device (100) can perform cleaning operations more efficiently through a 3D map that includes obstacles located on the floor.

[0161] In one or more embodiments, the electronic device (100) may display an editing UI for editing a two-dimensional map or a three-dimensional map on the display (120) based on a user input. Specifically, when a user views a map (two-dimensional map or three-dimensional map) completed by the electronic device (100) and inputs map modification information for additional map modification, the electronic device (100) may perform a modification and simplification operation on the two-dimensional map based on the map modification information, and display the generated two-dimensional map on the display (120). The user may check the simplified two-dimensional map displayed on the display (120) and perform repetitive simplification operations. In addition, when the user inputs map modification information for a three-dimensional map, the electronic device (100) may change the modification information for the three-dimensional map into modification information for the two-dimensional map and perform the modification and simplification operations.

[0162] FIGS. 15A, 15B, and 15C are diagrams illustrating a UI for modifying a two-dimensional map or a three-dimensional map according to one embodiment of the present disclosure.

[0163] FIG. 15A is a diagram illustrating a UI for modifying a two-dimensional map according to one embodiment of the present disclosure. At this time, the electronic device (100) may display an editing UI (1510) for modifying a two-dimensional map and a two-dimensional map (1520) on the display (120).

[0164] At this time, when a user input for drawing a rectangle corresponding to a room area is received through a rectangle menu included in the editing UI (1510), the electronic device (100) can display the room area in a rectangle shape on a two-dimensional map (1520), as illustrated in FIG. 15A. At this time, if the room area is not horizontal, the electronic device (100) can rotate the rectangle representing the room area through the user input.

[0165] Alternatively, when a user input for merging rectangles is received through a merge menu included in the edit UI (1510), the electronic device (100) may merge two rectangles to display a room area in the shape of a single polygon. For example, as illustrated in FIG. 15c, the electronic device (100) may display a room area in the shape of a polygon (1560) through a merge command for a first rectangle (1550-1) and a second rectangle (1550-2).

[0166] Additionally, when a user input for drawing a wall is received through a line drawing menu included in the editing UI (1510), the electronic device (100) can modify the two-dimensional map so that a wall corresponding to the line is included. In this case, if the wall is not horizontal, the electronic device (100) can rotate the line through the user input.

[0167] Additionally, when a user input for removing a room or line is received through a removal menu included in the edit UI (1510), the electronic device (100) can remove the room or line based on the user input.

[0168] FIG. 15b is a diagram illustrating a UI for modifying a 3D map according to one embodiment of the present disclosure. At this time, the electronic device (100) can display an editing UI (1530) for modifying a 3D map and a 3D map (1540) on the display (120).

[0169] Modifications to the 3D map illustrated in Fig. 15b are identical to those described in Fig. 15a, but the height of the square and the height / width of the line can be set through user input. The electronic device (100) can modify the height / width of the wall included in the indoor space through the set height / width of the square and line.

[0170] When map modification information is obtained through the editing UI as described in FIGS. 15a to 15c, the electronic device (100) can modify the two-dimensional map and perform a simplification operation based on the obtained map modification information.

[0171] Specifically, the electronic device (100) can perform a simplification task by preferentially connecting vectors that match the map correction information among linear vectors extracted from a two-dimensional grid map. In addition, the electronic device (100) can determine vectors that do not match the map correction information as obstacles or noise and remove them. Even if the map correction information does not exactly match the linear vector, the electronic device (100) can search for similar patterns and perform a simplification task.

[0172] FIG. 16 is a diagram illustrating a UI for inquiring whether to perform a simplification operation on some areas among a plurality of areas included in a two-dimensional map according to one embodiment of the present disclosure.

[0173] In one or more embodiments, the electronic device (100) may display a UI on the display (120) that inquires the user about the simplification operation during the simplification operation. In one embodiment, the electronic device (100) may display a UI (1610) that inquires whether to perform a simplification operation on an area included in a two-dimensional map, as illustrated in FIG. 16. At this time, the area may be an area in which it is ambiguous whether to perform the simplification operation. For example, if the length of a line included in a multi-line is determined to be within a critical range or the irregularity is determined to be within a critical range, the electronic device (100) may display a UI that inquires whether to perform the simplification operation on the area. If a user input for performing the simplification operation is received through the UI (1610), the electronic device (100) may perform the simplification operation on the area, and if a user input not to perform the simplification operation is received through the UI (1610), the electronic device (100) may not perform the simplification operation on the area.

[0174] In one or more embodiments, the electronic device (100) may perform a simplification task when the robot cleaner (200) generates an initial map, but this is only one embodiment, and the electronic device (100) may perform the simplification task based on a user input, a preset period, or a preset event. For example, when a user input for performing the simplification task is received, the electronic device (100) may perform the simplification task based on the user input. As another example, the electronic device (100) may perform the simplification task based on a preset period (e.g., once a month) or a preset event (e.g., adding a home appliance or changing the furniture arrangement, etc.).

[0175] FIG. 17 is a control method of an electronic device for generating a map according to one embodiment of the present disclosure.

[0176] The electronic device (100) acquires a two-dimensional map of an indoor space generated based on sensing data and driving data acquired by at least one external device (step S1710). At this time, the electronic device (100) may directly generate the two-dimensional map based on the sensing data and driving data, but this is only one embodiment, and the electronic device (100) may receive a two-dimensional map generated by an external device (robot cleaner (200) or external server (1200)).

[0177] The electronic device (100) simplifies a two-dimensional map acquired based on driving data (step S1720). Specifically, the electronic device (100) can acquire a two-dimensional grid map generated based on sensing data. The electronic device (100) can acquire a binary image based on information about obstacles included in the two-dimensional grid map. The electronic device (100) can perform a thinning operation that converts the thickness of lines included in the binary image to a constant. The electronic device (100) can extract straight linear vector candidates from the binary image. The electronic device (100) can connect the extracted straight linear vector candidates and convert them into linear vectors to acquire a two-dimensional linear vector map.

[0178] In addition, the electronic device (100) can remove noise included in a two-dimensional linear vector map. The electronic device (100) can simplify multiple lines included in the two-dimensional linear vector map from which noise has been removed based on driving data. The electronic device (100) can correct a room area included in an indoor space based on the simplified two-dimensional linear vector map.

[0179] At this time, the electronic device (100) can determine as noise and remove lines whose length is less than a threshold value or whose irregularity is greater than a threshold value among the multiple lines included in the two-dimensional linear vector map.

[0180] Additionally, the electronic device (100) can simplify the multiple lines included in the two-dimensional linear vector map from which noise has been removed so that the area in which the robot cleaner has driven is included as an indoor area based on the driving data.

[0181] In addition, the electronic device (100) can delete an area outside the boundary line included in the simplified two-dimensional linear vector map among the room areas included in the indoor space from the existing room area, and add an area inside the boundary line included in the simplified two-dimensional linear vector map but not included in the room area as a new room area.

[0182] The electronic device (100) generates a three-dimensional map based on the simplified two-dimensional map (step S1730). Furthermore, the electronic device (100) can render the generated three-dimensional map and display it on the display (120). This allows the user to control various home appliances contained in an indoor space and check cleaning operations through the three-dimensional map.

[0183] Meanwhile, the electronic device (100) may display a UI that prompts the user for a simplification task during a simplification task. In addition, the electronic device (100) may adjust the degree of simplification of a two-dimensional map based on obstacle information obtained from user input or sensing data. In addition, the electronic device (100) may determine at least one of the shape, height, and thickness of a wall included in a three-dimensional map based on obstacle information obtained from sensing data. In addition, the electronic device (100) may display an editing UI for editing a two-dimensional map or a three-dimensional map based on user input.

[0184] Meanwhile, the methods according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0185] The methods according to various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device according to the disclosed embodiments, as a device capable of calling the instructions stored in the storage medium and operating according to the called instructions.

[0186] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0187] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0188] While the present disclosure has been illustrated and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure, which may be defined by the appended claims and their equivalents.

Claims

1. In electronic devices, communication interface; a memory storing at least one computer program; and comprising one or more processors connected to the communication interface and the memory so as to be communicatively connected; The one or more computer programs, when executed by the one or more processors, Obtaining a two-dimensional map of the indoor space generated based on sensing data and driving data acquired by at least one external device, Simplify the acquired two-dimensional map based on the above driving data, An electronic device comprising computer-executable instructions for controlling the electronic device to generate a three-dimensional map based on the simplified two-dimensional map.

2. In paragraph 1, The one or more computer programs, when executed by the one or more processors, Obtaining a two-dimensional grid map generated based on the above sensing data, A binary image is obtained based on information about obstacles included in the above two-dimensional grid map, A thinning operation is performed to uniformly convert the thickness of the lines included in the above binary image. Extract linear vector candidates from the above binary image, An electronic device storing computer-executable instructions for controlling the electronic device to connect the extracted straight line linear vector candidates and convert them into linear vectors to obtain a two-dimensional linear vector map.

3. In paragraph 2, The one or more computer programs, when executed by the one or more processors, Remove noise contained in the above two-dimensional linear vector map, Simplify the polylines included in the two-dimensional linear vector map from which the noise has been removed based on the above driving data, An electronic device storing computer-executable instructions for controlling the electronic device to compensate for a room area included in an indoor space based on the simplified two-dimensional linear vector map.

4. In paragraph 3, The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions for controlling the electronic device to determine, among a plurality of lines included in the two-dimensional linear vector map, a line having a length less than a threshold value or a line having an irregularity greater than a threshold value as noise.

5. In paragraph 3, The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions for controlling the electronic device to simplify a polyline included in the two-dimensional linear vector map from which noise has been removed so that the area in which the robot cleaner has driven is included as an indoor area based on the driving data.

6. In paragraph 3, The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions that control the electronic device to delete an area outside the boundary line included in the simplified two-dimensional linear vector map among the room areas included in the indoor space from the existing room area, and to add an area inside the boundary line included in the simplified two-dimensional linear vector map but not included in the room area as a new room area.

7. In paragraph 3, including display; The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions that control the electronic device to display on the display a UI prompting the user to perform a simplification task.

8. In paragraph 1, The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions for controlling the electronic device to adjust the degree of simplification of the two-dimensional map based on user input or obstacle information obtained from the sensing data.

9. In paragraph 1, The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions for controlling the electronic device to determine at least one of a shape, height, and thickness of a wall included in the three-dimensional map based on obstacle information acquired from the sensing data.

10. In paragraph 1, including display; The one or more computer programs, when executed by the one or more processors, An electronic device storing computer-executable instructions for controlling the electronic device to display on the display an editing UI for editing the two-dimensional map or the three-dimensional map based on user input.

11. In a control method performed by an electronic device, A step of obtaining a two-dimensional map of an indoor space generated based on sensing data and driving data obtained by at least one external device; A step of simplifying the acquired two-dimensional map based on the driving data; and A control method comprising: a step of generating a three-dimensional map based on the simplified two-dimensional map.

12. In paragraph 11, The step of obtaining the above two-dimensional map is: A step of obtaining a two-dimensional grid map generated based on the above sensing data; A step of obtaining a binary image based on information about obstacles included in the above two-dimensional grid map; A step of performing a thinning operation to uniformly convert the thickness of lines included in the above binary image; A step of extracting a straight linear vector candidate from the above binary image; and A control method comprising: a step of converting the extracted straight line linear vector candidates into linear vectors by connecting them to obtain a two-dimensional linear vector map.

13. In paragraph 12, The above simplification steps are: A step of removing noise included in the above two-dimensional linear vector map; A step of simplifying a polyline included in a two-dimensional linear vector map from which noise has been removed based on the driving data; A control method comprising: a step of correcting a room area included in an indoor space based on the simplified two-dimensional linear vector map.

14. In paragraph 13, The steps for removing the above noise are: A control method for determining, among a plurality of lines included in the above two-dimensional linear vector map, a line whose length is less than a threshold value or a line whose irregularity is greater than a threshold value as noise.

15. In paragraph 13, The above simplification steps are: An electronic device that simplifies multiple lines included in the two-dimensional linear vector map from which noise has been removed so that the area in which the robot cleaner has driven is included as an indoor area based on the driving data.

Images