Apparatus and method for topology-based semantic map generation

Abstract

A topology-based semantic map generation method according to one embodiment is a topology-based semantic map generation method performed by a computing device including a processor and memory, comprising the steps of: mapping an environment through a sensor of a robot and generating a first map; generating a topology space through data preprocessing based on the first map; generating a topology node based on the topology space; generating a topology edge for the topology node; and assigning semantic space information for the topology space and generating a semantic map based on the topology node and the topology edge.

Description

Technology Field

[0001] The present invention relates to a topology-based semantic map generation apparatus and method, and more specifically, to a topology-based semantic map generation apparatus and method for generating a service environment space definition-based semantic map in a mobile robot in an embedded environment. Background Technology

[0002] There are many cases where mobile robots need to adopt different driving strategies depending on whether they are indoors, outdoors, or in specific areas. Accordingly, research is underway on efficient and safe driving strategies by distinguishing information about the driving area for the robot through computer vision technology and deep learning.

[0003] Accordingly, research is currently being conducted in two main ways. Computer vision methods extract shape information (triangles, squares, circles, etc.) from a given raw 2D image map to distinguish the space.

[0004] Representative deep learning methods include floor plans trained on models using large amounts of training data, and research that projects object information onto 2D maps by distinguishing objects through semantic information from image sensors.

[0005] Conventional technologies fundamentally face challenges in spatial differentiation in non-linear environments (such as offices with non-linear structural layouts), require large amounts of training data and time, and necessitate devices capable of high computational power for modeling. In addition to time and cost issues, this presents problems such as the difficulty of accurate spatial differentiation for low-spec embedded system-based robots and the difficulty of distinguishing spaces through model inference. The problem to be solved

[0006] One embodiment of the present invention aims to provide a topology-based semantic map generation apparatus and method for generating a semantic map in which drivable and non-drivable areas are distinguished based on topology in accordance with the driving strategy of a mobile robot in a low-spec embedded environment. means of solving the problem

[0007] Among the embodiments, the topology-based semantic map generation method is a topology-based semantic map generation method performed by a computing device including a processor and memory, wherein the processor maps an environment through a sensor of a robot and generates an initial map; the processor generates a topology space through data preprocessing based on the initial map; the processor generates a topology node based on the topology space; the processor generates a topology edge for the topology node; and the processor assigns semantic space information for the topology space and generates a semantic map based on the topology node and the topology edge.

[0008] The step of mapping the environment and generating an initial map through the sensor of the robot may include the step of the processor collecting data of the environment through LiDAR and IMU sensors, the step of the processor estimating and optimizing the position of the robot based on the collected data, the step of the processor generating a 3D map through a 3D point cloud, and the step of the processor converting the 3D map into a 2D occupancy grid map.

[0009] The step of generating a topology space through data preprocessing based on the initial map above may include the step of the processor converting the 2D occupancy grid map into a binarized map by binarizing it, the step of the processor generating a distance transformation map through distance transformation based on the binarized map, and the step of the processor generating the topology space by normalizing the distance transformation map.

[0010] The step of generating topology nodes based on the above topology space may include the step of the processor setting the size of grid pixels by reflecting the size of the robot and the step of the processor setting the spacing between nodes based on the size of the grid pixels.

[0011] The step of generating a topology edge for the topology node may include the step of the processor searching for the distance between nodes based on the interval between nodes and generating the edge connecting adjacent nodes where the shortest distance falls within a preset minimum distance.

[0012] The step of searching for distances between nodes based on the above-mentioned interval between nodes and generating edges connecting adjacent nodes where the shortest distance falls within a preset minimum distance may include: the processor optimizing the search for distances between nodes using a KD Tree; the processor connecting adjacent nodes based on the minimum distance and generating edge information; the processor checking for obstacles between edges based on the edge information and deleting some edges that have obstacles; and the processor checking whether the generated edge information duplicates existing stored edge information and updating the edge information.

[0013] The step of assigning semantic space information for the topology space and generating a semantic map based on the topology nodes and topology edges may include the step of the processor grouping the topology space using the topology nodes and topology edges, the step of the processor semantically distinguishing the topology space, and the step of the processor assigning an ID to the distinguished topology space.

[0014] The step of grouping the topology space using the topology nodes and the topology edges may include an initialization step in which the processor sets all nodes into independent groups through a Union-Find algorithm, and a step in which the processor determines nodes connected to each other into the same group using the edges.

[0015] The step of generating a topology space through data preprocessing based on the initial map above may include the step of the processor dividing the topology space into an open space where the robot can move and a closed space where the robot cannot move.

[0016] The step of dividing the topology space into an open space where the robot can move and a closed space where the robot cannot move may include the step of the processor generating the open space through preprocessing on the initial map, and the step of the processor defining the open space as a Region of Interest (ROI) and determining the area outside the ROI as the closed space.

[0017] Among the embodiments, the topology-based semantic map generation device is a topology-based semantic map generation device that generates a semantic map in a mobile robot in an embedded environment based on topology by executing program code loaded into one or more memory devices through one or more processors, wherein the program code is executed to map an environment through the robot's sensors and generate an initial map, generate a topology space through data preprocessing based on the initial map, generate topology nodes based on the topology space, generate topology edges for the topology nodes, assign semantic space information for the topology space based on the topology nodes and the topology edges, and generate a semantic map.

[0018] Mapping the environment and generating an initial map through the sensors of the robot may include collecting data of the environment through LiDAR and IMU sensors, estimating and optimizing the position of the robot based on the collected data, generating a 3D map through a 3D point cloud, and converting the 3D map into a 2D occupancy grid map.

[0019] Generating a topology space through data preprocessing based on the above initial map may include binarizing the above 2D occupancy grid map to convert it into a binarized map, generating a distance transformation map through distance transformation based on the above binarized map, and generating the topology space by normalizing the distance transformation map.

[0020] Creating topology nodes based on the above topology space may include setting the size of grid pixels to reflect the size of the robot and setting the spacing between nodes based on the size of the grid pixels.

[0021] Generating a topology edge for the above topology node may include searching for the distance between nodes based on the interval between the nodes and generating the edge connecting adjacent nodes where the shortest distance falls within a preset minimum distance.

[0022] Searching for distances between nodes based on the above-mentioned interval between nodes and generating the above-mentioned edges connecting adjacent nodes where the shortest distance falls within a preset minimum distance may include optimizing the search for distances between nodes using a KD Tree, connecting adjacent nodes based on the above-mentioned minimum distance and generating edge information, checking for obstacles between edges based on the above-mentioned edge information and deleting some edges that have obstacles, checking whether the generated edge information duplicates existing stored edge information, and updating the edge information.

[0023] Assigning semantic spatial information to the topology space and generating a semantic map based on the topology nodes and topology edges may include grouping the topology space using the topology nodes and topology edges, semantically distinguishing the topology space, and assigning an ID to the distinguished topology space.

[0024] Grouping the topology space using the topology nodes and topology edges may include initializing all nodes into independent groups using the Union-Find algorithm, and determining nodes connected to each other into the same group using the edges.

[0025] Generating a topology space through data preprocessing based on the above initial map may include dividing the topology space into an open space where the robot can move and a closed space where the robot cannot move.

[0026] Dividing the above topology space into an open space where the robot can move and a closed space where the robot cannot move may include generating the open space through preprocessing on the initial map, defining the open space as a Region of Interest (ROI), and determining the area outside the ROI as the closed space. Effects of the invention

[0027] A topology-based semantic map generation device and method according to one embodiment of the present invention can efficiently generate a semantic map based on topology in a low-spec embedded environment to separate non-linear spaces, eliminate the need to build a dataset, and enable rapid adaptation to services without expensive computing devices.

[0028] A topology-based semantic map generation device and method according to one embodiment of the present invention can acquire a map in which spatial regions are separated in various indoor environments where structures are clear, and can visualize spatial information including places where a robot can go and places where it cannot. Brief explanation of the drawing

[0029] FIGS. 1 to 8 are flowcharts of a topology-based semantic map generation method according to an embodiment of the present invention. FIG. 9 is a block diagram of a topology-based semantic map generation device according to one embodiment of the present invention. FIG. 10 is an illustrative diagram for explaining a topology-based semantic map generation process according to an embodiment of the present invention. FIG. 11 is a drawing for explaining a computing device according to an embodiment of the present invention. Specific details for implementing the invention

[0030] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0031] Throughout the specification and claims, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another.

[0032] Terms such as "...part," "...unit," and "module" as used in the specification may refer to a unit capable of processing at least one function or operation described in this specification, and may be implemented as hardware or a circuit, software, or a combination of hardware or a circuit and software.

[0033] In addition, at least some configurations or functions of the topology-based semantic map generation device and method according to the embodiments described below may be implemented as a program or software, and the program or software may be stored on a computer-readable medium.

[0034] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0035] FIGS. 1 to 8 are flowcharts of a topology-based semantic map generation method according to an embodiment of the present invention.

[0036] A topology-based semantic map generation method according to one embodiment of FIGS. 1 to 8 can be performed through a topology-based semantic map generation device (100, see FIG. 9).

[0037] A topology-based semantic map generation device (100) according to one embodiment can execute program code or instructions loaded into one or more memory devices through one or more processors.

[0038] For example, the topology-based semantic map generation device (100) may be implemented as a computing device (900) as described below in relation to FIG. 11. In this case, one or more processors may correspond to the processor (910) of the computing device (900), and one or more memory devices may correspond to the memory (930) of the computing device (900).

[0039] Program code or instructions can be executed by one or more processors to generate a semantic map in a mobile robot in an embedded environment based on topology.

[0040] FIG. 1 is a flowchart illustrating a topology-based semantic map generation method according to one embodiment.

[0041] In FIG. 1, the topology-based semantic map generation device (100) can collect sensor information from the robot's sensors (step S100).

[0042] For example, the robot's sensors may include LiDAR sensors, RGB-D sensors, and IMU sensors.

[0043] The topology-based semantic map generating device (100) can generate 3D and / or 2D maps using a LiDAR-based platform (step S200).

[0044] The topology-based semantic map generation device (100) can acquire a topology space through a data preprocessing process (step S300).

[0045] For example, a topology-based semantic map generating device (100) can acquire an open space where a robot can move and a closed space where a robot cannot move, respectively, as a topology space.

[0046] In other words, topological space includes both open space and closed space.

[0047] A topology-based semantic map generation device (100) can first acquire an open space and then sequentially acquire a closed space based on the open space.

[0048] The topology-based semantic map generation device (100) can generate a node for the current topology space (step S400).

[0049] The topology-based semantic map generation device (100) can generate nodes by using the generated topology space map and taking into account the size of the robot.

[0050] The topology-based semantic map generation device (100) can generate an edge for the current topology node (step S500).

[0051] A topology-based semantic map generation device (100) can give spatial meaning to a semantic space and generate a semantic map (step S600).

[0052] A topology-based semantic map generation device (100) can provide a unique ID to the topology space using acquired topology graph information (nodes and edges).

[0053] FIG. 2 is a flowchart for explaining the 3D / 2D Map generation step (step S200) using the LiDAR-based platform of FIG. 1.

[0054] In FIG. 2, the topology-based semantic map generation device (100) can collect data using LiDAR and IMU sensors (step S210).

[0055] A topology-based semantic map generation device (100) can collect data of the surrounding environment using LiDAR and IMU sensors of a robot or handheld device.

[0056] The topology-based semantic map generation device (100) can process data to perform position estimation and optimization (step S220).

[0057] That is, the topology-based semantic map generation device (100) can extract features of LiDAR data from the SLAM (Simultaneous Localization and Mapping) Front-End and estimate the position of the robot.

[0058] The topology-based semantic map generation device (100) can perform optimization by utilizing IMU data in the SLAM Back-End.

[0059] The topology-based semantic map generating device (100) can generate a 3D map and generate a 3D PointCloud or OctoMap (step S230).

[0060] The topology-based semantic map generating device (100) can convert 3D data into a 2D occupancy grid map (Occupancy Grid Map (e.g., PGM format)) (step S240).

[0061] Figure 3 is a flowchart illustrating the topology space acquisition step (step S300) through the data preprocessing process of Figure 1.

[0062] In FIG. 3, the topology-based semantic map generation device (100) can prepare a 2D occupancy grid map mapped from a robot (step S310).

[0063] A topology-based semantic map generation device (100) can generate a binarized map that distinguishes between free space (open space) and non-free space (closed space) by binarizing a 2D occupied grid map (step S320).

[0064] The topology-based semantic map generation device (100) can generate a distance transformation map based on a binarized map through a distance transformation algorithm (step S330).

[0065] At this time, the topology-based semantic map generation device (100) can generate a distance transformation map by calculating the shortest distance from a white pixel to the nearest black pixel within a 5 x 5 area (kernel) around the pixel.

[0066] The topology-based semantic map generation device (100) can generate a final topology space by normalizing the distance transformation map (step S340).

[0067] Figure 4 is a flowchart illustrating the node creation step (step S400) for the current topology space of Figure 1.

[0068] A topology-based semantic map generation device (100) can generate nodes using a binarized map, robot size, and resolution size as inputs using the Particle Uniformly Generate On Free Space function.

[0069] In FIG. 4, the topology-based semantic map generating device (100) can prepare a topology space map containing the generated topology space (step S410).

[0070] The topology-based semantic map generation device (100) can resize the map to which nodes are to be generated using padding size (step S420).

[0071] The topology-based semantic map generation device (100) can remove noise parts by trimming the edges of the map image.

[0072] The topology-based semantic map generation device (100) can set the size of the grid pixels by taking into account the size of the robot (step S430).

[0073] For example, if the robot size is 1m when the resolution is 0.05 at actual scale, the robot size on the grid map is 50.

[0074] The grid size determines the grid interval at which nodes are created. For example, if the grid size is 10, nodes can be created at intervals of 0.2m. Padding is the process of extracting meaningful areas from a map image.

[0075] For example, a topology-based semantic map generating device (100) can call a pixel coordinate system by a grid size and if the value of the corresponding pixel is 255, it can be considered as a Topology Space.

[0076] The topology-based semantic map generating device (100) can set the spacing between nodes by taking into account the size of the grid pixels (step S440).

[0077] The topology-based semantic map generating device (100) can secondarily determine whether the topology space is one that the robot can actually pass through by using a Radius Distance that considers the size of the robot based on the spacing between nodes in the pixel coordinates that are primarily considered as the topology space (step S440).

[0078] The topology-based semantic map generation device (100) can store nodes that reflect the set pixel size and the spacing between nodes (step S450).

[0079] The topology-based semantic map generation device (100) can store points that satisfy both primary and secondary judgments as nodes.

[0080] The topology-based semantic map generating device (100) can generate nodes according to robot grid pixel size (e.g., 5 pixels).

[0081] The topology-based semantic map generating device (100) can remove grid pixels from the Node candidate group if the Pixel information around the robot size node is black (0).

[0082] FIG. 5 is a flowchart illustrating the edge generation step (step S500) for the current topology node of FIG. 1.

[0083] A topology-based semantic map generating device (100) can search for distances between nodes based on the spacing between nodes and generate edges connecting adjacent nodes where the shortest distance between nodes falls within a preset minimum distance.

[0084] In FIG. 5, the topology-based semantic map generation device (100) can generate a KD TREE MAP for the stored nodes (step S510).

[0085] A KD TREE MAP is a data structure-based map that enables efficient storage and exploration of point data in space, and may be a tree structure that enables fast searching by dividing a k-dimensional (2D, 3D, etc.) space.

[0086] The topology-based semantic map generation device (100) can generate edges by searching for nodes within the minimum distance among each node using the generated KDTREE MAP (step S520).

[0087] That is, the topology-based semantic map generating device (100) can perform pair operations between each Node through a preset minimum distance (e.g., 50 pixels) via Nearest Search.

[0088] The topology-based semantic map generation device (100) can check if there are obstacles among the generated edges and, if there are obstacles, delete the edges between the corresponding nodes (step S530).

[0089] The topology-based semantic map generating device (100) checks whether the generated edge information overlaps with existing stored edge information, and if it overlaps, it may not store it (step S540).

[0090] The topology-based semantic map generation device (100) can update existing edges with newly generated edges and save them (step S550).

[0091] FIG. 6 is a flowchart illustrating the steps of assigning spatial meaning to the semantic space of FIG. 1 and generating a semantic map (step S600).

[0092] In FIG. 6, the topology-based semantic map generating device (100) can receive acquired topology graph information (nodes and edges) (step S610).

[0093] The topology-based semantic map generation device (100) can initialize a Union Find Tree based on stored node information (step S620).

[0094] A Union-Find Tree can be a data structure that efficiently handles the management of multiple sets.

[0095] The topology-based semantic map generation device (100) can generate a Union Find Tree based on stored edge information and initialization information (step S630).

[0096] The topology-based semantic map generation device (100) can perform Tree Clustering (grouping) by grouping nodes into clusters using a Union-Find data structure (step S640).

[0097] For example, a topology-based semantic map generating device (100) groups each pair in the edge list into the same set.

[0098] A topology-based semantic map generating device (100) can find a representative root belonging to each point (node) and form a cluster based on this.

[0099] A topology-based semantic map generation device (100) can group topology spaces using generated topology nodes and topology edges.

[0100] That is, the topology-based semantic map generation device (100) initializes all nodes into independent groups through the Union-Find algorithm and can determine nodes connected to each other using edges as the same group.

[0101] A topology-based semantic map generation device (100) can generate a semantic map by semantically separating grouped topology spaces into semantic spaces and assigning a unique ID to each (step S650).

[0102] FIG. 7 is a flowchart illustrating a topology-based semantic map generation method according to one embodiment.

[0103] Figure 7 is a flowchart illustrating the steps of dividing a topology space into an open space where a robot can move and a closed space where a robot cannot move.

[0104] Figure 7 is a flowchart illustrating the steps of first generating a semantic map for open space and then generating a semantic map for closed space.

[0105] In FIG. 7, the topology-based semantic map generation device (100) can prepare a topology space (open space) that is generated first for the area where the robot can go (step S710).

[0106] The topology-based semantic map generation device (100) can generate a topology space (closed space) for an area that a robot cannot go to through ROI setting and image binarization (step S720).

[0107] That is, the topology-based semantic map generation device (100) can define a predetermined open space as an ROI and determine an area outside the ROI as a closed space.

[0108] The topology-based semantic map generation device (100) can generate topology nodes for a determined closed space (step S730). A method for generating topology nodes for a closed space can be referenced from the one described in FIG. 4.

[0109] The topology-based semantic map generation device (100) can generate topology edges based on topology nodes for a closed space (step S740). The method for generating edges can be described by referring to FIG. 5.

[0110] The topology-based semantic map generation device (100) can finally generate a semantic map by assigning an ID to each of the closed spaces through semantic distinction (step S750).

[0111] Figure 8 is a diagram illustrating the step of assigning spatial meaning to a semantic space.

[0112] In FIG. 8, the topology-based semantic map generating device (100) can obtain (n) the number of nodes with a radius of 100 pixels (r) based on the center coordinates of the semantic open space using Kdtree (step S811).

[0113] Additionally, the topology-based semantic map generation device (100) can obtain (n) the number of nodes with a radius of 50 pixels with a radius (r) based on the center coordinates of the semantic close space using Kdtree (step S821).

[0114] The topology-based semantic map generation device (100) can calculate density based on the number of nodes acquired for each of the open space and the closed space (steps S812, S822).

[0115] A topology-based semantic map generating device (100) calculates the original area (A) based on the surrounding radius (r) and determines the density (D) as the result of dividing the number of nodes (n) by the original area (A).

[0116] The topology-based semantic map generation device (100) can assign a room characteristic to the space where a robot can go if the density of the open space is greater than a predetermined threshold (e.g., 0.9) (step S813).

[0117] The topology-based semantic map generating device (100) assigns an unknow characteristic if the density of the open space is less than or equal to a predetermined threshold (e.g., 0.9) (step S814). The unknow characteristic may be a space that a robot can go to, which is not a room type.

[0118] The topology-based semantic map generation device (100) can define an open semantic space including spaces with assigned characteristics (step S815).

[0119] The topology-based semantic map generating device (100) can assign an unknown characteristic to the entire structure of a building if the density of the closed space is greater than a predetermined threshold (e.g., 0.9) (step S823).

[0120] The topology-based semantic map generation device (100) can assign pillar characteristics if the density of the closed space is less than or equal to a predetermined threshold (e.g., 0.9) (step S824). The topology-based semantic map generation device (100) can define a closed semantic space including the spaces to which the characteristics are assigned (step S825).

[0121] The topology-based semantic map generation device (100) combines semantic information of an open semantic space with characteristics and semantic information of a closed space with characteristics (steps S830 and S840).

[0122] The topology-based semantic map generation device (100) can provide open semantic space and closed semantic space that can be defined by the user (step S850).

[0123] The topology-based semantic map generation device (100) can generate a semantic map containing semantic spaces determined according to user definition (step S860).

[0124] FIG. 9 is a block diagram of a topology-based semantic map generation device according to one embodiment of the present invention.

[0125] Referring to FIG. 9, the topology-based semantic map generation device (100) may include a mapping module (110), a data preprocessing module (120), a topology node generation module (130), a topology edge generation module (140), and a semantic map generation module (150).

[0126] The mapping module (110) can map the environment through the robot's sensors and generate an initial map.

[0127] The mapping module (110) can generate an occupancy grid map (OGM) through LiDAR-based mapping.

[0128] The data preprocessing module (120) can generate a topology space through data preprocessing based on the initial OGM map.

[0129] The data preprocessing module (120) can convert the OGM map into a binarized map and generate a distance conversion map based on the binarized map.

[0130] The topology node creation module (130) can create topology nodes based on the topology space.

[0131] The topology node generation module (130) normalizes the distance transformation map to generate a topology space.

[0132] The topology node generation module (130) can determine the space that the robot can go and the space that it cannot go based on the distance conversion map, determine the space that the robot can go as an open space, and determine the space that it cannot go as a closed space.

[0133] The topology node creation module (130) can determine the closed space based on the ROI designated as the open space.

[0134] The topology node creation module (130) can create nodes of a topology graph including open space and closed space.

[0135] For example, the topology node generation module (130) can set the size of the grid pixels by reflecting the size of the robot and set the spacing between nodes based on the size of the grid pixels.

[0136] The topology edge generation module (140) can generate topology edges for topology nodes.

[0137] The topology edge generation module (140) can generate edges for each of the open space topology graph and the closed space topology graph.

[0138] The topology edge generation module (140) can generate an edge connecting adjacent nodes for each of the open space topology graph and the closed space topology graph by searching for the distance between nodes based on the spacing between nodes and the shortest distance within a preset minimum distance.

[0139] The semantic map generation module (150) can generate a semantic map by providing semantic space information for the topology space based on topology nodes and topology edges.

[0140] The semantic map generation module (150) can generate semantic information of open space and semantic information of closed space, respectively.

[0141] The semantic map generation module (150) can distinguish the space of the open space based on the semantic information of the open space.

[0142] The semantic map generation module (150) can distinguish the space of the closed space based on the semantic information of the closed space.

[0143] The semantic map generation module (150) can assign meaning to the separate spaces of the open space and the closed space according to the user's definition of the semantic information of the open space.

[0144] FIG. 10 is an illustrative diagram for explaining a topology-based semantic map generation process according to an embodiment of the present invention.

[0145] In FIG. 10, when a topology-based semantic map generating device (100) receives an occupancy grid map (10), it can generate a topology space map (20) through binarization, distance transformation, and normalization.

[0146] A topology-based semantic map generating device (100) can generate a first topology node (30) for open space in a topology space map (20).

[0147] A topology-based semantic map generating device (100) can generate a first topology edge (40) based on a first topology node (30).

[0148] A topology-based semantic map generation device (100) can semantically distinguish open spaces based on a first topology node (30) and a first topology edge (40) and generate a first semantic map (50).

[0149] A topology-based semantic map generation device (100) can generate a second topology node (60) for a closed space excluding an open space based on a first semantic map (50).

[0150] The topology-based semantic map generation device (100) can generate a second topology edge (70) based on a second topology node (60) in a closed space.

[0151] A topology-based semantic map generation device (100) can generate a final semantic map (80) by semantically separating closed spaces based on a second topology node (60) and a second topology edge (70) of a closed space.

[0152] FIG. 11 is a drawing for explaining a computing device according to an embodiment of the present invention.

[0153] Referring to FIG. 11, a topology-based semantic map generation device and method according to embodiments can be implemented using a computing device (900).

[0154] The computing device (900) may include at least one of a processor (910), memory (930), user interface input device (940), user interface output device (950), and storage device (560) that communicate via a bus (920). The computing device (900) may also include a network interface (970) that is electrically connected to a network (90). The network interface (970) may transmit or receive signals to or from other entities via the network (90).

[0155] The processor (910) can be implemented in various types such as an MCU (Micro Controller Unit), AP (Application Processor), CPU (Central Processing Unit), GPU (Graphic Processing Unit), NPU (Neural Processing Unit), etc., and may be any semiconductor device that executes instructions stored in memory (930) or storage device (960). The processor (910) may be configured to implement the functions and methods described above in relation to FIGS. 1 to 10.

[0156] The memory (930) and storage device (960) may include various forms of volatile or non-volatile storage media. For example, the memory may include ROM (read-only memory) (931) and RAM (random access memory) (932). In this embodiment, the memory (930) may be located inside or outside the processor (910), and the memory (930) may be connected to the processor (910) through various known means.

[0157] In some embodiments, at least some configurations or functions of the topology-based semantic map generation device and method according to the embodiments may be implemented as a program or software executed on a computing device (900), and the program or software may be stored on a computer-readable medium.

[0158] In some embodiments, at least some configurations or functions of the topology-based semantic map generation device and method according to the embodiments may be implemented using hardware or circuits of the computing device (900), or may be implemented using separate hardware or circuits that can be electrically connected to the computing device (900).

[0159] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art to which the present invention belongs, utilizing the basic concept of the present invention as defined in the following claims, also fall within the scope of the present invention. Explanation of the symbols

[0160] 100: Topology-based semantic map generator 110: Mapping Module 120: Data Preprocessing Module 130: Topology Node Creation Module 140: Topology Edge Generation Module 150: Semantic Map Generation Module

Claims

Claim 1 A topology-based semantic map generation method performed by a computing device including a processor and memory, comprising: a step in which the processor maps an environment through a sensor of a robot and generates an initial map; a step in which the processor generates a topology space through data preprocessing based on the initial map; a step in which the processor generates a topology node based on the topology space; a step in which the processor generates a topology edge for the topology node; and a step in which the processor assigns semantic space information for the topology space and generates a semantic map based on the topology node and the topology edge. Claim 2 A topology-based semantic map generation method according to claim 1, wherein the step of mapping an environment and generating an initial map through a sensor of the robot comprises: a step in which the processor collects data of the environment through a LiDAR and IMU sensor; a step in which the processor estimates the position of the robot based on the collected data; a step in which the processor generates a 3D map through a 3D point cloud; and a step in which the processor converts the 3D map into a 2D occupancy grid map. Claim 3 In claim 2, the step of generating a topology space through data preprocessing based on the initial map comprises: a step in which the processor converts the 2D occupancy grid map into a binarized map by binarizing it; a step in which the processor generates a distance transformation map through distance transformation based on the binarized map; and a step in which the processor generates the topology space by normalizing the distance transformation map. Claim 4 A method for generating a topology-based semantic map according to claim 3, wherein the step of generating topology nodes based on the topology space comprises: a step in which the processor sets the size of grid pixels reflecting the size of the robot; and a step in which the processor sets the spacing between nodes based on the size of the grid pixels. Claim 5 A method for generating a topology-based semantic map according to claim 4, wherein the step of generating a topology edge for the topology node comprises the step of the processor searching for a distance between nodes based on the interval between nodes and generating the edge connecting adjacent nodes where the distance falls within a preset minimum distance. Claim 6 A method for generating a topology-based semantic map according to claim 5, wherein the step of searching for distances between nodes based on the interval between nodes and generating edges connecting adjacent nodes where the distance falls within a preset minimum distance comprises: a step in which the processor searches for distances between nodes using a KD Tree; a step in which the processor connects adjacent nodes based on the minimum distance and generates edge information; a step in which the processor checks for obstacles between edges based on the edge information and deletes some edges containing obstacles; and a step in which the processor checks whether the generated edge information overlaps with existing stored edge information and updates the edge information. Claim 7 In claim 6, the step of assigning semantic space information for the topology space and generating a semantic map based on the topology nodes and the topology edges comprises: the step of the processor grouping the topology space using the topology nodes and the topology edges; the step of the processor semantically distinguishing the topology space; and the step of the processor assigning an ID to the distinguished topology space. Claim 8 A method for generating a topology-based semantic map according to claim 7, wherein the step of grouping the topology space using the topology nodes and the topology edges comprises: an initialization step in which the processor sets all nodes into independent groups through a Union-Find algorithm; and a step in which the processor determines nodes connected to each other to be in the same group using the edges. Claim 9 A method for generating a topology-based semantic map according to claim 1, wherein the step of generating a topology space through data preprocessing based on the initial map comprises the step of the processor dividing the topology space into an open space in which the robot can move and a closed space in which the robot cannot move. Claim 10 A method for generating a topology-based semantic map according to claim 9, wherein the step of dividing the topology space into an open space in which the robot can move and a closed space in which the robot cannot move comprises: a step in which the processor generates the open space through preprocessing on the initial map; and a step in which the processor defines the open space as a Region of Interest (ROI) and determines the area outside the ROI as the closed space. Claim 11 A topology-based semantic map generation device that generates a semantic map in a mobile robot in an embedded environment based on topology by executing program code loaded into one or more memory devices through one or more processors, wherein the program code is executed to map an environment through a robot's sensor and generate an initial map, generate a topology space through data preprocessing based on the initial map, generate topology nodes based on the topology space, generate topology edges for the topology nodes, and assign semantic space information for the topology space based on the topology nodes and the topology edges and generate a semantic map. Claim 12 A topology-based semantic map generation device according to claim 11, wherein mapping an environment and generating an initial map through the sensors of the robot comprises collecting data of the environment through LiDAR and IMU sensors, estimating the position of the robot based on the collected data, generating a 3D map through a 3D point cloud, and converting the 3D map into a 2D occupancy grid map. Claim 13 A topology-based semantic map generation device according to claim 12, wherein generating a topology space through data preprocessing based on the initial map comprises binarizing the 2D occupancy grid map to convert it into a binarized map, generating a distance transformation map through distance transformation based on the binarized map, and generating the topology space by normalizing the distance transformation map. Claim 14 A topology-based semantic map generation device according to claim 13, wherein generating topology nodes based on the topology space includes setting the size of grid pixels by reflecting the size of the robot and setting the spacing between nodes based on the size of the grid pixels. Claim 15 A topology-based semantic map generation device according to claim 14, wherein generating a topology edge for the topology node comprises searching for the distance between nodes based on the interval between nodes and generating the edge connecting adjacent nodes where the distance falls within a preset minimum distance. Claim 16 A topology-based semantic map generation device according to claim 15, wherein the method of searching for distances between nodes based on the interval between nodes and generating edges connecting adjacent nodes where the distance falls within a preset minimum distance includes searching for distances between nodes using a KD Tree, connecting adjacent nodes based on the minimum distance and generating edge information, checking for obstacles between edges based on the edge information and deleting some edges with obstacles, checking whether the generated edge information duplicates existing stored edge information, and updating the edge information. Claim 17 A topology-based semantic map generation device according to claim 16, wherein the method of assigning semantic space information to the topology space and generating a semantic map based on the topology nodes and topology edges comprises grouping the topology space using the topology nodes and topology edges, semantically distinguishing the topology space, and assigning an ID to the distinguished topology space. Claim 18 A topology-based semantic map generation device according to claim 17, wherein grouping the topology space using the topology nodes and the topology edges includes initializing all nodes into independent groups through a Union-Find algorithm and determining nodes connected to each other into the same group using the edges. Claim 19 A topology-based semantic map generation device according to claim 11, wherein generating a topology space through data preprocessing based on the initial map includes dividing the topology space into an open space where the robot can move and a closed space where the robot cannot move. Claim 20 A topology-based semantic map generation device according to claim 19, wherein the topology space is divided into an open space in which the robot can move and a closed space in which the robot cannot move, the method comprises generating the open space through preprocessing on the initial map, defining the open space as a Region of Interest (ROI), and determining the area outside the ROI as the closed space.

Images