Method for controlling robot on basis of hybrid map

Abstract

An exemplary embodiment of the present invention discloses a method for controlling a robot on the basis of a hybrid map, comprising the steps of: receiving satellite information of a target area; generating a first map of the target area on the basis of the satellite information; dividing an area in the first map according to preset criteria; receiving three-dimensional information (point cloud data) about an area including a road on which the robot can travel among the divided areas; generating a second map on the basis of the three-dimensional information; generating a hybrid map by overlapping and aligning the first map and the second map; and determining the position of the robot within the target area, wherein the robot drives using one of maps of different formats forming the hybrid map according to the position within the target area.

Description

Control method for hybrid map-based robots

[0001] The present invention relates to a method for estimating the position of a robot and selecting a map generated based on one format from a map formed by overlapping different formats based on the estimated position.

[0002] The map required for autonomous driving of a mobile robot refers to information that digitizes the space in which the robot moves and works, and is divided into two types: metric and topology depending on the form of expression of the map.

[0003] The dual metric format is a method of precisely expressing the robot's surroundings in the form of occupied / unoccupied, and is widely used in the SLAM (Simultaneous Localization And Mapping) method, in which the robot uses its own sensors to recognize the environment and form a map based on that information, and it is a method of updating the map by probabilistically modeling the uncertainty generated from the sensors.

[0004] This type of map is one in which a robot can recognize and create its own environment in an unknown environment, and has the advantage of being able to estimate the robot's location using the created map information. However, because a lot of memory is required for map creation and the environment is only expressed as occupied / unoccupied, there is a problem in that, for example, parts that connect spaces cannot be expressed as semantic information such as doors, stairs, and hallways. Therefore, maps using a semantic segmentation technique are generally used to recognize areas.

[0005] Meanwhile, the topology form is a method of simply and clearly expressing the environment in a graphical form, and has the advantage of being able to efficiently express a large space with a small memory and simplifying path planning for driving, but has the problem that it is difficult for the robot to find topology information on its own and it is difficult to estimate the robot's location using only map information.

[0006] Therefore, as the target space expands due to the development of autonomous driving technology for robots, more memory is required to create maps. If a map requiring a large amount of memory is used, the autonomous driving performance of the robot may deteriorate, so a solution to this problem is needed.

[0007] Furthermore, existing robot control systems are limited to receiving and controlling information about robots of the same type, making simultaneous control of heterogeneous robots impossible. Furthermore, when transmitting the same content to multiple heterogeneous robots, the problem arises of having to input content tailored to each robot type.

[0008] In addition, there is an inconvenient problem in using robots because commands cannot be automatically given or assigned to robots from legacy systems used for existing work, and commands must be given manually to robots directly from the control system for each robot type.

[0009] The number of robots and the corresponding increase in the number of robots are increasing. While this increases productivity in terms of replacing humans, the resulting increase in management complexity and time is problematic. Therefore, a system that integrates and controls the management of diverse robots is needed.

[0010] In addition, in existing robot control systems, there is a problem that when heterogeneous robots interfere with each other while performing individual missions assigned to each robot, the user must manually control each robot to directly resolve the interference situation.

[0011] Accordingly, the present invention aims to improve the autonomous driving performance of a robot by generating a driving path of a robot through a hybrid map generated by overlapping and aligning a plurality of maps formed in different formats for a target space, and selecting a map of an appropriate format based on the generated driving path to generate a simple map required for driving of the robot.

[0012] One of the various tasks of the present invention is to provide a system that can detect and control interference situations that may occur when heterogeneous robots perform their respective assigned tasks.

[0013] In particular, when the movement paths of heterogeneous robots overlap during movement, a control method is provided that grants driving priority to one robot.

[0014] One of the various tasks of the present invention is to provide a criterion for selecting one of various map data included in a hybrid map based on the estimated location of a mobile robot.

[0015] One of the various tasks of the present invention is to provide a criterion for selecting a map data transition point between various map data included in a hybrid map based on a driving path of a mobile robot, a mission scenario, and the location of the robot.

[0016] Various embodiments for solving the problem of the present invention include a step of receiving satellite information of a target area, a step of generating a first map of the target area based on the satellite information, a step of dividing an area in the first map according to a preset standard, a step of receiving three-dimensional information (Point Cloud Data) about an area including a road on which the robot can drive among the divided areas, a step of generating a second map based on the three-dimensional information, a step of generating a hybrid map by overlapping and aligning the first map and the second map, and a step of determining the position of the robot within the target area, and a method for controlling a hybrid map-based robot can be provided, characterized in that the robot drives using any one of maps of different formats forming the hybrid map according to the position within the target area.

[0017] The above first map may be characterized as being a two-dimensional map of the target area based on the satellite information.

[0018] The above-mentioned preset standard may be characterized by whether or not the road is one on which the robots can drive.

[0019] The above hybrid map may be characterized in that the first map and the second map are generated by overlapping and aligning based on the boundary line of an area that meets the preset criteria.

[0020] The above hybrid map may be characterized in that it is generated by converting the coordinates of the first map into a local coordinate system and then overlapping and aligning them with the second map.

[0021] The method may further include a step of setting a driving path of the robot, wherein the robot drives using one of maps of different formats forming the hybrid map according to a location on the driving path.

[0022] The method may further include a step of generating a spatial map based on the above-mentioned three-dimensional information, a step of converting the spatial map into a 2D map, and a step of generating a third map by overlapping and aligning the first map with the converted 2D map, wherein the hybrid map may be generated by overlapping and aligning at least two maps among the first map, the second map, the third map, and the spatial map.

[0023] The above second map may be characterized as being a feature map generated based on feature points extracted from the above spatial map.

[0024] The method may further include a step of setting a driving path of the robot, and the spatial map may be generated based on three-dimensional information collected along the driving path of the robot.

[0025] The second map may include information generated based on feature points of objects existing on the driving route, and the third map may include two-dimensional location information of the objects.

[0026] The robot may be characterized in that it drives using one of different formats of maps that form the hybrid map based on the current location and information about the objects within the target area.

[0027] The robot may be characterized in that it drives using either the second map or the third map depending on the current location of the robot and the distance from the objects.

[0028] The method may further include a matching judgment step for judging whether the matching degree of a reference area is greater than a preset reference value when overlapping and aligning the first map with the converted 2D map.

[0029] It may be characterized in that the step of converting the above spatial map into a 2D map, the step of generating the third map, and the step of determining the degree of alignment are performed sequentially.

[0030] The step of converting the above-mentioned spatial map into a 2D map, the step of determining the degree of conformity, and the step of generating the third map are sequentially performed, and the step of generating the third map may be characterized in that it is performed when the degree of conformity of the reference area is greater than a preset reference value in the step of determining the degree of conformity.

[0031] The above hybrid map may be characterized in that the second map and the third map are generated by overlapping and aligning them.

[0032] An exemplary embodiment of the present invention provides a method for controlling a hybrid map-based robot, including the steps of: receiving satellite information of a target area; generating a first map of the target area based on the satellite information; dividing an area in the first map according to a preset criterion; setting a driving path of a robot; receiving three-dimensional information (Point Cloud Data) for an area including the driving path of the robot among the divided areas; generating a second map based on the three-dimensional information; overlapping and aligning the first map and the second map to create a hybrid map; and determining a position of the robot within the target area, wherein the robot drives using any one of maps of different formats forming the hybrid map according to the position within the target area.

[0033] An exemplary embodiment of the present invention provides a control device for a hybrid map-based robot that drives a target area, the control device including a communication unit that transmits and receives information necessary for map generation, a map generation unit that generates a map of the target area by overlapping and aligning two or more maps having different formats, a location estimation unit that determines a location of the robot, and a processor that selects one of the two or more maps having different formats according to the location of the robot, wherein the map generation unit overlapping and aligns a first map generated based on satellite information of the target area and a second map generated based on three-dimensional information of the target area, and the processor selects one of the first map and the second map as a map for driving of the robot based on a current location of the robot and location information of objects included in the three-dimensional information.

[0034] An exemplary embodiment of the present invention can provide a program stored on a computer-readable recording medium including a program code for executing the control method of the hybrid map-based robot described above.

[0035] An exemplary embodiment of the present invention can provide a computer-readable recording medium including a program code for executing the above-described method for controlling a hybrid map-based robot.

[0036] Each feature of the above-described embodiments may be implemented in combination in other embodiments as long as it is not inconsistent with or exclusive of other embodiments.

[0037] According to various embodiments of the present invention, a robot can be operated efficiently by creating a single integrated map without being limited to the specifications of the robot, the mission assigned to the robot, the driving target environment of the robot, etc.

[0038] In addition, when a situation arises where heterogeneous robots interfere with each other while performing their respective assigned tasks, priority can be given to one robot to prevent collisions between robots, hinder mission performance, and occurrence of abnormal situations.

[0039] The effects of the present invention are not limited to those described above, and other effects not mentioned will be clearly recognized by those skilled in the art from the description below.

[0040] FIG. 1 is a drawing showing a configuration for robot control and configurations of a robot according to an exemplary embodiment of the present invention.

[0041] FIG. 2 is a drawing showing the configuration of a robot control system according to an exemplary embodiment of the present invention.

[0042] FIG. 3 and FIG. 4 are drawings showing a map generation method according to an exemplary embodiment of the present invention.

[0043] Figures 5 to 13 are drawings showing maps according to exemplary embodiments of the present invention.

[0044] FIG. 14 and FIG. 15 are drawings showing a robot according to an exemplary embodiment of the present invention.

[0045] FIG. 16 and FIG. 17 are drawings showing a method for generating a driving path of a robot according to an exemplary embodiment of the present invention.

[0046] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, these are merely examples and the present invention is not limited thereto.

[0047] In describing embodiments of the present invention, detailed descriptions of known technologies related to the present invention will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined based on their functions in the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0048] The terminology used in this detailed description is for the purpose of describing embodiments of the present invention only and should not be construed as limiting. Unless expressly stated otherwise, singular forms include plural forms.

[0049] In this description, expressions such as "including" or "having" are intended to indicate certain features, numbers, steps, operations, elements, portions or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, portions or combinations thereof other than those described.

[0050] Additionally, in describing components of embodiments of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by the terms.

[0051] FIG. 1 is a drawing showing a configuration for map generation and a configuration of a robot according to an exemplary embodiment of the present invention, and FIG. 2 is a drawing showing a configuration of a robot control system according to an exemplary embodiment of the present invention.

[0052] The following description is provided with reference to Figures 1 and 2.

[0053] The robot control system of this embodiment may include a robot (10), a database (30), and a server (50).

[0054] The robot (10) may include a robot of various shapes and configurations that can drive indoors and outdoors, and may perform a mission by transmitting and receiving information such as map information, driving information, and route information with a database (30) and a server (50).

[0055] Meanwhile, referring to FIG. 1, the robot (10) of the present embodiment may include a control unit (210), a movement module (220), a sensor unit (230), a photographing unit (240), a display unit (250), a storage unit (260), and a communication unit (270).

[0056] The movement module (220) may be composed of a motor, a gear assembly, and wheels, and may provide driving force for the movement (driving) of the robot. Wheel-based robots mainly use DC motors and gears to control direction and speed, and this movement module allows the robot to move precisely to a desired location.

[0057] The sensor unit (230) may include various sensors that enable the robot to obtain information about the target space while moving through the target space. In other words, the sensor unit may play a role in allowing the robot to recognize the environment and collect data.

[0058] For example, the sensor unit (230) may include various sensors such as LIDAR, ultrasonic sensors, infrared sensors, and IMU (inertial measurement unit), and the robot may detect obstacles in the surroundings through the sensors and collect data in real time to perform path search.

[0059] The photographing unit (240) may include a real-image sensor unit (not shown) for photographing real-image images, a thermal image sensor unit (not shown) for acquiring thermal images, etc. For example, the real-image sensor unit (not shown) may be implemented as an RGB camera, and the thermal image sensor unit (not shown) may be implemented as an infrared (infra-red) camera. Through the configuration of the photographing unit (240), the robot (10) can accurately identify objects not only in a high-illuminance environment such as during the day, but also in a low-illuminance environment such as at night.

[0060] That is, the photographing unit (240) is a device necessary for the robot to collect visual data, and the robot can obtain important information necessary for object recognition, path setting, and environment mapping by analyzing the images captured by the photographing unit (240), and the above information can be used together with the information collected by the sensor unit to perform more precise environment recognition.

[0061] The display unit (250) can display the status of the robot (10), time, weather, information on the target space, etc. In terms of the robot control system of the present embodiment, for example, when the robot (10) is unable to drive or perform a mission due to the occurrence of an event, the robot (10) can transmit status information of the robot (10) to the server (50) and then display at least a portion of the status information of the robot (10) through the display unit (250) while waiting until a worker arrives at the location at which the status information was transmitted to the server (50) and a separate command is input.

[0062] That is, the display unit (250) can display the general status of the robot, information required for the mission, information to be provided to the user or worker, and of course, information required for the robot control system can also be displayed.

[0063] The storage unit (260) can store programs for the operation of the robot (10), map information, captured images, scenarios for performing missions, etc. More specifically, the storage unit (260) can store data collected while the robot (10) is driving, or information received from a database (30) or a server (50). The information may include map information, route information, sensor information, etc., and the information may be stored in real time.

[0064] The communication unit (270) may include one or more modules that enable the robot (10) to communicate with other devices (e.g., a remote control system, a server (50), another robot, a database (30), etc.).

[0065] The control unit (210) may include a processor (211), a path generation unit (2131), a driving path map generation unit (2133), a driving control unit (215), an event detection unit (217), and a location estimation unit (219).

[0066] The control unit (210) can manage and control all operations of the robot. The control unit (210) can process information collected from the sensor unit (230) and the camera unit (240), calculate the travel path of the robot (10), and issue commands to the movement module (220). In addition, the control unit (210) can also process responses to various events occurring while the robot (10) is moving.

[0067] The processor (211) can process information acquired from various components included in the robot described above. For example, the processor (211) can process information collected from the sensor unit (230) and the photographing unit (240) in real time and calculate the path and movement of the robot.

[0068] The path generation unit (2131) and the driving path map generation unit (2133) can select any one of the maps formed by arranging (merging) at least two different format maps generated through the map generation unit (280), and can generate a driving path of the robot through the path generation unit (2131) within the selected map, or can generate a map used by the robot based on the driving path generated through the driving path map generation unit (2133).

[0069] The above driving path map means a simplified map composed of nodes, edges, and context information included in each node or edge generated according to the example described below. The nodes may represent objects that are mission execution targets of the robot or points where a change in the status of the robot (e.g., change in driving direction, etc.) occurs on the driving path. The edges may connect the nodes and represent the driving path of the robot between the nodes. The context information may represent a control command value of the robot at the node or edge, information about the node, etc.

[0070] Meanwhile, the path generation unit (2131) can generate a driving path between objects using a map generated by the global map generation unit (281), and can generate a driving path more effectively by generating a path using a map generated by the local map generation unit (283) in the vicinity of the object (a predetermined area based on the object), and similarly, when modifying the generated driving path for other reasons, it will be possible to quickly respond to other reasons by modifying the driving path according to the exemplary criteria described above.

[0071] In addition, as an example, the path generation unit (2131) can generate a path candidate group including a plurality of driving paths on which the robot (10) can perform a mission in a target space (map information), and select an optimal path from the generated path candidate group based on preset criteria to generate a driving path for the robot (10).

[0072] The above-described preset criteria can be used to set the surveillance priority according to the mission of the robot (10) for objects existing on the map information, and can be used as a criterion for determining whether a surveillance blind spot occurs in the object when performing the assigned task (driving) according to the set priority.

[0073] Meanwhile, the driving route generated by the route generation unit (2131) or the driving route map generation unit (2133) may include routes for multiple robots. The routes for the multiple robots may refer to routes for robots in a master-slave relationship or routes for heterogeneous robots.

[0074] More specifically, since a situation may arise where the driving paths of a robot overlap with those of another robot when the robot is driving within the target area, the paths for the plurality of robots may mean at least a path that overlaps with the driving path of the robot among the driving paths of the other robots.

[0075] As described above, when there is an overlapping section between the driving paths of multiple robots, the decision on which robot to give driving priority to may be made from the perspective of the server (50), or may be made from the perspective of the robot (10) or the database (30), which will be described in more detail later.

[0076] Meanwhile, the driving control unit (215) can generate or transmit a signal for controlling the movement module (220). More specifically, the signal for controlling the movement module (220) may be generated by a control signal input from the server (50) through the communication unit (270), or the signal for controlling the movement module (220) may be generated based on a result of processing through the processor (211) based on information about the target space acquired from the sensor unit (230) or the photographing unit (240).

[0077] The event detection unit (217) can detect an abnormal situation occurring in any of the objects existing in the target space, an abnormal situation occurring on the robot's driving path, or an abnormal situation occurring in the robot, through the sensor unit (230), the camera unit (240), the communication unit (270), and the process (211).

[0078] The position estimation unit (219) can determine the position of a moving robot, and can estimate (and determine) the position of the robot in various ways together with the sensor unit (230), communication unit (270), processor (211), map generation unit (280), etc.

[0079] And the control unit (210) can use any one of the maps formed by overlapping different formats according to the location of the robot within the target area for the robot's driving.

[0080] The map generation unit (280) can convert the target space where the robot (10) drives into various formats and then generate a map for each format, and the map merging unit (287) can perform conversion and alignment of the maps generated in each format. The above-described map conversion can be performed through the map conversion unit (2871), and the map alignment can be performed through the map alignment unit (2872).

[0081] The area division unit (289) can perform area division on a map generated through the global map generation unit (281) or a satellite map received through the communication unit (270).

[0082] The above-described map generation unit (280), map merging unit (287), and area division unit (289) are described in more detail with reference to the drawings below.

[0083] Meanwhile, the map generation unit (280), the map merging unit (287), and the area division unit (289) may be included as a component of the robot (10) of the present embodiment. That is, the map generation unit (280), the map merging unit (287), and the area division unit (289) may be included as a sub-component (onboard system) of the robot.

[0084] However, it is not limited thereto, and from the perspective of the robot control system, it can be provided independently of the robot by being included in the database (30) and server (50). If the map generation unit (280), map merging unit (287), and area division unit (289) are provided independently of the robot, the generated map can be transmitted to the robot through the communication unit (270).

[0085] Meanwhile, the database (30) may store information on map information for the target space, multiple scenarios, driving routes, types of events, etc. For example, the map information may include various map information for expressing the target space, and may include, for example, a satellite map (2D Map), SLAM information (Simultaneous Localization and Mapping), PCD (Point Cloud Data), a topological map, a semantic map, an HD map (High-Definition Map), etc. for the target space.

[0086] The server (50) may transmit map information to the robot (10), or may assign a mission to the robot (10) or perform a robot (10) control task according to the assigned mission.

[0087] Of course, the server (50) may also serve as an information relay point between the robot (10) and the database (30) on the robot control system, and some of the above-described vehicle event type confirmation, map information transmission, and robot control command transmission may be performed in the database (30) or through a configuration installed in the robot (10).

[0088]

[0089] FIGS. 3 and 4 are drawings showing a map generation method according to an exemplary embodiment of the present invention, and FIGS. 5 to 13 are drawings showing maps according to an exemplary embodiment of the present invention.

[0090] The following description is provided with reference to Figures 1 to 13.

[0091] In this embodiment, the map used by the robot (10) to perform the mission is a map (hereinafter referred to as a hybrid map) in which maps of different formats are merged (overlapping, aligned, and stacked), and can be divided into a global map and a local map depending on the form of the map.

[0092] A global map may mean a 2D map generated through a global map generation unit (281) based on a satellite map received from a communication unit (270) or a database (30), and a local map may mean a 3D map (spatial map, feature map) generated through a local map generation unit (283) based on point cloud information (Point Cloud Data, PCD) received from a communication unit (270) or a database (30).

[0093] Of course, the point cloud information required for the above local map creation may be collected through the sensor unit (230) while the robot (10) drives for mapping the target space and may be stored in the storage unit (260).

[0094] The map merging unit (287) can generate a hybrid map using maps of different formats. More specifically, the map merging unit (287) can include a map conversion unit (2871) and a map alignment unit (2872). The map conversion unit (2871) can convert one of the different formats of maps into a reference format to generate a specific map, for example, and the map alignment unit (2872) can generate the specific map by overlapping and aligning maps that have been converted and formed into the same format.

[0095] Of course, the map alignment unit (2872) can also create a single map by overlapping and aligning map information of different formats as described above, and for example, objects that are mission targets based on the global map can be expressed as nodes, and detailed information of objects expressed as each node can be overlapping and aligning information of the local map to create a hybrid map.

[0096] More specifically, the 2D satellite map received through the communication unit (270) may be expressed as data using a geographic coordinate system (e.g., latitude, longitude), and the map generated through the local map generation unit (283) or the spatial map generation unit (285) may be expressed as data using a local coordinate system or a world coordinate system to express a 3D space (e.g., LiDAR, depth camera, etc.).

[0097] Therefore, the map merging unit (287) needs to convert the two coordinate systems used in the above-described different formats (2D format, 3D format) into one common coordinate system in order to align them.

[0098] According to one embodiment of the present invention, the map conversion unit (2871) converts the coordinates (GPS coordinates) of a 2D satellite map into a local coordinate system based on the position of the robot or a specific reference point, and the map alignment unit (2872) can align the 2D satellite map converted into the local coordinate system and the local map expressing the 3D space based on the above-described criteria.

[0099] Additionally, the map conversion unit (2871) can also find common features between a 2D satellite map and a 3D space map and set criteria for feature matching. For example, edge detection, which extracts road or building boundaries from a 2D satellite map and detects the same features in a 3D map, or an algorithm that extracts and matches feature points from a satellite map and a 3D space, can be used to find commonalities between the two maps and set criteria.

[0100] The map alignment unit (2872) aligns the center points of the 2D map and the 3D map to align the two maps that have been converted or matched as described above, adjusts the rotation angle between the two maps to match the direction of the maps, and adjusts the size of the maps as needed so that the two maps can be expressed in the same ratio.

[0101] And the map alignment unit (2872) can perform error verification (matching judgment step, S173) to check whether the two maps are well overlapped, and at this time, for example, the accuracy of matching the point cloud information of the 3D map with the boundary line or road of the 2D map can be used as a standard.

[0102] And, of course, the map generated in this way can continuously estimate the location of the robot using the SLAM (Simultaneous Localization and Mapping) algorithm as the robot (10) moves and update the alignment status between maps in real time.

[0103]

[0104] Referring to FIG. 3, a series of processes divided into steps S11 (S111 to S117) for generating a first map represents a process for generating the global map described above, a series of processes divided into steps S14 (S141 to S147) represents a process for generating a local map, and a series of processes divided into steps S16 (S161 to S165) represents a process for converting some information during the local map generation process into a 2D map and generating an intermediate map by merging the necessary information with the first map.

[0105] More specifically, first, looking at the process of creating a global map, the robot (10) can receive satellite information (S111) from a database (30) or server (50) through a communication unit (270).

[0106] The above satellite information may include aerial photographs, orthophotos, and true orthophotos of the target space, and may be expressed as photographs or images such as the image (70-1) shown in FIG. 5 or d1 shown in FIG. 13, for example.

[0107] More specifically, referring to the image (70-1) for satellite information expressed in FIG. 5, various information about the target space can be expressed in the satellite information. For example, a lake (711a, 711b), a playground (712), a parking lot (713), a road (731) on which vehicles can drive, a road (732a, 732b, 732c) that can be used by both people and robots or only by robots, a road on which people can travel (733), multiple buildings (751, 754), etc. can be expressed.

[0108] After receiving satellite information, the global map generation unit (281) can generate a first map (S117) by dividing an area (S113) and performing labeling (S115) based on the received satellite information.

[0109] The area division section (289) can express information included in the 2D Layout Data, such as the boundary of a road, the boundary of a building, and the road (d51) usable by a robot, as exemplarily expressed in d5 of FIG. 13, through area division (S113) on a satellite map.

[0110] More specifically, referring to the image (70-2) for satellite information expressed in FIG. 6, the area to be divided in the area division step (S113) may be divided into a1, a2, a3, a4, a5, a6 based on the boundary (701) based on the amount of information that the robot can receive or the size of the area in which the robot can drive. In addition, the area to be divided may be divided into predetermined areas (711a 711b) based on the boundary (703a, 703b) in which the robot cannot drive.

[0111] In addition, referring to the image (70-3) for satellite information expressed in FIG. 7, the areas divided in the area division step (S113) may be expressed as areas divided by lines of roads (7311) on which vehicles can drive, lines of roads (7321b) that can be used by both people and robots or only by robots, lines of roads (7331) on which people can travel, boundaries of multiple buildings, boundaries of areas such as playgrounds or parking lots, etc.

[0112] Meanwhile, labeling (S115) can be performed by inputting context information for each of the exemplarily divided areas, required areas, or expressed information. More specifically, by expressing selected reference points based on the robot's driving path or the robot's mission scenario as a type of node, a standard for receiving PCD (Point Cloud Data) described below can be set, and a topology map that generates the robot's driving path can be generated by expressing edges connecting the nodes on a global map.

[0113] At least one of the 2D maps generated through the above-described process may be referred to as a first map in the present embodiment. The first map refers to a map encompassing 2D maps generated for a target area based on satellite information, and may, for example, refer to a 2D satellite map for the target area based on satellite information, or may refer to a map in which areas are divided according to preset criteria based on a received satellite map, or may refer to a topology map composed of nodes and edges selected based on a robot's driving path or a robot's mission scenario.

[0114] That is, the first map may mean at least one of various types of maps that can be generated based on received satellite information.

[0115] Next, looking at the process of generating a local map, the robot (10) can collect point cloud information of the target space (S141), which can be expressed as d2 in Fig. 13 as an example. The collection refers to various methods of obtaining point cloud information (3D data), as described above.

[0116] Meanwhile, the spatial map generation unit (285) can generate a spatial map (d3) based on the above point cloud information. Briefly, the generation of the spatial map can be generated through a post-processing step of the acquired raw data (PCD, 3D data). For example, after performing outlier removal (noise removal) through the acquired raw information, each data set can be aligned and merged to perform a rendering operation.

[0117] After a series of processes for generating the above spatial map are performed, or during a series of processes, the local map generation unit (283) can generate a feature map (d4) (S147) through a step (S145) of extracting feature points for objects.

[0118] That is, the 3D map generated through the above-described process may be referred to as the second map (d4) in the present embodiment.

[0119] Meanwhile, the map merging unit (287) can convert a spatial map generated based on PCD into a 2D map (S161) through the map conversion unit (2871), and then merge the converted map with the first map (S163) through the map alignment unit (2872) to generate a third map (d5) (S165).

[0120] More specifically, the generated third map (d5) can be expressed (7511, 7521, 7531, 7541) by overlapping and aligning information about buildings necessary for the robot's mission scenario or driving path among the buildings expressed on the spatial map, as in the map image (70-4) expressed in FIG. 8, on the image (first map, 70-3) for satellite information expressed in FIG. 7.

[0121] That is, the third map can express some of the information not expressed in the satellite information by overlapping and aligning a 2D map converted through a spatial map (3D data) on the first map generated based on the satellite information (2D layout data). Therefore, in Fig. 13, the first map and the third map are expressed as d5 for convenience.

[0122] Meanwhile, the control unit (210) can create a hybrid map (S18) by merging at least two of the first map, the second map, and the third map through the map generation unit (280) and the map merging unit (287).

[0123] Additionally, according to various embodiments of the present invention, a hybrid map may be formed by merging a second map and a third map.

[0124] Additionally, according to various embodiments of the present invention, a hybrid map may be formed by merging a first map, a second map, and a third map.

[0125]

[0126] Referring to FIG. 4, the robot (10) of the present embodiment can create a global map (S11) through the above-described configuration and method, and the area division unit (289) can divide and extract (S13) a path (d51 of FIG. 5) that the robot (10) can use based on the created global map.

[0127] The path that the robot can use may be set as an area that includes a portion of the boundary demarcated as an area where a person can drive from a boundary demarcated as an area where a vehicle can drive, for example.

[0128] More specifically, referring to the map image (70-5) of FIG. 9, only the road line (7311) on which a vehicle can drive, the road line (7321b) that can be used by both people and robots or only robots, the road line (7331) on which people can walk, and the road line (7321b) that can only be used by robots among the boundaries of multiple buildings, playgrounds, parking lots, etc. can be segmented and extracted (S13).

[0129] In addition, in the step (S13) according to the exemplary embodiment of the present invention, the area division unit (289) can divide and extract roads (p6, p1, p2, p3) connecting the starting point (S) and the arrival point (G) where the robot is set to perform the mission, as expressed in the map image (70-6) of FIG. 10, among the road lines (7321b) that only the robot can use.

[0130] In addition, as described above, the area division unit (289) can divide the area into a1, a2, a3, a4, a5, and a6 based on the amount of information that the robot can receive or the size of the area that the robot can drive based on images (70-1 to 70-3) for satellite information, and can divide the area (711a 711b) that the robot cannot drive on, the line of the road (7311) that the vehicle can drive on, the line of the road (7321b) that can be used by both people and robots or only by robots, the line of the road (7331) that people can walk on, the boundary of a plurality of buildings, or the boundary of an area such as a playground or a parking lot.

[0131] And, the control unit (210) can extract roads (7321b) that only the robot can use and roads (p6, p1, p2, p3) that connect the starting point (S) and the destination point (G) where the robot is set to perform the mission by dividing the area through the area dividing unit (289) based on the route generation unit (2131) and the overlapping and aligned map images (70-4 to 70-6) based on the route generation unit (2133).

[0132] After the above step (S13), the robot (10) can receive (S151) 3D information (Point Cloud Data, PCD) stored in the database (30) through the communication unit (270). More specifically, the PCD reception can be based on the robot's usage path extracted in the above step (S13).

[0133] More specifically, referring to the map image (70-6) of FIG. 10, the PCD received in the step (S151) may mean three-dimensional information about a specific area located on the robot's usage path (p6, p1, p2, p3).

[0134] The above specific area may refer to, for example, an area (PC1) including a starting point, an area (PC2, PC3, PC4) including objects that the robot passes through to perform a given mission, and an area (PC5) including the robot's arrival point.

[0135] That is, rather than receiving a large amount of PCDs for the target space, it will be possible to more effectively generate the robot's driving path or map by receiving PCDs on the path that the robot can use.

[0136] Based on the map (70-6) generated in this way, the control unit (210) can select one of the maps of different formats that form a hybrid map based on the location of the robot determined through the location estimation unit (219) and use it for the robot's driving.

[0137] In more detail, the robot can navigate through any of the different map formats based on its current location and information about objects (S, 7511, 7521, 7531, 7541, G) within the target area.

[0138] For example, when a robot is located on a driving path (p2) formed in a second zone (a2), or when a robot enters a second zone (a2), the robot can drive using the first map or the third map.

[0139] Additionally, when the robot is located on a driving path (p3) formed in the third zone (a3), or when the robot enters the third zone (a3), the robot can drive using the second map.

[0140] That is, in sections (p2) where information about objects is not required, a map with a relatively small data volume can be used, and in sections (p3) where information about objects is required, a map containing 3D information can be used for driving safety, preventing deviation from the driving path, etc.

[0141] Meanwhile, the spatial map generation unit (285) generates a PCD-based spatial map (S153), and if the generated spatial map is converted into a 2D map by the map conversion unit (2871) (S155: Yes), the map alignment unit (2872) can merge the converted 2D map and the map (first map, global map) generated in the step (S11) to generate a hybrid map (S171). The hybrid map can be referred to in the sense that maps of different formats are merged (overlapping and aligned), and the hybrid map in this step (S171) can be referred to as a second map.

[0142] That is, the hybrid map generated in the above step (S171) can be referred to as a hybrid map in the sense that two maps containing different information are overlapping, but it is clear that the information contained is different from that of the hybrid map generated in the subsequent step S19.

[0143] After the above step (S171), the map alignment unit (2872) can compare the matching rate (degree of overlap) of the two maps with a reference value, and if the matching rate exceeds the reference value (S173: No), the two overlapping maps can be aligned to create an intermediate map (S175).

[0144] Alternatively, for example, the hybrid map generated in the step (S171) may mean a map in which two maps containing different information (the first map and the converted 2D map) are overlapped and aligned, and in this case, if the matching rate (degree of overlap) in the step (S173) exceeds the reference value by comparing it with the reference value (S173: No), the intermediate map referred to in the step S175 may mean a map already generated in the step (S171).

[0145] The above degree of overlap can be judged based on elements commonly included in the first map (2D layout data map based on satellite information) and the 2D map converted from the spatial map (3D data).

[0146] The above commonly included elements may include, for example, a robot's usage path, a road boundary, multiple nodes between a starting point and a destination set according to the robot's mission, edges connecting the nodes, and a road on which the robot drives to perform the mission.

[0147] Meanwhile, if the spatial map generated in the above step (S155) is not converted into a 2D map (S155: No), the feature map generation unit (2831) can extract feature points for objects from the spatial map and generate a feature map (S157), as described above.

[0148] And the control unit (210) can create a hybrid map (S19) by merging two or more maps created through the above-described process through the map creation unit (280) and the map merging unit (290).

[0149] In the hybrid map generated as described above, as expressed in the map image (70-7) of Fig. 11, the first robot can drive along the road (p6-1, p1-2, p2, p-3, p3-1) connecting the starting point (S1) and the destination point (G1), and the second robot can drive along the road (p5, p6-2, p1-2, p2, p3, p3-2, p4) connecting the starting point (S2) and the destination point (G2).

[0150] When describing roads based on the first area (a1) on the map image (70-7), the first robot that can drive can use roads p1-1 and p1-2 to perform a mission, and among these, road p1-2 can form a section that overlaps with the road on which the second robot drives to perform a mission.

[0151] That is, the first robot and the second robot have overlapping path sections for sections p1-2, p2, and p3 on their respective driving paths.

[0152] These overlapping sections can be determined based on the roads on which the robots divided in the first map drive to perform missions, and when such overlapping sections (p1-2, p2, p3) occur, the decision on which robot to give priority to driving can be made based on the second map.

[0153] More specifically, referring to Fig. 12, Fig. 12 shows a map image containing information about an avoidance zone (750a).

[0154] The avoidance zone (750a) may mean an area divided into a plurality of nodes (Node, N) in which the robot can move based on the facility (750), and in the present embodiment, it means an area including the facility (750) by a road (B) and a plurality of nodes (N).

[0155] The facility (750) can represent at least one of the buildings (7521, 7531) existing on the driving path of the robots in FIG. 11 or the buildings (7511, 7541) existing for performing a mission, and can represent one of the buildings (7521, 7531, 7541) existing in an overlapping section on the driving paths of the first robot and the second robot.

[0156] The robot's driving path (732) may include a center path (732-C) and left and right paths (732-L, 732-R), and the robot primarily moves along the center path (732-C) and may move along the left and right paths (732-L, 732-R) in an event situation.

[0157] Examples of the above event situations include cases where robots encounter each other in overlapping sections or are driving at the same time. In such cases, among multiple robots, priority may be given to a robot with an avoidance zone (750a) on its driving path.

[0158] To explain more specifically with an example, in a situation where the first robot drives based on the direction expressed on the driving path (732) (from bottom to top in FIG. 12) and the second robot drives based on the direction opposite to the direction expressed on the driving path (732) (from top to bottom in FIG. 12), whether or not an avoidance zone (750a) exists on the driving path can be determined based on the left side of the driving path.

[0159] That is, in the example described above, since the first robot has an avoidance zone (750a) on the left side of the driving path (732), if the first robot and the second robot encounter each other while driving in different directions on overlapping driving paths, the second robot may be granted driving priority. In this case, the first robot may pass through the avoidance zone (750a) or enter the avoidance zone (750a) and then return to the driving path (732) after a preset time.

[0160] In addition, for example, when a first robot and a second robot are driving along a driving path (732) in the same direction at the same time and an overlapping section may occur, the robot that entered the driving path (732) first may be given driving priority.

[0161] Examples of where such driving priority is granted are not limited to those described above and various examples may be applied.

[0162]

[0163] FIG. 14 and FIG. 15 are drawings showing a robot according to an exemplary embodiment of the present invention.

[0164] The following description is provided with reference to Figures 14 and 15.

[0165] The robot of this embodiment is a robot (10a) that is equipped with a display unit (111) on the front as shown in FIG. 14, and thus requires information exchange with a user, and may be suitable for driving in an indoor environment. The robot (10b) that is sealed by a housing (12) as shown in FIG. 12 and does not have a separate display unit may be suitable for driving in an outdoor environment.

[0166] The robot (10a) of Fig. 14 may include a body part (11), a driving part (151, 153), and a head part (13).

[0167] The head (13) is equipped with a display (131) on the front to display information for communication with the user, and a sensor (133) is equipped on the upper part of the head (13) to collect various information necessary for the driving of the robot (10a).

[0168] The body part (11) is provided with a display part (111) on the front to display information required by the user or the status of the robot, etc. For example, the display part (131) of the head part can display information in the form of feedback according to the input value of the user or administrator, and the display part (111) of the body part can display the time, weather, information of the target space, the status of the robot, etc., and can display the above-described information in various ways according to the user's operation.

[0169] The driving part of this embodiment may be provided in the form of a wheel at the lower part of the body part (11), and may include a main wheel (151) that rotates directly by a motor and an auxiliary wheel (153) that prevents the robot (10a) from overturning and rotates by friction with the ground according to the movement of the robot (10a).

[0170] The robot (10b) of Fig. 15 has a structure in which the internal structure of the robot is firmly sealed by a housing (12), and unlike the robot (10a) of Fig. 14, structures such as the display portion (111, 113) that are relatively less durable are omitted.

[0171] More specifically, the housing (12) forms a combined structure of an upper housing (12u) and a lower housing (12d), and a sensor (121) is provided on the upper surface of the upper housing (12u), and a sensor (123) is also provided on the front surface of the lower housing (12d). Of course, various types of sensors can be adopted for each sensor (121, 123) depending on the purpose.

[0172] In addition, as described above, the wheels (141) forming the driving part of the robot (10b) of the present embodiment for driving in an outdoor environment can be directly rotated by the power transmission of the motor.

[0173] The robots (10a, 10b) described above are exemplary drawings for expressing the robots described in the following embodiments, and are a type of robot that constitutes the logic and system described below, and therefore should not be interpreted as being limited thereto.

[0174]

[0175] FIG. 16 and FIG. 17 are drawings showing a method for generating a driving path of a robot according to an exemplary embodiment of the present invention.

[0176] The following description is provided with reference to the contents of Figures 16 and 17.

[0177] The method for generating a driving path map of a robot of the present embodiment may include a step of generating a hybrid map (S10), a step of generating a driving path (S50), and a step of generating a driving path map (S70).

[0178] A method for generating a driving path map of a robot according to various embodiments of the present invention may include a step of generating a hybrid map (S10), a step of generating a scenario (S30), a step of generating a driving path (S50), and a step of generating a driving path map (S70).

[0179] The step (S10) of generating the above hybrid map is omitted as described above and the steps are described below.

[0180] The robot (10) can generate a scenario based on a given mission within the generated map. For example, the generated mission may involve monitoring objects within a target space, and the scenario generated based on the given mission may involve monitoring (surveillance) of specific objects.

[0181] The robot (10) can create a scenario through the control unit (210) according to a mission assigned through the server (50), and can create a driving path (S50) through the path creation unit (2131) according to the created scenario.

[0182] Alternatively, the above mission and scenario may be set identically, in which case the robot (10) may generate a driving path (S50) through the path generation unit (2131) according to the scenario (or mission) input through the server (50).

[0183] The step (S50) of generating a driving route can be performed via the path generation unit (2131) as described above, and a path can be generated by selecting at least one of maps of different formats according to preset criteria. Furthermore, as an example, the process of generating a group of path candidates including multiple driving routes for performing the mission or scenario and selecting an optimal route among them is also as described above.

[0184] For example, the robot (10) can drive along a generated driving path and perform a mission, and can detect events that occur during the mission through various components included in the robot (10).

[0185] The above events include various abnormal situations in which the robot's mission or driving is interrupted and the robot must modify the preset driving path, and do not include events in which the robot does not need to modify the preset driving path even if some interruption occurs in the robot's mission or driving.

[0186] The above event detection may be performed through the components included in the robot (10) as described above, or may be performed by transmitting event information detected from an observer's perspective in the server (50) to the robot (10).

[0187] And you can also modify the preset driving route based on event detection.

[0188] Meanwhile, the robot (10) can generate a driving path map (S70) based on the generated driving path. Of course, the driving path map can also be generated based on a modified driving path according to the above-described event detection.

[0189] Meanwhile, after a driving route is generated (S50) through the route generation unit (2131), the driving route map generation unit (2133) can classify the driving route. The driving route classification is a step of classifying the generated driving route by determining whether the generated driving route is based on a map in a certain format. In this embodiment, since the generated driving route is a route generated based on a hybrid map in which maps in various formats are overlapped and aligned as described above, this can mean classifying the route by dividing the driving route according to a preset standard and then classifying whether the route is generated based on a map in a certain format for each divided part.

[0190] Alternatively, the driving path classified in the above driving path classification step may refer to the robot dividing the driving path generated for mission execution according to preset criteria. In this case, the criteria for division may include, for example, whether the path includes a mission target object or whether the path is a result of a mission execution.

[0191] The generated driving path (some of the segmented paths) is judged to be a driving path on the global map, and if it is not a driving path on the global map, it is judged to be a driving path on the local map, and if it is not a driving path on the local map, it can be seen that there is a problem with the information forming the generated driving path or the segmented driving path for classification, so the process can be repeated again from the driving path classification step.

[0192]

[0193] The present invention has been described so far, focusing on preferred embodiments thereof. All embodiments and conditional examples disclosed herein are intended to assist those skilled in the art in understanding the principles and concepts of the present invention. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics.

[0194] Therefore, the disclosed embodiments should be considered illustrative rather than restrictive. The scope of the present invention is determined by the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present invention.

[0195] Meanwhile, the methods according to the various embodiments of the present invention described above can be implemented as programs and provided to servers or devices. Accordingly, each device can access the server or device where the program is stored and download the program.

[0196] In addition, the methods according to the various embodiments of the present invention described above may be implemented as programs and stored and provided on various non-transitory computer-readable media. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on non-transitory computer-readable media, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0197] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the invention pertains without departing from the gist of the present invention claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. Step of receiving satellite information of the target area; A step of generating a first map of the target area based on the satellite information; A step of dividing an area according to criteria set in the first map; A step of receiving 3D information (Point Cloud Data) for an area including a road on which the robot can drive among the above divided areas; A step of generating a second map based on the above three-dimensional information; A step of creating a hybrid map by overlapping and aligning the first map and the second map; and A step of determining the location of the robot within the target area; A control method for a hybrid map-based robot, characterized in that the robot drives using one of different map formats forming the hybrid map according to its location within the target area.

2. In paragraph 1, A control method for a hybrid map-based robot, characterized in that the first map is a two-dimensional map of the target area based on the satellite information.

3. In paragraph 2, A control method for a hybrid map-based robot, characterized in that the above-mentioned preset criterion is whether or not the road is one on which the robots can drive.

4. In paragraph 3, A control method for a hybrid map-based robot, characterized in that the hybrid map is generated by overlapping and aligning the first map and the second map based on the boundary line of an area that meets the preset criteria.

5. In paragraph 4, A control method for a hybrid map-based robot, characterized in that the hybrid map is generated by converting the coordinates of the first map into a local coordinate system and then overlapping and aligning them with the second map.

6. In paragraph 2, further comprising a step of setting a driving path of the robot; A control method for a hybrid map-based robot, characterized in that the robot drives using any one of maps of different formats forming the hybrid map according to a location on the driving path.

7. In paragraph 2, A step of creating a spatial map based on the above three-dimensional information; A step of converting the above spatial map into a 2D map; and Further comprising a step of generating a third map by overlapping and aligning the first map with the converted 2D map; A control method for a hybrid map-based robot, characterized in that the hybrid map is generated by overlapping and aligning at least two maps among the first map, the second map, the third map, and the spatial map.

8. In paragraph 7, A control method for a hybrid map-based robot, characterized in that the second map is a feature map generated based on feature points extracted from the spatial map.

9. In paragraph 8, further comprising a step of setting a driving path of the robot; A control method for a hybrid map-based robot, characterized in that the above spatial map is generated based on three-dimensional information collected along the driving path of the robot.

10. In paragraph 9, The above second map contains information generated based on feature points for objects existing on the driving route, A method for controlling a hybrid map-based robot, characterized in that the third map contains two-dimensional location information for the objects.

11. In paragraph 10, A control method for a hybrid map-based robot, characterized in that the robot drives using one of maps of different formats that form the hybrid map based on the current location and information about the objects within the target area.

12. In paragraph 11, A control method for a hybrid map-based robot, characterized in that the robot drives using either the second map or the third map according to the current location of the robot and the distance from the objects.

13. In paragraph 8, A control method for a hybrid map-based robot, characterized in that it further includes a matching degree determination step for determining whether the matching degree of a reference area is greater than a preset reference value when overlapping and aligning the first map with the converted 2D map.

14. In paragraph 13, A control method for a hybrid map-based robot, characterized in that the step of converting the above-mentioned spatial map into a 2D map, the step of generating the third map, and the step of determining the degree of alignment are performed sequentially.

15. In paragraph 13, The step of converting the above spatial map into a 2D map, the step of determining the degree of alignment, and the step of generating the third map are performed sequentially. A method for controlling a hybrid map-based robot, characterized in that the step of generating the third map is performed when the matching degree of the reference area is greater than a preset reference value in the matching degree determination step.

16. In paragraph 8, A method for controlling a hybrid map-based robot, characterized in that the hybrid map is generated by overlapping and aligning the second map and the third map.

17. Step of receiving satellite information of the target area; A step of generating a first map of the target area based on the satellite information; A step of dividing an area according to criteria set in the first map; Step of setting the robot's driving path; A step of receiving 3D information (Point Cloud Data) for an area including the driving path of the robot among the above divided areas; A step of generating a second map based on the above three-dimensional information; A step of creating a hybrid map by overlapping and aligning the first map and the second map; and A step of determining the location of the robot within the target area; A control method for a hybrid map-based robot, characterized in that the robot drives using one of different map formats forming the hybrid map according to its location within the target area.

18. In a control device of a hybrid map-based robot driving in a target area, A communication unit that transmits and receives information necessary for map creation; A map generation unit that generates a map of the target area by overlapping and aligning two or more maps in different formats; A position estimation unit that determines the position of the robot; and a processor for selecting one of two or more maps having different formats according to the location of the robot; The above map generation unit, A first map generated based on satellite information of the target area and a second map generated based on 3D information of the target area are overlapped and aligned, The above processor, A control device for a hybrid map-based robot, characterized in that one of the first map and the second map is selected as a map for driving of the robot based on the current location of the robot and location information of objects included in the three-dimensional information.

19. A program stored on a computer-readable recording medium including a program code for executing a control method of a hybrid map-based robot described in any one of claims 1 to 17.

20. A computer-readable recording medium containing a program code for executing a method for controlling a hybrid map-based robot as described in any one of claims 1 to 17.

Images