Robot and map generation method thereof

Abstract

A robot is disclosed. A processor receives as input and stores in memory a guide map for guiding map generation, converts the guide map into a map form recognizable by the robot, identifies a search boundary determined by the converted guide map, controls a travel unit to travel within the search boundary, generates a sensing map for a space in which the robot is located on the basis of sensing values sensed by a plurality of sensors during travel, and generates a space map for the space by matching the guide map and the sensing map.

Description

How to create a robot and its map

[0001] The present disclosure relates to a robot and a method for generating a map of the robot.

[0002] Thanks to recent advancements in robotics technology, the use of robots in large spaces like factories and shopping malls is on the rise. To utilize robots in these spaces, an indoor map of the space is required.

[0003] However, users had the inconvenience of having to physically walk around large spaces with the robot to create indoor maps. Furthermore, users had to visit the area where they wanted to create the map, which was a hassle.

[0004] A robot according to at least one embodiment of the present disclosure includes a plurality of sensors, a driving unit, a memory, and a processor.

[0005] The processor receives a guide map for guiding map creation and stores it in the memory, converts the guide map into a map form recognizable by the robot, identifies a search boundary determined by the converted guide map, controls the driving unit to drive within the search boundary, generates a sensing map for a space in which the robot is located based on sensing values ​​sensed by the plurality of sensors during driving, and matches the guide map and the sensing map to generate a space map for the space.

[0006] A method for generating a map of a robot according to at least one embodiment of the present disclosure includes the steps of receiving and storing a guide map for guiding map generation, converting the guide map into a map form recognizable by the robot, identifying a search boundary determined by the converted guide map, generating a sensing map for a space in which the robot is located based on sensing values ​​of a sensor provided in the robot while the robot is driving within the search boundary, and generating a space map for the space by matching the guide map and the sensing map.

[0007] According to at least one embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions that, when executed by a processor of a robot, cause the robot to perform an operation, the operation includes the steps of: receiving and storing a guide map for guiding map creation; converting the guide map into a map form recognizable by the robot; identifying a search boundary determined by the converted guide map; generating a sensing map for a space in which the robot is located based on sensing values ​​sensed by the plurality of sensors while driving within the search boundary; and generating a space map for the space by matching the guide map and the sensing map.

[0008] FIG. 1 is a perspective view illustrating a robot according to at least one embodiment of the present disclosure.

[0009] FIG. 2 is a block diagram illustrating the configuration of a robot according to at least one embodiment of the present disclosure.

[0010] FIG. 3 and FIG. 4 are drawings for explaining a method for a robot to receive a guide map according to at least one embodiment of the present disclosure.

[0011] FIG. 5 is a drawing for explaining a method of receiving a guide map by utilizing a partial map of a robot according to at least one embodiment of the present disclosure.

[0012] FIG. 6 is a diagram illustrating a process for identifying a search boundary of a robot according to at least one embodiment of the present disclosure.

[0013] FIG. 7 is a diagram illustrating a spatial map matching process of a robot according to at least one embodiment of the present disclosure.

[0014] FIG. 8 is a diagram for explaining a matching process of graph nodes according to at least one embodiment of the present disclosure.

[0015] FIG. 9 is an overall diagram of a spatial map generation process of a robot according to at least one embodiment of the present disclosure.

[0016] FIG. 10 is a flowchart illustrating a method for generating a map of a robot according to at least one embodiment of the present disclosure.

[0017] The terms used in the various embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the terms and the overall content of this disclosure, rather than simply their names.

[0018] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0019] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0020] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0021] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0022] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this disclosure, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0023] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0024] In this disclosure, the term user may refer to a person using an electronic device or a device used by the person.

[0025] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0026] FIG. 1 is a perspective view illustrating a robot according to at least one embodiment of the present disclosure. A robot may be a device capable of driving without direct human control. The robot (100) may be referred to by various terms such as an autonomous driving device, an autonomous mobile robot (AMR), an automated guided vehicle (AGV), an unmanned ground vehicle (UGV), etc., but is described as a robot (100) in the present disclosure. The robot (100) may be implemented as various types of robots that drive through a space and perform necessary tasks, such as a cleaning robot, a serving robot, a mobile projector, an industrial robot, a guide robot, a delivery robot, etc., depending on its method of use or purpose.

[0027] Referring to FIG. 1, a user (10) can transmit a guide map (20) to a robot (100). The robot (100) can create a map of a space (30) in which the robot (100) is located based on the guide map (20) transmitted from the user (10).

[0028] The guide map (20) can be a map that can be used as initial data for the robot to create a map. For example, the guide map (20) can be a map that a user (10) roughly draws of the space (30) where the robot (100) is located. In other words, a map drawn only to the extent of a layout by a user who knows the rough structure, location, and direction of the space where the robot is located can be used as the guide map (20). However, it is not necessarily limited to this, and a blueprint and drawing of the space where the robot (100) is located can also be used as the guide map (20). Various methods for inputting the guide map can be implemented. This will be described in detail in the following section.

[0029] In the present disclosure, the guide map (20) is described as a “guide map” because it is used to guide the initial driving of the robot (100), but may be described in various ways, such as a hand-drawn map, a user map, an initial map, a drawing map, a sketch map, etc.

[0030] Space (30) refers to the location where the robot (100) is located. Space (30) may include not only indoor spaces but also outdoor spaces. Specifically, it may include various commercial facilities, office spaces, accommodation facilities, medical facilities, educational facilities, large-scale shopping malls, airports, transportation terminals, laboratories, factories, parks, etc.

[0031] The robot (100) can generate a final map of the space (30) in which the robot (100) is located based on the guide map (20) provided by the user (10). In the present disclosure, the finally generated map is referred to as a space map.

[0032] In FIG. 1, when a user (10) provides a guide map for a space (30) to a robot (100), the robot (100) can drive in the space (30) based on the guide map (20) and identify terrain and objects based on sensing values ​​of a plurality of sensors to create a sensing map of the space.

[0033] Without a guide map, a robot (100) typically generates a sensing map by exploring all spaces it can navigate while moving through a space, i.e., all areas where the robot's driving section can navigate and where there are no walls or objects through which it can move. Accordingly, the resulting map may include areas where the user does not work with the robot or areas where map creation is not required. In particular, since the robot (100) acquires all map data of a space to generate a map, it may take a long time or, worse, the spatial map creation may not be completed.

[0034] Accordingly, the user (10) can select a space in which the robot wants to drive among the spaces, draw a guide map (20) for the space, and provide it to the robot (100). If there is a blueprint, the user (10) can mark only the desired space within the blueprint and use it as a guide map (20). The robot (100) can drive in a space specified based on the guide map (20) provided by the user (10), rather than the entire space, and can create a sensing map using multiple sensors.

[0035] The robot (100) creates a single spatial map by matching the sensing map generated while moving through the space with the guide map (20) provided by the user (10). In the present disclosure, the guide map, the sensing map, and the spatial map are each described separately; however, the guide map may alternatively be described as the first map, the sensing map as the second map, and the spatial map as the third map or the final map.

[0036] FIG. 2 is a block diagram illustrating the configuration of a robot according to at least one embodiment of the present disclosure.

[0037] According to FIG. 2, the robot (100) includes a plurality of sensors (110), a driving unit (120), a memory (130), a display (140), and a processor (150). However, the present invention is not limited thereto, and the robot (100) may be implemented in a form in which some components are excluded, or may be implemented in a form in which other components are further included. For example, the display (140) may be omitted depending on the embodiment. According to another embodiment, components such as various input / output interfaces or communication units may be further added. For the convenience of explanation, independent drawings for each embodiment are omitted.

[0038] The plurality of sensors (110) are sensors for detecting the surrounding environment. Specifically, they may include at least one of a LiDAR sensor, a vision sensor, an image sensor, an infrared sensor, an ultrasonic sensor, a gyro sensor, an acceleration sensor, and a proximity sensor. In addition, the plurality of sensors (110) may include at least one of a 2D camera, a TOF (Time of Flight) camera, a depth camera, a multi-lens array camera, a stereo vision system, a fused lidar camera, and a 3D camera.

[0039] A robot (100) can recognize surrounding walls, objects, pillars, terrain, features, etc. based on sensing values ​​sensed by each of a plurality of sensors (110) while moving through a space, and can obtain depth information with respect to the object to determine the distance and spatial location of the object. The robot (100) can generate a sensing map of the space based on the sensing values ​​of the plurality of sensors (110).

[0040] The driving unit (120) is a component for moving the main body of the robot (100). The driving unit (120) may include components such as a plurality of wheels, a driving motor for rotating each of the plurality of wheels, a gear, and a shaft. The plurality of wheels are provided on the lower or side of the main body of the robot (100) and support the main body of the robot (100) from the floor. When the driving motor operates and the driving force is transmitted to the plurality of wheels so that each wheel rotates, the robot (100) can move by the frictional force between the floor and the wheels. In addition, the driving unit (120) may vary the rotational speed of at least one wheel among the plurality of wheels or adjust the alignment direction of the wheels differently when changing direction. Depending on the type of the robot (100), the weight of the loaded item, and the usage environment of the robot (100), an infinite track or the like may be used instead of the wheels.

[0041] The memory (130) can store at least one command, data, program, etc. required for the operation of the robot (100). For example, the memory (130) can store a guide map provided by a user. The memory (130) may be implemented in the form of a memory embedded in the robot (100) or in the form of a memory detachable from the robot (100) depending on the purpose of data storage. For example, data for driving the robot (100) may be stored in a memory embedded in the robot (100), and data for the expansion function of the robot (100) may be stored in a memory detachable from the robot (100).

[0042] In the case of memory embedded in the robot (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0043] The memory (130) may be implemented as a single memory that stores data generated in various operations according to the present disclosure, but is not limited thereto, and the memory (130) may be implemented to include multiple memories that each store different types of data or each store data generated in different stages.

[0044] The display (140) is configured to display various screens. For example, the user (10) can provide a guide map of a space through the display (140). The display (140) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which multiple display modules are physically connected, etc. When at least a portion of the display (140) is implemented as a touch screen, the user (10) can provide a guide map by directly drawing a space on the touch screen.

[0045] The processor (150) is a component for controlling the operation of the robot (100). The processor (150) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, but is not limited thereto, and may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, and an artificial intelligence (AI) processor, or may be defined by the relevant terminology. In addition, the processor (150) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a processing algorithm built in, or may be implemented in the form of a field programmable gate array (FPGA). The processor (150) may perform various functions by executing computer executable instructions stored in the memory (130).

[0046] The processor (150) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (150) are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0047] The processor (150) can receive a guide map to guide map creation and store it in the memory (130). The processor (150) can convert the guide map into a map form that the robot can recognize. The processor (150) can identify a search boundary determined by the converted guide map and control the driving unit (120) to drive within the search boundary. The processor (150) can generate a sensing map for a space in which the robot is located based on sensing values ​​sensed by a plurality of sensors (110) while the robot (100) is driving. The processor (150) can match the guide map and the sensing map to generate a spatial map for the space.

[0048] The processor (150) may receive a guide map (20) from a user (10) in various ways. Below, a method for the robot (100) to receive a guide map will be described in detail.

[0049] FIG. 3 and FIG. 4 are drawings for explaining a method for a robot to receive a guide map according to at least one embodiment of the present disclosure.

[0050] Referring to FIG. 3, a user (10) can use a drawing map roughly drawn directly on the display (140) of the space where the robot (100) is located as a guide map. The user (10) can draw a drawing map (310) of the space based on his / her own experience, such as visiting the space or performing a task.

[0051] In the present disclosure, drawing includes all actions such as a user drawing a line directly on a display screen using a hand, pen, or other object, or drawing a line while moving a cursor within the display screen using an input means connected to the display (e.g., a mouse, keyboard, joystick, etc.).

[0052] A user (10) may be provided with a blueprint or drawing of a space and draw a drawing map (310) based thereon. At this time, the blueprint or drawing of the space is about the initial appearance of the space, and various facilities (devices, furniture, and other items) may have been added later. The user (10) may draw a guide map based on the current appearance while referring to the blueprint or drawing. For example, if the appearance of the blueprint or drawing of the space secured by the user (10) is the appearance before remodeling or repair work of the current space, the user may create a drawing map (310) by drawing the added or changed space.

[0053] Meanwhile, the user (10) may display (330) information about the initial location of the robot (100) on the drawing map (310) drawn by the user. In Fig. 3, the initial location of the robot (100) is displayed using an arrow, but this is not limited thereto, and any display surface capable of identifying the location of the robot may be used. The user (10) may provide the robot (100) with a drawing map (320) that displays the initial location of the robot (100) on the drawing map (310) drawn in space.

[0054] When the processor (150) receives a drawing map (310, 320) drawn by a user (10), it can recognize it as a guide map (20).

[0055] As another example, if the robot (100) includes a scanner or camera, the user (10) may provide the robot (100) with a map drawn directly on paper, a blueprint used when building the space, or a map drawn by the user (10) by adding the location of objects or changed internal space to the blueprint, using the scanner or camera.

[0056] As another example, if the robot (100) is equipped with various input / output interfaces such as a USB port, the user (10) can connect a USB memory in which data for a guide map drawn by the user is stored to the input / output interface, thereby providing the guide map to the robot (100).

[0057] As another example, if a communication unit is provided in the robot (100), the user (10) can connect a terminal device such as a mobile phone or tablet PC through the communication unit and draw a guide map through the terminal device screen and transmit it to the robot (100).

[0058] The processor (150) can recognize a map provided in various ways as described above as a guide map (20).

[0059] FIG. 4 is a drawing for explaining a method of receiving a guide map using a camera. Referring to FIG. 4, the processor (150) can receive a drawing map (310, 320) provided by a user through a camera (410) among a plurality of sensors (110). The user (10) can input a shooting command while placing the drawing map (310, 320) drawn by the user in front of the camera (410). The processor (150) can operate the camera (410) according to the shooting command to capture the drawing map (310, 320).

[0060] If a scanner is included among the plurality of sensors (110), the processor (150) may scan the drawing map through the scanner and use it as a guide map.

[0061] The processor (150) can store the captured image or scanned image in the memory (130). The processor (150) can also display (420) the captured image or scanned image through the display (140). The processor (150) can analyze the captured image or scanned image to recognize the guide map (20).

[0062] Meanwhile, the user (10) can bring the blueprint or drawing of the space itself to the camera (410). The processor (150) can capture the blueprint or drawing provided by the user (10) through the camera (410). The processor (150) can identify the captured blueprint or drawing and recognize it as a guide map (20).

[0063] The processor (150) can automatically capture a drawing map (310, 320) or blueprint placed in front of the camera (410) or can be captured by a user's operation. Specifically, the user can select the guide map input function through a button on the body of the robot (100) or a UI screen displayed on the display (140). In this case, the processor (150) can turn on the camera (410) to put it in a shooting standby state and automatically perform shooting when a map is placed in front of the camera (410).

[0064] The processor (150) can analyze multiple image frames within a photographed image or scanned image to identify a drawing map or blueprint provided by a user. Specifically, the processor (150) divides all pixels included in each of multiple consecutive image frames into multiple block units consisting of n*m pixels. The processor (150) can detect a representative value representing the characteristics of the pixels within each block. The representative value may be, but is not limited to, an average pixel value of the pixels within each block, and may also be a maximum pixel value, a minimum pixel value, or an RMS (Root Means Square) value.

[0065] The processor (150) can connect blocks that have representative values ​​within a similar range and are arranged in consecutive positions among a plurality of blocks to form a closed loop, and identify the closed loop as an edge of an object included in a photographed image. The processor (150) can identify the size of the object based on the number of blocks included in the edge. Additionally, the processor (150) can identify the shape of the object based on the shape of the edge.

[0066] Meanwhile, the processor (150) may also receive a guide map through a partial map provided by the robot (100), rather than a drawing map or blueprint drawn from the beginning by the user (10). This will be described in detail below.

[0067] FIG. 5 is a drawing for explaining a method of receiving a guide map by utilizing a partial map of a robot according to at least one embodiment of the present disclosure.

[0068] The processor (150) can control the display (140) to display a partial map (510) generated at the current location of the robot (100) among the spaces where the robot (100) is located. When a user (10) additionally draws on the partial map (510) on the display to create a map, the processor (150) can recognize the created map as a guide map (520) and store it in the memory (130). According to FIG. 5, it shows that one guide map (520) is generated by including a partial map (511) additionally drawn by the user in addition to the partial map (510) generated by sensing by the robot (100) before the guide map is provided.

[0069] When a user (10) directly draws a guide map, it may be difficult to determine which space in the entire space should be used as a reference for the drawing. Therefore, the processor (150) can sense the space around the robot through a plurality of sensors (110) and generate a rough partial map (510) of the surrounding space based on the sensed data. The partial map refers to a map generated for a part of the space where the robot is located among the entire indoor space. The partial map may be a part of the entire map. The partial map may be displayed as a bold line, blinking, highlighted, or displayed in a different color to make it more distinguishable from other parts in the entire map, as shown in FIG. 5.

[0070] The processor (150) can display a partial map (510) through the display (140). The user (10) can directly draw the remaining portion (511) following the partial map displayed on the display (140) to create a drawing map (520). Alternatively, the user (10) can receive the partial map to his / her terminal device through a communication unit, USB, cable, etc., and further draw and complete the partial map on his / her terminal device. The processor (150) can receive the drawing map (520) created in this way and recognize it as a guide map (20).

[0071] The processor (150) can receive a guide map (20) through the user or the communication unit in the manner described above.

[0072] Thereafter, the processor (150) may perform a preprocessing operation to convert the input guide map (20) into a map format recognizable by the robot (100). The preprocessing operation may be a operation to convert the guide map into a map format recognizable by the robot. Specifically, the preprocessing operation may include a series of operations such as straightening each line within the guide map and connecting them to each other.

[0073] The drawing map provided by the user (10) may be curved lines rather than straight lines, and straight lines may be broken. In addition, lines intended to represent a flat wall may be displayed as curved lines rather than straight lines, corners where lines meet may not be right angles, and a single line may be expressed as two lines or may be an overlaid line. In such cases, the processor (150) may have difficulty determining the outermost boundary in the guide map and may have difficulty generating a sensing map based on the guide map.

[0074] The processor (150) can convert the input guide map into a map form in which each side is composed of a straight line (rectilinear) through a preprocessing operation. Specifically, the processor (150) analyzes the guide map to obtain the position coordinate values ​​(e.g., x, y coordinates) of the pixels constituting each line included in the guide map. The y-coordinate values ​​of the plurality of pixels constituting one horizontal straight line are all the same, and the x-coordinate values ​​of the plurality of pixels constituting one vertical straight line are all the same. The plurality of pixels constituting a line that extends horizontally and bends have their y-coordinate values ​​change while their x-coordinate values ​​constantly increase or decrease, and the plurality of pixels constituting a line that extends vertically and bends have their y-coordinate values ​​constantly increase or decrease while their x-coordinate values ​​change.

[0075] Accordingly, the processor (150) compares the x and y coordinate values ​​of a plurality of pixels constituting a single line and corrects the x or y coordinate values ​​of a minority of pixels based on the coordinate values ​​of the majority of pixels. Accordingly, a curved or crooked line can be corrected to a straight line. Based on the directionality of each line, the processor (150) extends and connects at least one of two lines that are separated by a certain distance.

[0076] Meanwhile, there may be a curved wall or object in the spatial structure. In this case, the user may draw a curve on the guide map. However, if the user draws directly, the curvature of the curve may not be expressed consistently. The processor (150) calculates the curvature by comparing the x and y coordinates of consecutive pixels, and if the curvature is greater than a certain size, it can be determined that the user has drawn a curve. Accordingly, the relative curvature of the coordinate values ​​of all pixels included in the line is checked to identify multiple curvatures and correct the coordinate values ​​of some pixels based on the identified curvatures. Accordingly, the line can be expressed as having a consistent curvature.

[0077] The processor (150) can convert the guide map through this preprocessing work and store the converted guide map in the memory (130).

[0078] The processor (150) can identify a navigation boundary within which the robot (100) can navigate based on the converted guide map. The processor (150) can generate a sensing map based on the identified navigation boundary. A detailed description thereof will be provided below.

[0079] FIG. 6 is a diagram illustrating a process for identifying a search boundary of a robot according to at least one embodiment of the present disclosure.

[0080] When a guide map (20) is input, the robot (100) can convert it through the above-described preprocessing work and explore the surrounding space based on the converted guide map (610). The robot (100) can explore the surrounding space and create a sensing map using a plurality of sensors (110). At this time, the robot (100) can create the sensing map (620) and estimate the location of the robot using SLAM (Simultaneous Localization and Mapping). SLAM refers to simultaneous location estimation and map creation technology, and is a technology for a mobile body such as a robot to estimate its own location and simultaneously create a map of the surrounding environment. SLAM can autonomously explore a space and automatically create a map even in an environment where GPS signals are limited or networks are unavailable.

[0081] The robot (100) can collect information about the surrounding environment using a LiDAR sensor, a depth camera, etc. among a plurality of sensors (110). The robot (100) can extract features of the surrounding environment, such as pillars, corners, and objects, from data sensed by the plurality of sensors. The robot (100) can estimate the current position and posture by using the sensed data and the extracted features. Specifically, the processor (150) can recognize the direction in which the front of the robot (100) is facing based on sensed values ​​such as a geomagnetic sensor. In addition, the processor (150) can calculate the current position in space based on the distance to the walls sensed in the east, west, south, and north directions using the plurality of sensors (110). In this case, when various surrounding objects other than the wall are sensed, the size of the object can be estimated by comparing the distance to the object and the distance to the wall behind it.

[0082] The robot (100) can create a map of space using data sensed from multiple sensors and characteristics of the surrounding environment.

[0083] A robot (100) can generate a sensing map (620) of a specific space while navigating through the space. Here, the sensing map refers to a map generated by the robot while navigating through the space using multiple sensors. The sensing map is not limited to this, and may be called by various names such as a driving map, a SLAM map, a robot map, etc., but in the present disclosure, it will be collectively referred to as a sensing map.

[0084] In the absence of a guide map, the robot (100) basically explores all spaces to generate a sensing map. The sensing map (620) of Fig. 6 represents a sensing map sensed without a guide map (610) within a specific space (640) among spaces.

[0085] As illustrated in FIG. 6, when comparing a specific space (640) of the converted guide map (610) with the sensing map (620) generated by the robot (100), it can be seen that some spaces (650) are added to the sensing map (620). That is, since the robot (100) explores all spaces where it can move through the driving unit (120) in the absence of a guide map, a part (650) not included in the guide map (610) may be included.

[0086] This part (650) may be a space where the robot does not need to work, or a space where the user does not want to explore.

[0087] The robot (100) in the present disclosure can obtain a sensing map including a specific space (640) excluding a portion (650) by generating a sensing map based on a guide map (610). Depending on whether the guide map is used, some portions (650) may differ.

[0088] In this way, the robot (100) can determine a search boundary (630) that can be searched based on the converted guide map (610). The search boundary refers to a boundary that the robot can search while driving in a space. The search boundary can be determined through the guide map by the user's arbitrary selection, or can be determined by setting an area that is dangerous or does not require work when the robot drives. For example, if a part of the space has a cliff or a high ledge, and a major accident may occur when the robot drives in that area, the user (10) can create a guide map that excludes that area when providing the guide map.

[0089] The processor (150) can determine a search boundary (630) that the robot (100) can search by matching the converted guide map (610) with the sensing map (620) generated in real time while the robot is moving. As described in FIG. 5, the user (10) can receive a partial map indicating the current location of the robot from the robot (100). The user (10) can provide the robot with a guide map in which a space is additionally drawn on the provided partial map. Accordingly, the robot (100) can determine the current location of the robot from the converted guide map (610) based on the partial map, and can also determine the current location of the robot from the sensing map (620), thereby enabling matching between the two maps by comparing the current location for a specific space (640) among the spaces.

[0090] However, since the location indicated by the user may not be the exact current location, the processor (150) may first estimate the location of the robot (100) based on the converted guide map, and then correct the location based on the sensing values ​​acquired during driving.

[0091] The processor (150) can match the converted guide map (610) and the sensing map (620) in real time to ultimately determine the search boundary (630). Thus, the robot (100) can drive and generate a map only within the determined search boundary. Even if there are more searchable areas, the robot (100) can only perform searches within the search boundary (630), and can terminate further searches once the sensing map is completed within the search boundary.

[0092] The processor (150) can generate a sensing map for the space based on the determined search boundary. To create a single spatial map by matching the generated sensing map with the guide map, the processor (150) may require a matching process between the two maps. A detailed description of this will be provided below.

[0093] FIG. 7 is a diagram illustrating a spatial map matching process of a robot according to at least one embodiment of the present disclosure.

[0094] The processor (150) can decompose each of the guide map (610) and sensing map converted through the preprocessing operation into node units and reconstruct each into a graph form. The processor (150) can match nodes within each reconstructed graph based on similarity to generate a spatial map.

[0095] The processor (150) can generate a sensing map (710) for a space based on the search boundary (630). The processor (150) can decompose each of the converted guide map (610) and the sensing map (710) into nodes. The processor (150) can generate a single spatial map through matching between each of the decomposed nodes.

[0096] The processor (150) can generate a guide map graph (720) by decomposing the converted guide map (610) into node units and reconstructing each node into a connected graph form.

[0097] In addition, the processor (150) can generate a sensing map graph (730) by decomposing the sensing map (710) into node units and reconstructing each node into a connected graph form.

[0098] In SLAM-based position estimation, a node can be a structure that identifies a specific location, landmark, or point within a graph. Figure 7 represents each subspace that constitutes the entire space as a node, and illustrates a graph in which each node is connected by a line.

[0099] The processor (150) can decompose nodes based on the shape of each area within the map. Typically, a single independent space (e.g., a factory) can be composed of multiple subspaces (workshops, material rooms, offices, break rooms, etc.). Each subspace can be connected via a passageway or directly connected via a door, and each subspace can be separated from other subspaces by walls, doors, etc. The processor (150) can recognize each separated subspace as a node and decompose it.

[0100] Referring to FIG. 7, the guide map graph (720) and the sensing map graph (730) can be decomposed into node units, and each node can be numbered 1 to 6. In the case of the guide map graph (720), it can be seen that each node is numbered 1 to 6 in order from left to right. In contrast, in the case of the sensing map graph (730), it can be seen that each node is numbered 1 to 6 from left to right, centered around the largest node in the center. The number assigned to each node is expressed arbitrarily, and even if they are at the same location in the guide map graph (720) and the sensing map graph (730), they can be numbered differently. However, this is not limited thereto, and the same node number may be assigned for each location.

[0101] Meanwhile, the guide map graph (720) and the sensing map graph (730) may differ in size. Since the guide map is a rough drawing of an actual space, there may be differences in scale and width / height ratio from the sensing map generated by the robot. Therefore, when comparing node 1 of the guide map graph (720) with node 2 of the sensing map graph (730), there may be differences in scale even in the same space. In addition, when comparing node 4 of the guide map graph (720) with node 5 of the sensing map graph (730), there may be differences in width / height ratio even in the same space.

[0102] The processor (150) can generate a sensing map based on the guide map and generate a node-by-node graph in real time. The robot (100) can terminate the search once mapping of all nodes of the sensing map within the determined search boundary is completed.

[0103] The processor (150) can extract nodes from each graph, which is composed of nodes. The processor (150) can match two graphs by comparing the similarity of the extracted nodes. Below, a method for comparing the similarity of the nodes extracted from each graph will be described in detail.

[0104] FIG. 8 is a diagram for explaining a matching process of graph nodes according to at least one embodiment of the present disclosure.

[0105] The processor (150) can extract a straight line from each node of the extracted sensing map. The processor (150) can extract each convex hull based on the vertex of the straight line of each node of the extracted sensing map and the vertex of each node extracted from the guide map. The processor (150) can calculate a turning function graph of each extracted convex hull and identify candidate nodes to be matched with each other within the sensing map and the guide map based on the similarity of the calculated turning function graph. The processor (150) can match nodes with high similarity with each other based on scale information, rotation information, and translation information of the candidate nodes to generate a spatial map.

[0106] Referring to FIG. 8, the processor (150) can extract (820) a Hough line corresponding to a straight line from node 2 (810) of the sensing map graph (730). The processor (150) can extract (830) a convex hull based on the vertices of the extracted lines. A convex hull refers to a minimum convex polygon surrounded by a set of points.

[0107] The reason for extracting a convex hull from a specific node of a sensing map is that, in the case of a robot (100), when generating a sensing map while driving through a specific rectangular space, if an object is located within the specific space, a "ㄷ" shaped sensing map is generated as shown in 810. However, in the case of a guide map such as a drawing map or blueprint drawn by a user, a guide map is generated as a rectangular space excluding various objects within the space. In that case, it may be difficult to compare a "ㄷ" shaped sensing map with a rectangular guide map. Therefore, the processor (150) can extract a convex hull from a specific node of a sensing map graph that takes into account objects in the space, and compare the convex hull of the extracted specific node with the nodes of the guide map graph to match between the nodes.

[0108] The processor (150) can also extract a convex hull from each node of the guide map graph. The processor (150) can produce a turning function graph (840) of each convex hull extracted from each graph.

[0109] A turning function is a graph that considers the length, angle, and direction of each straight line at a specific node. The x-axis of the turning function represents length, and the y-axis represents angle.

[0110] When expressing a specific node as a turning function, the processor (150) can perform a process of normalizing the entire length to a value within a specific range. In addition, when any point of a specific node is rotated 90 degrees counterclockwise, the y-axis value of the turning function graph increases, and when rotated 90 degrees clockwise, the y-axis value of the turning function graph decreases. Therefore, the processor (150) can obtain scale and rotation information of each node through the turning function graph. The processor (150) can identify a candidate node to be matched based on the scale and rotation information obtained through the turning function graph.

[0111] The processor (150) can select node 1 (851) among the nodes of the guide map graph and node 2 (852) among the nodes of the sensing map graph as candidate nodes (850) based on the turning function graph. The processor (150) can determine translation information having a minimum error value between the selected candidate nodes. Translation information refers to the positional movement distance between two coordinates and can be the distance between two coordinates.

[0112] The processor (150) can extract pixel coordinates in units of pixels from the selected candidate nodes. The processor (150) can obtain the distance for each pixel from the pixel coordinates of the sensing map graph and the pixel coordinates of the guide map graph, add up the distances for all pixels, and determine the minimum value as translation information. The minimum distance for the pixel coordinates in each node means that the distance difference for two candidate nodes is minimum, and accordingly, it can be determined that the error value is minimum. Therefore, the processor (150) can determine the similarity between candidate nodes based on the translation information, and match nodes with high similarity to each other to generate a spatial map.

[0113] Accordingly, the processor (150) can determine the similarity based on the scale information, rotation information, and translation information of the nodes of each graph for the two maps, and match nodes with high similarity to create a spatial map.

[0114] Meanwhile, the processor (150) can determine the driving direction based on sensing values ​​sensed from a plurality of sensors (110) while the robot (100) is driving in space. Based on the driving direction and the guide map, the processor (150) can estimate in real time the point where the robot is located within the guide map.

[0115] For example, when the robot (100) moves through a space based on a plurality of sensors (110), the robot (100) can determine the driving direction as left, right, or front. When the robot moves from a specific space to another space, if the only directions in which the robot can move are right or front, the robot (100) can estimate an area where it cannot move to the left from the input guide map as the point in which it is located. As another example, when the only direction in which the robot can move is left, the robot can estimate an area where there is a path that can only move to the left from the input guide map as the point in which it is located.

[0116] FIG. 9 is an overall diagram of a spatial map generation process of a robot according to at least one embodiment of the present disclosure.

[0117] If the processor (150) does not receive a guide map, it can generate a full sensing map for the entire space. However, if the processor (150) receives a guide map (610) drawn by a user, it can generate a sensing map (910) based on the determined search boundary. The processor (150) can match the guide map (610) and the sensing map (910) to ultimately generate a space map (920). The space map refers to a map that is ultimately generated for the robot (100) to navigate the space based on conditions set by the user. Therefore, the robot (100) can navigate the space based on the space map (920).

[0118] The processor (150) can extract nodes from each of the guide map graph and the sensing map graph, and compare and match each node. The processor (150) can match each node to create a single spatial map (920). The spatial map (920) can be a map created by matching similar nodes among the nodes extracted from the guide map (610) and the sensing map (910) and overlapping them into one. The robot (100) can drive only within a range set by the user through the spatial map (920). Thus, the robot (100) can perform tasks or provide services while driving through a space using the spatial map (920).

[0119] FIG. 10 is a flowchart illustrating a method for generating a map of a robot according to at least one embodiment of the present disclosure.

[0120] Referring to FIG. 10, a robot receives and stores a guide map to guide map creation (S1010). The robot converts the guide map into a map format recognizable by the robot (S1020). The robot identifies a search boundary determined by the converted guide map (S1030). The robot generates a sensing map for the space in which the robot is located based on sensing values ​​sensed by multiple sensors while driving within the search boundary (S1040). The robot matches the guide map and the sensing map (S1050) to generate a spatial map for the space (S1060).

[0121] The method of creating a guide map and a sensing map to guide map creation, and the method of creating a spatial map by matching the guide map and the sensing map have been specifically described in the various embodiments described above, so a duplicate description will be omitted.

[0122] The control method described in Fig. 10 can be performed by a robot (100) having the configuration of Fig. 2 described above, but is not necessarily limited thereto, and can be performed by a robot having various configurations.

[0123] The various embodiments described above may be implemented as a single embodiment, or at least one of the embodiments may be combined in whole or in part to be implemented together in one device.

[0124] According to the various embodiments described above, the robot receives a guide map to guide map creation, enabling more accurate and efficient space map creation without requiring the user to visit the space in person and create the map together with the robot.

[0125] Meanwhile, the various embodiments described above may be applied to a product as an embodiment alone, but at least some of the contents may be implemented in combination with other embodiments of the present disclosure.

[0126] The various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine is a device that can call instructions stored in the storage medium and operate according to the called instructions, and may include an electronic device (e.g., a robot (100)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium can be provided in the form of a non-transitory computer-readable storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0127] Additionally, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product.

[0128] Specifically, a non-transitory readable storage medium or a computer program product storing computer instructions for causing the computer to perform operations including a step of receiving and storing a guide map for guiding map creation, a step of converting the guide map into a map form recognizable by the robot, a step of identifying a search boundary determined by the converted guide map, a step of generating a sensing map for a space in which the robot is located based on sensing values ​​sensed by a plurality of sensors provided in the robot while the robot is moving within the search boundary, and a step of generating a space map for the space by matching the guide map and the sensing map may be provided.

[0129] The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0130] In addition, computer instructions or programs for performing the map generation method of the robot according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0131] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In robots, Multiple sensors; driving part; memory; Processor; including; The above processor, Input a guide map to guide map creation and store it in the above memory, Convert the above guide map into a map format that the robot can recognize, Identifying a navigation boundary determined by the above-mentioned converted guide map, and controlling the driving unit to drive within the navigation boundary, Generate a sensing map for the space where the robot is located based on the sensing values ​​sensed by the plurality of sensors while driving, A robot that creates a spatial map for the space by matching the above guide map and the above sensing map.

2. In paragraph 1, display; including more; The above processor, A robot that recognizes a drawing map drawn by a user on the above display as the guide map.

3. In paragraph 1, The above processor, When a drawing map drawn by a user or a blueprint for the space is captured by a camera among the plurality of sensors, A robot that identifies the drawing map or blueprint from the photographed image and recognizes it as the guide map.

4. In paragraph 1, The above processor, Preprocessing is performed to convert the above guide map into a map format that can be recognized by the robot. Each of the guide map and the sensing map converted by the above preprocessing task is decomposed into node units and reconstructed in graph form. A robot that creates a spatial map by matching nodes within each of the reconstructed graphs based on similarity.

5. In paragraph 4, The above processor, Extract a straight line from each node of the above extracted sensing map, Each convex hull is extracted based on the vertex of the straight line of each node of the extracted sensing map and the vertex of each node extracted from the guide map. A turning function graph of each extracted convex hull is calculated, and candidate nodes to be matched with each other in the sensing map and the guide map are identified based on the similarity of the calculated turning function graphs. A robot that creates a spatial map by matching nodes with high similarity to each other based on scale information, rotation information, and translation information of the candidate nodes.

6. In paragraph 1, display; including more; The above processor, Controlling the display to display a partial map generated at the current location of the robot among the spaces where the robot is located; A robot that, when the user additionally draws a map on the partial map on the display, stores the created map as the guide map in the memory.

7. In paragraph 1, The above processor, While the robot is driving in the space, the driving direction is determined based on the sensing values ​​sensed from the plurality of sensors, A robot that estimates in real time the point where the robot is located within the guide map based on the driving direction and the guide map.

8. In the method of creating a robot map, A step for receiving and saving a guide map to guide map creation; A step of converting the above guide map into a map form recognizable by the robot; A step of identifying a navigation boundary determined by the above-mentioned converted guide map; A step of generating a sensing map for a space where the robot is located based on sensing values ​​of a sensor equipped in the robot while the robot is driving within the search boundary; and A method for generating a map, comprising: a step of generating a spatial map for the space by matching the guide map and the sensing map.

9. In paragraph 8, The step of entering and saving the above guide map is: A method for generating a map, comprising: a step of recognizing a drawing map drawn by a user on a display included in the robot as the guide map.

10. In paragraph 8, The step of entering and saving the above guide map is: When a drawing map drawn by a user or a blueprint for the space is captured by a camera among the plurality of sensors, A method for generating a map, comprising: a step of identifying the drawing map or design drawing from the photographed image and recognizing it as the guide map.

11. In paragraph 8, The steps for generating the above spatial map are: A step of reconstructing each of the above-mentioned converted guide map and the above-mentioned sensing map into a graph form by decomposing them into node units; and A method for generating a map, comprising: a step of matching nodes within each of the reconstructed graphs based on similarity.

12. In paragraph 11, The steps for generating the above spatial map are: A step of extracting a straight line from each node of the above sensing map; A step of extracting each convex hull based on the vertex of the straight line of each node of the sensing map and the vertex of each node extracted from the guide map; A step of producing a turning function graph of each extracted convex hull; A step of identifying candidate nodes to be matched with each other within the sensing map and the guide map based on the similarity of the generated turning function graphs; and A method for creating a map, further comprising: a step of matching nodes with high similarity to each other based on scale information, rotation information, and translation information of the candidate nodes.

13. In paragraph 8, The step of entering and saving the above guide map is: A step of displaying a partial map generated at the current location of the robot among the spaces where the robot is located through a display included in the robot; and A method for creating a map, further comprising: a step of saving the created map as the guide map when the user additionally draws on the partial map on the display to create a map.

14. In paragraph 8, The steps for generating the above sensing map are: A step of determining a driving direction based on sensing values ​​sensed from the plurality of sensors while the robot drives in the space; and A map generation method further comprising: a step of estimating in real time a point where the robot is located within the guide map based on the driving direction and the guide map.

15. A non-transitory computer-readable storage medium storing computer instructions that, when executed by a processor of the robot, cause the robot to perform an action, the action being: A step for receiving and saving a guide map to guide map creation; A step of converting the above guide map into a map form recognizable by the robot; A step of identifying a navigation boundary determined by the above-mentioned converted guide map; A step of generating a sensing map for a space where the robot is located based on sensing values ​​sensed by a sensor equipped in the robot while the robot is driving within the search boundary; and A non-transitory computer-readable storage medium, comprising: a step of generating a spatial map for the space by matching the guide map and the sensing map.

Images