A map generation method and robot

Abstract

The present application discloses a map generation method and a robot, which uses a preset object model to identify an object captured by a camera, maps the coordinates of the object acquired by the camera into coordinates of a specified dimension, and annotates the map generated by the SLAM system according to the recognition result and the coordinates to generate a map annotated with the object. By applying the technical solution proposed in the embodiment of the present application, the robot can automatically annotate objects on the map when building a map. This allows users to annotate on the map more conveniently, thereby solving the problem that the robot cannot annotate objects when building a map.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a map generation method. The present invention also designs a robot. Background Art

[0002] Robot positioning has always been the focus and difficulty of robot technology research. It is also the prerequisite for robot navigation and is of great significance to improving the level of robot intelligence.

[0003] SLAM (Simultaneous Localization and Mapping) is currently one of the key technologies for realizing truly autonomous mobile robots. This technology enables the robot to start moving from an unknown position in an unknown environment, locate itself according to position estimation and maps during the movement, and build incremental maps based on its own positioning, thereby realizing autonomous positioning and navigation of the robot.

[0004] The implementation and difficulty of SLAM are closely related to the form and installation method of the sensor. Sensors are divided into two categories: laser and vision:

[0005] (1) LiDAR (Laser SLAM)

[0006] LiDAR is the oldest and most researched SLAM sensor. It provides distance information between the robot and the surrounding obstacles. Common LiDARs, such as SICK, Velodyne, and our domestic rplidar, can be used for SLAM. LiDAR can measure the angle and distance of obstacles around the robot with high accuracy, thus easily realizing SLAM, obstacle avoidance and other functions.

[0007] (2) Visual SLAM

[0008] Due to the increase in CPU and GPU processing speed, many visual algorithms that were previously considered impossible to run in real time can now run at speeds above 10Hz. The improvement of hardware has also promoted the development of visual SLAM.

[0009] In terms of sensors, visual SLAM research is mainly divided into three categories: monocular, binocular (or multi-ocular), and RGBD.

[0010] In the process of implementing the present application, the inventor discovered that when the prior art uses laser SLAM technology to build a 2D map, the information obtained is a two-dimensional plane map of the map. When a 3D map is built through a sparse point cloud of visual SLAM, the information obtained is a sparse 3D point cloud. However, to match these two types of information with the real world, people need to rely on imagination to match them. For example, when a laser SLAM scans an open door, only a blank space will be displayed on the map instead of a door.

[0011] It can be seen that how to enable the robot to automatically mark objects when building a map has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0012] The present application provides a map generation method to enable automatic labeling of objects when a robot is building a map.

[0013] In order to achieve the above objectives, the present application provides a map generation method, which is applied to a robot, characterized in that the robot is provided with a camera and a SLAM system, and the method comprises:

[0014] Using a preset object model to identify the object captured by the camera;

[0015] Mapping the coordinates of the object acquired by the camera into coordinates of a specified dimension;

[0016] The map generated by the SLAM system is annotated according to the recognition result and the coordinates to generate a map annotated with the object.

[0017] Preferably, before using a preset object model to identify the object captured by the camera, the method further includes:

[0018] Object model training is performed on all objects that need to be labeled in the SLAM system to generate object models corresponding to each of the objects.

[0019] Preferably, when the SLAM system is a laser SLAM system, the map is generated by laser data collected by the SLAM system, and the specified dimension is two-dimensional.

[0020] Preferably, when the SLAM system is a visual SLAM system, the map is generated by data collected by the SLAM system through the camera, and the specified dimension is three-dimensional.

[0021] Preferably, the type of the camera corresponding to the laser SLAM system is a depth camera;

[0022] The types of the cameras corresponding to the visual SLAM system include a monocular camera, a binocular camera and the depth camera.

[0023] The present application provides a robot, characterized in that the robot is provided with a camera and a SLAM system, and the robot also includes:

[0024] A recognition module, used to recognize the object captured by the camera using a preset object model;

[0025] A mapping module, used to map the coordinates of the object acquired by the camera into coordinates of a specified dimension;

[0026] A labeling module is used to label the map generated by the SLAM system according to the recognition result and the coordinates to generate a map labeled with the object.

[0027] Preferably, before using a preset object model to identify the object captured by the camera, the method further includes:

[0028] The model training module is used to perform object model training on all objects that need to be labeled in the SLAM system to generate object models corresponding to each of the objects.

[0029] Preferably, when the SLAM system is a laser SLAM system, the map is generated by laser data collected by the SLAM system, and the specified dimension is two-dimensional.

[0030] Preferably, when the SLAM system is a visual SLAM system, the map is generated by data collected by the SLAM system through the camera, and the specified dimension is three-dimensional.

[0031] Preferably, the type of the camera corresponding to the laser SLAM system is a depth camera;

[0032] The types of the cameras corresponding to the visual SLAM system include a monocular camera, a binocular camera and the depth camera.

[0033] Compared with the prior art, the beneficial technical effects of the technical solution proposed in the embodiment of the present application include:

[0034] The method uses a preset object model to identify the object captured by the camera, maps the coordinates of the object acquired by the camera into coordinates of a specified dimension, and annotates the map generated by the SLAM system according to the recognition result and the coordinates to generate a map annotated with the object. By applying the technical solution proposed in the embodiment of the present application, the robot can automatically annotate objects on the map when building a map. This allows users to annotate on the map more conveniently, solving the problem that the robot cannot annotate objects when building a map. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the map generation process provided in the embodiment of the present application;

[0036] Figure 2 A schematic diagram of training objects that need to be labeled provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the process of generating a map by the laser SLAM system provided in Example 1 of the present application;

[0038] Figure 4 A schematic diagram of the process of generating a map by a visual SLAM system provided in Example 2 of the present application;

[0039] Figure 5 A module architecture diagram for map generation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] As stated in the background of this application, when the prior art uses laser SLAM technology to build a 2D map, the information obtained is a two-dimensional plane map of the map. When the sparse point cloud of visual SLAM is used to build a 3D map, the information obtained is a sparse 3D point cloud. There is a lack of a solution that enables the robot to automatically annotate objects when building a map.

[0041] In view of the problems in the background technology, the present application proposes a map generation method, which can provide a unified and complete methodology to realize the automatic labeling of objects when the robot is building a map, so as to clearly and directly know what the object is, thereby solving the problem of automatic labeling of objects when the robot is building a map.

[0042] like Figure 1 FIG. 1 is a flow chart of a method for automatically labeling objects when a robot establishes a map according to an embodiment of the present application. The method is applied to a robot, and the robot is provided with a camera and a SLAM system. Specifically, the method comprises the following steps:

[0043] S101, using a preset object model to identify the object captured by the camera.

[0044] In a preferred embodiment, before using a preset object model to identify the object captured by the camera, object model training is performed on all objects that need to be labeled in the SLAM system to generate object models corresponding to each of the objects.

[0045] The present application provides a multi-camera robot or SLAM system, wherein a front camera, a rear camera, a left camera, and a rear camera are installed on the robot body. Obviously, the camera positions described are only part of the embodiments of the present application, not all of the embodiments. The camera device provided in the present application is a depth camera or a monocular camera or a binocular camera.

[0046] The pre-set object model is obtained by scanning a large number of physical objects with the laser or visual system of the SLAM system, and the SLAM system records and stores the obtained object image information, so that the object captured by the camera can be identified.

[0047] S102: Map the coordinates of the object acquired by the camera into coordinates of a specified dimension.

[0048] Among them, how to obtain the coordinates of the object, that is, how the SLAM system generates a map, is the same as the acquisition and generation method of the prior art, so it will not be elaborated here;

[0049] According to the implementation and difficulty of SLAM, it is closely related to the form and installation method of the sensor (the sensor is divided into two categories: laser and vision). After the camera obtains the coordinates of the object,

[0050] When the SLAM system is a laser SLAM system, the map is generated by laser data collected by the SLAM system, the specified dimension is two-dimensional, and the type of the camera corresponding to the laser SLAM system is a depth camera; or

[0051] When the SLAM system is a visual SLAM system, the map is generated by the data collected by the SLAM system through the camera, the specified dimension is three-dimensional, and the types of the cameras corresponding to the visual SLAM system include a monocular camera, a binocular camera and the depth camera.

[0052] The SLAM system will match the corresponding camera according to different SLAM systems (laser / vision) and then generate the corresponding dimensional coordinates.

[0053] S103: annotate the map generated by the SLAM system according to the recognition result and the coordinates to generate a map annotated with the object.

[0054] The SLAM system marks the recognition results and the coordinates on the map generated by the SLAM system. The marking clarifies the information of the object. For example, when an open door is scanned, the marking will match the coordinates to mark the door's matching information on the map generated by the SLAM system, thereby generating a map marked with the object.

[0055] Compared with the prior art, the beneficial technical effects of the technical solution proposed in the embodiment of the present application also include:

[0056] The embodiment of the present application discloses a map generation method and a robot. In a robot with multiple cameras, the method uses a preset object model to identify the object captured by the camera, maps the coordinates of the object acquired by the camera to coordinates of a specified dimension, and annotates the map generated by the SLAM system according to the recognition result and the coordinates to generate a map annotated with the object. By applying the technical solution proposed in the embodiment of the present application, the robot can automatically annotate the objects that need to be annotated on the map when building a map. This allows users to annotate on the map more conveniently, solving the problem that the robot cannot automatically annotate objects when building a map.

[0057] In order to further explain the technical idea of ​​this application, the technical solution of this application is now described in combination with specific application scenarios. And according to the implementation and difficulty of SLAM and the close relationship between the form and installation method of the sensor (the sensor is divided into two categories: laser and vision), this application has two different implementation schemes:

[0058] Example 1, see Figure 3 , is a schematic diagram of the process of laser Slam provided in an embodiment of the present application, and the process may include:

[0059] S301, the robot uses laser scanning and a depth camera to obtain laser data and depth camera data respectively.

[0060] Among them, the laser Slam system includes a laser scanning the target object to capture the image information of the target object, and the robot records the image information and generates laser data;

[0061] At the same time, the depth camera also scans the target object and captures image information of the target object to generate depth camera data.

[0062] This application provides a depth camera, which can be implemented in many different ways, such as dual-camera parallax, a single camera capturing the same scene at different angles by moving, photometric stereo, etc. There are even ML methods to reconstruct the scene model, or calculate the distance by focusing at different distances multiple times.

[0063] S302, the laser SLAM system acquires laser data and generates a corresponding map.

[0064] The map can be represented in three ways: grid representation, geometric feature representation and topological representation. A two-dimensional map is established based on laser SLAM, and SLAM has three quantities in the two-dimensional world, the x-axis, the y-axis, and the direction angle.

[0065] S303: Using a preset object model to identify the data collected by the depth camera.

[0066] The preset object model is obtained by scanning a large number of physical objects with a laser or visual method of the SLAM system, and the SLAM system records and stores the obtained object image information, so that the object model can identify the object captured by the camera.

[0067] S304, mapping two-dimensional coordinates according to the depth information coordinates.

[0068] S305, regenerating a marked two-dimensional map on the map generated by the laser SLAM system in combination with the object model recognition result.

[0069] In this embodiment, the robot uses laser scanning and a depth camera to obtain laser data and depth camera data respectively, the laser SLAM system obtains laser data and generates a corresponding map, uses a preset object model to identify the data collected by the depth camera, maps two-dimensional coordinates according to the depth information coordinates, and combines the object model recognition results to regenerate a marked two-dimensional map on the map generated by the laser SLAM system. This enables the robot to automatically mark the objects that need to be marked on the map when building the map.

[0070] Example 2, see Figure 4 , is a schematic diagram of the process of visual Slam provided in an embodiment of the present application, and the process may include:

[0071] S401, the robot acquires data using monocular, binocular and depth cameras respectively.

[0072] Wherein, the visual SLAM system includes the camera described above scanning the target object, capturing the image information of the target object, and the robot records the image information and generates corresponding data;

[0073] The present application provides a visual SLAM system, which is divided into four modules (excluding sensor data reading): VO, (Visual Odometry) backend, mapping, and loop detection.

[0074] S402, the visual SLAM system acquires the data and generates a corresponding map.

[0075] Among them, the map can be represented in three ways: grid representation, geometric feature representation and topological map representation. The three-dimensional map established based on visual SLAM will be much more complicated in the three-dimensional world, with 6 quantities, x, y, z, roll (roll angle Φ), yaw (yaw angle ψ), pitch (pitch angle θ).

[0076] S403: Using a preset object model to identify the data collected by the depth camera.

[0077] The preset object model is obtained by scanning a large number of physical objects with a laser or visual method of the SLAM system, and the SLAM system records and stores the obtained object image information, so that the object model can identify the object captured by the camera.

[0078] S404, mapping three-dimensional coordinates according to the depth information coordinates.

[0079] S405, regenerating a three-dimensional map with annotations on the map generated by the laser SLAM system in combination with the object model recognition result.

[0080] In this embodiment, the robot uses a monocular, binocular and depth camera to obtain data respectively, the visual SLAM system obtains the data and generates a corresponding map, uses a preset object model to identify the data collected by the depth camera, maps three-dimensional coordinates according to the depth information coordinates, and combines the object model recognition results to regenerate a three-dimensional map with annotations on the map generated by the laser SLAM system. This enables the robot to automatically mark the objects that need to be marked on the map when building the map.

[0081] Based on the same inventive concept as the above method, the present application also provides a robot, such as Figure 5 As shown, including:

[0082] The recognition module 52 is used to recognize the object captured by the camera using a preset object model;

[0083] A mapping module 53, used to map the coordinates of the object acquired by the camera into coordinates of a specified dimension;

[0084] The labeling module 54 is used to label the map generated by the SLAM system according to the recognition result and the coordinates to generate a map labeled with the object.

[0085] In a preferred embodiment, before using a preset object model to identify the object captured by the camera, it also includes:

[0086] The model training module 51 is used to perform object model training on all objects that need to be labeled in the SLAM system to generate object models corresponding to each of the objects.

[0087] Preferably, when the SLAM system is a laser SLAM system, the map is generated by laser data collected by the SLAM system, and the specified dimension is two-dimensional.

[0088] Preferably, when the SLAM system is a visual SLAM system, the map is generated by data collected by the SLAM system through the camera, and the specified dimension is three-dimensional.

[0089] Preferably, the type of the camera corresponding to the laser SLAM system is a depth camera;

[0090] The types of the cameras corresponding to the visual SLAM system include a monocular camera, a binocular camera and the depth camera.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0092] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0093] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0094] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0095] The above disclosure only discloses several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by technicians in this field should fall within the scope of protection of the present application.

Claims

1. A map generation method, applied to a robot, characterized in that: The robot is provided with a camera and a SLAM system, and the method comprises: using a preset object model to identify the object captured by the camera: mapping the coordinates of the object acquired by the camera to coordinates of a specified dimension; annotating a map generated by the SLAM system according to the recognition result and the coordinates to generate a map annotated with the object; before using the preset object model to identify the object captured by the camera, the method further comprises: performing object model training on all objects that need to be annotated in the SLAM system to generate object models corresponding to each of the objects; when the SLAM system is a laser SLAM system, the map is generated by laser data collected by the SLAM system, and the specified dimension is two-dimensional; when the SLAM system is a visual SLAM system, the map is generated by data collected by the SLAM system through the camera, and the specified dimension is three-dimensional.

2. The method according to claim 1, characterized in that The type of the camera corresponding to the laser SLAM system is a depth camera; The types of cameras corresponding to the visual SLAM system include monocular cameras, binocular cameras, and Depth camera.

3. A robot, characterized in that: The robot is provided with a camera and a SLAM system, and the robot also includes: The recognition module is used to recognize the object captured by the camera using a preset object model: A mapping module, used to map the coordinates of the object acquired by the camera into coordinates of a specified dimension; A labeling module is used to label the map generated by the SLAM system according to the recognition result and the coordinates. before using the preset object model to identify the object captured by the camera, the method further includes: The model training module is used to train the object model of all objects that need to be labeled in the SLAM system to generate to form an object model corresponding to each of the objects; when the SLAM system is a laser SLAM system, the The map is generated by laser data collected by the SLAM system, and the specified dimension is two-dimensional; when the SLAM system is a visual SLAM system, the map is generated by data collected by the SLAM system through the camera, and the specified dimension is three-dimensional.

4. The robot according to claim 3, characterized in that: The type of the camera corresponding to the laser SLAM system is a depth camera; the type of the camera corresponding to the visual SLAM system includes a monocular camera, a binocular camera and the depth camera.

Images