An electronic map generation method and device, a mobile robot, and a storage medium

Abstract

Embodiments of the present application provide an electronic map generation method and device, a mobile robot and a storage medium. The method comprises: acquiring a plurality of depth images of an indoor scene captured by a binocular camera; establishing a two-dimensional grid map according to each depth image and determining position information of a roof and a door frame in the indoor scene; and determining a passable area boundary and a closed area boundary of the two-dimensional grid map according to the position information of the roof and the door frame, to obtain an electronic map of the indoor scene. The technical solution provided by the embodiments of the present application can avoid the problem that the binocular vision sensor cannot detect the closed boundary and the passable area in the grid map when the field of view of the binocular vision sensor is upward, and improve the accuracy of the grid map.

Description

Technical Field

[0001] This application relates to the field of robot vision technology, and in particular to an electronic map generation method, apparatus, mobile robot, and storage medium. Background Technology

[0002] In recent years, with the development of artificial intelligence, indoor mobile robots have attracted increasing attention, and map building is a major problem that needs to be solved in the unmanned navigation process of indoor mobile robots. Two-dimensional grid maps are the most common type of map in unmanned navigation, so how to quickly build accurate two-dimensional grid maps has become a very important issue.

[0003] In related technologies, when constructing a two-dimensional grid map of an indoor scene using a vision sensor, the first step is to use the vision sensor to acquire a depth image of the indoor scene. Then, the depth information of obstacles on the robot's travel plane in the depth image is used to construct the two-dimensional grid map. However, two-dimensional grid maps constructed using the above method often suffer from low boundary accuracy. Summary of the Invention

[0004] The purpose of this application is to provide an electronic map generation method, apparatus, mobile robot, and storage medium to improve the accuracy of two-dimensional raster map boundaries. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an electronic map generation method, the method comprising:

[0006] Acquire multiple depth images of an indoor scene captured by a stereo camera;

[0007] Based on the depth images, a two-dimensional grid map is constructed and the location information of the roof and door frame in the indoor scene is determined.

[0008] Based on the location information of the roof and the door frame, the boundaries of the passable area and the closed area of ​​the two-dimensional grid map are determined to obtain an electronic map of the indoor scene.

[0009] In one possible implementation, the step of establishing a two-dimensional grid map based on each of the depth images and determining the location information of the roof and door frames in the indoor scene includes:

[0010] For each depth image, determine the device pose, obstacle positions of each obstacle, and first position information of the roof and door frame in that depth image, wherein the device is the binocular camera or a mobile robot equipped with the binocular camera;

[0011] Based on the obstacle positions of each obstacle in the depth image, a two-dimensional raster map is constructed in the depth image.

[0012] The step of determining the passable area boundary and closed area boundary of the two-dimensional grid map based on the location information of the door frame and the roof to obtain an electronic map of the indoor scene includes:

[0013] For each depth image, using the first position information of the door frame and / or roof under the depth image, the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image are determined, and the boundary two-dimensional raster map of the depth image is obtained.

[0014] Based on the device pose under each depth image, the boundary two-dimensional raster maps of each depth image are fused and corrected to obtain an electronic map of the indoor scene.

[0015] In one possible implementation, the step of establishing a two-dimensional grid map based on each of the depth images and determining the location information of the roof and door frames in the indoor scene includes:

[0016] For each depth image, determine the device pose, obstacle positions of each obstacle, and first position information of the roof and door frame in that depth image, wherein the device is the binocular camera or a mobile robot equipped with the binocular camera;

[0017] Based on the device pose in each depth image and the obstacle position of each obstacle, a two-dimensional grid map of the indoor scene is established;

[0018] Based on the device pose in each depth image and the first position information of the roof and the door frame, the second position information of the roof and the door frame in the indoor scene is determined;

[0019] The step of determining the accessible area boundaries and closed area boundaries of the two-dimensional grid map based on the location information of the roof and the door frame to obtain an electronic map of the indoor scene includes:

[0020] Based on the second position information of the roof and the door frame in the indoor scene, the passable area boundary and the closed area boundary of the two-dimensional raster map of the indoor scene are determined, and an electronic map of the indoor scene is obtained.

[0021] In one possible implementation, determining the device pose, obstacle positions, and first position information of the roof and door frame for each depth image includes:

[0022] For each depth image, the robot pose of the mobile robot and the obstacle positions of each obstacle are obtained in that depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle positions are the positions of the obstacles in the camera coordinate system of the binocular camera.

[0023] The depth image is used to detect single right-angle feature regions and coordinate system-like right-angle feature regions. When a single right-angle feature region is detected, the position information of the single right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the door frame in the depth image. When a coordinate system-like right-angle feature region is detected, the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the roof in the depth image.

[0024] In one possible implementation, for each depth image, using the first position information of the door frame and / or roof under the depth image, determining the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image to obtain the boundary two-dimensional raster map of the depth image includes:

[0025] For each depth image, the first position information of the door frame and / or roof under the depth image is transformed from the camera coordinate system to the robot coordinate system to obtain the third position information of the door frame and / or roof under the depth image;

[0026] Based on the third position information of the door frame and / or roof in the depth image, the passable area boundary and / or roof closed area boundary of the two-dimensional grid map in the depth image are determined, wherein the two-dimensional grid map in the depth image is a map in the robot coordinate system.

[0027] In one possible implementation, determining the accessible area boundary and / or roof enclosure area boundary of the two-dimensional raster map under the depth image based on the third location information of the door frame and / or roof under the depth image includes:

[0028] Based on the third position information of the door frame and / or roof in the depth image, the corner points of the door frame and / or roof in the depth image are used as the center points and extended outward by a predetermined length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or the roof.

[0029] The endpoints of the door frame and / or the roof are combined and projected onto a two-dimensional raster map under the depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof;

[0030] In the case of two projected right angles with collinear right angles in the two-dimensional raster map of the depth image, the raster state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional raster map under the depth image is obtained.

[0031] In the case where there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thus obtaining the passable area boundary of the two-dimensional raster map under the depth image.

[0032] In one possible implementation, the method further includes:

[0033] Based on the third position information of the door frame in the depth image, calculate the first minimum Euclidean distance between the door frame and the mobile robot in the depth image; when the first minimum Euclidean distance is less than a preset threshold, determine the door frame in the depth image as a false detection; and / or,

[0034] Based on the third location information of the roof in the depth image, the second minimum Euclidean distance between the roof and the mobile robot in the depth image is calculated; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

[0035] Secondly, embodiments of this application provide an electronic map generation apparatus, the apparatus comprising:

[0036] The image acquisition module is used to acquire multiple depth images of an indoor scene captured by a stereo camera;

[0037] The map building module is used to build a two-dimensional raster map based on the depth images and to determine the location information of the roof and door frame in the indoor scene;

[0038] The boundary determination module is used to determine the passable area boundary and the closed area boundary of the two-dimensional grid map based on the location information of the roof and the door frame, so as to obtain an electronic map of the indoor scene.

[0039] In one possible implementation, the map building module includes:

[0040] The location information confirmation submodule is used to determine the device pose, obstacle position of each obstacle, and first position information of the roof and door frame for each depth image. The device is the binocular camera or a mobile robot equipped with the binocular camera.

[0041] The first raster map generation submodule is used to build a two-dimensional raster map under the depth image based on the obstacle positions of each obstacle under the depth image;

[0042] The boundary determination module includes:

[0043] The first boundary determination submodule is used to determine the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image for each depth image by using the first position information of the door frame and / or roof under the depth image, so as to obtain the boundary two-dimensional raster map of the depth image.

[0044] The fusion correction submodule is used to fuse and correct the boundary two-dimensional raster maps of each depth image based on the device pose under each depth image, so as to obtain an electronic map of the indoor scene.

[0045] In one possible implementation, the map building module includes:

[0046] The location information confirmation submodule is used to determine the device pose, obstacle position of each obstacle, and first position information of the roof and door frame for each depth image. The device is the binocular camera or a mobile robot equipped with the binocular camera.

[0047] The second grid map generation submodule is used to establish a two-dimensional grid map of the indoor scene based on the device pose under each depth image and the obstacle position of each obstacle.

[0048] The second location information confirmation submodule is used to determine the second location information of the roof and the door frame in the indoor scene based on the device pose under each depth image and the first location information of the roof and the door frame;

[0049] The boundary determination module includes:

[0050] The second boundary determination submodule is used to determine the passable area boundary and closed area boundary of the two-dimensional raster map of the indoor scene based on the second position information of the roof and the door frame in the indoor scene, so as to obtain the electronic map of the indoor scene.

[0051] In one possible implementation, the location information confirmation submodule is specifically used for:

[0052] For each depth image, the robot pose of the mobile robot and the obstacle positions of each obstacle are obtained in that depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle positions are the positions of the obstacles in the camera coordinate system of the binocular camera.

[0053] The depth image is used to detect single right-angle feature regions and coordinate system-like right-angle feature regions. When a single right-angle feature region is detected, the position information of the single right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the door frame in the depth image. When a coordinate system-like right-angle feature region is detected, the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the roof in the depth image.

[0054] In one possible implementation, the first boundary determination submodule includes:

[0055] The coordinate transformation unit is used to transform the first position information of the door frame and / or roof in the depth image from the camera coordinate system to the robot coordinate system for each depth image, so as to obtain the third position information of the door frame and / or roof in the depth image.

[0056] The boundary determination unit is used to determine the passable area boundary and / or roof closed area boundary of the two-dimensional grid map under the depth image based on the third position information of the door frame and / or roof under the depth image, wherein the two-dimensional grid map under the depth image is a map in the robot coordinate system.

[0057] In one possible implementation, the boundary determination unit is specifically used for:

[0058] Based on the third position information of the door frame and / or roof in the depth image, the corner points of the door frame and / or roof in the depth image are used as the center points and extended outward by a predetermined length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or the roof.

[0059] The endpoints of the door frame and / or the roof are combined and projected onto a two-dimensional raster map under the depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof;

[0060] In the case of two projected right angles with collinear right angles in the two-dimensional raster map of the depth image, the raster state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional raster map under the depth image is obtained.

[0061] In the case where there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thus obtaining the passable area boundary of the two-dimensional raster map under the depth image.

[0062] In one possible implementation, the device further includes:

[0063] The Euclidean distance calculation module is used to calculate a first minimum Euclidean distance between the door frame and the mobile robot in the depth image based on the third position information of the door frame in the depth image; when the first minimum Euclidean distance is less than a preset threshold, the door frame in the depth image is determined to be a false detection; and / or,

[0064] Based on the third location information of the roof in the depth image, the second minimum Euclidean distance between the roof and the mobile robot in the depth image is calculated; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

[0065] Thirdly, this application provides a mobile robot, which includes a binocular camera, a memory, and a processor. The binocular camera is used to acquire left and right eye images of an indoor scene and generate a depth image of the indoor scene based on the left and right eye images.

[0066] The memory is used to store computer programs;

[0067] When the processor executes the program stored in the memory, it implements any of the electronic map generation methods described in this application.

[0068] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the electronic map generation methods described in this application.

[0069] The electronic map generation method, apparatus, and mobile robot provided in this application embodiment establish a two-dimensional grid map based on depth images of an indoor scene captured by a binocular camera, determine the positional information of rooftops and door frames in the indoor scene, and then determine the boundaries of passable and closed areas of the two-dimensional grid map based on the positional information of the rooftops and door frames to obtain an electronic map of the indoor scene. It can be seen that the method of this application embodiment, by utilizing the positional information of rooftops and door frames while constructing the two-dimensional grid map, determines the boundaries of passable and closed areas of the two-dimensional grid map, thereby improving the accuracy of the two-dimensional grid map boundaries.

[0070] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0072] Figure 1 This is a first schematic diagram of an electronic map generation method provided in an embodiment of this application;

[0073] Figure 2 An example diagram of a mobile robot equipped with a binocular camera provided in an embodiment of this application;

[0074] Figure 3 This is a second schematic diagram of the electronic map generation method provided in the embodiments of this application;

[0075] Figure 4 An example diagram of the electronic map generation method provided in the embodiments of this application;

[0076] Figure 5 This is a third schematic diagram of the electronic map generation method provided in the embodiments of this application;

[0077] Figure 6 This is a schematic diagram of one possible implementation of step S121 in this application;

[0078] Figure 7 This is a schematic diagram of one possible implementation of step S131 in this application;

[0079] Figure 8 This is a schematic diagram of one possible implementation of step S1312 in this application;

[0080] Figure 9 This is a flowchart of an electronic map generation method according to an embodiment of this application;

[0081] Figure 10 This is a schematic diagram of an electronic map generation device according to an embodiment of this application. Detailed Implementation

[0082] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0083] First, the technical terms used in the embodiments of this application will be explained:

[0084] SLAM: The full name of SLAM is Simultaneous Localization and Mapping.

[0085] Line characteristics: A representation unit that uses the entire line segment as a constraint;

[0086] Keyframe: An image frame that represents key information at a specific moment during motion or during a scene change;

[0087] Raster map: A map whose status is indicated by whether a single raster cell is occupied;

[0088] Holes in depth maps: When some areas lack texture information or reflect light, depth values ​​will be missing in the depth map, thus forming holes.

[0089] Route Finder: Search the map for accessible paths to the next destination.

[0090] To improve the accuracy of the boundaries of two-dimensional raster maps, a first aspect of this application provides an electronic map generation method that can be applied to electronic devices. In specific applications, the electronic device can be a computing device, such as a server or a mobile robot, all of which fall within the scope of this application.

[0091] See Figure 1 This application provides an electronic map generation method, which includes:

[0092] Step S11: Acquire multiple depth images of the indoor scene captured by the binocular camera;

[0093] The binocular camera can be mounted on a mobile robot. For example, such as... Figure 2 As shown, the ground mobile robot equipped with a logic operation unit is equipped with a binocular camera. The angle between the binocular camera and the horizontal plane is α, which can be set to be greater than or equal to 45°. The vertical field of view of the binocular camera is β, which is related to the actual model of the binocular camera. In this embodiment of the application, the size of the angle α between the binocular camera sensor and the horizontal plane and the vertical field of view β are not specifically limited.

[0094] The depth image is acquired in real time by the stereo camera. Specifically, the left and right eyes of the stereo camera acquire left and right eye images respectively. Pixel matching is performed on the left and right eye images, and the depth of each pixel is calculated based on the matching results to obtain the depth image. The specific algorithm for obtaining the depth image based on the left and right eye images can be found in existing technologies and is not specifically limited in this application. In one example, the depth image can be the depth image of a keyframe created after the left and right eye images are synchronized. By continuously creating keyframes, multiple depth images can be obtained.

[0095] For example, during its movement, the mobile robot continuously acquires left and right eye images of the indoor scene using its onboard binocular camera. By analyzing the left and right eye images acquired at the same time, it obtains the depth image at that moment, thereby obtaining the depth images of the robot at multiple locations in the indoor scene.

[0096] Step S12: Based on the depth images, establish a two-dimensional raster map and determine the location information of the roof and door frames in the indoor scene.

[0097] In this embodiment, the two-dimensional grid map can be drawn from depth images captured by a binocular camera. In one example, a three-dimensional point cloud of an indoor scene can be created from the depth image, and the three-dimensional point cloud can be projected onto a two-dimensional map to form a two-dimensional discrete obstacle map. The two-dimensional discrete obstacle map is then scanned to obtain the two-dimensional grid map. Specific methods for creating a three-dimensional point cloud or a two-dimensional grid map of an indoor scene using depth images can be found in relevant SLAM algorithms, and are not specifically limited in this application.

[0098] Specifically, the positions of roofs and door frames in indoor scenes can be obtained by detecting roof and door frame features in depth images. For example, detecting whether point clouds in a coordinate-like rectangular form and a single rectangular form appear in the depth image can identify the point cloud in the coordinate-like rectangular form as the roof and the point cloud in the single rectangular form as the door frame, thereby determining the position information of the roof and door frame. Alternatively, computer vision technology can be used to detect the roof and door frame regions in the left and / or right eye images, and then the positions of the roof and door frame in the depth image can be obtained through coordinate system transformation.

[0099] Step S13: Based on the location information of the roof and door frame, determine the boundaries of the passable area and the closed area of ​​the two-dimensional grid map to obtain an electronic map of the indoor scene.

[0100] In this embodiment, the passable area refers to a passable door in an indoor scene (the door in this embodiment does not specifically refer to an openable structural door, but can also be a passage without a structural door), and the boundary of the closed area refers to the boundary of the non-passable indoor scene formed by the roofs (e.g., a wall structure).

[0101] In this embodiment of the application, determining the passable and closed areas of the two-dimensional grid map based on the location information of the roof and door frame can be achieved by determining the closed area boundary of the two-dimensional grid map based on the location information of the roof and the passable area in the two-dimensional grid map based on the location information of the door frame after obtaining the two-dimensional grid map.

[0102] As can be seen, the method of this application embodiment can determine the boundaries of the passable and closed areas of the two-dimensional grid map by utilizing the positional information of the roof and door frame while constructing the two-dimensional grid map, thereby improving the accuracy of the boundaries of the two-dimensional grid map.

[0103] Figure 3 This is a second schematic diagram of an electronic map generation method provided in an embodiment of this application, as shown below. Figure 3 As shown, in one specific implementation, step S12 may include the following steps:

[0104] Step S121: For each depth image, determine the device pose, obstacle positions of each obstacle, and first position information of the roof and door frame in that depth image, wherein the device is a binocular camera or a mobile robot equipped with a binocular camera.

[0105] When a stereo camera is mounted on a mobile robot, it can be assumed that the relative pose between the stereo camera and the mobile robot will not change; the transformation relationship between the robot coordinate system and the camera coordinate system can be:

[0106]

[0107] in, Indicates the robot's coordinate system. Indicates the camera coordinate system. The transformation parameters between the binocular camera coordinate system and the robot coordinate system are represented. In one example, these transformation parameters can be represented by a selection matrix and a translation vector. Specifically, they can be obtained through pre-calibration. The calibration process for these transformation parameters is existing technology and is not specifically limited in this application.

[0108] A mobile robot acquires multiple depth images from different positions. Therefore, the pose of the device (binocular camera or mobile robot) differs when acquiring each depth image. In this embodiment, the device pose can be the pose within the indoor scene coordinate system (also referred to as the world coordinate system). Changes in device pose can include rotation and translation. For example, the world coordinate system can be established using the origin of the robot's coordinate system at its initial position as the origin of the world coordinate system. Alternatively, a mobile robot coordinate system Or can be established with the robot's center O, and the position of the mobile robot at the moment of successful binocular camera initialization can be used as the initial position, with the origin O of the robot's coordinate system at that moment serving as the origin of the world coordinate system.

[0109] In this embodiment of the application, the obstacle position of each obstacle in the depth image can be obtained after obstacle detection of the depth image. The obstacle position information represents the point cloud region of the obstacle in the depth image, which can be represented by the coordinates of the obstacle point cloud region in the camera coordinate system. The number of obstacles included in the depth image can be one or more, depending on the actual obstacle distribution in the indoor scene.

[0110] Step S122: Based on the obstacle positions of each obstacle in the depth image, establish a two-dimensional raster map in the depth image.

[0111] For each depth image, a two-dimensional grid map can be built based on that depth image. This two-dimensional grid map can be a two-dimensional grid map in the robot coordinate system or the camera coordinate system.

[0112] Step S13 above may specifically include the following steps:

[0113] Step S131: For each depth image, using the first position information of the door frame and / or roof under the depth image, determine the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image, and obtain the boundary two-dimensional raster map of the depth image.

[0114] The first position information of the door frame and / or roof in the depth image can be the position information of the door frame and / or roof in the robot coordinate system or camera coordinate system in the depth image. The first position information of the roof in the depth image can be used to determine the boundary of the closed area of ​​the two-dimensional grid map in the depth image; the first position information of the door frame in the depth image can be used to determine the boundary of the passable area of ​​the two-dimensional grid map in the depth image. For example, the projected right angle obtained by projecting the coordinate system representing the roof can be determined as the roof, and the projected line segment obtained by projecting the single right angle representing the door frame can be determined as the door frame. It is verified whether the sides of the projected right angles are collinear, and the grid state between the collinear projected right angle sides is determined as the boundary of the closed area. It is also verified whether the projected line segments are collinear, and the occupied state of the grid between two collinear door frames is cleared, thus determining the area as a passable area. For example, as shown... Figure 4 As shown, the grid state between collinear right-angled sides is set as the boundary of a closed region, and the region between collinear line segments is set as a passable region.

[0115] Generally, the image coordinate system of the depth image is the same as the camera coordinate system. However, in some other embodiments, the image coordinate system of the depth image and the camera coordinate system are different. In this case, it is necessary to obtain the transformation relationship between the image coordinate system and the camera coordinate system. For example, for the depth image corresponding to the i-th keyframe, i.e., the i-th depth image, the transformation relationship between the image coordinate system and the camera coordinate system of the i-th depth image can be expressed as follows: The total number of depth images is n; the j-th point in the i-th depth image can be... , where m is the number of points in the i-th depth image.

[0116] The point cloud sets of door frame and roof features in the i-th depth image are respectively represented as follows: and Transform it to the robot coordinate system:

[0117]

[0118]

[0119] Let be the set of point clouds of the gate frame in the i-th depth image in the robot coordinate system. Let be the set of point clouds of the rooftops in the i-th depth image in the robot coordinate system.

[0120] according to and We can obtain the accessible area boundary and closed area boundary of the two-dimensional raster map under the i-th depth image.

[0121] Step S132: Based on the device pose under each depth image, fuse and correct the boundary two-dimensional raster map of each depth image to obtain an electronic map of the indoor scene.

[0122] In one example, based on the device pose in each depth image, the boundary 2D raster maps of each depth image can be transformed to the world coordinate system. Then, the boundary 2D raster maps of each depth image in the world coordinate system are fused and corrected to obtain an electronic map of the indoor scene. For example, for each boundary 2D raster map in the world coordinate system, for the same boundary, the final boundary line can be obtained directly by averaging; for the same boundary, some boundary lines with significant deviations can be removed first, and then the final boundary line can be obtained by averaging, etc.

[0123] By employing the embodiments of this application, the boundaries of passable and closed areas of the two-dimensional grid map are determined by utilizing the positional information of rooftops and door frames while constructing the two-dimensional grid map. This avoids the problem that the binocular vision sensor cannot detect closed boundaries and passable areas in the grid map when its field of view is facing upwards, thereby improving the accuracy of the grid map.

[0124] Figure 5 This is a third schematic diagram of the electronic map generation method provided in the embodiments of this application, such as... Figure 5 As shown, in one specific implementation, step S12 may include the following steps:

[0125] Step S121: For each depth image, determine the device pose, obstacle positions of each obstacle, and first position information of the roof and door frame in that depth image, wherein the device is a binocular camera or a mobile robot equipped with a binocular camera.

[0126] Step S201: Based on the device pose and obstacle position of each obstacle in the depth images, a two-dimensional grid map of the indoor scene is established.

[0127] In this embodiment of the application, unlike step S122, a two-dimensional grid map of the indoor scene can be built based on the obstacle information in each depth image after all depth images are acquired. This two-dimensional grid map can be a map of the indoor scene in the world coordinate system.

[0128] Step S202: Based on the device pose in each depth image and the first position information of the roof and door frame, determine the second position information of the roof and door frame in the indoor scene.

[0129] In one example, based on the device pose (in the world coordinate system) when acquiring the depth image, the first position information of the roof and door frame in the depth image can be transformed into the world coordinate system, thereby obtaining the second position information of the roof and door frame in the world coordinate system.

[0130] Step S13 above may specifically include the following steps:

[0131] Step S203: Based on the second position information of the roof and door frame in the indoor scene, determine the passable area boundary and closed area boundary of the two-dimensional raster map of the indoor scene to obtain the electronic map of the indoor scene.

[0132] In this embodiment of the application, after acquiring all depth images, the second position information of the roof and door frame in each depth image can be determined, wherein the second position information of the roof and door frame can be the pose of the roof and door frame in the world coordinate system.

[0133] By employing the embodiments of this application, after constructing a two-dimensional grid map, the location information of the roof and door frame is used to determine the boundaries of the passable and closed areas of the two-dimensional grid map. This avoids the problem that the binocular vision sensor cannot detect the closed boundaries and passable areas in the grid map when its field of view is facing upwards, thus improving the accuracy of the grid map.

[0134] Figure 6 This is an intentional description of one possible implementation of step S121 in this application, such as... Figure 6 As shown, in one specific implementation, step S121 may include the following steps:

[0135] Step S1211: For each depth image, obtain the robot pose of the mobile robot and the obstacle position of each obstacle in the depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle position is the position of the obstacle in the camera coordinate system of the stereo camera.

[0136] In one example, the position of an obstacle in the camera coordinate system of the stereo camera can be converted to the position of the obstacle in the robot coordinate system, and then to the position of the obstacle in the world coordinate system. The conversion relationship between the camera coordinate system and the robot coordinate system is the same as that described in step S121, and will not be repeated here.

[0137] Step S1212: Detect single right-angle feature regions and coordinate system-like right-angle feature regions in the depth image; if a single right-angle feature region is detected, obtain the position information of the single right-angle feature region in the depth image in the camera coordinate system to obtain the first position information of the door frame in the depth image; if a coordinate system-like right-angle feature region is detected, obtain the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system to obtain the first position information of the roof in the depth image.

[0138] By employing the embodiments of this application, the positional information of door frames and roofs in the depth image is obtained by detecting single right-angle feature regions and coordinate system-like right-angle feature regions in the depth image, ensuring that all door frames and roofs in the indoor scene can be detected. In this way, the two-dimensional grid map can be optimized based on the door frames and roofs, thereby improving the accuracy of the grid map.

[0139] Optionally, in one specific implementation, the following steps are included before step S201:

[0140] Step S200: Obtain the pose of the device in the initialization state.

[0141] In this embodiment of the application, the images captured by the left and right eyes of the binocular camera at the same time during device initialization are aligned in time. The pose of the binocular camera and the robot at that time is obtained. The pose of the binocular camera can be the pose of the binocular camera in the camera coordinate system.

[0142] The process of creating a two-dimensional grid map of an indoor scene can be found in relevant SLAM algorithms. Optionally, in one specific implementation, step S201 above includes the following steps:

[0143] Step 1: Based on the pose of the device in the initial state, calculate the obstacle information of each obstacle to obtain a 3D point cloud map.

[0144] Step 2: Project and scan the 3D point cloud map to obtain a 2D raster map of the indoor scene.

[0145] Optionally, in one specific implementation, the method of this application includes the following steps:

[0146] Step 11: Based on the first position information of the door frame and / or roof in each depth image, take the corner point of the door frame and / or the corner point of the roof in each depth image as the center point and extend it outward by a preset length along the direction of the edge to obtain the combination of the endpoints of each door frame and / or roof.

[0147] Step 12: Project the endpoints of the door frame and / or roof onto the two-dimensional raster map under each depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof under each depth image.

[0148] Step 13: Merge the projected line segments of the door frame and / or the projected right angles of the roof in each depth image to obtain the third map;

[0149] Step 14: In the case of two projected right angles with collinear right angles in the third map, set the grid state between the collinear right angles to the occupied state to obtain the boundary of the roof closed area of ​​the third map.

[0150] Step 15: In the case of two collinear projected line segments in the third map, set the grid state between the two collinear projected line segments as a passable area to obtain the passable area boundary of the third map.

[0151] Optionally, in one specific implementation, the method of this application includes the following steps:

[0152] Step 22: Merge the third map with the two-dimensional raster map of the indoor scene to determine the boundaries of the passable area and the closed area of ​​the two-dimensional raster map of the indoor scene, and obtain the first electronic map of the indoor scene.

[0153] Optionally, in one specific implementation, after step 22 above, the method further includes the following steps:

[0154] Step 23: Based on the first position information of the door frame in each depth image, calculate the first minimum Euclidean distance between the door frame and the mobile robot in each depth image; when the first minimum Euclidean distance is less than a preset threshold, determine the door frame in that depth image as a false detection; and / or,

[0155] Step 24: Calculate the second minimum Euclidean distance between the roof and the mobile robot in each depth image based on the third position information of the roof in each depth image; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

[0156] In this embodiment, the first minimum Euclidean distance between the door frame or roof and the mobile robot can be the minimum Euclidean distance between the pose of the door frame or roof in the robot coordinate system and all poses of the robot in the depth image, or the minimum Euclidean distance between the position of the door frame or roof in the world coordinate system and all positions of the robot in the depth image. The preset threshold can be the radius of the robot or can be set according to the actual situation. For example, if the set of coordinate points of a certain door frame or roof in the world coordinate system is Pij, and the set of all coordinate points of the robot after acquiring the indoor scene is Qxy, and the robot radius is 10cm, the Euclidean distance between each coordinate point in the set of all coordinate points of the robot in the set of all coordinate points ...

[0157] Figure 7 This is a schematic diagram of a possible implementation of step S131 in this application, as shown below. Figure 7 As shown, optionally, in one specific implementation, step S131 may include the following steps:

[0158] Step S1311: For each depth image, the first position information of the door frame and / or roof in the depth image is transformed from the camera coordinate system to the robot coordinate system to obtain the third position information of the door frame and / or roof in the depth image.

[0159] In this embodiment, the transformation relationship between the camera coordinate system and the robot coordinate system is the same as that described in step S121, and will not be repeated here.

[0160] Step S1312: Based on the third position information of the door frame and / or roof in the depth image, determine the passable area boundary and / or roof closed area boundary of the two-dimensional grid map in the depth image, wherein the two-dimensional grid map in the depth image is a map in the robot coordinate system.

[0161] In this embodiment, the position information of the door frame and roof in the camera coordinate system is transformed into the world coordinate system. This allows the feature information of the door frame and roof in the world coordinate system to be directly projected when optimizing the two-dimensional grid map, so as to obtain the passable area and closed area boundary of the two-dimensional grid map. This enables the two-dimensional grid map to be optimized intuitively based on the position information of the door frame and roof.

[0162] Figure 8 This is a schematic diagram of a possible implementation of step S1312 in this application, as shown below. Figure 8 As shown, in one specific implementation, step S1312 may include the following steps:

[0163] Step S13121: Based on the third position information of the door frame and / or roof in the depth image, extend the corner point of the door frame and / or roof in the depth image as the center point outward by a preset length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or roof.

[0164] For example, in an embodiment of this application, the endpoint combination can be obtained by extending 10cm outward from the center point representing the corner of the door frame and / or the corner of the roof.

[0165] Step S13122: Project the combined endpoints of the door frame and / or roof onto the two-dimensional raster map under the depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof.

[0166] Step S13123: In the case that there are two projected right angles with collinear right angles in the two-dimensional grid map of the depth image, the grid state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional grid map under the depth image is obtained.

[0167] Step S13124: In the case that there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thereby obtaining the passable area boundary of the two-dimensional raster map under the depth image.

[0168] In this embodiment of the application, by projecting the endpoints representing the door frame and the roof together, and when there are right angles with collinear right angles, the collinear right angles are determined to be in an occupied state, and the grid state between two collinear projected line segments is determined to be a passable area. In this way, the initial closed area and passable area of ​​the two-dimensional grid map can be obtained quickly.

[0169] Optionally, in one specific implementation, after step S1312 above, the method further includes the following steps:

[0170] Step S1313: Based on the third position information of the door frame in the depth image, calculate the first minimum Euclidean distance between the door frame and the mobile robot in the depth image; when the first minimum Euclidean distance is less than a preset threshold, determine the door frame in the depth image as a false detection; and / or,

[0171] Step S1314: Based on the third position information of the roof in the depth image, calculate the second minimum Euclidean distance between the roof and the mobile robot in the depth image; when the second minimum Euclidean distance is less than a preset threshold, determine the roof in the depth image as a false detection.

[0172] In this embodiment, the first minimum Euclidean distance between the door frame or roof and the mobile robot can be the minimum Euclidean distance between the pose of the door frame or roof in the robot coordinate system and the pose of the robot in the depth image, or it can be the minimum Euclidean distance between the position of the door frame or roof in the world coordinate system and the position of the robot in the depth image. The preset threshold can be an empirical value set according to actual conditions. The principle for setting the preset threshold is: when the Euclidean distance between the door frame / roof and the mobile robot is less than the preset threshold, it is considered a false detection; for example, the preset threshold can be the radius of the robot. In one example, the set of coordinate points of the door frame or roof in the robot coordinate system in the depth image is Pij, the coordinate point of the robot in the depth image is Q, and the robot radius is 10cm. The Euclidean distances between the robot's coordinate point Q and each coordinate point in the set of coordinate points Pij are calculated respectively. When the minimum Euclidean distance is less than 10cm, the detected door frame or roof is considered a false detection.

[0173] In this embodiment of the application, the accuracy of the grid map is further improved by comparing the minimum Euclidean distance between the door frame and the mobile robot with a threshold, and by comparing the minimum Euclidean distance between the roof and the mobile robot, and by determining whether false detections have occurred in the initial closed area and passable area of ​​the two-dimensional grid map based on the comparison results.

[0174] To illustrate the methods of the embodiments of this application, the following description is provided in conjunction with specific examples. (See attached examples.) Figure 9 , Figure 9The flowchart of the electronic map generation method is as follows: First, it is determined whether the left and right eye images of the binocular camera are synchronized (if they are not synchronized, they need to be adjusted to be synchronized in terms of timing). After synchronization, the left and right eye images of the binocular camera are acquired and matched to obtain the camera's initial pose. Then, keyframe data is continuously created (keyframe data includes depth images, and may also include at least one of the left eye image, right eye image, and composite left and right eye images). On the one hand, the keyframe data is detected to obtain the positions of door frames and roofs, and the passable and closed area boundaries of the grid map are determined based on the positions of door frames and roofs in each keyframe. On the other hand, the camera pose and map point positions are estimated based on each keyframe data, and the map points are projected to construct an obstacle map. Finally, the boundary recognition of the obstacle map is optimized using the passable and closed area boundaries of the grid map to obtain the optimized electronic map.

[0175] Corresponding to the above method embodiments, this application also provides an electronic map generation device, such as... Figure 10 As shown, the device may include the following modules:

[0176] Image acquisition module 1001 is used to acquire multiple depth images of an indoor scene captured by a binocular camera;

[0177] The map creation module 1002 is used to create a two-dimensional raster map based on images at various depths and to determine the location information of roofs and door frames in indoor scenes;

[0178] The boundary determination module 1003 is used to determine the boundaries of the passable area and the closed area of ​​the two-dimensional raster map based on the location information of the roof and door frame, so as to obtain an electronic map of the indoor scene.

[0179] As can be seen, the method of this application embodiment utilizes the positional information of rooftops and door frames to determine the boundaries of passable and closed areas of the two-dimensional grid map while constructing the two-dimensional grid map. This avoids the problem that the binocular vision sensor cannot detect closed boundaries and passable areas in the grid map when its field of view is facing upwards, thereby improving the accuracy of the grid map.

[0180] Optionally, in one specific implementation, the map building module 1002 includes:

[0181] The location information confirmation submodule is used to determine the device pose, obstacle position of each obstacle, and first position information of the roof and door frame for each depth image. The device is a binocular camera or a mobile robot equipped with a binocular camera.

[0182] The first raster map generation submodule is used to create a two-dimensional raster map of the depth image based on the obstacle positions of each obstacle in the depth image.

[0183] The boundary determination module 1003 includes:

[0184] The first boundary determination submodule is used to determine the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image for each depth image by using the first position information of the door frame and / or roof under the depth image, so as to obtain the boundary two-dimensional raster map of the depth image.

[0185] The fusion correction submodule is used to fuse and correct the boundary two-dimensional raster maps of each depth image based on the device pose under each depth image to obtain an electronic map of the indoor scene.

[0186] Optionally, in one specific implementation, the map building module 1002 includes:

[0187] The location information confirmation submodule is used to determine the device pose, obstacle position of each obstacle, and first position information of the roof and door frame for each depth image. The device is a binocular camera or a mobile robot equipped with a binocular camera.

[0188] The second raster map generation submodule is used to create a two-dimensional raster map of the indoor scene based on the device pose and obstacle position of each obstacle in the image at each depth.

[0189] The second position information confirmation submodule is used to determine the second position information of the roof and the door frame in the indoor scene based on the device pose under each depth image and the first position information of the roof and the door frame.

[0190] The boundary determination module 1003 includes:

[0191] The second boundary determination submodule is used to determine the passable area boundary and closed area boundary of the two-dimensional raster map of the indoor scene based on the second position information of the roof and door frame in the indoor scene, so as to obtain an electronic map of the indoor scene.

[0192] Optionally, in one specific implementation, the location information confirmation submodule is specifically used for:

[0193] For each depth image, the robot pose of the mobile robot and the obstacle positions of each obstacle are obtained in that depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle positions are the positions of the obstacles in the camera coordinate system of the binocular camera.

[0194] The depth image is used to detect single right-angle feature regions and coordinate system-like right-angle feature regions. When a single right-angle feature region is detected, the position information of the single right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the door frame in the depth image. When a coordinate system-like right-angle feature region is detected, the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the roof in the depth image.

[0195] Optionally, in one specific implementation, the first boundary determination submodule includes:

[0196] The coordinate transformation unit is used to transform the first position information of the door frame and / or roof in the depth image from the camera coordinate system to the robot coordinate system for each depth image, so as to obtain the third position information of the door frame and / or roof in the depth image.

[0197] The boundary determination unit is used to determine the passable area boundary and / or roof closed area boundary of the two-dimensional grid map under the depth image based on the third position information of the door frame and / or roof under the depth image, wherein the two-dimensional grid map under the depth image is a map in the robot coordinate system.

[0198] Optionally, in one specific implementation, the boundary determination unit is specifically used for:

[0199] Based on the third position information of the door frame and / or roof in the depth image, the corner points of the door frame and / or roof in the depth image are used as the center points and extended outward by a predetermined length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or the roof.

[0200] Project the combined endpoints of the door frame and / or roof onto a two-dimensional raster map under this depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof.

[0201] In the case of two projected right angles with collinear right angles in the two-dimensional raster map of the depth image, the raster state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional raster map under the depth image is obtained.

[0202] In the case where there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thus obtaining the passable area boundary of the two-dimensional raster map under the depth image.

[0203] Optionally, in one specific implementation, the apparatus further includes:

[0204] The Euclidean distance calculation module is used to calculate the first minimum Euclidean distance between the door frame and the mobile robot in the depth image based on the third position information of the door frame in the depth image; when the first minimum Euclidean distance is less than a preset threshold, the door frame in the depth image is determined to be a false detection; and / or,

[0205] Based on the third location information of the roof in the depth image, the second minimum Euclidean distance between the roof and the mobile robot in the depth image is calculated; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

[0206] In another embodiment of the present invention, a robot is also provided, which is equipped with a binocular camera and is used to implement any of the electronic map generation methods described in the above embodiments during operation.

[0207] This application also provides a mobile robot, including:

[0208] The device includes a binocular camera, a memory, and a processor. The binocular camera is used to acquire left and right eye images of an indoor scene and generate a depth image of the indoor scene based on the left and right eye images.

[0209] The memory is used to store computer programs;

[0210] When the processor executes the program stored in the memory, it implements any of the electronic map generation methods described in this application.

[0211] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0212] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0213] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described electronic map generation methods.

[0214] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the electronic map generation methods described above.

[0215] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0216] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0217] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of devices, mobile robots, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0218] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for generating electronic maps, characterized in that, include: Acquire multiple depth images of an indoor scene captured by a stereo camera; Based on the depth images, a two-dimensional grid map is constructed and the location information of the roof and door frame in the indoor scene is determined. Based on the location information of the roof and the door frame, the accessible area boundary and the closed area boundary of the two-dimensional grid map are determined to obtain an electronic map of the indoor scene. The step of establishing a two-dimensional grid map based on each depth image and determining the location information of the roof and door frames in the indoor scene includes: For each depth image, determine the device pose, obstacle positions of each obstacle, and first position information of the roof and door frame in that depth image. The device is the binocular camera or a mobile robot equipped with the binocular camera, and the first position information of the roof and door frame is the position information of the roof and door frame in the camera coordinate system. Based on the obstacle positions of each obstacle in the depth image, a two-dimensional raster map is constructed in the depth image. The step of determining the passable area boundary and closed area boundary of the two-dimensional grid map based on the location information of the door frame and the roof to obtain an electronic map of the indoor scene includes: For each depth image, the first position information of the door frame and / or roof under the depth image is transformed from the camera coordinate system to the robot coordinate system to obtain the third position information of the door frame and / or roof under the depth image; Based on the third position information of the door frame and / or roof in the depth image, the corner points of the door frame and / or roof in the depth image are used as the center points and extended outward by a predetermined length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or the roof. The endpoints of the door frame and / or the roof are combined and projected onto a two-dimensional raster map under the depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof; In the case of two projected right angles with collinear right angles in the two-dimensional raster map of the depth image, the raster state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional raster map under the depth image is obtained. In the case where there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thus obtaining the passable area boundary of the two-dimensional raster map under the depth image. Based on the device pose under each depth image, the boundary two-dimensional raster maps of each depth image are fused and corrected to obtain an electronic map of the indoor scene.

2. The method according to claim 1, characterized in that, For each depth image, determining the device pose, obstacle positions, and first position information of the roof and door frame in that depth image includes: For each depth image, the robot pose of the mobile robot and the obstacle positions of each obstacle are obtained in that depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle positions are the positions of the obstacles in the camera coordinate system of the binocular camera. The depth image is used to detect single right-angle feature regions and coordinate system-like right-angle feature regions. When a single right-angle feature region is detected, the position information of the single right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the door frame in the depth image. When a coordinate system-like right-angle feature region is detected, the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the roof in the depth image.

3. The method according to claim 1, characterized in that, The method further includes: Based on the third position information of the door frame in the depth image, calculate the first minimum Euclidean distance between the door frame and the mobile robot in the depth image; when the first minimum Euclidean distance is less than a preset threshold, determine the door frame in the depth image as a false detection; and / or, Based on the third location information of the roof in the depth image, the second minimum Euclidean distance between the roof and the mobile robot in the depth image is calculated; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

4. An electronic map generating device, characterized in that, The device includes: The image acquisition module is used to acquire multiple depth images of an indoor scene captured by a stereo camera; The map building module is used to build a two-dimensional raster map based on the depth images and to determine the location information of the roof and door frame in the indoor scene; The boundary determination module is used to determine the passable area boundary and the closed area boundary of the two-dimensional grid map based on the position information of the roof and the door frame, so as to obtain an electronic map of the indoor scene. The map building module includes: The location information confirmation submodule is used to determine the device pose, obstacle position of each obstacle, and first position information of the roof and door frame for each depth image. The device is the binocular camera or a mobile robot equipped with the binocular camera. The first raster map generation submodule is used to build a two-dimensional raster map under the depth image based on the obstacle positions of each obstacle under the depth image; The boundary determination module includes: The first boundary determination submodule is used to determine the passable area boundary and / or closed area boundary of the two-dimensional raster map under the depth image for each depth image by using the first position information of the door frame and / or roof under the depth image, so as to obtain the boundary two-dimensional raster map of the depth image. The fusion correction submodule is used to fuse and correct the boundary two-dimensional grid maps of each depth image based on the device pose under each depth image to obtain an electronic map of the indoor scene. The first boundary determination submodule includes: The coordinate transformation unit is used to transform the first position information of the door frame and / or roof in the depth image from the camera coordinate system to the robot coordinate system for each depth image, so as to obtain the third position information of the door frame and / or roof in the depth image. The boundary determination unit is used to determine the passable area boundary and / or roof closed area boundary of the two-dimensional grid map under the depth image based on the third position information of the door frame and / or roof under the depth image, wherein the two-dimensional grid map under the depth image is a map under the robot coordinate system. The boundary determination unit is specifically used for: Based on the third position information of the door frame and / or roof in the depth image, the corner points of the door frame and / or roof in the depth image are used as the center points and extended outward by a predetermined length along the direction of the edge to obtain the combination of the endpoints of the door frame and / or the roof. The endpoints of the door frame and / or the roof are combined and projected onto a two-dimensional raster map under the depth image to obtain the projected line segments of the door frame and / or the projected right angles of the roof; In the case of two projected right angles with collinear right angles in the two-dimensional raster map of the depth image, the raster state between the collinear right angles is set to the occupied state, and the roof closed area boundary of the two-dimensional raster map under the depth image is obtained. In the case where there are two collinear projection line segments in the two-dimensional raster map of the depth image, the raster state between the two collinear projection line segments is set as a passable area, thus obtaining the passable area boundary of the two-dimensional raster map under the depth image.

5. The apparatus according to claim 4, characterized in that, The location information confirmation submodule is specifically used for: For each depth image, the robot pose of the mobile robot and the obstacle positions of each obstacle are obtained in that depth image. The robot pose is the pose of the mobile robot in the world coordinate system of the indoor scene, and the obstacle positions are the positions of the obstacles in the camera coordinate system of the binocular camera. The depth image is used to detect single right-angle feature regions and coordinate system-like right-angle feature regions. When a single right-angle feature region is detected, the position information of the single right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the door frame in the depth image. When a coordinate system-like right-angle feature region is detected, the position information of the coordinate system-like right-angle feature region in the depth image in the camera coordinate system is obtained to obtain the first position information of the roof in the depth image. The device further includes: The Euclidean distance calculation module is used to calculate a first minimum Euclidean distance between the door frame and the mobile robot in the depth image based on the third position information of the door frame in the depth image; when the first minimum Euclidean distance is less than a preset threshold, the door frame in the depth image is determined to be a false detection; and / or, Based on the third location information of the roof in the depth image, the second minimum Euclidean distance between the roof and the mobile robot in the depth image is calculated; when the second minimum Euclidean distance is less than a preset threshold, the roof in the depth image is determined to be a false detection.

6. A mobile robot, characterized in that, The mobile robot includes: a binocular camera, a memory, and a processor. The binocular camera is used to acquire left and right eye images of an indoor scene and generate a depth image of the indoor scene based on the left and right eye images. The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the electronic map generation method according to any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the electronic map generation method according to any one of claims 1-3.

Images