A simultaneous localization and mapping method for establishing a drivable area in a map

Abstract

The application provides a method for establishing a drivable area in a map, which comprises the following steps: obtaining an image sequence to be processed, and sequentially detecting drivable areas of images in the image sequence; after extracting the drivable area of a current image, determining a corresponding three-dimensional approximate drivable area of the current image in a constructed point cloud map based on the drivable area of the current image; if it is determined that the current image is a key frame based on a preset selection rule, then fusing the corresponding three-dimensional approximate drivable area of the current image with a local fusion result corresponding to the previous image in the point cloud map to obtain a local fusion result corresponding to the current image in the point cloud map; and fusing the local fusion result corresponding to the current image in the point cloud map with a global three-dimensional drivable area already established in the point cloud map to obtain an updated global three-dimensional drivable area. The application can adapt to various indoor and outdoor scenes and has a low cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simultaneous localization and mapping, and particularly relates to a method for establishing drivable areas in a map in simultaneous localization and mapping. BACKGROUND

[0002] Simultaneous localization and mapping (SLAM) technology solves the problem of simultaneous localization and map establishment of robots in unknown environments, and is widely used in the fields of VR, unmanned driving, robots, etc. The map generated by SLAM is often used to guide the autonomous movement of intelligent agents such as robots. According to the different sensors used, SLAM can be divided into two categories: vision-based and laser radar-based methods. Vision-based SLAM methods have become the focus of research due to the maturity of camera technology, lower prices, longer life, and rich information acquisition capabilities. However, the map established by traditional vision SLAM algorithms is a sparse point cloud map, which is beneficial to the establishment of large-scale scene maps and can be used for localization. However, due to the lack of drivable area information, it cannot effectively guide the movement of robots and provide path planning assistance for robots. In order to effectively guide intelligent agents such as robots to automatically plan routes and avoid obstacles and ultimately reach the designated destination, it is of great significance to establish drivable areas in sparse point cloud maps.

[0003] Vision semantic SLAM systems perform semantic segmentation on image frames through deep learning neural networks and map semantic information to maps, and classify point clouds in maps by semantics. Laser radar SLAM detects drivable areas through road extraction and plane hypothesis, and divides point clouds into road surfaces and non-road surfaces. Fusing vision and laser radar data for drivable area detection improves the accuracy of detection results. Laser radar-based SLAM systems generally use grid maps to mark drivable areas by dividing the map space into grids and marking grid occupancy probabilities to generate drivable and non-drivable areas. Some methods are through image segmentation, then based on the assumption that the front of the robot is the ground, select sparse three-dimensional map points in the front ground segmentation to fit the ground plane mathematical model, then filter other ground segmentation results through the mathematical model, finally combine the camera pose obtained by SLAM and the ground mathematical model to project the correct ground segmentation image into the SLAM map, and realize dense three-dimensional reconstruction of the ground. However, laser radar-based SLAM methods are expensive.

[0004] Semantic visual SLAM system currently mainly uses dense point cloud map, which results in that it is only applicable to small-scale scenes such as indoor. The grid map can be converted from the point cloud map, but when the sparse point cloud map is converted into the grid map, due to the sparse characteristics of the sparse point cloud map, many unknown areas are generated, causing the drivable area to be discrete and discontinuous. Although some methods realize the dense three-dimensional reconstruction of the indoor ground, due to the assumption that the ground is flat, the method is only suitable for small-scale scenes such as indoor, and cannot adapt to the ground undulation in outdoor large-scale scenes.

[0005] Therefore, the prior art has defects and needs to be improved and developed. SUMMARY

[0006] The technical problem solved by the present application is to provide a method for establishing a drivable area in a map, aiming at solving the problem that the SLAM method based on a laser radar has a high cost when a drivable area is established in a sparse point cloud map, and the visual semantic SLAM system is not applicable to outdoor large-scale scenes.

[0007] The technical solution adopted by the present application to solve the technical problem is as follows:

[0008] A method for establishing a drivable area in a map, wherein the method comprises:

[0009] obtaining an image sequence to be processed, and detecting drivable areas of images in the image sequence in sequence;

[0010] After extracting the drivable area of the current image, determining a three-dimensional approximate drivable area corresponding to the current image in the constructed point cloud map based on the drivable area of the current image;

[0011] If the current image is determined to be a key frame based on a preset selection rule, then the three-dimensional approximate drivable area corresponding to the current image is fused with a local fusion result corresponding to the last frame image in the point cloud map, to obtain a local fusion result corresponding to the current image in the point cloud map;

[0012] The local fusion result corresponding to the current image in the point cloud map is fused with the global three-dimensional drivable area already established in the point cloud map, to obtain an updated global three-dimensional drivable area.

[0013] In an embodiment of the present application, after extracting the drivable area of the current image, determining a three-dimensional approximate drivable area corresponding to the current image in the constructed point cloud map based on the drivable area of the current image, comprises:

[0014] extracting a drivable region boundary of the current image after extracting the drivable region of the current image, to obtain the drivable region boundary of the current image;

[0015] matching feature points between the drivable region boundary of the current image and the constructed point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image;

[0016] connecting the plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image into a closed polygon to form a corresponding three-dimensional approximate drivable region of the current image in the point cloud map.

[0017] In an embodiment of the present application, the drivable region is represented in the form of an image mask; after extracting the drivable region of the current image, the drivable region boundary of the current image is extracted, comprising:

[0018] extracting the image mask of the drivable region of the current image after extracting the drivable region of the current image to obtain the drivable region boundary.

[0019] In an embodiment of the present application, matching feature points between the drivable region boundary of the current image and the constructed point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image, comprises:

[0020] matching feature points within a predetermined radius range of the drivable region boundary of the current image with the point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image.

[0021] In an embodiment of the present application, the preset selection rule is to select the current image as a key frame every predetermined number of image frames; or, if the number of feature point matching points between the current image and the point cloud map is less than a preset threshold, the current image is selected as a key frame.

[0022] In an embodiment of the present application, after extracting the drivable region of the current image, determining the corresponding three-dimensional approximate drivable region of the current image in the constructed point cloud map based on the drivable region of the current image, further comprises:

[0023] if the current image is a next frame image of the key frame, the corresponding three-dimensional approximate drivable region of the current image in the point cloud map is taken as a local fusion result corresponding to the current image in the point cloud map.

[0024] In an embodiment of the present application, after extracting the drivable region of the current image, the method further comprises:

[0025] If the current image is not a key frame, the three-dimensional approximate drivable region corresponding to the current image is fused with the local fusion result corresponding to the previous frame image in the point cloud map to obtain the local fusion result corresponding to the current image in the point cloud map, and the drivable region of the next frame image is extracted.

[0026] The present application also provides a device for establishing a drivable region in a map, wherein the device comprises:

[0027] a region extraction module, configured to acquire an image sequence to be processed, and detect drivable regions of images in the image sequence in sequence;

[0028] a region determination module, configured to, after extracting the drivable region of the current image, determine a three-dimensional approximate drivable region corresponding to the current image in the point cloud map based on the drivable region of the current image;

[0029] a local fusion module, configured to fuse the three-dimensional approximate drivable region corresponding to the current image with the local fusion result corresponding to the previous frame image in the point cloud map to obtain the local fusion result corresponding to the current image in the point cloud map;

[0030] a global fusion module, configured to, if the current image is determined to be a key frame based on a preset selection rule, fuse the local fusion result corresponding to the current image in the point cloud map with the global three-dimensional drivable region established in the point cloud map to obtain an updated global three-dimensional drivable region.

[0031] The present application also provides a terminal, comprising a memory, a processor, and a simultaneous localization and mapping program for establishing a drivable region in a map stored in the memory and executable on the processor, wherein the simultaneous localization and mapping program for establishing a drivable region in a map, when executed by the processor, implements the steps of the simultaneous localization and mapping method for establishing a drivable region in a map.

[0032] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the simultaneous localization and mapping method for establishing a drivable region in a map.

[0033] This invention provides a synchronous localization and mapping method for establishing drivable areas in a map. The method includes: acquiring an image sequence to be processed; sequentially detecting drivable areas in the images of the image sequence; after extracting drivable areas from the current image, determining the corresponding three-dimensional approximate drivable area in a constructed point cloud map based on the drivable area of ​​the current image; if the current image is determined to be a keyframe based on a preset selection rule, fusing the corresponding three-dimensional approximate drivable area with the local fusion result of the previous image in the point cloud map to obtain the corresponding local fusion result of the current image in the point cloud map; and fusing the corresponding local fusion result of the current image with the established global three-dimensional drivable area in the point cloud map to obtain an updated global three-dimensional drivable area. This invention extracts drivable areas from images in an image sequence, determines three-dimensional approximate drivable areas based on these drivable areas, and then performs local and global fusion to ultimately establish continuous drivable areas in a point cloud map. This method is adaptable to various indoor and outdoor scenarios and has low cost. Attached Figure Description

[0034] Figure 1 This is a flowchart of a preferred embodiment of the synchronous positioning and mapping method for establishing drivable areas in a map in this invention;

[0035] Figure 2 It is the image mask of the drivable area in this invention;

[0036] Figure 3 This refers to the drivable region boundary extracted based on image mask in this invention;

[0037] Figure 4 This is the local fusion result of the current image's approximately drivable three-dimensional region and the previous frame image in this invention;

[0038] Figure 5 This is a schematic diagram illustrating the fusion of the local fusion result of the current image in the point cloud map with the globally three-dimensional drivable area already established in the point cloud map in this invention.

[0039] Figure 6 This is a flowchart of the synchronous positioning and mapping method for establishing drivable areas in a map in this invention;

[0040] Figure 7 This is a functional principle block diagram of a preferred embodiment of the synchronous positioning and mapping device for establishing drivable areas on a map in this invention;

[0041] Figure 8 This is a functional principle block diagram of a preferred embodiment of the terminal in this invention. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are merely used to explain the present application and should not be used to limit the present application.

[0043] The present application only relies on a visual sensor to achieve the effect of establishing a drivable area in a sparse point cloud map without prior assumptions, and does not need to add additional map points in the map, which is conducive to the storage, fast loading and maintenance of the map. The method of the present application can not only be used in small-scale scenes such as indoors, but also be applicable to large-scale scenes with outdoor ground undulations. Intelligent agents such as robots can plan paths and avoid obstacles in advance according to the established drivable area in the map, thereby laying a good foundation for successfully reaching the destination.

[0044] Please refer to Figure 1 , Figure 1 is a flowchart of the simultaneous localization and mapping method for establishing a drivable area in a map in the present application. As shown in Figure 1 , the simultaneous localization and mapping method for establishing a drivable area in a map according to the present application embodiment comprises:

[0045] Step S100, acquiring an image sequence to be processed, and detecting drivable areas of images in the image sequence in sequence.

[0046] Specifically, the conventional visual SLAM system extracts feature points from the input video sequence of the camera, and performs feature point matching to establish and optimize the point cloud map while optimizing the camera pose. However, the point cloud map does not contain drivable area information, and even if it is converted into a grid map, due to the sparse nature of the point cloud map, discrete and discontinuous drivable areas and a large number of unknown areas will be generated during the conversion. The present application embodiment extracts drivable areas from images in the image sequence, and then establishes continuous drivable areas in the point cloud map.

[0047] As shown in Figure 1 , the simultaneous localization and mapping method for establishing a drivable area in a map according to the present application embodiment further comprises:

[0048] Step S200, after extracting the drivable area of the current image, determining the corresponding three-dimensional approximate drivable area of the current image in the constructed point cloud map based on the drivable area of the current image.

[0049] Specifically, step S200 completes the conversion of the two-dimensional drivable area to the three-dimensional drivable area.

[0050] In the present application embodiment, the step S200 specifically comprises:

[0051] Step S210, after extracting the drivable region of the current image, performing boundary extraction on the drivable region of the current image to obtain the drivable region boundary of the current image;

[0052] Step S220, performing feature point matching between the drivable region boundary of the current image and the constructed point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image;

[0053] Step S230, connecting the plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable region boundary of the current image into a closed polygon to form a three-dimensional approximate drivable region corresponding to the current image in the point cloud map.

[0054] Specifically, the embodiment of the present application detects the drivable region according to the texture, road edge, vanishing point and the like, and can also use a deep learning method to obtain the drivable region, and identifies the drivable region boundary of the current image. Feature points in the current frame that are successfully matched with three-dimensional map points in the point cloud map are searched along the boundary, and the three-dimensional map points corresponding to the searched feature points in the current frame are connected into a closed polygon to form a three-dimensional approximate drivable region.

[0055] That is, in the SLAM system, feature points are detected for each frame of image, and matched with three-dimensional map points in the map. The three-dimensional map points corresponding to the searched feature points in the current frame are connected into a closed polygon, a three-dimensional polygon that is most similar to the drivable region can be generated, and the conversion of the two-dimensional drivable region of the current frame to the three-dimensional drivable region is completed. The three-dimensional approximate drivable region established by the method does not have a ground plane assumption, can adapt to complex situations such as ground undulation, and does not increase any new map points, which is beneficial to the storage and rapid loading of the map.

[0056] The embodiment of the present application realizes the conversion of the two-dimensional drivable region to the three-dimensional drivable region through the steps of drivable region boundary extraction and feature point matching, and further provides a basis for establishing a continuous drivable region in the point cloud map.

[0057] In an embodiment of the present application, the drivable region is represented in the form of an image mask; and the step S210 specifically comprises: after extracting the drivable region of the current image, performing boundary extraction on the image mask of the drivable region of the current image to obtain the drivable region boundary.

[0058] Specifically, the drivable region obtained by the embodiment of the present application is represented in the form of an image mask, as shown in FIG. 2B. Figure 2 As shown in FIG. 2C, boundary extraction is performed on the image mask of the drivable region, and the white part represents the boundary. Figure 3

[0059] ​The drivable area of the embodiment of the present application is represented in an image mask manner, and the mask can effectively shield the non-target area in the image, and further accurately identify the drivable area in the image.

[0060] In an embodiment of the present application, the step S220 is specifically: performing feature point matching on the feature points in the predetermined radius range of the drivable area boundary of the current image and the point cloud map, to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable area boundary of the current image.

[0061] The embodiment of the present application increases the feature point matching probability and accuracy by searching for the nearest feature point that is successfully matched with the three-dimensional map point in the point cloud map along the drivable area boundary.

[0062] As shown in Figure 1 The method for establishing a drivable area in a map according to the embodiment of the present application further includes:

[0063] In step S300, if it is determined that the current image is a key frame based on a preset selection rule, the three-dimensional approximate drivable area corresponding to the current image is fused with the local fusion result corresponding to the previous frame image in the point cloud map, to obtain the local fusion result corresponding to the current image in the point cloud map.

[0064] Specifically, the three-dimensional approximate drivable area of the current image is fused with the local fusion result of the previous frame, and saved as the local fusion result of the current image, as shown in Figure 4 .

[0065] In the embodiment of the present application, the preset selection rule is to select the current image as a key frame every predetermined number of image frames; or, if the number of feature point matching points between the current image and the point cloud map is less than a preset threshold, the current image is selected as a key frame. That is, the embodiment of the present application is to perform local fusion first, and therefore, a key frame is selected as an end point of one local fusion. When the current frame is selected as a key frame by the SLAM algorithm, the union of the three-dimensional approximate drivable area polygon after fusion of the current frame and the drivable area polygon in the map forms a global three-dimensional approximate drivable area polygon.

[0066] As shown in Figure 1 The method for establishing a drivable area in a map according to the embodiment of the present application further includes:

[0067] In step S400, the local fusion result corresponding to the current image in the point cloud map is fused with the global three-dimensional drivable area that has been established in the point cloud map, to obtain an updated global three-dimensional drivable area.

[0068] Specifically, in order to form a complete drivable area in the map, the three-dimensional approximate drivable area polygon after fusion of the current key frame needs to be combined with the existing three-dimensional drivable area in the point cloud map to achieve global fusion of the drivable area, and the process is as shown in Figure 5 Figure 5 The points in the figure are the point clouds in the point cloud map, the larger area is the existing drivable area, and the smaller area is the local fusion result corresponding to the current key frame of the figure.

[0069] The method for establishing a drivable area in a map provided in the embodiments of the present application is integrated in the SLAM algorithm framework, and the overall flowchart is as shown in Figure 6 The traditional SLAM algorithm includes three modules of tracking, local optimization and global optimization. The above five steps for establishing a drivable area in a sparse point map are integrated into the three modules. Among them, the four steps of drivable area segmentation, drivable area boundary extraction, three-dimensional approximate drivable area generation and multi-frame fusion are integrated in the tracking module, and each frame of image is processed. When the frame of image is selected as a key frame, the cumulative fusion three-dimensional approximate drivable area polygon stored by the frame of image is combined with the drivable area polygon in the point cloud map in the global optimization module to obtain a global drivable area fusion result. That is, the step of drivable area global fusion is integrated in the global optimization module of the SLAM algorithm, and the establishment of the drivable area in the sparse point cloud map is finally completed.

[0070] The embodiments of the present application realize the establishment of a continuous drivable area in a sparse point cloud map of a visual SLAM system, and simultaneously adapt to various indoor and outdoor scenes.

[0071] In the embodiments of the present application, the step S200 further includes: if the current image is a next frame of image of the key frame, taking the three-dimensional approximate drivable area corresponding to the current image in the point cloud map as the local fusion result corresponding to the current image in the point cloud map.

[0072] That is, after the calculation of the key frame is completed, the next frame of image will be the first frame of local fusion, and the multi-frame fusion will be completed in turn until the next key frame appears.

[0073] The embodiments of the present application save the local fusion result by first performing local fusion, which is beneficial to the storage of the drivable area.

[0074] In the embodiments of the present application, the step S200 further includes: if the current image is not a key frame, fusing the three-dimensional approximate drivable area corresponding to the current image with the local fusion result corresponding to the last frame of image in the point cloud map to obtain the local fusion result corresponding to the current image in the point cloud map, and continuing to extract the drivable area of the next frame of image. ​

[0075] That is, if the current image is not a key frame, global fusion is not needed, and local fusion is continued.

[0076] The present application achieves the following effects:

[0077] First, compared with the dense point cloud generated by semantic visual SLAM, the method of the present application can establish drivable regions while inheriting the advantage of large-scale mapping of sparse point cloud maps.

[0078] Second, compared with a laser radar SLAM system, the method of the present application only relies on a visual sensor, and is low in cost and simple to maintain.

[0079] Third, the present application has no prior assumptions and can be applied to various environments such as indoors and outdoors.

[0080] Fourth, the method of the present application does not need to add additional map points in the map, which is conducive to the storage, rapid loading and maintenance of the map.

[0081] In one embodiment, as shown in Figure 7 Based on the above method of establishing a drivable region in a map for simultaneous localization and mapping, the present application also correspondingly provides a device for establishing a drivable region in a map for simultaneous localization and mapping, which comprises:

[0082] A region extraction module 100 is configured to acquire an image sequence to be processed, and detect drivable regions of images in the image sequence in sequence.

[0083] A region determination module 200 is configured to, after extracting a drivable region of a current image, determine a corresponding three-dimensional approximate drivable region of the current image in a constructed point cloud map based on the drivable region of the current image.

[0084] A local fusion module 300 is configured to, if the current image is determined to be a key frame based on a preset selection rule, fuse the corresponding three-dimensional approximate drivable region of the current image with a local fusion result corresponding to the previous frame image in the point cloud map, to obtain a local fusion result corresponding to the current image in the point cloud map.

[0085] A global fusion module 400 is configured to fuse the local fusion result corresponding to the current image in the point cloud map with a global three-dimensional drivable region already established in the point cloud map, to obtain an updated global three-dimensional drivable region.

[0086] Figure 8 The terminal provided by the embodiment of the present application is shown in a structural schematic diagram. The terminal can include:

[0087] The memory 501, the processor 502 and the computer program stored in the memory 501 and executable on the processor 502.

[0088] The processor 502 implements the simultaneous localization and mapping method for establishing a drivable area in a map provided in the above embodiments when executing the program.

[0089] Further, the terminal further comprises:

[0090] The communication interface 503 is used for communication between the memory 501 and the processor 502.

[0091] The memory 501 is used for storing the computer program executable on the processor 502.

[0092] The memory 501 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0093] If the memory 501, the processor 502 and the communication interface 503 are independently implemented, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, only one line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0094] Optionally, in specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can complete communication between each other through an internal interface.

[0095] The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0096] The embodiment also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method for simultaneously positioning and mapping a drivable area in a map as described above.

[0097] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0098] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0099] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for performing a step of a process described in connection with that code module, segment, or portion of code. The various embodiments of the preferred implementation of the present application can also include additional or fewer steps or processes, in some embodiments, the steps or processes can be performed in the order shown or discussed, including sub-steps, additional steps, or in an opposite order, including in an essentially simultaneous manner or in reverse order, as appropriate, as will be appreciated by one skilled in the art through reading the specification.

[0100] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a list of instructions to implement a logical function, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electronic connection having one or N wires (electronic devices), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program can be printed, because the program can be electronically captured, via the optical scan of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in the computer memory.

[0101] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware and in another embodiment, any of the following technologies known in the art or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0102] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, which can be stored in a computer-readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0103] In addition, each of the function units in each of the embodiments of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0104] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0105] In summary, the synchronous positioning and mapping method for establishing drivable areas in a map disclosed by the present application comprises: acquiring an image sequence to be processed, and sequentially detecting drivable areas of images in the image sequence; after extracting the drivable area of the current image, determining the corresponding three-dimensional approximate drivable area of the current image in the constructed point cloud map based on the drivable area of the current image; if it is determined that the current image is a key frame based on a preset selection rule, then the three-dimensional approximate drivable area corresponding to the current image is fused with the local fusion result corresponding to the last frame image in the point cloud map, to obtain the local fusion result corresponding to the current image in the point cloud map; the local fusion result corresponding to the current image in the point cloud map is fused with the global three-dimensional drivable area already established in the point cloud map, to obtain an updated global three-dimensional drivable area. The present application extracts drivable areas from images in an image sequence, determines a three-dimensional approximate drivable area based on the drivable area, and then performs local fusion and global fusion, so that a continuous drivable area is finally established in the point cloud map, which can adapt to various indoor and outdoor scenes and has a relatively low cost.

[0106] It should be understood that the application of the present application is not limited to the above examples, and those skilled in the art can make improvements or modifications according to the above description, and all these improvements and modifications shall fall within the protection scope of the claims of the present application.

Claims

1. A simultaneous localization and mapping method of establishing a drivable region in a map, characterized by, The method comprises: acquiring an image sequence to be processed, and sequentially detecting drivable areas of images in the image sequence; after extracting the drivable area of the current image, determining a corresponding three-dimensional approximate drivable area of the current image in a constructed point cloud map based on the drivable area of the current image; if it is determined that the current image is a key frame based on a preset selection rule, fusing the three-dimensional approximate drivable area corresponding to the current image with a local fusion result corresponding to the last frame image in the point cloud map to obtain a local fusion result corresponding to the current image in the point cloud map; fusing the local fusion result corresponding to the current image in the point cloud map with an established global three-dimensional drivable area in the point cloud map to obtain an updated global three-dimensional drivable area.

2. The simultaneous localization and mapping method of establishing a drivable region in a map according to claim 1, characterized in that, After extracting the drivable area of the current image, determining a corresponding three-dimensional approximate drivable area of the current image in a constructed point cloud map based on the drivable area of the current image, comprises: after extracting the drivable area of the current image, performing boundary extraction on the drivable area of the current image to obtain a drivable area boundary of the current image; performing feature point matching on the drivable area boundary of the current image and the constructed point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable area boundary of the current image; connecting the plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable area boundary of the current image into a closed polygon to form the three-dimensional approximate drivable area corresponding to the current image in the point cloud map. 3.The Simultaneous Localization and Mapping method of establishing drivable regions in a map according to claim 2, characterized in that, The drivable area is represented in an image mask manner; After extracting the drivable area of the current image, performing boundary extraction on the drivable area of the current image to obtain a drivable area boundary of the current image, comprises: After extracting the drivable area of the current image, performing boundary extraction on the image mask of the drivable area of the current image to obtain a drivable area boundary.

4. The simultaneous localization and mapping method of establishing a drivable region in a map according to claim 2, characterized in that, Performing feature point matching on the drivable area boundary of the current image and the constructed point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable area boundary of the current image, comprises: Performing feature point matching on feature points within a predetermined radius range of the drivable area boundary of the current image and the point cloud map to obtain a plurality of three-dimensional map points in the point cloud map that are successfully matched with the drivable area boundary of the current image.

5. The simultaneous localization and mapping method of establishing a drivable region in a map according to claim 1, wherein, The preset selection rule is to select the current image as a key frame every predetermined number of image frames, or to select the current image as a key frame if the number of feature point matching points between the current image and the point cloud map is less than a preset threshold.

6. The simultaneous localization and mapping method of establishing a drivable region in a map according to claim 1, wherein, After extracting the drivable area of the current image, determining a corresponding three-dimensional approximate drivable area of the current image in a constructed point cloud map based on the drivable area of the current image, further comprises: If the current image is a next frame image of a key frame, the three-dimensional approximate drivable area corresponding to the current image in the point cloud map is taken as a local fusion result corresponding to the current image in the point cloud map. 7.The Simultaneous Localization and Mapping method of establishing drivable regions in a map according to claim 1, wherein, After extracting the drivable region of the current image, determining the corresponding three-dimensional approximate drivable region of the current image in the constructed point cloud map based on the drivable region of the current image, further comprising: If the current image is not a key frame, the three-dimensional approximate drivable region corresponding to the current image is fused with the local fusion result corresponding to the last frame image in the point cloud map to obtain the local fusion result corresponding to the current image in the point cloud map, and the drivable region of the next frame image is extracted.

8. A simultaneous localization and mapping device that establishes a drivable area in a map, characterized by, The device comprises: a region extraction module configured to acquire an image sequence to be processed and detect drivable regions of images in the image sequence in sequence; a region determination module configured to, after extracting the drivable region of the current image, determine the corresponding three-dimensional approximate drivable region of the current image in the constructed point cloud map based on the drivable region of the current image; a local fusion module configured to fuse the three-dimensional approximate drivable region corresponding to the current image with the local fusion result corresponding to the last frame image in the point cloud map to obtain the local fusion result corresponding to the current image in the point cloud map; a global fusion module configured to, if the current image is determined to be a key frame based on a preset selection rule, fuse the local fusion result corresponding to the current image in the point cloud map with the global three-dimensional drivable region established in the point cloud map to obtain an updated global three-dimensional drivable region.

9. A terminal, characterized by comprising: comprise: a memory, a processor, and a simultaneous localization and mapping program for establishing a drivable region in a map stored on the memory and executable on the processor, the simultaneous localization and mapping program for establishing a drivable region in a map, when executed by the processor, implements the steps of the simultaneous localization and mapping method for establishing a drivable region in a map according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program which can be executed to implement the steps of the simultaneous localization and mapping method for establishing a drivable region in a map according to any one of claims 1-7.

Images