Map construction method, machine readable storage medium and processor

Abstract

The embodiment of the invention provides a map construction method, a machine readable storage medium and a processor, and belongs to the technical field of maps. The map construction method comprises the following steps: acquiring field 3D point cloud data through a sensor; preprocessing the obtained 3D point cloud data to obtain point cloud data; performing semantic segmentation on the point cloud data; and performing map construction according to the segmented point cloud data. According to the map construction method, the semantic map can be quickly and accurately constructed.

Description

Technical Field

[0001] The present invention relates to the field of map technology, and in particular to a method, device, storage medium and processor for processing field information. Background Art

[0002] With the continuous development of deep learning and the popularization of 3D LiDAR, the algorithms applied to semantic segmentation of laser point cloud data have also received extensive attention and research. At present, most laser SLAM only use the geometric information of laser data. Although it can obtain relatively accurate positioning and maps, the final map is only composed of corresponding point clouds stacked together, which is difficult to understand and not easy for subsequent navigation applications. Therefore, a method that can quickly and accurately construct semantic maps is needed. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a map construction method, a machine-readable storage medium and a processor, wherein the map construction method can quickly and accurately construct a semantic map.

[0004] In order to achieve the above object, an embodiment of the present invention provides a map construction method, the map construction method comprising:

[0005] Acquire 3D point cloud data on site through sensors;

[0006] Preprocessing the acquired 3D point cloud data to obtain point cloud data;

[0007] Performing semantic segmentation on the point cloud data;

[0008] A map is constructed according to the segmented point cloud data.

[0009] Optionally, preprocessing the acquired 3D point cloud data to obtain point cloud data includes:

[0010] Get 3D point cloud data for each frame;

[0011] Relative to each point in the point cloud, place the origin of the LiDAR coordinate system at the center of the 2D image.

[0012] According to the formula (1), the yaw angle and pitch angle of each point in the 3D point cloud data are obtained:

[0013]

[0014] Among them, yaw is the yaw angle, pitch is the pitch angle, x, y, z are the coordinates of the point in the 3D point cloud;

[0015] Obtaining the width and height of the two-dimensional image corresponding to the 3D point cloud data;

[0016] According to formula (2), the three-dimensional coordinates of the points in the 3D point cloud are converted into two-dimensional image coordinates:

[0017]

[0018] Among them, (u, v) is the coordinate point of the point in the 3D point cloud corresponding to the 2D image, fov down is the vertical downward field of view angle of the lidar sensor, fov up is the vertical upward field of view angle of the lidar sensor, col is the width of the two-dimensional image, and row is the width of the two-dimensional image.

[0019] Optionally, preprocessing the acquired 3D point cloud data to obtain point cloud data includes:

[0020] Acquire point cloud data on the two-dimensional image;

[0021] The point cloud in the point cloud data is marked and divided into a plurality of regions;

[0022] Acquire an area in the point cloud data where the number of point clouds is greater than a preset threshold;

[0023] Selecting a point cloud at a central position in the region;

[0024] According to the point cloud at the midpoint position in the region, calculating the distance between the remaining point clouds in the region and the point cloud at the center position;

[0025] Screening point clouds whose distance from the point cloud at the center position is less than a preset distance;

[0026] The point clouds of all the filtered areas are aggregated into a plane point cloud;

[0027] The corresponding 3D point cloud data is acquired according to the plane point cloud to obtain point cloud data.

[0028] Optionally, performing semantic segmentation on the preprocessed point cloud data includes:

[0029] Acquire the point cloud data;

[0030] Performing downsampling on the point cloud data in a horizontal direction;

[0031] Performing feature extraction on the downsampled point cloud data, and upsampling the extracted features to the original image resolution;

[0032] The upsampled point cloud data is semantically segmented using a softmax function.

[0033] Optionally, constructing a map according to the segmented point cloud data includes:

[0034] Construct a 3D point cloud map of the scene through the SLAM method;

[0035] Obtaining the point cloud data after semantic segmentation;

[0036] Extract point cloud features of the three-dimensional point cloud map constructed by the SLAM method;

[0037] Matching the point cloud data with the extracted point cloud features to screen mutually matching point clouds;

[0038] Build semantic maps based on matched point clouds.

[0039] Optionally, constructing a map according to the segmented point cloud data includes:

[0040] Get the point cloud data of the currently scanned scene;

[0041] Performing semantic definition on the point cloud data currently scanned;

[0042] Matching the semantic definition of the current scan with the semantics in the semantic map for consistency;

[0043] The semantics whose consistency between the semantic definition in the semantic map and the currently scanned semantic definition is greater than a set threshold are removed.

[0044] On the other hand, the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute a map construction method as described above.

[0045] In yet another aspect, the present invention further provides a processor for running a program, wherein the program, when run, is used to execute a map construction method as described above.

[0046] Through the above technical solution, a map construction method, a machine-readable storage medium and a processor provided by the present invention can obtain 3D point cloud data of the scene through a sensor. After obtaining the 3D point cloud data of the scene, the 3D point cloud data can be preprocessed to obtain point cloud data. After obtaining the point cloud data, the point cloud data can be semantically segmented, and a map can be constructed based on the semantically segmented point cloud data. The present invention can conveniently, quickly and accurately construct a semantic map.

[0047] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0049] Figure 1 is a flow chart of a map construction method according to one embodiment of the present invention;

[0050] Figure 2 is a first flow chart of preprocessing 3D point cloud data in a map construction method according to an embodiment of the present invention;

[0051] Figure 3 is a second flow chart of preprocessing 3D point cloud data in a map building method according to an embodiment of the present invention;

[0052] Figure 4 is a flowchart of performing semantic segmentation on point cloud data according to a map construction method according to an embodiment of the present invention;

[0053] Figure 5 is a first flow chart of performing map construction in a map construction method according to an embodiment of the present invention;

[0054] Figure 6 This is a second flow chart of performing map construction in a map construction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0056] Figure 1 is a flow chart of a map construction method according to an embodiment of the present invention. In the present invention, the process of the map construction method may include:

[0057] In step S1, on-site 3D point cloud data is acquired through sensors.

[0058] In step S2, the acquired 3D point cloud data is preprocessed to obtain point cloud data.

[0059] In step S3, semantic segmentation is performed on the point cloud data.

[0060] In step S4, a map is constructed based on the segmented point cloud data.

[0061] In the present invention, after obtaining the 3D point cloud data of the scene through the sensor, the obtained 3D point cloud data can be preprocessed to obtain point cloud data that meets the requirements. After obtaining the point cloud data, the point cloud data can be semantically segmented, and then a map can be constructed based on the segmented point cloud data. This method can conveniently, quickly and accurately construct a semantic map.

[0062] In one embodiment of the present invention, Figure 2 As shown, the first process of preprocessing 3D point cloud data may include:

[0063] In step S5, 3D point cloud data of each frame is obtained;

[0064] In step S6, for each point in the point cloud, the laser radar coordinate origin is placed at the center of the two-dimensional image;

[0065] In step S7, the yaw angle and pitch angle of each point in the 3D point cloud data are obtained according to formula (1):

[0066]

[0067] Among them, yaw is the yaw angle, pitch is the pitch angle, x, y, z are the coordinates of the point in the 3D point cloud;

[0068] In step S8, the width and height of the two-dimensional image corresponding to the 3D point cloud data are obtained;

[0069] In step S9, the three-dimensional coordinates of the points in the 3D point cloud are converted into two-dimensional image coordinates according to formula (2):

[0070]

[0071] Among them, (u, v) is the coordinate point of the point in the 3D point cloud corresponding to the 2D image, fov down is the vertical downward field of view angle of the lidar sensor, fov up is the vertical upward field of view angle of the lidar sensor, col is the width of the two-dimensional image, and row is the width of the two-dimensional image.

[0072] In the present invention, when acquiring the 3D point cloud data of the scene, it is generally scanned on a vertical plane by mechanical rotation of a mechanical laser radar and then imaged. This method similar to the line-by-line exposure of a camera will bring about a jelly effect, that is, when the carrier robot or the detected object is moving, due to the line-by-line exposure, different parts of the moving object are not exposed at the same time, resulting in shaking, slope images and other phenomena. Therefore, in order to obtain a representation of the geometric consistency of each laser scanning environment, the 3D point cloud data of each frame can be converted into a two-dimensional image data representation. When converted to a two-dimensional image representation, the 3D point cloud data of each frame can be obtained, and the laser radar coordinate origin can be placed at the center of the two-dimensional image relative to each point in the point cloud. The two-dimensional image can be set according to the needs of the staff. According to formula (1), the yaw angle and pitch angle of each point in the 3D point cloud data can be obtained. After obtaining the yaw angle and pitch angle, the width and height of the two-dimensional image corresponding to the 3D point cloud data can be obtained. Then, the three-dimensional coordinates of the points in the 3D point cloud can be converted into two-dimensional image coordinates according to formula (2).

[0073] In one embodiment of the present invention, Figure 3 As shown, the second process of preprocessing 3D point cloud data may include:

[0074] In step S10 , point cloud data on the two-dimensional image is acquired.

[0075] In step S11, the point cloud in the point cloud data is marked and segmented into multiple regions.

[0076] In step S12, an area in the point cloud data where the number of point clouds is greater than a preset threshold is obtained.

[0077] In step S13, a point cloud at a midpoint position in the region is selected.

[0078] In step S14, based on the point cloud at the midpoint position in the region, the distances between the remaining point clouds in the region and the point cloud at the midpoint position are calculated.

[0079] In step S15 , point clouds whose distance from the point cloud at the center is less than a preset distance are screened.

[0080] In step S16, the point clouds of all the filtered areas are aggregated into a plane point cloud.

[0081] In step S17, corresponding 3D point cloud data is acquired according to the plane point cloud to obtain point cloud data.

[0082] In the present invention, after converting the points in the 3D point cloud into two-dimensional points, the point cloud data on the two-dimensional image can be obtained. After obtaining the point cloud data, the point cloud in the point cloud data can be marked, and then the point cloud data can be divided into multiple regions. In the point cloud data, an area where the number of point clouds is greater than a preset threshold is obtained, and then the point cloud at the midpoint position in the area can be selected. After obtaining the point cloud at the center position of each area, the distance between the remaining point clouds in the area and the point cloud at the center position can be calculated. By calculating the distance between the remaining point clouds in an area and the point cloud at the center position, the evacuation relationship of the point clouds in the area can be determined. The point cloud whose distance from the point cloud at the center position is less than the preset distance can be filtered, so that the filtered point cloud can be more concentrated. After filtering the point cloud, the point clouds of all filtered areas can be aggregated into a plane point cloud, and then the corresponding 3D point cloud data can be obtained according to the plane point cloud to obtain the point cloud data.

[0083] In one embodiment of the present invention, Figure 4 As shown, the process of semantic segmentation of point cloud data may include:

[0084] In step S18, point cloud data is acquired.

[0085] In step S19, downsampling is performed on the point cloud data in the horizontal direction.

[0086] In step S20, feature extraction is performed on the downsampled point cloud data, and the extracted features are upsampled to the original image resolution.

[0087] In step S21, semantic segmentation is performed on the upsampled point cloud data using a softmax function.

[0088] In the present invention, after acquiring the point cloud data, downsampling can be performed in the horizontal direction according to the point cloud data, and then feature extraction can be performed on the downsampled point cloud data, and the extracted features can be upsampled to the original image resolution, and finally, the upsampled point cloud data can be semantically segmented by the softmax function to obtain the classification of each object.

[0089] In one embodiment of the present invention, Figure 5 As shown, the first process of map construction may include:

[0090] In step S22, a three-dimensional point cloud map of the scene is constructed by using the SLAM method.

[0091] In step S23, the point cloud data after semantic segmentation is obtained.

[0092] In step S24, point cloud features of the three-dimensional point cloud map constructed by the SLAM method are extracted.

[0093] In step S25, the point cloud data and the extracted point cloud features are matched to screen mutually matching point clouds.

[0094] In step S26, a semantic map is established based on the mutually matched point clouds.

[0095] In the present invention, when constructing a map, a three-dimensional point cloud map of the scene can be constructed by the SLAM method, and then the point cloud data after semantic segmentation can be obtained. The point cloud features of the three-dimensional point cloud map constructed by the SLAM method are extracted, and then the point cloud data and the extracted point cloud features can be matched to screen mutually matching point clouds, so that a semantic map can be established based on the mutually matching point clouds.

[0096] In one embodiment of the present invention, Figure 6 As shown, the second process of map construction may include:

[0097] In step S27, the point cloud data of the currently scanned scene is obtained.

[0098] In step S28, semantic definition is performed on the currently scanned point cloud data.

[0099] In step S29, the current scanned semantic definition is matched with the semantics in the semantic map for consistency.

[0100] In step S30, the semantics whose consistency with the current scan semantic definition in the semantic map is greater than a set threshold are removed.

[0101] In the present invention, when constructing a map, dynamic objects can be filtered out. When filtering out dynamic objects, point cloud data of the current scanned scene can be obtained, and then semantics can be defined for the point cloud data of the current scan. After the semantics are defined, the semantic definition of the current scan can be matched with the semantics in the semantic map for consistency, and then the semantics whose consistency with the semantic definition of the current scan in the semantic map is greater than a set threshold can be removed. This semantic can be regarded as a moving object and can therefore be removed.

[0102] On the other hand, the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute a map construction method as described above.

[0103] In yet another aspect, the present invention further provides a processor for running a program, wherein the program, when run, is used to execute a map construction method as described above.

[0104] Through the above technical solution, a map construction method, a machine-readable storage medium and a processor provided by the present invention can obtain 3D point cloud data of the scene through a sensor. After obtaining the 3D point cloud data of the scene, the 3D point cloud data can be preprocessed to obtain point cloud data. After obtaining the point cloud data, the point cloud data can be semantically segmented, and a map can be constructed based on the semantically segmented point cloud data. The present invention can conveniently, quickly and accurately construct a semantic map.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0109] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0110] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0111] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0112] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0113] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A map construction method, characterized in that: The map construction method comprises: Acquire 3D point cloud data on site through sensors; Preprocessing the acquired 3D point cloud data to obtain point cloud data; Performing semantic segmentation on the point cloud data; A map is constructed according to the segmented point cloud data.

2. The map construction method according to claim 1, characterized in that: Preprocessing the acquired 3D point cloud data to obtain point cloud data includes: Get 3D point cloud data for each frame; Relative to each point in the point cloud, place the origin of the LiDAR coordinate system at the center of the 2D image. According to the formula (1), the yaw angle and pitch angle of each point in the 3D point cloud data are obtained: Among them, yaw is the yaw angle, pitch is the pitch angle, x, y, z are the coordinates of the point in the 3D point cloud; Obtaining the width and height of the two-dimensional image corresponding to the 3D point cloud data; According to formula (2), the three-dimensional coordinates of the points in the 3D point cloud are converted into two-dimensional image coordinates: Among them, (u, v) is the coordinate point of the point in the 3D point cloud corresponding to the 2D image, fov down is the vertical downward field of view angle of the lidar sensor, fov up is the vertical upward field of view angle of the lidar sensor, col is the width of the two-dimensional image, and row is the width of the two-dimensional image.

3. The map construction method according to claim 2, characterized in that: Preprocessing the acquired 3D point cloud data to obtain point cloud data includes: Acquire point cloud data on the two-dimensional image; The point cloud in the point cloud data is marked and divided into a plurality of regions; Acquire an area in the point cloud data where the number of point clouds is greater than a preset threshold; Selecting a point cloud at a central position in the region; According to the point cloud at the midpoint position in the region, calculating the distance between the remaining point clouds in the region and the point cloud at the center position; Screening point clouds whose distance from the point cloud at the center position is less than a preset distance; The point clouds of all the filtered areas are aggregated into a plane point cloud; The corresponding 3D point cloud data is acquired according to the plane point cloud to obtain point cloud data.

4. The map construction method according to claim 2, characterized in that: Performing semantic segmentation on the preprocessed point cloud data includes: Acquire the point cloud data; Performing downsampling on the point cloud data in a horizontal direction; Performing feature extraction on the downsampled point cloud data, and upsampling the extracted features to the original image resolution; The upsampled point cloud data is semantically segmented using a softmax function.

5. The map construction method according to claim 4, characterized in that: Map construction based on the segmented point cloud data includes: Construct a 3D point cloud map of the scene using the SLAM method; Obtaining the point cloud data after semantic segmentation; Extract point cloud features of the three-dimensional point cloud map constructed by the SLAM method; Matching the point cloud data with the extracted point cloud features to screen mutually matching point clouds; Build semantic maps based on matched point clouds.

6. The map construction method according to claim 5, characterized in that: Map construction based on the segmented point cloud data includes: Get the point cloud data of the current scanned scene; Performing semantic definition on the point cloud data currently scanned; Matching the semantic definition of the current scan with the semantics in the semantic map for consistency; The semantics whose consistency between the semantic definition in the semantic map and the currently scanned semantic definition is greater than a set threshold are removed.

7. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions, and the instructions are used to enable a machine to execute a map construction method as described in any one of claims 1-6.

8. A processor, characterized in that: Used to run a program, wherein the program, when run, is used to execute a map construction method as described in any one of claims 1-6.

Images