Occupancy grid map construction method and device

Abstract

The application discloses a kind of occupancy grid map construction method and device, it is related to map construction technical field, the method includes: obtaining the radar point cloud data of specified area, optical image, occupancy grid map;With grid coordinates matching, the first conversion parameter information of point cloud coordinates is determined;With pixel coordinates matching, the second conversion parameter information of point cloud coordinates is determined;According to the first, second conversion parameter information, the conversion relationship of grid coordinates and pixel coordinates is determined;According to conversion relationship and time synchronization parameter, grid coordinates are mapped to optical image and obtain pixel coordinate point set;Image processing is carried out to pixel coordinate point set, and the multi-path false pixel coordinate of determining;According to the second conversion parameter information and frame sequence matching algorithm, multi-path false pixel coordinate is mapped to radar point cloud data, and multi-path false point cloud coordinates are determined and deleted, and occupancy grid map is reconstructed.The application can simply and effectively identify multi-path false point cloud in point cloud, improve the accuracy of map positioning.

Description

Technical Field

[0001] This invention relates to the field of map construction technology, and in particular to a method and apparatus for occupying a raster map. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Simultaneous Localization and Mapping (SLAM) technology is widely used in fields such as autonomous driving, robot autonomous navigation, and drones. This technology relies on a single sensor or multiple sensors to observe the surrounding environment, thereby achieving mapping and self-localization.

[0004] Commonly used sensors include millimeter-wave radar, a radar system that uses electromagnetic waves in the millimeter-wave frequency band for detection and ranging, and can operate normally in adverse weather conditions such as rain and snow. Occupancy grid maps constructed using SLAM algorithms and point cloud data collected by millimeter-wave radar can perceive the surrounding environment. However, because radar signals undergo multiple reflection or scattering paths during propagation, additional multipath spurious point clouds appear in the received signal. The presence of multipath spurious point clouds can interfere with the correct calculation of the SLAM algorithm. The SLAM algorithm may incorrectly include multipath spurious point clouds in the position estimation calculation, resulting in inaccurate positioning in the constructed occupancy grid map. Simultaneously, during the occupancy grid map construction process, the SLAM algorithm may incorrectly identify multipath spurious point clouds as real environmental feature points, leading to inaccurate point cloud representations on the map. This, in turn, results in numerous noisy and false points appearing on the map during subsequent path planning and navigation.

[0005] Because millimeter-wave radar has sparse point clouds and lacks rich semantic information, it cannot intuitively identify multipath false points in occupied grid maps. Most existing technologies use machine learning models to detect and filter multipath false point clouds. However, machine learning models require a large amount of real data for training, the training process is relatively complex, and the model needs to be updated frequently. Otherwise, it may not be able to accurately identify multipath false points, resulting in inaccurate positioning in occupied grid maps. Summary of the Invention

[0006] This invention provides a method for constructing an occupancy grid map, which can simply and effectively identify and remove multipath false point clouds in radar point cloud data, thereby improving the accuracy of occupancy grid map positioning. The method includes:

[0007] Acquire continuous frame radar point cloud data and optical images of a specified area, and acquire an occupied grid map constructed based on the continuous frame radar point cloud data;

[0008] The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the grid map to determine the first transformation parameter information.

[0009] The point cloud coordinates of continuous frame radar point cloud data are matched with the pixel coordinates of optical images to determine the second transformation parameter information;

[0010] Based on the first and second conversion parameter information, the conversion relationship between raster coordinates and pixel coordinates is determined; based on the conversion relationship between raster coordinates and pixel coordinates and the preset time synchronization parameter, the raster coordinates occupying the raster map are mapped to the optical image to obtain the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image; the time synchronization parameter is used to represent the correspondence between the frame number of raster coordinates and the frame number of pixel coordinates.

[0011] Image processing is performed on the pixel coordinate point set, and the region of interest is determined based on the image processing results. The pixel coordinates of the region of interest are used as the multipath spurious pixel coordinates in the pixel coordinate point set.

[0012] The frame sequence point cloud of each radar point cloud data is determined according to the preset frame sequence matching algorithm.

[0013] Based on the second conversion parameter information, the multipath false pixel coordinates are converted into point cloud coordinates; the frame sequence point cloud to which the converted point cloud coordinates belong is determined, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinates in the continuous frame radar point cloud data.

[0014] Remove the multipath false point cloud coordinates from the continuous frame radar point cloud data to obtain the updated continuous frame radar point cloud data;

[0015] The occupied grid map is reconstructed using the updated continuous frame radar point cloud data.

[0016] This invention also provides an occupancy grid map construction device for easily and effectively identifying and removing multipath false point clouds in radar point cloud data, thereby improving the accuracy of occupancy grid map positioning. The device includes:

[0017] The acquisition module is used to acquire continuous frame radar point cloud data and optical images of a specified area, as well as to acquire an occupied grid map constructed based on the continuous frame radar point cloud data.

[0018] The first parameter determination module is used to match the point cloud coordinates of the first frame radar point cloud data with the grid coordinates of the first frame occupying the grid map to determine the first transformation parameter information.

[0019] The second parameter determination module is used to match the point cloud coordinates of continuous frame radar point cloud data with the pixel coordinates of optical images to determine the second transformation parameter information.

[0020] The transformation relationship determination module is used to determine the transformation relationship between raster coordinates and pixel coordinates based on the first transformation parameter information and the second transformation parameter information;

[0021] The first mapping module is used to map the grid coordinates occupying the grid map to the optical image according to the conversion relationship between grid coordinates and pixel coordinates and the preset time synchronization parameters, so as to obtain the set of pixel coordinate points of the grid coordinates occupying the grid map in the optical image; the time synchronization parameters are used to represent the correspondence between the frame number of grid coordinates and the frame number of pixel coordinates.

[0022] The spurious point determination module is used to perform image processing on the pixel coordinate point set, determine the region of interest based on the image processing results, and use the pixel coordinates of the region of interest as the multipath spurious pixel coordinates in the pixel coordinate point set.

[0023] The frame sequence determination module is used to determine the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm.

[0024] The second mapping module is used to convert multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determine the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong, and the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong are multipath false point cloud coordinates in continuous frame radar point cloud data.

[0025] The deletion module is used to delete multipath false point cloud coordinates in continuous frame radar point cloud data to obtain updated continuous frame radar point cloud data.

[0026] The mapping module is used to reconstruct the occupied grid map using updated continuous frame radar point cloud data.

[0027] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for constructing an occupied grid map.

[0028] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for constructing an occupied grid map.

[0029] In this embodiment of the invention, continuous frame radar point cloud data and optical images of a specified area are acquired, as well as an occupied grid map constructed based on the continuous frame radar point cloud data; the point cloud coordinates of the first frame radar point cloud data are matched with the grid coordinates of the first frame of the occupied grid map to determine first transformation parameter information; the point cloud coordinates of the continuous frame radar point cloud data are matched with the pixel coordinates of the optical image to determine second transformation parameter information; the transformation relationship between grid coordinates and pixel coordinates is determined based on the first and second transformation parameter information; based on the transformation relationship between grid coordinates and pixel coordinates and a preset time synchronization parameter, the grid coordinates of the occupied grid map are mapped to the optical image to obtain a set of pixel coordinate points of the occupied grid map in the optical image; the time synchronization parameter is used to represent the grid coordinates. The system establishes a correspondence between the frame number and the frame number of pixel coordinates; performs image processing on the pixel coordinate point set, determines the region of interest based on the image processing results, and uses the pixel coordinates of the region of interest as the multipath false pixel coordinates in the pixel coordinate point set; determines the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm; converts the multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determines the frame sequence point cloud to which the converted point cloud coordinates belong, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinate in the continuous frame radar point cloud data; deletes the multipath false point cloud coordinates in the continuous frame radar point cloud data to obtain updated continuous frame radar point cloud data; and reconstructs the occupied grid map using the updated continuous frame radar point cloud data. In this way, by mapping the occupied grid map onto the optical image, and using image processing techniques, the region of interest is determined from the pixel coordinates of the grid coordinates of the occupied grid map in the optical image. Then, the pixel coordinates of the region of interest are extracted as multipath false pixel coordinates in the pixel coordinate set. Then, the multipath false pixel coordinates are mapped onto the radar point cloud data, which can effectively identify the multipath false point cloud coordinates in the radar point cloud data. Furthermore, by deleting the multipath false point cloud coordinates in the radar point cloud data, the occupied grid map can be reconstructed using the updated radar point cloud data, thereby improving the accuracy of the occupied grid map positioning. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0031] Figure 1 This is a flowchart of an occupation grid map construction method provided in an embodiment of the present invention;

[0032] Figure 2 This is a flowchart provided in an embodiment of the present invention for matching the point cloud coordinates of the first frame of radar point cloud data with the grid coordinates of the first frame of the grid map to determine the first transformation parameter information.

[0033] Figure 3 This is a flowchart illustrating how to map the grid coordinates of a grid map to an optical image, according to an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram of an occupation grid map construction device according to an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0037] In the description of this specification, the terms "comprising," "including," "having," and "containing" are open-ended terms, meaning that they include but are not limited to. The terms "an embodiment," "a specific embodiment," "some embodiments," and "for example," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The order of steps involved in the various embodiments is used to illustrate the implementation of this application, and the order of steps is not limited and can be adjusted appropriately as needed.

[0038] Research has revealed that while occupancy grid maps constructed using SLAM algorithms and millimeter-wave radar-acquired point cloud data can perceive the surrounding environment, the presence of multipath spurious point clouds in the received signals results from multiple reflection or scattering paths during radar signal propagation. These multipath spurious point clouds interfere with the correct calculations of the SLAM algorithm, potentially leading to incorrect inclusion of them in the position estimation calculations and resulting in inaccurate localization of the constructed occupancy grid map. Furthermore, during the occupancy grid map construction process, the SLAM algorithm may incorrectly identify multipath spurious point clouds as real environmental feature points, leading to inaccurate point cloud representations on the map. This, in turn, results in numerous noisy and false points appearing on the map during subsequent path planning and navigation. Because millimeter-wave radar has sparse point clouds and lacks rich semantic information, it cannot intuitively identify multipath false points in occupied grid maps. Most existing technologies use machine learning models to detect and filter multipath false point clouds. However, machine learning models require a large amount of real data for training, the training process is relatively complex, and the model needs to be updated frequently. Otherwise, it may not be able to accurately identify multipath false points, resulting in inaccurate positioning in occupied grid maps.

[0039] In response to the above research, and in order to easily and effectively identify and remove multipath false point clouds in radar point cloud data and improve the accuracy of occupied grid map positioning, this invention provides an occupied grid map construction method and apparatus.

[0040] like Figure 1 The diagram shown is a flowchart of a method for constructing an occupied grid map according to an embodiment of the present invention. The method may include the following steps:

[0041] Step 101: Obtain continuous frame radar point cloud data and optical images of the specified area, and obtain the occupied grid map constructed based on the continuous frame radar point cloud data.

[0042] Step 102: Match the point cloud coordinates of the first frame of radar point cloud data with the grid coordinates of the first frame of the grid map to determine the first transformation parameter information.

[0043] Step 103: Match the point cloud coordinates of the continuous frame radar point cloud data with the pixel coordinates of the optical image to determine the second transformation parameter information;

[0044] Step 104: Determine the transformation relationship between raster coordinates and pixel coordinates based on the first transformation parameter information and the second transformation parameter information;

[0045] Step 105: Based on the conversion relationship between raster coordinates and pixel coordinates and the preset time synchronization parameters, the raster coordinates occupying the raster map are mapped to the optical image to obtain the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image; the time synchronization parameters are used to represent the correspondence between the frame number of the raster coordinates and the frame number of the pixel coordinates.

[0046] Step 106: Perform image processing on the pixel coordinate point set, determine the region of interest based on the image processing result, and use the pixel coordinates of the region of interest as the multipath dummy pixel coordinates in the pixel coordinate point set.

[0047] Step 107: Determine the frame sequence point cloud for each radar point cloud data according to the preset frame sequence matching algorithm.

[0048] Step 108: Based on the second conversion parameter information, convert the multipath false pixel coordinates into point cloud coordinates; determine the frame sequence point cloud to which the converted point cloud coordinates belong, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinates in the continuous frame radar point cloud data.

[0049] Step 109: Delete the multipath false point cloud coordinates in the continuous frame radar point cloud data to obtain the updated continuous frame radar point cloud data.

[0050] Step 110: Reconstruct the occupied grid map using the updated continuous frame radar point cloud data.

[0051] In this embodiment of the invention, continuous frame radar point cloud data and optical images of a specified area are acquired, as well as an occupied grid map constructed based on the continuous frame radar point cloud data; the point cloud coordinates of the first frame radar point cloud data are matched with the grid coordinates of the first frame of the occupied grid map to determine first transformation parameter information; the point cloud coordinates of the continuous frame radar point cloud data are matched with the pixel coordinates of the optical image to determine second transformation parameter information; the transformation relationship between grid coordinates and pixel coordinates is determined based on the first and second transformation parameter information; based on the transformation relationship between grid coordinates and pixel coordinates and a preset time synchronization parameter, the grid coordinates of the occupied grid map are mapped to the optical image to obtain a set of pixel coordinate points of the occupied grid map in the optical image; the time synchronization parameter is used to represent the grid coordinates. The system establishes a correspondence between the frame number and the frame number of pixel coordinates; performs image processing on the pixel coordinate point set, determines the region of interest based on the image processing results, and uses the pixel coordinates of the region of interest as the multipath false pixel coordinates in the pixel coordinate point set; determines the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm; converts the multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determines the frame sequence point cloud to which the converted point cloud coordinates belong, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinate in the continuous frame radar point cloud data; deletes the multipath false point cloud coordinates in the continuous frame radar point cloud data to obtain updated continuous frame radar point cloud data; and reconstructs the occupied grid map using the updated continuous frame radar point cloud data. In this way, by mapping the occupied grid map onto the optical image, and using image processing techniques, the region of interest is determined from the pixel coordinates of the grid coordinates of the occupied grid map in the optical image. Then, the pixel coordinates of the region of interest are extracted as multipath false pixel coordinates in the pixel coordinate set. Then, the multipath false pixel coordinates are mapped onto the radar point cloud data, which can effectively identify the multipath false point cloud coordinates in the radar point cloud data. Furthermore, by deleting the multipath false point cloud coordinates in the radar point cloud data, the occupied grid map can be reconstructed using the updated radar point cloud data, thereby improving the accuracy of the occupied grid map positioning.

[0052] The following is about Figure 1 The method for constructing the occupied grid map is described in detail.

[0053] Before step 101 above, firstly, radar point cloud data of a designated area is acquired using millimeter-wave radar; optical images of the designated area are acquired using a camera. The millimeter-wave radar and camera can be mounted at fixed locations on a mobile vehicle, and the acquired radar point cloud data is continuous frame radar point cloud data. Then, using the SLAM algorithm, an occupancy grid map is constructed based on the continuous frame radar point cloud data.

[0054] In step 101 above, continuous frame radar point cloud data and optical images of the specified area are acquired, as well as an occupied grid map constructed based on the continuous frame radar point cloud data.

[0055] In step 102 above, the point cloud coordinates of the first frame radar point cloud data can be matched with the grid coordinates of the first frame occupying the grid map to determine the first transformation parameter information.

[0056] In practice, it is only necessary to match the point cloud coordinates of the first frame of radar point cloud data with the grid coordinates of the first frame of the occupying grid map. Based on the matching result, the first transformation parameter information is determined. Then, using the first transformation parameter information, the point cloud coordinates of other frames of radar point cloud data can be matched with the grid coordinates of the occupying grid map.

[0057] In one embodiment, such as Figure 2 As shown, step 102 above may specifically include:

[0058] Step 201: Match the point cloud coordinates of the first frame of radar point cloud data with the grid coordinates of the first frame of the grid map to obtain pairwise matched coordinates.

[0059] Step 202: Determine the first transformation parameter information based on the pairwise paired coordinates and rigid body transformations. The first transformation parameter information includes the first translation amount, the second translation amount, and the rotation matrix.

[0060] In practice, rigid body transformation can be used to obtain the conversion parameters between the matching point cloud coordinates and the raster coordinates, namely the first translation, the second translation, and the rotation matrix.

[0061] The first translation can be the translation of the horizontal axis, and the second translation can be the translation of the vertical axis.

[0062] In step 103 above, the point cloud coordinates of the continuous frame radar point cloud data can be matched with the pixel coordinates of the optical image to determine the second transformation parameter information.

[0063] In practice, a marker can be set, and the point cloud coordinates of the marker in the radar point cloud data can be matched with the pixel coordinates of the marker in the optical image to determine the second conversion parameter information.

[0064] In one embodiment, step 103 above may specifically include:

[0065] Set up markers and determine the point cloud coordinates of the markers in continuous frames of radar point cloud data, and the pixel coordinates of the markers in optical images;

[0066] Using homography transformation, the point cloud coordinates of the marker in continuous frame radar point cloud data and the pixel coordinates of the marker in the optical image are transformed to determine the second transformation parameter information; the second transformation parameter information includes the homography matrix and the scaling factor.

[0067] In practice, homography transformation can be used to convert the point cloud coordinates and pixel coordinates of the marker to obtain transformation parameter information, namely the homography matrix and scaling factor.

[0068] In step 104 above, the conversion relationship between raster coordinates and pixel coordinates can be determined based on the first conversion parameter information and the second conversion parameter information.

[0069] In one embodiment, step 104 above may specifically include:

[0070] Using homography transformation, the first and second transformation parameter information are used as transformation parameters for homography transformation to determine the transformation relationship between raster coordinates and pixel coordinates.

[0071] In one embodiment, the conversion relationship between raster coordinates and pixel coordinates can be expressed by the following formula 1:

[0072]

[0073] Where tx is the first translation, ty is the second translation, R is the rotation matrix, H is the homography matrix, S is the scaling factor, i is the number of frames in the radar point cloud data, (u i ,v i () represents pixel coordinates. These are raster coordinates.

[0074] In practice, the point cloud coordinates of radar point cloud data can be used as a bridge to construct the conversion relationship between raster coordinates and pixel coordinates using homography transformation, i.e., Formula 1 above.

[0075] In step 105 above, the raster coordinates occupying the raster map can be mapped to the optical image based on the conversion relationship between raster coordinates and pixel coordinates and the preset time synchronization parameters, thereby obtaining the set of pixel coordinate points in the optical image corresponding to the raster coordinates occupying the raster map. The time synchronization parameters are used to represent the correspondence between the frame number of the raster coordinates and the frame number of the pixel coordinates.

[0076] In practical implementation, since the point cloud coordinates of continuous frame radar point cloud data are consecutive frames, the grid coordinates of the constructed grid map are also consecutive frames. However, the number of frames for pixel coordinates in the optical image is greater than the number of frames for point cloud coordinates in the radar point cloud data. Consequently, the number of frames for pixel coordinates is greater than the number of frames for grid coordinates. Therefore, when mapping the grid coordinates of the grid map to the optical image, the number of frames is discontinuous, with time intervals. In this embodiment, the number of frames for pixel coordinates corresponding to grid coordinates can be determined by setting a correspondence between the number of frames for grid coordinates and the number of frames for pixel coordinates. Then, the grid coordinates corresponding to the number of frames are converted to pixel coordinates through the conversion relationship between grid coordinates and pixel coordinates.

[0077] In one embodiment, such as Figure 3 As shown, step 105 above may specifically include:

[0078] Step 301: Based on the preset time synchronization parameters, using the optical image as a reference, match the frame number of pixel coordinates in the optical image with the frame number of grid coordinates occupying the grid map.

[0079] Step 302: Based on the conversion relationship and matching results between raster coordinates and pixel coordinates, the raster coordinates occupying the raster map are mapped to the pixel coordinates of the frame number in the optical image that matches the raster coordinates, thus obtaining the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image.

[0080] In practice, since the number of frames for pixel coordinates in an optical image is greater than the number of frames for raster coordinates, the number of frames for pixel coordinates in the optical image can be used as a benchmark to correspond the number of frames for raster coordinates occupying the raster map to them, thus obtaining the raster coordinates corresponding to each frame of pixel coordinates in the optical image. Then, using Formula 1 above, the raster coordinates corresponding to each frame of pixel coordinates occupying the raster map can be converted into pixel coordinates and mapped onto the optical image to form a set of pixel coordinate points (the set of pixel coordinate points in the optical image corresponding to the raster coordinates occupying the raster map).

[0081] In step 106 above, image processing can be performed on the pixel coordinate point set, and the region of interest can be determined based on the image processing result. The pixel coordinates of the region of interest are then used as the multipath dummy pixel coordinates in the pixel coordinate point set.

[0082] In practice, image processing can be performed on the pixel coordinate point set. Based on the image processing results, the position of the multipath false points in the pixel coordinate point set can be observed intuitively. The position of the multipath false points is taken as the region of interest. The pixel coordinates in the region of interest are determined by the horizontal and vertical coordinate measurement information in the pixel coordinate point set, and the multipath false pixel coordinates are obtained.

[0083] In this embodiment of the invention, it is necessary to map the multipath spurious pixel coordinates to continuous frame radar point cloud data.

[0084] In step 107 above, due to the time synchronization relationship, a frame of mapped optical image may contain multiple frames of radar point cloud. Therefore, the pixel coordinates of one frame cannot correspond one-to-one with the point cloud coordinates of each frame. One pixel coordinate may correspond to multiple points in multiple consecutive frames. In this embodiment of the invention, the frame sequence point cloud of each radar point cloud data can be determined according to the preset frame sequence matching algorithm.

[0085] In practice, starting from the first frame, the coordinates of the same radar point cloud data within two time intervals can be searched in continuous frame radar point cloud data with radius r. This process can be repeated to determine multiple frame sequence point clouds for each radar point cloud data. Here, r can be set according to the time error between the frame number of point cloud coordinates and the frame number of pixel coordinates. In this way, frame sequence point clouds with two time synchronization intervals can be obtained.

[0086] For example, the pixel coordinates of the 20th frame of the optical image correspond to the point cloud coordinates of the 5th frame of the continuous frame radar point cloud data, and the pixel coordinates of the 30th frame of the optical image correspond to the point cloud coordinates of the 9th frame of the continuous frame radar point cloud data. The point cloud of the two time-synchronized frame sequences is equivalent to the point cloud coordinates of the 5th to 9th frames.

[0087] In step 108 above, the multipath false pixel coordinates can be converted into point cloud coordinates according to the second conversion parameter information; the frame sequence point cloud to which the converted point cloud coordinates belong is determined, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinates in the continuous frame radar point cloud data.

[0088] In practice, the second conversion parameter information represents the conversion relationship between point cloud coordinates and pixel coordinates. Based on the second conversion parameter information, the conversion relationship between point cloud coordinates and pixel coordinates can be determined. Based on the conversion relationship between point cloud coordinates and pixel coordinates, the multipath false pixel coordinates are converted into point cloud coordinates. The converted point cloud coordinates are matched with the frame sequence point cloud to obtain the frame sequence point cloud to which the converted point cloud coordinates of the multipath false pixels belong. Thus, the multipath false point cloud coordinates in the continuous frame radar point cloud data are obtained.

[0089] In one embodiment, the conversion relationship between point cloud coordinates and pixel coordinates is shown in Formula 2 below:

[0090]

[0091] in, This represents the point cloud coordinates corresponding to the multipath spurious pixel coordinates, (u' i ,v' i) represents the coordinates of a spurious pixel in a multipath path.

[0092] In steps 109 and 110 above, the multipath false point cloud coordinates in the continuous frame radar point cloud data can be deleted, and the occupied grid map can be reconstructed using the SLAM algorithm based on the continuous frame radar point cloud data with the multipath false point cloud coordinates deleted.

[0093] This can improve the accuracy of positioning on the grid map.

[0094] To better understand the occupation grid map construction method provided in the embodiments of the present invention, a specific example is given below.

[0095] First, radar point cloud data of the surrounding environment (a designated area) is acquired using millimeter-wave radar; optical images of the surrounding environment are acquired using an optical camera; and an occupancy grid map is constructed based on the radar point cloud data using a SLAM algorithm. Then, the specific steps are as follows:

[0096] 1. Define continuous frame radar point cloud data as located in the local coordinate system, optical images as located in the pixel coordinate system, and the occupied raster map as located in the global coordinate system.

[0097] Let the point cloud coordinates of each frame of radar point cloud data in the local coordinate system be... The pixel coordinates in the pixel coordinate system are (u i ,v i In the global coordinate system, the raster coordinates are...

[0098] 2. Transfer the point cloud coordinates of the first frame of radar point cloud data. raster coordinates of the first frame of the raster map Perform matching, based on the matching and Using rigid body transformation, the transformation parameters R, tx, ty between the local coordinate system and the global coordinate system are obtained (i.e., the first transformation parameter information: rotation matrix, first translation amount, second translation amount, which can also be called the transformation parameters between point cloud coordinates and raster coordinates).

[0099] 3. Using the principle of homography transformation, the calibration matrix is ​​obtained by using the point cloud coordinates and pixel coordinates of millimeter-wave radar and optical camera (marker) to obtain the transformation parameters H and S between the local coordinate system and the pixel coordinate system (i.e., the second transformation parameter information: homography matrix, scaling factor, which can also be called the transformation parameters between point cloud coordinates and pixel coordinates).

[0100] 4. Using the local coordinate system as a bridge, the homography transformation is used to construct the transformation relationship between raster coordinates in the global coordinate system and pixel coordinates in the pixel coordinate system. The formula for the transformation relationship is as follows:

[0101]

[0102] Where tx is the first translation, ty is the second translation, R is the rotation matrix, H is the homography matrix, S is the scaling factor, i is the number of frames in the continuous radar point cloud data, (u i ,v i () represents pixel coordinates. These are raster coordinates.

[0103] 5. Based on the preset time synchronization parameters, match the frame number of optical image pixel coordinates with the frame number of raster coordinates; according to the above formula for the conversion relationship between raster coordinates and pixel coordinates, the raster coordinates of each frame of radar point cloud data in the global coordinate system can be obtained as follows: Mapping the matched frames to the pixel coordinate system yields the pixel coordinates (u) of each frame of radar point cloud data in the pixel coordinate system. i ,v i This forms a set of pixel coordinates in the optical image representing the raster coordinates that occupy the raster map.

[0104] 6. Extract multipath spurious pixel coordinates (u') from the pixel coordinate set. i ,v' i Specifically, image processing algorithms are used to process the pixel coordinate point set. Based on the image processing results, the positions of multipath false points in the pixel coordinate point set can be observed intuitively. The positions of the multipath false points are taken as the region of interest. The pixel coordinates within the region of interest are determined by the horizontal and vertical coordinate measurement information of the pixel coordinate point set, thus obtaining the multipath false pixel coordinates.

[0105] 7. Based on the preset frame sequence matching algorithm, determine the frame sequence point cloud for each radar point cloud data; based on the second transformation parameters H and S, construct the transformation relationship from the pixel coordinate system to the local coordinate system (i.e., the transformation relationship from pixel coordinates to point cloud coordinates). The formula for the transformation relationship from the pixel coordinate system to the local coordinate system is:

[0106]

[0107] Based on the formula for the transformation relationship between the pixel coordinate system and the local coordinate system, the multipath false pixel coordinates are transformed to the local coordinate system to obtain the point cloud coordinates corresponding to the multipath false pixel coordinates. The point cloud coordinates of the frame sequence are then queried one by one with the point cloud of the frame sequence to determine the point cloud of the frame sequence to which the point cloud coordinates transformed from the multipath false pixel coordinates belong as the multipath false point cloud coordinates in the continuous frame radar point cloud data.

[0108] 8. Delete multipath false point cloud coordinates from radar point cloud data.

[0109] 9. The radar point cloud data with the multipath false point cloud coordinates removed is re-inputted into the millimeter-wave radar SLAM algorithm model to construct an occupied grid map, thereby optimizing the accuracy of positioning and removing noise points in the occupied grid map.

[0110] Thus, the above method can realize a multipath false point cloud suppression localization mapping method that integrates vision and radar. Compared with machine learning recognition methods, this method does not require a large amount of data for training and is relatively simple to implement. Furthermore, by enhancing the semantic features of the occupied grid map constructed by millimeter-wave radar SLAM with visual assistance, multipath false points can be intuitively identified. Based on the coordinate system transformation relationship, multipath false points can be deleted from the original radar point cloud data, thereby reducing their impact on positioning inaccuracy and eliminating interference from noise points in the map.

[0111] This invention also provides an occupation grid map construction apparatus, as described in the following embodiments. Since the principle by which this apparatus solves the problem is similar to that of the occupation grid map construction method, the implementation of this apparatus can be found in the implementation of the occupation grid map construction method, and repeated details will not be elaborated further.

[0112] like Figure 4 The diagram shown is a schematic representation of an occupational grid map construction device provided in an embodiment of the present invention, which may include:

[0113] The acquisition module 401 is used to acquire continuous frame radar point cloud data and optical images of a specified area, as well as to acquire an occupied grid map constructed based on the continuous frame radar point cloud data.

[0114] The first parameter determination module 402 is used to match the point cloud coordinates of the first frame radar point cloud data with the grid coordinates of the first frame occupying the grid map to determine the first transformation parameter information.

[0115] The second parameter determination module 403 is used to match the point cloud coordinates of the continuous frame radar point cloud data with the pixel coordinates of the optical image to determine the second transformation parameter information.

[0116] The transformation relationship determination module 404 is used to determine the transformation relationship between raster coordinates and pixel coordinates based on the first transformation parameter information and the second transformation parameter information;

[0117] The first mapping module 405 is used to map the grid coordinates occupying the grid map to the optical image according to the conversion relationship between grid coordinates and pixel coordinates and the preset time synchronization parameters, so as to obtain the set of pixel coordinate points of the grid coordinates occupying the grid map in the optical image; the time synchronization parameters are used to represent the correspondence between the frame number of grid coordinates and the frame number of pixel coordinates.

[0118] The false point determination module 406 is used to perform image processing on the pixel coordinate point set, determine the region of interest based on the image processing result, and use the pixel coordinates of the region of interest as the multipath false pixel coordinates in the pixel coordinate point set.

[0119] The frame sequence determination module 407 is used to determine the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm.

[0120] The second mapping module 408 is used to convert multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determine the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong, and the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong are multipath false point cloud coordinates in continuous frame radar point cloud data.

[0121] The deletion module 409 is used to delete the multipath false point cloud coordinates in the continuous frame radar point cloud data to obtain the updated continuous frame radar point cloud data.

[0122] The mapping module 410 is used to reconstruct the occupied grid map using updated continuous frame radar point cloud data.

[0123] In one embodiment, the first parameter determining module can be specifically used for:

[0124] The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the grid map to obtain pairwise paired coordinates.

[0125] Based on the pairwise paired coordinates and rigid body transformations, the first transformation parameter information is determined, which includes a first translation amount, a second translation amount, and a rotation matrix.

[0126] In one embodiment, the second parameter determining module can be specifically used for:

[0127] Set up markers and determine the point cloud coordinates of the markers in continuous frames of radar point cloud data, and the pixel coordinates of the markers in optical images;

[0128] Using homography transformation, the point cloud coordinates of the marker in continuous frame radar point cloud data and the pixel coordinates of the marker in the optical image are transformed to determine the second transformation parameter information; the second transformation parameter information includes the homography matrix and the scaling factor.

[0129] In one embodiment, the transformation relationship determination module can be specifically used for:

[0130] Using homography transformation, the first and second transformation parameter information are used as transformation parameters between raster coordinates and pixel coordinates to determine the transformation relationship between raster coordinates and pixel coordinates.

[0131] In one embodiment, the first mapping module can be specifically used for:

[0132] Based on preset time synchronization parameters, using the optical image as a reference, the frame number of pixel coordinates in the optical image is matched with the frame number of grid coordinates occupying the grid map.

[0133] Based on the conversion relationship and matching results between raster coordinates and pixel coordinates, the raster coordinates occupying the raster map are mapped to the pixel coordinates of the frame number in the optical image that matches the raster coordinates, thus obtaining the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image.

[0134] This invention also provides a computer device. Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements the above-described method for constructing a grid map.

[0135] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for constructing an occupied grid map.

[0136] In this embodiment of the invention, continuous frame radar point cloud data and optical images of a specified area are acquired, as well as an occupied grid map constructed based on the continuous frame radar point cloud data; the point cloud coordinates of the first frame radar point cloud data are matched with the grid coordinates of the first frame of the occupied grid map to determine first transformation parameter information; the point cloud coordinates of the continuous frame radar point cloud data are matched with the pixel coordinates of the optical image to determine second transformation parameter information; the transformation relationship between grid coordinates and pixel coordinates is determined based on the first and second transformation parameter information; based on the transformation relationship between grid coordinates and pixel coordinates and a preset time synchronization parameter, the grid coordinates of the occupied grid map are mapped to the optical image to obtain a set of pixel coordinate points of the occupied grid map in the optical image; the time synchronization parameter is used to represent the grid coordinates. The system establishes a correspondence between the frame number and the frame number of pixel coordinates; performs image processing on the pixel coordinate point set, determines the region of interest based on the image processing results, and uses the pixel coordinates of the region of interest as the multipath false pixel coordinates in the pixel coordinate point set; determines the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm; converts the multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determines the frame sequence point cloud to which the converted point cloud coordinates belong, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinate in the continuous frame radar point cloud data; deletes the multipath false point cloud coordinates in the continuous frame radar point cloud data to obtain updated continuous frame radar point cloud data; and reconstructs the occupied grid map using the updated continuous frame radar point cloud data. In this way, by mapping the occupied grid map onto the optical image, and using image processing techniques, the region of interest is determined from the pixel coordinates of the grid coordinates of the occupied grid map in the optical image. Then, the pixel coordinates of the region of interest are extracted as multipath false pixel coordinates in the pixel coordinate set. Then, the multipath false pixel coordinates are mapped onto the radar point cloud data, which can effectively identify the multipath false point cloud coordinates in the radar point cloud data. Furthermore, by deleting the multipath false point cloud coordinates in the radar point cloud data, the occupied grid map can be reconstructed using the updated radar point cloud data, thereby improving the accuracy of the occupied grid map positioning.

[0137] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing an occupational grid map, characterized in that, include: Acquire continuous frame radar point cloud data and optical images of a specified area, and acquire an occupied grid map constructed based on the continuous frame radar point cloud data; The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the grid map to determine the first transformation parameter information. The point cloud coordinates of continuous frame radar point cloud data are matched with the pixel coordinates of optical images to determine the second transformation parameter information; Based on the first and second transformation parameter information, the transformation relationship between raster coordinates and pixel coordinates is determined; Based on the conversion relationship between raster coordinates and pixel coordinates and the preset time synchronization parameters, the raster coordinates occupying the raster map are mapped to the optical image to obtain the set of pixel coordinate points in the optical image of the raster coordinates occupying the raster map; the time synchronization parameters are used to represent the correspondence between the frame number of the raster coordinates and the frame number of the pixel coordinates. Image processing is performed on the pixel coordinate point set, and the region of interest is determined based on the image processing results. The pixel coordinates of the region of interest are used as the multipath spurious pixel coordinates in the pixel coordinate point set. Based on the preset frame sequence matching algorithm, the frame sequence point cloud of each radar point cloud data is determined; Based on the second conversion parameter information, the multipath false pixel coordinates are converted into point cloud coordinates; the frame sequence point cloud to which the converted point cloud coordinates belong is determined, and the frame sequence point cloud to which the converted point cloud coordinates belong is the multipath false point cloud coordinates in the continuous frame radar point cloud data. Remove the multipath false point cloud coordinates from the continuous frame radar point cloud data to obtain the updated continuous frame radar point cloud data; The occupied grid map is reconstructed using the updated continuous frame radar point cloud data.

2. The method as described in claim 1, characterized in that, The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the occupying grid map to determine the first transformation parameter information, including: The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the grid map to obtain pairwise paired coordinates. Based on the pairwise paired coordinates and rigid body transformations, the first transformation parameter information is determined, which includes a first translation amount, a second translation amount, and a rotation matrix.

3. The method as described in claim 1, characterized in that, The point cloud coordinates of continuous frame radar point cloud data are matched with the pixel coordinates of the optical image to determine the second transformation parameter information, including: Set up markers and determine the point cloud coordinates of the markers in continuous frames of radar point cloud data, and the pixel coordinates of the markers in optical images; Using homography transformation, the point cloud coordinates of the marker in continuous frame radar point cloud data and the pixel coordinates of the marker in the optical image are transformed to determine the second transformation parameter information; the second transformation parameter information includes the homography matrix and the scaling factor.

4. The method as described in claim 1, characterized in that, Based on the first and second transformation parameter information, the transformation relationship between raster coordinates and pixel coordinates is determined, including: Using homography transformation, the first and second transformation parameter information are used as transformation parameters between raster coordinates and pixel coordinates to determine the transformation relationship between raster coordinates and pixel coordinates.

5. The method as described in claim 1, characterized in that, Based on the conversion relationship between raster coordinates and pixel coordinates and preset time synchronization parameters, the raster coordinates occupying the raster map are mapped to the optical image, resulting in a set of pixel coordinates in the optical image representing the raster coordinates occupying the raster map, including: Based on preset time synchronization parameters, using the optical image as a reference, the frame number of pixel coordinates in the optical image is matched with the frame number of grid coordinates occupying the grid map. Based on the conversion relationship and matching results between raster coordinates and pixel coordinates, the raster coordinates occupying the raster map are mapped to the pixel coordinates of the frame number in the optical image that matches the raster coordinates, thus obtaining the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image.

6. An apparatus for constructing a grid map, characterized in that, include: The acquisition module is used to acquire continuous frame radar point cloud data and optical images of a specified area, as well as to acquire an occupied grid map constructed based on the continuous frame radar point cloud data. The first parameter determination module is used to match the point cloud coordinates of the first frame radar point cloud data with the grid coordinates of the first frame occupying the grid map to determine the first transformation parameter information. The second parameter determination module is used to match the point cloud coordinates of continuous frame radar point cloud data with the pixel coordinates of optical images to determine the second transformation parameter information. The transformation relationship determination module is used to determine the transformation relationship between raster coordinates and pixel coordinates based on the first transformation parameter information and the second transformation parameter information; The first mapping module is used to map the grid coordinates occupying the grid map to the optical image according to the conversion relationship between grid coordinates and pixel coordinates and the preset time synchronization parameters, so as to obtain the set of pixel coordinate points of the grid coordinates occupying the grid map in the optical image; the time synchronization parameters are used to represent the correspondence between the frame number of grid coordinates and the frame number of pixel coordinates. The spurious point determination module is used to perform image processing on the pixel coordinate point set, determine the region of interest based on the image processing results, and use the pixel coordinates of the region of interest as the multipath spurious pixel coordinates in the pixel coordinate point set. The frame sequence determination module is used to determine the frame sequence point cloud of each radar point cloud data according to a preset frame sequence matching algorithm. The second mapping module is used to convert multipath false pixel coordinates into point cloud coordinates according to the second conversion parameter information; determine the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong, and the frame sequence point cloud to which the point cloud coordinates converted from multipath false pixel coordinates belong are multipath false point cloud coordinates in continuous frame radar point cloud data. The deletion module is used to delete multipath false point cloud coordinates in continuous frame radar point cloud data to obtain updated continuous frame radar point cloud data. The mapping module is used to reconstruct the occupied grid map using updated continuous frame radar point cloud data.

7. The apparatus as claimed in claim 6, characterized in that, The first parameter determination module is specifically used for: The point cloud coordinates of the first frame of radar point cloud data are matched with the grid coordinates of the first frame of the grid map to obtain pairwise paired coordinates. Based on the pairwise paired coordinates and rigid body transformations, the first transformation parameter information is determined, which includes a first translation amount, a second translation amount, and a rotation matrix.

8. The apparatus as claimed in claim 6, characterized in that, The second parameter determination module is specifically used for: Set up markers and determine the point cloud coordinates of the markers in continuous frames of radar point cloud data, and the pixel coordinates of the markers in optical images; Using homography transformation, the point cloud coordinates of the marker in continuous frame radar point cloud data and the pixel coordinates of the marker in the optical image are transformed to determine the second transformation parameter information; the second transformation parameter information includes the homography matrix and the scaling factor.

9. The apparatus as claimed in claim 6, characterized in that, The conversion relationship determination module is specifically used for: Using homography transformation, the first and second transformation parameter information are used as transformation parameters between raster coordinates and pixel coordinates to determine the transformation relationship between raster coordinates and pixel coordinates.

10. The apparatus as claimed in claim 6, characterized in that, The first mapping module is specifically used for: Based on preset time synchronization parameters, using the optical image as a reference, the frame number of pixel coordinates in the optical image is matched with the frame number of grid coordinates occupying the grid map. Based on the conversion relationship and matching results between raster coordinates and pixel coordinates, the raster coordinates occupying the raster map are mapped to the pixel coordinates of the frame number in the optical image that matches the raster coordinates, thus obtaining the set of pixel coordinate points of the raster coordinates occupying the raster map in the optical image.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 5.

Images