A Cognitive Map Localization Method and System Based on Sparse Point Cloud-Texture Information

Abstract

The present invention discloses a cognitive map positioning method and system based on sparse point cloud-texture information, belonging to the field of positioning technology. The present invention makes full use of the accurate depth estimation ability of lidar and the good feature expression ability of cameras, adopts a geometric center estimation algorithm to obtain traffic sign geometric information, and after fusing point cloud data and image data through the traffic sign geometric information, generates a cognitive map. Based on the cognitive map, using the obtained traffic sign geometric center information, a two-stage search strategy is adopted to match and position the traffic signs at the current position of the autonomous vehicle and the traffic sign features stored in the semantic layer data of the cognitive map, obtain the corresponding positions of the traffic signs where the vehicle is currently traveling in the cognitive map, and fuse them with the local positioning results of the vehicle's own lidar odometer to achieve high-precision positioning of the autonomous vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of positioning, and in particular, to a cognitive map positioning method and system based on sparse point cloud-texture information. Background Art

[0002] With the progress of technology, important breakthroughs and developments have been made in autonomous driving technology in recent years. As the core of autonomous driving, mapping and positioning technology has become increasingly important for the development of autonomous driving technology. As an important part of the future development of autonomous driving, high-precision maps have gradually attracted public attention and received extensive attention. High-precision maps contain a large amount of static auxiliary information such as road traffic markings and the surrounding geographical environment, which can help autonomous driving vehicles quickly identify the surrounding environment and improve the calculation efficiency and accuracy. Compared with ordinary maps, high-precision maps have a higher absolute coordinate accuracy and contain more abundant road element attribute information, which completely restores the static object information in the real world. Among the static object information, traffic signs, as important marking features on the road, provide important guiding information for drivers.

[0003] The traditional method for extracting road traffic signs is to first collect high-density point cloud data through a high-line-number lidar, and then extract it through an extraction algorithm. The on-vehicle sensors used in the extraction process are expensive and require huge computing power, resulting in low production efficiency and complex update of high-precision maps. Compared with high-line-number lidars, vision sensors have more advantages in terms of cost, information collection, and feature expression ability. However, relying solely on image features has the defect of being easily affected by weather and seasons. As an autonomous driving map designed based on the human cognitive mechanism, the cognitive map has the characteristics of being lightweight and having low computing resource requirements. By using vision sensors, information such as lanes and traffic signs on the road is recorded to form a lightweight map, and semantic-level positioning can be referenced based on the cognitive map during vehicle driving. Due to its small data volume and rich information, it has achieved remarkable application results in the positioning and navigation of autonomous driving vehicles.

[0004] In existing cognitive map positioning and navigation solutions, the center of the detection frame is often defined as the center of the traffic sign. However, due to the uncertainty of the algorithm, there is a deviation between the actual center of the traffic sign and the center of the detection frame. Using the center of the detection frame as the storage of cognitive map elements introduces errors, which affects the positioning accuracy based on the cognitive map. In addition, only one vision sensor, the camera, is used in existing cognitive map methods, lacking accurate measurement of the distance to traffic signs, which further affects the positioning accuracy. Summary of the Invention

[0005] In view of the shortcomings of the above-mentioned existing cognitive map positioning and navigation, the present invention provides a cognitive map positioning method based on sparse point cloud-texture information. After the point cloud data and the image data are fused based on the geometric information of the traffic sign obtained by the method, a cognitive map is generated, so that the accurate depth estimation capability of the laser radar and the good feature expression capability of the camera can be fully utilized. Based on the cognitive map generated by the method, the geometric information of the sign and the two-stage search strategy are used to fuse with the local positioning results of the laser radar odometer on the vehicle to achieve high-precision positioning of the autonomous driving vehicle.

[0006] The technical solution of the present invention is as follows:

[0007] A cognitive map positioning method based on sparse point cloud-texture information comprises the following steps:

[0008] Step 1: Collect current road data, including image data collected by the camera and point cloud data collected by the lidar; use GNSS to establish a global coordinate system;

[0009] Step 2: extract the traffic sign from the image data collected in step 1, perform binary mask processing on the extracted traffic sign using an image segmentation algorithm, and then use a geometric center estimation method to calculate the center point data of the traffic sign as geometric prior information of the traffic sign;

[0010] Step 3: Jointly calibrate the laser radar and the camera to obtain the laser radar-camera coordinate system transformation matrix, and use the matrix to unify the coordinate systems of the camera and the laser radar; project the point cloud data obtained in step 1 into the image data obtained in step 1 through the cone projection guidance method to construct image-point cloud association data;

[0011] Step 4: In the constructed image-point cloud association data, the geometric contour information of the traffic sign is extracted using the center point data obtained in step 2, and the point cloud data within the contour range is extracted based on the contour information;

[0012] Step 5: Select the image data corresponding to the outline of the traffic sign in the point cloud data, perform Gabor wavelet transform on it, use the LBP operator to extract the binary pattern, and obtain the LBP-Gabor texture feature image; divide the LBP-Gabor texture feature image into n sub-blocks, and perform histogram statistics and normalization processing on each sub-block to generate n histogram feature vectors; after cascading these n vectors, store them as the final texture features of the traffic sign; where n≥4;

[0013] Step 6: Repeat steps 2 to 4 to obtain the final texture features of all traffic signs on the current road, obtain the semantic layer of the cognitive map, and complete the generation of the cognitive map;

[0014] Step 7: Use a two-stage search strategy to match and locate the features in the traffic signs at the current position of the autonomous vehicle and the semantic layer data of the cognitive map, so as to obtain the coordinates of the traffic signs where the current autonomous vehicle is driving in the cognitive map;

[0015] Step 8: Align the coordinates of the traffic signs where the current autonomous vehicle is driving obtained in Step 7 to the global coordinate system, and fuse them with the local positioning result of the lidar odometer to obtain the vehicle pose estimation.

[0016] Further, the process of calculating the center point coordinates of the traffic sign by using the geometric center estimation method in Step 2 is as follows:

[0017] Judge whether the shape of the traffic sign is circular or triangular according to the binary mask processing result; if it is judged to be circular, use the Hough transform algorithm to extract the center point of the traffic sign to obtain the center point and radius data, and if it is judged to be triangular, use the method of statistically extracting discrete small regions in the region or the HSV color space transformation method to obtain the center point data of the traffic sign;

[0018] Further, the detailed process of obtaining the pose estimation in Step 7 is as follows:

[0019] Step 7.1: First-stage matching: Eliminate the interference of the same traffic signs in the same road section on the positioning result:

[0020] 7.1.1 According to the vehicle-end observation result, take the traffic sign at the current position of the autonomous vehicle as the center as the region of interest, and extract a rectangular frame; at the same time, for each traffic sign in the cognitive map of the same road section, extract a rectangular frame;

[0021] 7.1.2 Calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame in the cognitive map by using two measures of the Bhattacharyya distance and the perceptual hash respectively,

[0022] 7.1.3 Calculate the comprehensive correlation coefficient between each rectangular frame in the cognitive map and the rectangular frame of the vehicle-end observation result according to the calculation result in Step 7.1.2. The comprehensive correlation coefficient is defined as:

[0023]

[0024] 7.1.4 Sort the comprehensive correlation coefficients calculated in Step 7.1.3 in descending order, and the one with the smallest value is matched as the same traffic sign object.

[0025] Step 7.2: Second-stage matching: Match the texture features of each traffic sign stored in the semantic layer of the cognitive map according to the result of the first-stage matching to obtain the corresponding position of the traffic sign where the current vehicle is driving in the cognitive map.

[0026] Furthermore, step 1 further includes performing time synchronization processing on GNSS, lidar, and camera, and aligning the data collected by the camera and the lidar point cloud data in the time dimension through time synchronization processing; the time synchronization processing process is as follows:

[0027] Step 1.1: Add timestamps to each frame of camera image data and the point cloud data collected by the lidar based on the high-frequency GPS signal of the clock;

[0028] Step 1.2: Search for the timestamp of the image data according to the timestamp of the point cloud data with a lower frequency, and find the timestamp of the image data closest to the timestamp of the point cloud data with a lower frequency for matching to achieve the synchronization processing of the image data and the point cloud data.

[0029] Furthermore, the detailed process of step 8 is as follows:

[0030] Suppose the relative pose obtained by the lidar odometer is:

[0031] P = {P 0 , P 1 , P 2 ...}

[0032] P i = [R|T]

[0033] where P i is the pose of the i-th frame relative to the (i - 1)-th frame. R and T are pose transformation matrices

[0034] Suppose the global positioning result obtained by matching road signs in the map is:

[0035] G = {G 0 , G k ,...}

[0036] G = [R|T]

[0037] where G i is the global matching positioning result of the map, and R and T are pose transformation matrices

[0038] Assume the true pose of the camera to be solved is:

[0039] X = {X 0 , X 1 , X 2 ...}

[0040] X i = [R|T]

[0041] where X is the true pose of the camera to be solved, and R and T are pose transformation matrices

[0042] Construct the fusion positioning problem as a maximum a posteriori probability problem:

[0043] X = argmax(f(X|P,G))

[0044] When the observations G and X are known, maximize the posterior probability and construct a pose graph to solve for the true pose of the vehicle.

[0045] A cognitive map positioning method based on sparse point cloud-texture information, including a data acquisition module, a feature extraction module, a map generation module, a matching and positioning module, and a fusion positioning module.

[0046] The data acquisition module is respectively connected to the map feature extraction module and the matching and positioning module; it is used to collect image data and point cloud data; preprocess the collected image data to obtain the central points of traffic signs in the image data; establish a global coordinate system using GNSS, and complete the coordinate, time, and space unification of three sensors, namely the camera, lidar, and GNSS; in this embodiment, the image data is collected by a camera, and the point cloud data is collected by a lidar.

[0047] The map feature extraction module is connected to the map generation module. Under the coordinate system of the unified camera and lidar provided by the data acquisition module, use the received central point data to extract the geometric prior information of traffic signs, and based on this, extract the point cloud data within the contour of traffic signs, that is, generate sparse point clouds; select the image data corresponding to the contour of traffic signs in the point cloud data, and use the Gabor wavelet transform, LBP operator, histogram statistics, and normalization algorithm to process it to obtain the texture information of traffic signs and provide it to the map generation module.

[0048] The map generation module is connected to the matching and positioning module, and generates a cognitive map according to the received traffic sign texture signal;

[0049] The matching and positioning module is connected to the fusion positioning module, and uses two-stage matching to match the traffic signs scanned near the current position of the autonomous vehicle with the traffic signs in the cognitive map to obtain the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map;

[0050] The fusion positioning module aligns the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map to the global coordinate system, and then fuses them with the local positioning results of the lidar odometer to obtain the vehicle pose estimation.

[0051] A cognitive map positioning method and system based on sparse point cloud-texture information provided by the present invention make full use of the accurate depth estimation ability of lidar and the good feature expression ability of cameras. The geometric center estimation algorithm is used to obtain the geometric information of traffic signs, and after fusing the point cloud data and image data through the geometric information of traffic signs, a cognitive map is generated. Based on the cognitive map, using the obtained geometric center information of traffic signs, a two-stage search strategy is adopted to match and locate the traffic signs at the current position of the autonomous vehicle and the traffic sign features stored in the semantic layer data of the cognitive map, obtain the corresponding position of the traffic signs where the vehicle is traveling in the cognitive map, and fuse it with the local positioning result of the vehicle's own lidar odometer to achieve high-precision positioning of the autonomous vehicle.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] 1. The detection deviation between the actual center of the traffic sign and the center of the detection frame is reduced through the geometric center estimation algorithm.

[0054] 2. In the process of generating the semantic layer of the cognitive map, first use the Gabor wavelet to extract the features of the image data in the image-point cloud association data, and then use the LBP operator to extract the texture features. By combining the Gabor and LBP operators, the amount of calculation is reduced while richer texture features are obtained.

[0055] 3. When positioning based on the cognitive map, the two-stage search strategy is adopted to solve the problem of no match caused by the possible existence of multiple identical traffic signs in the same section of the road, and the positioning accuracy is improved by fusing the cognitive map positioning result with the local positioning result of the vehicle's own lidar odometer. Description of the Drawings

[0056] Figure 1 is the flow chart of geometric center estimation in the embodiment;

[0057] Figure 2 is the texture feature extraction process in the embodiment;

[0058] Figure 3 is the cognitive map generation diagram of the present invention;

[0059] Figure 4 is the schematic diagram of the first-stage matching in the embodiment;

[0060] Figure 5 is the schematic diagram of pose graph fusion in the embodiment;

[0061] Figure 6 is the system block diagram of the present invention. Detailed Embodiments

[0062] The technical solutions of the present invention will be described in detail below with reference to the drawings and embodiments.

[0063] A cognitive map positioning method based on sparse point cloud-texture information provided by the present invention includes the following steps:

[0064] Step 1: Collect road data, which includes image data collected by a camera and point cloud data collected by a lidar. A global coordinate system is established using GNSS. Since the working frequencies of the three sensors are different, there is a time difference in the data between the sensors. Therefore, it is necessary to perform time and space synchronization processing on the three sensors of GNSS, camera, and lidar to align them in the time dimension and space dimension. The time synchronization processing process is as follows:

[0065] Step 1.1: Add timestamps to each frame of camera image data and point cloud data collected by the lidar based on the high-frequency GPS signal of the clock;

[0066] Step 1.2: Search for the timestamp of the image data according to the timestamp of the point cloud data with a lower frequency, and find the timestamp of the image data closest to the timestamp of the point cloud data with a lower frequency for matching to achieve the synchronization processing of the image data and the point cloud data.

[0067] Step 2: Extract traffic signs from the image data collected in Step 1. After performing binary masking processing on the extracted traffic signs using an image segmentation algorithm, calculate the center point data of the traffic sign using a geometric center estimation algorithm.

[0068] In practical applications, the shapes of traffic signs can be divided into circular or triangular. Determine whether the traffic sign is circular or triangular according to the result of binary masking processing, and then adopt different calculation methods according to different shapes. For example Figure 1 As shown: In this embodiment, the Otsu algorithm is first used to perform threshold segmentation on the image data to obtain the binary mask region of the traffic sign. If the obtained binary mask region of the traffic sign is circular, the Hough transform algorithm is used to extract the center point of the traffic sign to obtain the center point and radius data. If the obtained binary mask region of the traffic sign is triangular, the method of statistically extracting discrete small regions within the region is adopted: extract the discrete small region with the largest area from the statistical discrete regions, and then calculate the first moment of the discrete small region. Calculate the center point of the triangle according to the calculation result of the first moment. If the binary mask region of the traffic sign cannot be obtained by using the Otsu algorithm or the obtained binary mask region of the traffic sign is incorrect, the HSV color space transformation method is adopted to convert the detected region map from the RGB space to the HSV space to calculate the center point. Taking the triangle as an example, the detailed calculation process of the HSV color space transformation method is:

[0069] Perform region segmentation on the detection area according to the colors of traffic signs in actual applications, and obtain the largest circumscribed triangle that meets the conditions as the mask area of the traffic sign; at the same time, calculate the areas of each discrete area in the detection area, select the triangular area with the largest area as the traffic sign area, and calculate the first moment of this area to obtain the geometric center point of the traffic sign, which is used as the geometric prior information of the traffic sign;

[0070] Step 3: Perform joint calibration on the lidar and the camera, obtain the lidar-camera coordinate system transformation matrix, and use this matrix to unify the coordinate systems of the camera and the lidar; through the method guided by cone projection, project the point cloud data obtained in Step 1 onto the image data obtained in Step 1 to construct image-point cloud associated data. The detailed process is as follows:

[0071] Based on the prior information of the initial parameters of the detection box, project the 2D mask area of the image data, search for the point cloud coordinates in the 3D coordinate space corresponding to this 2D detection box, and extract the target points on the planar traffic sign within the geometric contour of the traffic sign. Since the 2D area of the image detection box lacks depth information, a point on the image corresponds to a ray in the lidar coordinate system. Therefore, the planar rectangular box on the image corresponds to a pyramid in the lidar coordinate system.

[0072] After projecting the lidar point cloud data onto the image data, through the one-to-one mapping relationship between the point cloud data and the image data, calculate the pixel coordinates of the point cloud data in the image by the following formula.

[0073]

[0074] Where P is the camera internal parameter matrix, and Tr_velo_to_cam is the lidar-camera transformation matrix obtained by joint calibration. Construct image-point cloud associated data, and the data structure of the association is [u, v, x, y, z], where u / v are the coordinates of the point cloud projected onto the image. By judging whether the projected point coordinate values belong to the traffic sign, determine whether the corresponding original point cloud is retained, so as to achieve the purpose of only retaining the point cloud data with cognitive information and realize the lightweight of the point cloud data.

[0075] Step 4: In the constructed image-point cloud associated data, use the center point data obtained in Step 2 to extract the geometric contour information of the traffic sign, and based on this contour information, extract the point cloud data within the contour range.

[0076] After the point cloud data is projected onto the image, it is discrete points. To ensure semantic continuity. Therefore, it is necessary to select the image strip area corresponding to the point cloud projection point for feature extraction. Such as Figure 2As shown in the figure, select the image data corresponding to the traffic sign contour of the point cloud data. After performing Gabor wavelet transform on it, use the LBP operator to extract the binary pattern, and generate an LBP-Gabor texture feature image. Divide the LBP-Gabor texture feature image into n sub-blocks, and perform histogram statistics and normalization processing on each sub-block to generate n histogram feature vectors; after concatenating these n vectors, store them as the final texture feature of the traffic sign. Reduce the data calculation amount through Gabor wavelet transform and improve the robustness of the system.

[0077] Step 6: Repeat Steps 2 to 4 to obtain the final texture features of all traffic signs on the current road, obtain the semantic layer of the cognitive map, and generate the cognitive map as shown in Figure 3 the figure.

[0078] Step 7: Adopt a two-stage search strategy to match and locate the features in the traffic signs at the current position scanned by the autonomous vehicle and the data in the semantic layer of the cognitive map, so as to obtain the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map; the detailed process of matching and locating using the two-stage search strategy is as follows:

[0079] As shown in Figure 4 the figure, the first-stage matching includes the following steps:

[0080] 7.1.1 According to the vehicle-end observation results, take the traffic sign at the current position of the autonomous vehicle as the center as the region of interest, and extract a rectangular special frame; at the same time, for each traffic sign in the cognitive map of the same road section, extract a rectangular frame respectively;

[0081] 7.1.2 Calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame in the cognitive map respectively using two measures: the Bhattacharyya distance and the perceptual hash.

[0082] 7.1.3 Calculate the comprehensive correlation coefficient between each rectangular frame in the cognitive map and the rectangular frame of the vehicle-end observation result according to the calculation results in Step 7.1.2. The comprehensive correlation coefficient is defined as:

[0083]

[0084] 7.1.4 Sort the comprehensive correlation coefficients calculated in Step 7.1.3 in descending order, and the one with the smallest value is the object matched as the same traffic sign.

[0085] Second-stage matching: According to the results of the first-stage matching, match the texture features of each traffic sign stored in the semantic layer of the cognitive map to obtain the corresponding position of the traffic sign scanned at the current position of the autonomous vehicle in the cognitive map.

[0086] Step 8: Align the coordinates of the traffic signs passed by the current autonomous vehicle obtained in Step 7 in the cognitive map to the global coordinate system, and fuse them with the local positioning result of the lidar odometer to obtain the pose estimation of the autonomous vehicle. The detailed process is as follows:

[0087] Refer to Figure 5 , where the vertices in the figure are composed of the poses of each frame, and the edges are composed of the lidar odometer factor constraints and the cognitive map matching result factor constraints; the poses and constraints can be modeled as a joint probability distribution problem. Assume that the nodes in the pose graph are the vehicle motion states χ = {x 0 ,x 1 ,...,x n}; then:

[0088]

[0089] where X is the vehicle state, z L , z R are the measured values of the vehicle states corresponding to the two constraint factors. S is the set of measured values, including the local pose measurement of the lidar odometer and the pose measurement obtained by cognitive map matching localization. The essence of the pose graph is a maximum likelihood estimation problem. Assuming that all probability measurements within the same time period are independent, the above problem can be expressed as:

[0090]

[0091] Assume that the uncertainty of the observation in formula (2) follows a Gaussian distribution, then formula (3) can be obtained:

[0092]

[0093] z k is the measured value of the vehicle state, X is the state prediction value, h is the observation equation. By formula (3), the observation is converted into a non-linear least squares problem, and the objective function is calculated as the sum of squares of the constraint factors from the lidar observation and the constraint factors from the global matching observation of the cognitive map, thus obtaining as the lidar odometer and cognitive map matching observation result model, which can be obtained through the inter-frame pose transformation.

[0094] Step 8.2: Complete the optimization of the lidar odometer and cognitive map matching observation result model by analyzing the residual factors in the pose graph:

[0095] (1) The constraint factor of the lidar observation

[0096] The present invention solves the local pose through a lidar odometer, and the pose estimation within a short time is accurate and stable. The pose relationship between two adjacent frames is constructed as a factor of the global pose:

[0097]

[0098] and are the local coordinate system poses at time t-1 and time t, is the change operator between the poses.

[0099] (2) Constraint factor for global matching observation of the cognitive map

[0100] In the present invention, the starting point of the vehicle is used as the origin of the coordinate system, and the position information obtained through map matching calculation is set as For the state node with traffic sign observation, a map matching position constraint can be included, and the map matching factor is as follows:

[0101]

[0102] The establishment of the global optimized pose graph is completed through the above two constraint factors. The essence of solving the optimization problem is to find a set of state variables to complete the matching of all constraint factors, so that the Mahalanobis norm sum of all error factors is minimized.

[0103] According to the above derivation content, the process of completing the current pose estimation of the autonomous vehicle in this embodiment is as follows:

[0104] Step 8.1. Set the relative pose obtained by the lidar odometer as:

[0105] P = {P 0 , P 1 , P 2 ...}

[0106] P i = [R|T]

[0107] where P i is the pose of the i-th frame relative to the (i-1)-th frame. R and T are pose transformation matrices

[0108] Suppose the global positioning result obtained by matching road signs in the map is:

[0109] G = {G 0 , G k ,...}

[0110] G = [R|T]

[0111] where G iFor the global matching and positioning result of the map, R and T are pose transformation matrices

[0112] Assume that the true pose of the camera to be solved is:

[0113] X = {X 0 , X 1 , X 2 ...}

[0114] X i = [R|T]

[0115] where X is the true pose of the camera to be solved, and R and T are pose transformation matrices

[0116] Construct the fusion positioning problem as a maximum a posteriori probability problem:

[0117] X = argmax(f(X|P, G))

[0118] When the observations G and X are known, maximize the posterior probability and construct a pose graph to solve the true pose of the vehicle

[0119] In the positioning method of this embodiment, it includes two positioning sources: the pose estimation of the lidar odometer and the vehicle positioning based on traffic signs. The two positioning sources complement each other. Through the fusion of the two positioning sources, a high-precision positioning result of the autonomous driving vehicle is obtained. Since the lidar odometer outputs results throughout the process and its error accumulates over time, resulting in a drift within a certain range, the positioning method based on the cognitive map is a sparse positioning result. By fusing the positioning results of traffic signs, the long-term drift error of the lidar odometer is corrected, and the cumulative error of the lidar odometer is eliminated. The fusion algorithm regards the positioning problem as an optimization problem under known observations. By using the relative pose provided by the lidar odometer and the global positioning result based on traffic signs, a pose graph optimization problem is constructed to obtain the optimal positioning description

[0120] A cognitive map positioning based on sparse point cloud-texture information includes a data acquisition module, a feature extraction module, a map generation module, a matching positioning module, and a fusion positioning module

[0121] The data acquisition module is respectively connected to the map feature extraction module and the matching positioning module; it is used to collect image data and point cloud data; preprocess the collected image data to obtain the center points of traffic signs in the image data; establish a global coordinate system using GNSS, and complete the coordinate, time, and space unification of the three sensors of the camera, lidar, and GNSS. In this embodiment, the image data is collected by a camera, and the point cloud data is collected by a lidar

[0122] The map feature extraction module is connected to the map generation module. Under the coordinate system of the unified camera and lidar provided by the data acquisition module, it extracts the geometric prior information of traffic signs using the received center point data, and based on this, extracts the point cloud data within the range of the geometric prior information, that is, sparse point cloud generation; selects the image data corresponding to the traffic sign contour of the point cloud data, and after processing using the Gabor wavelet transform, LBP operator, histogram statistics, and normalization algorithm, provides the texture information of the traffic sign to the map generation module.

[0123] The map generation module is connected to the matching and positioning module, and generates a cognitive map according to the received traffic sign texture signal;

[0124] The matching and positioning module is connected to the fusion positioning module, and is used to match the traffic signs scanned near the current position of the autonomous vehicle with the traffic signs in the cognitive map to obtain the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map;

[0125] The fusion positioning module is connected to the traffic sign positioning module. After aligning the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map to the global coordinate system, it is fused with the local positioning result of the lidar odometer to obtain the vehicle pose estimation.

Claims

1. A cognitive map localization method based on sparse point cloud-texture information, characterized in that: It includes the following steps: Step 1, collect current road data, and the data includes image data collected by a camera and point cloud data collected by a lidar; Establish a global coordinate system using GNSS; Step 2, extract traffic signs from the image data collected in Step 1. After performing binary masking on the extracted traffic signs using an image segmentation algorithm, calculate the center point data of the traffic signs using a geometric center estimation algorithm as the geometric prior information of the traffic signs; Step 3, perform joint calibration on the lidar and the camera to obtain the lidar-camera coordinate system transformation matrix, and use this matrix to unify the coordinate systems of the camera and the lidar; through the method guided by cone projection, project the point cloud data obtained in Step 1 onto the image data obtained in Step 1 to construct image-point cloud association data; Step 4, in the constructed image-point cloud association data, extract the geometric contour information of the traffic signs using the center point data obtained in Step 2, and based on this contour information, extract the point cloud data within the contour range; Step 5, select the image data corresponding to the traffic sign contour of the point cloud data, perform Gabor wavelet transform on it, and then use the LBP operator to extract the binary pattern to obtain the LBP-Gabor texture feature image; divide the LBP-Gabor texture feature image into n sub-blocks, and perform histogram statistics and normalization processing on each sub-block to generate n histogram feature vectors; After cascading these n vectors, store them as the final texture feature of the traffic sign; Step 6, repeat Steps 2 to 4 to obtain the final texture features of all traffic signs on the current road, obtain the semantic layer of the cognitive map, and complete the generation of the cognitive map; Step 7, use a two-stage search strategy to match and locate the features in the traffic signs at the current position of the autonomous vehicle and the data in the semantic layer of the cognitive map to obtain the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map; the detailed process of the two-stage search strategy for positioning and matching is as follows: Step 7.1, the first-stage matching: 7.1.1 According to the vehicle-end observation results, take the traffic sign at the current position of the autonomous vehicle as the center as the region of interest, and extract a rectangular special frame; at the same time, for each traffic sign in the cognitive map of the same road section, extract a rectangular frame respectively; 7.1.2 Calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame in the cognitive map using the Bhattacharyya distance and the perceptual hashing measures respectively; 7.1.3, according to the settlement results in Step 7.1.2, calculate the comprehensive correlation coefficient of each rectangular frame in the cognitive map and the rectangular frame of the vehicle-end observation result respectively; the comprehensive correlation coefficient is defined as: 7.1.4, sort the comprehensive correlation coefficients calculated in Step 7.1.3 in descending order, and the one with the smallest value is the matching object of the same traffic sign board; Step 7.2, the second-stage matching: According to the results of the first-stage matching, match the texture features of the traffic signs stored in the semantic layer of the cognitive map to obtain the corresponding position of the traffic signs at the current position of the vehicle in the cognitive map; Step 8: Align the corresponding position of the traffic signs at the current vehicle position obtained in Step 7 in the cognitive map to the global coordinate system, and fuse it with the local positioning result of the lidar odometer to obtain the vehicle pose estimation.

2. A cognitive map positioning method based on sparse point cloud-texture information according to claim 1, characterized in that: The process of calculating the center point data of the traffic sign by using the geometric center estimation method in Step 2 is as follows: Judge whether the shape of the traffic sign is circular or triangular according to the binary mask processing result; if it is judged to be circular, use the Hough transform algorithm to extract the center point of the traffic sign to obtain the center point and radius data, and if it is judged to be triangular, use the statistical method to extract the discrete small areas in the area or the HSV color space transformation method to obtain the center point data of the traffic sign.

3. A cognitive map positioning method based on sparse point cloud-texture information according to claim 1, characterized in that: Step 1 further includes performing time synchronization processing on GNSS, lidar, and camera, and aligning the data collected by the camera and the lidar point cloud data in the time dimension through time synchronization processing; the time synchronization processing process is as follows: Step 1.1: Add timestamps to each frame of camera image data and the point cloud data collected by the lidar based on the high-frequency GPS signal of the clock; Step 1.2: Search for the timestamp of the image data according to the timestamp of the point cloud data with a lower frequency, and find the timestamp of the image data closest to the timestamp of the point cloud data with a lower frequency for matching to achieve the synchronization processing of the image data and the point cloud data.

4. A cognitive map positioning based on sparse point cloud-texture information, including a data acquisition module, a feature extraction module, a map generation module, a matching positioning module, and a fusion positioning module; The data acquisition module is respectively connected to the map feature extraction module and the matching positioning module; it is used to collect image data and point cloud data; preprocess the collected image data to obtain the center point of the traffic sign in the image data ； Establish a global coordinate using GNSS, and complete the unification of the coordinates, time, and space of the three sensors of the camera, lidar, and GNSS; among them, The image data is collected by the camera, and the point cloud data is collected by the lidar; The map feature extraction module is connected to the map generation module. Under the coordinate system of the unified camera and lidar provided by the data acquisition module, use the received center point data to extract the geometric prior information of the traffic sign, and based on this, extract the point cloud data within the contour range, that is, generate sparse point cloud; Select the image data corresponding to the contour of the traffic sign in the point cloud data, and use the Gabor wavelet transform, LBP operator, histogram statistics, and normalization algorithm to process it to obtain the texture information of the traffic sign and provide it to the map generation module; The map generation module is connected to the matching positioning module, and generates a cognitive map according to the received traffic sign texture signal; The matching and positioning module is connected to the fusion positioning module. It uses two-stage matching to match the traffic signs scanned near the current position of the autonomous vehicle with the traffic signs in the cognitive map, and obtains the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map. The detailed process of the two-stage matching is as follows: Step 7.1: First-stage matching 7.1.1 According to the vehicle-end observation results, a rectangular special frame is extracted with the traffic sign at the current position of the autonomous vehicle as the center as the region of interest. At the same time, for each traffic sign in the cognitive map of the same road section, a rectangular frame is extracted respectively; 7.1.2 The similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame in the cognitive map is calculated using two measures: the Bhattacharyya distance and the perceptual hash respectively; 7.1.3 According to the calculation results in step 7.1.2, the comprehensive correlation coefficients between each rectangular frame in the cognitive map and the rectangular frame of the vehicle-end observation result are obtained respectively. The comprehensive correlation coefficient is defined as: 7.1.4 The comprehensive correlation coefficients calculated in step 7.1.3 are sorted in descending order, and the one with the smallest value is the matching object of the same traffic sign board; Step 7.2: Second-stage matching: According to the result of the first-stage matching, the texture features of the traffic signs stored in the semantic layer of the cognitive map are matched to obtain the corresponding position of the traffic signs at the current position of the vehicle in the cognitive map; The fusion positioning module aligns the coordinates of the traffic signs at the current position of the autonomous vehicle in the cognitive map to the global coordinate system, and then fuses them with the local positioning result of the lidar odometer to obtain the vehicle pose estimation.

Images