A positioning method based on a lightweight road semantic map

Abstract

This invention discloses a localization method based on a lightweight road semantic map. The invention includes the following steps: First, visual odometry information is estimated from binocular images captured by a camera; then, the initial pose T of the camera is estimated based on the visual odometry information. c ; for the initial pose T of the camera c Perform a coordinate system transformation to obtain the initial pose T in the world coordinate system. w Then, extract various semantic targets from the current target image to obtain the corresponding semantic feature points, thereby obtaining the semantic feature point set of the current target image; finally, based on the initial pose T in the world coordinate system... w The method uses a set of semantic feature points and a particle filter to estimate the current pose. Experiments on a self-collected dataset show that, compared with traditional visual SLAM and particle filter methods based on road network maps, the method of this invention achieves higher accuracy and more reliable localization performance.

Description

Technical Field

[0001] This invention relates to map-based positioning methods, and in particular to a positioning method based on a lightweight road semantic map. Background Technology

[0002] With the rapid development of autonomous driving, the demand for high-precision vehicle positioning is becoming increasingly urgent. In recent years, although the accuracy of various onboard sensors has improved, sensor-based odometry or SLAM positioning is still affected by accumulated errors. Maps, however, encompass global information and can effectively compensate for the accumulated errors caused by methods such as odometry; therefore, map-integrated positioning methods have become a major trend.

[0003] Traditional topology-based localization methods typically correct localization by matching odometer trajectories with road network topology maps, and then use filtering or optimization algorithms to refine the pose. Topology maps are simple in structure and easy to process, making it convenient to construct matching relationships. However, their limited information content leads to lower accuracy in localization results. Furthermore, this method cannot eliminate errors introduced by odometer readings on long, straight roads. Semantic maps, on the other hand, are rich in semantic information about the roads and offer higher accuracy, effectively addressing these map-related problems.

[0004] Dense 3D semantic point cloud maps built by fusing multiple sensors offer high accuracy, but their large storage requirements and computational demands limit their practical application. Existing methods extract edge information from dense semantic maps for localization, but factors such as noise complicate data preprocessing and result in poor edge detection, thus impacting localization accuracy. Summary of the Invention

[0005] To address the problems existing in the background technology, the present invention aims to provide a lightweight road semantic map localization method, applicable to lightweight map localization problems in road scenarios, and effectively solving the aforementioned problems. By estimating the initial pose through sensors, designing a special matching method to associate visual and map semantic information, and using a particle filtering method to estimate the optimal pose, relatively reliable localization results can be obtained.

[0006] This invention proposes a vehicle localization algorithm based on a particle filter framework, incorporating a lightweight semantic map. The algorithm designs methods for extracting observational features such as points, lines, and bounding boxes from images; it also proposes a multi-modal feature matching loss model for similarity calculation; and finally, vehicle localization is achieved based on the particle filter framework. Experiments on multiple self-collected datasets demonstrate that this invention achieves more reliable localization performance compared to traditional visual SLAM and particle filter methods based on road network maps.

[0007] The steps of the technical solution adopted in this invention are as follows:

[0008] 1) Visual odometry information is estimated based on the binocular images acquired by the camera, and then the initial pose T of the camera is estimated based on the visual odometry information. c ; for the initial pose T of the camera c Perform a coordinate system transformation to obtain the initial pose T in the world coordinate system. w ;

[0009] 2) Denote one of the images in the binocular images as the target image, extract various semantic targets from the target image, obtain the corresponding semantic feature points, and thus obtain the set of semantic feature points of the target image;

[0010] 3) Based on the initial pose T in the world coordinate system w The set of semantic feature points is used to estimate the current pose using a particle filter method.

[0011] In step 2), the semantic targets in the target image are targets above the road surface, arrows, zebra crossings, and lane lines. After extracting the targets above the road surface in the target image using the YOLOv5 method, a detection box is obtained, and the four endpoints of the detection box are used as feature points corresponding to the targets above the road surface.

[0012] After extracting the arrows / zebra stripes on the ground in the target image using the YOLOv5 method, the detection box is obtained. Then, the feature point information within the area covered by the detection box is extracted using the SuperPoint method to obtain the feature points corresponding to the arrows / zebra stripes.

[0013] The target image is segmented using the RESA network to obtain the lane line region. Then, the feature point information within the lane line region is extracted using the SuperPoint method to obtain the feature points corresponding to the lane lines.

[0014] Specifically, 3) refers to:

[0015] 3.1) Initial pose T in world coordinate system w N particles are generated around it;

[0016] 3.2) Based on the lightweight road semantic map, extract and project the local semantic map features of N particles respectively to obtain the projected semantic feature set corresponding to N particles;

[0017] 3.3) Combining the matching loss model, the semantic feature point set and the projected semantic feature set corresponding to N particles are matched respectively to obtain the matching loss cost. The weights corresponding to N particles are calculated based on the matching loss cost, and the current estimated pose is obtained based on the weight of each particle.

[0018] In section 3.1), the initial pose T in the world coordinate system w N particles are generated around it in a Gaussian distribution.

[0019] In step 3.2), for each particle, semantic map feature points within a preset range of the current particle's location in the lightweight road semantic map are extracted to obtain a local semantic map feature point set. The local semantic map feature point set is then projected onto the pixel coordinate system corresponding to the current particle using internal and external parameter matrices to obtain a projected semantic feature set.

[0020] In section 3.3), the pose corresponding to the particle with the largest weight is taken as the current estimated pose.

[0021] In section 3.3), the matching loss model includes the semantic loss of road arrows, the semantic loss of targets above the road, and the semantic loss of lane lines / zebra crossings. The semantic loss of road arrows is calculated by taking the average Euclidean distance of point-to-point matching as the error; the semantic loss of targets above the road is calculated by taking the Euclidean distance loss of point-to-point matching and the error of the IOU measurement box as the error; the semantic loss of lane lines / zebra crossings is calculated by combining the average Euclidean distance error of point-to-point matching with the angle error of the straight line where the feature point is located.

[0022] This invention extracts semantic features from images, matches these image features with semantic map features, and then uses a matching loss model for particle filtering weight calculation. This invention utilizes a lightweight semantic map to effectively control positioning accuracy within the meter range.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1) This invention combines a lightweight semantic feature point map with a particle filter framework for vehicle localization. Reliable localization is achieved using a semantic map with minimal storage requirements.

[0025] 2) This invention can maintain reliable positioning even when the initial positioning error is large or the initial pose of the trajectory is highly uncertain;

[0026] 3) This invention utilizes a multi-modal feature matching loss model to calculate the matching loss of different semantic targets, which is used for particle similarity calculation to improve positioning accuracy.

[0027] In summary, this invention enables reliable localization using lightweight semantic maps. It also exhibits good robustness even under conditions of large initial localization errors or significant uncertainty in the initial pose of the trajectory. Attached Figure Description

[0028] Figure 1 It is a basic framework for localization based on lightweight road semantic maps.

[0029] Figure 2 These are the feature points and the fitted straight line.

[0030] Figure 3 It is the result of actual map feature matching.

[0031] Figure 4 This is a diagram illustrating IOU calculation.

[0032] Figure 5 It is a (L+S+T) combined distance-error diagram.

[0033] Figure 6 It is a (L+A+Z) combination distance-error diagram.

[0034] Figure 7 The first data path-error diagram is shown in (a) for comparison of SLAM methods and (b) for comparison of road network filtering methods.

[0035] Figure 8 The second data path-error diagram is shown in (a) for comparison of SLAM methods and (b) for comparison of road network filtering methods.

[0036] Figure 9 This is a schematic diagram of the location trajectory of data in a certain place.

[0037] Figure 10 This is a schematic diagram of the location trajectory of data from a certain location.

[0038] Figure 11 These are schematic diagrams showing different initial positions. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] like Figure 1 As shown, the semantic map localization method proposed in this invention includes the following steps:

[0041] 1) Visual odometry information is estimated based on the binocular images acquired by the camera, and then the initial pose T of the camera is estimated based on the visual odometry information. c Initial pose T c Located in the camera coordinate system, initial pose T c Includes two-dimensional coordinate information and yaw angle information; initial pose T of the camera c Perform a coordinate system transformation to obtain the initial pose T in the world coordinate system. w In practice, the world coordinate system is the Northeast-Universal (ENU) coordinate system.

[0042] 2) Denote one of the images in the binocular images as the target image, extract various semantic targets from the target image, obtain the corresponding semantic feature points, and thus obtain the set of semantic feature points of the target image;

[0043] In section 2), the semantic targets in the target image are targets above the road surface, arrows, zebra crossings, and lane lines. For each semantic target category, different extraction methods are used according to their different characteristics. For targets above the road surface, the YOLOv5 method is used to extract the targets above the road surface in the target image to obtain detection boxes. The four endpoints of the detection boxes are used as feature points corresponding to the targets above the road surface. Targets above the road surface include signs and traffic lights.

[0044] For arrows and zebra crossings, the YOLOv5 method is used to extract the arrows / zebra crossings on the ground in the target image to obtain the detection box. Then, the SuperPoint method is used to extract the feature point information within the area covered by the detection box to obtain the feature points corresponding to the arrows / zebra crossings.

[0045] For lane lines, the target image is segmented using the RESA network to obtain the lane line region. Then, the feature point information within the lane line region is extracted using the SuperPoint method to obtain the feature points corresponding to the lane lines.

[0046] The SuperPoint network is used to extract feature points for the semantics of road arrows, zebra crossings, and lane lines. For zebra crossings and lane lines, in addition to feature point information, it is also necessary to fit the straight line information where the feature points are located. The fitted semantic straight line information will participate in the subsequent matching loss weight calculation.

[0047] Using the RANSAC algorithm, the straight lines at both ends of the zebra crossing and the dashed lines of the two lanes were fitted respectively, and the results are as follows: Figure 2 As shown, it consists of the following steps:

[0048] S1. Select four points in the dataset (four points are needed to estimate a double line).

[0049] S2. Use these four points to calculate the parameters k1, b1, k2, b2 in the data model y = k1x + b1, y = k2x + b2. Since the slope difference between the two straight lines before and after the zebra crossing is not significant in the image, if the difference between k1 and k2 is too large, return to step one and reselect four points. Additionally, since the slopes of the two straight lines of the lane line have a certain angular difference in the image, to prevent the four points selected in step one from being on the same line, this algorithm presets a threshold for the angle between the two lines. Only when the angle is greater than the threshold can the next step be performed; otherwise, return to S1.

[0050] S3. Input all data points into the model and calculate the number of "interior points". Since a data point may match either of the two lines, if the distance from the point to either line is less than the preset error threshold, then the point is an interior point.

[0051] S4. If the number of internal points is greater than a certain proportion of the total number of points, then the current model is taken as the final fitted model and the iteration ends; otherwise, return to S1.

[0052] In the RANSAC method used, the error threshold set in step S3 is 3, meaning that if the distance from a point to a line is less than 3 pixels, it is considered an inlier. In step four, the inlier rate threshold is set to 80%, meaning that the iteration ends when the number of inliers accounts for more than 80% of the total number of points.

[0053] 3) Based on the initial pose T in the world coordinate system w The set of semantic feature points is used to estimate the current pose using a particle filter method.

[0054] 3) Specifically:

[0055] 3.1) Initial pose T in world coordinate system w N particles are generated around the vehicle, each containing two-dimensional coordinate information and yaw angle information. Since this invention assumes the vehicle is traveling on a level road, each particle contains two-dimensional coordinate information (x, y) and a yaw angle (yaw).

[0056] In 3.1), the initial pose T in the world coordinate system w N particles are generated around it in a Gaussian distribution. satisfy This represents the two-dimensional coordinates of particle i in the world coordinate system. This represents the yaw angle of particle i in the world coordinate system.

[0057] In practice, during the particle initialization process, a Gaussian distribution method is used to initialize the particles, based on the estimated pose T. w The particle generation is centered on the position and initialized with a preset variance. Considering that a small variance might not cover the optimal position, while a large variance reduces semantic constraints and increases computational redundancy for low-precision positions, the variance is set to the lane width (typically 3 meters), and the particle number N is set to 500, taking into account practical road conditions.

[0058] 3.2) Based on the lightweight road semantic map, extract and project the local semantic map features of N particles respectively to obtain the projected semantic feature set corresponding to N particles;

[0059] In section 3.2), for each particle, semantic map feature points within a preset range of the current particle's location in the lightweight road semantic map are extracted to obtain a local semantic map feature point set. This local semantic map feature point set is then projected onto the pixel coordinate system corresponding to the current particle using an intrinsic and extrinsic parameter matrix, resulting in a projected semantic feature set. Specifically, map feature points within a 30-meter range are projected onto the particle's pixel coordinate system. The lightweight road semantic map in this invention includes representative, stable semantic targets with certain geometric properties in the road scene, such as corner features of road arrows, lane lines, zebra crossings, road signs, and traffic lights.

[0060] Specifically, the local semantic map feature points P in the ENU coordinate system are projected onto the particle pixel coordinate system using intrinsic and extrinsic parameter matrices to obtain the projected semantic feature points p. m The local semantic map feature point P satisfies P = [X w ,Y w Z w ], Projection semantic feature point p m Satisfy p m =[u m ,v m The conversion formula is as follows:

[0061]

[0062]

[0063] Among them, X w ,Y w Z w These represent the x-axis, y-axis, and z-axis values ​​of a local semantic map feature point P in the world coordinate system, respectively. m ,v m R' represents the u-axis and v-axis values ​​of a local semantic map feature point P in the pixel coordinate system, R′ represents the rotation matrix, and Z represents the local semantic map feature point P in the pixel coordinate system. c This represents the normalized value, and K represents the camera intrinsic parameter. This represents the x-axis, y-axis, and z-axis values ​​of the i-th particle in the world coordinate system.

[0064] 3.3) Combining the matching loss model, the Hungarian (KM) algorithm is used to perform feature matching between the set of semantic feature points and the projected semantic feature sets corresponding to the N particles, respectively, to obtain the matching loss cost. Based on the matching loss cost, the weights corresponding to the N particles are calculated, and the current estimated pose is obtained based on the weights of each particle. In specific implementation, the pose corresponding to the particle with the largest weight is taken as the current estimated pose.

[0065] Specifically:

[0066] Each semantic class point set projected onto the semantic map in particle coordinates (representing the i-th point of semantic class l) and the set of semantic feature points extracted from the image using a network. Matching is performed on (the j-th point representing semantic class l). This matching process can be implemented using the KM algorithm, where the weight w ij Indicating the same semantic category and The weights are defined as the pixel coordinates of the two points. Euclidean distance Since a smaller Euclidean distance indicates a closer distance between two points, the minimum value is selected for matching during the matching process. The KM algorithm matching result is as follows: Figure 3 As shown, triangles represent feature points extracted from the image, and circles are the corresponding points projected onto the pixel coordinate system from the map feature points. Matching results are represented by connecting lines. It can be seen that this invention still achieves good results even in real-world scenarios where there are missed or false detections of image semantic feature points.

[0067] After point matching is completed, the matching error needs to be calculated to measure the similarity between the current observation and the map, which is then used to estimate the camera pose correction. Based on the characteristics of semantic elements of each category, this invention designs a multi-modal matching loss model, which can better calculate the matching cost.

[0068] The matching loss model includes semantic loss for road arrows, semantic loss for targets above the road, and semantic loss for lane lines / zebra crossings. The semantic loss for road arrows is calculated by using the average Euclidean distance of point-to-point matching as the error. The semantic loss for targets above the road (i.e., rectangular semantic objects such as signs and traffic lights) is calculated by using the Euclidean distance loss of point-to-point matching and the IOU error measurement box error as the error. The semantic loss for lane lines / zebra crossings is calculated by combining the average Euclidean distance error of point-to-point matching with the angle error of the line where the feature point is located.

[0069] The average Euclidean distance of the point-to-point matching is the basic matching error, and the calculation formula is as follows:

[0070]

[0071] Where n1 represents the set of semantic class points in semantic category l. and semantic feature point set The number of matching points and Each represents a set of semantic class points. The x and y coordinates of the i-th feature point pixel and Representative semantic feature point set The x and y coordinates of the j-th feature point pixel, de l λ represents the matching error of semantic category l in the pixel coordinate system, and λ1 represents the point error coefficient.

[0072] The formulas for calculating the corresponding straight-line angle error and the corresponding straight-line spacing error of the matching points are as follows:

[0073]

[0074]

[0075] Where n2 represents the set of semantic class points in semantic category l. and semantic feature point set The number of lines matched and Each represents a set of semantic class points. The slope and offset of the i-th line segment. and They represent semantic feature point sets respectively. The slope and offset of the j-th line segment, arctan(·) is the arctangent function, || represents the absolute value, i.e. l λ2 represents the slope matching error of semantic category l in the pixel coordinate system, while t represents the line segment error coefficient. ij x1 represents the error in the spacing between matching lines, and W represents the width coefficient and the image frame width.

[0076] Inspired by the IOU metric in object detection networks such as YOLO, this invention uses IOU error to measure the bounding box, as shown in the following formula:

[0077]

[0078] Where n3 represents the set of semantic class points in semantic category l. and semantic feature point set The number of matching card-shaped items, and Each represents a set of semantic class points. The pixel length and width of the i-th card-shaped object. and They represent semantic feature point sets respectively. The pixel length and width of the j-th sign-shaped object, se l λ3 represents the effective area matching error of semantic category l in the pixel coordinate system, and λ3 represents the effective area error coefficient.

[0079] The definition of matching IOU is:

[0080]

[0081] x1 = max(x a1 ,x b1 ),y1=max(y a1 ,y b1 ),x2=min(x a2 ,x b2 ),y2=min(y a2 ,y b2 )

[0082] Among them, a l a w Represents each semantic class point set The semantic box's pixel length and width, b l b w Represents a set of semantic feature points The semantic box's pixel length and width, and each semantic class point set The coordinates of the top left corner of the sign are (x a1 ,y a1 The coordinates of the lower right corner point are (x a2 ,y a2 Semantic feature point set The coordinates of the top left corner of the sign are (x b1 ,y b1 ), bottom right corner point (x b2 ,y b2 x1, y1, x2, y2 represent the x-axis and y-axis coordinates of the top-left and bottom-right corners of the overlapping box, respectively. Figure 4 As shown.

[0083] Since the number of semantic matching pairs in each class can be very large, the mean absolute error is used to measure the error of each semantic category, and different weighting coefficients are assigned to different semantic categories. Therefore, the total map matching error is as follows:

[0084] er=λ a *de a +λ lane *(de lane +ie lane )+λ z *(de z +ie z )+λ t *(de t +se t )+λ s *(de s +se s )

[0085] Where, λ a , λ lane , λ z , λt , λ s This represents the semantic weight coefficients for road arrows, lane lines, zebra crossings, traffic lights, and signs. The semantics of road arrows only include matching point errors; lane lines and zebra crossings consider both point errors and line segment slope errors; and traffic lights and signs consider both matching point errors and effective overlapping area errors. This loss model considers the average loss error of each semantic piece of information. Since different semantic pieces of information provide different levels of location confidence during localization, the loss weights for each semantic piece of information are also set differently.

[0086] The experimental datasets include two datasets for different locations. Both scenarios contain rich road semantic information, and the roads are wide and contain dynamic objects, posing a significant challenge to the localization algorithm. The first dataset is 1.5 km long, using RTK-GPS provided by a certain company as the ground truth for localization, with centimeter-level accuracy. The second dataset is 2.3 km long, using GPS data from a self-collected system as the ground truth, with an accuracy of approximately 0.5 meters. The input data consists of stereo images and a self-built lightweight semantic map as prior information. To validate the experimental results, GPS timestamps and image timestamps were pre-aligned for times with available GPS data, and error analysis was performed using the aligned data.

[0087] Since the true positioning value does not include information such as pitch and yaw angles, only two-dimensional position is used to measure the positioning error, where the ENU coordinate axis is represented by (x, y). Assume the true value at time k has coordinates (x, y). k ,y k The algorithm predicts that the result at time k is (x′). k ,y′ k This invention analyzes positioning errors from three aspects: longitudinal error, lateral error, and overall error, as shown in the following formula:

[0088]

[0089] Among them, e x The lateral error at time k is defined as the positional error value perpendicular to the true trajectory's direction of travel, and the longitudinal error is e. y The positioning error is defined as the driving direction parallel to the true trajectory. The total error e is defined as the Euclidean distance between the estimated value and the true value. a represents the slope of the straight line and b represents the deviation of the straight line. b = y k -ax k Horizontal average error Longitudinal average error and overall average error This represents the average error over all time points.

[0090] SA and semantic feature ablation experiments

[0091] Semantic information is a crucial source of the localization fusion method of this invention. Different semantic information weights are related not only to their quantity distributed along the road, but also to their location and observable range. Since sign information is located above the road, it is easier to observe, and the confidence level of semantic extraction at long distances is higher, contributing more significantly to localization accuracy. Therefore, it is given the largest weight, λ. s =λ t =5. Road surface arrows are only highly reliable when the distance is close enough, therefore a small weight is assigned to them, λ. a =1. Lane lines can effectively correct lateral distances, and point-to-point matching can also constrain longitudinal positions in areas with dashed lines; therefore, a weight λ is set. lane =2. Zebra crossings generally appear at intersections, and because their semantic occurrence is relatively infrequent, a weight λ is assigned. z =3. In addition, the weights λ1, λ2, and λ3 for points, lines, and signs are set to 1, 2, and 2, respectively.

[0092] This invention divides road semantic information into road surface semantic information and road surface semantic information, and conducts semantic-based filtering experiments on a first location dataset and a second location dataset, respectively. Since lane line information is present throughout the entire dataset and can correct the localization results within a certain error range, the experiments are divided into two semantic combinations: lane line + road surface arrow + zebra crossing (L+A+Z) and lane line + sign + traffic light (L+S+T). The experimental results are shown in Tables 1 and 2.

[0093] As can be seen from the table, the (L+S+T) combination has better longitudinal accuracy than the (L+A+Z) combination in both datasets. This is because the sign semantic information is located above the road and can be observed from a greater distance, thus providing a wider range of constraints on the location information. In contrast, under the same conditions, information such as road arrows and zebra crossings is easily obscured by dynamic objects on the road, and even when observed from a distance, their shapes are not necessarily accurate; reliable observation information is only provided when they are close by.

[0094] Table 1 shows the average semantic localization error (in meters) for different datasets in a certain location.

[0095]

[0096] Table 2 shows the average semantic localization error (in meters) for different locations in the second dataset.

[0097]

[0098] To more intuitively analyze the impact of different combinations of semantic information on positioning accuracy, this invention presents experimental error results using a second dataset of a location that is richer in semantic information, such as... Figures 5 to 6 As shown in the figure, vertical lines are used to mark the semantic information of the corresponding routes, and the dots represent the total error in the corresponding routes. Figure 5 The positions of the black and gray vertical lines indicate the locations of the signs and traffic lights within the route, respectively. Figure 6 The positions of the black and gray vertical lines indicate the location of the zebra crossing within the route, respectively.

[0099] from Figure 5 and Figure 6 As can be seen, the error decreases slightly before the semantic information location, and the (L+A+Z) combination shows a steeper decrease in error than the (L+S+T) combination. This is because the (L+S+T) combination can be observed from a greater distance and thus be constrained earlier, resulting in a gentler trend. Zebra crossings are generally located at intersections, and some intersections lack traffic lights or signs. Therefore, at turning intersections, where visual odometry is most prone to drift, the (L+A+Z) combination corrects earlier than (L+S+T), leading to better lateral accuracy.

[0100] The filtering experiment results, which integrate all semantic information, effectively combine the semantic structural information in the road, thus obtaining more accurate results.

[0101] SB, comparative analysis of positioning results

[0102] The coarse pose is estimated using the visual odometry portion of ORB-SLAM2, and the localization result is obtained by fusing self-built semantic map information through particle filtering. The localization method of this invention is compared with ORB-SLAM2 and road network map-based filtering methods. Comparative experiments were conducted on a first and a second local dataset, and the results are shown in Tables 3 and 4. The tables show that the semantic map-based filtering method proposed in this invention significantly reduces the coarse pose error (VO), achieving meter-level accuracy. Compared with road network filtering, the method of this invention is also more advantageous. Furthermore, the error of the method of this invention is generally within 3 meters, demonstrating that the lightweight semantic map filtering method of this invention has considerable localization accuracy and is more meaningful in practical localization. Moreover, because lane line constraints exist in most scenarios, the lateral accuracy is even higher, reaching decimeter or even centimeter levels in both datasets.

[0103] Table 3 shows the average positioning error and the proportion of errors within 3 meters for each method in the first dataset.

[0104]

[0105] Table 4 shows the average positioning error and the proportion of errors within 3 meters for each method in the second dataset.

[0106]

[0107] Figures 7 to 8 The localization errors of two datasets along the path length are shown, and the semantic map filtering method of this invention is compared with the path-error maps of the ORB-SLAM2 method and the road network map filtering method. It can be seen that the error of ORB-SLAM2 is very small in areas with loops, such as... Figure 8 (Left) At a distance of 1000 meters, however, "loopbacks" are rare in real-world large-scale localization scenarios. Therefore, in most cases, SLAM errors drift as distance increases. The filtering method based on road network maps offers a significant improvement over SLAM. Its accuracy relies not only on the accuracy of the VO itself but also on correcting the trajectory within the current lane using the road network map. The image shows a marked decrease in localization error at intersections and turns. Figure 8 (Right) Positions at distances of 300 meters, 500 meters, and 1000 meters. However, in real-world scenarios involving long straight stretches, this method can only correct the lateral position using lane lines, without any longitudinal constraints. In contrast, the semantic map filtering method of this invention effectively addresses the problems faced by both methods. Based on the rich semantic information inherent in the road scene, it effectively solves the positioning accuracy problem in real-world road scenarios.

[0108] Figure 9 and Figure 10 The localization trajectories of the proposed method and the ORB-SLAM2 method are shown in two datasets. The left figure shows the trajectory of the proposed method in the semantic map, and the right figure shows a comparison of the proposed method's semantic map filtered trajectory, GPS trajectory, and ORB-SLAM trajectory. As can be seen from the figures, with prior information provided by the semantic map, the proposed method can reliably locate the target even in road segments where the VO (Voice of Root) drifts significantly.

[0109] SC, Experiments with Uncertain Starting Position

[0110] To simulate the accuracy and robustness of the positioning method in this invention under uncertain initial position conditions, a random error range of 15 meters is introduced from the GPS initial position as the vehicle's starting point to simulate possible initial positions. As the vehicle moves, the number of filters increases. This invention uses the number of filters required to achieve a positioning accuracy of 1 meter as the experimental evaluation metric, conducting experiments at different locations on the map, such as... Figure 11 As shown in the figure, the red dots indicate the true value positions at the beginning of each experiment. The experimental results are shown in Table 5.

[0111] Table 5 shows the number of filters required to achieve a 1-meter error at each position when the initial position is uncertain.

[0112]

[0113] The experiments show that in road sections with rich semantic information, fewer filtering iterations are required to achieve a filtering accuracy of 1 meter, such as in locations 6 and 7. Furthermore, even when the initial position is uncertain, the method of this invention can converge the result to the correct position, indicating that the method is also applicable to scenarios with uncertain starting positions and possesses robustness.

[0114] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A localization method based on a lightweight road semantic map, characterized in that, Includes the following steps: 1) Visual odometry information is estimated based on the binocular images captured by the camera, and then the initial pose of the camera is estimated based on the visual odometry information. ; Initial pose of the camera Perform a coordinate system transformation to obtain the initial pose in the world coordinate system. ; 2) Denote one of the images in the binocular images as the target image, extract various semantic targets from the target image, obtain the corresponding semantic feature points, and thus obtain the set of semantic feature points of the target image; 3) Based on the initial pose in the world coordinate system And a set of semantic feature points, using particle filtering method to estimate the current estimated pose; In step 2), the semantic targets in the target image are targets above the road surface, arrows, zebra crossings, and lane lines. After extracting the targets above the road surface in the target image using the YOLOv5 method, a detection box is obtained, and the four endpoints of the detection box are used as the feature points corresponding to the targets above the road surface. After extracting the arrows / zebra stripes on the ground in the target image using the YOLOv5 method, the detection box is obtained. Then, the feature point information within the area covered by the detection box is extracted using the SuperPoint method to obtain the feature points corresponding to the arrows / zebra stripes. The target image is segmented using the RESA network to obtain the lane line region. Then, the feature point information within the lane line region is extracted using the SuperPoint method to obtain the feature points corresponding to the lane lines. Specifically, 3) refers to: 3.1) Initial pose in world coordinate system Surrounding generation One particle; 3.2) Based on the lightweight road semantic map, for Each particle is individually subjected to local semantic map feature extraction and projection to obtain... The set of projected semantic features corresponding to each particle; 3.3) Combining the matching loss model with the semantic feature point set... After performing feature matching between the projected semantic feature sets corresponding to each particle, the matching loss cost is obtained, and the result is calculated based on the matching loss cost. The weights corresponding to each particle are used to estimate the current pose.

2. The localization method based on a lightweight road semantic map according to claim 1, characterized in that, In section 3.1), the initial pose in the world coordinate system The surrounding area is generated according to a Gaussian distribution. One particle.

3. The localization method based on a lightweight road semantic map according to claim 1, characterized in that, In section 3.2), for each particle, semantic map feature points within a preset range of the current particle's location in the lightweight road semantic map are extracted to obtain a local semantic map feature point set. The local semantic map feature point set is then projected onto the pixel coordinate system corresponding to the current particle using internal and external parameter matrices to obtain a projected semantic feature set.

4. The localization method based on a lightweight road semantic map according to claim 1, characterized in that, In section 3.3), the pose corresponding to the particle with the largest weight is taken as the current estimated pose.

5. The localization method based on a lightweight road semantic map according to claim 1, characterized in that, In section 3.3), the matching loss model includes the semantic loss of road arrows, the semantic loss of targets above the road, and the semantic loss of lane lines / zebra crossings. The semantic loss of road arrows is calculated by taking the average Euclidean distance of point-to-point matching as the error; the semantic loss of targets above the road is calculated by taking the Euclidean distance loss of point-to-point matching and the error of the IOU measurement box as the error; the semantic loss of lane lines / zebra crossings is calculated by combining the average Euclidean distance error of point-to-point matching with the angle error of the straight line where the feature point is located.

Images