Vehicle semantic map matching and positioning method and device based on space-based remote sensing image

Abstract

The invention discloses a vehicle semantic map matching and positioning method and device based on space-based remote sensing images, and belongs to the field of vehicle autonomous positioning. The method comprises the following steps: firstly, performing multi-scale cutting on a space-based remote sensing image to construct an image pyramid, identifying roads and buildings by using a deep learning model, and constructing a lightweight semantic map through vectorization extraction; an upper layer matches a vehicle track with a road center line vector through adaptive sliding window matching, and a track feature detection function and an incremental search mechanism are introduced to realize rapid coarse positioning; and the lower layer takes the coarse positioning result as the center, the laser point cloud is matched with the building contour vector by adopting a simplified ICP algorithm in the candidate area, only translation is estimated by fixed rotation, and finally, RANSAC conversion is performed once to realize accurate positioning. According to the method, the problem of quick and accurate positioning in a non-cooperative area, a strange scene and a satellite denial condition is solved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of vehicle autonomous localization and computer vision, and proposes a vehicle semantic map matching and localization method and device based on space-based remote sensing images. Background Technology

[0002] Common map data acquisition methods include manual acquisition and computer-aided automated acquisition. Manual data acquisition requires on-site measurements using theodolites and other positional reference devices to record the coordinates and other necessary information of spatial objects, and then creating a map based on the measured data. While this method produces detailed and accurate maps, it is labor-intensive, time-consuming, and cannot meet the needs for rapid application in large-scale, unfamiliar scenarios, nor can it be applied to non-cooperative areas.

[0003] In recent years, with the development of remote sensing and image processing technologies, scholars both domestically and internationally have conducted extensive research on automated map data acquisition methods based on remote sensing images. Compared with manual acquisition methods, automated extraction of map elements such as roads and buildings based on remote sensing images does not require on-site measurement, has shorter mapping time, and greatly reduces reliance on manual labor. Common map extraction solutions can be divided into two steps: the first step is image segmentation, which involves generating pixel-level labels based on satellite remote sensing images, defining the target extraction task as a type II image segmentation task; the second step is vector extraction, which involves vectorizing the segmented targets, such as extracting center lines and contours, and storing the vectorized representation of the targets in an electronic map.

[0004] The principle of the correlation map matching algorithm is to first assemble the collected vehicle latitude and longitude location information into a vehicle location trajectory in sequence, then perform correlation matching between the location trajectory and the curve trajectory of the path to be matched found from the road vector map. By calculating the correlation coefficient between the two, the similarity between the location trajectory and the path to be matched is determined, and finally, the path with the highest similarity is taken as the matching result. This algorithm is very suitable for complex scenarios such as curves and intersections, but the matching effect is only good when the vehicle's driving trajectory has obvious flight path characteristics. If the vehicle's heading does not change much along the entire trajectory, the matching accuracy will be significantly reduced.

[0005] The Iterative Closest Point (ICP) algorithm is the most classic and well-known fine-grained registration algorithm for point cloud matching. The core idea of ​​this algorithm is to iteratively find the optimal rigid body transformation between two point clouds, minimizing the sum of the distances between corresponding points in the transformed source and target point clouds, ultimately achieving high-precision alignment of the two point clouds. While this algorithm offers high accuracy, it heavily relies on the initial pose, requiring that the initial positions of the two point clouds not deviate too much, and its convergence speed is slow when dealing with a large number of point clouds.

[0006] Navigation and positioning are among the most crucial functions of autonomous and driverless vehicles. Current vehicle positioning systems primarily rely on satellite navigation or inertial navigation. However, in urban environments, satellite signals are not always available due to limitations imposed by tall buildings and roadside trees. In contrast, inertial navigation systems offer advantages such as high stability, high output frequency, full autonomy, and rich navigation information, making them more suitable for navigation and positioning in urban environments.

[0007] In large-scale, long-range applications, the positioning results obtained by inertial navigation recursive calculations suffer from accumulated positioning errors over time, necessitating the introduction of other methods for auxiliary correction. Commonly used methods include satellite navigation positioning information assistance and landmark information assistance. However, satellite navigation, as a radio system based on space satellites, is easily affected by external factors such as terrain and electronic interference, leading to positioning failures. Landmarks require prior on-site mapping, and active marking technologies such as magnetic track pin identification require the pre-deployment of dedicated equipment. While these methods can effectively suppress error divergence in inertial navigation and improve positioning accuracy, they cannot be used in situations without satellite positioning signals or on-site mapping, hindering rapid application in non-cooperative areas and unfamiliar scenarios. Summary of the Invention

[0008] To address the above technical problems, this invention proposes a vehicle semantic map matching and localization technology method and apparatus based on space-based remote sensing images, comprising two parts: semantic map construction and vehicle matching and localization. In the semantic map construction part, a space-based remote sensing image of the application area is input. First, a deep learning model is used to segment roads and buildings, and these two types of map elements are vectorized to construct a lightweight semantic map of the application area. In the vehicle matching and localization part, combining the global nature of road map matching and the local accuracy of building point cloud matching, an innovative hierarchical localization strategy is designed: first, a preliminary localization is performed using a road map matching method to determine the possible coarse location and heading angle of the vehicle; then, building point cloud matching is performed with this location as the center to obtain the accurate vehicle localization result. The specific technical solution is as follows:

[0009] A vehicle semantic map matching and localization method based on space-based remote sensing images includes the following steps:

[0010] Step 1: Acquire space-based remote sensing images of the application area and cut them into small remote sensing images of various scales to construct an image pyramid;

[0011] Step 2: Perform road identification on the remote sensing images at each scale to obtain the corresponding binary road images. Then, through stitching and morphological processing, obtain the binary road images of the application area at each scale. Extract the road centerline vector from the binary road images, fuse the extraction results from multiple scales, and perform optimization processing to obtain usable road centerline vector data.

[0012] Step 3: Perform building recognition on the remote sensing images at each scale to obtain the corresponding polygon vertex coordinate sequence, stitch them together to obtain the building outline vector of the application area at each scale, fuse the results of multiple scales, and perform optimization processing to obtain usable building outline vector data.

[0013] Step 4: Merge the road centerline vector data and the building outline vector data to obtain a lightweight semantic map;

[0014] Step 5: Acquire vehicle trajectory data and LiDAR point cloud data, and ensure timestamp alignment;

[0015] Step 6: Perform map matching and localization based on adaptive sliding window matching. Preprocess the vehicle trajectory data and the road centerline vector data of the semantic map, and match the vehicle trajectory sequence with the path sequence to be matched to obtain the coarse localization of the vehicle.

[0016] Step 7: Using the coarse positioning result as the center, define the candidate area, transform the laser point cloud data to the global coordinate system, match the point cloud projection points in the candidate area with the building outline vector data of the semantic map, and obtain the transformation from point cloud projection to building outline. Perform the same transformation on the vehicle coarse positioning to obtain the accurate positioning result.

[0017] Preferably, in step 1, a multi-scale representation of the same target is obtained by cutting the high-resolution remote sensing image of the region into multiple different sizes.

[0018] Preferably, in step 2, the building vector data extraction module is used to segment, extract, and vectorize the building map elements. In the building vector data extraction module, the input large-area high-resolution image is first cut into different sizes to construct an image pyramid; for a remote sensing small image of a certain scale, it is input into the building segmentation model for building recognition to obtain the corresponding polygon vertex coordinate sequence, and the building segmentation results are stitched together to obtain the building outline vector; finally, the building outline vectors of multiple scales are fused to obtain usable building outline vector data.

[0019] Preferably, in step 2, the road vector data extraction module is used to segment, extract, and vectorize road map elements. In the road vector data extraction module, the input large-area high-resolution image is first segmented into different sizes to construct an image pyramid; a remote sensing small image at a certain scale is input into the road segmentation model for road recognition, and the road segmentation results are stitched and superimposed to obtain the road binary image of the area to be tested. After operations such as centerline breakage and reconnection, the road centerline vector is obtained; finally, the road centerline vectors at multiple scales are fused to obtain the road centerline vector data.

[0020] Preferably, the adaptive sliding window matching in step 5 includes: introducing a trajectory feature detection function to detect the features of vehicle trajectories, using an adaptive variable-length window for matching, and ensuring that trajectory features exist in the sequence to be matched; when searching for the path to be matched, an incremental search method is used, and when the historical matching results are continuous and the correlation coefficient is greater than the threshold, an incremental query is performed based on the previous matching path, otherwise a traversal query is performed.

[0021] Preferably, the two parts being matched are preprocessed vehicle trajectory sequences. The set of paths to be matched, with the length of the matching window being the length of the vehicle trajectory sequence. The window sliding length is The matching process involves calculating and comparing the correlation coefficients of the heading angle sequences to find the best matching result.

[0022] Preferably, the track feature detection function is as follows: The sequence of vehicle waypoints to be detected is as follows:

[0023] ① Extract the first and last points of the flight path and connect them. This line segment serves as the reference line segment for detection;

[0024] ② Calculation points Distance to the reference line segment ,like If the value is greater than the pre-set threshold D, the vehicle's trajectory is considered to have obvious trajectory characteristics, and the detection is completed; otherwise, continue to detect the next point.

[0025] ③ Repeat step ②, starting from the second point. The second to last point was detected. If the distance from these points to the reference line segment is not greater than the threshold D, then the vehicle track is considered to have no obvious track characteristics, and the detection ends.

[0026] A vehicle semantic map matching and localization device based on space-based remote sensing images, comprising:

[0027] The image pyramid construction module acquires space-based remote sensing images of the application area, cuts them into small remote sensing images of various scales, and constructs an image pyramid.

[0028] The road vector data extraction module performs road identification on remote sensing small images at each scale to obtain corresponding road binary images. Then, through stitching and morphological processing, it obtains road binary images of the application area at each scale. The road centerline vector is extracted from the road binary images, the extraction results at multiple scales are fused, and optimization processing is performed to obtain usable road centerline vector data.

[0029] The building vector data extraction module performs building recognition on remote sensing images at each scale, obtains the corresponding polygon vertex coordinate sequence, stitches them together to obtain the building outline vector of the application area at each scale, fuses the results of multiple scales, and performs optimization processing to obtain usable building outline vector data.

[0030] The semantic map model building module merges road centerline vector data and building outline vector data to obtain a lightweight semantic map.

[0031] The data acquisition module acquires vehicle trajectory data and LiDAR point cloud data, and ensures timestamp alignment.

[0032] The coarse localization module performs map matching and localization based on adaptive sliding window matching. It preprocesses the vehicle trajectory data and the road centerline vector data of the semantic map, and matches the vehicle trajectory sequence with the path sequence to be matched to obtain the coarse localization of the vehicle.

[0033] The fine positioning module defines the candidate area with the coarse positioning result as the center, transforms the laser point cloud data to the global coordinate system, matches the point cloud projection points in the candidate area with the building outline vector data of the semantic map, and obtains the transformation from point cloud projection to building outline. The same transformation is performed on the vehicle coarse positioning to obtain the accurate positioning result.

[0034] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method.

[0035] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement the method described thereon.

[0036] The present invention has the following beneficial effects:

[0037] (1) Before using a deep learning model to extract targets from remote sensing images, this invention adds a step of constructing an image pyramid. Compared with traditional map extraction solutions that directly segment images, this invention cuts high-resolution remote sensing images of large areas into segments of different sizes, thereby obtaining multi-scale representations of the same target and improving the problem of omissions in single-scale recognition.

[0038] (2) This invention considers that traditional correlation map matching algorithms only achieve good matching results when the vehicle's trajectory has obvious track features. If the vehicle's heading does not change much along the entire trajectory, the matching accuracy will be significantly reduced. The map matching and positioning method based on adaptive sliding window matching proposed in this invention introduces a track feature detection function and uses an adaptive variable-length window for matching to ensure that track features exist in the sequence to be matched. This solves the problem that traditional correlation map matching algorithms are only suitable for road segments with relatively obvious road information features. When searching for the path to be matched, an incremental search method is added, which makes full use of historical matching information, shortens the time required to search for the path to be matched in complex road network areas, and greatly improves the real-time performance of matching.

[0039] (3) When using the iterative nearest point algorithm alone, the quality of the matching result heavily depends on the initial pose, requiring that the initial positions of the two point clouds not deviate too much, and the convergence speed is slow when the number of point clouds is large. This invention proposes a hierarchical localization strategy that combines road map matching and building point cloud matching. The result of road map matching serves as a global position constraint for building point cloud matching, reducing the number of candidate positions while providing a better initial pose for point cloud matching, thus avoiding point cloud matching from getting trapped in local optima; building point cloud matching compensates for the insufficient accuracy of road matching, further refining vehicle localization and significantly improving the accuracy and robustness of vehicle localization in complex environments.

[0040] (4) This invention constructs a lightweight semantic map model using readily available space-based remote sensing images, solving the problem that existing methods cannot be quickly applied in non-cooperative areas or unfamiliar scenarios; and solves the problem that existing methods cannot achieve rapid and accurate positioning under satellite-denied conditions by using a hierarchical positioning strategy that combines road map matching and building point cloud matching. Under satellite navigation denial conditions, this invention can be quickly applied without on-site data collection, enabling large-scale, long-range autonomous and accurate positioning, which is of great significance for enhancing the autonomous and accurate positioning capabilities of vehicles and reducing the cost and assurance requirements of positioning correction. Attached Figure Description

[0041] Figure 1 This is a system architecture diagram for a vehicle semantic map matching and localization technology based on space-based remote sensing images;

[0042] Figure 2 This is a schematic diagram of an image pyramid.

[0043] Figure 3 Schematic diagram for matching sliding windows;

[0044] Figure 4 Schematic diagram of vehicle trajectory feature detection function;

[0045] Figure 5 The flowchart shows the process of searching for a matching path. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, this invention adopts the following technical solution.

[0047] The present invention provides a vehicle semantic map matching and localization method based on space-based remote sensing images, which specifically includes the following steps:

[0048] Step 1: Acquire space-based remote sensing images of the application area and cut them into small remote sensing images of various scales to construct an image pyramid;

[0049] Step 2: Perform road identification on the remote sensing images at each scale to obtain the corresponding binary road images. Then, through stitching and morphological processing, obtain the binary road images of the application area at each scale. Extract the road centerline vector from the binary road images, fuse the extraction results from multiple scales, and perform optimization processing to obtain usable road centerline vector data.

[0050] Step 3: Perform building recognition on the remote sensing images at each scale to obtain the corresponding polygon vertex coordinate sequence, stitch them together to obtain the building outline vector of the application area at each scale, fuse the results of multiple scales, and perform optimization processing to obtain usable building outline vector data.

[0051] Step 4: Merge the road centerline vector data and the building outline vector data to obtain a lightweight semantic map with good continuity and integrity;

[0052] Step 5: Acquire vehicle trajectory data and LiDAR point cloud data, and ensure timestamp alignment;

[0053] Step 6: Perform map matching and localization based on adaptive sliding window matching: preprocess the vehicle trajectory data and the road centerline vector data of the semantic map, match the vehicle trajectory sequence with the path sequence to be matched, and obtain the coarse localization of the vehicle.

[0054] Step 7: Using the coarse positioning result as the center, define the candidate area, transform the laser point cloud data to the global coordinate system, match the point cloud projection points in the candidate area with the building outline vector data in the semantic map, and obtain the transformation from point cloud projection to building outline. Perform the same transformation on the vehicle coarse positioning to obtain the precise positioning result.

[0055] More specifically, this invention provides a vehicle semantic map matching and localization method based on space-based remote sensing images, comprising two parts: semantic map construction and vehicle matching and localization. The overall system architecture is as follows: Figure 1 As shown.

[0056] The semantic map construction section segments, extracts, and vectorizes two types of map elements: roads and buildings. It includes two modules: building vector data extraction and road vector data extraction. In the building vector data extraction module, the input large-area high-resolution image is first segmented at different sizes to construct an image pyramid. A small remote sensing image at a certain scale is input into the building segmentation model for building recognition, obtaining the corresponding polygon vertex coordinate sequence. These sequences are then stitched together to obtain the building outline vectors of the application area at each scale. The results from multiple scales are fused and optimized to obtain usable building outline vector data. In the road vector data extraction module, the input large-area high-resolution image is first segmented at different sizes to construct an image pyramid. A small remote sensing image at a certain scale is input into the road segmentation model for road recognition. The road segmentation results are then stitched and overlaid to obtain a binary road image of the area to be tested. Operations such as centerline breakage and reconnection are performed to obtain the road centerline vectors of the application area at each scale. The road centerline vectors from multiple scales are fused to obtain usable road centerline vector data for subsequent road map matching. Finally, the road centerline vector data and building outline vector data are merged to obtain a lightweight semantic map with good continuity and integrity.

[0057] The vehicle matching and localization section innovatively designs a hierarchical localization strategy combining road map matching and building point cloud matching, comprising two modules: road map matching and building point cloud matching. In the road map matching module, an adaptive sliding window matching-based map matching and localization method is proposed: first, the original vehicle trajectory data is preprocessed to obtain a vehicle trajectory sequence for matching; then, a path to be matched is searched on the known road vector map, using the current vehicle position measured by inertial navigation to search for candidate road segments, obtaining the road path to be matched; finally, the vehicle heading sequence with trajectory features is matched with multiple equal-length road heading angle sequences using an adaptive length sliding window, and the road segment traveled and the current position are determined based on the correlation coefficient, obtaining the vehicle's coarse localization point and heading angle. A candidate region is constructed centered on the coarse localization point. In the building point cloud matching module, for each candidate position, the vehicle point cloud data is transformed to a global coordinate system based on the vehicle's orientation. The transformed vehicle point cloud is matched with the building outline vector data. Combining the global nature of road map matching and the local accuracy of building point cloud matching, the precise vehicle localization result is finally obtained.

[0058] Furthermore, multi-scale target segmentation based on image pyramids specifically involves:

[0059] Before segmenting roads or buildings, an image pyramid was constructed to cut high-resolution remote sensing images of large areas into different sizes, thereby obtaining multi-scale representations of the same target. This reduces the impact of scale ambiguity on the recognition results and improves the problem of omissions in single-scale recognition.

[0060] In image processing, scale ambiguity refers to the fact that computers can only perceive the pixel size of an object in an image, but cannot infer the object's actual physical size from this. Taking road segmentation as an example, if a road occupies different pixel ranges in an 8192 pixel × 8192 pixel segmentation size than in a 16384 pixel × 16384 pixel segmentation size, although it is the same road, it appears "different sizes" to the computer. Figure 2 As shown. After the road segmentation model is trained, the parameters and sizes of the filters and convolution kernels used in the model are fixed, and the extracted features are only applicable to detecting roads with pixel sizes within a certain range. Since the widths of different roads vary greatly, if only a single scale of recognition is performed on the remote sensing image, the recognition results will have many omissions.

[0061] To address the aforementioned problems, this invention constructs an image pyramid. While maintaining a fixed observation window, it segments the remote sensing image using various different sizes, thereby obtaining representations of the remote sensing image at different scales (different resolutions), and also achieving multi-scale representations of the same target. Figure 2For example, the original image is a wide main road with a narrower branch road. After being segmented at different sizes A and B, multi-scale representations of the main road and the branch road are obtained, allowing the road segmentation model to identify both roads with significant width differences. Even if the pixel range of the branch road in segmentation result (a) is too small to be identified as a road, its pixel range in segmentation result (b) is appropriate; similarly, even if the pixel range of the main road in segmentation result (b) is too large to be identified as a road, its pixel range in segmentation result (a) is appropriate. The construction of the image pyramid increases the likelihood of the target being identified by the deep learning segmentation model, greatly reducing the impact of scale ambiguity on the recognition results.

[0062] This invention innovatively designs a hierarchical positioning strategy that combines road map matching and building point cloud matching in the vehicle matching and positioning part. It includes two modules: road map matching and building point cloud matching. First, a map matching and positioning method based on adaptive sliding window matching is used to match the vehicle's driving trajectory with the road centerline vector data to obtain a coarse positioning of the vehicle. Then, based on the coarse positioning of the vehicle, the laser point cloud data is matched with the building outline vector data to obtain the precise positioning of the vehicle.

[0063] Road map matching offers good global continuity, but its accuracy is limited by the extraction precision of road centerline vector data. Building point cloud matching provides high accuracy but heavily relies on the initial pose and has a slow convergence speed. This method combines the advantages of both approaches through a progressive localization approach, moving from coarse global localization to precise local localization. The results of road map matching serve as global positional constraints for building point cloud matching, reducing the number of candidate locations while providing a better initial pose and preventing the matching from getting trapped in local optima. Building point cloud matching compensates for the insufficient accuracy of road map matching, further refining vehicle localization and significantly improving the accuracy and robustness of vehicle localization in complex environments.

[0064] The map matching and localization method based on adaptive sliding window matching performs adaptive variable-length sliding window matching between the vehicle's relative heading angle sequence and multiple equal-length road heading angle sequences to be matched. It then determines the vehicle's traveled road segment and current position based on the correlation coefficient, correcting the vehicle's localization. Compared to traditional correlation map matching algorithms, this method introduces a track feature detection function and uses an adaptive variable-length window for matching, ensuring that track features exist in the sequence to be matched. This solves the problem that traditional correlation map matching algorithms only achieve good matching results when the vehicle's trajectory has obvious track features. Furthermore, this method adds an incremental search approach when searching for matching paths, fully utilizing historical matching information and shortening the time required to search for matching paths in complex road network areas, significantly improving the real-time performance of the matching.

[0065] Sliding window matching is a method that dynamically maintains a window, slides the window along the sequence, and examines the data within the window sequentially. The two parts being matched are preprocessed vehicle trajectory sequences. The set of paths to be matched, with the length of the matching window being the length of the vehicle trajectory sequence. The window sliding length is The matching process involves calculating and comparing the correlation coefficients of the heading angle sequences to find the best matching result, such as... Figure 3 As shown. Vehicle trajectory sequence It is a sequence consisting of a set of equally spaced points, with a length of . To perform matching, one path from the set of paths to be matched is selected and interpolated at equal intervals to obtain a sequence of paths to be matched that has the same spacing between adjacent points as the vehicle trajectory sequence. The path to be matched consists of multiple straight line segments connecting two points end to end. The first straight line segment is a candidate road segment searched based on the current vehicle position measured by inertial navigation. In the sequence of paths to be matched, the indices of all points corresponding to the candidate road segments are marked as 0~ Then find the point among these points that is closest to the current vehicle position measured by the inertial navigation system, and mark its index as... From the sequence of paths to be matched, the self-index is... Starting from the point, extract a length of... Calculate the correlation coefficient between the relative heading angle sequence and the relative heading angle, and record the current value. Corresponding location points and correlation coefficients. The minimum value is The maximum value is The process involves iterating through all paths in the set of paths to be matched, repeating the above steps, and finally determining the location point corresponding to the highest correlation coefficient as the best match result.

[0066] Since the matching effect is better only when the vehicle trajectory has obvious track features, the map matching and localization method based on adaptive sliding window matching incorporates a track feature detection function in its design. This function detects track features on the vehicle tracks to be matched, performs variable-length reading and matching of vehicle tracks, and uses adaptive length sliding window matching to ensure that the vehicle track sequence used for matching has obvious track features.

[0067] The principle of the track feature detection function is as follows: Figure 4 As shown, assuming The sequence of vehicle waypoints to be detected is as follows:

[0068] ① Extract the first and last points of the flight path and connect them. This line segment serves as the reference line segment for detection.

[0069] ② Calculation points Distance to the reference line segment ,like If the value is greater than the pre-set threshold D, the vehicle's trajectory is considered to have obvious trajectory characteristics, and the detection is completed; otherwise, the detection continues to the next point.

[0070] ③ Repeat step ②, starting from the second point. The second to last point was detected. If the distance from these points to the reference line segment is not greater than the threshold D, then the vehicle track is considered to have no obvious track characteristics, and the detection ends.

[0071] The set of paths to be matched consists of all possible paths corresponding to the vehicle's trajectory sequence. The map matching and localization method based on adaptive sliding window matching incorporates an incremental search approach for the paths to be matched. It uses a combination of traversal search and incremental search to search the set of paths to be matched, as shown in the flowchart below. Figure 5 As shown. If the correlation coefficient of the previous matching result is greater than the threshold and the recent matching results are consecutive, then an incremental query is performed based on the previous matching path to reduce the number of paths to be matched, speed up the search, and thus improve the real-time performance of the matching; if the above judgment conditions are not met, then a traversal query is performed to ensure that the correct matching path can be found.

[0072] When searching for a matching path, the vehicle's current position measured by inertial navigation is first used as a reference point. Then, using the spatial index (R-tree) established when importing the road vector map, the path is centered on the reference point and a distance threshold is applied. A circular buffer is constructed with a radius, and straight road segments whose distance from the reference point is less than a threshold are queried to obtain a set of candidate road segments. During the traversal search, all paths that meet the length requirement connected to each candidate road segment need to be traversed. During the incremental search, the common parts of the previously matched paths are retained, and the connection relationship between each candidate road segment and this part is detected to obtain a new set of paths to be matched.

[0073] Vehicle localization methods that match point cloud data with building outlines include:

[0074] The building point cloud matching module first projects the point cloud data onto the bird's-eye view, then uses the Iterative Closest Point (ICP) algorithm to estimate the translation transformation, initially aligning the projected point cloud points to the building edges to establish a "point-edge" correspondence. Finally, RANSAC fine registration and error checking are performed to further eliminate rotation and scaling errors. Because when the vehicle's approximate position overlaps with its true position, the point cloud projection on the bird's-eye view lies on the building outline edge, the transformation from point cloud projection to building outline is the same as the transformation from the vehicle's approximate position to its true position.

[0075] The ICP algorithm used in the building point cloud matching module is a simplified version of the ICP algorithm, which fixes the rotation part and only estimates the translation vector. The objective function of the simplified ICP algorithm is shown in equation (1), where the source point set... It is obtained from the original vehicle point cloud data through coordinate projection and heading angle rotation correction, that is, each point The corrected point cloud projection coordinates contain a total of Individual points; target point set For building outline points; These are the weighting coefficients; Let be the translation vector to be determined; For the matching function, for each source point Find the nearest target point This simplification reduces the degrees of freedom, improving the speed and stability of the algorithm.

[0076] (1)

[0077] The specific iteration steps are as follows:

[0078] ① Nearest neighbor search: for the source set Each point in In the target point set Find the point in the middle that is closest to it by Euclidean distance. And record the index of the nearest target point. After traversing the source point set, we obtain the set of nearest target points, called the "nearest target point set" (this set is a subset of the target point set, may not contain all its points, and may contain duplicates).

[0079] (2)

[0080] ② Translation vector estimation: Calculate the centroids of the source point set and the corresponding nearest target point set, and solve for the translation vector using the least squares method.

[0081] (3)

[0082] (4)

[0083] ③ Apply the estimated transformation to the source point set: .

[0084] ④ Repeat the above steps until the convergence condition is met.

[0085] make To simplify the cumulative translation vector in the ICP algorithm, Point cloud projection coordinates The final point coordinates after simplified ICP calculation, i.e. Since ICP is sensitive to its initial location and easily affected by outliers, the RANSAC algorithm is used for further fine registration to eliminate the influence of outliers. The specific steps are as follows:

[0086] ① For each initially aligned point cloud projection point Find the nearest building edge point. (do (projection on the building edge), the preservation distance is less than the threshold. Point pairs, construct a set of matching pairs .

[0087] (5)

[0088] ②Use these matching point pairs to estimate a similarity transformation using the RANSAC algorithm.

[0089] ③ The estimated transformation is used to transform the point cloud points again, and the corrected position error is calculated to solve the problem of outlier interference.

[0090] When performing point cloud matching for buildings, it is necessary to first map the point cloud data from three dimensions to two dimensions through coordinate transformation, rotation transformation, and scale normalization so that it can match the building outline. Let the coordinates of the point cloud in the vehicle coordinate system be... Where x is the positive direction of vehicle movement, y is the positive direction of vehicle leftward movement, and z is the positive direction of vertical upward movement. First, the vehicle coordinate system is transformed to the local northeast-sky coordinate system. Then, the point cloud is rotated according to the vehicle's heading angle. Finally, the rotated point cloud coordinates are scaled and projected onto the image pixel coordinate system. The complete projection formula for the entire process is shown in equation (6), where... These are the pixel coordinates of the point cloud projection points in the image. 's' is the vehicle heading angle, and 's' is the scaling factor (unit: pixels / meter). These are the pixel coordinates of the image center.

[0091] (6).

[0092] A vehicle semantic map matching and localization device based on space-based remote sensing images, comprising:

[0093] The image pyramid construction module acquires space-based remote sensing images of the application area, cuts them into small remote sensing images of various scales, and constructs an image pyramid.

[0094] The road vector data extraction module performs road identification on remote sensing small images at each scale to obtain corresponding road binary images. Then, through stitching and morphological processing, it obtains road binary images of the application area at each scale. The road centerline vector is extracted from the road binary images, the extraction results at multiple scales are fused, and optimization processing is performed to obtain usable road centerline vector data.

[0095] The building vector data extraction module performs building recognition on remote sensing images at each scale, obtains the corresponding polygon vertex coordinate sequence, stitches them together to obtain the building outline vector of the application area at each scale, fuses the results of multiple scales, and performs optimization processing to obtain usable building outline vector data.

[0096] The semantic map model building module merges road centerline vector data and building outline vector data to obtain a lightweight semantic map.

[0097] The data acquisition module acquires vehicle trajectory data and LiDAR point cloud data, and ensures timestamp alignment.

[0098] The coarse localization module performs map matching and localization based on adaptive sliding window matching. It preprocesses the vehicle trajectory data and the road centerline vector data of the semantic map, and matches the vehicle trajectory sequence with the path sequence to be matched to obtain the coarse localization of the vehicle.

[0099] The fine positioning module defines the candidate area with the coarse positioning result as the center, transforms the laser point cloud data to the global coordinate system, matches the point cloud projection points in the candidate area with the building outline vector data of the semantic map, and obtains the transformation from point cloud projection to building outline. The same transformation is performed on the vehicle coarse positioning to obtain the accurate positioning result.

[0100] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method.

[0101] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement the method described thereon.

Claims

1. A vehicle semantic map matching and localization method based on space-based remote sensing images, characterized in that, Includes the following steps: Step 1: Acquire space-based remote sensing images of the application area and cut them into small remote sensing images of various scales to construct an image pyramid; Step 2: Perform road identification on the remote sensing images at each scale to obtain the corresponding binary road images. Then, through stitching and morphological processing, obtain the binary road images of the application area at each scale. Extract the road centerline vector from the binary road images, fuse the extraction results from multiple scales, and perform optimization processing to obtain usable road centerline vector data. Step 3: Perform building recognition on the remote sensing images at each scale to obtain the corresponding polygon vertex coordinate sequence, stitch them together to obtain the building outline vector of the application area at each scale, fuse the results of multiple scales, and perform optimization processing to obtain usable building outline vector data. Step 4: Merge the road centerline vector data and the building outline vector data to obtain a lightweight semantic map; Step 5: Acquire vehicle trajectory data and LiDAR point cloud data, and ensure timestamp alignment; Step 6: Perform map matching and localization based on adaptive sliding window matching. Preprocess the vehicle trajectory data and the road centerline vector data of the semantic map, and match the vehicle trajectory sequence with the path sequence to be matched to obtain the coarse localization of the vehicle. Step 7: Using the coarse positioning result as the center, define the candidate area, transform the laser point cloud data to the global coordinate system, match the point cloud projection points in the candidate area with the building outline vector data of the semantic map, and obtain the transformation from point cloud projection to building outline. Perform the same transformation on the vehicle coarse positioning to obtain the accurate positioning result.

2. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 1, characterized in that, In step 1, the same target is represented at multiple scales by cutting the high-resolution remote sensing image of the region into multiple different sizes.

3. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 1, characterized in that, In step 3, the building vector data extraction module is used to segment, extract, and vectorize building map elements. In the building vector data extraction module, the input large-area high-resolution image is first cut into different sizes to construct an image pyramid; for a remote sensing small image of a certain scale, it is input into the building segmentation model for building recognition to obtain the corresponding polygon vertex coordinate sequence, and the building segmentation results are stitched together to obtain the building outline vector; finally, the building outline vectors of multiple scales are fused to obtain usable building outline vector data.

4. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 3, characterized in that, In step 2, the road vector data extraction module is used to segment, extract, and vectorize road map elements. In the road vector data extraction module, the input large-area high-resolution image is first segmented into different sizes to construct an image pyramid; a small remote sensing image at a certain scale is input into the road segmentation model for road recognition, and the road segmentation results are stitched and superimposed to obtain the binary image of the road in the area to be tested. After operations such as centerline breakage and reconnection, the road centerline vector is obtained; finally, the road centerline vectors at multiple scales are fused to obtain usable road centerline vector data.

5. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 3, characterized in that, The adaptive sliding window matching in step 6 includes: introducing a trajectory feature detection function to detect the features of vehicle trajectories, using an adaptive variable-length window for matching to ensure that trajectory features exist in the sequence to be matched; and using an incremental search method when searching for the path to be matched. When the historical matching results are continuous and the correlation coefficient is greater than the threshold, an incremental query is performed based on the previous matching path; otherwise, a traversal query is performed.

6. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 5, characterized in that, The two parts being matched are preprocessed vehicle trajectory sequences. The set of paths to be matched, with the length of the matching window being the length of the vehicle trajectory sequence. The window sliding length is The matching process involves calculating and comparing the correlation coefficients of the heading angle sequences to find the best matching result.

7. The vehicle semantic map matching and localization method based on space-based remote sensing images according to claim 5, characterized in that, The specific function for detecting track features is as follows: The sequence of vehicle waypoints to be detected is as follows: ① Extract the first and last points of the flight path and connect them. This line segment serves as the reference line segment for detection; ② Calculation points Distance to the reference line segment ,like If the value is greater than the pre-set threshold D, the vehicle's trajectory is considered to have obvious trajectory characteristics, and the detection is completed; otherwise, continue to detect the next point. ③ Repeat step ②, starting from the second point. The second to last point was detected. If the distance from these points to the reference line segment is not greater than the threshold D, then the vehicle track is considered to have no obvious track characteristics, and the detection ends.

8. A vehicle semantic map matching and positioning device based on space-based remote sensing images, characterized in that, include: The image pyramid construction module acquires space-based remote sensing images of the application area, cuts them into small remote sensing images of various scales, and constructs an image pyramid. The road vector data extraction module performs road identification on remote sensing small images at each scale to obtain corresponding road binary images. Then, through stitching and morphological processing, it obtains road binary images of the application area at each scale. The road centerline vector is extracted from the road binary images, the extraction results at multiple scales are fused, and optimization processing is performed to obtain usable road centerline vector data. The building vector data extraction module performs building recognition on remote sensing images at each scale, obtains the corresponding polygon vertex coordinate sequence, stitches them together to obtain the building outline vector of the application area at each scale, fuses the results of multiple scales, and performs optimization processing to obtain usable building outline vector data. The semantic map model building module merges road centerline vector data and building outline vector data to obtain a lightweight semantic map. The data acquisition module acquires vehicle trajectory data and LiDAR point cloud data, and ensures timestamp alignment. The coarse localization module performs map matching and localization based on adaptive sliding window matching. It preprocesses the vehicle trajectory data and the road centerline vector data of the semantic map, and matches the vehicle trajectory sequence with the path sequence to be matched to obtain the coarse localization of the vehicle. The fine positioning module defines the candidate area with the coarse positioning result as the center, transforms the laser point cloud data to the global coordinate system, matches the point cloud projection points in the candidate area with the building outline vector data of the semantic map, and obtains the transformation from point cloud projection to building outline. The same transformation is performed on the vehicle coarse positioning to obtain the accurate positioning result.

9. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 7.

Images