Visual localization prior map construction method and device based on laser radar

Abstract

The invention provides a visual localization prior map construction method and device based on a laser radar. The method comprises the following steps: acquiring foundation laser radar point cloud data and mobile laser radar image data of a localization area; constructing a local map of the ground-based laser radar point cloud data, and matching the frame of the mobile laser radar image data with the local map to obtain the current pose estimation of the laser radar; global positioning is carried out on the laser radar according to the key frame in the current pose estimation, an optimization factor graph is constructed based on a global positioning result, and pose constraint is carried out on the initial pose estimation of the laser radar; and based on the final pose estimation of the laser radar after pose constraint, constructing a priori map database for visual localization in the localization area. Through the method and the device, the defects of failure and poor robustness of a prior map positioning method in a complex scene with serious GNSS positioning error or even failure in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to a method and device for constructing a priori maps of visual positioning based on laser radar. Background Art

[0002] Visual geolocation can retrieve and calculate the user's current geographic location based on the visual image input by the user. Image data is easy to obtain and the acquisition cost is low. Reliable location information can be obtained in complex environments such as cities, indoors, and mountainous areas where global navigation satellite system (GNNS) positioning fails. Therefore, visual geolocation technology has wide and important applications in the fields of autonomous driving, augmented reality (AR), robot navigation and positioning, etc.

[0003] The visual geolocation methods in the prior art are mainly divided into RGB image-based positioning methods and prior map-based positioning methods. The RGB image-based method uses a large number of reference images with position marks, retrieves similar images of the query image through image matching algorithms, and then uses the location of similar images to approximately replace the user's approximate location, but the accuracy is low, and in large and complex outdoor scenes, due to the huge number of reference images, the similar image retrieval process will take a long time, and if there is an error match, it will further affect the positioning accuracy. The positioning method based on the prior map uses SLAM technology to estimate the three-dimensional structure in the scene by matching feature points in the image and using geometric constraints, and determines the position and posture of the camera through camera motion estimation. However, in large and complex outdoor scenes, three-dimensional mapping is easily disturbed by environmental factors such as occlusion, illumination, and seasonal changes. SLAM technology is difficult to cope with the three-dimensional mapping tasks of large and complex outdoor scenes, and there will be problems such as low accuracy, easy failure of mapping, and poor robustness. In addition, both SfM image reconstruction and SLAM three-dimensional mapping rely on accurate GNSS real-time differential positioning (RTK) position information to provide high-precision position constraints and convert the prior map to a geographic coordinate system. Therefore, in complex scenarios such as mountainous areas or cities where GNSS positioning errors are severe or even fail, there will be problems with visual positioning where the prior map positioning method fails and has poor robustness. Summary of the invention

[0004] The present invention provides a method and device for constructing a priori map for visual positioning based on laser radar, so as to solve the defects in the prior art that the priori map positioning method may fail and have poor robustness in complex scenarios where the GNSS positioning error is serious or even fails.

[0005] The present invention provides a method for constructing a priori maps based on laser radar, the method comprising the following steps: Obtain ground-based LiDAR point cloud data and mobile LiDAR image data of the positioning area; Constructing a local map of the ground-based laser radar point cloud data, and matching the moving frames of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar; Performing global positioning of the laser radar according to the key frames in the current pose estimation, and constructing an optimization factor graph according to the result of the global positioning to perform pose constraints on the initial pose estimation of the laser radar; Based on the final pose estimation of the laser radar after pose constraints, a priori map database for visual positioning in the positioning area is constructed.

[0006] In some embodiments, constructing a local map of the ground-based laser radar point cloud data includes: Extracting corner point features of the ground-based laser radar point cloud data in each frame of the mobile laser radar image data; In the neighborhood of each ground-based laser radar point cloud data, a covariance matrix is ​​constructed according to the coordinate values ​​of the ground-based laser radar point cloud data in the neighborhood; Determining multiple eigenvalues ​​of the covariance matrix by eigenvalue decomposition method; Dividing the ground-based laser radar point cloud data into plane feature point cloud data and irregular feature point cloud data according to the feature value; The corner point features, plane feature point cloud data and irregular feature point cloud data of each frame are combined to obtain a local map of the ground-based laser radar point cloud data.

[0007] In some embodiments, matching the moving frame of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar includes: Matching the moving frames of the mobile laser radar image data with the point cloud feature points in the local map to obtain matching point cloud features; Calling the ICP algorithm to determine the transformation matrix between the performing frame and the matching point cloud feature; The mobile laser radar image data is transformed frame by frame through the transformation matrix to obtain the current posture estimation of the laser radar.

[0008] In some embodiments, performing global positioning of the laser radar according to the key frame in the current pose estimation includes: Obtaining an initial pose estimate of the laser radar, and clipping the ground-based laser radar point cloud data using the initial pose estimate to obtain a global prior map; Matching the key frame in the current pose estimation with the global prior map to obtain a laser point cloud of the global prior map; Calling the ICP algorithm to determine the optimal transformation matrix between the laser point cloud of the key frame and the laser point cloud of the global prior map; Based on the optimal transformation matrix, the position information of the laser radar in the ground-based laser radar point cloud data is determined as the global positioning result of the laser radar.

[0009] In some embodiments, the optimization factor graph includes an inter-frame motion constraint factor graph, a loop constraint factor graph, and a global positioning constraint factor graph. The optimization factor graph is constructed according to the result of global positioning to constrain the initial pose estimation of the laser radar, including: Construct a global positioning constraint factor graph according to the result of global positioning; Determining the inter-frame motion constraint factor graph according to a pose transformation relationship between two adjacent key frames in a current pose estimation of the laser radar; Performing loop closure detection on the key frames in the current pose estimation to determine a loop closure constraint factor graph; According to the motion constraint factor graph, the loop constraint factor graph and the global positioning constraint factor graph, the initial pose estimation of the laser radar is subjected to pose constraints.

[0010] In some embodiments, performing loop detection on the key frames in the current pose estimation to determine a loop constraint factor graph includes: Determining a historical frame before a current key frame in the current pose estimate; Determine the Euclidean distance between the laser point cloud of the current key frame and the laser point cloud of the historical frame; When the Euclidean distance is less than a preset distance threshold, it is determined that a loop occurs in the laser radar, and the laser point cloud of the current key frame is matched with the laser point cloud of the historical frame to obtain a position transformation relationship between the current key frame and the historical frame; A loop constraint factor graph is determined based on the position transformation relationship.

[0011] In some embodiments, the constructing of a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after pose constraints includes: Determine a reference image corresponding to each frame in the mobile laser radar image data, and determine a final pose estimate of the laser radar corresponding to each frame; Obtaining external parameters of the laser radar and the camera, and calculating the final pose estimation corresponding to each frame using the external parameters to obtain the pose of the reference image; Calling the SuperPoint model to extract candidate feature points of the reference image and descriptors of the candidate feature points, calling the SuperGlue model to match the descriptors of the candidate feature points, and filtering out local feature points and corresponding local feature descriptors from the candidate feature points according to the matching results; Calling the NetVLAD model to perform global feature extraction on the reference image to obtain an image global feature descriptor of the reference image; Projecting the two-dimensional coordinates of the local feature points according to the external parameters and the internal parameters of the laser radar camera to obtain the three-dimensional coordinates of the local feature points; Based on the reference image, the pose of the reference image, the image global feature descriptor of the reference image, the three-dimensional coordinates of the local feature points of the reference image, the local feature points of the reference image and the corresponding local feature descriptors, a priori map database for performing visual positioning in the positioning area is constructed.

[0012] In some embodiments, the method further comprises: performing visual geolocation of the query image of the positioning area through the prior map database, and the process of visual geolocation of the query image comprises: Performing global feature extraction on the query image to obtain an image global feature descriptor of the query image; Matching the image global feature descriptor with the prior map database to determine a target reference image similar to the query image; Matching the first local feature points and the corresponding first local feature descriptor of the query image with the second local feature points and the corresponding second local feature descriptor of the target reference image; The pose estimation of the query image is determined according to the matched second local feature points as the visual geographic positioning result of the query image.

[0013] In some embodiments, determining the pose estimation of the query image according to the matched second local feature points includes: According to the matched second local feature point, query the corresponding target three-dimensional coordinates from the prior map database; Construct a perspective projection model of the query image corresponding to the target camera; Substituting the target two-dimensional coordinates of the first local feature point and the target three-dimensional coordinates into the perspective projection model for iterative solution to obtain the target camera pose and projection error; The target camera pose obtained by an iterative solution with the minimum projection error is determined as the pose estimate of the query image, as the visual geographic positioning result of the query image.

[0014] The present invention also provides a device for constructing a priori maps based on visual positioning of laser radar, the device comprising the following modules: An acquisition module is used to acquire ground-based laser radar point cloud data and mobile laser radar image data of the positioning area; A matching module, used to construct a local map of the ground-based laser radar point cloud data, and match the moving frame of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar; A constraint module, used to perform global positioning of the laser radar according to the key frame in the current pose estimation, and to construct an optimization factor graph according to the result of the global positioning to perform pose constraints on the initial pose estimation of the laser radar; A construction module is used to construct a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after pose constraints.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for constructing a priori visual positioning map based on laser radar as described in any one of the above-mentioned methods is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for constructing a priori maps based on visual positioning using laser radar.

[0017] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for constructing a priori maps based on visual positioning using laser radar.

[0018] The method and device for constructing a priori map for visual positioning based on laser radar provided by the present invention first construct a local map of ground-based laser radar point cloud data, and match the frames of mobile laser radar image data with the local map to obtain the current pose estimate of the laser radar; then the laser radar is globally positioned, and an optimization factor graph is constructed based on the result of global positioning, and the initial pose estimate of the laser radar is subjected to pose constraints. Finally, based on the final pose estimate of the laser radar after pose constraints, an a priori map database for visual positioning in the positioning area is constructed. Thus, on the basis of the SLAM algorithm of the prior art, steps such as laser radar global positioning and factor graph optimization are integrated to fully utilize the geographical location constraints brought by the ground-based laser radar point cloud to construct the a priori map database, thereby improving the mapping accuracy, global consistency and coverage, and solving the problem of visual geographic positioning in areas with unstable GNSS positioning in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the method for constructing a priori map of visual positioning based on laser radar provided by the present invention.

[0021] Figure 2 It is a schematic diagram of the framework of the laser radar-based visual positioning priori map construction method provided by the present invention.

[0022] Figure 3 It is a schematic diagram of errors in the global positioning results of the laser radar provided by the present invention.

[0023] Figure 4 It is a schematic diagram of loop generation of laser radar key frames provided by the present invention.

[0024] Figure 5 It is a schematic diagram of the global optimization factor graph model provided by the present invention.

[0025] Figure 6 It is a schematic diagram of camera pose estimation provided by the present invention.

[0026] Figure 7 It is a structural schematic diagram of the laser radar-based visual positioning prior map construction device provided by the present invention.

[0027] Figure 8 It is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Combine the following Figures 1 to 6 To describe the laser radar-based visual positioning priori map construction method of the present invention, Figure 1 is a flow chart of a method for constructing a priori maps based on laser radar for visual positioning provided by the present invention, such as Figure 1As shown, the method includes the following steps 101 to 104.

[0030] Step 101: Acquire ground-based LiDAR point cloud data and mobile LiDAR image data of the positioning area.

[0031] In the embodiment of the present invention, the area where GNSS real-time differential positioning is available is used as the positioning area. In the positioning area, a high-precision laser scanner is used to collect and obtain ground-based laser radar point cloud data. Figure 2 , Figure 2 This is a schematic diagram of the framework of the method for constructing a priori map based on laser radar visual positioning provided by the present invention. After collecting ground-based laser radar data, high-precision control points are collected using a total station and an RTK surveying instrument in the same scene. These control points are used to geo-reference the ground-based laser radar point cloud data to convert it to the world geographic coordinate system for global positioning of the laser radar. At the same time, the laser radar camera will also collect and acquire mobile laser radar image data (specifically RGB images, as reference images) in real time, which is convenient for subsequent optimization of the laser radar's posture estimation and construction of a priori map database, ensuring that each frame of mobile laser radar image data has corresponding ground-based laser radar point cloud data.

[0032] Step 102: construct a local map of the ground-based lidar point cloud data, and match the moving frames of the mobile lidar image data with the local map to obtain a current pose estimate of the lidar.

[0033] To determine the current pose estimate of the lidar, it is necessary to first build a local map of the ground-based lidar point cloud data. The local map is a frame set based on a time sliding window with a fixed size limit (such as 50 frames). In each frame of the mobile lidar image data, the corner features, plane feature point cloud data and irregular feature point cloud data of the ground-based lidar point cloud data are stored and constructed into a local map to facilitate the matching of subsequent image data key frames with the local map.

[0034] The process of constructing the local map of ground-based lidar point cloud data is as follows: first, in each frame of the mobile lidar image data, the corner point features of the ground-based lidar point cloud data are extracted, and then the principal component analysis method is used to distinguish the ground-based lidar point cloud data into plane feature point cloud data and irregular feature point cloud data.

[0035] First, for each ground-based lidar point cloud data in the neighborhood, a covariance matrix is ​​constructed based on the coordinate values ​​of the ground-based lidar point cloud data in the neighborhood. This covariance matrix describes the variability and distribution of the ground-based lidar point cloud data and other ground-based lidar point cloud data in the neighborhood on different coordinate axes. Next, the eigenvalue decomposition method is used to determine multiple eigenvalues ​​(for example, 3) of the covariance matrix, and each eigenvalue has a corresponding eigenvector. The eigenvalue represents the degree of variation of the point cloud data in the variation direction, and the eigenvector is used to represent these variation directions. Then, the ground-based lidar point cloud data is divided into plane feature point cloud data and irregular feature point cloud data according to the eigenvalues. Here, among the three eigenvalues ​​of the ground-based lidar point cloud data, if two of the eigenvalues ​​are relatively large and the other eigenvalue is relatively small, it means that the ground-based lidar point cloud data in the neighborhood is mainly distributed in one plane. In this case, such ground-based lidar point cloud data can be divided into plane feature point cloud data. If the three eigenvalues ​​are similar in size, or no eigenvalue is significantly smaller than the other two eigenvalues, it means that the ground-based lidar point cloud data in the neighborhood is distributed more evenly and has no obvious linear or planar structure. In this case, such ground-based lidar point cloud data can be divided into irregular feature point cloud data.

[0036] Through the above process, the ground-based lidar point cloud data can be divided into plane feature point cloud data and irregular feature point cloud data in each frame of the mobile lidar image data. Finally, the corner point features, plane feature point cloud data and irregular feature point cloud data of each frame (for example, 50 frames) are combined to obtain a local map of the ground-based lidar point cloud data.

[0037] like Figure 2 As shown in the figure, after the local map is constructed, the LiDAR SLAM algorithm stage is carried out. The LiDAR-Inertial Navigation Odometry will match the frames of the mobile LiDAR image data with the local map to obtain the current pose estimate of the LiDAR.

[0038] Specifically, the moving frame of the mobile LiDAR image data is first matched with the point cloud feature points in the local map to obtain the matching point cloud features, and then the ICP algorithm can be called to determine the transformation matrix between the moving frame and the matching point cloud features. Here, the ICP algorithm can iteratively solve the change relationship between the moving frame and the local map, thereby estimating the motion state of the LiDAR and restoring the pose information. Finally, the moving frame of the mobile LiDAR image data is transformed through the transformation matrix to obtain the current pose estimation of the LiDAR, thereby realizing the pose estimation of the moving frame of the LiDAR by the LiDAR-Inertial Navigation Odometry.

[0039] Step 103: perform global positioning of the laser radar according to the key frame in the current pose estimation, and construct an optimization factor graph according to the result of the global positioning to perform pose constraints on the initial pose estimation of the laser radar.

[0040] After determining the current pose estimation of the laser radar frame in step 102, the laser radar can be globally positioned, and a number of key frames with significant features, high quality and representativeness are selected from the current pose estimation of the laser radar, which are recorded as , the lidar is globally positioned according to the key frames in the current pose estimation, as described below.

[0041] For any lidar keyframe (i represents the i-th frame), both can be compared with high-precision ground-based lidar point cloud data Perform frame-to-map scan matching to solve the optimal transformation matrix from the lidar coordinate system to the geographic coordinate system In the global positioning process, in order to improve the computational efficiency, the initial pose estimation of the lidar is obtained , through the initial pose estimation Ground-based LiDAR point cloud data Crop to get the global prior map. Here, the ground-based LiDAR point cloud data Will be downsampled, and the downsampled point cloud data is recorded as , the corresponding laser point coordinates are marked as (X, Y, Z), and then estimated according to the initial pose and the lidar field of view angle threshold Crop and get the global prior map, denoted as .

[0042] Initial pose estimation For LiDAR point cloud data The laser point coordinates (X, Y, Z) in the laser radar coordinate system are converted to the laser point coordinates in the laser radar coordinate system, recorded as , the formula is expressed as follows (1): = (1)

[0043] Then, the laser radar field of view angle threshold is used Crop to get the global prior map point cloud , expressed as the following formula (2): (2)

[0044] In the above formula (2), represents downsampled point cloud data, (X, Y, Z) represents lidar point cloud data The laser point coordinates in Represents the laser point coordinates in the laser radar coordinate system.

[0045] Next, the key frame in the current pose estimation is matched with the global prior map to obtain the optimal transformation matrix Here the lidar keyframe With global prior map Perform point cloud matching to obtain a global prior map The laser point cloud is denoted as , and then call the ICP algorithm to determine the key frame Laser point cloud , and the global prior map Laser point cloud The optimal transformation matrix between , expressed as the following formula (3): (3)

[0046] In the above formula (3), represents the initial pose estimate, Represents a lidar keyframe , represents the laser point cloud corresponding to the j-th key frame, Represents the global prior map The laser point cloud that matches the laser point cloud corresponding to the j-th key frame in , Indicates calculating the Euclidean distance.

[0047] It should be noted that the laser radar key frame The point cloud matching process depends on the accurate initial pose estimation of the lidar Otherwise, it is easy to fall into the local optimal solution, resulting in errors in the calculated global positioning results. Therefore, the starting point of point cloud data collection must be set in an area with a priori point cloud map, so that an approximate specific location can be provided manually and ensure that global positioning can be carried out smoothly. In this process, the lidar-inertial odometer estimates the motion state of the lidar sensor and determines the distance between two adjacent key frames in the current pose estimation (for example, and ), denoted as Then according to this posture transformation relationship Determine the next keyframe Initial pose estimation used for point cloud matching , expressed as the following formula (4): (4)

[0048] Here, between two adjacent key frames (for example and )’s posture transformation relationship Also known as inter-frame motion constraint, it is used to constrain pose estimation and eliminate possible errors in pose information. It will be described below and will not be repeated here.

[0049] Finally, the position information of the lidar in the ground-based lidar point cloud data is determined based on the optimal transformation matrix as the global positioning result of the lidar. Here, the optimal transformation matrix is ​​solved Then, according to the optimal transformation matrix and the position information of the laser radar in the ground-based laser radar point cloud data , calculate the pose information , this relationship is expressed as the following formula (5): (5)

[0050] In the above formula (5), is the translation vector from the local coordinates of the lidar sensor (i.e., the lidar coordinate system) to the world geographic coordinate system at the i-th key frame of the lidar. It is expressed as the following formula (6): (6)

[0051] In the above formula (6), , , Represents the coordinate position of the local coordinates of the lidar sensor to the world geographic coordinate system.

[0052] In the above formula (5), When it is the i-th key frame of the lidar, the rotation matrix of the local coordinates of the lidar sensor (i.e., the lidar coordinate system) to the world geographic coordinate system is expressed as the following formula (7): (7)

[0053] In the above formula (7), the parameters Represents the attitude information of the LiDAR sensor in the LiDAR coordinate system.

[0054] Combining the above formulas (3), (5), (6), and (7), we can calculate the coordinate position of the local coordinates of the lidar sensor in the geographic coordinate system: , , , and the attitude information of the lidar sensor in the lidar coordinate system The coordinate position and attitude information here are the position and attitude information of the laser radar in the ground-based laser radar point cloud data , that is , and then each key frame of the lidar (i.e., i key frames) Combined as the global positioning result of the lidar, it is finally expressed as .

[0055] In some embodiments, as the key frames selected by the lidar-inertial navigation odometer As the number of sensors increases, the cumulative error will also increase, which will lead to larger errors in the global positioning results of the LiDAR or even failure. Figure 3 , Figure 3 Schematic diagram of the error in the laser radar global positioning result provided by the present invention. Figure 3 As shown in the figure, according to the motion trajectory obtained by the lidar-inertial navigation odometer, the trajectory obtained after the algorithm (ICP algorithm) optimization and the actual trajectory of the lidar, the calculated lidar global positioning results are different, that is, errors occur. , the trajectory will become longer and longer, and the error will be larger. and )’s posture transformation relationship As an inter-frame motion constraint, it is impossible to eliminate the error of the global positioning result.

[0056] Therefore, if Figure 2 As shown, after the laser radar is globally positioned, the embodiment of the present invention constructs an optimization factor graph based on the result of global positioning, and constrains the initial pose estimation of the laser radar. The optimization factor graph here includes an inter-frame motion constraint factor graph, a loop constraint factor graph, and a global positioning constraint factor graph.

[0057] First, a global positioning constraint factor graph is constructed based on the results of global positioning. The global positioning constraint refers to the positioning result of the lidar global positioning module. The global positioning constraint can provide accurate position information of the key frames, so it can be added to the optimization factor graph to participate in the global optimization operation.

[0058] The inter-frame motion constraint factor graph is determined according to the pose transformation relationship between two adjacent key frames in the current pose estimation of the lidar. The inter-frame motion constraint is to constrain the pose information between two adjacent key frames through the pose transformation relationship.

[0059] The loop constraint requires loop detection on the key frames in the current pose estimation to determine the loop constraint factor graph. The purpose of loop detection is to determine whether the laser radar has returned to the place it has passed in the process of moving and collecting data. Therefore, loop detection can effectively eliminate the cumulative error of the laser radar-inertial navigation odometer. The process of loop detection is to identify the same place between the current key frame of the laser radar and the historical frame (that is, the frame before the current key frame, which may not be adjacent), so as to solve the change of the laser radar motion state.

[0060] See also Figure 4 , Figure 4 Schematic diagram of the laser radar key frame loop provided by the present invention. Figure 4 As shown, the laser radar Departure, move to location Move back to position , which indicates the location and location Form a loop, and It indicates the location and location The relative position relationship between them can effectively eliminate the cumulative error caused by the lidar-inertial navigation odometer.

[0061] Next, the process of determining the loop constraint factor graph is introduced: first, the historical frame before the current key frame in the current pose estimation is determined. There can be multiple historical frames. Next, the Euclidean distance between the laser point cloud of the current key frame and the laser point cloud of the historical frame is determined. When the Euclidean distance is less than the preset distance threshold, it is determined that a loop has occurred in the laser radar, and the ICP algorithm is used to match the laser point cloud of the current key frame with the laser point cloud of the historical frame to obtain the position transformation relationship between the current key frame and the historical frame. Finally, the loop constraint factor graph is determined based on the position transformation relationship.

[0062] Here, considering the limited number of laser point clouds obtained by a single-frame scan of the LiDAR, in order to improve the accuracy of point cloud matching, a certain number of continuous key frames will be converted to a unified world coordinate system and merged before scanning and matching as a historical frame set. When matching the point cloud, the current key frame can be matched with the historical frame set. After successfully matching the point cloud using the ICP algorithm, the relative transformation relationship between the current key frame and the historical frame is used as a loop constraint factor graph and added to the optimization factor graph.

[0063] After determining the optimization factor graph, the initial pose estimation of the LiDAR is constrained according to the motion constraint factor graph, loop constraint factor graph, and global positioning constraint factor graph. Figure 5 , Figure 5It is a schematic diagram of the global optimization factor graph model provided by the present invention. Starting from the initial pose estimation of the laser radar, that is, the first key frame of the laser radar At the beginning, the state node (also known as the position node) corresponding to the laser radar is , it is necessary to perform global positioning of the laser radar and constrain it through the global positioning factor (i.e. the global positioning constraint factor graph above). When the state node corresponding to the laser radar reaches When , the corresponding key frame is , it is also necessary to perform lidar global positioning and constrain it through the global positioning factor, where in the key frame During global positioning, the two adjacent key frames ( and ) to determine the optimal change matrix, so as to transform the second key frame Perform global positioning using LiDAR.

[0064] During the global positioning process, the current key frame may also be generated The process of matching with global positioning fails, and it is necessary to re-target the keyframes Perform global positioning of the LiDAR. As the odometer factor in the LiDAR-INS odometer accumulates, the subsequent LiDAR state nodes are obtained. , , until , , and the corresponding key frame is During the laser radar-inertial navigation odometer accumulation process, for example, in the laser radar state node , the key frame is , you need to change the lidar status node The corresponding sub-keyframe of the lidar arrive As a historical frame, loop detection is performed, and the state nodes are calibrated by the loop detection factor (i.e. the loop constraint factor graph above). and state nodes Perform point cloud matching, calculate the position transformation relationship, constrain the current pose estimation of the lidar, and obtain the final pose estimation of the lidar.

[0065] Starting from the initial pose estimation of the laser radar, during the movement of the laser radar, the initial pose estimation of the laser radar is constrained in real time according to the motion constraint factor graph, the loop constraint factor graph and the global positioning constraint factor graph to obtain the final pose estimation of the laser radar, thereby improving the accuracy of the laser radar pose estimation.

[0066] Step 104: construct a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after the pose constraint.

[0067] like Figure 2 As shown in Figure 1, after the pose optimization is performed through the factor graph to obtain the final pose estimate, the prior map construction phase is subsequently entered. In this phase, based on the final pose estimate of the LiDAR after pose constraints, a prior map database for visual positioning in the positioning area is constructed.

[0068] In an embodiment of the present invention, the prior map database includes a reference image, a pose of the reference image, local feature points of the reference image, and three-dimensional coordinates of the local feature points of the reference image. Each running frame of the mobile laser radar image data is an RGB image taken by the laser radar camera, and this RGB image is used as a reference image, so the reference image corresponding to each running frame in the mobile laser radar image data can be determined, and through factor graph optimization, the running frames of the laser radar have corresponding final pose estimates, so the final pose estimate corresponding to each running frame of the laser radar can be determined.

[0069] Considering that the mobile LiDAR image data and the ground-based LiDAR point cloud data are collected at the same time, there is a corresponding relationship between the two. Therefore, the reference image pose corresponding to each frame in the mobile LiDAR image data can be calculated by estimating the final pose of the LiDAR in the corresponding frame. , the final pose corresponding to each frame is estimated through external parameters Calculate and get the pose of the reference image , the calculation formula is as follows formula (8): (8)

[0070] In the above formula (8), is the pose of the reference image, is the final pose estimation of the lidar, Represents the external parameters of the laser radar and the camera, which are obtained after calibration.

[0071] like Figure 2As shown, in the local and global feature extraction of the image in the prior map construction stage, for local feature extraction, the embodiment of the present invention calls the SuperPoint model to extract the candidate feature points and the descriptors of the candidate feature points of the reference image, and calls the SuperGlue model to match the descriptors of the candidate feature points, and screens out local feature points from the candidate feature points of the reference image according to the matching results. Here, considering that there are differences between the reference images, their local feature points are uncertain, so feature point matching is required between the reference images. The SuperPoint model is an end-to-end deep learning architecture for detecting and describing feature points in reference images. It combines feature detectors and descriptor generators, and can directly extract candidate feature points and their corresponding descriptors that can be used for matching from the reference image. The SuperGlue model is a post-processing model based on a graph neural network (GNN), which receives candidate feature points and their corresponding descriptors from different reference image pairs (for example, candidate feature points and corresponding descriptors generated by SuperPoint), calculates the matching scores of the corresponding descriptors of the candidate feature points of the two reference images, and performs the best match on the candidate feature points, thereby screening out local feature points.

[0072] Through the combined use of the SuperGlue model and the SuperGlue model, not only single feature points are taken into account, but also the spatial relationship between feature points is comprehensively considered. It is very effective for multiple uncertain reference images in natural scenes and can significantly improve the robustness and accuracy of matching.

[0073] As for the global features of the image, the embodiment of the present invention calls the NetVLAD model to extract the global features of the reference image to obtain the image global feature descriptor of the reference image. Here, the NetVLAD model is directly used to extract the global features of the reference image, and the global features can be used for image matching.

[0074] like Figure 2 As shown in the figure, after the local feature points and global features are extracted, laser point-image projection is required to determine the 3D coordinates of the local feature points of the reference image, that is, the 3D coordinates on the prior map. The 3D coordinates of the local feature points are determined by projecting the 2D coordinates of the local feature points of the reference image. Here, the extrinsic parameters of the laser radar camera are used to determine the 3D coordinates of the local feature points of the reference image. And the camera internal parameter K, project the two-dimensional coordinates of the local feature points to obtain the three-dimensional coordinates of the local feature points. Because the coordinates of the local feature points extracted by the SuperGlue model are two-dimensional coordinates , laser point cloud can provide accurate point measurement information of the external environment, so using laser point cloud 3D Coordinates The corresponding local feature points of the reference image are directly used as The three-dimensional coordinates of Due to the non-repetitive scanning characteristics of the new solid-state laser radar, a high-density laser point cloud can be collected in a short time, so that most of the two-dimensional local feature points of the reference image can be aligned to the three-dimensional laser point cloud. If the camera intrinsic parameter is , the external parameters of the laser radar and camera are , then the laser point cloud can be projected onto the image plane using the following formula (9). The camera intrinsic parameters, laser radar and camera extrinsic parameters can be obtained after calibration. If the image plane coordinates after projection are within a certain error range (less than 0.2 pixel), the local feature point will use the 3D coordinates of the nearest projected laser point cloud as the 3D coordinates in the prior map. Formula (9) is expressed as follows: (9)

[0075] In the above formula (9), K is the camera internal parameter, are the external parameters of the laser radar and camera, Represents the local feature points of the reference image The corresponding laser point cloud The three-dimensional coordinates of Represents local feature points of the image Coordinates in the lidar coordinate system.

[0076] The corresponding , as the Z-axis coordinate of the local feature point in the prior map, thereby projecting the two-dimensional coordinates of the local feature point of the reference image into three-dimensional coordinates.

[0077] Finally, the prior map database is constructed. According to the reference image, the reference image's posture, the reference image's image global feature descriptor, the three-dimensional coordinates of the reference image's local feature points, the reference image's local feature points and the corresponding local feature descriptors, the prior map database for visual positioning in the positioning area is constructed. Here, each reference image in the mobile lidar image data, the reference image's posture, the reference image's local feature points and the corresponding local feature descriptors, the three-dimensional coordinates of the reference image's local feature points, and the reference image's image global feature descriptors are combined to obtain the prior map database.

[0078] In the embodiment of the present invention, a local map of ground-based laser radar point cloud data is first constructed, and the frames of mobile laser radar image data are matched with the local map to obtain the current pose estimate of the laser radar; then the laser radar is globally positioned, and an optimization factor graph is constructed according to the result of global positioning, and the initial pose estimate of the laser radar is subjected to pose constraints. Finally, based on the final pose estimate of the laser radar after the pose constraints, a priori map database for visual positioning in the positioning area is constructed. Thus, on the basis of the SLAM algorithm of the prior art, steps such as laser radar global positioning and factor graph optimization are integrated to fully utilize the geographic location constraints brought by the ground-based laser radar point cloud to construct the priori map database, thereby improving the mapping accuracy, global consistency and coverage, and solving the problem of visual geographic positioning in areas where GNSS positioning is unstable in complex scenarios.

[0079] In some embodiments, Figure 2 As shown, after constructing the prior map database, the query image of the positioning area is visually geolocated through the prior map database. The process of visual geolocation of the query image is introduced below.

[0080] First, similar image retrieval is performed, that is, the corresponding reference image is retrieved and matched from the prior map database based on the query image. Then local feature extraction and matching are performed, and the NetVLAD model is called to extract global features of the query image to obtain the image global feature descriptor of the query image. Since the prior map database records the global feature descriptor corresponding to each reference image, the image global feature descriptor is matched with the prior map database to determine the target reference image similar to the query image. By matching the global feature descriptor, the target reference image that matches the query image can be queried from the prior map database.

[0081] After determining the target reference image, the SuperPoint model is used to extract the second local feature points and the corresponding second local feature descriptors of the target reference image. The SuperGlue model is then used to match the first local feature points and the corresponding first local feature descriptors of the query image with the second local feature points and the corresponding second local feature descriptors of the target reference image, that is, 2D-2D matching of local feature points. The three-dimensional coordinates in the prior map database are then queried to establish a 2D-3D matching relationship. Finally, pose estimation is performed. Here, the pose estimation of the query image is determined from the prior map database based on the matched second local feature points as the visual geographic positioning result of the query image, which is described in detail below.

[0082] First, the corresponding target three-dimensional coordinates are queried from the prior map database based on the matched second local feature points. Here, based on the 2D-2D matching, the first local feature points of the target reference image can be obtained, and then based on the 2D-3D matching relationship, the target three-dimensional coordinates of the first local feature point can be queried from the prior map database.

[0083] See also Figure 6 , Figure 6 is a schematic diagram of the camera pose estimation provided by the present invention, performing visual geolocation on the query image, that is, performing pose estimation on the target camera corresponding to the query image (for example, the query image is taken by the target camera). Figure 6 As shown, , , To query the coordinates of the first local feature point of the image in the camera coordinate system, represents the origin of the camera coordinate system, Represents the laser point cloud corresponding to the first local feature point in the camera coordinate system, , , To query the target 3D coordinates of the first local feature point from the prior map database, Represents the coordinate origin in the world geographic coordinate system, Represents the laser point cloud projected by the first local feature point in the world geographic coordinate system, Represents the projection between laser point clouds. The position of the target camera can be transformed from the camera coordinate system to the world geographic coordinate system through the rotation matrix and translation vector The pose estimation of the target camera can be determined by constructing a perspective projection model of the query image corresponding to the target camera. The constructed perspective projection model is shown in the following formula (10): (10)

[0084] In the above formula (10), To query the Z-axis coordinate of the first local feature point of the image in the camera coordinate system, , , is the target 3D coordinates of the first local feature point queried from the prior map database, and R is the rotation matrix of the camera coordinate system converted to the world geographic coordinate system , t is the translation vector from the camera coordinate system to the world geographic coordinate system, , is the target two-dimensional coordinate of the first local feature point in the image plane, and K is the intrinsic parameter of the camera.

[0085] When estimating the target camera’s pose, substitute the target 2D coordinates and target 3D coordinates of the first local feature point into the perspective projection model of formula (10) for iterative solution to obtain the target camera pose and projection error. In formula (10), at least 6 sets of 2D-3D matching relationships are required to solve, that is, the first local feature point needs to be matched to the second local feature point of the 6 reference images, so as to query the target 3D coordinates of the 6 reference images and calculate the parameters R, t, and , as the target camera pose. In the actual matching process, the number of matched local feature points is usually more than 6 groups, and there may be wrongly matched local feature point pairs.

[0086] Based on this, the embodiment of the present invention uses the RANSAC algorithm to remove false matches and solve the camera's pose. The RANSAC algorithm first randomly selects the minimum number of local feature point pairs from all local feature point pairs, and uses these local feature point pairs to preliminarily calculate the target camera pose; then the obtained results are applied to all local feature point pairs to calculate the projection error; repeat the above steps multiple times, each time selecting a different random local feature point pair; finally, the target camera pose obtained by the iteration with the smallest projection error in the iterative process is determined as the pose estimate of the query image, as the visual geolocation result of the query image.

[0087] The embodiments of the present invention effectively combine the mobile LiDAR SLAM algorithm with the static ground-based LiDAR scanning to realize the incremental 3D reconstruction from coarse to fine and from known to unknown in complex large scenes, and integrate the steps of LiDAR global positioning and factor graph optimization on the basis of the SLAM algorithm, so as to fully utilize the geographic location constraints brought by the ground-based LiDAR point cloud to construct a priori map database, and realize the priori map construction and geographic visual positioning in the area where GNSS fails, thereby improving the mapping accuracy, global consistency and coverage, and solving the visual geographic positioning problem in the area where GNSS positioning is unstable in complex scenes.

[0088] The following is a description of the visual positioning prior map construction device based on laser radar provided by the present invention. The visual positioning prior map construction device based on laser radar described below and the visual positioning prior map construction method based on laser radar described above can be referenced to each other.

[0089] See also Figure 7 , Figure 7 is a schematic diagram of the structure of the laser radar-based visual positioning prior map construction device provided by the present invention, such as Figure 7As shown, the device for constructing a priori map for visual positioning based on laser radar specifically includes: an acquisition module 701, a matching module 702, a constraint module 703, and a construction module 704. Among them, the acquisition module 701 is used to acquire the ground-based laser radar point cloud data and the mobile laser radar image data of the positioning area; the matching module 702 is used to construct a local map of the ground-based laser radar point cloud data, and match the moving frame of the mobile laser radar image data with the local map to obtain the current pose estimation of the laser radar; the constraint module 703 is used to perform global positioning of the laser radar according to the key frame in the current pose estimation, and construct an optimization factor graph according to the result of global positioning, and perform pose constraints on the initial pose estimation of the laser radar; the construction module 704 is used to construct a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after pose constraints.

[0090] It can be seen from this that the beneficial effects produced by the visual positioning prior map construction device based on laser radar provided by the present invention should be the same as the visual positioning prior map construction method based on laser radar described above. Therefore, the beneficial effects produced by the visual positioning prior map construction device based on laser radar will not be repeated here.

[0091] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a method for constructing a priori map of visual positioning based on a laser radar, the method comprising: obtaining ground-based laser radar point cloud data and mobile laser radar image data of a positioning area; constructing a local map of the ground-based laser radar point cloud data, and matching the moving frame of the mobile laser radar image data with the local map to obtain a current pose estimation of the laser radar; performing global positioning of the laser radar according to the key frame in the current pose estimation, and constructing an optimization factor graph according to the result of global positioning, and constraining the initial pose estimation of the laser radar; constructing a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after the pose constraint.

[0092] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0093] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the laser radar-based visual positioning prior map construction method provided by the above-mentioned methods, and the method includes: obtaining ground-based laser radar point cloud data and mobile laser radar image data of the positioning area; constructing a local map of the ground-based laser radar point cloud data, and matching the current frame of the mobile laser radar image data with the local map to obtain the current pose estimate of the laser radar; globally positioning the laser radar according to the key frames in the current pose estimate, and constructing an optimization factor graph according to the results of the global positioning, and performing pose constraints on the initial pose estimate of the laser radar; based on the final pose estimate of the laser radar after the pose constraint, constructing a prior map database for visual positioning in the positioning area.

[0094] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the laser radar-based visual positioning prior map construction method provided by the above-mentioned methods, the method comprising: obtaining ground-based laser radar point cloud data and mobile laser radar image data of the positioning area; constructing a local map of the ground-based laser radar point cloud data, and matching the current frame of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar; globally positioning the laser radar according to the key frames in the current pose estimate, and constructing an optimization factor graph based on the result of the global positioning, and performing pose constraints on the initial pose estimate of the laser radar; based on the final pose estimate of the laser radar after the pose constraints, constructing a prior map database for visual positioning in the positioning area.

[0095] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0096] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a priori maps for visual positioning based on laser radar, characterized in that: The method comprises: Obtain ground-based LiDAR point cloud data and mobile LiDAR image data of the positioning area; Constructing a local map of the ground-based laser radar point cloud data, and matching the moving frames of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar; Performing global positioning of the laser radar according to the key frames in the current pose estimation, and constructing an optimization factor graph according to the result of the global positioning to perform pose constraints on the initial pose estimation of the laser radar; Based on the final pose estimation of the laser radar after pose constraints, a priori map database for visual positioning in the positioning area is constructed.

2. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The constructing of a local map of the ground-based laser radar point cloud data includes: Extracting corner point features of the ground-based laser radar point cloud data in each frame of the mobile laser radar image data; In the neighborhood of each ground-based laser radar point cloud data, a covariance matrix is ​​constructed according to the coordinate values ​​of the ground-based laser radar point cloud data in the neighborhood; Determining multiple eigenvalues ​​of the covariance matrix by eigenvalue decomposition method; Dividing the ground-based laser radar point cloud data into plane feature point cloud data and irregular feature point cloud data according to the feature value; The corner point features, plane feature point cloud data and irregular feature point cloud data of each frame are combined to obtain a local map of the ground-based laser radar point cloud data.

3. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The step of matching the moving frame of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar includes: Matching the moving frames of the mobile laser radar image data with the point cloud feature points in the local map to obtain matching point cloud features; Calling the ICP algorithm to determine the transformation matrix between the performing frame and the matching point cloud feature; The mobile laser radar image data is transformed frame by frame through the transformation matrix to obtain the current posture estimation of the laser radar.

4. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The globally positioning the laser radar according to the key frame in the current pose estimation includes: Obtaining an initial pose estimate of the laser radar, and clipping the ground-based laser radar point cloud data using the initial pose estimate to obtain a global prior map; Matching the key frame in the current pose estimation with the global prior map to obtain a laser point cloud of the global prior map; Calling the ICP algorithm to determine the optimal transformation matrix between the laser point cloud of the key frame and the laser point cloud of the global prior map; Based on the optimal transformation matrix, the position information of the laser radar in the ground-based laser radar point cloud data is determined as the global positioning result of the laser radar.

5. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The optimization factor graph includes an inter-frame motion constraint factor graph, a loop constraint factor graph, and a global positioning constraint factor graph. The optimization factor graph is constructed according to the result of global positioning to constrain the initial pose estimation of the laser radar, including: Construct a global positioning constraint factor graph according to the result of global positioning; Determining the inter-frame motion constraint factor graph according to a pose transformation relationship between two adjacent key frames in a current pose estimation of the laser radar; Performing loop closure detection on the key frames in the current pose estimation to determine a loop closure constraint factor graph; According to the motion constraint factor graph, the loop constraint factor graph and the global positioning constraint factor graph, the initial pose estimation of the laser radar is subjected to pose constraints.

6. The method for constructing a priori maps based on laser radar for visual positioning according to claim 5, characterized in that: The performing loop detection on the key frames in the current pose estimation to determine a loop constraint factor graph includes: Determining a historical frame before a current key frame in the current pose estimate; Determine the Euclidean distance between the laser point cloud of the current key frame and the laser point cloud of the historical frame; When the Euclidean distance is less than a preset distance threshold, it is determined that a loop occurs in the laser radar, and the laser point cloud of the current key frame is matched with the laser point cloud of the historical frame to obtain a position transformation relationship between the current key frame and the historical frame; A loop constraint factor graph is determined based on the position transformation relationship.

7. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The method of constructing a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after the pose constraint comprises: Determine a reference image corresponding to each frame in the mobile laser radar image data, and determine a final pose estimate of the laser radar corresponding to each frame; Obtaining external parameters of the laser radar and the camera, and calculating the final pose estimation corresponding to each frame using the external parameters to obtain the pose of the reference image; Calling the SuperPoint model to extract candidate feature points of the reference image and descriptors of the candidate feature points, calling the SuperGlue model to match the descriptors of the candidate feature points, and filtering out local feature points and corresponding local feature descriptors from the candidate feature points according to the matching results; Calling the NetVLAD model to perform global feature extraction on the reference image to obtain an image global feature descriptor of the reference image; Projecting the two-dimensional coordinates of the local feature points according to the external parameters and the internal parameters of the laser radar camera to obtain the three-dimensional coordinates of the local feature points; Based on the reference image, the pose of the reference image, the image global feature descriptor of the reference image, the three-dimensional coordinates of the local feature points of the reference image, the local feature points of the reference image and the corresponding local feature descriptors, a priori map database for performing visual positioning in the positioning area is constructed.

8. The method for constructing a priori maps based on laser radar for visual positioning according to claim 1, characterized in that: The method further comprises: performing visual geographic positioning of a query image of a positioning area through the prior map database, wherein the process of visual geographic positioning of the query image comprises: Performing global feature extraction on the query image to obtain an image global feature descriptor of the query image; Matching the image global feature descriptor with the prior map database to determine a target reference image similar to the query image; Matching the first local feature points and the corresponding first local feature descriptor of the query image with the second local feature points and the corresponding second local feature descriptor of the target reference image; The pose estimation of the query image is determined according to the matched second local feature points as the visual geographic positioning result of the query image.

9. The method for constructing a priori maps based on laser radar for visual positioning according to claim 8, characterized in that: Determining the pose estimation of the query image according to the matched second local feature points includes: According to the matched second local feature point, query the corresponding target three-dimensional coordinates from the prior map database; Construct a perspective projection model of the query image corresponding to the target camera; Substituting the target two-dimensional coordinates of the first local feature point and the target three-dimensional coordinates into the perspective projection model for iterative solution to obtain the target camera pose and projection error; The target camera pose obtained by an iterative solution with the minimum projection error is determined as the pose estimate of the query image, as the visual geographic positioning result of the query image.

10. A device for constructing a priori maps based on visual positioning of laser radar, characterized in that: The device comprises: An acquisition module is used to acquire ground-based laser radar point cloud data and mobile laser radar image data of the positioning area; A matching module, used to construct a local map of the ground-based laser radar point cloud data, and match the moving frame of the mobile laser radar image data with the local map to obtain a current pose estimate of the laser radar; A constraint module, used to perform global positioning of the laser radar according to the key frame in the current pose estimation, and to construct an optimization factor graph according to the result of the global positioning to perform pose constraints on the initial pose estimation of the laser radar; A construction module is used to construct a priori map database for visual positioning in the positioning area based on the final pose estimation of the laser radar after pose constraints.

Images