A vehicle-infrastructure collaborative positioning method based on multi-view degradation perception

Abstract

The application discloses a vehicle-infrastructure collaborative positioning method based on multi-view degradation perception, relates to the technical field of intelligent transportation and automatic driving, and comprises the following steps: acquiring vehicle end point cloud data, and constructing a vehicle end point-surface residual error factor; acquiring measurement data of a vehicle-mounted inertial measurement unit, and constructing an IMU pre-integration factor through mechanical arrangement; acquiring roadside point cloud data, constructing a roadside residual error based on the roadside point cloud data and the vehicle end point cloud data, and constructing a roadside DICP factor; constructing a factor graph model based on the vehicle end point-surface residual error factor, the IMU pre-integration factor and the roadside DICP factor, updating the pose information of a vehicle, and obtaining a positioning result. The application can effectively overcome the defects of drift and unstable registration caused by the insufficient observability of a laser radar in a degradation scene, solve the problems of high delay and insufficient constraints of multi-view vehicle-road collaborative positioning, and realize long-time, stable and high-precision real-time positioning in a complex traffic environment.

Description

Technical Field

[0001] This application relates to the fields of intelligent transportation and autonomous driving technology, and in particular to a vehicle-infrastructure cooperative localization method based on multi-view degradation perception. Background Technology

[0002] With the rapid development of intelligent transportation systems and autonomous driving technology, achieving centimeter-level stable positioning of vehicles in complex traffic environments has become a key fundamental capability. Traditional single-sensor positioning methods, such as laser odometry based on lidar, inertial navigation systems based on inertial measurement units, and vision-based visual odometry, all exhibit varying degrees of limitations in real-world environments, making it difficult to meet the positioning requirements of long-term, high robustness, and low latency.

[0003] LiDAR, with its high precision and strong resistance to light, is widely used in autonomous driving positioning. However, in degraded scenarios with low structure and weak texture, such as tunnels, corridor-type roads, urban "street valley effect", long-distance sparse point clouds, painted surfaces of lane lines or walls, point cloud features exhibit low observability (e.g., planar repetition, near-zero curvature, unidirectional structure, etc.). This results in traditional point-to-point or point-to-surface 3D point cloud registration methods (Iterative Closest Point, ICP) lacking effective constraints in the degradation direction, easily leading to drift accumulation or even registration failure.

[0004] To address the degradation problem, related technologies can introduce geometric decomposition methods or multi-view point cloud compensation strategies. However, point cloud fusion under multi-view conditions often requires waiting for all sensor data to arrive completely and undergo global optimization before the pose can be updated, resulting in significant system latency. Furthermore, traditional ICP registration algorithms fail to adaptively identify degradation directions during the optimization process, resulting in insufficient utilization efficiency of degradation features and poor robustness of the system in critical scenarios.

[0005] On the other hand, related technologies can also improve positioning performance through the tight coupling method of laser-inertial measurement unit (IMU). However, inertial drift caused by this method is unavoidable, and it is difficult to maintain positioning accuracy in the long term when used alone. Although IMU pre-integration can constrain short-term drift, the system will still accumulate significant errors when laser constraint is weak in degraded environments.

[0006] On the other hand, some related technologies have adopted vehicle-road cooperative positioning strategies, leveraging the advantages of roadside LiDAR, such as its higher installation position, wider field of view, and greater variability in viewing angles, to significantly improve the robustness of the positioning system in degraded scenarios. However, existing vehicle-road cooperative positioning schemes rely on a global optimization process when performing multi-view point cloud fusion, making it impossible to perform delay-free correction of onboard odometer results and lacking real-time perception and adaptive optimization strategies for the degradation level of roadside radar point clouds. Therefore, there is currently a lack of a vehicle-infrastructure cooperative localization method that simultaneously possesses vehicle-mounted laser-inertial tight coupling capability, degradation observability perception capability, multi-view point cloud adaptive registration capability, delay-free keyframe insertion strategy, and multi-source factor graph global optimization capability, so as to achieve long-term, stable, and high-precision localization in complex traffic environments. Summary of the Invention

[0007] This application provides a vehicle-infrastructure cooperative localization method based on multi-view degradation perception to address the shortcomings of the aforementioned related technologies. The technical solution is as follows: In a first aspect, this application provides a vehicle-infrastructure cooperative localization method based on multi-view degradation perception, the method comprising: Acquire vehicle endpoint cloud data collected by vehicle-mounted LiDAR, construct vehicle endpoint local surface elements based on vehicle endpoint cloud key frames, calculate vehicle endpoint residuals from each point in the vehicle endpoint local surface elements to the surface elements, and construct vehicle endpoint-surface residual factors based on the vehicle endpoint residuals. The measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes is acquired. Based on the measurement data, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit, mechanical arrangement is performed to construct the IMU pre-integration factor. Obtain roadside point cloud data collected from roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals; A factor graph model is constructed based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor. The factor graph model is solved to update the vehicle's pose information and obtain the vehicle's positioning result.

[0008] In one alternative embodiment of the first aspect, the construction of vehicle-side local polygons based on vehicle-side endpoint cloud keyframes includes: For each vehicle-end cloud keyframe, a neighborhood is searched with each point as the center and a preset neighborhood radius to obtain the neighborhood, and all points in the neighborhood are taken as local surface elements of the vehicle end. The center of the vehicle end local surface element is obtained by calculating the mean value of the field. The local covariance matrix of the vehicle end is calculated based on the distance between each point in the local surface element and the center of the vehicle end element. Based on the local covariance matrix of the vehicle end, eigenvalue decomposition is performed to obtain an eigenvalue diagonal matrix. Based on the eigenvalues ​​in the eigenvalue diagonal matrix, the normal vector of the vehicle end surface element is determined, and the planar confidence of the local surface element of the vehicle end is calculated.

[0009] In one alternative embodiment of the first aspect, the calculation of the vehicle-end residual from each point to the local surface element in the vehicle-end area includes: Calculate the pose of each point in the local surface element of the vehicle end in the vehicle coordinate system, transform the pose to the local map coordinate system, and calculate the normal distance to the center of the vehicle end surface element based on the transformed pose. The vehicle end residual from the point to the surface element is calculated based on the normal vector of the vehicle end surface element and the normal distance, using the following formula: ; in, This represents the function used to calculate the pose. Let i be the i-th point in the center of the vehicle end face element. For the center of the vehicle end face element, For the residual at the car end, This is the normal vector of the vehicle end face element.

[0010] In one alternative embodiment of the first aspect, the step of constructing the vehicle end-surface residual factor based on the vehicle end residual includes: Based on the vehicle-end residual and the vehicle-end local covariance matrix corresponding to each vehicle-end local surface element, the vehicle-end point-surface residual factor is constructed: ; in, The end-to-surface residual factor; This is the information matrix corresponding to the residual.

[0011] In one alternative of the first aspect, the degradation-aware ICP performed based on roadside point cloud data and vehicle-side point cloud data to construct roadside residuals includes: Based on the roadside keyframes in the roadside point cloud data, the corresponding roadside local surface elements are constructed, and the center of the roadside surface elements is calculated. The local covariance matrix of the roadside is calculated based on the distance between each point in the local surface element and the center of the roadside surface element. Eigenvalue decomposition is performed based on the local covariance matrix of the roadside to obtain the eigenvalue diagonal matrix. Based on the eigenvalues ​​in the eigenvalue diagonal matrix, the normal vector of the roadside element and the degree of geometric degradation are determined. The corresponding viewpoint weight is calculated based on the preset suppression coefficient and the degree of geometric degradation. The roadside residuals are constructed, and the formula is applied: ; in, The normal vector of the roadside element. For the center of the roadside element, For pose variables, For vehicle-end cloud, For roadside residuals, This refers to the translation amount in coordinate system transformation. For viewpoint weights.

[0012] In one alternative to the first aspect, the step of constructing the roadside DICP factor based on the roadside residual includes: The roadside DICP factor is constructed based on the roadside residual and roadside local covariance matrix corresponding to each roadside local element: ; in, This is the information matrix corresponding to the residual. The roadside DICP factor is mentioned.

[0013] In one alternative of the first aspect, the factor graph model is constructed based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor, and the objective function of the factor graph model is expressed as: ; in, Let the objective function be... This is the IMU pre-integration factor.

[0014] Secondly, this application also provides a vehicle-infrastructure cooperative positioning device based on multi-view degradation perception, comprising: The vehicle-mounted data processing unit is used to acquire vehicle-end point cloud data collected by the vehicle-mounted lidar, construct vehicle-end local surface elements based on vehicle-end point cloud key frames, calculate the vehicle-end residual from each point in the vehicle-end local surface element to the surface element, and construct the vehicle-end point-surface residual factor based on the vehicle-end residual. The IMU data processing unit is used to acquire measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes, and to perform mechanical arrangement based on the measurement data, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit to construct the IMU pre-integration factor. The roadside data processing unit is used to acquire roadside point cloud data collected by roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals. The positioning unit is used to construct a factor graph model based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor, solve the factor graph model to update the vehicle's pose information, and obtain the vehicle's positioning result.

[0015] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method provided by the first aspect of this application or any implementation thereof.

[0016] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of this application or any implementation thereof.

[0017] The beneficial effects of the technical solution provided in this application include at least the following: This application constructs local surface elements and calculates vehicle-side residuals by combining them with IMU pre-integration factors to form a tight laser-IMU coupling. In degraded scenarios, surface element modeling naturally highlights effective directions and suppresses degraded directions, avoiding drift and improving short-range robustness. On the roadside, degradation-aware ICP adaptive analysis of point cloud observability is adopted, and roadside DICP factors are constructed. Multi-view point clouds bring different main direction information, which, after fusion, enhances the overall constraint and significantly improves registration accuracy and robustness. Through a zero-delay keyframe insertion strategy, the vehicle-side can immediately utilize strong roadside observations without waiting for global optimization, achieving zero-wait collaboration and significantly reducing system latency. The vehicle-side point-surface residual factors, IMU pre-integration factors, and roadside DICP factors are uniformly constructed into a factor graph model for solution, realizing integrated optimization of temporal, geometric, viewpoint, synchronization, and calibration, reducing fragmentation and suboptimal results, and improving positioning consistency and accuracy.

[0018] The method provided in this application has good scalability and can be easily integrated with loop closure, visual factors, or geometric priors to form a stronger combination. This effectively overcomes the drift and registration instability caused by insufficient observability of lidar in degraded scenarios such as tunnels and street canyons, and solves the problems of high latency and insufficient constraints in multi-view vehicle-road cooperative positioning, achieving long-term, stable, and high-precision real-time positioning in complex traffic environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a vehicle-infrastructure cooperative localization method based on multi-view degradation perception provided in an embodiment of this application. Figure 2 This is one of the data illustrations comparing a vehicle-infrastructure cooperative localization method based on multi-view degradation perception with related technologies provided in the embodiments of this application; Figure 3 This is the second data illustration comparing a vehicle-infrastructure cooperative localization method based on multi-view degradation perception with related technologies provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of a vehicle-infrastructure cooperative positioning device based on multi-view degradation perception provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or apparatus.

[0023] It should be noted that the terms "first" and "second" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in an order other than those described or illustrated herein.

[0024] The present application will now be described in detail with reference to specific embodiments.

[0025] Next, combine Figure 1This paper introduces a vehicle-infrastructure cooperative localization method based on multi-view degradation perception, provided by an embodiment of this application. For details, please refer to... Figure 1 , Figure 1 This illustration shows a flowchart of a vehicle-infrastructure cooperative localization method based on multi-view degradation perception, provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101, acquire vehicle endpoint cloud data collected by vehicle-mounted lidar, construct vehicle endpoint local surface elements based on vehicle endpoint cloud key frames, calculate vehicle endpoint residual from each point to the surface element in the vehicle endpoint local surface element, and construct vehicle endpoint-surface residual factor based on the vehicle endpoint residual. S102: Acquire the measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes, and perform mechanical arrangement based on the measurement data, the kinematic model of the vehicle-mounted inertial measurement unit and the error state propagation model to construct the IMU pre-integration factor; S103, acquire roadside point cloud data collected by roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals; S104. A factor graph model is constructed based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor. The factor graph model is solved to update the vehicle's pose information and obtain the vehicle's positioning result.

[0026] Specifically, S101 can collect vehicle-end cloud data via onboard LiDAR during vehicle operation, and construct vehicle-end local element data based on each key point cloud frame in the vehicle-end cloud data, including: For each frame of point cloud, the neighborhood can be searched based on a preset neighborhood radius, with each point as the center. Local surface elements at the vehicle end are constructed based on each point within the domain; Calculate the neighborhood mean of the local surface elements at the vehicle end to obtain the neighborhood centroid: ; Where i is the ordinal number of the midpoint of the local surface element at the vehicle end; Let i represent the i-th point; k represents the total number of points in the neighborhood; This represents the neighborhood centroid, which is also the center of the vehicle end face element.

[0027] Furthermore, the local covariance matrix of the vehicle end is calculated based on the distance between each point within the local surface element and the center of the vehicle end surface element, to measure the discreteness and distribution shape of the neighborhood points in the three principal directions, using the formula: ; Next, eigenvalue decomposition is performed based on the local covariance matrix of the vehicle end to obtain an eigenvalue diagonal matrix. The normal vector of the vehicle end surface element is determined based on the eigenvalues ​​in the eigenvalue diagonal matrix, using the formula: ; Specifically, normal vector The calculation process can be represented as follows: ; in, The eigenvalue diagonal matrix is ​​used to extract eigenvalues ​​along the three principal directions. , These are the primary, secondary, and minor eigenvalues, respectively. Furthermore, select the smallest eigenvalue. corresponding vector As the normal vector of the vehicle end face element .

[0028] It should be noted that, if A very small size indicates a clear surface orientation and strong planarity; if Locally, it resembles voxel noise, and the normal direction is unreliable.

[0029] Furthermore, the curvature of the local surface element at the vehicle end is calculated as the planar confidence level of the local surface element at the vehicle end: ; in, Indicates the plane confidence level. The smaller the value, the flatter the local surface element, and the more reliable the normal. Increasing the weight of a local surface element at the vehicle end indicates that the structure is not obvious or the noise is high. The weight of that surface element in the optimization can be adjusted accordingly.

[0030] Furthermore, each point in the local surface element of the vehicle end is located in the vehicle coordinate system. The pose of each point in the local surface element in the vehicle coordinate system is calculated, and the pose is transformed to the local map coordinate system. Based on the transformed pose, the normal distance to the center of the vehicle end surface element is calculated, and the normal vector of the vehicle end surface element is used as the basis for the calculation. The vehicle end residual from the point to the surface element is obtained by calculating the distance from the point to the surface element. Apply the formula: ; in, This represents the function used to calculate the pose.

[0031] Subsequently, based on the vehicle-end residual corresponding to each local surface element at the vehicle end... and the vehicle-side local covariance matrix The end-to-surface residual factor of the vehicle is constructed as follows: ; in, For the end-to-surface residual factor, This is the information matrix corresponding to the residual, specifically the vehicle-side local covariance matrix. The reverse.

[0032] In some embodiments, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit can be pre-constructed in S102.

[0033] Specifically, the kinematic model can be represented as: ; Among them, posture From angular velocity Advancement, speed Gravity Compared to Impact, location Obtained by velocity integration; These are the gyroscope and the accelerator bias, respectively. Indicates the rate of change of velocity. Indicates attitude variation. This indicates that the rate of change of position is equal to the rate of change of velocity. .

[0034] Specifically, the error state propagation model can be expressed as: ; in, The coefficient matrix, This refers to the error of the vehicle-mounted inertial measurement unit. The coefficient matrix of the noise term. For noise terms, This is the first-order differential term of the state variable.

[0035] Specifically, between every two adjacent keypoint cloud frames, intermediate quantities can be obtained through pre-integration. These intermediate quantities, combined with the error state propagation model, constitute a priori factor-constrained short-time motion to obtain the pre-integrated residual term. .

[0036] The resulting IMU pre-integration factor is then expressed as: ; in, For the pre-integrated residual term, This is the pre-integrated information matrix.

[0037] In some implementations, S103 specifically includes: It can acquire roadside point cloud data collected simultaneously by roadside point cloud acquisition devices during vehicle operation, construct corresponding roadside local pixels based on roadside keyframes in the roadside point cloud data, and extract the centroid of the roadside local pixels to obtain the roadside pixel center. ; Calculate the roadside local covariance matrix of roadside local elements. : ; in, This represents the j-th point in the local surface element of the roadside.

[0038] Furthermore, eigenvalue decomposition is performed based on the roadside local covariance matrix: ; Based on the obtained eigenvalue diagonal matrix The feature values ​​in the three principal directions were extracted. .

[0039] Specifically, These are the primary, secondary, and smallest eigenvalues, respectively. The vector corresponding to the smallest eigenvalue is selected as the roadside element normal vector. ; It should be noted that the roadside local covariance matrix Similar to the vehicle-side local covariance matrix C, but with the roadside local covariance matrix focusing on the roadside viewpoint, it can reveal the reliability of this point cloud in three directions. For example: like The local surface element is like a "thin line", which is strong in the direction of perpendicularity and weak in other directions; like The local surface element image is a "thin plane", which is strong in the normal direction and weak in the tangential direction; If the three are close: the point cloud of local surface elements is like "uniform noise", which is difficult to provide clear constraints.

[0040] Furthermore, the degree of geometric degradation of local surface elements on the roadside is calculated based on the eigenvalues ​​in the three main directions. : ; Specifically, As an indicator reflecting the degree of geometric degradation, a perspective with poor observability has a greater impact. value.

[0041] To suppress views with poor observability, view weights can be calculated based on the degree of geometric degradation, using the following formula: ; It can automatically adjust the perspective weight based on the degree of degradation; in, The lower the weight, the more effectively the error propagation in the degradation direction can be suppressed, thus improving registration stability; The suppression coefficient, Larger → More cautious Smaller individuals tend to trust observations from a different perspective more.

[0042] Then, based on the parameters calculated above and combined with the vehicle-end cloud data, the roadside residuals can be constructed using the following formula: ; in, For the corresponding roadside element normal vector, For the center of the roadside element, For pose variables, For vehicle-end cloud, For roadside residuals, The translation amount in coordinate system transformation is determined by introducing viewpoint weights. It can automatically reduce the contribution of the degenerate perspective, making the optimization more stable.

[0043] Then, based on the roadside residuals and roadside local covariance matrix corresponding to each roadside local element, the roadside DICP (Degeneracy-aware Point Cloud Registration) factor is constructed: ; in, This is the information matrix corresponding to the residual, specifically the inverse of the roadside local covariance matrix.

[0044] In some embodiments, the vehicle pose can be estimated based on the vehicle-end cloud data to obtain the estimated vehicle pose at the current moment. It can estimate the vehicle's pose based on roadside point cloud data, thus obtaining the estimated roadside pose at the current moment. Based on vehicle-side pose estimation Roadside estimated pose Error terms can be calculated , represented as: ; Lie algebra Error terms Logarithmic mapping to Lie algebra vector L Further calculation of the L2 norm can serve as a measure of consistency. (You can also separate the rotation / translation components and set dual values); If the consistency metric is less than the error threshold : ; This indicates that the vehicle's self-estimation is sufficiently consistent with the roadside's observation of the vehicle. The current keyframe of the vehicle can be inserted into the factor graph without waiting for loop closure or global convergence. In this way, timeliness is preserved, and the system can immediately correct vehicle drift with the help of strong roadside constraints, greatly reducing the loss of "good information that comes late".

[0045] In some embodiments, S104 specifically includes: A factor graph model can be constructed based on the vehicle end-to-surface residual factor, the IMU pre-integration factor, and the roadside DICP factor. The objective function of the factor graph model is expressed as: ; Understandably, the objective function constructed above incorporates the residuals from various sources. With the corresponding information matrix (The inverse of the covariance matrix) weighted unified solution: the data from which the reliability is higher corresponds to a larger weight value; in degenerate environments, the information matrix... It will automatically weaken to reduce unreliable degenerate perspectives.

[0046] The vehicle's pose is updated based on the calculation results by solving the objective function: ; in, Is The minimum increment on, Ensure rotation update in It is continuous and numerically stable; it is usually paired with Gauss-Newton / LM iteration for more reliable convergence. and These represent the pose matrices before and after the update, respectively. Therefore, the vehicle's pose information in the current frame can be updated to obtain the vehicle's localization result, which specifically includes the vehicle's position, speed, and attitude information at the corresponding moment.

[0047] In some embodiments, the updated pose information can be mapped onto the target reference frame.

[0048] For example, the following formula can be applied: ; in, For vehicles in The coordinates of a point in the current coordinate system. For vehicles in Coordinates in the target coordinate system The translation amount used for coordinate system transformation. This is the rotation matrix from the current coordinate system to the target coordinate system.

[0049] For example, you can set the target coordinate system to the world coordinate system to complete the transition from the coordinate system. Connect to the world coordinate system Rigid body transformation; the target coordinate system can also be specified as a road reference system (for convenient road asset management / high-precision map docking) or a vehicle system (for convenient control and obstacle avoidance), etc., but the embodiments of this application do not limit this.

[0050] In some embodiments, a time synchronization error term can also be added to the above objective function, expressed as: ; in, For time offset, For the first frame Data observation time, For the first The observation time of frame IMU observation data.

[0051] The new objective function can be expressed as: ; By introducing a time synchronization error term, the time synchronization accuracy of vehicle-side point clouds and roadside point clouds can be improved, reducing the error caused by asynchronous observation time. This allows for better coordination of multi-view information and helps improve positioning accuracy.

[0052] In some specific embodiments, a vehicle-infrastructure cooperative localization method based on multi-view degradation perception provided in this application is verified using a real tunnel scenario as an experimental scenario. Data collected jointly by vehicle-mounted and roadside multi-source sensors are used to conduct a detailed analysis of the effectiveness, robustness, and trajectory accuracy improvement of the algorithm proposed in this application in degradation scenarios, and compare it with a variety of existing mainstream methods to demonstrate the technical advantages of this application.

[0053] Specifically, this application selects a typical urban tunnel as the test environment. This tunnel is a closed structure with uniform wall heights on both sides and a flat ceiling, exhibiting strong linear characteristics overall. In this type of degraded scenario, the point cloud obtained by the vehicle-mounted LiDAR mostly shows a parallel distribution, with extremely sparse forward structure. Especially when the vehicle is traveling at high speed or encounters obstruction, the normals of local surface elements are highly repetitive, resulting in severe degradation of geometric observability. Traditional ICP and LiDAR-Inertial Fusion algorithms are prone to drift accumulation in the forward direction, leading to significant trajectory deviations. Therefore, this scenario is typical for verifying the performance of this application in degraded environments.

[0054] Specifically, the experimental platform consists of two parts: a vehicle-mounted unit and a roadside unit. The vehicle-mounted unit includes a Velodyne-32C lidar, a MEMS IMU (ADIS-16470), a tactical-grade fiber optic IMU (XW-GI7660), a GNSS receiver (PolaRx5), and a high-performance antenna (Zephyr Model 2). The MEMS IMU is used as a reference ground truth trajectory through post-processing, and the lidar and IMU data are used as online positioning inputs for the algorithm in this application. For the roadside infrastructure, multiple fixed lidars can be deployed at several locations at the tunnel entrance and in the middle, supplementing the structural information in the forward and ceiling directions by scanning the vehicle from different angles. In addition, in order to construct a high-precision benchmark map that unifies the tunnel coordinate system with the roadside sensor coordinates, the experiment used a mobile data acquisition vehicle carrying a high-resolution lidar and a combined navigation system to conduct offline mapping inside the tunnel. At the same time, high-reflectivity features such as cones and reflective strips were deployed inside the tunnel for manual calibration of corresponding points. These artificial markers can be clearly presented in both the vehicle-mounted and roadside point clouds, which facilitates the establishment of accurate external parameter relationships.

[0055] During the data processing phase, the vehicle-mounted LiDAR first undergoes preprocessing steps such as distortion correction, time synchronization, and point cloud filtering, and then performs surface element construction and normal estimation on the vehicle. IMU data is pre-integrated between keyframes to maintain the stability of continuous estimation over a short period. The roadside LiDAR is unified to the tunnel navigation system through extrinsic parameter mapping, and the degree of degradation is analyzed based on the covariance matrix of the point cloud normals. For views with highly repetitive structures or excessive distances, the degree of degradation increases significantly, and the corresponding degradation weight automatically decreases. For views with ideal viewing angles and clear geometric structures, higher weights are assigned, ensuring that the contribution of each viewpoint in DICP registration matches its actual observability, thereby avoiding matching errors caused by degraded views.

[0056] The results of this application were compared with common laser odometry methods, specifically FAST-LIO2 and LIO-SAM. Fastlio2 stands for Fast Direct LiDAR-inertial Odometry, and LIO-SAM stands for Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping. SSP-LIO stands for Same-Surfel-Point LiDAR-Inertial Odometry, indicating a method based on vehicle-mounted laser point clouds and IMU data. PRLIO-SAM stands for Probability Roadside LiDAR Inertial Odometry – iSAM, which is the method provided in this application.

[0057] In this embodiment, the post-processed results of tactical-level IMU + GNSS are used as the ground truth trajectory. First, the position error RMSE is compared, such as... Figure 2 As shown, the errors of FAST-LIO2 in the east and north directions are 2.04 m and 3.38 m, respectively, while the error in the vertical direction is as high as 6.32 m. LIO-SAM and SSP-LIO also exhibit varying degrees of drift in these three directions, with the vertical error consistently remaining between 2.87 and 4.80 m. In contrast, the method of this application achieves a significant advantage in the same tunnel section, with errors in the east and north directions both controlled within 0.48 m, and the vertical error only 0.28 m. This order-of-magnitude improvement reflects the effective suppression of vertical drift by roadside geometric anchors and DICP.

[0058] Regarding attitude angle error, such as Figure 3 As shown, the method provided in this application demonstrates superior performance. FAST-LIO2 achieves a roll and pitch angle error (RMSE) exceeding 1°, while LIO-SAM and SSP-LIO maintain errors within the 0.5–1° range, respectively. In the comparison, the method provided in this application shows the most significant improvement in attitude estimation accuracy, with roll and pitch angle errors controlled at 0.33° and 0.35°, respectively. Due to the more pronounced geometric orientation in the tunnel environment, the heading angle error remains optimal at 0.06°. This result indicates that multi-view structural compensation can significantly improve attitude drift, especially in the pitch direction, while the degradation weighting mechanism suppresses jumps caused by unreliable directions, maintaining optimization stability.

[0059] Finally, considering the overall trajectory performance, FAST-LIO2 and LIO-SAM exhibited significant trajectory drift in the middle section of the tunnel due to severe structural repetition; while SSP-LIO performed relatively stably in the initial section, it still showed deviations in the deeper parts of the tunnel. In contrast, the trajectory of this invention remained smooth, continuous, and highly consistent with the true value throughout the entire tunnel section. Especially in the three regions of entrance, middle section, and exit, the trajectory did not show significant deflection or jumps, demonstrating superior robustness throughout the entire section.

[0060] In summary, this embodiment demonstrates that the vehicle-infrastructure cooperative localization method based on multi-view degradation perception provided in this application has significant advantages in geometrically degraded environments such as tunnels. Through degradation perception weighting, multi-view geometric compensation, delay-free keyframe insertion, and multi-source factor graph optimization, this application can robustly handle heterogeneous information from the vehicle and roadside, achieving high-precision, low-drift vehicle-infrastructure cooperative localization results.

[0061] The following are apparatus embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of this application.

[0062] Please see below. Figure 4 The image below is a schematic diagram of a vehicle-infrastructure cooperative localization device based on multi-view degradation perception, provided as an exemplary embodiment of this application. The device includes: The vehicle-mounted data processing unit is used to acquire vehicle-end point cloud data collected by the vehicle-mounted lidar, construct vehicle-end local surface elements based on vehicle-end point cloud key frames, calculate the vehicle-end residual from each point in the vehicle-end local surface element to the surface element, and construct the vehicle-end point-surface residual factor based on the vehicle-end residual. The IMU data processing unit is used to acquire measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes, and to perform mechanical arrangement based on the measurement data, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit to construct the IMU pre-integration factor. The roadside data processing unit is used to acquire roadside point cloud data collected by roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals. The positioning unit is used to construct a factor graph model based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor, solve the factor graph model to update the vehicle's pose information, and obtain the vehicle's positioning result.

[0063] It should be noted that the device provided in the above embodiments, when executing a vehicle-infrastructure cooperative localization method based on multi-view degradation perception, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0064] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0065] Please see Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0066] like Figure 5 As shown, the electronic device includes a processor and a memory.

[0067] In this embodiment, the processor is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array).

[0068] A processor can also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state and is also called the CPU (Central Processing Unit). The coprocessor is a low-power processor used to process data in the standby state.

[0069] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this application, the non-transitory computer-readable storage media in the memory are used to store at least one instruction, which is executed by a processor to implement the methods in the embodiments of this application.

[0070] In some embodiments, the electronic device further includes a peripheral device interface and at least one peripheral device. The processor, memory, and peripheral device interface are connected via a bus or signal line. Each peripheral device is connected to the peripheral device interface via a bus, signal line, or circuit board. Specifically, the peripheral device includes: a display screen, a camera, and audio circuitry. The peripheral device interface can be used to connect at least one I / O (Input / Output) related peripheral device to the processor and memory.

[0071] In some embodiments of this application, the processor, memory, and peripheral device interfaces are integrated on the same chip or circuit board; in other embodiments of this application, any one or two of the processor, memory, and peripheral device interfaces can be implemented on separate chips or circuit boards. This application does not specifically limit the implementation in this regard.

[0072] The electronic device structural block diagrams shown in the embodiments of this application do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0073] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the methods in any of the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A vehicle-infrastructure cooperative localization method based on multi-view degradation perception, characterized in that, include: Acquire vehicle endpoint cloud data collected by vehicle-mounted LiDAR, construct vehicle endpoint local surface elements based on vehicle endpoint cloud key frames, calculate vehicle endpoint residuals from each point in the vehicle endpoint local surface elements to the surface elements, and construct vehicle endpoint-surface residual factors based on the vehicle endpoint residuals. The measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes is acquired. Based on the measurement data, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit, mechanical arrangement is performed to construct the IMU pre-integration factor. Obtain roadside point cloud data collected from roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals; A factor graph model is constructed based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor. The factor graph model is solved to update the vehicle's pose information and obtain the vehicle's positioning result.

2. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 1, characterized in that, The construction of vehicle-side local surface elements based on vehicle-side endpoint cloud keyframes includes: For each vehicle-end cloud keyframe, a neighborhood is searched with each point as the center and a preset neighborhood radius to obtain the neighborhood, and all points in the neighborhood are taken as local surface elements of the vehicle end. The center of the vehicle end local surface element is obtained by calculating the mean value of the field. The local covariance matrix of the vehicle end is calculated based on the distance between each point in the local surface element and the center of the vehicle end element. Based on the local covariance matrix of the vehicle end, eigenvalue decomposition is performed to obtain an eigenvalue diagonal matrix. Based on the eigenvalues ​​in the eigenvalue diagonal matrix, the normal vector of the vehicle end surface element is determined, and the planar confidence of the local surface element of the vehicle end is calculated.

3. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 2, characterized in that, The calculation yields the vehicle-end residual from each point in the local surface element to the surface element, including: Calculate the pose of each point in the local surface element of the vehicle end in the vehicle coordinate system, transform the pose to the local map coordinate system, and calculate the normal distance to the center of the vehicle end surface element based on the transformed pose. The vehicle end residual from the point to the surface element is calculated based on the normal vector of the vehicle end surface element and the normal distance, using the following formula: ； in, This represents the function used to calculate the pose. Let i be the i-th point in the center of the vehicle end face element. For the center of the vehicle end face element, For the residual at the car end, This is the normal vector of the vehicle end face element.

4. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 3, characterized in that, The process of constructing the vehicle end-surface residual factor based on the vehicle end residual includes: Based on the vehicle-end residual and the vehicle-end local covariance matrix corresponding to each vehicle-end local surface element, the vehicle-end point-surface residual factor is constructed: ； in, The end-to-surface residual factor; This is the information matrix corresponding to the residual.

5. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 4, characterized in that, The degradation-aware ICP performed based on roadside point cloud data and vehicle-side point cloud data is used to construct roadside residuals, including: Based on the roadside keyframes in the roadside point cloud data, the corresponding roadside local surface elements are constructed, and the center of the roadside surface elements is calculated. The local covariance matrix of the roadside is calculated based on the distance between each point in the local surface element and the center of the roadside surface element. Eigenvalue decomposition is performed based on the local covariance matrix of the roadside to obtain the eigenvalue diagonal matrix. Based on the eigenvalues ​​in the eigenvalue diagonal matrix, the normal vector of the roadside element and the degree of geometric degradation are determined. The corresponding viewpoint weight is calculated based on the preset suppression coefficient and the degree of geometric degradation. The roadside residuals are constructed, and the formula is applied: ； in, The normal vector of the roadside element. For the center of the roadside element, For pose variables, For vehicle-end cloud, For roadside residuals, This refers to the translation amount in coordinate system transformation. For viewpoint weights.

6. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 5, characterized in that, The method of constructing the roadside DICP factor based on the roadside residuals includes: The roadside DICP factor is constructed based on the roadside residual and roadside local covariance matrix corresponding to each roadside local element: ； in, This is the information matrix corresponding to the residual. The roadside DICP factor is mentioned.

7. The vehicle-infrastructure cooperative localization method based on multi-view degradation perception according to claim 6, characterized in that, The factor graph model is constructed based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor. The objective function of the factor graph model is expressed as: ； in, Let the objective function be... This is the IMU pre-integration factor.

8. A vehicle-infrastructure cooperative positioning device based on multi-view degradation perception, characterized in that, include: The vehicle-mounted data processing unit is used to acquire vehicle-end point cloud data collected by the vehicle-mounted lidar, construct vehicle-end local surface elements based on vehicle-end point cloud key frames, calculate the vehicle-end residual from each point in the vehicle-end local surface element to the surface element, and construct the vehicle-end point-surface residual factor based on the vehicle-end residual. The IMU data processing unit is used to acquire measurement data of the vehicle-mounted inertial measurement unit between two adjacent keyframes, and to perform mechanical arrangement based on the measurement data, the kinematic model and error state propagation model of the vehicle-mounted inertial measurement unit to construct the IMU pre-integration factor. The roadside data processing unit is used to acquire roadside point cloud data collected by roadside infrastructure at the same time, perform degradation perception ICP based on roadside point cloud data and vehicle point cloud data, construct roadside residuals, and construct roadside DICP factors based on the roadside residuals. The positioning unit is used to construct a factor graph model based on the vehicle end-face residual factor, the IMU pre-integration factor, and the roadside DICP factor, solve the factor graph model to update the vehicle's pose information, and obtain the vehicle's positioning result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images