Lightweight multi-source fusion high-precision positioning method based on traffic sign recognition

Abstract

The invention discloses a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, and belongs to the technical field of satellite positioning, and the method comprises the steps: building a traffic sign detection data set, and training a traffic sign recognition model; detecting the road traffic sign in real time based on the traffic sign recognition model to obtain a road traffic sign detection result; on the basis of a road traffic sign detection result, plug-and-play lightweight multi-source fusion positioning is realized on the basis of the factor graph; according to the lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, the effectiveness of a sensor fusion process is dynamically adjusted, and plug and play among sensors is realized, so that excessive energy consumption introduced by sensor fusion is remarkably reduced; the data fusion difficulty is reduced, and the positioning precision in a complex environment is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of satellite positioning technology, and in particular relates to a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition. Background Art

[0002] In special environments such as urban canyons or tunnels, traditional global satellite navigation systems (GNSS) are prone to signal loss and are difficult to provide stable positioning services. With the development of inertial navigation systems (INS), integrated navigation technology of INS and GNSS has emerged. This integrated system uses INS to provide short-term and high-frequency positioning services when the GNSS signal is weak or lost. Thanks to the INS's anti-interference ability to changes in the external environment, it can achieve continuous and independent positioning. However, when INS operates independently, its performance will drift over time, especially when using low-cost micro-electromechanical systems (MEMS)-inertial measurement units (IMUs), the drift of this error will gradually increase.

[0003] In the field of computer vision and robotics, simultaneous localization and mapping (SLAM) technology achieves precise navigation by estimating the position and building a map of the environment during movement. This technology relies on incremental maps and self-localization to achieve autonomous navigation. In recent years, single-sensor SLAM systems such as LG-SLAM, IMLS-SLAM, SUMA, and LOAM have made significant progress. Despite the use of closed-loop correction technology, the errors of these SLAM systems will still accumulate as the movement distance increases. In large-scale outdoor activities, due to the difficulty in forming a closed loop, a single SLAM system cannot provide absolute navigation information.

[0004] Therefore, the integration of GNSS / INS combined navigation system and SLAM technology can not only reduce navigation drift and reduce the reliance on closed-loop correction in an environment with weak GNSS signals, but also provide accurate absolute navigation information, realizing the complementary advantages of these technologies. This fusion strategy provides a new solution for positioning and navigation in complex environments and optimizes the performance of the navigation system.

[0005] In recent years, studies have combined inertial navigation systems (INS) with visual SLAM to develop high-precision visual inertial odometry (VINS) and general navigation frameworks such as VINS-Fusion. These systems integrate IMU pre-integration measurements with visual feature observations through tightly coupled nonlinear optimization techniques.

[0006] In the VINS-Fusion system, GNSS is integrated into the global estimator and cooperates with VINS as a local estimator to estimate the IMU bias with GNSS assistance. For details, see the literature: Qin T, Cao S, Pan J, et al. A general optimization-based framework for global pose estimation with multiple sensors [J]. arXiv preprint arXiv: 1901.03642, 2019. The literature Jin R, Liu J, Zhang H, et al. Fast and accurate initialization for monocular vision / INS / GNSS integrated system onland vehicle [J]. IEEE Sensors Journal, 2021, 21 (22): 26074-26085 describes the parallel startup of GNSS / INS and VINS, which is used to initialize the GNSS-visual inertial navigation system of land vehicles, but the coupling of this integration method is weak. Reference: Xiong L, Kang R, Zhao J, et al. G-VIDO: A vehicle dynamics and intermittent GNSS-aided visual-inertial state estimator for autonomous driving [J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11845-11861. G-VIDO [3] is a similar system, but it improves the accuracy of the system by integrating vehicle dynamic information. GNSS can help initialize INS first, and then further initialize VINS. Therefore, GNSS and VINS can work in a unified world coordinate system without additional conversion.

[0007] Although the current positioning fusion method has good performance in positioning accuracy, it has high requirements on computer performance. At the same time, the absolute positioning in a wide environment provided by the Global Navigation Satellite System (GNSS) is not suitable for the positioning of the satellites. Therefore, it is necessary to develop a fusion high-precision positioning method that can take into account computing performance, does not need to fuse the two positioning methods all the time, and only needs to realize the plug-and-play fusion of the two methods in environments such as occlusion, so as to improve the system integration, optimize accuracy, and enhance practicality to solve the existing problems. Summary of the invention

[0008] The purpose of the present invention is to provide a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition to solve the problems of low positioning accuracy in complex environments of current smart transportation and high energy consumption in multi-source fusion positioning.

[0009] To achieve the above object, the present invention provides the following technical solution: a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, comprising:

[0010] (1) Establish a traffic sign detection dataset and train a traffic sign recognition model based on the YOLOv5 algorithm;

[0011] (2) Real-time detection of road traffic signs based on traffic sign recognition model;

[0012] (3) Based on the road traffic sign detection results, plug-and-play lightweight multi-source fusion positioning is achieved based on the factor graph.

[0013] Preferably, the data types of the YOLOv5 traffic sign recognition data set include two types: road height limit signs and highway toll station ETC display signs.

[0014] Preferably, the output end of the traffic sign recognition model based on the YOLOv5 algorithm is a trained weight file;

[0015] Preferably, a real-time GNSS high-precision model based on RTK is constructed, and the road height limit and the ETC display signs of the expressway toll station are detected in real time simultaneously; under the condition that the GNSS signal and the network RTK enhanced information can be received in an open environment, RTK is used for real-time high-precision positioning;

[0016] Preferably, after detecting the relevant landmarks, a GNSS / INS / visual SLAM plug-and-play positioning model is constructed based on the factor graph;

[0017] Preferably, the GNSS / INS / visual SLAM plug-and-play positioning model is constructed, assuming that the observation values ​​of each sensor are independent of each other, and according to the Bayesian formula, when the state and observation value are given:

[0018]

[0019] Where, X MAP It refers to the maximum a posteriori estimate of state X, where Z represents the sensor measurement value; p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density of state X when the observed value is known; p(Z|X) is the prior conditional probability density function, which is only related to the state after the measurement value is given; p(Z) is the probability density function of the measurement value and has nothing to do with the state estimate; therefore, the following formula is obtained:

[0020]

[0021] The estimated value of the current state is calculated through the observed value and the state; then t k Status at the moment:

[0022]

[0023] k represents the current epoch, i represents the historical epoch,

[0024] For positioning, the starting point is given directly, where p(x 0 ) represents the probability of the starting point; then at time tk, the maximum estimate of the state is:

[0025] Formula (5) corresponds to the marginal function formula of the factor graph in form. By decomposing the continuous multiplication in the maximum a posteriori estimation probability density function of the factor graph optimization and writing it into factor nodes, we can get:

[0026]

[0027] In the formula, Xi represents the variable nodes included in the current moment, f i (X i ) is the corresponding local function, i.e., factor node;

[0028] The relationship between the factor node and the error function can be expressed as follows: i (X i )=d(err i (X i ,z i ))(7);

[0029] err i (X i ,z i ) is the error function, d() is the cost function;

[0030] The measurement equations of different sensors are ultimately used to calculate the residual between the predicted value and the measured value, so the error function can be expressed in the form of a residual matrix: err i (X i ,z i )=h(X i )-z i (8);

[0031] The cost function d() represents f i (X i ) is proportional to the error function. When the error function is smaller, f i (X i) is larger; the maximum a posteriori estimation of factor graph optimization is a nonlinear least squares problem: Where h represents the state equation.

[0032] The technical effects and advantages of the present invention are as follows: In this lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, traffic sign recognition is used to identify the environment. When an occluded environment is identified, GNSS / INS / visual SLAM is fused through factor graph technology, and positioning is performed through GNSS / INS / vision. The effectiveness of the sensor fusion process is dynamically adjusted to achieve plug-and-play between sensors, which significantly reduces the excessive energy consumption introduced by sensor fusion; reduces the difficulty of data fusion, and enhances the positioning accuracy in complex environments; compared with traditional filtering methods, the GNSS / INS / SLAM fusion positioning method constructed with factor graphs effectively corrects the drift error caused by long-term operation of VIO by introducing GNSS observation data on a global scale, thereby ensuring the provision of long-term stable position information. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The present invention provides Figure 1 A lightweight multi-source fusion high-precision positioning method based on traffic sign recognition is shown in, including:

[0036] Step 1: Establish a traffic sign detection dataset and train a traffic sign recognition model;

[0037] Step 2: Detect road traffic signs in real time based on the traffic sign recognition model;

[0038] Step 3: Based on the road traffic sign detection results, plug-and-play lightweight multi-source fusion positioning is implemented based on the factor graph. In this embodiment, the traffic sign recognition model is based on the YOLOv5 algorithm; the data types of the traffic sign recognition data set include two types: road height limit signs and highway toll station ETC display signs; the traffic sign recognition model based on the YOLOv5 algorithm, whose output end is the trained weight file; build a real-time GNSS high-precision model based on RTK, and simultaneously detect road height limit and highway toll station ETC display signs in real time; after detecting the relevant signs, build a GNSS / INS / visual SLAM plug-and-play positioning model based on the factor graph;

[0039] Assuming that the observations of each sensor are independent of each other, according to the Bayesian formula:

[0040]

[0041] Where, X MAP Refers to the maximum a posteriori estimate of state X, where Z represents the sensor measurement value. p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density of state X when the observed value is known; p(Z|X) is the prior conditional probability density function, which is only related to the state after the measurement value is given; p(Z) is the probability density function of the measurement value, which has nothing to do with the state estimate. Therefore, the following formula is obtained:

[0042]

[0043] For real-time positioning, the estimated value of the current state can only be derived from the previous observations and states. k The state at the moment is:

[0044] Where k represents the current epoch, i represents the historical epoch,

[0045] For positioning, the starting point is usually given directly, where p(x 0 ) represents the probability of the starting point.

[0046] Then at t k At this moment, the maximum estimate of the state is:

[0047]

[0048] The above formula corresponds to the marginal function formula of the factor graph in form. By decomposing the multiplication in the probability density function of the maximum a posteriori estimation of the factor graph optimization and writing it into factor nodes, we can get:

[0049]

[0050] In the formula, Xi represents the variable nodes included in the current moment, f i (X i ) is the corresponding local function, that is, the factor node.

[0051] For each factor node, it is related to the error function corresponding to the sensor observation. The optimization of the state is actually to minimize the error function. The relationship between the error function and the factor node can be expressed as follows: i (X i )=d(err i (X i ,z i ))(7);

[0052] err i (X i ,z i ) is the error function and d() is the cost function.

[0053] The observation equations of different sensors are ultimately used to calculate the residual between the state and the observed state, so the error function can be expressed in the form of a residual matrix: err i (X i ,z i )=h(X i )-z i (8);

[0054] The cost function d() represents f i (X i ) is proportional to the error function. When the error function is smaller, f i (X i ) is larger. The maximum a posteriori estimation problem of factor graph optimization can be transformed into a nonlinear least squares problem: Where h represents the state equation;

[0055] Based on the above factor graph, the integration of GNSS, INS, and V-SLAM can effectively improve the positioning accuracy in complex environments, reduce the number of jump points and drift time, and significantly reduce the requirements for computing performance in areas with severe obstruction such as under viaducts and tunnels, thereby achieving lightweight and high-precision positioning.

[0056] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, characterized by: include: Establish a traffic sign detection dataset and train a traffic sign recognition model; Based on the traffic sign recognition model, the road traffic signs are detected in real time to obtain the road traffic sign detection results; Based on the road traffic sign detection results, plug-and-play lightweight multi-source fusion positioning is achieved based on factor graph.

2. According to claim 1, a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition is characterized in that: The data types of the traffic sign recognition dataset include: road height limit signs and ETC display boards at expressway toll stations.

3. The lightweight multi-source fusion high-precision positioning method based on traffic sign recognition according to claim 1 is characterized in that: The output end of the traffic sign recognition model is a trained weight file.

4. The lightweight multi-source fusion high-precision positioning method based on traffic sign recognition according to claim 1 is characterized in that: The road height limit signs and highway toll station ETC display boards are synchronously detected in real time through the traffic sign recognition model.

5. The lightweight multi-source fusion high-precision positioning method based on traffic sign recognition according to claim 4 is characterized in that: The detection marks obtained after the detection are used as the basis for constructing the GNSS / INS / visual SLAM plug-and-play positioning model.

6. The lightweight multi-source fusion high-precision positioning method based on traffic sign recognition according to claim 5 is characterized by: The construction of the GNSS / INS / visual SLAM plug-and-play positioning model includes: assuming that the observation values ​​of each sensor are independent of each other, according to the Bayesian formula: Where, X MAP It refers to the maximum a posteriori estimate of state X, where Z represents the sensor measurement value; p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density of state X given the observed value; p(Z|X) is the prior conditional probability density function; p(Z) is the probability density function of the measurement value; therefore: The estimated value of the current state is calculated through the observed value and the state; then t k Status at the moment: k represents the current epoch, i represents the historical epoch, and for positioning, the starting point is given directly, where p(x0) represents the probability of the starting point; then at time tk, the maximum estimate of the state is: Formula (5) corresponds to the marginal function formula of the factor graph in form. By decomposing the continuous multiplication in the maximum a posteriori estimation probability density function of the factor graph optimization and writing it into factor nodes, we can get: In the formula, Xi represents the variable nodes included in the current moment, f i (X i ) is the corresponding local function, i.e., factor node; The relationship between the factor node and the error function can be expressed as follows: i (X i )=d(err i (X i ,z i ))(7); err i (X i ,z i ) is the error function, d() is the cost function; The error function is expressed as a residual matrix: err i (X i ,z i )=h(X i )-z i (8); The cost function d() represents f i (X i ) is proportional to the error function. When the error function is smaller, f i (X i ) is larger; the maximum a posteriori estimation of factor graph optimization is a nonlinear least squares problem: Where h represents the state equation.

Images