A lightweight multi-source fusion high-precision positioning method based on traffic sign recognition

Abstract

The present invention discloses a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, which belongs to the field of satellite positioning technology. The method comprises the following steps: establishing a traffic sign detection data set and training a traffic sign recognition model; based on the traffic sign recognition model, detecting road traffic signs in real time to obtain road traffic sign detection results; based on the road traffic sign detection results, implementing plug-and-play lightweight multi-source fusion positioning based on a factor graph; the lightweight multi-source fusion high-precision positioning method based on traffic sign recognition significantly reduces excessive energy consumption introduced by sensor fusion by dynamically adjusting the effectiveness of the sensor fusion process and implementing plug-and-play between sensors; the method reduces the difficulty of data fusion and enhances positioning accuracy in complex environments.

Description

Technical Field

[0001] The present invention belongs to the field of satellite positioning technology, and in particular relates to a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition. Background Art

[0002] In challenging environments such as urban canyons and tunnels, traditional global satellite navigation systems (GNSS) are prone to signal loss, making it difficult to provide stable positioning services. With the development of inertial navigation systems (INS), integrated navigation technology combining INS and GNSS has emerged. This integrated system utilizes the INS to provide short, high-frequency positioning services when the GNSS signal is weak or lost. Thanks to the INS's ability to resist interference from changing external environments, it can achieve continuous, independent positioning. However, when operating independently, the performance of the INS will drift over time, especially when using low-cost microelectromechanical systems (MEMS)-inertial measurement units (IMUs), where this drift can become increasingly severe.

[0003] In the fields of computer vision and robotics, simultaneous localization and mapping (SLAM) enables precise navigation by estimating position and building a map of the environment during movement. This technology relies on incremental mapping and self-localization for autonomous navigation. In recent years, single-sensor SLAM systems such as LG-SLAM, IMLS-SLAM, SUMA, and LOAM have made significant progress. Despite the use of closed-loop correction techniques, the errors of these SLAM systems accumulate as the distance traveled increases. In large-scale outdoor activities, due to the difficulty in forming a closed loop, a single SLAM system cannot provide absolute navigation information.

[0004] Therefore, integrating the GNSS / INS integrated navigation system with SLAM technology not only reduces navigation drift and reliance on closed-loop corrections in environments with weak GNSS signals, but also provides precise absolute navigation information, achieving the complementary advantages of these technologies. This fusion strategy provides a new solution for positioning and navigation in complex environments, optimizing navigation system performance.

[0005] In recent years, research has combined inertial navigation systems (INS) with visual SLAM to develop high-precision visual-inertial odometry (VINS) and general navigation frameworks such as VINS-Fusion. These systems integrate IMU pre-integration measurements with visual feature observations through tightly coupled nonlinear optimization techniques.

[0006] In the VINS-Fusion system, GNSS is integrated into the global estimator and cooperates with VINS as a local estimator to estimate the IMU bias with GNSS assistance. For details, see the literature: Qin T, Cao S, Pan J, et al. A general optimization-based framework for global pose estimation with multiple sensors [J]. arXiv preprint arXiv: 1901.03642, 2019. The literature Jin R, Liu J, Zhang H, et al. Fast and accurate initialization for monocular vision / INS / GNSS integrated system onland vehicle [J]. IEEE Sensors Journal, 2021, 21(22): 26074-26085 describes the parallel startup of GNSS / INS and VINS, which is used to initialize the GNSS-visual inertial navigation system of land vehicles. However, the coupling of this integration method is weak. Reference: Xiong L, Kang R, Zhao J, et al. G-VIDO: A vehicle dynamics and intermittent GNSS-aided visual-inertial state estimator for autonomous driving [J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11845-11861. G-VIDO[3] is a similar system, but it improves the accuracy of the system by integrating vehicle dynamic information. GNSS can help initialize INS first, and then further initialize VINS. Therefore, GNSS and VINS can work in a unified world coordinate system without additional conversion.

[0007] Although current positioning fusion methods have good performance in terms of positioning accuracy, they require high computer performance. At the same time, the absolute positioning provided by the Global Navigation Satellite System (GNSS) in a wide range of environments is difficult to achieve. Therefore, it is necessary to develop a fusion high-precision positioning method that can take into account computing performance, eliminate the need to constantly fuse two positioning methods, and only achieve plug-and-play fusion of the two methods in environments such as occlusion. This can improve system integration, optimize accuracy, and enhance practicality to solve existing problems. Summary of the Invention

[0008] The purpose of the present invention is to provide a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition to solve the current problems of low positioning accuracy in complex environments of smart transportation and high energy consumption in multi-source fusion positioning.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, comprising:

[0010] (1) Establish a traffic sign detection dataset and train a traffic sign recognition model based on the YOLOv5 algorithm;

[0011] (2) Real-time detection of road traffic signs based on traffic sign recognition model;

[0012] (3) Based on the road traffic sign detection results, plug-and-play lightweight multi-source fusion positioning is achieved based on factor graphs.

[0013] Preferably, the data types of the YOLOv5 traffic sign recognition dataset include two categories: road height limit signs and highway toll station ETC display signs.

[0014] Preferably, the traffic sign recognition model based on the YOLOv5 algorithm has an output end which is a trained weight file;

[0015] Preferably, a real-time GNSS high-precision model based on RTK is constructed, and road height restrictions and ETC display signs at highway toll stations are detected in real time. In an open environment where GNSS signals and network RTK enhancement information can be received, RTK is used for real-time high-precision positioning.

[0016] Preferably, after detecting the relevant landmarks, a GNSS / INS / visual SLAM plug-and-play positioning model is constructed based on the factor graph;

[0017] Preferably, the GNSS / INS / visual SLAM plug-and-play positioning model is constructed, assuming that the observation values ​​of each sensor are independent of each other. According to the Bayesian formula, when the state and observation value are given, there is:

[0018]

[0019] Where, X MAP It refers to the maximum a posteriori estimate of the state X, where Z represents the sensor measurement value; p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density function of the state X given the observed value; p(Z|X) is the prior conditional probability density function, which is only related to the state after the measurement value is given; p(Z) is the probability density function of the measurement value and is independent of the state estimate; therefore, the following formula is obtained:

[0020]

[0021] The estimated value of the current state is calculated by the observed value and the state; then t k Status at the moment:

[0022]

[0023] k represents the current epoch, i represents the historical epoch,

[0024] For positioning, the starting point is given directly, where p(x0) represents the probability of the starting point; then at time tk, the maximum estimate of the state is:

[0025] Formula (5) corresponds to the marginal function formula of the factor graph in form. By decomposing the continuous multiplication in the factor graph optimization maximum a posteriori estimation probability density function and writing it into factor nodes, we can obtain:

[0026]

[0027] In the formula, Xi represents the variable nodes included in the current moment, f i (X i ) is the corresponding local function, i.e., factor node;

[0028] The relationship between factor nodes and error functions can be expressed as follows: i (X i )=d(err i (X i ,z i ))(7);

[0029] err i (X i ,z i ) is the error function, d() is the cost function;

[0030] The measurement equations of different sensors are ultimately used to calculate the residual between the predicted value and the measured value, so the error function can be expressed in the form of a residual matrix: err i (X i ,z i )=h(X i )-z i (8);

[0031] The cost function d() represents f i (X i ) is proportional to the negative logarithm of the error function. When the error function is smaller, f i (X i ) is larger; the maximum a posteriori estimation of factor graph optimization is a nonlinear least squares problem: Where h represents the equation of state.

[0032] The technical effects and advantages of the present invention are as follows: This lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, traffic sign recognition is used to identify the environment, and when an occluded environment is identified, GNSS / INS / visual SLAM is integrated through factor graph technology, and positioning is performed through GNSS / INS / vision. By dynamically adjusting the effectiveness of the sensor fusion process, plug-and-play between sensors is achieved, which significantly reduces the excessive energy consumption introduced by sensor fusion; reduces the difficulty of data fusion and enhances positioning accuracy in complex environments; compared with traditional filtering methods, the GNSS / INS / SLAM fusion positioning method constructed with factor graphs effectively corrects the drift error caused by long-term VIO operation by introducing GNSS observation data on a global scale, ensuring the provision of long-term stable position information. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] The present invention provides Figure 1 A lightweight multi-source fusion high-precision positioning method based on traffic sign recognition is shown in , including:

[0036] Step 1: Establish a traffic sign detection dataset and train a traffic sign recognition model;

[0037] Step 2: Detect road traffic signs in real time based on the traffic sign recognition model;

[0038] Step 3: Based on the road traffic sign detection results, a plug-and-play lightweight multi-source fusion positioning is implemented based on the factor graph. In this embodiment, the traffic sign recognition model is based on the YOLOv5 algorithm; the traffic sign recognition dataset data types include two categories: road height limit signs and highway toll station ETC display signs; the traffic sign recognition model based on the YOLOv5 algorithm has a trained weight file as its output; a real-time GNSS high-precision model based on RTK is constructed, and road height limit and highway toll station ETC display signs are detected simultaneously in real time; after detecting the relevant signs, a GNSS / INS / visual SLAM plug-and-play positioning model is constructed based on the factor graph;

[0039] Assuming that the observations of each sensor are independent of each other, according to the Bayesian formula:

[0040]

[0041] Where, X MAP It refers to the maximum a posteriori estimate of the state X, where Z represents the sensor measurement value. p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density function of the state X given the observed value; p(Z|X) is the prior conditional probability density function, which is only related to the state after the measurement value is given; and p(Z) is the probability density function of the measurement value, which is independent of the state estimate. Therefore, the following formula is obtained:

[0042]

[0043] For real-time positioning, the estimated value of the current state can only be derived from the previous observations and states. k The state at the moment is:

[0044] Where k represents the current epoch, i represents the historical epoch,

[0045] For positioning, the starting point is generally given directly, where p(x0) represents the probability of the starting point.

[0046] Then at t k At this moment, the maximum estimate of the state is:

[0047]

[0048] The above formula corresponds to the marginal function formula of the factor graph in form. By decomposing the continuous multiplication in the maximum a posteriori estimation probability density function of the factor graph optimization and writing it into factor nodes, we can get:

[0049]

[0050] In the formula, Xi represents the variable nodes included in the current moment, f i (X i ) is the corresponding local function, that is, the factor node.

[0051] For each factor node, it is related to the error function corresponding to the sensor observation. The optimization of the state is actually to minimize the error function. The relationship between the error function and the factor node can be expressed as follows: i (X i )=d(err i (X i ,z i))(7);

[0052] err i (X i ,z i ) is the error function, and d() is the cost function.

[0053] The observation equations of different sensors are ultimately used to calculate the residual between the state and the observed state, so the error function can be expressed in the form of a residual matrix: err i (X i ,z i )=h(X i )-z i (8);

[0054] The cost function d() represents f i (X i ) is proportional to the negative logarithm of the error function. When the error function is smaller, f i (X i ) is larger. The maximum a posteriori estimation problem of factor graph optimization can be transformed into a nonlinear least squares problem: Where h represents the equation of state;

[0055] Based on the above factor graph, the integration of GNSS, INS, and V-SLAM effectively improves positioning accuracy in complex environments, such as under overpasses and in tunnels, where there is severe obstruction. This reduces the number of jump points and drift time, significantly lowers the requirements for computing performance, and achieves lightweight, high-precision positioning.

[0056] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A lightweight multi-source fusion high-precision positioning method based on traffic sign recognition, characterized by: include: Establish a traffic sign detection dataset and train a traffic sign recognition model; Based on the traffic sign recognition model, the road traffic signs are detected in real time to obtain the road traffic sign detection results; Based on the road traffic sign detection results, a plug-and-play lightweight multi-source fusion positioning is achieved based on the factor graph; The data types of the traffic sign detection dataset include: road height limit signs and ETC display boards at highway toll stations; the output end of the traffic sign recognition model is a trained weight file; the road height limit signs and ETC display boards at highway toll stations are synchronously detected in real time by the traffic sign recognition model; the detection signs obtained after the detection are used as the basis for constructing a GNSS / INS / visual SLAM plug-and-play positioning model based on the factor graph.

2. The lightweight multi-source fusion high-precision positioning method based on traffic sign recognition according to claim 1 is characterized by: The construction of the GNSS / INS / visual SLAM plug-and-play positioning model includes: assuming that the observation values ​​of each sensor are independent of each other, according to the Bayesian formula: Where, X MAP It refers to the maximum a posteriori estimate of the state X, where Z represents the sensor measurement value; p(X) is the prior probability density function of the state; p(X|Z) is the posterior probability density function of the state X given the observed value; p(Z|X) is the prior conditional probability density function; and p(Z) is the probability density function of the measurement value. Therefore: The estimated value of the current state is calculated by the observed value and the state; then t k Status at the moment: k represents the current epoch, i represents the historical epoch. For positioning, the starting point is directly given, where p(x0) represents the probability of the starting point; then at time tk, the maximum estimate of the state is: Formula (5) corresponds to the marginal function formula of the factor graph in form. By decomposing the continuous multiplication in the factor graph optimization maximum a posteriori estimation probability density function and writing it into factor nodes, we can obtain: In the formula, Xi is expressed as the variable nodes included in the current moment, f i ( X i ) is the corresponding local function, i.e., factor node; The relationship between factor nodes and error functions can be expressed as follows: f i ( X i )= d ( err i ( X i ,z i ))(7); err i ( X i ,z i ) is the error function, d () is the cost function; The error function is expressed in the form of a residual matrix: err i ( X i ,z i )= h i ( X i )- z i (8); The cost function d() represents f i (X i ) is proportional to the negative logarithm of the error function. When the error function is smaller, f i (X i ) is larger; the maximum a posteriori estimation of factor graph optimization is a nonlinear least squares problem: Where h represents the equation of state.

Images