Low-cost vehicle-mounted navigation positioning method based on assistance of vision and double speedometers in GNSS denial environment

Abstract

The invention relates to a low-cost vehicle-mounted navigation positioning method based on vision and double-speedometer assistance in a GNSS denial environment. The method comprises the following steps: acquiring surrounding environment image information of a vehicle in the GNSS denial environment; acquiring inertial measurement information of the vehicle in the GNSS denial environment; processing the image information of the surrounding environment by using an optical flow method, and judging the state information of the vehicle; calculating the initial navigation state of the vehicle by using the inertial measurement information; and correcting the initial navigation state by using the state information of the vehicle and the constraint model corresponding to the state information to obtain a final vehicle navigation state result. According to the method, the three-dimensional position, speed and attitude of the vehicle are accurately estimated, the method is low in cost and high in real-time performance, the obtained result is stable and reliable, and a high-precision vehicle inertial navigation positioning result can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of vehicle inertial navigation technology, and in particular to a low-cost vehicle-mounted navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment. Background Art

[0002] In recent years, with the rapid development and popularization of high-tech technologies such as artificial intelligence, "5G" communications, the Internet of Things, and smart cars, intelligent transportation systems such as unmanned driving, smart logistics, and shared travel have put forward high requirements on the accuracy and reliability of vehicle navigation. All of these are inseparable from the support of the navigation and positioning system. A single navigation and positioning system is easily affected by various factors, while an integrated system can complement each other's advantages and reduce errors. Vehicle navigation usually adopts an integrated system based on the global satellite navigation system (GNSS). At present, the most common integrated system is the integrated GNSS / INS combined navigation system composed of GNSS and inertial navigation system (INS).

[0003] The global navigation satellite systems (GNSS) in the integrated system include China's Beidou Navigation Satellite System (BDS), the United States' Global Positioning System (GPS), Russia's GLONASS and the European Union's Galileo Navigation Satellite System (GALILEO). GNSS has the advantages of wide coverage, all-weather, and high precision, which can meet the vehicle's demand for navigation performance. In the presence of GNSS signals, high-precision navigation and positioning information can be continuously provided through GNSS / INS integration. The key component of the inertial navigation system (INS) is the inertial measurement unit (IMU), whose performance determines the navigation and positioning accuracy of the pure inertial navigation system. It has the advantages of strong autonomy, short-term high precision, good dynamic performance and no dependence on external information. However, due to the high cost, high-precision IMU is generally only used in special fields and is difficult to promote in the civilian market.

[0004] When GNSS signals are blocked by tall buildings, viaducts, tunnels and other GNSS-denied environments, GNSS cannot provide accurate, continuous and reliable positioning information, and its dynamic performance is poor, so its accuracy is not high when used alone. The navigation accuracy of IMU will also continue to decline over time. When GNSS is interrupted, navigation positioning errors will accumulate rapidly over time. Improving vehicle positioning accuracy in GNSS-denied environments is urgent. Summary of the invention

[0005] In order to solve the problem of inaccurate vehicle positioning in a GNSS-denied environment, the present invention proposes a low-cost vehicle-mounted navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment. The method requires loading an astronomical fisheye camera, an odometer and an inertial measurement unit onto an experimental vehicle, and using the astronomical camera to determine the stationary and moving states of the experimental vehicle, and using the dual odometer to further determine the straight-line driving and turning states of the experimental vehicle, initializing the navigation state of the inertial measurement unit, and collecting the acceleration and angular velocity information observed by the inertial sensor; and using the constraint model corresponding to the motion state based on the experimental vehicle motion state obtained after the astronomical fisheye camera and the dual odometer, thereby correcting the inertial navigation calculation error, obtaining an accurate experimental vehicle navigation state estimation result, and realizing accurate estimation of the three-dimensional position, speed and posture of the experimental vehicle. The present invention realizes easy-to-install, stable and reliable vehicle inertial navigation and positioning.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in GNSS-denied environments, including:

[0008] (1) Obtaining image information of the vehicle’s surrounding environment in a GNSS-denied environment;

[0009] (2) Obtain inertial measurement information of the vehicle in a GNSS-denied environment;

[0010] (3) Processing the surrounding environment image information using an optical flow method to determine the state information of the vehicle;

[0011] (4) using the inertial measurement information to estimate the initial navigation state of the vehicle;

[0012] (5) Using the vehicle state information and a constraint model corresponding to the state information, the initial navigation state is corrected to obtain a final vehicle navigation state result.

[0013] Optionally, the method for obtaining surrounding environment image information of a vehicle in a GNSS-denied environment includes:

[0014] Install a preset image acquisition device on the top of the vehicle, wherein the preset image acquisition device should be in the same direction as the Z axis of the carrier coordinate system (perpendicular to the X axis and Y axis of the carrier coordinate system pointing to the sky);

[0015] The preset image acquisition device is used to capture the image information of the surrounding environment of the vehicle.

[0016] Optionally, the method for obtaining inertial measurement information of a vehicle in a GNSS-denied environment includes:

[0017] Install a preset inertial measurement device on the vehicle, wherein the Y-axis direction of the preset inertial measurement device must be consistent with the forward direction of the vehicle, and the X-axis direction of the preset inertial measurement device points to the right of the vehicle;

[0018] The preset inertial measurement device is used to collect acceleration information and angular velocity information of the vehicle.

[0019] Optionally, determining the status information of the vehicle includes:

[0020] Processing the surrounding environment image information by using an optical flow method to obtain a moving state or a stationary state of the vehicle;

[0021] When the vehicle is in the moving state, the mileage information of the vehicle is used to determine the straight-line driving state or the turning driving state of the vehicle; wherein the mileage information is collected based on a preset dual mileage measurement device.

[0022] Optionally, the process of processing the surrounding environment image information by using the optical flow method includes:

[0023] Assume that the current time is t and the next frame time is t+δt; then the position of the current pixel point A(x,y,z,t) in the next frame is A(x+δx,y+δy,z+δz,t+δt);

[0024] Where x, y, z are the positions of pixel point A in the x, y, z directions, δ is the change, and t is the current time;

[0025] Assuming the brightness is constant, the formula for the change of pixel A is expressed as:

[0026] A(x,y,z,t)=A(x+δx,y+δy,z+δz,t+δt)

[0027] The motion path of pixel A during motion (V x ,V y )for:

[0028]

[0029] Among them, I xi with I yi is the i-th pixel at (x, y), I ti is the corresponding moment;

[0030] The mileage data collected by the preset dual mileage measurement device are respectively recorded as V A , V B , then the speed difference is:

[0031] ΔV=V A -V B

[0032] When ΔV≤ε, the vehicle is considered to be moving in a straight line; when ΔV>ε, the vehicle is considered to be turning right; when ΔV<-ε, the vehicle is considered to be turning left; ε is the limiting threshold.

[0033] Optionally, using the state information of the vehicle and a constraint model corresponding to the state information, correcting the initial navigation state includes:

[0034] Using the state information of the vehicle and the constraint model corresponding to the state information, the data is corrected using a zero-speed correction when the vehicle is stationary, and the data is corrected using a non-holonomic constraint when the vehicle is moving;

[0035] When the experimental vehicle is stationary, the coordinates of the vehicle are considered to remain unchanged, and the coordinates of the vehicle at time t are assigned to the coordinates at time t+1;

[0036]

[0037] Among them, x t ,y t 、z t represents the coordinates of the experimental vehicle at time t, and the coordinates at time t+1 are used as the observed value to correct other errors;

[0038] When the vehicle is moving, the lateral and elevation speeds are assumed to be approximately 0, so the vehicle speed in the carrier coordinate system is:

[0039]

[0040] in, represents the corrected three-dimensional vehicle velocity, v od Indicates the speed in the forward direction. The other errors are corrected as observed quantities.

[0041] The beneficial effects of the present invention are:

[0042] The present invention is based on the GNSS denial environment. Aiming at the problem that the vehicle-mounted GNSS / INS combined navigation in the city is easily affected by the GNSS signal shielding and has low positioning accuracy, a low-cost vehicle-mounted navigation positioning method based on vision and dual odometer assistance in the GNSS denial environment is proposed. This method uses the sky-facing fisheye camera on the top of the vehicle to obtain the vehicle's surrounding environment information, judge the vehicle's static and moving state, and use the dual odometer to further judge the straight driving and turning state of the experimental vehicle. And with the help of the corresponding constraint model, the inertial navigation calculation error is corrected to obtain an accurate experimental vehicle navigation state estimation result, realizing the accurate estimation of the three-dimensional position, speed, and posture of the experimental vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 It is a flowchart of a low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to an embodiment of the present invention;

[0045] Figure 2 This is a fisheye camera processing image of the sky when the experimental vehicle is stationary in an embodiment of the present invention, without pixel point path display;

[0046] Figure 3 This is a fisheye camera processing diagram of the sky when the experimental vehicle moves in an embodiment of the present invention, with the pixel point movement path displayed;

[0047] Figure 4 4 is a comparison diagram of the root mean square errors of different methods according to the embodiments of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] This embodiment proposes a low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment, including:

[0051] (1) Obtaining image information of the vehicle’s surrounding environment in a GNSS-denied environment;

[0052] (2) Obtain inertial measurement information of the vehicle in a GNSS-denied environment;

[0053] (3) Use the optical flow method to process the surrounding environment image information and determine the vehicle status information;

[0054] (4) Using inertial measurement information, calculate the initial navigation state of the vehicle;

[0055] (5) Using the vehicle state information and the constraint model corresponding to the state information, the initial navigation state is corrected to obtain the final vehicle navigation state result.

[0056] Specifically, the flowchart of this embodiment is as follows: Figure 1 As shown, the method needs to first load the celestial fisheye camera, odometer and inertial measurement unit onto the experimental vehicle, and use the celestial fisheye camera to determine the stationary and moving states of the experimental vehicle, and use the dual odometers to further determine the straight-line driving and turning states of the experimental vehicle. Initialize the navigation state of the inertial measurement unit, collect the acceleration and angular velocity information observed by the inertial sensor; and use the constraint model corresponding to the motion state based on the experimental vehicle motion state obtained after the celestial fisheye camera and the dual odometers, so as to correct the inertial navigation calculation error, obtain an accurate experimental vehicle navigation state estimation result, and realize accurate estimation of the experimental vehicle position, speed, and posture. This embodiment realizes easy-to-install, stable and reliable vehicle inertial navigation positioning.

[0057] This embodiment adds a visual sensor and an odometer while building a motion constraint model based on the vehicle's motion law. This method is called a low-cost vehicle-mounted navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment. In vehicle-mounted inertial navigation, zero velocity update (ZUPT) and non-holonomic constraint (NHC) are also commonly used. ZUPT can effectively improve the accuracy of long-term navigation. When the carrier is stationary, its velocity is theoretically zero. This information is used as an observation to update the Kalman filter to correct the speed and attitude errors; however, ZUPT will misjudge the stationary state when the vehicle is stationary due to vehicle vibration, etc.; non-holonomic constraint (NHC) can improve navigation accuracy by constraining lateral and vertical speeds, but it is not completely established when the vehicle skids and bumps on the road; vision can solve the above problems well, but vision cannot judge the vehicle's turn well, so dual odometers are introduced to judge the vehicle's turn. Thereby achieving complementary advantages and improving navigation accuracy. This embodiment aims to combine the visual sensor and dual odometer with zero-speed correction and non-holonomic constraint algorithms to improve the performance of the navigation system.

[0058] Furthermore, the method for obtaining the surrounding image information of the vehicle in the GNSS denied environment includes:

[0059] Install the preset image acquisition device on the top of the vehicle, wherein the preset image acquisition device should be consistent with the Z-axis direction of the carrier coordinate system;

[0060] The preset image acquisition device is used to capture the image information of the vehicle's surrounding environment.

[0061] Specifically, in this embodiment, a celestial fisheye camera is mounted on the experimental vehicle to capture the surrounding environment information of the experimental vehicle. The celestial fisheye camera is mounted on the top of the experimental vehicle, wherein the celestial fisheye camera should be consistent with the Z-axis direction of the carrier coordinate system.

[0062] Furthermore, the method for obtaining inertial measurement information of a vehicle in a GNSS-denied environment includes:

[0063] Install the preset inertial measurement device on the vehicle, wherein the Y-axis direction of the preset inertial measurement device must be consistent with the forward direction of the vehicle, and the X-axis direction of the preset inertial measurement device points to the right of the vehicle;

[0064] The preset inertial measurement device is used to collect the acceleration information and angular velocity information of the vehicle.

[0065] Specifically, in this embodiment, the inertial measurement unit is installed on the experimental vehicle, the navigation state of the inertial measurement unit is initialized, and the acceleration and angular velocity information of the experimental vehicle are collected. When the inertial measurement unit is installed on the experimental vehicle, the Y-axis direction of the inertial measurement unit must be consistent with the forward direction of the experimental vehicle, and the X-axis points to the right of the experimental vehicle.

[0066] In this embodiment, two odometers are also mounted on the non-steering wheels of the vehicle.

[0067] The fisheye camera is used to capture the surrounding environment information of the experimental vehicle, and the optical flow method is used to process the image information to determine the moving and stationary states of the experimental vehicle. The dual odometer is used to further determine the straight-line driving and turning states of the experimental vehicle.

[0068] The inertial navigation algorithm is used to estimate the position, speed and attitude of the inertial sensor in real time, and the constraint model corresponding to the motion state is used according to the experimental vehicle motion state obtained in the previous step to correct the inertial navigation calculation error, obtain the accurate experimental vehicle navigation state estimation result and output it.

[0069] Furthermore, determining the vehicle status information includes:

[0070] Use the optical flow method to process the surrounding image information to obtain the vehicle's motion state or stationary state;

[0071] When the vehicle is in motion, the mileage information of the vehicle is used to determine the straight-line driving state or the turning driving state of the vehicle; wherein the mileage information is collected based on a preset dual mileage measurement device.

[0072] Furthermore, the process of processing the surrounding environment image information using the optical flow method includes:

[0073] Assume that the current time is t and the next frame time is t+δt; then the position of the current pixel point A(x,y,z,t) in the next frame is A(x+δx,y+δy,z+δz,t+δt);

[0074] Where x, y, z are the positions of pixel point A in the x, y, z directions, δ is the change, and t is the current time;

[0075] Assuming the brightness is constant, the formula for the change of pixel A is expressed as:

[0076] A(x,y,z,t)=A(x+δx,y+δy,z+δz,t+δt)

[0077] In this embodiment, the information image captured by the azimuth fisheye camera is a two-dimensional image. Only x, y, and t are considered. The moving direction of the pixel point A during movement (V x ,V y )for:

[0078]

[0079] Among them, I xi with I yi is the i-th pixel at (x, y), I ti is the corresponding moment;

[0080] The mileage data collected by the preset dual mileage measurement device are respectively recorded as V A , V B , then the speed difference is:

[0081] ΔV=V A -V B

[0082] When ΔV≤ε, the vehicle is considered to be moving in a straight line; when ΔV>ε, the vehicle is considered to be turning right; when ΔV<-ε, the vehicle is considered to be turning left; ε is the limiting threshold.

[0083] The process of processing the surrounding image information is as follows: Figure 2-Figure 3 shown.

[0084] Further, using the state information of the vehicle and the constraint model corresponding to the state information, correcting the initial navigation state includes:

[0085] When the experimental vehicle is stationary, the coordinates of the vehicle are considered to remain unchanged, and the coordinates of the vehicle at time t are assigned to the coordinates at time t+1;

[0086]

[0087] The coordinates at time t+1 are used as observations to correct other errors.

[0088] When the vehicle is moving, the lateral and elevation speeds are assumed to be approximately 0, so the vehicle speed in the carrier coordinate system is:

[0089]

[0090] Will The other errors are corrected as observed quantities.

[0091] In this embodiment, first, an inertial measurement unit is used to collect acceleration and angular velocity information of the experimental vehicle.

[0092] Secondly, a fisheye camera is used to capture the surrounding environment of the experimental vehicle in real time, and the optical flow method is used to process the image information to determine the moving and stationary states of the experimental vehicle. When the vehicle is in motion, the dual odometer is used to further determine the straight-line driving and turning states of the vehicle.

[0093] Finally, based on the accurate experimental vehicle motion state obtained by the azimuth fisheye camera and the dual odometers, supplemented by the corresponding constraint model, the inertial navigation error is corrected to obtain an accurate experimental vehicle navigation state estimation result, realizing an accurate estimation of the experimental vehicle's three-dimensional position, speed, and attitude. This method is low-cost, has strong real-time performance, and the results obtained are stable and reliable, and can obtain high-precision vehicle inertial navigation positioning results.

[0094] Through the method proposed in this embodiment, the three-dimensional position, speed and attitude of the experimental vehicle are accurately estimated in a GNSS-denied environment. Figure 4 As shown in the figure, three methods are used to compare the positioning results. The blue, red and orange are the root mean square errors of the three coordinate axes XYZ respectively. The left side uses only ZUPT to process the data; the middle one uses ZUPT+NHC to process the data; the rightmost one is the method designed in this embodiment; Figure 4 It can be concluded that the accuracy of the method used in this embodiment is much higher than that of the other two groups.

[0095] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in GNSS-denied environment, characterized by: include: (1) Obtaining image information of the vehicle’s surrounding environment in a GNSS-denied environment; (2) Obtain inertial measurement information of the vehicle in a GNSS-denied environment; (3) Processing the surrounding environment image information using an optical flow method to determine the state information of the vehicle; (4) using the inertial measurement information to estimate the initial navigation state of the vehicle; (5) Using the vehicle state information and a constraint model corresponding to the state information, the initial navigation state is corrected to obtain a final vehicle navigation state result.

2. The low-cost vehicle-mounted navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to claim 1 is characterized in that: The method of obtaining the surrounding image information of the vehicle in the GNSS denied environment includes: Install a preset image acquisition device on the top of the vehicle, wherein the preset image acquisition device should be consistent with the Z-axis direction of the carrier coordinate system; The preset image acquisition device is used to capture the image information of the surrounding environment of the vehicle.

3. The low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to claim 1 is characterized in that: Methods for obtaining inertial measurement information of a vehicle in a GNSS-denied environment include: Install a preset inertial measurement device on the vehicle, wherein the Y-axis direction of the preset inertial measurement device must be consistent with the forward direction of the vehicle, and the X-axis direction of the preset inertial measurement device points to the right of the vehicle; The preset inertial measurement device is used to collect acceleration information and angular velocity information of the vehicle.

4. The low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to claim 1 is characterized in that: Determining the state information of the vehicle includes: Processing the surrounding environment image information by using an optical flow method to obtain a moving state or a stationary state of the vehicle; When the vehicle is in the moving state, the mileage information of the vehicle is used to determine the straight-line driving state or the turning driving state of the vehicle; wherein the mileage information is collected based on a preset dual mileage measurement device.

5. The low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to claim 4 is characterized in that: The process of processing the surrounding environment image information using the optical flow method includes: Assume that the current time is t and the next frame time is t+δt; then the position of the current pixel point A(x,y,z,t) in the next frame is A(x+δx,y+δy,z+δz,t+δt); Where x, y, z are the positions of pixel point A in the x, y, z directions, δ is the change, and t is the current time; Assuming the brightness is constant, the change formula of pixel A is expressed as: A(x,y,z,t)=A(x+δx,y+δy,z+δz,t+δt) The motion path of pixel A during motion (V x ,V y )for: Among them, I xi with I yi is the i-th pixel at (x, y), I ti is the corresponding moment; The mileage data collected by the preset dual mileage measuring device are respectively recorded as V A , V B , then the speed difference is: ΔV=V A -V B When ΔV≤ε, the vehicle is considered to be moving in a straight line; when ΔV>ε, the vehicle is considered to be turning right; when ΔV<-ε, the vehicle is considered to be turning left; ε is the limiting threshold.

6. The low-cost vehicle navigation and positioning method based on vision and dual odometer assistance in a GNSS-denied environment according to claim 1 is characterized in that: Using the state information of the vehicle and a constraint model corresponding to the state information, correcting the initial navigation state includes: Using the state information of the vehicle and the constraint model corresponding to the state information, the data is corrected using a zero-speed correction when the vehicle is stationary, and the data is corrected using a non-holonomic constraint when the vehicle is moving; When the experimental vehicle is stationary, the coordinates of the vehicle are considered to remain unchanged, and the coordinates of the vehicle at time t are assigned to the coordinates at time t+1; Among them, x t ,y t 、z t represents the coordinates of the experimental vehicle at time t, and the coordinates at time t+1 are used as the observed value to correct other errors; When the vehicle is moving, the lateral and elevation speeds are considered to be 0, so the vehicle speed in the carrier coordinate system is: in, represents the corrected three-dimensional vehicle velocity, v od Indicates the speed in the forward direction. The other errors are corrected as observed quantities.

Images