Vehicle positioning methods, devices, vehicles and storage media

Abstract

This invention relates to the field of intelligent driving technology and discloses a vehicle positioning method, device, vehicle, and storage medium. The method includes: acquiring image information collected by an image sensor and vehicle motion information and / or position information collected by a target sensor, wherein the target sensor is a sensor other than the image sensor; acquiring visually detected lane lines based on the image information; acquiring first measurement data of lane line factors based on the visually detected lane lines and a map; acquiring second measurement data of corresponding factors based on the vehicle motion information and / or position information; constructing a factor graph including lane line factors and target factors corresponding to the target sensor based on the first and second measurement data; and performing sliding window optimization on the factor graph to obtain vehicle pose information. This invention utilizes factor graphs to fuse multi-source data to achieve vehicle positioning. The factor graph solution process involves multiple iterations and fully considers historical positioning data, which can improve the accuracy of vehicle fusion positioning.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to a vehicle positioning method, device, vehicle, and storage medium. Background Technology

[0002] With the development of intelligent connected vehicle technology, intelligent transportation systems, represented by autonomous passenger cars, commercial vehicles, and taxis, have undergone tremendous changes in their perception of complex scenarios. High-precision positioning is a key problem that autonomous driving needs to solve. Accurate position, speed, and attitude information are crucial for vehicles to achieve autonomous driving, affecting motion planning, decision-making, and motion tracking performance. To achieve high-precision positioning in various complex scenarios (such as urban canyons, tunnels, and overpasses), it is difficult to rely on a single sensor and often requires fusion positioning based on multi-sensor information. For example, methods based on the Global Navigation Satellite System (GNSS) can achieve centimeter-level accuracy in open scenarios, but are not reliable enough under occlusion and multipath conditions. To solve the GNSS problem, methods that fuse GNSS with Inertial Measurement Units (IMUs) or odometry have been proposed. Of course, there are many other combined positioning methods, such as fusion positioning based on IMUs, GNSS, LiDAR, image sensors, and high-precision maps. However, current fusion positioning methods still have certain limitations, resulting in relatively low positioning accuracy. Summary of the Invention

[0003] In view of this, the present invention provides a vehicle positioning method, device, vehicle and storage medium to solve the problem that the positioning accuracy of current fusion positioning methods is still relatively low.

[0004] In a first aspect, the present invention provides a vehicle positioning method, the method comprising:

[0005] Acquire image information collected by an image sensor, and vehicle motion information and / or position information collected by a target sensor, which is a sensor other than an image sensor;

[0006] The visually detected lane lines are obtained based on image information; the first measurement data of lane line factors are obtained based on the visually detected lane lines and the map; and the second measurement data of the corresponding factors are obtained based on vehicle motion information and / or position information.

[0007] A factor map including lane line factors and target factors corresponding to the target sensor is constructed based on the first and second measurement data;

[0008] The factor map is optimized using a sliding window to obtain vehicle pose information.

[0009] In one optional implementation, after performing sliding window optimization on the factor graph to obtain the vehicle pose information, the method further includes:

[0010] Acquire target information collected by inertial sensors at a target time, which is after the time corresponding to the vehicle pose information;

[0011] Vehicle pose prediction is performed based on vehicle pose information and target information to obtain predicted vehicle pose information at the target time.

[0012] In one alternative implementation, first measurement data of lane line factors is obtained based on visually detected lane lines and a map, including:

[0013] Based on the latest vehicle position information, obtain at least one first lane line from the map;

[0014] From at least one first lane line, obtain the target first lane line that matches each visually detected lane line;

[0015] Obtain the target line segment in the first lane line corresponding to the sampling point of the visually detected lane line to obtain the first measurement data.

[0016] In one alternative implementation, obtaining a target first lane line that matches each visually detected lane line from at least one first lane line includes:

[0017] Based on the lane line edge attribute features, the visually detected lane lines are associated with the corresponding first lane lines;

[0018] For a pair of visually detected lane lines and the first lane line, based on the latest acquired vehicle pose information, the first distance between the visually detected lane line and the first lane line is obtained;

[0019] The first distance is compensated to the latest acquired vehicle pose information to obtain the prior vehicle pose information;

[0020] Using prior vehicle pose information, the visually detected lane lines are projected onto the map, and the second distance between the visually detected lane lines and the corresponding first lane lines is calculated.

[0021] If the second distance is less than a preset threshold, the first lane line corresponding to the visually detected lane line is determined as the target first lane line for matching the visually detected lane line.

[0022] In one alternative implementation, the residual of the lane line factor is the distance between the sampling point of the visually detected lane line and the corresponding target line segment.

[0023] In one optional implementation, the vehicle motion information and / or position information collected by the target sensor includes at least one of the following: acceleration and angular velocity information collected by an inertial sensor, position information and first velocity information collected by a global navigation satellite system sensor, and second velocity information collected by a wheel speed sensor.

[0024] In one alternative implementation, the objective function used for sliding window optimization includes residuals of lane line factors, residuals of inertial pre-integration, residuals of wheel speed integration, residuals of wheel speed factors, and residuals of the position and velocity factors of the Global Navigation Satellite System.

[0025] In a second aspect, the present invention provides a vehicle positioning device, the device comprising:

[0026] The information acquisition module is used to acquire image information acquired by the image sensor, as well as vehicle motion information and / or position information acquired by the target sensor, which is a sensor other than the image sensor.

[0027] The measurement data acquisition module is used to acquire visually detected lane lines based on image information, acquire first measurement data of lane line factors based on visually detected lane lines and maps, and acquire second measurement data of corresponding factors based on vehicle motion information and / or position information.

[0028] The factor graph construction module is used to construct a factor graph, including lane line factors and target factors corresponding to the target sensor, based on the first measurement data and the second measurement data.

[0029] The positioning module is used to perform sliding window optimization on the factor map to obtain vehicle pose information.

[0030] Thirdly, the present invention provides a vehicle comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the vehicle positioning method of the first aspect or any corresponding embodiment thereof.

[0031] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the vehicle positioning method of the first aspect or any corresponding embodiment thereof.

[0032] The vehicle positioning method, device, vehicle, and storage medium provided in this embodiment utilize factor graphs to fuse multi-source data to achieve vehicle positioning. The factor graph solution process iterates multiple times and fully considers historical positioning data, significantly improving the accuracy of vehicle fusion positioning. In actual testing, the vehicle positioning method provided in this embodiment achieves a lateral positioning accuracy within 12 cm and a longitudinal accuracy within 80 cm relative to high-precision maps. In other words, the vehicle positioning method provided in this embodiment can effectively maintain decimeter-level positioning accuracy and exhibits strong robustness, making it applicable to complex scenarios such as tunnels, overpasses, and urban canyons, without any vehicle positioning error divergence problem. It also boasts advantages such as low hardware cost and strong scalability. Attached Figure Description

[0033] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating a vehicle positioning method according to an embodiment of the present invention;

[0035] Figure 2 This is a flowchart illustrating another vehicle positioning method according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of a vehicle positioning method according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the factor graph provided in an embodiment of the present invention;

[0038] Figure 5 This is a structural block diagram of a vehicle positioning device according to an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the hardware structure of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] In fusion positioning methods, Kalman filtering can be used for fusion, but it has the following shortcomings: Kalman filter only uses the observation data of each frame for estimation and correction, and cannot use historical information to correct and adjust the estimated value of the current state quantity. When there is noise and outliers, the positioning accuracy will be severely reduced, thereby affecting the safety of vehicle driving.

[0042] This invention provides a vehicle positioning method that uses factor maps to fuse relevant information from visually detected lane lines, high-precision maps, and other sensors to improve the accuracy of fusion positioning.

[0043] According to an embodiment of the present invention, a vehicle positioning method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of executable computer instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] This embodiment provides a vehicle positioning method, which can be used in vehicles equipped with the aforementioned computer system. Figure 1 This is a flowchart of a vehicle positioning method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0045] Step S101: Obtain image information collected by the image sensor and vehicle motion information and / or position information collected by the target sensor. The target sensor is a sensor other than the image sensor.

[0046] Image sensors and target sensors can be sensors installed on vehicles. Target sensors can be IMU sensors, GNSS sensors, lidar, etc.

[0047] Step S102: Obtain visually detected lane lines based on image information; obtain first measurement data of lane line factors based on visually detected lane lines and map; obtain second measurement data of corresponding factors based on vehicle motion information and / or position information.

[0048] Measurement data (first measurement data, second measurement data), also known as observation data, requires processing from sensor-collected information to measurement data, including coordinate transformation (to the world coordinate system), integration, and pre-integration. The map above is a high-precision map, with an accuracy down to the centimeter level.

[0049] Step S103: Construct a factor map based on the first measurement data and the second measurement data, including lane line factors and target factors corresponding to the target sensor;

[0050] Step S104 involves performing sliding window optimization on the factor graph to obtain vehicle pose information. This embodiment of the invention is a least squares problem, which can be solved using the Gauss-Newton iterative method or the Levenberg-Marquardt method. This localization framework uses the GTSAM (Georgia Tech Smoothing and Mapping, a C++ library based on factor graphs) library to solve for the vehicle pose information (state variables). Furthermore, to ensure real-time vehicle localization, sliding window optimization is used here, with a window size of, for example, 1-2 seconds.

[0051] The vehicle positioning method provided in this embodiment utilizes factor graphs to fuse multi-source data for vehicle positioning. The factor graph solution process iterates multiple times and fully considers historical positioning data, significantly improving the accuracy of vehicle fusion positioning. In actual testing, the vehicle positioning method provided by this embodiment achieves a lateral positioning accuracy within 12 cm and a longitudinal accuracy within 80 cm relative to high-precision maps. In other words, the vehicle positioning method provided by this embodiment can effectively maintain decimeter-level positioning accuracy and exhibits strong robustness, making it applicable to complex scenarios such as tunnels, overpasses, and urban canyons, without the problem of vehicle positioning error divergence. It also boasts advantages such as low hardware cost and strong scalability.

[0052] This embodiment provides a vehicle positioning method, which can be used in the aforementioned mobile terminals, such as mobile phones and tablets. Figure 2 This is a flowchart of a vehicle positioning method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0053] Step S201: Obtain image information collected by the image sensor and vehicle motion information and / or position information collected by the target sensor. The target sensor is a sensor other than the image sensor.

[0054] Please see Figure 3 The information collected by the sensors can be input into the measurement manager to obtain the latest measurement information when locating the vehicle. The measurement manager can also perform interpolation based on the information collected by the sensors to achieve data alignment.

[0055] Specifically, the target sensor may include at least one of the following: an inertial sensor, a global navigation satellite system sensor, and a wheel speed sensor, wherein the inertial sensor acquires acceleration information and angular velocity information, the global navigation satellite system sensor acquires position information and a first velocity information, and the wheel speed sensor acquires a second velocity information.

[0056] Step S202: Obtain visually detected lane lines based on image information; obtain first measurement data of lane line factors based on visually detected lane lines and map; obtain second measurement data of corresponding factors based on vehicle motion information and / or position information.

[0057] Lane lines can be visually detected based on image information. Specifically, image recognition technology can be used to detect lane lines, such as machine learning.

[0058] In some optional implementations, step S202, namely, obtaining the first measurement data of lane line factors based on visually detected lane lines and a map, includes:

[0059] Step S2021: Based on the latest vehicle position information, obtain at least one first lane line from the map;

[0060] The latest vehicle pose information can be the vehicle pose information obtained from the previous factor graph optimization, or it can be the prediction information obtained by predicting the vehicle pose information obtained from the factor graph optimization. The prediction method can be found below, or other methods can be used for prediction. The accuracy of the vehicle pose information here can be lower than that of the vehicle pose information obtained from the factor graph optimization.

[0061] This step mainly involves using the latest vehicle pose information to extract lane lines from the high-precision map that match (are close to) the vehicle's position; at this point, the first lane line obtained may be more than one.

[0062] Step S2022: Obtain the target first lane line that matches each visually detected lane line from at least one first lane line;

[0063] There is usually more than one visually detected lane line based on image information. This step is to match each visually detected lane line with the first lane line obtained from the high-precision map.

[0064] Specifically, step S2022, obtaining the target first lane line matching each visually detected lane line from at least one first lane line, includes:

[0065] Step S20221: Based on the lane line edge attribute features, associate the visually detected lane line with the corresponding first lane line;

[0066] Here, feature matching is used to associate the visually detected lane line with the first lane line.

[0067] In this embodiment of the invention, when obtaining visually detected lane lines based on image information, it is necessary to obtain lane line edge information and lane line shape information.

[0068] Step S20222: For the associated pair of visually detected lane lines and the first lane line, based on the latest acquired vehicle pose information, obtain the first distance between the visually detected lane lines and the first lane line;

[0069] The following steps S20223-S20225 are also performed separately for each pair of visually detected lane lines and the first lane line.

[0070] Step S20223: The first distance is compensated to the latest acquired vehicle pose information to obtain the prior vehicle pose information;

[0071] Step S20224: Using prior vehicle pose information, project the visually detected lane line onto the map and calculate the second distance between the visually detected lane line and the corresponding first lane line.

[0072] Step S20225: If the second distance is less than a preset threshold, determine the first lane line corresponding to the visually detected lane line, which is the target first lane line for matching the visually detected lane line.

[0073] Step S2023: Obtain the target line segment in the first target lane line corresponding to the sampling point of the visually detected lane line to obtain the first measurement data, i.e., the measurement data of the lane line factor. Specifically, for each visually detected lane line, sampling is performed and the target line segment corresponding to the sampling point in the first target lane line is obtained. The variance of the sampling points is set, and finally the measurement data of the lane line factor is obtained.

[0074] In this embodiment of the invention, the measurement data of the lane line factor is obtained by associating visual detection with lane lines on a high-precision map, then obtaining multiple sets of matching between visual detection lane line sampling points and map line segments, and adding them to the factor graph as constraints, thereby improving the lateral accuracy of positioning.

[0075] The vehicle motion information and / or position information includes at least one of the following: acceleration and angular velocity information collected by inertial sensors, position and first velocity information collected by global navigation satellite system sensors, and second velocity information collected by wheel speed sensors.

[0076] The following example illustrates the specific process of obtaining the second measurement data of corresponding factors based on vehicle motion information and / or position information, which simultaneously includes acceleration and angular velocity information collected by inertial sensors, position information and first velocity information collected by global navigation satellite system sensors, and second velocity information collected by wheel speed sensors.

[0077] The position and velocity information collected by the sensors of the Global Navigation Satellite System are in the WGS84 coordinate system. Therefore, they need to be transformed to the world coordinate system before they can be added to the factor graph to constrain the position and velocity. Then, according to the actual sensor measurement accuracy, the corresponding measurement variance is set to finally obtain the measurement data of the position and velocity factors of the Global Navigation Satellite System.

[0078] For the acceleration information (specifically acquired by the accelerometer) and angular velocity information (specifically acquired by the gyroscope) collected by the inertial sensor, the changes in rotation and displacement between two different moments can be obtained through integration. During subsequent factor map optimization, pose optimization is required. Each pose adjustment necessitates re-transferring the acceleration and angular velocity information collected by the inertial sensor between them, requiring re-integration, which is very time-consuming. A pre-integration strategy can be adopted to avoid re-transferring the acceleration and angular velocity information. In this embodiment of the invention, the median method can be used for discretization to obtain the IMU pre-integrated measurement data, which can be accomplished using the GTSAM library.

[0079] The second speed information collected by the wheel speed sensor can be used to generate wheel speed integral measurement data. Wheel speed integral refers to calculating the vehicle's displacement using the instantaneous speed of the wheels. The formula for calculating wheel speed integral can be:

[0080]

[0081] Among them, R i V represents the rotation at time i, which can be obtained from IMU pre-integrated data. ij Let dt be the average velocity from time i to j, dt be the duration from i to j, and n be the total number of measurement data frames in the wheel speed integral.

[0082] Step S203: Construct a factor map based on the first measurement data and the second measurement data, including lane line factors and target factors corresponding to the target sensor.

[0083] Specifically, a nonlinear factor graph can be defined using the GTSAM library, such as... Figure 4 As shown, relevant factors are then added, such as the global navigation satellite system position and velocity factor, the IMU pre-integration factor, the lane line factor, and the wheel speed integration factor, thus completing the construction of the factor map.

[0084] Step S204: Perform sliding window optimization on the factor graph to obtain vehicle pose information. Since the sensors continuously collect new information, the factor graph will continuously grow. To reduce computational complexity and memory usage, this embodiment of the invention employs a sliding window strategy. The sliding window defines a fixed-size window on the factor graph, retaining only the factors and variables within the window for optimization. When new collected information arrives, the window slides forward, and older factors and variables are removed.

[0085] In this embodiment of the invention, if the factor graph simultaneously includes the Global Navigation Satellite System position and velocity factor, the IMU pre-integration factor, the lane line factor, the wheel speed integration factor, and the wheel speed factor, the state variables of the factor graph are:

[0086]

[0087] Where, p i and v i R represents the vehicle's position and velocity in the world coordinate system. i This refers to the vehicle's rotation relative to the world coordinate system. and For zero bias in acceleration and zero bias in angular velocity.

[0088] In this embodiment of the invention, due to the presence of zero bias in the inertial sensor—that is, internal errors caused by various physical factors such as internal mechanics and temperature—the pre-integration result may be affected. Therefore, it is necessary to estimate and compensate for the zero bias to ensure the accuracy of the inertial pre-integration.

[0089] Regarding lane line factors, the lane line factor measurement data (i.e., the first measurement data) after matching the visually detected lane lines with the high-precision map consists of the set of sampling points of the visually detected lane lines and the corresponding map line segments (target line segments in the target first lane line). The distance from the sampling points to the map line segments is calculated as a residual constraint, and the calculation formula is as follows:

[0090]

[0091] Where, r PLM p represents the residual from the sampling points of the lane line factor to the corresponding map line segments. wl =Rp+t, where p is the sampling point of the visually detected lane line in the vehicle coordinate system, R and t are the coordinate transformation parameters, where t is the translation amount, which is independent of time, and p w These are the sampling points for visual detection of lane lines in the world coordinate system. and represents the two endpoints of the map line segment (world coordinate system), and l represents the sampling point and the corresponding map line segment number.

[0092] For the IMU pre-integration factor, IMU pre-integration is used to constrain the relative motion between two adjacent time points, and the constraint variance of the residual is:

[0093]

[0094]

[0095] Where, r IP For the IMU pre-integration residual, ΔX ij The data is the IMU pre-integrated data from time i to time j. In gΔt, g represents the acceleration due to gravity, and Δt is the time interval from time i to time j. The pre-integrated data is calculated from the IMU measurement information (i.e., the acceleration and angular velocity information collected by the inertial sensor). Let R be the rotation matrix. i The transpose of .

[0096] For the wheel speed integral factor, the wheel speed integral is used to constrain the relative motion between two adjacent time points, and the residual constraint equation is:

[0097]

[0098]

[0099] Where, r WP The residual of the wheel speed integral, ΔW ij The integral data of wheel velocity from time i to time j. The integral data is calculated from the wheel speed measurement information (i.e., the second speed information).

[0100] The wheel speed factor is used to ensure that the wheel speed is consistent with the measurement data (second speed information) from the wheel speed sensor. The residual constraint equation is as follows:

[0101]

[0102] Where, r WS The residual of the wheel speed factor, v w Let the vehicle's speed be in the world coordinate system. This is wheel speed measurement information (i.e., second speed information).

[0103] The position and velocity factor for the Global Navigation Satellite System (GNSS) is used to ensure that the position and velocity are consistent with the measurement information from the GNSS sensors (i.e., the position and initial velocity information acquired by the GNSS sensors). The residual constraint equation is:

[0104]

[0105] Where, r gnss The residual of the position and velocity factor of the global navigation satellite system. and This includes position measurement data (i.e., position information collected by the sensors of the Global Navigation Satellite System) and velocity measurement data (i.e., the first velocity information collected by the sensors of the Global Navigation Satellite System).

[0106] In this embodiment of the invention, the objective function for optimizing the constructed factor graph is a nonlinear least squares objective function, which may be, for example:

[0107]

[0108] In this embodiment of the invention, acceleration and angular velocity information collected by inertial sensors, position and first velocity information collected by global navigation satellite system sensors, second velocity information collected by wheel speed sensors, lane lines detected by vision, and high-precision maps are fused together and factor graphs are used to jointly optimize the constructed residual terms and constraint variances. By establishing a nonlinear least squares objective function, the position, attitude, velocity, acceleration, and angular velocity zero-bias information of the vehicle are optimized.

[0109] Additionally, please see Figure 3 Step S204, which involves performing sliding window optimization on the factor map to obtain the vehicle pose information, also includes:

[0110] Step S205: Acquire the target information collected by the inertial sensor at the target time, which is after the time corresponding to the vehicle pose information;

[0111] Specifically, the time corresponding to the vehicle pose information is the time corresponding to the measurement data used to obtain that vehicle pose information. The time corresponding to the measurement data can be the time when the sensor acquired the corresponding information. If the measurement data is obtained through interpolation, then its corresponding time is the interpolation time. For real-time vehicle positioning, the target time can be the time corresponding to the vehicle pose information up to the current time, or the time up to the latest inertial information acquired by the inertial sensor.

[0112] Step S206: Predict the vehicle pose based on the vehicle pose information and the target information to obtain the predicted vehicle pose information for the target time. Specifically, the predicted vehicle pose information can be obtained by performing IMU integration using the target information after optimizing the vehicle pose information.

[0113] The vehicle positioning method provided in this embodiment has a delay in the vehicle pose information obtained based on factor graph optimization relative to the current time (latest time). In order to obtain vehicle pose information in real time, a vehicle pose information prediction thread can be maintained to achieve the following functions: After obtaining the latest vehicle pose information (i.e., state information) through factor graph optimization, the latest vehicle pose information can be used as the base state. Then, all information collected by the IMU sensor between the base state time (i.e., the time corresponding to the latest obtained vehicle position information) and the latest IMU time (i.e., the time when the inertial sensor latest collected inertial information) is obtained, and the latest vehicle pose information can be predicted by performing IMU integration on the base state.

[0114] In addition, vehicle pose information prediction can improve the output frequency of vehicle localization results. For example, the frequency of vehicle pose information obtained by factor graph optimization is 10Hz, and by predicting vehicle pose information, a vehicle localization result output frequency of 100Hz can be achieved.

[0115] This embodiment also provides a vehicle positioning device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0116] This embodiment provides a vehicle positioning device, such as... Figure 5 As shown, it includes:

[0117] The information acquisition module 501 is used to acquire image information acquired by the image sensor, as well as vehicle motion information and / or position information acquired by the target sensor, wherein the target sensor is a sensor other than the image sensor.

[0118] The measurement data acquisition module 502 is used to acquire visually detected lane lines based on image information, acquire first measurement data of lane line factors based on visually detected lane lines and maps, and acquire second measurement data of corresponding factors based on vehicle motion information and / or position information.

[0119] The factor graph construction module 503 is used to construct a factor graph including lane line factors and target factors corresponding to the target sensor based on the first measurement data and the second measurement data;

[0120] The positioning module 504 is used to perform sliding window optimization on the factor map to obtain vehicle pose information.

[0121] In some alternative implementations, the vehicle positioning device further includes:

[0122] The latest inertial information acquisition module is used to acquire target information collected by the inertial sensor at a target time, which is after the time corresponding to the vehicle pose information.

[0123] The prediction module is used to predict the vehicle pose based on the vehicle pose information and the target information, and obtain the predicted vehicle pose information at the target time.

[0124] In some optional implementations, the measurement data acquisition module 502 includes:

[0125] The first lane line acquisition unit is used to acquire at least one first lane line from the map based on the latest acquired vehicle pose information.

[0126] The target first lane line acquisition unit is used to acquire a target first lane line that matches each visually detected lane line from at least one first lane line;

[0127] The first vehicle data acquisition unit is used to acquire the target line segment in the first lane line corresponding to the sampling point of the visually detected lane line, and obtain the first measurement data.

[0128] In some alternative implementations, the target first lane line acquisition unit includes:

[0129] The association subunit is used to associate the visually detected lane line with the corresponding first lane line based on the lane line edge attribute features;

[0130] The first distance acquisition subunit is used to acquire the first distance between the visually detected lane line and the first lane line based on the latest acquired vehicle pose information for a pair of associated visually detected lane lines and the first lane line.

[0131] The prior vehicle position information acquisition subunit is used to compensate the first distance to the latest acquired vehicle pose information to obtain the prior vehicle pose information.

[0132] The second distance acquisition subunit is used to project the visually detected lane line onto the map using prior vehicle pose information and calculate the second distance between the visually detected lane line and the corresponding first lane line.

[0133] A sub-unit is defined to determine the first lane line corresponding to the visually detected lane line when the second distance is less than a preset threshold, and to be the target first lane line matched with the visually detected lane line.

[0134] In some alternative implementations, the residual of the lane line factor is the distance between the sampling point of the visually detected lane line and the corresponding target line segment.

[0135] In some optional implementations, the vehicle motion information and / or position information collected by the target sensor includes at least one of the following: acceleration and angular velocity information collected by an inertial sensor, position information and first velocity information collected by a global navigation satellite system sensor, and second velocity information collected by a wheel speed sensor.

[0136] In some alternative implementations, the objective function used for sliding window optimization includes residuals of lane line factors, residuals of inertial pre-integration, residuals of wheel speed integration, residuals of wheel speed factors, and residuals of the position and velocity factors of the Global Navigation Satellite System.

[0137] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0138] In this embodiment, the vehicle positioning device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0139] This invention also provides a vehicle having the above-described features. Figure 5 The vehicle positioning device shown.

[0140] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a vehicle provided in an optional embodiment of the present invention, such as... Figure 6 As shown, the vehicle includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the vehicle, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Figure 6 Take a processor 10 as an example.

[0141] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0142] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0143] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on vehicle usage. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the vehicle via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0145] The vehicle also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0146] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the vehicle, such as a touchscreen, keypad, mouse, trackpad, touchpad, indicator, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0147] The vehicle also includes a communication interface for communicating with other devices or communication networks.

[0148] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0149] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A vehicle positioning method, characterized in that, The method includes: The system acquires image information collected by an image sensor and vehicle motion information and / or position information collected by a target sensor, wherein the target sensor is a sensor other than the image sensor. Based on the image information, visually detected lane lines are obtained; based on the visually detected lane lines and the map, first measurement data of lane line factors are obtained; based on the vehicle motion information and / or position information, second measurement data of corresponding factors are obtained. A factor map including lane line factors and target factors corresponding to the target sensor is constructed based on the first measurement data and the second measurement data; Sliding window optimization is performed on the variables and factors in the factor graph to obtain vehicle pose information; The first measurement data for obtaining lane line factors based on the visually detected lane lines and the map includes: Based on the latest vehicle position information, at least one first lane line is obtained from the map; From the at least one first lane line, obtain the target first lane line that matches each of the visually detected lane lines; Obtain the target line segment in the first lane line corresponding to the sampling point of the visually detected lane line to obtain the first measurement data; The objective function used in the sliding window optimization includes the residuals of the lane line factor, the residuals of the inertial pre-integration, the residuals of the wheel speed integration, the residuals of the wheel speed factor, and the residuals of the position and velocity factor of the Global Navigation Satellite System. The residual of the lane line factor is the vertical distance from the sampling point of the visually detected lane line to the target line segment in the first lane line. The residuals of the inertial pre-integration are used to constrain the relative pose, velocity, and zero bias changes between adjacent time moments. The residuals of the wheel speed integration are used to constrain the relative translational motion between adjacent frames based on the wheel speed measurement value. The residuals of the wheel speed factor are used to keep the wheel speed consistent with the measurement data of the wheel speed sensor. The position and velocity factor of the Global Navigation Satellite System is used to keep the position and velocity consistent with the measurement information of the Global Navigation Satellite System sensor. The step of obtaining the target first lane line matching each of the visually detected lane lines from the at least one first lane line includes: Based on lane line edge attributes, line type, and color features, the visually detected lane lines are associated with the corresponding first lane lines; For a pair of visually detected lane lines and the first lane line, based on the latest acquired vehicle pose information, a first distance between the visually detected lane line and the first lane line is obtained; The first distance is compensated to the latest acquired vehicle pose information to obtain prior vehicle pose information; Using the prior vehicle pose information, the visually detected lane line is projected onto the map, and the second distance between the visually detected lane line and the corresponding first lane line is calculated. If the second distance is less than a preset threshold, the first lane line corresponding to the visually detected lane line is determined as the target first lane line matched by the visually detected lane line.

2. The method according to claim 1, characterized in that, After performing sliding window optimization on the factor graph to obtain vehicle pose information, the process further includes: Acquire target information collected by inertial sensors at a target time, wherein the target time is after the time corresponding to the vehicle pose information; Based on the vehicle pose information and the target information, vehicle pose prediction is performed to obtain predicted vehicle pose information for the target time.

3. The method according to claim 1 or 2, characterized in that, The vehicle motion information and / or position information collected by the target sensor includes at least one of the following: acceleration and angular velocity information collected by an inertial sensor, position information and first velocity information collected by a global navigation satellite system sensor, and second velocity information collected by a wheel speed sensor.

4. A vehicle positioning device, characterized in that, The device includes: The information acquisition module is used to acquire image information acquired by the image sensor, and vehicle motion information and / or position information acquired by the target sensor, wherein the target sensor is a sensor other than the image sensor; The measurement data acquisition module is used to acquire visually detected lane lines based on the image information, and to acquire first measurement data of lane line factors based on the visually detected lane lines and the map; and to acquire second measurement data of corresponding factors based on the vehicle motion information and / or position information. The acquisition of the first measurement data of lane line factors based on the visually detected lane lines and the map includes: acquiring at least one first lane line from the map based on the latest acquired vehicle pose information; acquiring target first lane lines matching each of the visually detected lane lines from the at least one first lane line; acquiring target line segments in the target first lane lines corresponding to the sampling points of the visually detected lane lines to obtain the first measurement data; the acquisition of target first lane lines matching each of the visually detected lane lines from the at least one first lane line includes: associating the visually detected lane lines with the corresponding first lane lines based on lane line edge attributes, line type, and color features; and for an associated pair of visually detected lane lines and the first lane line, acquiring a first distance between the visually detected lane lines and the first lane line based on the latest acquired vehicle pose information. A factor graph construction module is used to construct a factor graph including lane line factors and target factors corresponding to the target sensor based on the first measurement data and the second measurement data; The positioning module is used to perform sliding window optimization on the variables and factors in the factor graph to obtain vehicle pose information. The objective function used in the sliding window optimization includes the residuals of lane line factors, inertial pre-integration residuals, wheel speed integration residuals, wheel speed factor residuals, and the residuals of the position and velocity factors of the Global Navigation Satellite System (GNSS). The residual of the lane line factor is the vertical distance from the sampling point of the visually detected lane line to the target line segment in the first lane line. The residual of the inertial pre-integration is used to constrain the relative pose, velocity, and zero bias changes at adjacent time points. The residual of the wheel speed integration is used to constrain the relative translational motion of adjacent frames based on the wheel speed measurement value. The residual of the wheel speed factor is used to keep the wheel speed consistent with the measurement data of the wheel speed sensor. The position and velocity factors of the GNSS are used to keep the position and velocity consistent with the measurement information of the GNSS sensor.

5. A vehicle, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the vehicle positioning method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the vehicle positioning method according to any one of claims 1 to 3.

Images