Fusion localization methods, devices and electronic equipment for autonomous vehicles
By acquiring multi-sensor positioning data from autonomous vehicles and utilizing confidence thresholds and visual-assisted verification, a fusion strategy is determined, which solves the problem of reduced positioning accuracy and stability in multi-sensor fusion positioning and achieves more reliable positioning results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2023-03-15
- Publication Date
- 2026-05-26
Smart Images
Figure CN116295343B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to a fusion positioning method, device and electronic device for autonomous vehicles. Background Technology
[0002] As the cost of hardware related to autonomous vehicles decreases, high-performance sensors such as multi-line LiDAR, high-resolution cameras, and high-performance onboard computing units are increasingly being considered and incorporated into the hardware suites of autonomous vehicles. Correspondingly, traditional integrated navigation and positioning systems, such as IMU (Inertial Measurement Unit) + GNSS (Global Navigation Satellite System) / RTK (Real-time kinematic), are being replaced by multi-sensor fusion positioning solutions, such as those incorporating LiDAR and lane-matching positioning, to ensure that autonomous vehicles have additional observation sources to guarantee positioning accuracy and stability in the event of RTK failure.
[0003] Currently, most multi-sensor fusion positioning solutions choose a Kalman filter fusion approach that combines IMU, GNSS / RTK, and laser SLAM (Simultaneous Localization and Mapping). This means that when the RTK positioning signal is not good, laser positioning is used to replace or assist the integrated navigation and positioning system in optimizing the observation values to ensure the long-term stability of the overall positioning.
[0004] However, if the positioning confidence provided by the observation source is inaccurate, for example, at a certain moment, GNSS / RTK gives a low-confidence positioning result with correct position, while laser positioning gives a high-confidence positioning result with incorrect position, direct fusion will cause the final positioning of the autonomous vehicle to deviate from the lane, resulting in reduced positioning accuracy and positioning stability. Summary of the Invention
[0005] This application provides a fusion positioning method, apparatus, and electronic device for autonomous vehicles to improve the stability and accuracy of fusion positioning for autonomous vehicles.
[0006] The embodiments of this application adopt the following technical solutions:
[0007] In a first aspect, embodiments of this application provide a fusion positioning method for autonomous vehicles, wherein the method includes:
[0008] The system acquires positioning data from multiple sensors of an autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data.
[0009] The confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are determined using preset confidence threshold conditions;
[0010] The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0011] Fusion positioning is performed according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0012] Optionally, determining the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition includes:
[0013] The confidence level corresponding to the satellite positioning data is compared with a first preset confidence threshold to obtain the confidence level type corresponding to the satellite positioning data;
[0014] The confidence level corresponding to the laser positioning data is compared with a second preset confidence threshold to obtain the confidence level type corresponding to the laser positioning data.
[0015] Optionally, determining the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data includes:
[0016] The positioning error between the satellite positioning data and the laser positioning data is determined based on the satellite positioning data and the laser positioning data.
[0017] The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data.
[0018] Optionally, determining the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data includes:
[0019] If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is less than a preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data and / or laser positioning data.
[0020] If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than the preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data, the laser positioning data and the visual positioning data.
[0021] If both the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are low confidence, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the visual positioning data.
[0022] Optionally, the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data and / or laser positioning data, and the step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes:
[0023] Perform fusion positioning based on the aforementioned satellite positioning data; or...
[0024] Perform fusion positioning based on the laser positioning data; or...
[0025] The fusion weight of the satellite positioning data is determined based on the confidence level corresponding to the satellite positioning data, and the fusion weight of the laser positioning data is determined based on the confidence level corresponding to the laser positioning data. Then, fusion positioning is performed based on the fusion weight of the satellite positioning data and the fusion weight of the laser positioning data.
[0026] Optionally, the fusion positioning strategy is a fusion positioning strategy based on the satellite positioning data, the laser positioning data, and the visual positioning data. The step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes:
[0027] Determine the validity of the visual positioning data;
[0028] If the visual positioning data is valid, determine the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data, respectively.
[0029] Based on the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data, update the confidence level corresponding to the satellite positioning data and the confidence level corresponding to the laser positioning data;
[0030] The fusion positioning result of the autonomous vehicle is obtained by performing fusion positioning based on the confidence levels corresponding to the updated satellite positioning data and the updated laser positioning data.
[0031] Optionally, the visual positioning data includes the lateral positioning position of the lane line at the current moment, and determining the validity of the visual positioning data includes:
[0032] Obtain the heading angle change rate of the autonomous vehicle, and determine the heading angle reference value based on the heading angle change rate;
[0033] The heading angle at the current moment is determined based on the lateral positioning position of the lane line at the current moment and the fused positioning result of the previous moment;
[0034] The heading angle at the current moment is compared with the heading angle reference value, and the validity of the visual positioning data is determined based on the comparison result.
[0035] Secondly, embodiments of this application also provide a fusion positioning device for autonomous vehicles, wherein the device includes:
[0036] The acquisition unit is used to acquire positioning data from multiple sensors of the autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data.
[0037] The first determining unit is used to determine the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition;
[0038] The second determining unit is used to determine the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0039] The fusion positioning unit is used to perform fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0040] Thirdly, embodiments of this application also provide an electronic device, including:
[0041] Processor; and
[0042] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.
[0043] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.
[0044] The above-mentioned technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: The fusion positioning method for autonomous vehicles in the embodiments of this application first acquires positioning data from multiple sensors of the autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data; then, it uses a preset confidence threshold condition to determine the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data; next, it determines the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data; finally, it performs fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle. The fusion positioning method for autonomous vehicles in the embodiments of this application performs mutual verification of confidence based on the positioning data of multiple sensors, and adopts different fusion positioning strategies according to the verification results, ensuring that the fusion positioning algorithm has reliable observation input, and improving the stability and positioning accuracy of fusion positioning. Attached Figure Description
[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0046] Figure 1 This is a flowchart illustrating a fusion localization method for an autonomous vehicle according to an embodiment of this application.
[0047] Figure 2 This is a schematic diagram of the structure of a fusion positioning device for an autonomous vehicle according to an embodiment of this application;
[0048] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0050] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0051] This application provides a fusion localization method for autonomous vehicles, such as... Figure 1 The diagram provided illustrates a flowchart of a fusion localization method for an autonomous vehicle according to an embodiment of this application. The method includes at least the following steps S110 to S140:
[0052] Step S110: Obtain positioning data from multiple sensors of the autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data.
[0053] In this embodiment of the application, when performing fusion localization of autonomous vehicles, it is necessary to first obtain localization data output by multiple sensors of the autonomous vehicle. Specifically, this may include satellite localization data output by IMU+GNSS / RTK observation sources, laser localization data output by laser SLAM, and visual localization data output by vision.
[0054] Because different sensors have different output frequencies and are affected differently by the external environment, it is not always possible to receive positioning results from multiple sensors simultaneously at a given observation update time. Therefore, this embodiment of the application can first synchronize the time and frequency of satellite positioning data, laser positioning data, and visual positioning data to ensure that positioning data from all three observation sources are received simultaneously at the observation update time. Specifically, a low-frequency observation source such as GNSS / RTK can be used as a reference. Compared to the other two observation sources, its frequency is low and stable. It is determined whether there are corresponding positioning results from other sensors within a preset buffer time range. If so, vehicle speed information and time intervals can be used for prediction to obtain the positioning results of other sensors at the corresponding GNSS / RTK time.
[0055] Step S120: Determine the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition.
[0056] GNSS / RTK output satellite positioning data includes the location confidence level of satellite positioning data, and laser SLAM output laser positioning data also includes the location confidence level of laser positioning data. Therefore, different predefined confidence threshold conditions can be used to determine the confidence type corresponding to satellite positioning data and the confidence type corresponding to laser positioning data, such as whether it is high confidence or low confidence.
[0057] Since GNSS / RTK may output low-confidence positioning results that are correct at certain times, while laser positioning may output high-confidence positioning results that are incorrect, determining the confidence type corresponding to satellite positioning data and the confidence type corresponding to laser positioning data can provide a basis for mutual verification of the confidence of satellite positioning data and laser positioning data.
[0058] Step S130: Determine the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0059] When the confidence level based on the confidence level of satellite positioning data and laser positioning data cannot accurately determine the reliability of the positioning results of satellite positioning and laser SLAM, visual positioning data can be further combined for auxiliary verification. Based on the verification results, the reliability of the positioning results of different sensors can be determined, and different fusion positioning strategies can be adopted.
[0060] Step S140: Perform fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0061] The aforementioned fusion positioning strategy refers to fusion positioning based on the positioning data of currently reliable sensors. Therefore, according to this fusion positioning strategy, the positioning data of the corresponding sensors can be input into the Kalman filter for fusion positioning to obtain the fusion positioning result of the autonomous vehicle.
[0062] The autonomous vehicle fusion localization method of this application embodiment is based on mutual verification of the confidence of localization data from multiple sensors. Different fusion localization strategies are adopted according to the verification results, which ensures that the fusion localization algorithm has reliable observation input and improves the stability and localization accuracy of fusion localization.
[0063] In some embodiments of this application, determining the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using preset confidence threshold conditions includes: comparing the confidence level corresponding to the satellite positioning data with a first preset confidence threshold to obtain the confidence type corresponding to the satellite positioning data; and comparing the confidence level corresponding to the laser positioning data with a second preset confidence threshold to obtain the confidence type corresponding to the laser positioning data.
[0064] Since satellite positioning and laser SLAM positioning have different positioning accuracy requirements, different confidence thresholds can be set for each. For example, a first preset confidence threshold can be set for satellite positioning data, and a second preset confidence threshold can be set for laser positioning data. The size of the first preset confidence threshold and the second preset confidence threshold mainly depends on the positioning accuracy requirements, so they can be set flexibly according to needs, and no specific limitation is made here.
[0065] The confidence level of satellite positioning data is compared with a corresponding first preset confidence threshold. If the confidence level of the satellite positioning data is greater than the first preset confidence threshold, the confidence level of the satellite positioning data is determined to be high confidence; otherwise, it is determined to be low confidence. The confidence level of laser positioning data is compared with a corresponding second preset confidence threshold. If the confidence level of the laser positioning data is greater than the second preset confidence threshold, the confidence level of the laser positioning data is determined to be high confidence; otherwise, it is determined to be low confidence.
[0066] In some embodiments of this application, determining the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data includes: determining the positioning error between the satellite positioning data and the laser positioning data based on the satellite positioning data and the laser positioning data; and determining the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data.
[0067] When verifying the confidence levels of satellite positioning data and laser positioning data, the positioning error between satellite positioning data and laser positioning data can be further calculated. If the positioning error between satellite positioning data and laser positioning data is less than a certain error threshold, then it is highly likely that the positioning results of both satellite positioning and laser SLAM are reliable, that is, they meet the corresponding positioning accuracy requirements. This is because the positioning error between the two is likely to be less than the error threshold only when both positioning results are relatively accurate.
[0068] Therefore, based on the magnitude of the positioning error between satellite positioning data and laser positioning data, as well as the confidence level of satellite positioning data, the confidence level of laser positioning data, and visual positioning data, the reliability of the positioning results of each sensor can be determined, and thus the corresponding fusion positioning strategy can be determined.
[0069] In some embodiments of this application, determining the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data includes: if at least one of the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is less than a preset positioning error threshold, then the fusion positioning strategy is determined to be based on the satellite positioning data and / or the laser positioning data. The implemented fusion positioning strategy is as follows: if at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than the preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data, the laser positioning data, and the visual positioning data; if both the confidence types corresponding to the satellite positioning data and the laser positioning data are low confidence, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the visual positioning data.
[0070] When determining the fusion positioning strategy for autonomous vehicles based on the confidence type of satellite positioning data and laser positioning data, the positioning error between satellite positioning data and laser positioning data, and visual positioning data, the main situations can be divided into the following categories:
[0071] (1) Both the confidence level of the satellite positioning data and the confidence level of the laser positioning data are high confidence. Considering that satellite positioning or laser SLAM positioning may output high confidence positioning results with incorrect positions, we can further judge based on the magnitude of the positioning error between the satellite positioning data and the laser positioning data:
[0072] 1) If the positioning error between satellite positioning data and laser positioning data is less than the preset positioning error threshold, it can be basically considered that the confidence level of both satellite positioning data and laser positioning data is accurate. It can be determined that the positioning results of both are valid observation results. Therefore, the fusion positioning strategy can be to select either one or perform weighted fusion based on the confidence level.
[0073] 2) If the positioning error between satellite positioning data and laser positioning data is not less than the preset positioning error threshold, it indicates that at least one of the satellite positioning data and laser positioning data has an unreliable confidence level. Visual positioning data can be further combined for auxiliary verification, and then the corresponding fusion positioning strategy can be determined based on the verification results.
[0074] (2) If either the confidence level of the satellite positioning data or the confidence level of the laser positioning data is low, and considering that satellite positioning or laser SLAM positioning may output a low-confidence positioning result that is correct or a high-confidence positioning result that is incorrect, we can further judge based on the magnitude of the positioning error between the satellite positioning data and the laser positioning data:
[0075] 1) If the positioning error between satellite positioning data and laser positioning data is less than the preset positioning error threshold, it can be basically assumed that the sensor corresponding to the low confidence type outputs a low confidence positioning result with the correct position. That is, the positioning result itself is reliable. Therefore, the fusion positioning strategy at this time is the same as that in (1)-1), that is, select any one of them or perform weighted fusion based on confidence.
[0076] 2) If the positioning error between satellite positioning data and laser positioning data is not less than the preset positioning error threshold, the reliability of satellite positioning data and laser positioning data cannot be directly determined in this case. Visual positioning data can be further combined for auxiliary verification, and then the corresponding fusion positioning strategy can be determined based on the verification results.
[0077] (3) Both the confidence type of satellite positioning data and the confidence type of laser positioning data are low confidence. In this case, it can be basically considered that the positioning results of satellite positioning and laser SLAM positioning are invalid observation results. Fusion positioning can be achieved based on visual positioning data. Of course, when fusion positioning is achieved based on visual positioning data, the validity of visual positioning data can be judged first. If the visual positioning data is valid, lane keeping is performed using the lateral positioning position matched by the lane line, and the positioning results of the subsequent N time moments are judged. When GNSS / RTK and laser positioning meet the conditions of (1) or (2), lane keeping is turned off and normal fusion positioning output is performed.
[0078] In some embodiments of this application, the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data and / or laser positioning data. The step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes: performing fusion positioning based on the satellite positioning data; or, performing fusion positioning based on the laser positioning data; or, determining the fusion weight of the satellite positioning data based on the confidence level corresponding to the satellite positioning data, and determining the fusion weight of the laser positioning data based on the confidence level corresponding to the laser positioning data, and performing fusion positioning based on the fusion weight of the satellite positioning data and the fusion weight of the laser positioning data.
[0079] If the fusion positioning strategy is based on satellite positioning data and / or laser positioning data, it means that both satellite positioning data and laser positioning data are valid observation results. Therefore, either one can be selected as additional observation information and input into the Kalman filter for fusion positioning. Alternatively, the corresponding fusion weights can be determined according to the confidence levels of the two data and weighted fusion can be performed. That is, the higher the confidence level, the larger the corresponding fusion weight value.
[0080] In some embodiments of this application, the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data, the laser positioning data, and the visual positioning data. The step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes: determining the validity of the visual positioning data; if the visual positioning data is valid, determining the positioning errors between the satellite positioning data and the visual positioning data, and the positioning errors between the laser positioning data and the visual positioning data, respectively; updating the confidence levels corresponding to the satellite positioning data and the laser positioning data based on the positioning errors between the satellite positioning data and the visual positioning data, and the positioning errors between the laser positioning data and the visual positioning data; and performing fusion positioning based on the updated confidence levels corresponding to the satellite positioning data and the updated confidence levels corresponding to the laser positioning data to obtain the fusion positioning result of the autonomous vehicle.
[0081] If the fusion positioning strategy is based on satellite positioning data, laser positioning data, and visual positioning data, the positioning results in the visual positioning data may also be unreliable. Therefore, when using visual positioning data to assist in verifying the confidence of satellite positioning and laser SLAM positioning, the validity of the visual positioning data can be determined first. If the visual positioning data is valid, the positioning errors between satellite positioning data and visual positioning data, as well as the positioning errors between laser positioning data and visual positioning data, can be calculated separately.
[0082] It should be noted that since visual positioning data mainly refers to the lateral positioning position of lane lines, the lateral positioning positions of satellite positioning data and laser positioning data can be decomposed separately to calculate the lateral positioning errors between satellite positioning data and visual positioning data, as well as between laser positioning data and visual positioning data.
[0083] Next, the lateral positioning error between the satellite positioning data and the visual positioning data is compared with the lateral positioning error between the laser positioning data and the visual positioning data. Based on the comparison results, it is determined which of the satellite positioning data and the visual positioning data is closer to the lateral positioning position of the visual positioning data, that is, the one with the smaller lateral positioning error. The closer the lateral positioning data is, the more reliable the positioning data is. Then, the confidence scores corresponding to the satellite positioning data and the visual positioning data can be recalculated, that is, confidence score correction is performed. Finally, a weighted fusion is performed based on the recalculated confidence scores, thereby ensuring that the fusion positioning algorithm can use more reliable observation inputs, improving the fusion positioning accuracy and positioning stability.
[0084] To facilitate understanding, here is a way to recalculate the confidence level, which can be represented as follows:
[0085] Conf_GNSS = 1 - D_GNSS / (D_GNSS + D_lidar)
[0086] Conf_lidar = 1 - Conf_GNSS
[0087] Where D_GNSS is the lateral positioning error between satellite positioning data and visual positioning data, D_lidar is the lateral positioning error between laser positioning data and visual positioning data, Conf_GNSS is the confidence level corresponding to the updated satellite positioning data, and Conf_lidar is the confidence level corresponding to the updated laser positioning data.
[0088] Of course, those skilled in the art can flexibly set the specific method for recalculating the confidence level according to actual needs, as long as the smaller the lateral positioning error, the higher the corresponding confidence level.
[0089] It should be noted that although the confidence levels corresponding to the updated satellite positioning data and the updated laser positioning data are calculated based on the lateral positioning error, they can also be applied to the weighted fusion of longitudinal positioning in the fusion positioning process. That is, the lateral positioning and longitudinal positioning use the same weight to reduce the probability of lane keeping.
[0090] In some embodiments of this application, the visual positioning data includes the lateral positioning position of the lane line at the current moment, and determining the validity of the visual positioning data includes: obtaining the heading angle change rate of the autonomous vehicle and determining a heading angle reference value based on the heading angle change rate; determining the heading angle at the current moment based on the fusion positioning result of the lane line lateral positioning position at the current moment and the previous moment; comparing the heading angle at the current moment with the heading angle reference value, and determining the validity of the visual positioning data based on the comparison result.
[0091] When determining the validity of visual positioning data, the yaw rate output by the autonomous vehicle can be obtained first. Then, the yaw rate reference value is calculated by integration. Next, the yaw rate at the current moment is calculated based on the lateral positioning position of the lane line at the current moment and the lateral positioning position decomposed from the fused positioning result at the previous moment. The yaw rate at the current moment is compared with the above-mentioned yaw rate reference value. If the deviation between the two is within a certain range, it means that the lateral positioning position of the autonomous vehicle at the current moment has not deviated from the lane, and the lateral positioning position of the lane line is reliable. It can be used as a basis for auxiliary verification of the confidence of satellite positioning data and laser positioning data. If the deviation between the two exceeds a certain range, it means that the current lateral positioning position of the lane line may have deviated from the lane, and the lateral positioning position is unreliable and cannot be used as a basis for auxiliary verification. In this case, an alarm can be further triggered to facilitate timely manual intervention.
[0092] This application embodiment uses the heading angle change rate as additional auxiliary information to further determine the validity of the lane matching results, thereby reducing errors caused by lane matching mistakes.
[0093] In some embodiments of this application, determining the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data includes: if the visual positioning data cannot be obtained, and the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are not both high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than a preset positioning error threshold, then the heading angle change rate of the autonomous vehicle is obtained, and a heading angle reference value is determined based on the heading angle change rate; the heading angle corresponding to the satellite positioning data at the current moment is determined based on the fusion positioning result of the satellite positioning data and the previous moment, and the heading angle corresponding to the laser positioning data at the current moment is determined based on the fusion positioning result of the laser positioning data and the previous moment; the heading angle corresponding to the satellite positioning data at the current moment and the heading angle corresponding to the laser positioning data at the current moment are compared with the heading angle reference value respectively, and the fusion positioning strategy of the autonomous vehicle is determined based on the comparison result.
[0094] In some special scenarios, such as intersections or rainy days, there may be situations where lane lines cannot be identified and do not meet the requirements of (1)-1) in the aforementioned embodiments. In such cases, you can choose to stop with an alarm or rely on the rate of change of heading angle for observation selection. For example, you can choose one and only use the observation information that is closest to the change of heading angle for fusion positioning.
[0095] This application also provides a fusion positioning device 200 for autonomous vehicles, such as... Figure 2 As shown, a structural schematic diagram of a fusion positioning device for an autonomous vehicle according to an embodiment of this application is provided. The device 200 includes:
[0096] The acquisition unit 210 is used to acquire positioning data from multiple sensors of the autonomous vehicle, including satellite positioning data, laser positioning data and visual positioning data.
[0097] The first determining unit 220 is used to determine the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition;
[0098] The second determining unit 230 is used to determine the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0099] The fusion positioning unit 240 is used to perform fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0100] In some embodiments of this application, the first determining unit 220 is specifically used to: compare the confidence level corresponding to the satellite positioning data with a first preset confidence threshold to obtain the confidence level type corresponding to the satellite positioning data; and compare the confidence level corresponding to the laser positioning data with a second preset confidence threshold to obtain the confidence level type corresponding to the laser positioning data.
[0101] In some embodiments of this application, the second determining unit 230 is specifically used to: determine the positioning error between the satellite positioning data and the laser positioning data based on the satellite positioning data and the laser positioning data; and determine the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data.
[0102] In some embodiments of this application, the second determining unit 230 is specifically configured to: if at least one of the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is less than a preset positioning error threshold, then determine that the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data and / or the laser positioning data; if at least one of the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than the preset positioning error threshold, then determine that the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data, the laser positioning data, and the visual positioning data; if both the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are low confidence, then determine that the fusion positioning strategy is a fusion positioning strategy implemented based on the visual positioning data.
[0103] In some embodiments of this application, the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data and / or laser positioning data. The fusion positioning unit 240 is specifically used to: perform fusion positioning based on the satellite positioning data; or, perform fusion positioning based on the laser positioning data; or, determine the fusion weight of the satellite positioning data based on the confidence level corresponding to the satellite positioning data, and determine the fusion weight of the laser positioning data based on the confidence level corresponding to the laser positioning data, and perform fusion positioning based on the fusion weight of the satellite positioning data and the fusion weight of the laser positioning data.
[0104] In some embodiments of this application, the fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data, the laser positioning data, and the visual positioning data. The fusion positioning unit 240 is specifically used to: determine the validity of the visual positioning data; if the visual positioning data is valid, determine the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data, respectively; update the confidence level corresponding to the satellite positioning data and the confidence level corresponding to the laser positioning data based on the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data; and perform fusion positioning based on the updated confidence level corresponding to the satellite positioning data and the updated confidence level corresponding to the laser positioning data to obtain the fusion positioning result of the autonomous vehicle.
[0105] In some embodiments of this application, the visual positioning data includes the lateral positioning position of the lane line at the current moment, and the fusion positioning unit 240 is specifically used to: obtain the heading angle change rate of the autonomous vehicle, and determine the heading angle reference value based on the heading angle change rate; determine the heading angle at the current moment based on the lateral positioning position of the lane line at the current moment and the fusion positioning result at the previous moment; compare the heading angle at the current moment with the heading angle reference value, and determine the validity of the visual positioning data based on the comparison result.
[0106] It is understood that the aforementioned fusion positioning device for autonomous vehicles can implement each step of the fusion positioning method for autonomous vehicles provided in the foregoing embodiments. The relevant explanations regarding the fusion positioning method for autonomous vehicles are applicable to the fusion positioning device for autonomous vehicles, and will not be repeated here.
[0107] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.
[0108] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.
[0109] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.
[0110] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming the fusion positioning device for the autonomous vehicle at the logical level. The processor executes the program stored in memory and specifically performs the following operations:
[0111] The system acquires positioning data from multiple sensors of an autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data.
[0112] The confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are determined using preset confidence threshold conditions;
[0113] The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0114] Fusion positioning is performed according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0115] The above is as stated in this application. Figure 1 The method executed by the fusion positioning device of the autonomous vehicle disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0116] The electronic device can also perform Figure 1 The method for implementing the fusion positioning device in autonomous vehicles, and the realization of the fusion positioning device in autonomous vehicles. Figure 1 The functions of the embodiments shown are not described in detail here.
[0117] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the fusion positioning device of the autonomous vehicle in the illustrated embodiment is specifically used to perform:
[0118] The system acquires positioning data from multiple sensors of an autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data.
[0119] The confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are determined using preset confidence threshold conditions;
[0120] The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data.
[0121] Fusion positioning is performed according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle.
[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0127] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0131] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A fusion localization method for autonomous vehicles, wherein, The method includes: The system acquires positioning data from multiple sensors of an autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data. The confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are determined using preset confidence threshold conditions; The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data. Fusion positioning is performed according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle; The step of determining the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data includes: The positioning error between the satellite positioning data and the laser positioning data is determined based on the satellite positioning data and the laser positioning data. The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data. The step of determining the fusion positioning strategy for the autonomous vehicle based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data includes: If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is less than a preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data and / or laser positioning data. If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than the preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data, the laser positioning data and the visual positioning data. If both the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are low confidence, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the visual positioning data.
2. The method as described in claim 1, wherein, The step of determining the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition includes: The confidence level corresponding to the satellite positioning data is compared with a first preset confidence threshold to obtain the confidence level type corresponding to the satellite positioning data; The confidence level corresponding to the laser positioning data is compared with a second preset confidence threshold to obtain the confidence level type corresponding to the laser positioning data.
3. The method as described in claim 1, wherein, The fusion positioning strategy is a fusion positioning strategy implemented based on the satellite positioning data and / or laser positioning data. The step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes: Perform fusion positioning based on the aforementioned satellite positioning data; or... Perform fusion positioning based on the laser positioning data; or... The fusion weight of the satellite positioning data is determined based on the confidence level corresponding to the satellite positioning data, and the fusion weight of the laser positioning data is determined based on the confidence level corresponding to the laser positioning data. Then, fusion positioning is performed based on the fusion weight of the satellite positioning data and the fusion weight of the laser positioning data.
4. The method as described in claim 1, wherein, The fusion positioning strategy is a fusion positioning strategy based on the satellite positioning data, the laser positioning data, and the visual positioning data. The step of performing fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle includes: Determine the validity of the visual positioning data; If the visual positioning data is valid, determine the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data, respectively. Based on the positioning error between the satellite positioning data and the visual positioning data, and the positioning error between the laser positioning data and the visual positioning data, update the confidence level corresponding to the satellite positioning data and the confidence level corresponding to the laser positioning data; The fusion positioning result of the autonomous vehicle is obtained by performing fusion positioning based on the confidence levels corresponding to the updated satellite positioning data and the updated laser positioning data.
5. The method as described in claim 4, wherein, The visual positioning data includes the lateral positioning position of the lane line at the current moment, and determining the validity of the visual positioning data includes: Obtain the heading angle change rate of the autonomous vehicle, and determine the heading angle reference value based on the heading angle change rate; The heading angle at the current moment is determined based on the lateral positioning position of the lane line at the current moment and the fused positioning result of the previous moment; The heading angle at the current moment is compared with the heading angle reference value, and the validity of the visual positioning data is determined based on the comparison result.
6. A fusion positioning device for an autonomous vehicle, wherein, The device includes: The acquisition unit is used to acquire positioning data from multiple sensors of the autonomous vehicle, including satellite positioning data, laser positioning data, and visual positioning data. The first determining unit is used to determine the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data using a preset confidence threshold condition; The second determining unit is used to determine the fusion positioning strategy of the autonomous vehicle based on the confidence type corresponding to the satellite positioning data, the confidence type corresponding to the laser positioning data, and the visual positioning data. The fusion positioning unit is used to perform fusion positioning according to the fusion positioning strategy of the autonomous vehicle to obtain the fusion positioning result of the autonomous vehicle. The second determining unit is specifically used for: The positioning error between the satellite positioning data and the laser positioning data is determined based on the satellite positioning data and the laser positioning data. The fusion positioning strategy for the autonomous vehicle is determined based on the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data, the positioning error between the satellite positioning data and the laser positioning data, and the visual positioning data. The second determining unit is specifically used for: If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is less than a preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data and / or laser positioning data. If at least one of the confidence types corresponding to the satellite positioning data and the laser positioning data is high confidence, and the positioning error between the satellite positioning data and the laser positioning data is not less than the preset positioning error threshold, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the satellite positioning data, the laser positioning data and the visual positioning data. If both the confidence type corresponding to the satellite positioning data and the confidence type corresponding to the laser positioning data are low confidence, then the fusion positioning strategy is determined to be a fusion positioning strategy based on the visual positioning data.
7. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 5.
8. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 5.