Driving positioning multi-sensor data diagnosis method, device, equipment and medium
通过多传感器数据诊断方法,结合局部建图和GNSS状态,解决了自动驾驶车辆中传感器数据异常导致的定位精度问题,实现了在复杂场景下的高精度定位输出。
Patent Information
- Application Number
- CN202411202155.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-12
AI Technical Summary
In autonomous driving vehicles, abnormal input of sensor data causes the positioning function to fail to achieve high-precision positioning, especially in high-speed and urban expressway scenarios. How to accurately and timely discover and isolate abnormal data to output the correct status bits is a challenge.
By obtaining visual perception data, high-precision map data and global navigation satellite system data, combining local mapping mechanisms and pre-determined driving positioning trajectories, quality diagnosis of multi-sensor data is carried out, abnormal data is eliminated, and the update of high-precision map data is checked. and GNSS data quality to realize quality diagnosis of multi-sensor data.
Effectively eliminate abnormal data, ensure high-precision output of positioning function, and improve the positioning robustness and accuracy of autonomous driving vehicles in complex scenarios.
Smart Images

Figure CN118938266A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent driving positioning technology, and in particular to a driving positioning multi-sensor data diagnosis method, device, equipment and medium. Background Art
[0002] In the field of autonomous driving, high-precision positioning based on high-precision maps is a standard product for L3+ high-level intelligent driving, providing high-precision positioning functions for autonomous driving vehicles and providing strong support for planning control, fusion prediction, etc. In actual applications, the input signal of sensor data will cause abnormal Global Navigation Satellite System (GNSS) data due to environmental influences, abnormal perception results due to perception algorithm capabilities, and abnormal map data due to untimely updates of high-precision map data. The access of these abnormal information will cause the positioning function to be unable to output high-precision positioning results. In high-speed and urban expressway scenarios, robust sensor input is an advance guarantee for high-precision positioning. How to accurately and timely detect abnormal data, isolate it, and output the correct status bit is crucial. Summary of the invention
[0003] The embodiments of the present invention provide a method, device, equipment and medium for diagnosing multi-sensor data for vehicle positioning, and realize a multi-sensor data quality diagnosis function.
[0004] In a first aspect, an embodiment of the present invention provides a vehicle positioning multi-sensor data diagnosis method, comprising:
[0005] Obtain visual perception data, high-precision map data, and global navigation satellite system data to be diagnosed;
[0006] Determine the diagnosis result of the visual perception data according to the visual perception data and the predetermined vehicle positioning trajectory in combination with the local mapping mechanism, wherein the diagnosis result of the visual perception data includes abnormal isolation data, temporary element data and locked element data, and the temporary element data and the locked element data are used as the current frame data of the local mapping;
[0007] Performing mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data;
[0008] A diagnosis result of the global navigation satellite system data is determined according to the global navigation satellite system data and the vehicle positioning trajectory.
[0009] In a second aspect, an embodiment of the present invention provides a vehicle positioning multi-sensor data diagnosis device, comprising:
[0010] A data acquisition module, used to acquire visual perception data to be diagnosed, high-precision map data, and global navigation satellite system data;
[0011] A first diagnostic module is used to determine a diagnostic result of the visual perception data according to the visual perception data and a predetermined vehicle positioning trajectory in combination with a local mapping mechanism, wherein the diagnostic result of the visual perception data includes abnormal isolation data, temporarily stored element data, and locked element data, and the temporarily stored element data and the locked element data are used as current frame data for local mapping;
[0012] A second diagnosis module is used to perform mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data;
[0013] The third diagnosis module is used to determine the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory.
[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, including:
[0015] at least one processor; and
[0016] a memory communicatively connected to the at least one processor; wherein,
[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle positioning multi-sensor data diagnosis method as described in the first aspect embodiment.
[0018] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the vehicle positioning multi-sensor data diagnosis method as described in the embodiment of the first aspect.
[0019] An embodiment of the present invention provides a method, device, equipment and medium for diagnosing multi-sensor data for vehicle positioning, the method comprising: obtaining visual perception data to be diagnosed, high-precision map data and global navigation satellite system data; determining the diagnosis result of the visual perception data based on the visual perception data and a predetermined vehicle positioning trajectory, in combination with a local mapping mechanism, the diagnosis result of the visual perception data including abnormal isolation data, temporary element data and locked element data, the temporary element data and locked element data being used as local mapping current frame data; performing mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis result of the local mapping current frame data and the high-precision map data; determining the diagnosis result of the global navigation satellite system data based on the global navigation satellite system data and the vehicle positioning trajectory. The above technical solution introduces the local mapping function to construct a local perception map to eliminate abnormal visual perception data; combines the local mapping data to judge the high-precision map data that is not updated in time, and performs mutual verification between the current frame data of the local mapping and the high-precision map data; detects the GNSS data quality through the predetermined driving positioning trajectory combined with the GNSS status, thereby realizing the multi-sensor data quality diagnosis function.
[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0022] Figure 1 A schematic diagram of a flow chart of a vehicle positioning multi-sensor data diagnosis method provided in Embodiment 1 of the present invention;
[0023] Figure 2 A schematic flow chart of another method for diagnosing vehicle positioning multi-sensor data provided in the second embodiment of the present invention;
[0024] Figure 3 A schematic diagram of the structure of a vehicle positioning multi-sensor data diagnostic device provided in Embodiment 3 of the present invention;
[0025] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0028] Embodiment 1
[0029] Figure 1 This is a flow chart of a method for diagnosing multi-sensor data for vehicle positioning provided in the first embodiment of the present invention. The method can be applied to the situation of detecting and diagnosing the quality of multi-sensor data for vehicle positioning. The method can be executed by a multi-sensor data diagnosis device for vehicle positioning. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the vehicle positioning multi-sensor data diagnosis method provided in the first embodiment may specifically include the following steps:
[0030] S101. Acquire visual perception data to be diagnosed, high-precision map data, and global navigation satellite system data.
[0031] It should be noted that the high-precision positioning function based on high-precision maps provides high-precision positioning functions for autonomous driving vehicles. The high-precision positioning function achieves driving positioning by fusing multiple sensor data. Exemplarily, the multiple sensor data may include visual perception data, high-precision map data, and global navigation satellite system data. Considering that in actual applications, the input signal of sensor data may cause abnormal global navigation satellite system data due to environmental influences, abnormal perception results due to perception algorithm capabilities, and abnormal high-precision map data due to untimely updates of high-precision map data, the access of these abnormal data will cause the positioning function to be unable to output high-precision positioning results.
[0032] Based on the above description, in this embodiment, before the fusion processing of multiple sensor data, the sensor data collected by the vehicle will be diagnosed for quality, so as to screen out normal data and abnormal data, isolate the abnormal data, and use the normal data for subsequent fusion positioning. Exemplarily, the vehicle acquires visual perception data, high-precision map data and global navigation satellite system data in real time during driving, and performs real-time detection and diagnosis on the quality of these three types of data. Among them, the multi-sensor data includes visual perception data, which can be understood as data obtained by an image acquisition device; it also includes high-precision map data, which can be understood as map data containing detailed information; it also includes global navigation satellite system data, which can be understood as positioning data obtained using satellite technology. This step is used to obtain visual perception data, high-precision map data and global navigation satellite system data that need to be quality detected and diagnosed.
[0033] S102: Determine a diagnosis result of the visual perception data based on the visual perception data and the predetermined vehicle positioning trajectory in combination with a local mapping mechanism.
[0034] Among them, the diagnosis results of the visual perception data include abnormal isolation data, temporarily stored element data and locked element data, and the temporarily stored element data and locked element data are used as the current frame data of local mapping.
[0035] In this embodiment, a local mapping mechanism (Local Map) function is introduced for visual perception data, and abnormal perception results in the visual perception data are eliminated by constructing a local perception map. When the quality of visual perception data is detected and diagnosed based on the local mapping mechanism, since the visual perception data is in the vehicle coordinate system, it is necessary to convert the visual perception data to the global coordinate system based on the predicted driving positioning trajectory before subsequent processing. The visual perception data contains perceived elements, such as lane lines, poles, traffic lights, etc., and the perceived elements are recorded as visual perception elements.
[0036] After the visual perception data is converted to the global coordinate system, the visual perception elements in the frame of visual perception data can be verified with the local map. Specifically, based on the type and distance, the visual perception data containing the global positioning posture is registered with the local map to determine the diagnostic result of the visual perception data. In this embodiment, when the visual perception data in the global coordinate system is verified with the local map, it can be determined whether the visual perception element contained in the frame of visual perception data appears for the first time. If it appears for the first time, it is used as a temporary element. If the included visual perception element does not appear for the first time, a visual perception position check is performed, and the visual perception elements of the same category are checked with the corresponding elements in the local map. The distance between the two is calculated and compared with the threshold, so as to determine whether the visual perception element is normal data or abnormal data, and the normal data is recorded as locked perception data. The temporary element data and the locked element data are used as the current frame data of the local map. By introducing the local map function, a local perception map is constructed to eliminate abnormal visual perception abnormal data.
[0037] Exemplarily, extract each visual perception element contained in the visual perception data; for each visual perception element, determine whether the visual perception element appears for the first time; if the visual perception element appears for the first time, use the visual perception element as temporary element data; if the visual perception element does not appear for the first time, determine from the local map a corresponding map element that is similar to the global positioning posture and of the same type as the visual perception element, and determine the distance between the visual perception element and the corresponding map element; if the distance is greater than a set distance threshold, determine the visual perception element as abnormal isolation data; if the distance is less than or equal to the set distance threshold, determine the visual perception element as locked element data.
[0038] S103: Perform mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data.
[0039] In this step, the high-precision map data that is not updated in time is determined by combining the current frame data of local mapping and the high-precision map data, and the current frame data of local mapping and the high-precision map data are mutually verified.
[0040] In this embodiment, the distance and type matching conformity check is performed on the current frame data of the local mapping and the high-precision map data, and the locked element data and the temporarily stored element data in the current frame data of the local mapping are obtained as the element data to be matched, and the element data to be matched is preliminarily screened out based on the high-precision map data to determine whether it is normal data or abnormal data, and the result is recorded as the initial matching result. If the initial matching result does not meet the set matching conditions, the element data to be matched is determined to be abnormal isolation data, and the abnormal isolation data is not used for subsequent positioning fusion processing. If the initial matching result meets the set matching conditions, the element data to be matched and the high-precision map data that meet the set matching conditions will be further integrity checked to determine whether the current frame data of the local mapping and the high-precision map data are consistent, and the integrity check result is recorded as the diagnosis result of the current frame data of the local mapping and the high-precision map data.
[0041] S104: Determine a diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory.
[0042] In this embodiment, the GNSS will have an estimate of the vehicle's location, and the error range of the vehicle is recorded as the GNSS data. At the same time, the vehicle positioning trajectory will also have an estimate of the vehicle's location, and the error range of the vehicle is recorded as the GNSS data. This step is used to verify the consistency of the GNSS data and the vehicle positioning trajectory through a chi-square test. If the chi-square test condition is met, the GNSS data is considered to be normal input data. If the chi-square test condition is not met, the GNSS data is determined to be abnormal isolation data.
[0043] It should be noted that, whether it is visual perception data, global navigation satellite system data or high-precision map data, the determined abnormal isolation data will not be used for subsequent positioning fusion processing, while the determined normal input data can be used for subsequent positioning fusion processing.
[0044] An embodiment of the present invention provides a method for diagnosing multi-sensor data for vehicle positioning. By introducing a local mapping function, a local perception map is constructed to eliminate abnormal visual perception data; high-precision map data that is not updated in time is judged in combination with the local mapping data, and the current frame data of the local mapping and the high-precision map data are mutually verified; the GNSS data quality is detected by combining a predetermined vehicle positioning trajectory with the GNSS status, thereby realizing a multi-sensor data quality diagnosis function.
[0045] As an optional embodiment of the embodiment of the present invention, based on the above embodiment, the steps for determining the vehicle positioning trajectory can be optimized, including: determining the vehicle positioning trajectory at the current moment based on the previous vehicle positioning trajectory obtained by multi-sensor fusion positioning at the previous moment and the inertial measurement data at the current moment.
[0046] In this embodiment, the fusion of multiple sensors also takes time, and the fused result is not the current result. Therefore, it is necessary to combine with the inertial measurement unit (IMU) to calculate the current result as the vehicle positioning trajectory. Among them, the vehicle positioning trajectory obtained by multi-sensor fusion positioning at the previous moment is recorded as the previous vehicle positioning trajectory, and combined with the inertial measurement data at the current moment, such as acceleration, angular velocity, etc., to obtain the displacement between the previous moment and the current moment, thereby obtaining the vehicle positioning trajectory at the current moment. Exemplarily, each time the vehicle positioning trajectory is inferred to the current moment, it will be saved in the cache.
[0047] The above technical solution specifies the steps for determining the vehicle positioning trajectory and provides basic data for the follow-up.
[0048] Embodiment 2
[0049] Figure 2 A flow chart of another method for diagnosing multi-sensor data for vehicle positioning provided in Embodiment 2 of the present invention. This embodiment is a further optimization of the above embodiment. In this embodiment, "determining the diagnosis result of the visual perception data based on the visual perception data and the predetermined vehicle positioning trajectory in combination with the local mapping mechanism" is further limited and optimized, and "determining the diagnosis result of the local mapping current frame data and the high-precision map data by mutual verification based on the local mapping current frame data and the high-precision map data" is further limited and optimized, and "determining the diagnosis result of the global navigation satellite system data based on the global navigation satellite system data and the vehicle positioning trajectory" is limited and optimized.
[0050] like Figure 2 As shown, this embodiment 2 provides a vehicle positioning multi-sensor data diagnosis method, which specifically includes the following steps:
[0051] S201. Obtain visual perception data, high-precision map data, and global navigation satellite system data to be diagnosed.
[0052] S202: According to the timestamp of the visual perception data, determine the positioning posture corresponding to the timestamp from the driving positioning trajectory as the global positioning posture of the visual perception data in the global coordinate system.
[0053] In this embodiment, whenever a new frame of visual perception data is to be detected and diagnosed, the timestamp of the visual perception data is used to find the nearest positioning posture in the vehicle positioning trajectory, and the visual perception data is transferred to the global coordinate system through the positioning posture, thereby obtaining the visual perception data under the global positioning posture.
[0054] S203: Based on the type and the distance, the visual perception data including the global positioning posture is registered with the local map to determine the diagnosis result of the visual perception data.
[0055] In this embodiment, when the visual perception data in the global coordinate system is verified with the local map, it can be determined whether the visual perception element contained in the frame of visual perception data appears for the first time. If it appears for the first time, it is used as a temporary element. If the included visual perception element does not appear for the first time, a visual perception position check is performed, and the visual perception elements of the same category are checked with the corresponding elements in the local map, and the distance between the two is calculated and compared with the threshold, so as to determine whether the visual perception element is normal data or abnormal data.
[0056] As a specific implementation method, the registration of visual perception data including global positioning pose and local mapping can be optimized based on type and distance to determine the diagnostic results of visual perception data, including:
[0057] a1) extracting various visual perception elements contained in the visual perception data.
[0058] Specifically, various visual perception elements contained in the visual perception data are extracted, such as lane lines, ground signs, poles, traffic lights, etc.
[0059] b1) For each visual perception element, determine whether the visual perception element appears for the first time.
[0060] For each visual perception element, steps b1) to f1) need to be performed. This step is used to determine whether the visual perception element appears for the first time.
[0061] c1) If the visual perception element appears for the first time, the visual perception element is used as temporary element data.
[0062] It should be clear that since perception is not very stable, not every object can be detected every time. Sometimes it can be detected, sometimes it cannot be detected, and sometimes it is detected incorrectly. Therefore, it is necessary to determine whether it is the first time it appears, because it is a continuous process, and what you see this time may have been seen last time. Therefore, it is necessary to determine whether it is the first time it appears. Specifically, if the visual perception element appears for the first time, the visual perception element is used as temporary element data. For temporary element data, since the temporary element data is not included in the local map, it cannot be determined whether it is normal or abnormal.
[0063] d1) If the visual perception element does not appear for the first time, a corresponding map element that is close to the global positioning pose and of the same type as the visual perception element is determined from the local map, and the distance between the visual perception element and the corresponding map element is determined.
[0064] Specifically, if the visual perception element does not appear for the first time, the corresponding map element with a similar position to the global positioning and the same type of visual perception element is determined from the local map, and the distance between the visual perception element and the corresponding map element is calculated. This step is equivalent to visual perception position verification, and the visual perception elements of the same category are verified with the corresponding elements in the local map, and the distance between the two is compared.
[0065] e1) If the distance is greater than a set distance threshold, the visual perception element is determined to be abnormal isolation data.
[0066] The set distance threshold can be set according to an empirical value to distinguish whether the visual perception element is recognized normally. Specifically, if the position distance between the visual perception element and the corresponding map element is greater than the set distance threshold, it is considered that the position difference between the visual perception element and the corresponding map element is large, and the visual perception element is determined to be abnormal isolation data.
[0067] f1) If the distance is less than or equal to the set distance threshold, the visually perceived element is determined as the locked element data.
[0068] Specifically, if the distance between the visual perception element and the corresponding map element is less than or equal to the set distance threshold, the visual perception element is considered to match the corresponding map element, and the visual perception element is determined as trustworthy data and recorded as locked element data.
[0069] In this embodiment, both the temporarily stored element data and the locked element data are pushed into the current frame data of the local mapping, and the temporarily stored element data and the locked element data will be further diagnosed later.
[0070] S204: Obtain locked element data and temporarily stored element data in the current frame data of the local mapping as element data to be matched, match each element data to be matched with the high-precision map data by type and distance, and obtain an initial matching result.
[0071] In this embodiment, the distance and type matching conformity check is performed between the current frame data of the local mapping and the high-precision map data, the locked element data and the temporarily stored element data in the current frame data of the local mapping are obtained as the element data to be matched, and the element data to be matched is preliminarily screened out based on the high-precision map data to determine whether it is normal data or abnormal data, and the result is recorded as the initial matching result.
[0072] As a specific implementation method, the type and distance of each element data to be matched are matched with the high-precision map data to obtain an initial matching result, including:
[0073] a2) Determine the corresponding map elements that are close to the global positioning posture from the high-precision map data, and match the type and distance of the element data to be matched with the corresponding map elements.
[0074] Specifically, the corresponding map elements close to the global positioning position are determined from the high-precision map data, and the element data to be matched is matched with the corresponding map elements in terms of type and distance.
[0075] b2) If the type of the element data to be matched is different from that of the corresponding map element or the distance is greater than a set distance threshold, it is determined that the element data to be matched does not meet the set matching condition as an initial matching result.
[0076] Specifically, if the type of the element data to be matched is different from that of the corresponding map element or the distance is greater than the set distance threshold, it is determined that the element data to be matched does not meet the set matching condition, and the failure to meet the set matching condition is taken as the initial matching result.
[0077] c2) If the type of the element data to be matched is the same as that of the corresponding map element and the distance is less than or equal to the set distance threshold, the element data to be matched that meets the set matching condition is taken as the initial matching result.
[0078] Specifically, if the to-be-matched element data is of the same type as the corresponding map element and the distance is less than or equal to the set distance threshold, it is determined that the to-be-matched element data meets the set matching condition, and the satisfaction of the set matching condition is taken as the initial matching result.
[0079] The above technical solution specifies the initial matching steps and performs a preliminary screening of the temporarily stored element data and the locked element data.
[0080] S205: If the initial matching result does not satisfy the set matching condition, determine that the element data to be matched is abnormal isolation data.
[0081] In this embodiment, if the initial matching result does not meet the set matching condition, the element data to be matched is determined to be abnormal isolated data, and the abnormal isolated data is not used for subsequent positioning fusion processing.
[0082] S206: If the initial matching result satisfies the set matching conditions, integrity check is performed on the element data to be matched and the high-precision map data to determine the diagnosis results of the current frame data of the local mapping and the high-precision map data.
[0083] Specifically, if the initial matching result meets the set matching conditions, the integrity of the element data to be matched and the high-precision map data that meet the set matching conditions will be further checked to determine whether the current frame data of the local mapping and the high-precision map data are consistent, and the integrity check result will be recorded as the diagnostic result of the current frame data of the local mapping and the high-precision map data.
[0084] As a specific implementation method, the integrity check of the matching element data and the high-precision map data can be optimized to determine the diagnosis results of the current frame data of the local map and the high-precision map data, including:
[0085] a3) Obtaining the to-be-matched attribute information of the to-be-matched element data that meets the set matching conditions, and the map attribute information of the corresponding map elements in the high-precision map data.
[0086] In this embodiment, the attribute information of the element data to be matched that meets the set matching conditions is recorded as the attribute information to be matched. Exemplarily, the attribute information may be information such as type, quantity, length, etc. The element data to be matched may be lane lines, traffic signs, poles, ground signs, speed limits or crosswalks, etc., which can be considered as types of element data to be matched. Exemplarily, assuming that the element data to be matched is a lane line, when comparing the lane line attributes, the left lane line, the left-left lane line, the right lane line, and the right-right lane line can be compared to perform integrity verification with the length, type, and quantity of the lane lines on the high-precision map.
[0087] b3) Matching the attribute information to be matched with the map attribute information to determine the matching degree between the element data to be matched and the corresponding map element.
[0088] Specifically, the attribute information to be matched is matched with the map attribute information to obtain the matching degree between the element data to be matched and the corresponding map element.
[0089] c3) If the matching degree is less than the set matching degree threshold, it is determined that the to-be-matched element data does not match the corresponding map element.
[0090] The matching degree threshold is set according to historical experience. Specifically, if the matching degree is less than the set matching degree threshold, it is considered that the matching degree is low, and it is determined that the element data to be matched does not match the corresponding map element, and the low matching degree count is increased by 1.
[0091] d3) If the element data to be matched for a set number of consecutive frames are inconsistent with the high-precision map data, it is determined that the current frame data of the local mapping is inconsistent with the high-precision map data, and the current frame data of the local mapping and the high-precision map data are used as abnormal isolation data.
[0092] Specifically, if the element data to be matched for a set number of consecutive frames are inconsistent with the high-precision map data, which is equivalent to the cumulative value of the low matching count being greater than the threshold, the map and perception data matching is judged to be abnormal, that is, the current frame data of the local mapping and the high-precision map data are used as abnormal isolation data.
[0093] e3) Otherwise, determine that the visual perception data is consistent with the high-precision map data, and use the current frame data of the local mapping and the high-precision map data as normal input data.
[0094] Specifically, if the visual perception data is consistent with the high-precision map data, the local map current frame data and the high-precision map data are used as normal input data. In addition, if the current frame matches normally and several consecutive frames match normally, the count value is zero.
[0095] S207. Based on the chi-square test, verify the consistency of the global navigation satellite system data and the vehicle positioning trajectory.
[0096] Among them, the global navigation satellite system will have an estimate of the vehicle's positioning, and the vehicle is within a certain error range, which is recorded as the global navigation satellite system data. At the same time, the driving positioning trajectory will also have an estimate of the vehicle's positioning, and the vehicle is within a certain error range. This step is used to verify the consistency of the global navigation satellite system data and the driving positioning trajectory through the chi-square test.
[0097] S208. If the chi-square test condition is met, it is determined that the global navigation satellite system data is normal input data.
[0098] Specifically, if the chi-square test condition is met, the GNSS data is considered to be normal input data.
[0099] S209: If the chi-square test condition is not met, the global navigation satellite system data is determined to be abnormal isolation data.
[0100] Specifically, if the chi-square test condition is not satisfied, the global navigation satellite system data is determined to be abnormal isolation data.
[0101] The above technical solution introduces local mapping. Local mapping is a past result. The visual perception data of the current frame is the current result. By comparing the past and the present, it not only pays attention to the current frame, but also draws on the past information to make a comprehensive judgment. That is, by constructing a local mapping mechanism, the context comparison verification of the perception results can be realized, and the preliminary self-check of the perception results can be realized. In addition, the access to high-precision map data can further check the integrity of the perception results and the preliminary data accuracy, and at the same time verify the consistency of the map and the perception results, enhance the accuracy of the local mapping data, and increase the success rate of the self-verification of the local mapping perception results mentioned above. Furthermore, the verification of global navigation satellite data, in addition to the determination of its own state bit, adds and fuses the chi-square test of the posture to exclude abnormal global navigation satellite data. The multi-sensor data quality detection and diagnosis of the vehicle positioning function is realized, and the accuracy of the diagnosis is improved.
[0102] As an optional embodiment of the embodiment of the present invention, based on the above embodiment, it can be optimized that before determining the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory, the method further includes:
[0103] a4) Get the current status of the global navigation satellite system.
[0104] In this embodiment, before performing quality diagnosis on the GNSS data, it is necessary to first determine whether the GNSS is normal. This step is used to obtain the current status of the GNSS.
[0105] b4) If the current state is the locked state, the step of determining the diagnosis result of the global navigation satellite system data is performed.
[0106] Specifically, if the current state of the global navigation satellite system is a locked state, it means that the receiver of the global navigation satellite system can stably and continuously receive the signal transmitted by the satellite and effectively lock the signal, then enter the verification mechanism and execute the steps of determining the diagnostic result of the global navigation satellite system data.
[0107] The above technical solution adds a step of determining whether to execute the diagnosis result of the global navigation satellite system data.
[0108] Embodiment 3
[0109] Figure 3 The present invention provides a schematic diagram of the structure of a vehicle positioning multi-sensor data diagnosis device provided in the third embodiment of the present invention. The device can be used to detect and diagnose the quality of vehicle positioning multi-sensor data. The vehicle positioning multi-sensor data diagnosis device can be configured in an electronic device, such as Figure 3 As shown, the device includes: a data acquisition module 31, a first diagnosis module 32, a second diagnosis module 33 and a third diagnosis module 34; wherein,
[0110] A data acquisition module 31 is used to acquire visual perception data to be diagnosed, high-precision map data, and global navigation satellite system data;
[0111] A first diagnostic module 32 is used to determine a diagnostic result of the visual perception data based on the visual perception data and a predetermined vehicle positioning trajectory in combination with a local mapping mechanism. The diagnostic result of the visual perception data includes abnormal isolation data, temporarily stored element data, and locked element data. The temporarily stored element data and the locked element data are used as current frame data for local mapping.
[0112] The second diagnosis module 33 is used to perform mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data;
[0113] The third diagnosis module 34 is used to determine the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory.
[0114] The above technical solution introduces the local mapping function to construct a local perception map to eliminate abnormal visual perception data; combines the local mapping data to judge the high-precision map data that is not updated in time, and performs mutual verification between the current frame data of the local mapping and the high-precision map data; detects the GNSS data quality through the predetermined driving positioning trajectory combined with the GNSS status, thereby realizing the multi-sensor data quality diagnosis function.
[0115] Optionally, the first diagnostic module 32 includes:
[0116] A posture determination unit is used to determine, according to the timestamp of the visual perception data, a positioning posture corresponding to the timestamp from the vehicle positioning trajectory as the global positioning posture of the visual perception data in the global coordinate system;
[0117] The data registration unit is used to register the visual perception data including the global positioning posture with the local map based on the type and distance, so as to determine the diagnosis result of the visual perception data.
[0118] Optionally, the data registration unit is specifically used to:
[0119] Extracting various visual perception elements contained in the visual perception data;
[0120] For each visual perception element, determine whether the visual perception element appears for the first time;
[0121] If the visual perception element appears for the first time, the visual perception element is used as temporary element data;
[0122] If the visual perception element does not appear for the first time, a corresponding map element that is close to the global positioning pose and of the same type as the visual perception element is determined from the local map, and the distance between the visual perception element and the corresponding map element is determined;
[0123] If the distance is greater than the set distance threshold, the visual perception element is determined to be abnormal isolation data;
[0124] If the distance is less than or equal to the set distance threshold, the visually perceived element is determined as the locked element data.
[0125] Optionally, the device includes a trajectory determination module, specifically configured to:
[0126] The vehicle positioning trajectory at the current moment is determined based on the previous vehicle positioning trajectory obtained by multi-sensor fusion positioning at the previous moment and the inertial measurement data at the current moment.
[0127] Optionally, the second diagnostic module 33 includes:
[0128] The initial matching determination unit is used to obtain the locked element data and the temporarily stored element data in the current frame data of the local mapping as the element data to be matched, and match the type and distance of each element data to be matched with the high-precision map data to obtain the initial matching result;
[0129] An abnormal data determination unit, configured to determine that the element data to be matched is abnormal isolation data if the initial matching result does not satisfy the set matching condition;
[0130] The integrity check unit is used to perform integrity check on the element data to be matched and the high-precision map data if the initial matching result meets the set matching conditions, and determine the diagnosis results of the current frame data of the local mapping and the high-precision map data.
[0131] Optionally, the initial matching determination unit is specifically configured to:
[0132] Determine the corresponding map elements that are close to the global positioning posture from the high-precision map data, and match the type and distance of the element data to be matched with the corresponding map elements;
[0133] If the type of the element data to be matched is different from that of the corresponding map element or the distance is greater than the set distance threshold, it is determined that the element data to be matched does not meet the set matching condition as the initial matching result;
[0134] If the type of the element data to be matched is the same as that of the corresponding map element and the distance is less than or equal to the set distance threshold, the element data to be matched that meets the set matching condition is taken as the initial matching result.
[0135] Optionally, the integrity checking unit is specifically configured to:
[0136] Obtaining the to-be-matched attribute information of the to-be-matched element data that meets the set matching conditions, as well as the map attribute information of the corresponding map elements in the high-precision map data;
[0137] Matching the to-be-matched attribute information with the map attribute information to determine the matching degree between the to-be-matched element data and the corresponding map element;
[0138] If the matching degree is less than the set matching degree threshold, it is determined that the element data to be matched does not match the corresponding map element;
[0139] If the element data to be matched for a set number of consecutive frames are inconsistent with the high-precision map data, it is determined that the current frame data of the local mapping is inconsistent with the high-precision map data, and the current frame data of the local mapping and the high-precision map data are used as abnormal isolation data;
[0140] Otherwise, it is determined that the visual perception data is consistent with the high-precision map data, and the current frame data of the local mapping and the high-precision map data are used as normal input data.
[0141] Optionally, the third diagnostic module 34 is specifically configured to:
[0142] Based on the chi-square test, the consistency of the GNSS data and the vehicle positioning trajectory was verified;
[0143] If the chi-square test conditions are met, the GNSS data are determined to be normal input data;
[0144] If the chi-square test conditions are not met, the GNSS data are determined to be abnormal isolation data.
[0145] Optionally, the device further comprises a system status determination module, which is used to:
[0146] Get the current status of the global navigation satellite system;
[0147] If the current state is the locked state, the step of determining the diagnosis result of the global navigation satellite system data is performed.
[0148] The vehicle positioning multi-sensor data diagnosis device provided in the embodiment of the present invention can execute the vehicle positioning multi-sensor data diagnosis method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0149] Embodiment 4
[0150] Figure 4 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0151] like Figure 4As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.
[0152] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0153] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 41 executes the various methods and processes described above, such as the vehicle positioning multi-sensor data diagnosis method.
[0154] In some embodiments, the driving positioning multi-sensor data diagnostic method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the driving positioning multi-sensor data diagnostic method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the driving positioning multi-sensor data diagnostic method in any other appropriate manner (for example, by means of firmware).
[0155] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0156] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0157] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0158] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0159] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0160] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0161] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle positioning multi-sensor data diagnosis method provided by any embodiment of the present invention.
[0162] In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code for the operation of the present disclosure, including but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0164] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A vehicle positioning multi-sensor data diagnosis method, characterized in that: include: Obtain visual perception data, high-precision map data, and global navigation satellite system data to be diagnosed; Determine the diagnosis result of the visual perception data according to the visual perception data and the predetermined vehicle positioning trajectory in combination with the local mapping mechanism, wherein the diagnosis result of the visual perception data includes abnormal isolation data, temporary element data and locked element data, and the temporary element data and the locked element data are used as the current frame data of the local mapping; Performing mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data; A diagnosis result of the global navigation satellite system data is determined according to the global navigation satellite system data and the vehicle positioning trajectory.
2. The method according to claim 1, characterized in that Determining the diagnosis result of the visual perception data based on the visual perception data and the predetermined vehicle positioning trajectory in combination with a local mapping mechanism includes: According to the timestamp of the visual perception data, determining the positioning posture corresponding to the timestamp from the driving positioning trajectory as the global positioning posture of the visual perception data in the global coordinate system; Based on the type and the distance, the visual perception data including the global positioning posture is registered with the local map to determine the diagnosis result of the visual perception data.
3. The method according to claim 2, characterized in that The method of registering the visual perception data including the global positioning posture with the local map based on the type and the distance to determine the diagnosis result of the visual perception data includes: Extracting each visual perception element contained in the visual perception data; For each visual perception element, determining whether the visual perception element appears for the first time; If the visually perceived element appears for the first time, the visually perceived element is used as temporary element data; If the visual perception element does not appear for the first time, determining from the local map a corresponding map element that is close to the global positioning posture and of the same type as the visual perception element, and determining a distance between the visual perception element and the corresponding map element; If the distance is greater than a set distance threshold, determining that the visual perception element is abnormal isolation data; If the distance is less than or equal to the set distance threshold, the visually perceived element is determined as the locked element data.
4. The method according to claim 1, characterized in that: The step of determining the driving positioning trajectory includes: The vehicle positioning trajectory at the current moment is determined based on the previous vehicle positioning trajectory obtained by multi-sensor fusion positioning at the previous moment and the inertial measurement data at the current moment.
5. The method according to claim 1, characterized in that The mutual verification of the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data includes: Obtaining locked element data and temporarily stored element data in the current frame data of the local mapping as element data to be matched, matching each element data to be matched with the high-precision map data by type and distance, and obtaining an initial matching result; If the initial matching result does not satisfy the set matching condition, determining that the element data to be matched is abnormal isolation data; If the initial matching result satisfies the set matching condition, an integrity check is performed on the element data to be matched and the high-precision map data to determine the diagnosis results of the current frame data of the local mapping and the high-precision map data.
6. The method according to claim 5, characterized in that The matching of the type and distance of each of the to-be-matched element data and the high-precision map data to obtain an initial matching result includes: Determine a corresponding map element close to the global positioning posture from the high-precision map data, and match the to-be-matched element data with the corresponding map element by type and distance; If the type of the to-be-matched element data is different from that of the corresponding map element or the distance is greater than a set distance threshold, determining that the to-be-matched element data does not satisfy the set matching condition as the initial matching result; If the type of the to-be-matched element data is the same as that of the corresponding map element and the distance is less than or equal to the set distance threshold, the to-be-matched element data satisfying the set matching condition is taken as the initial matching result.
7. The method according to claim 5, characterized in that The performing integrity check on the element data to be matched and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data includes: Acquire the to-be-matched attribute information of the to-be-matched element data that meets the set matching conditions, and the map attribute information of the corresponding map element in the high-precision map data; Matching the to-be-matched attribute information with the map attribute information to determine the matching degree between the to-be-matched element data and the corresponding map element; If the matching degree is less than a set matching degree threshold, it is determined that the to-be-matched element data does not match the corresponding map element; If the element data to be matched for a set number of consecutive frames are inconsistent with the high-precision map data, it is determined that the local mapping current frame data is inconsistent with the high-precision map data, and the local mapping current frame data and the high-precision map data are used as abnormal isolation data; Otherwise, it is determined that the visual perception data is consistent with the high-precision map data, and the local mapping current frame data and the high-precision map data are used as normal input data.
8. The method according to claim 1, characterized in that Determining the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory includes: Verifying the consistency of the GNSS data and the vehicle positioning trajectory based on a chi-square test; If the chi-square test condition is met, it is determined that the global navigation satellite system data is normal input data; If the chi-square test condition is not satisfied, the global navigation satellite system data is determined to be abnormal isolation data.
9. The method according to claim 1, characterized in that: Before determining the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory, the method further includes: Obtaining a current status of the global navigation satellite system; If the current state is a locked state, a step of determining a diagnosis result of the global navigation satellite system data is performed.
10. A vehicle positioning multi-sensor data diagnostic device, characterized in that: include: A data acquisition module, used to acquire visual perception data to be diagnosed, high-precision map data, and global navigation satellite system data; A first diagnostic module is used to determine a diagnostic result of the visual perception data according to the visual perception data and a predetermined vehicle positioning trajectory in combination with a local mapping mechanism, wherein the diagnostic result of the visual perception data includes abnormal isolation data, temporarily stored element data, and locked element data, and the temporarily stored element data and the locked element data are used as current frame data for local mapping; A second diagnosis module is used to perform mutual verification based on the local mapping current frame data and the high-precision map data to determine the diagnosis results of the local mapping current frame data and the high-precision map data; The third diagnosis module is used to determine the diagnosis result of the global navigation satellite system data according to the global navigation satellite system data and the vehicle positioning trajectory.
11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle positioning multi-sensor data diagnosis method according to any one of claims 1-9.
12. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the vehicle positioning multi-sensor data diagnosis method as described in any one of claims 1-9 when executed by a computer processor.
