Method and system for identifying, labeling, and correcting unexpected traffic sign data

Abstract

Methods and systems for identifying, labeling, and correcting for unexpected traffic sign data. Systems and methods for identifying, labeling, and correcting for unexpected traffic sign data include capturing traffic sign data with a plurality of vehicles, where each vehicle includes a front sensor to capture traffic sign data, and then collecting and storing the traffic sign data captured from the plurality of vehicles over a period of time. One or more potential false traffic sign detections are identified from the captured traffic sign data based on road categories and associated rules, including identification and labeling. An average vehicle speed during a non-peak traffic period for one or more traffic zones is determined, and then filtered based on traffic sign categories to determine a most likely traffic sign legend based on the determination of the average vehicle speed during the non-peak traffic period.

Description

[0001] introduction

[0002] Vehicles are rapidly integrating an ever-increasing number of technological components into their systems. Special-purpose microcontrollers, technologies, and sensors can be used in many different applications within vehicles. Automotive microcontrollers and sensors can be used to enhance automated architectures that provide customers with state-of-the-art experiences and services, for tasks such as body control, camera vision, information display, safety, and autonomous control. Furthermore, functions such as adaptive cruise control, lane change assist, and vehicle proximity detection can utilize a variety of sensors that employ cameras, light detection and ranging (LIDAR), radio detection and ranging (RADAR), ultrasound, and other technologies to achieve their functionality.

[0003] However, with the widespread use of such automated controls, such as optical recognition of traffic signs, the possibility of false detections is constantly increasing. Therefore, the ability to mitigate and correct false detections is crucial in situations where optical recognition may affect vehicle control (e.g., adaptive cruise control). Summary of the Invention

[0004] This document discloses systems and methods for identifying, marking, and correcting accidental traffic sign data. As disclosed herein, traffic sign data can be captured and analyzed using a large number of vehicles (e.g., using crowdsourcing algorithms). Such data can be captured during different parts of the day, such as when traffic is congested, and also during periods of minimal congestion where vehicles may travel at higher speeds. However, due to environmental conditions or optical aberrations, cameras in vehicles may misinterpret traffic signs, for example, by displaying a traffic speed sign as "25" instead of its actual value "45".

[0005] Therefore, a system for identifying, marking, and correcting erroneous traffic sign data may include multiple vehicles, each equipped with a front sensor (e.g., a camera module) to capture traffic sign data. The system may include a system, such as a traffic sign data aggregation system, that can collect and store traffic sign data captured from vehicles over a period of time. The traffic sign data aggregation system can then be used to identify and mark one or more potential erroneous traffic sign detections from the captured traffic sign data based on road categories and associated rules (e.g., speed limit rules, yield sign rules, or stop sign rules). A temporal and spatial filtering system can then be used to determine the average vehicle speed during non-peak traffic periods for one or more traffic areas and then filter the captured traffic sign data based on traffic sign categories (e.g., speed limit categories, stop sign categories, or yield sign categories). The traffic sign data aggregation system can then determine the most probable traffic sign legend (e.g., speed limit value, stop sign, or yield sign) from the filtered captured traffic sign data based on the determination made by the temporal and spatial filtering system.

[0006] Another aspect of this disclosure may include a system in which the traffic sign data aggregation system is further configured to perform data curation to filter out unsuitable data, including data from selected vehicles identified as sources of error in determining traffic sign speed limit values.

[0007] Another aspect of this disclosure may include systems in which traffic sign categories include, but are not limited to, speed limit signs, stop signs, or yield signs.

[0008] Another aspect of this disclosure may include a system in which road categories include primary roads, secondary roads, and tertiary roads.

[0009] Another aspect of this disclosure may include a system in which the traffic sign data aggregation system is further configured to mark one or more erroneous traffic sign detections as high risk when the road category is not a primary level and the speed limit is greater than a threshold.

[0010] Another aspect of this disclosure may include the system further performing data clustering by grouping traffic sign data associated with a single specific traffic sign.

[0011] Another aspect of this disclosure may include a system in which traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by a crowdsourcing algorithm.

[0012] Another aspect of this disclosure may include a system in which a temporal and spatial filtering system applies a distributed approach to process crowdsourced telemetry data, including estimated confidence scores.

[0013] Another aspect of this disclosure may include a system in which the distribution method further includes estimating confidence scores based on determining the maximum peak and qualified peak from crowdsourced telemetry data.

[0014] Another aspect of this disclosure may include a system in which a temporal and spatial filtering system performs a deduplication process based on the determined most likely traffic sign legend.

[0015] Another aspect of this disclosure may include a system in which a temporal and spatial filtering system is further configured to filter data based on a velocity limit category by filtering out velocity values ​​less than a threshold.

[0016] Another aspect of this disclosure may include a method for identifying, marking, and correcting erroneous traffic sign data, comprising: capturing traffic sign data using multiple vehicles, each vehicle including a front sensor to capture traffic sign data. The method may continue by: collecting and storing traffic sign data captured from the multiple vehicles over a period of time, and identifying and marking one or more potential erroneous traffic sign detections from the captured traffic sign data based on road categories and associated rules. The method may include determining the average vehicle speed during non-maximum traffic periods for one or more traffic areas, filtering the captured traffic sign data based on traffic sign categories, and determining the most probable traffic sign legend from the filtered captured traffic sign data based on the determination of the average vehicle speed during non-maximum traffic periods.

[0017] Another aspect of this disclosure may include a method that performs data management to filter out unsuitable traffic sign data, including data from selected vehicles identified as sources of error in determining traffic sign speed limit values.

[0018] Another aspect of this disclosure may include traffic sign categories including speed limit signs, stop signs, or yield signs.

[0019] Another aspect of this disclosure may include the method marking one or more erroneous traffic sign detections as high risk when the road category is not a primary level and the speed limit is greater than a threshold.

[0020] Another aspect of this disclosure may include a method that performs data clustering by grouping traffic sign data associated with a single specific traffic sign.

[0021] Another aspect of this disclosure may include traffic sign data captured from multiple vehicles over a period of time, including data collected and analyzed by crowdsourcing algorithms.

[0022] Another aspect of this disclosure may include a method in which a distributed approach is applied to process crowdsourced telemetry data, including estimating confidence scores based on determining the maximum peak and a set of qualifying peaks.

[0023] Another aspect of this disclosure may include a method that performs a deduplication process based on the determined most likely traffic sign legend.

[0024] Another aspect of this disclosure may include a method for identifying, marking, and correcting erroneous traffic sign data, comprising: capturing traffic sign data using multiple vehicles, each vehicle including a front sensor configured to capture traffic sign data. The method may further include collecting traffic sign data captured from the multiple vehicles over a period of time, and identifying and marking one or more potential erroneous traffic sign detections from the captured traffic sign data based on road categories and associated rules. The method may further determine the average vehicle speed during non-maximum traffic periods for one or more traffic areas, and filter the captured traffic sign data based on traffic sign categories, including speed limit signs, stop signs, or yield signs. The method may further determine the most probable traffic sign legends from the filtered captured traffic sign data based on the determination of the average vehicle speed during non-maximum traffic periods, while also performing data management to filter unsuitable traffic sign data, including data from selected vehicles identified as sources of error in determining traffic sign speed limit legends, and marking one or more erroneous traffic sign detections as high-risk when the road category is not a primary level and the speed limit is greater than a threshold. The method may also include performing data clustering by grouping traffic sign data associated with a single specific traffic sign, and applying distributed methods to process crowdsourced telemetry data, including estimating confidence scores based on determining the maximum peak and a set of qualified peaks, and performing a deduplication process based on the determined most probable traffic sign legend, wherein road categories include primary roads, secondary roads and tertiary roads, and wherein traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by a crowdsourcing algorithm.

[0025] This application provides the following technical solution:

[0026] 1. A system for identifying, marking, and correcting accidental traffic sign data, comprising:

[0027] Multiple vehicles, each equipped with a front sensor configured to capture traffic sign data;

[0028] A traffic sign data aggregation system, configured to collect and store traffic sign data captured from multiple vehicles over a period of time;

[0029] The traffic sign data aggregation system is further configured to identify and flag one or more potential erroneous traffic sign detections from the captured traffic sign data based on road category and associated rules;

[0030] A time and space filtering system, configured to determine the average vehicle speed during non-peak traffic periods for one or more traffic zones;

[0031] The temporal and spatial filtering system is further configured to filter the captured traffic sign data based on traffic sign category;

[0032] The traffic sign data aggregation system is further configured to determine the most likely traffic sign legend from the filtered captured traffic sign data based on a time and space filtering system.

[0033] 2. The system according to technical solution 1, wherein the traffic sign data aggregation system is further configured to perform data management to filter unsuitable data, including data from vehicles selected as sources of error when determining traffic sign speed limit values.

[0034] 3. The system according to technical solution 1, wherein the traffic sign categories include speed limit signs, stop signs, or yield signs.

[0035] 4. The system according to technical solution 1, wherein the road categories include main roads, secondary roads and tertiary roads.

[0036] 5. The system according to technical solution 1, wherein the traffic sign data aggregation system is further configured to mark one or more erroneous traffic sign detections as high risk when the road category is not a major level and the speed limit is greater than a threshold.

[0037] 6. The system according to technical solution 1, including a traffic sign data aggregation system, is further configured to perform data clustering by grouping traffic sign data associated with a single specific traffic sign.

[0038] 7. The system according to technical solution 1, wherein the traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by a crowdsourcing algorithm.

[0039] 8. The system according to technical solution 1, wherein the temporal and spatial filtering system is further configured to apply a distributed method to process crowdsourced telemetry data including estimated confidence scores.

[0040] 9. The system according to technical solution 8, wherein the distribution method further includes estimating the confidence score based on determining the maximum peak and qualified peak from crowdsourced telemetry data.

[0041] 10. The system according to technical solution 1, wherein the time and space filtering system is further configured to perform a deduplication process based on the determined most probable traffic sign legend.

[0042] 11. The system according to technical solution 1, wherein the time and space filtering system is further configured to filter data based on speed limit categories by filtering out speed values ​​less than a threshold.

[0043] 12. A method for identifying, marking, and correcting accidental traffic sign data, comprising:

[0044] Traffic sign data is captured using multiple vehicles, each of which includes a front sensor configured to capture traffic sign data.

[0045] Collect and store traffic sign data captured from multiple vehicles over a period of time;

[0046] Identify and mark one or more potential erroneous traffic signs from the captured traffic sign data based on road category and associated rules;

[0047] Determine the average vehicle speed during non-peak traffic periods for one or more traffic zones;

[0048] Traffic sign data captured based on traffic sign category filtering; and

[0049] Based on the determination of average vehicle speed during non-peak traffic periods, the most likely traffic sign legends are identified from the filtered captured traffic sign data.

[0050] 13. The method according to technical solution 12 further includes performing data management to filter unsuitable traffic sign data, including data from selected vehicles identified as sources of error in determining traffic sign speed limit values.

[0051] 14. The method according to technical solution 12, wherein the traffic sign categories include speed limit signs, stop signs, or yield signs.

[0052] 15. The method according to technical solution 12 further includes marking one or more erroneous traffic sign detections as high risk when the road category is not a major level and the speed limit is greater than a threshold.

[0053] 16. The method according to technical solution 12 further includes performing data clustering by grouping traffic sign data associated with a single specific traffic sign.

[0054] 17. The method according to technical solution 12, wherein the traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by a crowdsourcing algorithm.

[0055] 18. The method according to technical solution 12 further includes applying a distributed approach to process crowdsourced telemetry data, including estimating confidence scores based on determining the maximum peak value and a set of qualified peak values.

[0056] 19. The method according to technical solution 12 further includes performing a deduplication process based on the determined most probable traffic sign legend.

[0057] 20. A method for identifying, marking, and correcting accidental traffic sign data, comprising:

[0058] Traffic sign data is captured using multiple vehicles, each of which includes a front sensor configured to capture traffic sign data.

[0059] Collect traffic sign data captured from multiple vehicles over a period of time;

[0060] Identify and mark one or more potential erroneous traffic signs from the captured traffic sign data based on road category and associated rules;

[0061] Determine the average vehicle speed during non-peak traffic periods for one or more traffic zones;

[0062] The captured traffic sign data is filtered based on traffic sign categories, which include speed limit signs, stop signs, or yield signs.

[0063] Based on the determination of average vehicle speed during non-peak traffic periods, the most likely traffic sign legends are determined from the filtered captured traffic sign data;

[0064] Perform data management to filter out unsuitable traffic sign data, including data from vehicles selected as sources of errors when determining traffic sign speed limit legends;

[0065] When the road category is not a primary level and the speed limit is greater than the threshold, mark one or more erroneous traffic sign detections as high risk;

[0066] Data clustering is performed by grouping traffic sign data that are associated with a single, specific traffic sign.

[0067] Applying distributed methods to process crowdsourced telemetry data includes estimating confidence scores based on determining the maximum peak and a set of qualified peaks; and

[0068] The deduplication process is performed based on the most likely traffic sign legend.

[0069] The road categories include main roads, secondary roads, and tertiary roads; and

[0070] The traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by crowdsourcing algorithms.

[0071] The foregoing features and advantages, as well as other features and accompanying advantages, of this disclosure will become readily apparent from the following detailed description of illustrative examples and models used to carry out this disclosure, when taken in conjunction with the accompanying drawings and the appended claims. Furthermore, this disclosure explicitly includes combinations and sub-combinations of the elements and features presented above and below. Attached Figure Description

[0072] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate implementations of this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0073] Figure 1 This is an illustration of the detection of incorrect traffic signs on secondary roads according to this disclosure.

[0074] Figure 2 This is a flowchart of data for identifying, marking, and correcting accidental traffic signs, based on this disclosure.

[0075] Figure 3 It is a diagram illustrating the marking conditions for classifying risks associated with accidental traffic sign data according to this disclosure.

[0076] Figure 4 This is an illustration of an in-vehicle inspection for detecting potential errors in traffic signs according to this disclosure.

[0077] Figure 5 This is a diagram illustrating the use of confidence scores in crowdsourced traffic sign data processing according to this disclosure.

[0078] Figure 6 This is a diagram illustrating the processing of crowdsourced traffic sign data according to this disclosure.

[0079] Figure 7 This is a diagram illustrating the confidence score of the detected speed limit estimated based on the distribution of crowdsourced vehicle speed data according to this disclosure.

[0080] Figure 8 This is a diagram illustrating the distribution of crowdsourced vehicle speed data used in the deduplication process of crowdsourced traffic sign data according to this disclosure.

[0081] Figure 9A , 9B Figures 9C and 9C illustrate three examples of the use of raw telemetry data from high-speed vehicles in the deduplication process according to this disclosure.

[0082] Figure 10 A flowchart is depicted for a method of identifying, marking, and correcting accidental traffic sign data according to this disclosure.

[0083] The accompanying drawings are not necessarily to scale and may present slightly simplified representations of various preferred features of the present disclosure as disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes. Details associated with such features will be determined in part by the specific intended application and environment of use. Detailed Implementation

[0084] This disclosure allows for numerous different embodiments. Representative examples of this disclosure are shown in the accompanying drawings and are described in detail herein as non-limiting examples of the disclosed principles. Accordingly, elements and limitations described in the abstract, introduction, title, and detailed description but not expressly set forth in the claims should not be incorporated into the claims, individually or collectively, by implication, inference, or otherwise.

[0085] For the purposes of this specification, unless otherwise stated, the use of the singular includes the plural, and vice versa; the terms “and” and “or” shall be both conjunctions and disjunctive words; and the words “including,” “contains,” “includes,” “has,” “has,” and the like shall mean “including, but not limited to.” Furthermore, approximate terms such as “about,” “almost,” “substantially,” “generally,” “approximately,” etc., may be used herein in the sense of “within, close to, or nearly within,” or “within 0-5% of,” or “within acceptable manufacturing tolerances,” or logical combinations thereof. As used herein, a component “configured” to perform the specified function is capable of performing the specified function without alteration, and not merely has the potential to perform the specified function after further modification. In other words, the described hardware, when explicitly configured to perform the specified function, is specifically selected, created, implemented, utilized, programmed, and / or designed for the purpose of performing the specified function.

[0086] Referring to the accompanying drawings, the leftmost numeral of the reference numerals identifies the drawing in which the reference numeral first appears (e.g., reference numeral "310" indicates that the element so numbered is first labeled or first appears). Figure 3(in Chinese). In addition, elements having the same reference numerals followed by different letters of the alphabet or other distinguishing marks (e.g., apostrophes) indicate elements that may be identical in structure, operation, or form but can be identified as repeating elements in different locations in space or at different points in time (e.g., reference numerals "110a" and "110b" can indicate two different input devices that may be functionally identical but located at different points in the simulation arena).

[0087] Autonomous vehicles and advanced driver assistance systems (AV / ADAS), such as adaptive cruise control, traffic sign recognition, automated parking, automatic brake-holding, automatic braking, evasive steering assist, lane-keeping assist, adaptive headlights, reversing assist, blind spot detection, intersection traffic alert, local hazard warning, and rear automatic braking, can rely on information obtained from cameras and sensors on the vehicle. As these types of features become increasingly prevalent in vehicles, the sensors enabling these features are susceptible to misinterpretation; for example, weather conditions can blur camera sensors and lead to false detections. Crowdsourcing can also be used to improve confidence in detections, such as traffic signs. These sensors can be combined with data from multiple vehicles, rather than relying solely on the vehicle's sensors.

[0088] Figure 1 This is an illustration of a scenario 100 for detecting erroneous traffic signs on a secondary road according to an embodiment of this disclosure. Figure 1 In this scenario, the primary vehicle 110 is shown traveling on secondary road 120 and may be equipped with sensors 115, such as a front-facing camera module, which can be used for traffic sign recognition. Traffic sign recognition may include recognizing speed limit signs, such as traffic sign 125, but may also include other types of signs, such as stop signs, yield signs, intersection signs, one-way traffic signs, route or interstate highway number signs, turn signs, etc. However, scenario 100 illustrates that the primary vehicle 110 has processed a captured image of traffic sign 125 but incorrectly detected a speed limit of 80 mph as indicated by the detected speed limit 130, instead of the actual speed limit of 45 mph. Figure 3 and Figure 4 As the lieutenant general discussed, such false detections could indicate either high or low risk. For example, an 80 mph speed limit on a winding secondary road might not be an ideal scenario because the road may not have been designed and built for such high operating speeds. In a similar scenario, if a vehicle is on a highway with an actual speed limit of 70 mph but a 20 mph speed limit is falsely detected, reducing the vehicle's speed to that extent could lead to undesirable consequences.

[0089] Figure 2This is a flowchart 200 for identifying, marking, and correcting accidental traffic sign data according to embodiments of the present disclosure. At step 210, data is aggregated; traffic sign data can be collected from multiple vehicles over a period of time. For example, data from several days or weeks can be collected, aggregated, and stored in the cloud or by a third-party provider. Crowdsourcing algorithms can be used to collect and analyze such traffic sign data.

[0090] At step 220, data management allows for data filtering, enabling the deletion and / or ignoring of data unsuitable for further processing. For example, if traffic on a highway with a speed limit of 60 mph comes to a complete stop, for example due to some type of obstacle, data indicating vehicles traveling at 0 mph does not reflect the actual indicated speed limit and is therefore unsuitable for further processing.

[0091] At step 230, data clustering can group traffic sign data for a specific traffic sign. For example, if a crowdsourcing process is used, vehicles may report a stop sign at the corner of "Main Street" and "Center Street," but the actual locations of the reported signs may exist at multiple locations along the same side of the road, for example, within + / - 10 feet. This does not necessarily mean that there are multiple stop signs at that intersection.

[0092] At step 240, the marking conditions can be used to identify and mark unexpected traffic signs and objects, for example, such as Figure 1 The error detection of the 80 mph speed limit along secondary roads discussed in the text. The marking conditions will be... Figure 3 and Figure 4 Further detailed discussion is needed.

[0093] At step 250, temporal and spatial filtering allows for the detection of free-flowing traffic from the collected vehicle telemetry data. Free-flowing traffic can be considered as unimpeded traffic flow, e.g., without traffic congestion. Such free-flowing traffic data can provide a fairly accurate indication of the actual speed limits for specific sections or areas of a road.

[0094] At step 260, raw High-Speed ​​Vehicle Telemetry (HSVT) data from the vehicle can be analyzed to determine error detection. HSVT is a byproduct of a system that allows real-time data exchange between the vehicle and the central system.

[0095] At step 270, detection and deduplication can be performed to determine inferred traffic sign legends, such as speed limit values, stop signs, or yield signs, based on analysis of the raw HSVT data. Additionally, at step 270, a deduplication process can be invoked to eliminate duplicate data for specific traffic signs, such as the duplicate data of "multiple" traffic signs at intersections discussed above.

[0096] At step 280, the traffic sign legend is determined. The most likely traffic sign value or legend can be determined based on the deduplication data of the HSVT cross-check and saved / stored in the final traffic sign database.

[0097] Figure 3 This is an illustration of a labeling condition process 300 according to an embodiment of the present disclosure. Process 300 may receive a clustering table of data as input from an external source, for example, using road category and speed limit data to classify labeling conditions for potential false detection as high-risk or low-risk. Therefore, at step 310, a determination can be made regarding whether a road is a primary road type (e.g., a highway, a segmented highway, or an interstate highway possibly designed for highway-speed traffic) using open street maps or other road classification methods. In contrast, roads can be classified as secondary or tertiary roads. While a primary road may typically be a restricted-access highway with interchanges and ramps, a secondary road may include arterial roads that may or may not be segmented, and may also include intersections. Therefore, at step 310, if a road is determined to be a primary road, and the speed limit detected at step 315 is greater than, within a certain threshold, or equal to a typical primary road speed limit, the labeling condition can be set to low-risk at step 320. Then, in cases where the speed limit may be less than a typical primary road speed and within a certain threshold, the labeling condition can be set to high-risk at step 340. However, if the road is determined to be below the major road category, a determination can be made at step 330 regarding whether the speed limit is greater than the highway speed limit (e.g., 65 mph). If the speed limit is less than the highway speed limit, a determination can be made at step 320 that the situation can again be classified as low risk. However, if the speed limit is classified as greater than or equal to the highway speed limit and the road has been identified as below the major level, a high-risk detection can be determined at step 340.

[0098] Figure 4The illustration depicts an in-vehicle detection process 400 for potential error detection according to an embodiment of the present disclosure. Process 400 may not rely on external or crowdsourced traffic sign data, but may utilize internal sensors, such as a front-facing camera module. Therefore, at step 410, a front-facing camera module or other types of image capture devices (including, but not limited to, light detection and ranging (Lidar), radar, or the like) may be used to detect the sign. While this example illustrates a speed limit traffic sign, the type of traffic sign is not limited to speed and may be other types of traffic signs. At step 420, a determination may be made regarding whether the speed of the primary vehicle is within a threshold amount of the detected speed captured by the forward-facing sensor. If the primary vehicle speed is close to and within the threshold amount of the detected speed limit, the situation may be classified as low risk at step 430. However, if the primary vehicle speed is not close to the detected speed limit, a determination may be made at step 440 regarding whether the primary vehicle speed is below the detected speed limit. If it is less than the limit, a determination may be made at step 450 regarding whether the primary vehicle is part of a road congestion with slow-moving traffic, and if so, the associated risk marker may be determined as low risk at step 430. However, if the speed of the main vehicle is less than the detected speed limit at step 440, and no slow-moving traffic is detected at step 450, the situation can be identified as high-risk at step 460. Similarly, if the speed of the main vehicle is higher than the detected speed limit, the situation can also be marked as high-risk at step 460.

[0099] Figure 5The illustration depicts a process for high-speed vehicle telemetry 500 utilizing crowdsourced data according to an embodiment of the present disclosure. At step 510, crowdsourced traffic sign data may be collected. Such data may include various types of traffic signs and may also include the associated locations of the traffic signs. The crowdsourced traffic sign data may be collected over time and pertain to one or more roads. At step 520, based on the crowdsourced data collected in step 510, a determination may be made regarding reported problematic traffic signs. At step 525, an associated confidence score may also be present associated with the reported traffic signs. At step 530, a mixture of correct detections and one or more incorrect detections associated with the reported traffic signs may also be present. The outputs of both steps 525 and 530 may be presented in step 540 as inferred traffic signs based on the high-speed vehicle telemetry data from the process in step 535. The processed high-speed vehicle telemetry data may consist of multiple filters and distribution methods for the raw high-speed vehicle telemetry data. Such filtering can include temporal and spatial filtering to select free-flowing traffic data, where average vehicle speeds can be measured during low-traffic periods and / or high-congestion areas can also be filtered out. Furthermore, traffic sign filters can eliminate certain data by category type, such as removing low values ​​for speed limits, retaining values ​​associated with stop signs, and also retaining values ​​associated with yield signs. Processing raw high-speed vehicle telemetry data can also include using distribution methods (such as histograms, density plots, and kernel density estimation) to find the range of values ​​where most data exist, obtain the minimum and maximum values ​​of the highest bins, and determine the average value of the highest bins. Examples of distribution methods will be provided in [the following section / section / etc.]. Figure 7 Further discussion is needed.

[0100] Therefore, at step 540, the results of high-speed vehicle telemetry can lead to an increase in the traffic sign confidence score in step 545, and to the ability to identify and / or correct erroneous traffic sign detections in step 550.

[0101] Figure 6 This is a further discussion of the processing 600 of crowdsourced traffic sign data according to embodiments of the present disclosure. The high-speed vehicle telemetry processing 600 may begin at step 610 with obtaining traffic sign type and location data. Furthermore, the traffic sign type and location data may then be grouped by traffic sign type (e.g., by speed, stop, yield, etc.). Additionally, the data may be grouped by edge markers, road signs, or designations. Then, at step 620, the data can be managed by filtering out low-speed data based on the traffic sign type, for example, removing zero-speed data from traffic signs of the speed limit type.

[0102] At step 630, temporal and spatial filtering can be used to select free-flowing traffic data. For example, average vehicle speed can be measured during low-flow periods to filter our high-congestion periods. Furthermore, high-congestion periods can be defined by times when traffic in one or more directions and segments or portions of a road typically operates at speeds below free-flowing speeds, such as between 06:00 and 10:00, and between 15:00 and 18:00. These times are arbitrary and not restrictive.

[0103] Next, spatial filtering can be used to select the most suitable set of telemetry data samples for analysis. For example, when processing highway telemetry data, samples near exit / merging ramps can be filtered out. Similarly, for urban / residential roads, samples near intersections can be filtered out. This can be done by using OSM (or other) road topology information (such as vertices) to identify the locations of road intersections and filtering out samples using a distance threshold. Another approach could include the ability to select vehicle telemetry samples from within a distance threshold of detected traffic signs. Furthermore, behavior near posted traffic signs can be observed to further quantify the effectiveness of traffic signs and further confirm the legends associated with them.

[0104] At step 640, a distribution method can be applied to the data, and may include histograms, density plots, kernel density estimation, and similar uses. Such a method may include finding the range of values ​​that contains the majority of the data, such as a pattern in a histogram. The minimum and maximum values ​​of the highest bins can then be determined, the mean of the highest bins can be obtained, and the confidence scores can be estimated.

[0105] In step 650, based on the results of the distribution method in step 640, traffic sign legends inferred from high-speed vehicle telemetry can be determined, such as speed, stop, or yield. The inferred traffic sign data from step 650 can then be further processed in steps 660, 670, and 680. Step 660, the labeling conditions, can identify and label unexpected traffic signs and objects. Step 670 can also identify and further aggregate incorrect detections using vehicle sensors. Step 680 can also perform a deduplication process based on the inferred traffic sign data, which will... Figure 8 Further discussion is needed.

[0106] Then, at step 690, based on the determinations made in steps 660, 670, and 680, the final traffic sign value or legend can be identified as the most likely traffic sign legend.

[0107] Figure 7 It is an estimate 700 of the confidence score of high-speed vehicle telemetry data based on data distribution according to an embodiment of the present disclosure. Figure 7The diagram illustrates the distribution of vehicle speeds detected on a specific section of the road. Specifically, Figure 7 The diagram illustrates the high-speed vehicle telemetry distribution or histogram for a specific segment of the road, showing detected speeds from 0 mph to 80 mph. The histogram can also be normalized, and then N peaks in the high-speed vehicle telemetry box distribution can be detected, with significance at least greater than a threshold amount (e.g., 25%) for the largest peak, as shown by example largest peak 720 and non-largest peaks (e.g., peak 710). Next, support boxes around each peak, i.e., consecutive points of the peaks shown by support peak span 725, with significance greater than a threshold (e.g., 50% of the peak). The estimated confidence score can then be determined as follows:

[0108]

[0109] Where f i This represents the number of observations in the i-th bin, where the scaling factor is β = 0.5.

[0110] Figure 8 This is a diagram illustrating a HSVT-based filtering process based on raw telemetry data of high-speed vehicles, according to an embodiment of this disclosure. Figure 8 The illustration depicts an embodiment with a front-facing sensor (e.g., a camera) that may incorrectly detect and interpret traffic signs, i.e., false detections, such as... Figure 1 As shown in the image. The next step is to create labeling conditions to identify erroneous or incorrect detections, such as... Figure 5 As discussed in [the document]. Then, if the traffic speed distribution at that location is inconsistent with the detected sign type or speed limit value, high-speed vehicle telemetry can be used to correct erroneous detections. Therefore, Figure 8 The diagram illustrates a histogram of frequency versus speed (in miles per hour), where the maximum peak shown in region 810 can be used to filter out false detections.

[0111] The following will discuss three examples of the use of raw telemetry data from high-speed vehicles in the deduplication process, such as... Figure 9A , 9B As shown in Figure 9C.

[0112] like Figure 9A As shown, the first example could consist of a vehicle report using a front-mounted sensor that has incorrectly detected a traffic sign legend. This situation can then be flagged using the previously outlined process, and based on free-flowing traffic data, a distributional approach can be applied to the speed observations from the raw telemetry data of high-speed vehicles to determine the inferred traffic sign legend. Figure 9AFor example, this could be shown as an inferred speed limit of approximately 42 miles per hour, as indicated by the highest box sign 910, where the confidence score is greater than 0.5. Deduplication can then be applied, and the final value of the inferred traffic sign can be produced.

[0113] like Figure 9B As shown, the second example could consist of a report from a front-mounted sensor vehicle that has incorrectly detected a traffic sign legend (e.g., incorrectly detecting an 85 mph speed limit in a zone with an actual 35 mph speed limit). This example illustrates the incorrect identification of a "3" as an "8". The situation can then be flagged, for example, as... Figure 3 As discussed in [the paper], and based on free-flowing traffic data, a distributional approach can be applied to speed observations from raw telemetry data of high-speed vehicles to determine inferred traffic sign legends. Figure 9B In the example, this could be shown as an inferred speed limit of approximately 37 mph, as indicated by the highest box sign 920, which has a confidence score greater than 0.5, thus increasing the confidence score for "Speed ​​Limit 35" and decreasing the confidence score for "Speed ​​Limit 85". Deduplication can then be applied, and the final value of the inferred traffic sign can be generated based on the higher confidence result.

[0114] like Figure 9C As shown, the third example could consist of a front-mounted sensor vehicle report that has been incorrectly detected as a traffic sign legend (e.g., incorrectly detecting a traffic sign indicating multiple speed limits). For example, the traffic sign might indicate a speed limit of 70 mph, but it could also indicate a speed limit of 65 mph for trucks, and a minimum speed limit of 55 mph. The situation could then be labeled by indicating the highest detected speed relative to the state rule, and a distributional approach could be applied to the raw speed observations from high-speed vehicle telemetry data, based on free-flowing traffic data, to determine the inferred traffic sign legend. Figure 9C In the example, this could be shown as an inferred speed limit of approximately 74.5 mph, as indicated by the highest box sign 930, which has a confidence score greater than 0.5, thus increasing the confidence score for "Speed ​​Limit". Deduplication can then be applied, and a final inferred value for the traffic sign can be produced. In this case, the higher speed limit value would be determined based on observed traffic speeds.

[0115] Figure 10 An exemplary embodiment of a method for identifying, marking, and correcting accidental traffic sign data according to embodiments of the present disclosure is shown. Method 1000 begins at step 1005 with the capture of traffic sign data using a plurality of vehicles, wherein each vehicle includes a front sensor configured to capture traffic sign data. Figure 4 As discussed herein, traffic signs can be detected using a front-facing camera module in the vehicle or other types of image capture devices (including, but not limited to, light detection and ranging (Lidar), radar, or the like). While this example illustrates a speed limit traffic sign, the type of traffic sign is not limited to speed and can be other types of traffic signs.

[0116] At step 1010, the method can continue to collect and store traffic sign data captured from multiple vehicles over a period of time. The multiple vehicles can also operate as crowdsourcing entities, where crowdsourcing algorithms can be used to capture traffic sign data, and where such data can be captured during different parts of the day (e.g., when traffic is congested, and also during periods of minimal congestion where vehicles may travel at higher speeds). Furthermore, the method can utilize systems such as traffic sign data aggregation systems that can collect and store traffic sign data captured from vehicles over a period of time.

[0117] At step 1015, the method may continue to identify and flag one or more potential erroneous traffic sign detections from the captured traffic sign data based on road category and associated rules. Rules may include associated actions linked to a specific type of traffic sign. For example, a speed limit sign may be associated with a rule restricting the speed limit for vehicles on a specific road. A yield sign may be associated with a yield rule controlling which vehicle has the right-of-way. And a stop sign may be associated with a stop rule (e.g., at an intersection). Figure 3 As discussed herein, process 300 can receive a clustering table of data from an external source as input; for example, using road category and speed limit data, it can classify potential false detection labeling conditions as high-risk or low-risk. Also, Figure 3 As discussed herein, roads can be classified as primary, secondary, or tertiary types. While primary roads may typically be restricted-access highways with interchanges and ramps, secondary roads can include arterial roads that may or may not be separated, and may also include intersections. Tertiary roads can include roads outside urban areas with low to moderate traffic volume that link smaller villages or hamlets. Therefore, as... Figure 3 As discussed in step 310, it can be determined whether a road is a primary road type (e.g., a freeway, a median freeway, or an interstate freeway possibly designed for freeway speed traffic) using open street maps or other road classification methods. Furthermore, as... Figure 3As discussed earlier, if the road is identified as a major road at step 310, and the speed limit is greater than the highway speed limit at step 315, the marking condition can be set to low risk at step 320. However, if the road is identified as being below the major road category, a determination can be made at step 330 regarding whether the speed limit is greater than the highway speed limit (e.g., 65 mph). If the speed limit is less than the highway speed limit, a determination can be made at step 320 that the situation can again be classified as low risk. However, if the speed limit is classified as greater than or equal to the highway speed and the road has been identified as being below the major level, a detection can be determined as high risk at step 340.

[0118] At step 1020, the method may include determining the average vehicle speed during a non-maximum traffic period for one or more traffic zones. A traffic zone may be a reference to a specific portion of a road. For example, a portion of a road that typically handles non-congested traffic may be associated with a free-flowing traffic zone using a precise geospatial search method. Furthermore, the non-maximum reference is intended to describe the non-congested or free-flowing traffic conditions of a specific road that may be associated with one or more specific time periods, such as a day, a week, a month, or other time periods.

[0119] The method can continue at step 1025, filtering the captured traffic sign data based on traffic sign categories. Traffic sign categories can represent different types of traffic signs. For example... Figure 1 The categories of traffic signs discussed herein can be broad and include sign type categories such as stop signs, yield signs, intersection signs, one-way traffic signs, route or interstate highway number signs, turn signs, etc. Therefore, as... Figure 6 As discussed herein, filtering by traffic sign category can include temporal and spatial filtering to select free-flowing traffic data, where average vehicle speeds can be measured during periods of low traffic volume and / or high-congestion areas can also be filtered out. Furthermore, traffic sign filters can eliminate certain data by category type, such as removing low values ​​for speed limits, retaining values ​​associated with stop signs, and also retaining values ​​associated with yield signs.

[0120] At step 1030, the method can continue by determining the most probable traffic sign legend from the filtered captured traffic sign data based on the determination of the average vehicle speed during non-peak traffic periods. Figure 2 As discussed in step 280, "Determining Traffic Sign Legends," the most probable traffic sign values ​​or legends can be determined from the HSVT deduplication data. Furthermore, in... Figure 6 In step 690, based on the determinations made in steps 660, 670, and 680, the final traffic sign value or legend can be identified as the most likely traffic sign legend.

[0121] Method 1000 can then be completed.

[0122] The specification and abstract may set forth one or more embodiments of this disclosure as conceived by one or more inventors(s), and are therefore not intended to limit this disclosure and the appended claims.

[0123] The embodiments of this disclosure have been described above with the aid of illustrated functional building blocks that specify the implementation of functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined as long as the specified functions and their relationships can be properly performed.

[0124] The foregoing description of the specific embodiments so fully reveals the general nature of this disclosure that others can readily modify and / or adapt them to various applications of such specific embodiments without excessive experimentation, based on knowledge of the art, without departing from the general conception of this disclosure. Therefore, based on the teachings and guidance presented herein, such adaptations and modifications are intended to fall within the meaning and scope of equivalents of the disclosed embodiments. It is to be understood that the wording or terminology used herein is for descriptive purposes and not for limiting purposes, and that the terminology or terminology of this specification should be interpreted by those skilled in the art based on the teachings and guidance.

[0125] The breadth and scope of this disclosure should not be limited by the exemplary embodiments described above.

[0126] Exemplary embodiments of this disclosure have been presented. This disclosure is not limited to these examples. These examples are presented herein for illustrative purposes and not for limitation. Based on the teachings contained herein, alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to those skilled in the art(s) related to the subject matter. Such alternatives fall within the scope and spirit of this disclosure.

Claims

1. A system for identifying, marking, and correcting accidental traffic sign data, comprising: Multiple vehicles, each equipped with a front sensor configured to capture traffic sign data; A traffic sign data aggregation system, configured to collect and store traffic sign data captured from multiple vehicles over a period of time; The traffic sign data aggregation system is further configured to identify and flag one or more potential erroneous traffic sign detections from the captured traffic sign data based on road category and associated rules; A time and space filtering system, configured to determine the average vehicle speed during non-peak traffic periods for one or more traffic zones; The temporal and spatial filtering system is further configured to filter the captured traffic sign data based on traffic sign category; The traffic sign data aggregation system is further configured to determine the most likely traffic sign legend from the filtered captured traffic sign data based on a time and space filtering system.

2. The system of claim 1, wherein the traffic sign data aggregation system is further configured to perform data management to filter unsuitable data, including data from selected vehicles identified as sources of error in determining traffic sign speed limit values.

3. The system of claim 1, wherein the traffic sign categories include speed limit signs, stop signs, or yield signs.

4. The system according to claim 1, wherein the road categories include main roads, secondary roads and tertiary roads.

5. The system of claim 1, wherein the traffic sign data aggregation system is further configured to mark one or more erroneous traffic sign detections as high risk when the road category is not a primary level and the speed limit is greater than a threshold.

6. The system of claim 1, further comprising a traffic sign data aggregation system configured to perform data clustering by grouping traffic sign data associated with a single specific traffic sign.

7. The system of claim 1, wherein the traffic sign data captured from multiple vehicles over a period of time includes data collected and analyzed by a crowdsourcing algorithm.

8. The system of claim 1, wherein the temporal and spatial filtering system is further configured to apply a distributed approach to process crowdsourced telemetry data including estimated confidence scores.

9. The system of claim 8, wherein the distribution method further comprises estimating the confidence score based on determining the maximum peak and qualified peak from the crowdsourced telemetry data.

10. The system of claim 1, wherein the time and space filtering system is further configured to perform a deduplication process based on the determined most probable traffic sign legend.

Images