Additional information for the fusion of map and visual sensors for traffic sign recognition

Abstract

The invention relates to a method for adapting the semantic meaning of a traffic sign determined by an ego-vehicle from visual traffic sign recognition and / or from digital map data by using historical behavioral data from a plurality of other vehicles, wherein the behavioral data are determined by means of at least one acquisition unit arranged outside the ego-vehicle over a predetermined period under different conditions and made available to an evaluation unit.wherein, by means of the evaluation unit, a comparison is made between the data from visual traffic sign recognition and the behavioral data of the multitude of other vehicles in the vicinity of a geolocation of the ego-vehicle, and wherein, in the event of a deviation, information for adjusting the semantic meaning determined from visual traffic sign recognition and / or from digital map data is transmitted to a control and / or output device of the ego-vehicle.

Description

[0001] The invention relates to a method for supplementing or adapting a semantic meaning of a traffic sign for an ego-vehicle, determined from visual traffic sign recognition and / or from digital map data, and a system for supplementing or adapting a semantic meaning of a traffic sign for an ego-vehicle, determined from visual traffic sign recognition and / or from digital map data, based on behavioral data about a large number of other vehicles.

[0002] An automatic traffic sign recognition system based on optical sensors is known for self-driving vehicles such as passenger cars. The results are typically fused with information from a digital map to further improve the system and compensate for structural deficiencies, such as those caused by obstructions. The fused traffic sign recognition data can be displayed to the driver or fed into the vehicle's automatic driving control system or driver assistance system. Errors can occur during sensor-based traffic sign recognition, in the creation of the digital map, and ultimately during the fusion of the two data sources. For example, a misidentified traffic sign might lead to an incorrect speed limit being calculated.The consequence could be that the driver receives a false warning or that an assistance system sets an incorrect speed. At best, such errors therefore only represent a temporary inconvenience, while at worst they can encourage unlawful behavior or, in the case of automated driving, lead to dangerous situations by automatically increasing or decreasing the vehicle's speed.

[0003] In this context, DE 10 2021 107 543 A1 relates to a driving assistance device designed to be installed in an ego-vehicle and designed to assist the driver of the ego-vehicle in correctly observing the surroundings of a road on which the ego-vehicle is currently traveling, wherein the driving assistance device comprises: a sign identification unit designed to identify at least one specific type of road sign, wherein the sign identification unit is designed to base the identification on data recorded using a camera unit installed in the ego-vehicle and designed to capture images of the surroundings of the road on which the ego-vehicle is currently traveling.and to transmit the images to the sign identification unit and / or wherein the sign identification unit is configured to base the identification on data obtained from map data relating to the road on which the ego-vehicle is currently traveling, a traffic behavior monitoring unit configured to monitor the behavior of at least one road user who is near the ego-vehicle and is traveling on the same road and in the same direction as the ego-vehicle, a comparison unit configured to compare the identification result output by the sign identification unit with the monitored behavior of the at least one road user, which is output by the traffic behavior monitoring unit, and an identification adjustment unit configured toto adjust the determination result issued by the sign determination unit in the event that the comparison unit determines a deviation between the determination result issued by the sign determination unit and the monitored behavior of at least one road user, which is issued by the traffic behavior monitoring unit.

[0004] Since, as mentioned at the beginning, uncertainties can occur in traffic sign recognition, methods are known in the prior art to solve or reduce this problem, which perform a plausibility check based on the behavior of other road users. In particular, a method is known in which the meaning of a sign is determined using an image of a traffic sign captured by a vehicle camera and / or received map data, whereby the determined meaning is compared with the behavior of other road users and adjusted if necessary.

[0005] DE 10 2018 006 281 A1 relates to a method for operating a vehicle assistance system, wherein objects in the vehicle's environment are detected and a system action is triggered with respect to a detected object if a database contains an entry for the object that marks the system action as authorized, wherein the system action is nevertheless triggered if it is marked as unauthorized if the object is detected by means of several sensors, and / or is nevertheless triggered if its triggering is also required by a simultaneously running control procedure, and / or is triggered with a time delay compared to a system action marked as authorized, and / or is triggered with less intensity compared to a system action marked as authorized, wherein the intensity is subsequently increased step by step or continuously to the intensity of the system action marked as authorized.

[0006] German patent DE 10 2019 210 933 A1 discloses an environmental detection device for automated mobility, which is arranged, for example, at intersections or construction sites. The device comprises sensors arranged in a housing and an evaluation unit that determines objects and their state of motion from the sensor signals and provides this information to automated mobility systems, such as vehicles, via an interface.

[0007] German patent DE 10 2015 217 659 B4 discloses a method for extended driver information. After a traffic restriction is detected and a trigger signal is received, for example, user input, information explaining the traffic restriction is displayed. Furthermore, information is displayed if a traffic restriction is no longer valid but has not been explicitly lifted by signage.

[0008] From DE 10 2021 107 543 A1, a driving assistance device is known that identifies a specific type of road sign. The result of this identification is compared with the monitored behavior of at least one other road user moving in the same direction as the ego-vehicle. If there is a discrepancy between the identification result and the behavior, the result is adjusted.

[0009] German patent DE 10 2022 002 337 B3 discloses a method for determining a permissible maximum speed. Fleet data from vehicles in a fleet are transmitted to a central computer unit. A multidimensional traffic matrix is ​​created from the collected data, and the permissible maximum speed for a road segment is determined using a machine learning method. This determined speed is then used to validate attributes already stored in a digital map.

[0010] The object of the invention is to further improve the reliability of traffic sign recognition.

[0011] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0012] To adapt the semantic meaning of a traffic sign, determined by an ego-vehicle from visual traffic sign recognition and / or digital map data, using historical behavioral data from a large number of other vehicles, the behavioral data is collected over a specified period under various conditions by means of at least one acquisition unit located outside the ego-vehicle and made available to an evaluation unit. The evaluation unit compares the data from visual traffic sign recognition with the behavioral data of the large number of other vehicles in the vicinity of a geolocation of the ego-vehicle. In case of a discrepancy, information for adapting the semantic meaning determined from visual traffic sign recognition and / or digital map data is transmitted to a control and / or output device of the ego-vehicle.

[0013] The semantic meaning of a traffic sign can be, for example, a command, a prohibition, or information regarding traffic flow. Visual traffic sign recognition is performed using a camera, and the digital map data is stored on-board or off-board in the ego-vehicle. The evaluation unit, preferably implemented as a server, uses the semantic meaning received from the ego-vehicle to compare it with the behavioral data of other vehicles, detects any deviations, and sends information to a control and / or output device. For example, if the ego-vehicle detects a traffic sign indicating an exit, but none of the other vehicles have used the exit at the ego-vehicle's geolocation for an extended period, then the control and / or output device receives information that the exit is unavailable.The data acquisition unit provides the behavioral data of numerous other vehicles, along with their respective geolocations, to the evaluation unit, which then stores this data for analysis or comparison. Historical behavioral data refers to data from other road users that is captured, collected, and / or stored by the data acquisition unit before the ego-vehicle determines the semantic meaning of the traffic sign at its corresponding geolocation.

[0014] According to the invention, non-identifiable camera images are transmitted to the server by means of automatic visual traffic sign recognition to determine their semantic meaning. The traffic sign information determined by the server is used for plausibility checks and / or adjustments using behavioral data from other vehicles and / or digital map data in the ego-vehicle. Camera images are particularly unusable if they are too complex or if their format is not standardized.

[0015] In an embodiment of the invention, the visual traffic sign recognition and / or the semantic meaning determined from digital map data by the ego-vehicle relates to a speed limit of the traffic sign, and the behavioral data from the multitude of other vehicles include speed data, wherein, by comparison, a deviation of the speed limit of the traffic sign determined from the visual traffic sign recognition and / or from digital map data by the ego-vehicle is determined by the evaluation unit to a speed value representative of the behavioral data of the multitude of other vehicles, and wherein, in the event of a deviation above a predetermined threshold value, the information for adjusting the speed limit determined from visual traffic sign recognition and / or from digital map data is transmitted to the control and / or output device of the ego-vehicle.

[0016] The discrepancies between the semantic meaning of a traffic sign, determined by the ego-vehicle using visual traffic sign recognition and / or digital map data, and the behavioral data of numerous other vehicles in the vicinity of the ego-vehicle's geolocation, are preferably calculated using an evaluation unit implemented as an external server. For this purpose, the server receives the semantic meaning and geolocation of the traffic sign, determined by the ego-vehicle using visual traffic sign recognition and / or digital map data, from the ego-vehicle. The server then determines stored, i.e., historical, behavioral data of numerous other vehicles in the vicinity of the ego-vehicle's geolocation and compares this data with the semantic meaning determined by the ego-vehicle.If a discrepancy exists between the semantic meaning and the behavioral data of the numerous other vehicles, the server sends information in the form of a signal to a control or output device of the ego vehicle for adaptation. Advantageously, the information is already processed and made available to the ego vehicle, thus minimizing its processing load and the amount of data to be transmitted. Furthermore, the adaptation information is only transmitted as needed if a discrepancy is detected.

[0017] In the event that the semantic meaning of the traffic sign includes a speed limit, speed data stored on the evaluation unit is determined from the multitude of other vehicles, whereby as soon as a deviation of the speed limit determined by the ego-vehicle from visual traffic sign recognition and / or digital map data in the area of ​​the traffic sign is determined to a speed value representative of the speed data of the multitude of other vehicles in the vicinity of a geolocation of the ego-vehicle above a predefined threshold, the information is provided to adjust the speed limit determined from visual traffic sign recognition and / or digital map data to the control and / or output device of the ego-vehicle.

[0018] Different values ​​can be specified for the threshold depending on various conditions, for example depending on the time of day, weather conditions, type of road used, i.e. federal highway, country road or construction zone.

[0019] The control unit for automatic speed control then adjusts the speed according to the signal received from the server and / or an output device displays the speed limit associated with the signal instead of the speed limit determined by means of visual traffic sign recognition and / or map data.

[0020] The geolocation environment of the ego vehicle, within which the behavioral data of other vehicles is considered, is defined by an area, such as a circle or rectangle preferably limited to the lane being traveled, which can extend in front of and / or behind the geoposition of the ego vehicle. This geolocation environment preferably includes the road ahead of the ego vehicle, which is determined, for example, from steering wheel position, turn signal activation, and / or via a navigation device based on a destination input. The road ahead is thus considered a preview horizon of a predefined length, which can optionally be dynamically adjusted depending on environmental conditions, intersection density, and / or vehicle speeds.The area of ​​the traffic sign where the ego-vehicle determines a speed limit or other semantic meaning from visual traffic sign recognition and / or digital map data is generally located in the direction of travel before the sign's geolocation. The semantic meaning detected in this area often also applies to the area of ​​validity immediately following the traffic sign. A speed limit remains in effect until it is lifted. This can be explicitly lifted by a sign or implicitly by the road's design (e.g., urban road / federal highway / motorway) or by the end of roadworks. The representative speed is determined from the behavioral data of numerous other vehicles, encompassing speed data, using statistical methods and / or filtering.

[0021] The specified period can, for example, refer to specific times of day, periods determined by weather conditions, periods determined by construction sites, or generally to continuous recording by one or more recording units; in any case, the specified period must be chosen to be long enough so that the average speeds of the other vehicles under the respective conditions are sufficiently representative.

[0022] The at least one, i.e., in this case, a multitude of detection units, in one variant, represent the other road users themselves. These detection units comprise camera, lidar, or radar sensors, as well as speedometers and a geolocation device such as GPS. The speed data or other behavioral data of the other road users, including information such as time, environmental conditions, and geolocation, are transmitted to the server. The server can then analyze the numerous incoming data sets from the area around the ego-vehicle to identify potential malfunctions in the traffic sign recognition system, such as the determination of the speed limit within the ego-vehicle.This aggregated information can either be sufficiently informative on its own for the automatic generation of information for transmission to a control or output device of the ego vehicle, or it can be validated by other sources.

[0023] Alternatively or additionally, video data can be collected by a dedicated infrastructure unit to determine behavioral data from numerous other vehicles at relevant points along traffic routes. This behavioral data, similar to that collected from other road users, is then transmitted to the evaluation unit.

[0024] For example, the following cases can be identified and resolved using this method: 1) A traffic sign is located on a parallel street relative to the road or exit driven by the ego vehicle. Possible solutions: Behavioral data is collected from numerous other vehicles in the vicinity of the geolocation, i.e., in the lane of the ego vehicle, where the semantic meaning of the traffic sign on the parallel road is determined. If the semantic meaning deviates from the behavioral data of the other vehicles, preferably above a threshold, the traffic sign is ignored, as the visual traffic sign recognition system has evidently identified the traffic sign on the parallel road. In an advantageous embodiment, this evaluation can also be performed with lane-level precision. 2) Supplementary signs, such as "when wet," are misidentified or not recognized at all. Background: Especially when a self-driving vehicle is still far in front of a traffic sign, the actual sign can often be correctly identified by the vehicle's camera, but any accompanying information such as "when wet" or certain time restrictions are difficult to recognize from a great distance. Possible solutions: - If discrepancies occur between the semantic meaning determined by the ego vehicle and the behavioral data of the multitude of other vehicles in the vicinity of a geolocation of the ego vehicle or the supplementary sign, information for adjustment is provided, e.g. that a speed limit at a certain location only applies at a certain time. - Detecting the actual supplementary sign from close range is simpler and more reliable. This applies to traffic signs with restrictions on validity; i.e., in the interest of data-efficient transmission, it is advantageous to limit the focus to the necessary locations. Here, anomalies detected by the detection unit in the form of deviations exceeding a threshold can again be used. In addition to the difficulty of detection from a great distance, in certain regions of the world (e.g., the USA), unconventional and poorly standardized, often text-heavy additional conditions can make it very challenging to automatically determine the semantic meaning in the ego-vehicle. In these cases, images of the corresponding supplementary signs can be transmitted to the server to determine the semantic meaning. This makes it easier to process even complex sign information automatically. The actual information for the adjustment, such as "Sign X at position Y only applies under conditions Z," can be structured in the same way as described above. 3) Missing or late cancellation of speed limits or other restrictions. Background: Speed ​​limits are sometimes linked to specific conditions of the preceding section of road. The principle of necessity applies to the placement of road signs. A restriction, such as a speed limit, does not necessarily have to be lifted by a cancellation sign. For example, if a construction zone for which the speed limit applies ends, the speed limit also ends. These restrictions do not necessarily have to be explicitly lifted. In this case, it is anything but trivial for the ego-vehicle to recognize when, for example, the speed limit can be considered lifted.

[0025] Possible solutions: - Information derived from the behavioral data of the multitude of vehicles is provided to adjust the speed limit; i.e., at such locations, the information displayed by the output device includes virtual cancellation signs. - In order to position virtual cancellation signs in the correct location, suitable additional sources such as a live construction site service or a simple evaluation of the attributes (curves, bridge, tunnel, ...) in a digital map can now be used.

[0026] 4) Digital map data is permanently outdated. This is partly due to the update mechanism, which often requires surveys with special measuring vehicles. In contrast, actual changes to road infrastructure can be very dynamic. This typically results in periods of months or years during which digital maps can be outdated. The proposed method, however, is able to reliably detect changes to existing signage based on behavioral data, typically within hours or days.Outdated digital map data can cause problems when merging with automatic visual traffic sign recognition, particularly using camera-based traffic sign detection, if the detected semantic meaning and digital map data do not match. This can also occur, for example, if a traffic sign cannot be detected by the installed sensors because it is obscured, dirty, damaged, or difficult to see in the dark, and the digital map is also outdated. Possible solutions: - Information is provided to adjust the speed limit determined from automatic visual traffic sign recognition and / or digital map data: In the case of outdated map data, the information that a particular traffic sign at a particular location on the map can be ignored during fusion can be transmitted to the control or output device of the ego vehicle. -If a mismatch is detected between the visual traffic sign recognition and the semantic meaning of the traffic sign information determined from digital map data, confirmation can be obtained from other sources, in addition to information for adjusting the speed limit for safety reasons, that a different or new traffic sign is present or no longer present.

[0027] 5) Further traffic signs must be suppressed. Particularly in areas with heavy signage, it can happen that incorrect signs are detected by the sensors. As a possible solution, information is output to adjust the information recognized by visual traffic sign recognition if it deviates from the behavior data of the majority of other vehicles. For example, a traffic sign meaning "No Entry" or "No Overtaking" can be suppressed.

[0028] In an advantageous configuration, the server can also provide the ego-vehicle with speed-related information for adjustments that are perceived as convenient. This information can be useful, for example, if it is observed that the vast majority of other vehicles are consistently moving more slowly on a certain stretch of road, despite a higher speed limit. This could be due to sharp curves, cobblestones, or poor visibility on the road. In such cases, a virtual recommended speed limit sign (in Germany, a rectangular blue sign) can be transmitted to the ego-vehicle.

[0029] According to an advantageous embodiment, the detection unit includes a fleet of road users comprising numerous other vehicles, which transmit speed data or other behavioral data in the vicinity of the traffic sign to the central server. The data from the road user fleet includes all vehicles connected to the server, advantageously providing a data basis for the statistical determination of speed data or other behavioral data. The speed data or behavioral data can be classified, for example, according to weather, time of day, etc., in order to provide the vehicle fleet with corresponding real-time or historical speed data or behavioral data relevant to the conditions present for the ego vehicle.

[0030] According to an advantageous alternative or additional embodiment, a traffic monitoring camera is used as the detection unit, which transmits simultaneous or past speed data or behavioral data of the other vehicles in the vicinity of the traffic sign to the server.

[0031] According to a further advantageous embodiment, the server statistically evaluates the speed or behavioral data. The information for adjusting the speed limit includes statistical values ​​determined from the speed data of numerous other vehicles, for example, an average over a specified period, a filtered speed value, a median, a geometric mean, etc.

[0032] According to a further advantageous embodiment, the various conditions include at least one of the following: time restrictions, road conditions, wetness or dryness, weather conditions, categories of other vehicles. Examples include considering only trucks as other vehicles, considering only a predetermined number of hours and days, determining behavioral data within a time-restricted window, considering specific weather conditions (e.g., rain), or considering only information from other vehicles in a particular lane.To determine the deviation between the semantic meaning determined by the ego-vehicle from visual traffic sign recognition and / or digital map data, such as a speed limit on the traffic sign, and the behavioral data of the multitude of other vehicles determined by the evaluation unit, the historical behavioral data of the multitude of other vehicles are used, which were determined under the same, similar or at least comparable conditions that are relevant for the ego-vehicle when the information is transmitted for adaptation.

[0033] According to a further advantageous embodiment, the result of the automatic visual traffic sign recognition is discarded if, under predefined conditions, digital map data conflicts and the deviation from the speed data of the numerous other vehicles exceeds the predefined threshold. This applies in particular to cases where the traffic sign is detected in the wrong lane, the traffic sign is detected but without a restriction of validity, or a restriction has not been lifted.

[0034] According to a further advantageous embodiment, if the absence of a traffic sign indicating the cancellation of a restriction is determined from the behavioral data, a virtual cancellation signal is displayed to the driver. This display can be shown, for example, in the vehicle's instrument cluster, but also in a head-up display or another display unit in the vehicle's interior.

[0035] According to a further advantageous embodiment, digital map data is discarded and / or updated if, under given conditions, behavioral data of other vehicles and the result of automatic visual traffic sign recognition contradict the digital map data. Advantageously, this method allows the map data to be kept up to date at all times.

[0036] According to a further advantageous embodiment, the result of the automatic visual traffic sign recognition is discarded and replaced by the speed data of the multitude of other vehicles if sequences of traffic sign information are detected that do not meet a predetermined plausibility check.

[0037] For example, on highways or highway-like roads, the detected speed limit changes implausibly abruptly. However, especially in Europe, speed limits are often cascaded, from unlimited to 120 km / h, 120 km / h to 100 km / h, 100 km / h to 80 km / h, etc., but not abruptly from unlimited to a speed limit of 80 km / h. Furthermore, it is preferentially checked whether, under given conditions, digital map data and historical speed data of other road users contradict the result of the automatic visual traffic sign recognition.

[0038] Another aspect concerns a system for supplementing or adapting the semantic meaning of a traffic sign determined from visual traffic sign recognition and / or digital map data by an ego-vehicle using behavioral data from a large number of other vehicles, wherein at least one acquisition unit located outside the ego-vehicle provides a processing unit with behavioral data determined over a specified period under various conditions.The evaluation unit compares the semantic meaning of the traffic sign, determined by the ego-vehicle from visual traffic sign recognition and / or digital map data, with the behavioral data of numerous other vehicles. In case of a discrepancy, it transmits information to a control and / or output device of the ego-vehicle for adjusting the semantic meaning determined from visual traffic sign recognition and / or digital map data. According to the invention, the evaluation unit is configured to determine traffic sign information from transmitted camera images from the ego-vehicle's automatic visual traffic sign recognition, which cannot be evaluated by the ego-vehicle itself. The ego-vehicle uses the traffic sign information thus determined for plausibility checks and / or adjustments using the behavioral data of the other vehicles and / or the digital map data.

[0039] Advantages and preferred further developments of the proposed system result from an analogous and substantive transfer of the above statements made in connection with the proposed procedure.

[0040] Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.

[0041] They show: Fig. 1: A traffic situation with an ego vehicle in the area of ​​a traffic sign, for example with a speed limit, in which a method according to an embodiment of the invention is applied. Fig. 2: A method for supplementing or adapting traffic sign information obtained from visual traffic sign recognition and / or from digital map data according to an embodiment of the invention. Fig. 3, Fig. 4: further vehicles for data acquisition according to exemplary embodiments of the invention in the area of ​​a traffic sign.

[0042] The representations in the figures are schematic and not to scale.

[0043] Fig. Figure 1 schematically depicts a basic situation in which an ego-vehicle 9 is approaching a traffic sign 11, which indicates a maximum speed limit for the road ahead. For this purpose, the ego-vehicle 9 is equipped with a camera (not specified in detail) for visual traffic sign detection. This camera, indicated by the dashed lines, is directed into the area ahead and visually detects traffic sign 11. Furthermore, the ego-vehicle 9 has access to a digital map, represented by the traffic sign symbolized by XX. Digital map data is determined and used from this map based on the ego-vehicle 9's current position. The digital map also contains the semantic traffic sign information corresponding to the actual traffic sign 11 at the roadside.The data from the automatic visual traffic sign recognition system, using the camera unit directed towards the front of the Ego vehicle 9, and the digital map data can be fused together. This process can lead to inaccuracies, false detections, or other errors. A further explanation follows. Fig. 2. Evaluation unit 7 is described in more detail below. It is shown as an arrangement option for the procedure in the vehicle 9; the evaluation unit is preferably also located in a [context missing]. Fig. 3 and Fig. 4 can be integrated.

[0044] Fig. Figure 2 shows a method for supplementing or adapting a speed limit traffic sign information for an ego-vehicle 9 determined from visual traffic sign recognition and / or from digital map data according to Fig. 1. Behavioral data comprising speed data about a large number of other vehicles. 1. In step S1, the behavioral data comprising speed data is determined by means of at least one acquisition unit 3 arranged outside the ego vehicle 9 shown in Fig. over a specified period under different conditions and made available to an evaluation unit 7 implemented as a server 5, i.e. the behavioral data is sent to the server 5 with the respective geoposition and current environmental conditions and stored there.

[0045] In step S2, the historical, i.e., stored, behavioral data is used by the evaluation unit 7. This step includes a plausibility check and / or a comparison of a representative speed value determined from the speed data of the numerous other vehicles 1 in the vicinity of the geolocation of the ego-vehicle 9 with the speed limit information of the ego-vehicle 9, which is determined from visual traffic sign recognition and / or digital map data.Furthermore, this step includes the provision that, in the event of a deviation of the representative speed value from the speed limit determined from visual traffic sign recognition and / or digital map data above a specified threshold, information for adjusting the speed limit determined from visual traffic sign recognition and / or digital map data is transmitted to a control and / or output device of the Ego vehicle 9.

[0046] Fig. 3 and Fig. 4 show alternatives in the placement or design of a recording unit 3 for the execution of the procedure of Fig. 2. While in Fig. 3 a large number of other vehicles 1 together form the detection unit 3, is in Fig. 4 Alternatively or additionally, a stationary data collection unit 3 for traffic monitoring is provided. The stationary data collection unit 3 includes, for example, a traffic camera to record the behavior of other vehicles 1. In both cases, speed data of the other road users 1 are recorded and evaluated in relation to the position of the ego-vehicle 9.

[0047] The speed data and behavioral data of the other vehicles 1 are forwarded in both cases to the external server 5, which acts as a central switching point. The evaluation unit 7 of server 5 uses statistical methods to determine a representative speed value from the speed data of the vehicles 1, which is stored together with current conditions such as weather, time of day, etc., and the location. If, for example, as in Fig.If the ego-vehicle 9 is shown at the position of traffic sign 11, then, analogous to the prevailing conditions at the ego-vehicle 9, the corresponding representative speed of the multitude of other vehicles 1 is read out and compared with the speed limit determined by the ego-vehicle 9 from visual traffic sign recognition and / or digital map data, and if necessary, a difference, i.e., a deviation, is determined. If the deviation exceeds a predefined threshold, then a false recognition by the visual traffic sign recognition and / or the semantic meaning of the traffic sign determined from map data is assumed, and information for adaptation to a control and / or output device of the ego-vehicle 9 is transmitted.

[0048] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Claims

Method for adapting the semantic meaning of a traffic sign (11) determined from visual traffic sign recognition and / or from digital map data by an ego-vehicle (9) by using historical behavioral data from a large number of other vehicles (1), wherein the behavioral data are determined by means of at least one acquisition unit (3) arranged outside the ego-vehicle (9) over a specified period under different conditions and made available to an evaluation unit (7),wherein, by means of the evaluation unit (7) implemented as a server (5), a comparison is carried out between the semantic meaning determined by the ego-vehicle (9) from the visual traffic sign recognition and / or from the digital map data and the behavioral data of the multitude of other vehicles (1) in the vicinity of a geolocation of the ego-vehicle (9), and wherein, in the event of a deviation, information for adjusting the semantic meaning determined from visual traffic sign recognition and / or from digital map data is transmitted to a control and / or output device of the ego-vehicle (9), characterized in that camera images from the automatic visual traffic sign recognition, which are not evaluable, are transmitted to the server (5) for the determination of the traffic sign information.wherein the traffic sign information determined by the server (5) is used for plausibility checks and / or adjustments using the behavioral data of the other vehicles (1) and / or the digital map data in the ego-vehicle (9). The method according to claim 1, characterized in that the semantic meaning determined by visual traffic sign recognition and / or from digital map data by the ego-vehicle (9) comprises a speed limit of the traffic sign (11) and the behavioral data from the multitude of other vehicles (1) comprise speed data in the vicinity of the geolocation of the ego-vehicle.wherein a deviation of the speed limit of the traffic sign (11) determined by the ego-vehicle (9) from visual traffic sign recognition and / or digital map data is determined by the evaluation unit (7) from a speed value representative of the behavioral data of the multitude of other vehicles (1) and wherein, in the event of a deviation above a predetermined threshold value, the information for adjusting the speed limit determined from visual traffic sign recognition and / or digital map data is transmitted to the control and / or output device of the ego-vehicle (9). Method according to claim 1 or 2, wherein the deviation is determined by means of an evaluation unit (7) designed as a central server, which is set up to receive and process the semantic meaning of the traffic sign determined by the ego vehicle from the visual traffic sign recognition and / or from digital map data and the behavioral data in the vicinity of the traffic sign (11) from the multitude of other vehicles. Method according to one of claims 1 to 3, wherein the detection unit (3) includes a fleet of road users comprising the plurality of further vehicles (1) which transmit the behavioral data to the evaluation unit (7). Method according to one of claims 1 to 4, wherein a traffic monitoring camera is used as the recording unit (3), which transmits behavioral data of the other vehicles (1) to the evaluation unit (7). Method according to one of claims 1 to 5, by means of the evaluation unit (7) the representative speed value is determined from the speed data of the plurality of further vehicles (1) by means of a statistical evaluation. Method according to any of the preceding claims, wherein the various conditions include at least one of the following: time constraints, road condition, wetness or dryness, weather conditions, categories of other vehicles (1). Method according to one of the preceding claims, wherein the result of the visual traffic sign recognition is discarded if, under specified conditions, digital map data and the representative speed value of the other vehicles (1) are in agreement contrary to the result. Method according to one of the preceding claims, wherein if a missing traffic sign for lifting a restriction is determined from the behavioral data, a virtual lifting sign is displayed to a driver. Method according to one of the preceding claims, wherein digital map data are discarded and / or updated if, under given conditions, the behavioral data of the other vehicles (1) and the result of the automatic visual traffic sign recognition contradict the digital map data. Method according to one of the preceding claims, wherein the result of the automatic visual traffic sign recognition is discarded if sequences of traffic sign information are detected that do not meet a predetermined plausibility check. System for supplementing or adapting the semantic meaning of a traffic sign (11) determined from visual traffic sign recognition and / or from digital map data by an ego-vehicle (9) using behavioral data from a multitude of other vehicles (1), wherein at least one acquisition unit (3) arranged outside the ego-vehicle (9) provides the behavioral data determined over a specified period under different conditions to an evaluation unit (7) executed as a server,wherein the evaluation unit (7) compares the semantic meaning determined by the ego-vehicle (9) from visual traffic sign recognition and / or digital map data with the behavioral data of the multitude of other vehicles (1) and, in case of a deviation, transmits information to a control and / or output device of the ego-vehicle (9) for adjusting the semantic meaning determined from visual traffic sign recognition and / or digital map data, characterized in that the server (5) is further configured to: determine traffic sign information from transmitted camera images from the automatic visual traffic sign recognition of the ego-vehicle (9), which cannot be evaluated by the ego-vehicle (9) itself,and the ego vehicle (9) uses the traffic sign information thus determined for plausibility checks and / or adjustments using the behavioral data of the other vehicles (1) and / or the digital map data.

Images