Driving assistance device

Abstract

The present invention relates to a driving assistance device (110), comprising: a sign determination unit (112) adapted to determine at least one specific type of road signs (146a, 146b, 146c), wherein the sign determination unit (112) is adapted to make the determination based on data recorded using a camera unit (120) installed in the vehicle (100), and / or wherein the sign determination unit (112) is adapted to make the determination based on data obtained from map data about the road (140); and a traffic behavior monitoring unit (114) adapted to monitor traffic behavior in the vehicle (100). The invention relates to the behavior of at least one traffic participant (A, B, C) traveling in the vicinity of the vehicle (100) on the same road (140) and in the same direction (T); a comparison unit (116) adapted to compare the determination result with the monitored behavior of the at least one traffic participant (A, B, C); and a determination adjustment unit (118) adapted to adjust the determination result output by the sign determination unit (112) in the event of a mismatch between the determination result determined by the comparison unit (116) and the monitored behavior of the at least one traffic participant (A, B, C). The invention also relates to a corresponding vehicle and a corresponding method.

Description

Technical Field

[0001] This invention relates to a driving assistance device adapted for installation in a vehicle and configured to assist the driver of the vehicle in correctly observing the environment of the road on which the vehicle is currently traveling. The invention also relates to a vehicle including such a driving assistance device, and a method for assisting the driver of the vehicle in correctly observing the environment of the road on which the vehicle is currently traveling. Background Technology

[0002] Vehicles equipped with driver assistance devices suitable for detecting road signs are known in the prior art. The purpose of such driver assistance devices is to identify road signs as accurately as possible. Typically, driver assistance devices are adapted to use images recorded by cameras to determine the specific type of road sign, such as a specific speed limit indicated by a road sign.

[0003] When it is necessary to identify road signs displayed in traffic control and information systems (also known as "elevated gantry signs"), a particularly high failure rate occurs because they are more difficult to recognize. Furthermore, road signs displayed in traffic control and information systems may suddenly switch between different road sign types, and may also display several different specific types of road signs or even different road signs for adjacent lanes.

[0004] Other potential failures include the identification of markings on other objects, such as stickers on trucks, and the incorrect assignment of identified road signs to lanes or roads.

[0005] In particular, autonomous vehicles require high recognition quality for legally compliant driving. However, the recognition quality of existing driver assistance devices is insufficient for autonomous vehicles because the failure rate is too high when determining the specific type of road sign.

[0006] In view of the above, the object of the present invention is to overcome these limitations. In particular, the object of the present invention is to provide a driving assistance device with a reduced failure rate when identifying road signs. Summary of the Invention

[0007] According to a first aspect of the invention, this objective is achieved by a driving assistance device adapted to be installed in the vehicle and configured to assist the driver of the vehicle in correctly observing the environment of the road on which the vehicle is currently traveling, wherein the driving assistance device includes: a sign determination unit adapted to determine at least one specific type of road sign, wherein the sign determination unit is adapted to make the determination based on data recorded using a camera unit installed in the vehicle and adapted to record an image from the environment of the road on which the vehicle is currently traveling and forward the image to the sign determination unit, and / or wherein the sign determination unit is adapted to make the determination based on information from the road on which the vehicle is currently traveling. The system includes: a data acquisition unit; a traffic behavior monitoring unit adapted to monitor the behavior of at least one traffic participant traveling near the vehicle on the same road and in the same direction as the vehicle; a comparison unit adapted to compare a determination result output by a sign determination unit with the monitored behavior of the at least one traffic participant output by the traffic behavior monitoring unit; and an adjustment determination unit adapted to adjust the determination result output by the sign determination unit if the comparison unit determines a mismatch between the determination result output by the sign determination unit and the monitored behavior of the at least one traffic participant output by the traffic behavior monitoring unit.

[0008] As has already been noted, for simplicity, the invention will be discussed below in relation to right-hand traffic systems. Obviously, by simply mirroring the situation regarding the vehicle's direction of travel, the situation discussed for right-hand traffic can also be applied to left-hand traffic systems established, for example, in Japan and the United Kingdom. Furthermore, for better understanding, the invention will be described essentially by means of road signs indicating speed limits. Moreover, although the invention may be described below with respect to two or three traffic participants traveling in two or three lanes, it should be understood that the invention is applicable to roads with at least one lane for traffic participants traveling in the same direction as the vehicle, i.e., explicitly also applicable to roads with multiple such lanes, and applicable to at least one traffic participant traveling near the vehicle, i.e., explicitly also applicable to multiple such traffic participants.

[0009] Therefore, the present invention provides a solution to address mismatches between map data and camera-based recognition caused by, for example, the fact that the map data is outdated or defective, and / or that a sign is indicated in the map data but not recognized by the camera, and / or that a sign is recognized by the camera but not indicated in the map data, and / or that the map data indicates different signs or different specific types of road signs compared to the signs recognized by the camera, and / or that the signs are displayed in traffic control and information systems because these signs are dynamic traffic signs that cannot be stored in static maps and therefore the map data cannot support the recognition of dynamic traffic signs. In cases where such mismatches exist between map data and camera-based recognition, the present invention enables the determination of the correct specific type of road sign with high accuracy.

[0010] At least each time a road sign is determined by the sign determination unit, the driving assistance device of the present invention triggers the traffic behavior monitoring unit to monitor the behavior of at least one traffic participant near the vehicle. Then, the comparison unit compares the monitored behavior of the at least one traffic participant, output by the traffic behavior monitoring unit, with the determination result (e.g., the specific type of road sign) output by the sign determination unit. In this way, it can be determined whether the traffic around the vehicle is operating in a manner that matches the rules indicated by the determined specific type of road sign. If the monitored behavior of the surrounding traffic does not match the determined specific type of road sign, the determination adjustment unit can change the determination result, i.e., the determined specific type of road sign, to a determination result that more fully matches the behavior of the surrounding traffic. To provide a simple example, if the sign determination unit is identifying a road sign as indicating a speed limit of 120 km / h, but all surrounding traffic participants adjust their speed to 100 km / h, it can be inferred that the actual road sign type actually indicates a speed limit of 100 km / h and is incorrectly identified as indicating 120 km / h. This would cause the output of the driver assistance device, which instructs the driver of the vehicle on the speed limit and / or sets the vehicle's speed, to change to a value of 100 km / h. Of course, it's not only possible to monitor the deceleration of surrounding traffic participants, but also their acceleration and / or the basic maintenance of a specific speed.

[0011] By considering traffic behavior around the vehicle, the accuracy of the driver assistance system installed in the vehicle can be significantly improved because the accuracy of multiple different units can be combined. In other words, when determining the specific type of road sign, a redundancy can be achieved using the driver assistance system installed in the vehicle and the sign determination unit installed in other vehicles (even if it might be the driver of the other vehicle itself).

[0012] The driving assistance device according to the invention is suitable not only for monitoring the speed of at least one traffic participant at a specific point in time, but also for monitoring behavior over a predetermined time span, including, for example, acceleration, deceleration, or lane-changing maneuvers. This allows for the acquisition of additional data to cross-check the determination results output by the sign determination unit. For example, if the sign determination unit determines the specific type of a road sign to indicate a speed limit equal to or even slower than a previously determined road sign, but the at least one traffic participant still accelerates, a mismatch can be determined by the comparison unit. Therefore, the determination adjustment unit can adjust the determined road sign type to indicate a road sign type indicating a higher speed limit. Doing so further reduces recognition failures.

[0013] The driving assistance device according to the invention can also be adapted to compare images recorded by the camera unit with data obtained from map data. Sometimes, for example, road signs incorporated into the map data may not be recognized by the camera unit because other vehicles obstruct the line of sight between the camera unit and road signs. Such events can in particular trigger a traffic behavior monitoring unit to monitor the behavior of at least one traffic participant, such that at least the specific type of road sign indicated in the map data can be compared with traffic behavior around the vehicle. The map data may be stored in the vehicle and / or may be received from external sources, such as via V2X and / or cellular communication and / or GPS and / or via the Internet from a server.

[0014] Road signs can be any kind of road sign, not just a specific type of road sign indicating a particular speed limit. For example, a road sign can be a road sign indicating no overtaking, an additional road sign indicating when a related road sign is valid (e.g., over a specific time span, at a specific distance, for a specific vehicle weight, or when specific environmental conditions occur (e.g., for wet roads)), a lane guidance sign (e.g., to inform of the merging of multiple lanes, the narrowing of a lane), or a road sign that identifies the type of road (e.g., highway, public access road, or expressway) and its corresponding rules; only a few important examples are mentioned here.

[0015] Furthermore, this invention is not limited to road signs fixedly installed on the side of a road. It can also be applied to road markings, road signs displayed in traffic control and information systems, and road signs displayed on specific vehicles such as road maintenance vehicles, police cars, or heavy freight transport vehicles. Specifically, but not limited to, when road signs displayed in traffic control and information systems are indicated in map data, the traffic sign recognition threshold can be lowered in the desired area for focused recognition. Since it can be assumed that the road signs displayed in the traffic control and information system are substantially above the vehicle, the recognition algorithm of the sign determination unit can focus on that corresponding upper area.

[0016] It can be assumed that the behavior of the at least one traffic participant complies with traffic regulations and laws. However, to further improve the identification results, the comparison unit can be further adapted to consider the situation where at least one traffic participant is generally conforming to the speed limit indicated by the current road sign, but the traffic participant may still decide to drive faster than the indicated speed limit. Therefore, the comparison unit can be adapted to compare the determination result output by the sign determination unit with the monitored behavior of the at least one traffic participant output by the traffic behavior monitoring unit, taking into account, for example, a predetermined acceptable range of 30 km / h above the actual speed limit, preferably about 20 km / h.

[0017] To monitor the behavior of at least one traffic participant, the traffic behavior monitoring unit may be connected to at least one sensor. This at least one sensor may include, for example, a lidar system, a camera, a radar system, etc.

[0018] Based on this invention, the achieved recognition quality can be significantly improved, enabling the integration of driver assistance devices into automated vehicles. However, this invention can also be implemented as a standalone device, for example, suitable as an improvement to conventional vehicles.

[0019] Further improvements in road sign determination can be achieved when the sign determination unit is adapted to determine multiple possible different specific types for a single road sign, each specific type indicating different traffic rule information, particularly different speed limits on the road and / or lane on which the vehicle is currently traveling. In this case, each specific type can be associated with a different recognition probability based on the accuracy of the sign determination unit in determining the corresponding specific type of a single road sign. In particular, the determination of road sign types based on recorded images is sensitive to failure. Typical failures that may occur are blurred images or reduced contrast due to environmental factors, such as poor visibility or debris or damaged road signs. Associating the recognition probability with each of the multiple possible different specific types of a single road sign allows a comparison unit to compare the monitored behavior of at least one traffic participant with the possible road sign types based, for example, starting with their recognition probabilities, beginning with those of the highest recognition probability. Referring to the example given above, if the sign determination unit is determining the specific type of a road sign indicating a speed limit of 120 km / h with a recognition probability of 27% and the specific type of a road sign indicating a speed limit of 100 km / h with a recognition probability of 24%, but surrounding traffic participants adjust their speed to 100 km / h, the driver assistance device can discard the determination result indicating 120 km / h, even though it has a higher recognition probability, and the determination result adjustment unit can change the determination result to the value of 100 km / h.

[0020] The comparison process can stop when a match is detected. Therefore, the accuracy of the determination can be improved and / or the adjustment process for the determination result can be accelerated.

[0021] Cross-checking with the monitored behavior of at least one traffic participant can further improve recognition quality, especially when the recognition probabilities of at least two potentially different specific types of road signs are almost the same. For example, when the monitored behavior does not match the specific type of the road sign with the highest probability, but matches the specific type of the road sign with a second (or third, fourth, etc.) high probability, the determination adjustment unit can adjust the determination result output by the sign determination unit to the specific type of the road sign with the second (or third, fourth, etc.) high probability.

[0022] In this regard, the adjustment unit can be adapted to adjust the determination result output by the sign determination unit only when the difference between the highest recognition probability of a single road sign specific type or a first complete set of multiple road sign specific types and the lower recognition probability of a second complete set of multiple road sign specific types is less than a predetermined threshold, particularly less than 25%, preferably less than about 10%. In other words, a difference of less than 25% (or less than 10% respectively) can be taken into account not only for the recognition probabilities of the two single specific types of road signs, but also for the cumulative recognition probabilities of the first complete set of multiple road signs (e.g., including a first specific type of left road sign with a recognition probability of 33% and a first specific type of right road sign with a recognition probability of 31%, such that the cumulative recognition probability of the first complete set is equal to 64%) and the cumulative recognition probabilities of the second complete set of multiple road signs (e.g., including a second specific type of left road sign with a recognition probability of 28% and a second specific type of right road sign with a recognition probability of 27%, such that the cumulative recognition probability of the second complete set is equal to 55%).

[0023] The corresponding "cumulative recognition probability" should be understood as the combination of all recognition probabilities of the road signs that constitute the corresponding "complete set". Below, for better understanding, an example using the complete set of road signs is given. In this example:

[0024] The sign determination unit determines the left-hand road sign as having a 33% recognition probability indicating a speed limit of 120 km / h, a 28% recognition probability indicating a speed limit of 100 km / h, a 21% recognition probability indicating a speed limit of 130 km / h, and a 10% recognition probability indicating a speed limit of 80 km / h.

[0025] The sign determination unit determines the right-of-way sign as having a 31% recognition probability indicating a speed limit of 130 km / h, a 27% recognition probability indicating a speed limit of 100 km / h, a 25% recognition probability indicating a speed limit of 120 km / h, and an 11% recognition probability indicating a speed limit of 80 km / h.

[0026] The first set is now formed by the specific types of left and right road signs with the highest recognition probabilities: for left road signs, 120 km / h (33%); for right road signs, 130 km / h (31%). Here, the cumulative recognition probability of the first set is 64%. Therefore, assuming that the two road signs must be identical due to local traffic rules, the driver assistance device detects a mismatch. The traffic sign recognition results are then adjusted to a second set with the second highest cumulative recognition probability, which in this example is a combination of "left traffic sign: 120 km / h (33%); right road sign: 100 km / h (27%)" with a cumulative recognition probability of 60%. The cumulative recognition probability of the second set (60%) is within the allowable difference from the first set (64%) (a 4% difference is within the 10% range). However, a mismatch will still be identified. Therefore, the traffic sign recognition results were adjusted to a third set with the next highest cumulative recognition probability, which is the combination of "left traffic sign: 100km / h (28%); right road sign: 130km / h (31%)" with a cumulative recognition probability of 59%. The cumulative recognition probability of the third set (59%) is within the allowable difference from the first set (64%) (a 5% difference is within the 10% range). However, a mismatch was still identified. Therefore, the traffic sign recognition results were adjusted to a fourth set with the next highest cumulative recognition probability, which is the combination of "left traffic sign: 100km / h (28%); right road sign: 100km / h (27%)" with a cumulative recognition probability of 55%. The cumulative recognition probability of the fourth set (55%) is within the allowable difference from the first set (64%) (a 9% difference is within the 10% range).

[0027] As can be seen in the example above, the fourth set passed the plausibility check for not violating local traffic rules. If the fourth set also matches the behavior of other traffic participants, then it can be determined that the traffic sign identification based on the fourth set is likely correct.

[0028] Therefore, it is possible to prevent the determination results output by the sign determination unit from being adjusted to the unlikely road sign type. Furthermore, comparing the monitored behavior with the specific type of road sign with the highest probability, and, if applicable, comparing the monitored behavior with a finite number of further determined specific types of road signs within a predetermined threshold, can accelerate the adjustment process of the determination results. As a result, the vehicle's behavior can be adapted to road sign compliance behavior earlier and / or the result can be indicated to the driver of the vehicle earlier.

[0029] It is conceivable that even the specific type of a road sign with a second (or third) high probability of recognition may not match the monitored traffic behavior or may fail the plausibility check. Then, the next possible specific type of road sign can be selected. Advantageously, this can only be carried out if the next possible specific type of road sign is still within a predetermined threshold, specifically if the difference from the specific type of road sign with the highest probability of recognition is less than 25%, preferably less than about 10%.

[0030] Advantageously, the sign determination unit can be adapted to determine multiple road signs substantially simultaneously and to determine at least one specific type for each road sign, wherein the driving assistance device may further include a fidelity control unit adapted to re-examine at least one determined specific type for each road sign in light of existing road traffic rules. Implementing the fidelity control unit in the driving assistance device according to the invention enables the exclusion of obvious errors in the determination of the specific type of a road sign, for example, because they violate local traffic rules. This can also lead to a more accurate determination of specific road signs.

[0031] In an embodiment, the realism control unit may be adapted to indicate potential failures in the determination of multiple road signs when different specific road sign types are determined for road signs fixedly arranged on one side of a section of road and road signs fixedly arranged on the other side of the same section of road. That is, in many traffic systems, road signs fixedly installed on both sides of the same road section are defined to indicate the same traffic rule information, such as the same speed limit. Therefore, if the sign determination unit determines different specific types of road signs for road signs arranged on one side of the same section of road and road signs arranged on the other side of the same section of road, this may indicate that at least one of the determination results is incorrect.

[0032] Alternatively or additionally, the fidelity control unit can be adapted to indicate a possible failure in the determination of multiple road signs when two different specific types of road signs can be determined in the traffic control and information system for right-hand traffic systems, wherein the specific type of road sign associated with the left lane indicates a speed rule value that is lower than the speed rule value of the specific type of road sign associated with the right lane (when viewed in the direction of travel of the vehicle). For right-hand traffic systems, it is generally defined that if multiple road signs indicating different speed limits are displayed in the traffic control and information system, the different speed limits are arranged in ascending order from the right road sign to the left road sign. Therefore, a determination result for a right road sign indicating a higher speed limit than the left road sign can indicate a recognition failure. This can further improve the accuracy of the determination result.

[0033] According to another embodiment of the invention, when a road sign is detected in a traffic control and information system and the specific type of the road sign in the lane on which the vehicle is currently traveling differs from the specific types of road signs in other lanes, the traffic behavior monitoring unit may be adapted to monitor only the traffic behavior in the same lane of the road on which the vehicle is currently traveling, and / or may be adapted to use a multiplication factor to monitor the behavior of at least one traffic participant near the vehicle in at least one lane different from the lane on which the vehicle is currently traveling. This multiplication factor is associated with the specific types of other road signs in the corresponding other lanes and represents the difference in corresponding road rule information. By centrally monitoring the behavior of at least one traffic participant in the same lane as the vehicle, identification failures based on consideration of the behavior of at least one traffic participant traveling in a different lane with different road sign types can be prevented. For example, the vehicle may be traveling in a right-hand traffic system in a lane with an allocated speed limit of 100 km / h, where a speed limit of 120 km / h is allocated to the lane adjacent to the left. Now, when monitoring the behavior of traffic participants traveling to the left of the vehicle, a multiplication factor of 1.2 can be used to accurately determine traffic behavior. In other words, by using a multiplication factor when monitoring traffic participants traveling in lanes with speed limits different from the vehicle's lane, erroneous conclusions about the relationship between traffic behavior and specific determinations can be avoided. Of course, other multiplication factors are possible for other speed limits; for example, a multiplication factor of 1.25 could be used for an adjacent lane with an assigned speed limit of 80 km / h and an adjacent lane with an assigned speed limit of 100 km / h. In this context, the multiplication factor is not strictly limited to the factor that must be multiplied to obtain the speed limit of the other lane, but rather the multiplication factor can also indicate the offset value with respect to the different speed limits of at least two lanes. For example, in some traffic systems, the speed limits displayed in the gantry typically have a fixed speed difference of 20 km / h between adjacent lanes, and levels 60, 80, 100, and 120 are commonly used. Therefore, when monitoring traffic participants in adjacent lanes with different speed limits, an offset of 20 km / h can be considered.

[0034] Furthermore, the different road signs assigned to different lanes are stored by the driving assistance device and are considered when determining whether the determination adjustment unit needs to adjust the determination result. Based on the fact that the decrease and / or increase in speed limit from the first road sign to the second road sign following the first road sign in the vehicle's direction of travel is generally equal for all lanes, considering the previously stored speed limit of the first road sign can improve recognition accuracy when determining the specific type of the second road sign. Moreover, storing the road signs assigned to lanes adjacent to the lane the vehicle is currently traveling in simplifies lane-changing maneuvers, since the effective speed limit for the target lane is known.

[0035] The traffic behavior monitoring unit can be further adapted to: monitor the behavior of at least two traffic participants; detect the distance between the at least two traffic participants when the first traffic participant is traveling in the same lane ahead of the second traffic participant; identify at least one traffic participant as an independent traffic participant, wherein the independence of the traffic participant is determined when the distance between the at least two traffic participants substantially exceeds a predetermined distance, specifically a distance in meters equal to half the speed value of the second traffic participant in km / h, and / or when the second traffic participant is accelerating or decelerating and the first traffic participant substantially maintains its speed for a predetermined period of time; and output only the monitored behavior of the at least one independent traffic participant to the comparison unit. That is, the behavior of a traffic participant may strongly depend on the behavior of another traffic participant, such as when the traffic participant is following behind a preceding traffic participant, particularly when the following traffic participant is using an adaptive cruise control unit. Therefore, the behavior of the following traffic participant can be classified as non-independent, i.e., the following traffic participant decelerates / accelerates only in response to the preceding traffic participant. However, when a following traffic participant decelerates / accelerates independently of the preceding traffic participant, this can indicate that the following participant is attempting to adapt to changing traffic rules, such as a reduction / increase in speed limits, where the preceding participant has not yet made the corresponding adjustment. Furthermore, when the distance between two traffic participants is greater than a predetermined distance, it can also be determined that the two participants are acting independently of each other. Of course, the predetermined distance differs for different speeds of the traffic participants, as a medium distance is sufficient to induce independence at low speeds, but not at high speeds. Therefore, the time interval t = d / v (time interval t = distance d divided by speed v), which indicates the relationship between the distance between traffic participants and their speeds, can also be used to determine the dependence or independence of traffic participants. For example, the aforementioned distance value in meters, equal to half the speed value of the second traffic participant in km / h, is essentially equal to a time interval of 1.8 seconds. By considering only the behavior of independent traffic participants, the accuracy of sign determination can be improved because traffic participants who only react to other traffic participants without directly responding to road signs on the road can be excluded from traffic monitoring.

[0036] It should also be noted that a minimum monitoring period (e.g., 1.5 or 2 seconds) for the independent behavior of traffic participants may be necessary to avoid misclassification due to delayed reactions from second traffic participants following the first. However, keeping this minimum period relatively short may be additionally beneficial for accurately detecting the intent of traffic participants. For example, suppose there are two vehicles, A and B, classified as "independent," where B is driving behind A. If B attempts to adapt to a higher speed limit, but A does not adjust, B accelerates and closes the distance to A, but needs to decelerate shortly afterward to avoid colliding with A. Therefore, B's acceleration can only be observed for a very short time. By setting the time period correctly, B can be prevented from being incorrectly classified as "non-independent."

[0037] In this scenario, the traffic behavior monitoring unit can be further adapted to monitor the behavior of at least three independent traffic participants, wherein the comparison unit can be further adapted to compare the monitored behavior of these at least three independent traffic participants, and only consider behaviors similar to those of the majority of the monitored traffic participants. As the number of independent traffic participants exhibiting similar behavior increases, the probability that the specific type of the determined road sign matching that behavior is the same as the actual road sign increases.

[0038] For example, traffic changes caused by impending traffic congestion or obstacles due to accidents can also affect the behavior of at least one traffic participant. Considering the behavior of at least one traffic participant affected in this way may lead to erroneous conclusions and thus incorrect determinations. To improve accuracy, the driver assistance device may further include a traffic information receiving unit and a traffic information consideration unit. The traffic information receiving unit is adapted to receive traffic information about traffic conditions on a portion of the road ahead of the vehicle's current location, particularly traffic congestion, and the traffic information consideration unit is adapted to determine a predetermined traffic condition-compliant behavior of the traffic participant based on the traffic information output by the traffic information receiving unit. A comparison unit is also adapted to compare the monitored behavior of the at least one traffic participant output by the traffic behavior monitoring unit with the predetermined traffic condition-compliant behavior output by the traffic information consideration unit. Furthermore, a determination adjustment unit is also adapted to discard any mismatch between the determination result output by the sign determination unit and the indication output by the traffic behavior monitoring unit if the monitored behavior of the at least one traffic participant matches the predetermined traffic condition-compliant behavior. For example, the traffic information receiving unit may be adapted to receive traffic information from a global positioning system, wireless equipment, cellular networks, external servers, etc. As an example, if a traffic participant is reducing their speed, even if the sign determination unit indicates a much higher speed limit, but the speed reduction aligns with the traffic conditions due to an impending traffic congestion, the determination adjustment unit will not adjust the determination result. This is because it considers that the traffic participant is adapting their behavior to the surrounding traffic conditions rather than to the speed limit indicated by the current road sign. Furthermore, traffic information can be received via V2X. In this way, vehicles in a traffic jam can report the congestion to following vehicles.

[0039] In addition to receiving traffic information from external sources, the traffic information receiving unit can also be adapted to collect traffic information solely through onboard sensors, for example, to detect traffic congestion. As an example, a traffic congestion can be detected if all vehicles suddenly decelerate at a rate higher than normally observed when adapting to slower speed limits.

[0040] Advantageously, the driving assistance device may also include a server connection unit adapted to send to the server:

[0041] The determination result of the specific type of road sign output by the sign determination unit and / or the determination and adjustment unit output;

[0042] A unique identifier corresponding to road signs, especially a unique location on the road; and

[0043] Preferably, the date and time of sending the determination result;

[0044] The server connection unit may also be adapted to receive from the server:

[0045] Road sign information for different road signs, the different road signs indicating the specific type of the road sign previously determined by a number of other traffic participants;

[0046] The determination adjustment unit can also be adapted to adjust the determination result output by the sign determination unit when there is a mismatch between the determination result output by the sign determination unit and the road sign information received from the server. For example, a driver passing a road sign can confirm or reject the determination result output by the corresponding sign determination unit installed in the corresponding vehicle. The confirmed determination result can then be uploaded to the server as road sign information. The road sign information can then be sent to other traffic participants, for example, as updated map data, and / or when a traffic participant arrives at the corresponding road sign. The determination result output by the sign determination unit can then be compared with the road sign information received from the server. In the event of a mismatch, the determination adjustment unit can adjust the determination result.

[0047] In this regard, it may be beneficial to consider the number of confirmed specific types of road signs, for example, in the form of a threshold number of confirmed specific types of road signs that must be exceeded before considering road sign information. Doing so allows for the disregarding of road sign information uploaded to the server by only a very small number of traffic participants. Furthermore, road signs can change over time, for example, when displayed in traffic control and information systems. To determine such road signs, it is also beneficial to consider the date and time of the road sign information. As the time elapsed after the specific road sign information has been uploaded to the server increases, the probability that the determined road sign type is no longer valid also increases. To prevent the negative impact of outdated road sign information received from the server on the recognition results, the comparison unit can be adapted to consider only road sign information received from the server within a specific threshold, such as 15 minutes of the same day. However, since some gantries frequently (e.g., within minutes) change the indicated speed limits based on current traffic flow, for a robust implementation, the server can also apply intelligent algorithms. For example, if the server determines that the speed limit originates from a gantry, and a specific number of traffic participants, such as three, continuously report the same new speed limit within a short timeframe, say less than 3 minutes, the server adapts accordingly. Conversely, if no vehicles report traffic signs at the gantry location within a certain timeframe, the gantry should be assumed to be "closed," i.e., no speed limit is indicated.

[0048] It should be emphasized here that the driving assistance device according to the present invention can be partially or wholly implemented by a processor such as a central processing unit (CPU) that executes a program (software) stored in memory. All or some of the components can be implemented by hardware such as large-scale integrated circuits (LSI), application-specific integrated circuits (ASIC), or field-programmable gate arrays (FPGA), and / or can be implemented by a combination of software and hardware. The program can be pre-stored in a storage device such as a hard disk drive (HDD) or flash memory, and can be stored in a removable storage medium such as a DVD or CD-ROM, and can be installed on the storage device when the storage medium is installed in a vehicle.

[0049] According to a second aspect, the present invention relates to a vehicle including a driving assistance device according to a first aspect of the invention. The vehicle can adjust its behavior according to the specific type of road sign determined. As a result, the vehicle can drive with improved accuracy.

[0050] Furthermore, it should be understood that the vehicle according to the invention can be an automobile, truck, motorcycle, bus or any other vehicle suitable for driving on roads, especially an autonomous vehicle.

[0051] According to a third aspect, the present invention relates to a method for assisting a driver of a vehicle in correctly observing the environment of the road on which the vehicle is currently traveling, the method comprising the following steps:

[0052] A sign determination step, wherein at least one specific type of road sign is determined, wherein the determination is based on data recorded using a camera unit mounted in the vehicle and recording images of the environment of the road on which the vehicle is currently traveling, and / or wherein the determination is based on data obtained from map data regarding the road on which the vehicle is currently traveling.

[0053] A traffic behavior monitoring procedure in which the behavior of at least one traffic participant is monitored, said at least one traffic participant being near the vehicle and traveling on the same road as the vehicle and in the same direction as the vehicle.

[0054] The comparison step involves comparing the determination result output in the sign determination step with the monitored behavior of the at least one traffic participant output in the traffic behavior monitoring step.

[0055] An adjustment step is defined, wherein if, in the comparison step, it is determined that there is a mismatch between the determination result output in the sign determination step and the monitored behavior of the at least one traffic participant output in the traffic behavior monitoring step, the determination result output in the sign determination step is adjusted.

[0056] It has been noted that all the features, advantages, and functions described with respect to the driving assistance device according to the invention can also be applied to the method of the invention, and vice versa. The same applies to vehicles according to the second aspect of the invention.

[0057] As already described with respect to the driving assistance device according to the invention, the accuracy of the determination of the specific type of road sign can be significantly improved by comparing the results with the monitored behavior of traffic participants traveling in the surrounding environment of the vehicle, because mismatches and, consequently, incorrectly determined specific types of road signs can be identified based on the comparison results.

[0058] If, during the sign determination step, multiple possible different specific types of a single road sign are also determined, each specific type indicating different traffic rule information, and each specific type can be associated with a different recognition probability based on the accuracy of the sign determination step for the corresponding specific type of a single road sign, then the method according to the invention can be further improved. Therefore, in the case of a mismatch between the determined specific type of a road sign and the behavior of surrounding traffic, assuming that the corresponding specific type of the road sign matches the behavior of surrounding traffic, the determination result can be adjusted to a result associated with a second (or third, fourth, etc.) high recognition probability.

[0059] Advantageously, in the adjustment step, the determination result output by the sign determination step can be adjusted only if the difference between the highest recognition probability of a single road sign type or the highest recognition probability of a first set of multiple specific types of road signs and the lower recognition probability of a second set of multiple specific types of road signs is less than a predetermined threshold, particularly less than 25%, preferably less than about 10%. This avoids the determination result being adjusted to be associated with a lower recognition probability when the lower recognition probability is "not sufficiently likely," i.e., if the difference between the lower recognition probability and the highest recognition probability of a specific type of road sign exceeds 25% or 10%, respectively, the determination result can be left unadjusted. Attached Figure Description

[0060] The invention will be described in more detail with reference to the accompanying drawings and specific embodiments, wherein:

[0061] Figure 1An exemplary schematic layout of the vehicle equipped with a driving assistance device according to a first aspect of the present invention is shown;

[0062] Figures 2a to 2g The functions of the driver assistance device are shown in different exemplary traffic conditions;

[0063] Figure 3 A flowchart illustrating an embodiment of the operation of a driver assistance device is shown;

[0064] Figure 4 A flowchart illustrating a subroutine of an embodiment explaining the operation of the traffic behavior monitoring unit of a driving assistance device is shown;

[0065] Figure 5 It shows Figure 3 The flowchart of another subroutine of the flowchart; and

[0066] Figure 6 An exemplary flowchart of a method for assisting a driver according to a third aspect of the present invention is shown. Detailed Implementation

[0067] exist Figure 1 In this drawing, the vehicle is generally indicated by reference numeral 100, but is not shown in further detail. The vehicle 100 includes a brake 102, a throttle device 104, a vehicle speed control device 106 adapted to control vehicle speed by selectively activating the brake 102 and / or the throttle device 104, and a driving assistance device 110 according to the invention. To display information collected by the driving assistance device 110 to the driver of the vehicle 100, the driving assistance device 110 may be connected to a human-machine interface unit 108. This information may be collected by a sign determination unit 112, such that the sign determination unit 112 may also be directly connected to the human-machine interface unit 108.

[0068] In the illustrated example, the driver assistance device 110 includes a sign determination unit 112, a traffic behavior monitoring unit 114, a comparison unit 116, and a determination and adjustment unit 118. The driver assistance device 110 is connected to a camera unit 120 and a map data unit 122. The camera unit 120 is mounted in the vehicle 100 and adapted to record images of the environment from the road on which the vehicle 100 is currently traveling, and to forward the images to the sign determination unit 112. In this example, the map data unit 122 is adapted to determine the position of the vehicle 100 in map data from a global positioning system, for example via a GPS antenna 124, the map data being stored in a memory (not shown) connected to the driver assistance device 110. Based on the position of the vehicle 100, the map data unit 122 sends corresponding map data about the road on which the vehicle 100 is currently traveling to the sign determination unit 112, wherein the corresponding map data specifically indicates traffic rule information, such as road signs indicating specific speed limits (also referred to as "traffic signs") or the specific type of road sign.

[0069] Based on data received from map data unit 122 and / or images received from camera unit 120, sign determination unit 112 determines at least one specific type of road sign, such as the speed limit of the road and / or lane on which the vehicle 100 is currently traveling. To this end, sign determination unit 112 may access, for example, multiple previously stored specific types of road signs stored in a memory connected to sign determination unit 112, and may compare the detected specific type of road sign with the previously stored specific types.

[0070] The sign determination unit 112 can determine multiple possible specific types for a single road sign, each indicating different traffic rule information. Based on the accuracy of the sign determination unit 112 in determining the corresponding specific type of a single road sign, each specific type is associated with a different recognition probability. For example, the sign determination unit 112 compares the image received from the camera unit 120 with multiple road sign types, and the basic algorithm assigns a recognition probability to each of the multiple road sign types. The sign determination unit 112 outputs the road sign determination result to the comparison unit 116.

[0071] Traffic behavior monitoring unit 114 monitors the behavior of at least one traffic participant located near vehicle 100, traveling on the same road as vehicle 100, and in the same direction. For example, the monitored behavior may include acceleration, deceleration, and / or lane-changing maneuvers. To monitor the behavior of this at least one traffic participant, traffic behavior monitoring unit 114 is connected to at least one sensor 125. This at least one sensor 125 may include, for example, at least one lidar system, at least one camera, at least one radar system, etc. The monitored behavior of the at least one traffic participant is output by traffic behavior monitoring unit 114 to comparison unit 116.

[0072] By comparing the determination result output by the sign determination unit 112 with the monitored behavior of the at least one traffic participant output by the traffic behavior monitoring unit 114, the comparison unit 116 determines whether there is a mismatch between them. If the monitored behavior of the at least one traffic participant is different from the behavior that conforms to the road sign type, then a mismatch exists. This behavior that conforms to the road sign type can be determined by the comparison unit 116 based on the determination result output by the sign determination unit 112.

[0073] The comparison result is output to the determination adjustment unit 118. If the comparison unit 116 determines that there is a mismatch, the determination adjustment unit 118 is adapted to adjust the determination result. The determination adjustment unit 118 sends the adjusted road sign type to the human-machine interface 108 and / or the speed control device 106 to display the adjusted determination result and / or adjust the behavior (speed) of the vehicle.

[0074] In this scenario, the sign determination unit 112 outputs multiple possible different specific types with corresponding associated recognition probabilities for a single road sign, and the determination adjustment unit 118 can further determine the difference between each of the individual recognition probabilities associated with the possible different specific types of the road sign. In an exemplary case, if this difference between the highest and lowest recognition probabilities is less than a predetermined threshold, particularly less than 25%, preferably less than approximately 10%, the determination adjustment unit 118 can adjust only the determination result output by the sign determination unit 112. This prevents the determination result from being adjusted to a specific type of road sign with excessively low probabilities, even if surrounding traffic participants do not act according to the specific type of road sign with the highest recognition probability.

[0075] To more accurately determine the specific type of road sign, Figure 1The driving assistance device 110 shown in the example also includes a fidelity control unit 126. The fidelity control unit 126 re-examines the at least one specific type of road sign determined by the sign determination unit 112 according to existing road traffic rules and / or local laws / regulations. Furthermore, if the determined specific type of road sign violates existing road traffic regulations and / or local laws / regulations, the fidelity control unit 126 indicates a possible failure in the determination of the road sign. For example, if two road signs are fixedly installed on both sides of a road, local regulations typically require the two road signs to be identical. Therefore, if the determination results of the two road signs differ from each other, the fidelity control unit 126 can indicate that at least one of the determination results may be incorrect. Then, in particular, these two determination results of the two road signs can be compared with the monitored behavior of the at least one traffic participant.

[0076] In order to receive traffic information, Figure 1 The illustrated driver assistance device 110 also includes a traffic information receiving unit 130. The traffic information receiving unit 130 receives traffic information regarding the road conditions, particularly traffic congestion, on the section of road ahead of the current position of the vehicle 100. Furthermore, the driver assistance device 110 includes a traffic information consideration unit 132 adapted to determine a predetermined traffic condition compliance behavior of the at least one traffic participant based on the traffic information output by the traffic information receiving unit 130. The driver assistance device 110, particularly the traffic information receiving unit 130, can be connected to a cellular antenna 134, V2X communication, etc., to receive traffic information.

[0077] The driving assistance device 110 may also include a server connection unit 136. This server connection unit 136 receives the determination result of the specific type of road sign from the sign determination unit 112 and / or from the determination adjustment unit 118. The server connection unit 136 sends the determination result of the specific type of road sign and the unique identifier of the corresponding road sign to a server (not shown). The unique identifier includes, for example, the unique location of the corresponding road sign on the road, and preferably includes the date and time of the sent determination result. The unique identifier allows the server to assign the determination result to a specific road sign, for example, on a map. When sending the date and time of the sent determination result, the server is also able to specify the age of the determination result. Especially in this case, when road signs are deployed in traffic control and information systems, the specific type of road sign changes periodically, and the age of the determination result can indicate the reliability of the determination result.

[0078] Server connection unit 136 is also adapted to receive road sign information for different road signs from a server. This road sign information indicates the specific type of road sign previously determined by multiple other traffic participants. In the event of a mismatch between the determination result output by sign determination unit 112 and the road sign information received from the server, determination adjustment unit 118 can adjust the determination result output by sign determination unit 112. Comparison unit 116 can determine whether a mismatch exists. For this purpose, comparison unit 116 can be connected to server connection unit 136 and can receive road sign information for different road signs from the server via server connection unit 136.

[0079] Reference Figures 2a to 2g The function of the driver assistance device 110 is described in more detail regarding the specific conditions of the right-hand traffic system. Figures 2a to 2g The traffic conditions shown have some similarities. Therefore, similar parts are provided with the same reference numerals, and unless otherwise stated, a description of one of these figures can be applied to the other figures in these figures.

[0080] Figure 2a The vehicle 100 is shown traveling on road 140, which has left lane I, middle lane II, and right lane III. Figure 2a In the situation shown, the vehicle 100 is traveling in lane II along the direction of travel t. At a portion 144 of road 140, road sign 146a is fixedly set on the right side 148a of road 140 (also indicated by arrow R), and road sign 146b is fixedly set on the left side 148b of road 140 (also indicated by arrow L).

[0081] When the vehicle 100 approaches road signs 146a and 146b, camera unit 120 records at least one image of each road sign 146a and 146b, and / or map data unit 122 provides map data about road signs 146a and 146b. Based on the recorded images and / or map data, sign determination unit 112 determines at least one specific type for each road sign 146a and 146b, such as a speed limit of 100 km / h. To prevent incorrect determination of road signs 146a and 146b, such as incorrect determination of the specific type of road signs 146a and 146b, sign determination unit 112 and / or comparison unit 116 may compare at least one recorded image with map data. However, because a mismatch may occur between map data and camera-based recognition results, for example, due to the fact that map data is outdated or defective and / or that map data indicates different signs or different specific types of road signs than those identified by the camera, the present invention proposes monitoring the behavior of at least one traffic participant to determine which of the different specific types of road signs is correct.

[0082] exist Figure 2a In the situation shown, there are three other traffic participants, designated A, B, and C, traveling in lane I of road 140, and moving so close to each other that their actions may influence each other (as previously described regarding the dependent / independent behavior of traffic participants). Traffic participants A, B, and C are monitored by traffic behavior monitoring unit 114. It is then determined whether the behavior of traffic participants A, B, and C matches the specific type identified by the road sign, thereby identifying any errors in the identification of the specific type of road sign by the sign identification unit, as described above.

[0083] Here, given that the specific types of road signs 146a and 146b must be the same according to the regulations of certain traffic systems, the fidelity control unit 126 can re-examine the determined specific types of road signs 146a and 146b.

[0084] exist Figure 2b The text is essentially based on... Figure 2aThis is one scenario where traffic participant A slows down first, then traffic participant B, and finally traffic participant C. As a result, traffic participants B and C may slow down solely due to the actions of traffic participant A, rather than because they intend to adapt their speeds to the speed limits indicated on road signs 146a, 146b. Therefore, only traffic participant A acts independently of the other traffic participants; that is, only one traffic participant is reducing their speed. This makes the possibility of a mismatch between the monitored behavior of traffic participants and the specific type of road signs determined by the sign determination unit 112 if the behavior of traffic participant A does not match the specific type of road signs 146a, 146b determined by the sign determination unit 112.

[0085] You can also refer to this. Figure 2b In the described alternative scenario, traffic participant C decelerates first, then traffic participant B decelerates, and then traffic participant A decelerates. As a result, all three vehicles decelerate independently. Therefore, there is a high probability that the specific types of road signs 146a and 146b indicate a reduced speed limit (compared to the previous speed limit). This conclusion can be used to re-examine the specific types of road signs 146a and 146b determined by sign determination unit 112.

[0086] exist Figure 2c In this scenario, three traffic participants, A, B, and C, are driving at substantially the same speed as vehicle 100. Vehicle 100 is traveling in the middle lane II, traffic participants B and C are traveling in the left lane I, and traffic participant A is traveling in the right lane III. Then, traffic participant A begins to accelerate, and traffic participant C begins to closely follow traffic participant B, i.e., traffic participant C is also accelerating. As a result, two of the three traffic participants are accelerating (or at least attempting to accelerate), indicating that it is highly likely that the specific type of road signs 146a and 146b indicates an increased speed limit (compared to the previous speed limit).

[0087] exist Figure 2d The diagram shown is basically based on Figure 2c In an alternative exemplary scenario, traffic participant A maintains their speed even after road signs 146a and 146b. However, traffic participant B begins to accelerate, and traffic participant C begins to accelerate after traffic participant B. As a result, traffic participant C may simply follow traffic participant B without considering the actual speed limit. Therefore, only one traffic participant, B, is truly accelerating independently. This could indicate that an increased speed limit is unlikely.

[0088] Figure 2eThe aim is to imagine that if a predetermined distance, or even greater, exists between traffic participant A and traffic participant B, then even if B decelerates after A, traffic participant B, following traffic participant A, can be considered to decelerate independently of traffic participant A. This predetermined distance can depend on the current speed, for example, a distance in meters equal to half the speed value of traffic participant B in km / h, or in other words, the distance necessary for traffic participant B to avoid passing the point reached by traffic participant A on road 140 for at least approximately 2 seconds. Furthermore, it must be considered that even if A and B are "far apart," traffic participant B will eventually need to decelerate after a period of time to avoid colliding with traffic participant A. In one example, if the distance between traffic participants A and B is sufficiently large, for example, greater than the distance in meters of traffic participant B's speed value in km / h, it can be assumed that traffic participant A's (moderate) deceleration does not affect the subsequent speed of traffic participant B in the next 1.5 seconds; that is, B's reaction within 1.5 seconds after A's reaction (approximately 1 second reaction time + 0.5 seconds for deceleration) can indicate that both reactions are initiated independently.

[0089] Road signs can also be displayed in the traffic control and information system 150. Figure 2f Each lane may have one road sign, for example, in Figure 2f In the diagram, road sign 146a is for lane III, road sign 146b is for lane II, and road sign 146c is for lane I. In other cases, there may be more road signs, such as... Figure 2g As shown. Typically, the specific type of road signs displayed in the traffic control and information system 150 is dynamic and can change over time. However, in the traffic control and information system 150, at least in right-hand traffic systems, if the speed limits are completely different, the speed limits are arranged in descending order from left lane I to right lane III.

[0090] In the following text, reference will be made to Figure 3 The flowchart describes the operation of an exemplary embodiment of the driving assistance device 110 in more detail.

[0091] The process begins at step S100, in which the vehicle 100 is started. Then, the process proceeds to step S110, in which the behavior of at least one traffic participant is monitored.

[0092] The process then proceeds to step S120, where it is determined whether the road 140 on which the vehicle 100 is traveling has more than one lane. If this is not the case (step S120: No), the process returns to step S110.

[0093] If the vehicle 100 is traveling on a road 140 with more than one lane (step S120: Yes), the process proceeds to step S130, where it is determined whether at least one road sign is detected by the camera unit 120 and / or the map data unit 122. If no road sign is detected (step S130: No), the process returns to step S110.

[0094] If at least one road sign is detected (step S130: Yes), the process proceeds to step S140, where the at least one road sign is determined. If multiple road signs are detected, all road signs are determined. As described above, the specific types of road signs can form at least one complete set of road signs.

[0095] Next, in step S150, the process determines whether the at least one road sign is present in the traffic control and information system 150. If, in step S150, the at least one road sign is detected in the traffic control and information system 150 (step S150: Yes), the process proceeds to the following reference. Figure 5 The subroutine S152 is described further. If no road sign is detected in the traffic control and information system 150 in step S150 (step S150: No), the process determines in step S210 whether the behavior of more than one traffic participant can be monitored. If this is not the case (step S210: No), the process returns to step S110, where the behavior of all traffic participants is monitored.

[0096] If the behavior of more than one traffic participant can be monitored (step S210: Yes), the process continues to step S220, where it is determined whether only one road sign is identified and / or whether a mismatch between at least two road signs 146a, 146b can be identified and / or whether a mismatch between at least one of the identified road signs 146a, 146b and the map data can be identified. If this is not the case (step S220: No), the process returns to step S110.

[0097] If the result of step S220 is positive (step S220: Yes), the process continues with subroutine S180 (described below). Data D190 is received from subroutine S180.

[0098] In the subsequent step S200, it is determined, based on data D190, whether at least one specific road sign differs from the assumed result of subroutine S180. Specifically, a first overall analysis result is set based on data regarding the behavior of traffic participants and on the specific type of all detected road signs, including their corresponding recognition probabilities (in the case of the first execution of step S200; otherwise, it is a newly adjusted overall analysis result). In step S200, the first (or current) overall analysis result is checked for mismatches, and plausibility control is performed as described above. In the negative case (S200: No), i.e., if no mismatch is detected in the current overall analysis result, the process proceeds to step S202, where the current overall analysis result is accepted, i.e., rejected or rewritten. The process then returns to step S110 to prepare for a new process iteration in the event, for example, the detection of a new road sign.

[0099] If step S200 is affirmative (S200: Yes), the process proceeds to step S204. In step S204, the first / current overall analysis result is adjusted, for example, by selecting a second set of road signs with the next highest cumulative recognition probability, as described above.

[0100] Then, the process proceeds to step S170. In step S170, it is determined whether the new overall analysis result and / or the new complete set of road signs differs from the first overall analysis result and / or the first complete set of road signs (specifically, the overall analysis result and / or the complete set of road signs with the highest overall recognition probability) by more than 10% (in this example, a value of 25% is also conceivable)). If the determination result is negative (step S170: No), the process returns to step S110. If the result is positive in step S170 (step S170: Yes), the process returns to and repeats step S200 above.

[0101] The loop S200->S204->S170->S200 is repeated multiple times as needed, including the adaptation of different variables of the current overall analysis result and / or the entire set of road signs, until it is determined in step S200 that there is no mismatch. Then, the current overall analysis result is accepted via step S202, i.e., the previous overall analysis result is rewritten, or until the difference exceeds 10% / 25%, then the process returns to step S110. The result is that there is a mismatch in the current overall analysis result, but it is not possible to resolve it at this time.

[0102] Now, refer to Figure 4 An embodiment of the description subroutine S180 is designed to classify the behavior of at least one traffic participant.

[0103] Subroutine S180 begins at step S270 and proceeds to step S272, in which all traffic participants near the vehicle 100 and traveling on the same road 140 as the vehicle 100 and in the same direction t as the vehicle are detected, wherein, in particular, speed change information of all traffic participants within the sensor range is collected, for example, by continuously observing their behavior.

[0104] Next, subroutine S180 proceeds to step S274, where the acceleration counter data D276, deceleration counter data D278, constant speed counter data D280, and blocking counter data D282 are set to zero. The sum of all data D276, D278, D280, and D282 equals the number of traffic participants detected at the end of subroutine S180.

[0105] Subroutine S180 then proceeds to step S284, where the first traffic participant A is analyzed. Of course, this and subsequent steps can be repeated for all detected traffic participants, but the first traffic participant A is shown below to provide an example.

[0106] Subroutine S180 then proceeds to step S286, where it is determined whether a first traffic participant A whose behavior must be classified can be detected. This iteration continues until all detected traffic participants have been processed. If step S286 is negative (step S286: No), subroutine S180 terminates in step S288.

[0107] If step S286 is affirmative (step S286: Yes), subroutine S180 proceeds to step S290. In step S290, it is determined whether the first traffic participant A is significantly slower or faster than the previous speed limit, because traffic participants significantly slower (e.g., trucks) or faster (e.g., speeding vehicles) than the previous speed limit can be ignored, as they are not indicators of an upcoming speed limit. If step S290 is affirmative (step S290: Yes), subroutine S180 proceeds to step S292, where the next traffic participant B is selected, and subroutine S180 proceeds to step S286 and performs another iteration, as described above.

[0108] If the first traffic participant A is not significantly slower or faster than the previous speed limit (step S290: No), subroutine S180 proceeds to step S294, where it is determined whether the first traffic participant A is accelerating or starting to tailgate. If this is the case (step S294: Yes), subroutine S180 proceeds to step S296, where the acceleration counter data D276 is incremented. Then subroutine S180 proceeds to step S292.

[0109] If the first traffic participant A is neither accelerating nor beginning to tailgate (step S294: No), then subroutine S180 proceeds to step S298. In step S298, it is determined whether the distance between the first traffic participant A and the traffic participant ahead of the first traffic participant A exceeds a predetermined distance. If this is the case, then the first traffic participant A can be considered to be acting independently.

[0110] If step S298 is affirmative (step S298: Yes), subroutine S180 proceeds to step S300, where it is determined whether the first traffic participant A is decelerating. If the first traffic participant A is decelerating (step S300: Yes), it is assumed that the analyzed first traffic participant A is decelerating independently of the preceding traffic participants, and subroutine S180 first increments the deceleration counter data D278 in step S302, then proceeds to step S292. If the first traffic participant A is not decelerating (step S300: No), it is assumed that the analyzed first traffic participant A maintains its current speed without obstruction, and subroutine S180 first increments the constant speed counter data D280 in step S304, then proceeds to step S292.

[0111] If the distance between the first traffic participant A and the preceding traffic participant does not exceed a predetermined distance (step S298: No), then subroutine S180 proceeds to step S306, in which it is determined whether the first traffic participant A is decelerating. If this is not the case (step S306: No), it is assumed that the analyzed traffic participant is driving at approximately the same speed due to the preceding traffic participant (which is blocking the analyzed first traffic participant A), and subroutine S180 first proceeds to step S308, in which the blocking counter data D282 is incremented, and then proceeds to step S292.

[0112] If the first traffic participant A is decelerating (step S306: Yes), then the behavior of the preceding traffic participant must be analyzed, and subroutine S180 continues to step S310. In step S310, it is determined whether the traffic participant preceding the first traffic participant A started decelerating earlier than the first traffic participant A (considering time intervals, as described above, e.g., A starts decelerating no later than 2 seconds after its predecessor). If this is not the case (step S310: No), then it is assumed that the analyzed first traffic participant A decelerates independently of its predecessor, and subroutine S180 proceeds to step S302.

[0113] If step S310 outputs a positive result (step S310: Yes), it is assumed that the first traffic participant A being analyzed slowed down due to the traffic participant in front of them, rather than due to the lower speed limit. Therefore, subroutine S180 proceeds to step S308, where the blocking counter data D282 is incremented. Next, subroutine S180 proceeds to step S292.

[0114] It should be noted that the subroutine S180 described above can be designed to run continuously so that the outputs of multiple runs, namely data D276, D278, D280, and D282, can be filtered. For example, the average of several acceleration data values ​​D276 can be calculated to introduce hysteresis. In this way, the output values ​​become more robust.

[0115] If the value of acceleration counter data D276 is greater than the sum of the values ​​of constant speed counter data D280 and deceleration counter data D278, a higher speed limit can be assumed. If the value of deceleration counter data D278 is greater than the sum of the values ​​of constant speed counter data D280 and acceleration counter data D276, a lower speed limit can be assumed.

[0116] Based on the assumed results output by subroutine S180 and the road sign 146 determined according to this process, as referenced above. Figure 3 The determination and adjustment unit 118 can adjust the determination result of the specific type of road sign indicating a specific speed limit.

[0117] Now, refer to Figure 5 An embodiment of the described subroutine S152 is designed to process signs displayed on a gantry (traffic control and information system 150).

[0118] Subroutine S152 begins at step S154.

[0119] Next, subroutine S152 proceeds to subroutine S180, which is the same as the above-described subroutine S180.

[0120] Then, subroutine S152 proceeds to step S160. In step S160, it is determined whether a road sign dependency relationship of at least one road sign is violated, i.e., whether existing traffic regulations and / or rules are violated. For example, whether a road sign associated with the right lane in traffic control and information system 150 displays a higher speed limit than a road sign associated with the left lane in traffic control and information system 150 (which is violated in most right-hand traffic systems). If this is the case (step S160: Yes), then in step S204, the first / current set of road signs is adjusted, for example, by selecting the next set with the second highest cumulative recognition probability (similar to step S204 above).

[0121] Then, the process proceeds to step S170, which is the same as step S170 described above. Therefore, if the current total set and / or current overall analysis result of the road signs with the relevant specific cumulative recognition probability differs from the first (and / or highest) total set / overall analysis result of the road signs by no more than 10% or 25% (step S170: Yes), then subroutine S152 re-enters step S160. If step S170 is negative (step S170: No), subroutine S152 ends in step S156, and the process proceeds as described above. Figure 3 Proceed as described. (As per the above) Figure 3 The negative result of step S170 may indicate the existence of a mismatch, but there is no (better) solution at this point.

[0122] If step S160 outputs a negative result (step S160: No), the process proceeds to step S200a, which is almost identical to step S200 described above, except that step S160 is shown as a step separated from step S200a by 200a. Figure 3 Step S200 also includes the plausibility control of step S160. Therefore, refer to the above description whenever applicable.

[0123] It should be mentioned at this point that even if a specific step is stated here, in particular a specific step of subroutine S152, it may be similar to / identical to another step, that is, their general logic may be similar to / identical. These steps may handle completely different data, such as the lack of realism that can be identified by the realism check incorporated in step S160 if multiple road signs installed in the gantry are not different from each other in the correct order, and the lack of realism that can be identified by the realism check incorporated in step S200 if two road signs installed on both sides of the road are different from each other.

[0124] The data D190 derived from subroutine S180 (see the description of data D190 above) is input into step S200a as input data.

[0125] Now, if step S200a outputs a positive result (step S200a: Yes), the process proceeds to step S204 above. If step S200a outputs a negative result (step S200a: No), then in step S202, the current overall analysis result is accepted, i.e., the previous result is rejected or rewritten (as in the reference). Figure 3 Step S202 (described).

[0126] After step S202, subroutine S152 ends in step S156, and the process is as follows: Figure 3 The process will be carried out on the site described.

[0127] refer to Figure 6 The flowchart below will describe in more detail an embodiment of a method for assisting the driver of the vehicle 100 to correctly observe the environment of the road 140 on which the vehicle 100 is currently traveling.

[0128] The method begins with a sign determination step S400, in which at least one specific type of road sign is determined. The determination in sign determination step S400 is based on data recorded using camera unit 120 and / or on data obtained from map data regarding the road 140 on which the vehicle 100 is currently traveling. Camera unit 120 is mounted in the vehicle 100 and records images of the environment from the road 140 on which the vehicle 100 is currently traveling.

[0129] Furthermore, in the sign determination step S400, multiple possible different specific types of a single road sign can also be determined. Each specific type indicates different traffic rule information. Moreover, based on the accuracy of the determination of the corresponding specific type of the single road sign in the sign determination step S400, each specific type can be associated with a different recognition probability.

[0130] The method then proceeds to traffic behavior monitoring step S410, in which the behavior of at least one traffic participant A is monitored. Traffic participant A is located near vehicle 100 and is traveling on the same road as vehicle 100 and in the same direction as vehicle 100. According to another embodiment, traffic behavior monitoring step S410 can be performed continuously and independently of sign determination step S400.

[0131] The results of the sign determination step S400 and the traffic behavior monitoring step S410 are output to the comparison step S420. In the comparison step S420, the determination result output from the sign determination step S400 is compared with the monitored behavior of the at least one traffic participant A output from the traffic behavior monitoring step S410.

[0132] Then, the method can proceed to the determination adjustment step S430, wherein, in the case that a mismatch is determined in the comparison step S420 between the determination result output from the sign determination step S400 and the monitored behavior of the at least one traffic participant A output from the traffic behavior monitoring step S410, the determination result output from the sign determination step S400 is adjusted.

[0133] Furthermore, in the adjustment step S430, the determination result output in the sign determination step S400 can be adjusted only if the difference between the highest recognition probability of the specific type of the identified road sign and the second highest recognition probability of the same specific type of road sign is less than 25%, preferably less than 10%. Finally, the method ends in step S440.

Claims

1. A driving assistance device (110) adapted to be installed in a host vehicle (100) and configured to assist a driver of the host vehicle (100) to correctly observe an environment of a road (140) on which the host vehicle (100) is currently travelling, wherein, The driving assistance device (110) includes: A sign determination unit (112) is adapted to determine at least one specific type of road sign (146a, 146b, 146c), wherein the sign determination unit (112) is adapted to base the determination on data recorded using a camera unit (120), the camera unit being mounted in the vehicle (100) and adapted to record images from the environment of the road (140) on which the vehicle (100) is currently traveling and to forward the images to the sign determination unit (112), and / or wherein the sign determination unit (112) is adapted to base the determination on data obtained from map data regarding the road (140) on which the vehicle (100) is currently traveling. A traffic behavior monitoring unit (114) is adapted to monitor the behavior of at least one traffic participant (A, B, C) traveling near the vehicle (100) and on the same road (140) as the vehicle (100) and in the same direction (t) as the vehicle (100). The comparison unit (116) is adapted to compare the determination result output by the sign determination unit (112) with the monitored behavior of the at least one traffic participant (A, B, C) output by the traffic behavior monitoring unit (114), and The adjustment unit (118) is adapted to adjust the determination result output by the sign determination unit (112) when the comparison unit (116) determines a mismatch between the determination result output by the sign determination unit (112) and the monitored behavior of the at least one traffic participant (A, B, C) output by the traffic behavior monitoring unit (114). If a road sign (146a, 146b, 146c) is detected in the traffic control and information system (150) and the specific type of the road sign (146a, 146b, 146c) for the lane (I, II, III) on which the vehicle (100) is currently traveling is different from the specific type of the road sign (146a, 146b, 146c) for other lanes (I, II, III), the traffic behavior monitoring unit (114) is adapted to monitor only the same lane (I, II, III) of the road (140) on which the vehicle (100) is currently traveling. Traffic behavior near the vehicle (100) in lanes (I, II, III) and / or suitable for using a multiplication factor to monitor the behavior of at least one traffic participant (A, B, C) near the vehicle (100) in at least one lane (I, II, III) different from the lane (I, II, III) on which the vehicle (100) is currently traveling, the multiplication factor being associated with other specific types of road signs (146a, 146b, 146c) for the corresponding other lanes (I, II, III) and representing differences in the corresponding road rule information.

2. The driving assistance device (110) according to claim 1, characterized in that The sign determination unit (112) is adapted to determine multiple possible different specific types for a single road sign (146a, 146b, 146c), each specific type indicating different traffic rule information, wherein each specific type is associated with a different recognition probability based on the determination accuracy of the sign determination unit (112) for that specific type.

3. The driving assistance device (110) according to claim 2, characterized in that, Each specific type indicates a different speed limit on the road (140) and / or lane (I, II, III) on which the vehicle (100) is currently traveling.

4. The driving assistance device (110) according to claim 2, characterized in that The determination and adjustment unit (118) is adapted to adjust the determination result output by the sign determination unit (112) only if the difference between the highest recognition probability of the first set of multiple specific types of the single road sign (146a, 146b, 146c) and the lower recognition probability of the second set of multiple specific types of the single road sign (146a, 146b, 146c) is less than a predetermined threshold.

5. The driving assistance device (110) according to claim 4, characterized by The determination adjustment unit (118) is adapted to adjust the determination result output by the flag determination unit (112) only when the difference is less than 25%.

6. The driving assistance device (110) according to claim 5, characterized in that, The determination adjustment unit (118) is adapted to adjust the determination result output by the flag determination unit (112) only when the difference is less than 10%.

7. The driving assistance device (110) according to any one of claims 1 to 6, Its features are, The sign determination unit (112) is adapted to simultaneously determine multiple road signs (146a, 146b, 146c) and determine at least one specific type of each road sign (146a, 146b, 146c), wherein the driving assistance device (110) further includes A plausibility control unit (126) is adapted to re-examine the at least one specific type of road sign (146a, 146b, 146c) in light of existing road traffic rules.

8. The driving assistance device (110) according to claim 7, Its features are, The realism control unit (126) is adapted to indicate possible failures in the determination of the plurality of road signs (146a, 146b, 146c) when different specific types are determined for road signs (146a, 146b, 146c) fixedly arranged on one side (148a) of a portion (144) of the road (140) and for road signs (146a, 146b, 146c) fixedly arranged on the other side (148b) of the same portion (144) of the road (140).

9. The driving assistance device (110) according to claim 7, characterized in that The realism control unit (126) is adapted to indicate a possible failure in the determination of the plurality of road signs (146a, 146b, 146c) in the following situation: for a right-hand traffic system, for a first road sign (146a, 146b, 146c) displayed on the left in the traffic control and information system (150) and a second road sign (146a, 146b, 146c) displayed on the right in the same traffic control and information system (150), it determines two different specific types of road signs (146a, 146b, 146c), wherein the specific type of the road sign associated with the first road sign indicates a lower speed rule value than the speed rule value of the specific type of the road sign associated with the second road sign (146a, 146b, 146c).

10. The driving assistance device (110) according to claim 1, characterized in that The traffic behavior monitoring unit (114) is also adapted to Detect the behavior of at least two traffic participants (A, B); When a first traffic participant (A) is traveling in the same lane (I, II, III) in front of a second traffic participant (B), detect the distance between at least two traffic participants (A, B); At least one traffic participant (A, B) is identified as an independent traffic participant (A, B), wherein the independence of traffic participants (A, B) is determined when the distance between the at least two traffic participants (A, B) exceeds a predetermined distance, and / or when the second traffic participant (B) is accelerating or decelerating and the first traffic participant (A) maintains its speed for a predetermined time period; and The monitored behavior of only at least one independent traffic participant (A, B) is output to the comparison unit (116).

11. The driving assistance device (110) according to claim 10, characterized by The predetermined distance is a distance in meters that is at least half of the speed value of the second traffic participant (B) in km / h.

12. The driving assistance device (110) according to claim 10, characterized in that The traffic behavior monitoring unit (114) is also adapted to monitor the behavior of at least three independent traffic participants (A, B, C), wherein the comparison unit (116) is also adapted to compare the monitored behavior of the at least three independent traffic participants (A, B, C) and only consider behaviors that are similar to the majority of the monitored traffic participants (A, B, C).

13. The driving assistance device (110) according to claim 1, characterized in that The driving assistance device (110) also includes Traffic information receiving unit (130) is adapted to receive traffic information about the traffic conditions on the portion of the road (140) in front of the current position of the vehicle (100); as well as Traffic information consideration unit (132) is adapted to determine the predetermined traffic condition compliance behavior of traffic participants (A, B, C) based on the traffic information output by the traffic information receiving unit (130); The comparison unit (116) is further adapted to compare the monitored behavior of the at least one traffic participant (A, B, C) output by the traffic behavior monitoring unit (114) with the predetermined traffic condition conformity behavior output by the traffic information consideration unit (132); and The determination adjustment unit (118) is further adapted to discard the indicated mismatch between the determination result output by the sign determination unit (112) and the behavior of the at least one traffic participant (A, B, C) output by the traffic behavior monitoring unit (114) when the monitored behavior of the at least one traffic participant (A, B, C) matches the predetermined traffic condition.

14. The driving assistance device (110) according to claim 13, characterized by The traffic situation described is traffic congestion.

15. The driving assistance device (110) according to claim 1, characterized in that The driving assistance device (110) further includes a server connection unit (136), which is adapted to send to the server: The determination result of the specific type of the road signs (146a, 146b, 146c) output by the sign determination unit (112) and / or the determination adjustment unit (118); and The unique identifier corresponding to the road signs (146a, 146b, 146c) The server connection unit (136) is also adapted to receive from the server: Road sign information for different road signs (146a, 146b, 146c), indicating the specific type of the road sign (146a, 146b, 146c) previously determined by multiple other traffic participants (A, B, C); The determination adjustment unit (118) is further adapted to adjust the determination result output by the sign determination unit (112) when there is a mismatch between the determination result output by the sign determination unit (112) and the road sign information received from the server.

16. The driving assistance device (110) according to claim 15, characterized in that, The unique identifier of the corresponding road sign (146a, 146b, 146c) is the unique location on the road (140).

17. The driving assistance device (110) according to claim 15, characterized in that, The server connection unit (136) is also adapted to send to the server the date and time of the sent determination result.

18. A vehicle (100) comprising a driving assistance device (110) according to any one of claims 1 to 17.

19. A method for assisting a driver of a vehicle (100) in correctly observing the environment of a road (140) on which the vehicle (100) is currently traveling, the method comprising the steps of: A sign determination step (S400) in which at least one specific type of road sign (146a, 146b, 146c) is determined, wherein in the sign determination step (S400), the determination is based on data recorded using a camera unit (120) mounted in the vehicle (100) and recording images of the environment of the road (140) on which the vehicle (100) is currently traveling, and / or wherein in the sign determination step (S400), the determination is based on data obtained from map data regarding the road (140) on which the vehicle (100) is currently traveling. Traffic behavior monitoring step (S410), wherein the behavior of at least one traffic participant (A, B, C) is monitored, the at least one traffic participant being near the vehicle (100) and on the same road (140) as the vehicle (100) and traveling in the same direction (t). The comparison step (S420) involves comparing the determination result output in the sign determination step (S400) with the monitored behavior of the at least one traffic participant (A, B, C) output in the traffic behavior monitoring step (S410), and... The adjustment step (S430) involves adjusting the determination result output in the sign determination step (S400) if a mismatch is found in the comparison step (S420) between the determination result output in the sign determination step (S400) and the monitored behavior of the at least one traffic participant (A, B, C) output in the traffic behavior monitoring step (S410). If road signs (146a, 146b, 146c) are detected, and the specific type of the road signs (146a, 146b, 146c) for the lane (I, II, III) on which the vehicle (100) is currently traveling is different from the specific type of the road signs (146a, 146b, 146c) for other lanes (I, II, III), in the traffic behavior monitoring step (S410), only the same lane (I, II, III) of the road (140) on which the vehicle (100) is currently traveling is monitored. Traffic behavior near the vehicle (100) in the context of the vehicle (100), and / or suitable for using a multiplication factor to monitor the behavior of at least one traffic participant (A, B, C) located near the vehicle (100) in at least one lane (I, II, III) different from the lane (I, II, III) in which the vehicle (100) is currently traveling, the multiplication factor being associated with other specific types of road signs (146a, 146b, 146c) for the corresponding other lanes (I, II, III), and representing differences in the corresponding road rule information.

20. The method according to claim 19, characterized in that In the sign determination step (S400), multiple possible different specific types of a single road sign (146a, 146b, 146c) are also determined, each specific type indicating different traffic rule information, wherein each specific type is associated with a different recognition probability based on the determination accuracy of the sign determination step (S400) for that specific type.

21. The method according to claim 20, Its features are, In the determination and adjustment step (S430), the determination result output by the sign determination step (S400) is adjusted only if the difference between the highest recognition probability of the first complete set of multiple specific types of the single road sign (146a, 146b, 146c) and the lower recognition probability of the second complete set of multiple specific types of the single road sign (146a, 146b, 146c) is less than a predetermined threshold.

22. The method according to claim 21, characterized in that, In the determination adjustment step (S430), the determination result output in the flag determination step (S400) is adjusted only if the difference is less than 25%.

23. The method of claim 22, wherein, In the determination adjustment step (S430), the determination result output in the flag determination step (S400) is adjusted only if the difference is less than 10%.

Images