Method and device for determining vehicle-relevant traffic signs

Abstract

Method for determining the traffic signs relevant to a vehicle, wherein the lanes (40 to 44, 54 to 58) are recorded at least in the direction of travel of the vehicle; at least the traffic signs relevant to the direction of travel of the vehicle (46 to 52, 62 to 66) are recorded; wherein the traffic signs (46 to 52, 62 to 66) assignable to one of the identified lanes (40 to 44, 54 to 58) are recorded lane-specifically, wherein The lane (40 to 44, 54 to 58) is determined using a navigation system and a sequence of images captured by a camera system (12) of the area in front of the vehicle, wherein a lane (40 to 44, 54 to 58) determined using the sequence of images captured by the camera system (12) is checked using the lane (40 to 44, 54 to 58) determined using the navigation system or where the lane (40 to 44, 54 to 58) detected by the navigation system is checked against the lane (40 to 44, 54 to 58) determined by the image sequence detected by the camera system, where the traffic signs (46 to 52, 62 to 66) assigned to a determined lane (40 to 44, 54 to 58) are stored lane-related and the identified lane (40 to 44, 54 to 58) is selected as the current lane (40 to 44, 54 to 58) of the vehicle to which a lane change is intended, and information about the identified traffic signs (46 to 52, 62 to 66) relating to the current lane (40 to 44, 54 to 58) is displayed, the intended lane change (40 to 44, 54 to 58) being determined by an activated turn signal, and the information about the identified traffic signs is given to a driver by displaying a recording of the traffic sign (46 to 52, 62 to 66) taken with the help of the vehicle's camera system (12), and Additional information will be provided about the identified traffic signs (46 to 52, 62 to 66) that could not be clearly assigned to the current lane (40 to 44, 54 to 58).

Description

[0001] Several systems for automated traffic sign recognition are known, which can detect at least the traffic signs relevant to a vehicle's direction of travel. Furthermore, information about recognized traffic signs is displayed to the driver in the vehicle via a display unit and / or processed further by driver assistance systems.

[0002] The traffic sign recognition systems known in the prior art, such as those known from documents EP 1 145 186 B1 and EP 1 383 098 B1, detect and analyze all traffic signs present in a vehicle's driving tube.

[0003] DE 103 34 620 A1 describes a method for detecting traffic signs in synchronization with the current driving movement. The relevance and validity of the traffic signs are checked and determined. However, the document does not describe a method for recognizing and validating traffic lanes.

[0004] US 2007 / 0043491A1 describes a system for capturing and analyzing scenarios outside the vehicle. This analysis is fused with information from the vehicle's internal sensors and with driver information. The document does not describe any prioritization of traffic signs.

[0005] DE 10 2005 035 342 A1 describes a device for wireless emergency signal transmission to or between vehicles. The device comprises a transponder receiving module and a transponder transmitting unit, which can be installed in the road surface in each traffic lane.

[0006] DE 10 2005 007 802 A1 describes a method for lane recognition. The document does not describe how traffic signs could be assigned to a lane.

[0007] Due to the large number of traffic signs present in the vehicle's lane, the driver is also confronted with traffic sign information about signs that are not relevant to him because of the vehicle's current lane.

[0008] The object of the invention is to automatically determine the traffic signs relevant to a vehicle in a simple manner.

[0009] This problem is solved by a method with the features of claim 1 and a device with the features of claim 4. Advantageous embodiments of the invention are specified in the dependent claims.

[0010] A method comprising the features of claim 1 and a device comprising the features of claim 4 enable the simple determination of traffic signs relevant to a vehicle by identifying and assigning to a specific lane the traffic signs assigned to that lane. For this purpose, the lanes of the roadway are identified, at least in the direction of travel of the vehicle. This allows, in particular, lane-dependent commands and prohibitions for adjacent lanes in the same direction of travel to be identified and assigned to the lane to which they are legally assigned. For example, traffic signs generated by variable message signs on overhead gantries of traffic management systems are each positioned above the lane to which they are assigned.The traffic signs generated by these traffic signal systems are primarily prohibition signs, mandatory signs, warning signs, and / or information signs, which, according to traffic law, take precedence over static traffic signs at the roadside. By detecting lane-specific traffic signs, the driver can be provided with precise information about the applicable prohibitions or mandatory signs for the lane in which the vehicle is currently located, is changing lanes to, or intends to change lanes.

[0011] Furthermore, lanes such as turning lanes and exit lanes on highways and expressways often have different restrictions than the main lanes of the highway or expressway itself. When a vehicle turns into a turning lane or an exit lane, information about the rules and / or prohibitions applicable to that lane can be displayed. The individual lanes are preferably identified using a lane model derived from images of the area in front of the vehicle captured by the vehicle's camera system and / or from data transmitted by the vehicle's navigation system.

[0012] In addition, preferably all relevant classes of traffic signs arranged in the vehicle's lane are detected. The lane to which the vehicle intends to change is selected as the current lane. The intended lane change is determined by an activated turn signal. Preferably, only information about the detected traffic signs pertaining to the current lane(s) is output and / or processed. This ensures that only the information relevant to the vehicle's requirements is actually output and / or processed.

[0013] Additionally, information about detected traffic signs that apply to all lanes can be displayed. Furthermore, information about detected traffic signs that could not be clearly assigned to the current lane is also displayed. This ensures that information about traffic signs that could not be clearly assigned to any lane and that apply to all lanes is also displayed and / or processed. This information is preferably only processed and / or displayed if the traffic signs clearly assigned to a specific lane do not have priority over these other traffic signs under traffic law.

[0014] The traffic signs assigned to a specific lane are stored lane-specifically. This ensures that these traffic signs are available as information when the vehicle later changes lanes and can be displayed and / or further processed.

[0015] Furthermore, it is advantageous that, before a lane change by the vehicle, information about traffic signs for the lane currently in use before the lane change, that after a lane change by the vehicle, information about traffic signs for the lane currently in use after the lane change, and that, when the vehicle intends to change lanes, information about traffic signs for the lane currently in use before and after the lane change, or only for the lane currently in use after the lane change, is displayed, in particular on a display unit, or transmitted to another control unit and / or to another control process. This ensures that only the traffic signs relevant to the current traffic situation are displayed and / or processed.

[0016] Information about the detected traffic sign(s) is displayed by showing an image of the traffic sign captured by the vehicle's camera system. Additionally, further information about the detected traffic sign(s) can be provided to the driver by displaying a symbol of the traffic sign, by displaying text associated with that traffic sign, and / or by providing an audible message. Alternatively or additionally, the information about the lane-specific traffic signs detected is further processed.

[0017] The lane can be detected using either the navigation system or the image sequence captured by the camera system. For each detected lane, an individual probability is determined that this lane is the current lane in which the vehicle is located, with the lane with the highest probability preferably being identified as the current lane. To this end, the individual probability that this lane is the current lane is preferably determined for each detected lane.

[0018] A device for determining the traffic signs relevant to a vehicle, comprising the features of claim 4, can achieve the same advantages as the method according to claim 1. Furthermore, this device can be further developed in the same manner as specified for the method according to claim 1. In particular, the device comprising the features of claim 4 can also be further developed with the features specified in the dependent claims for the further development of the method according to claim 1.

[0019] Further features and advantages of the invention will become apparent from the following description, which, in conjunction with the accompanying figures, explains the invention in more detail using an exemplary embodiment.

[0020] They show: Fig. 1 a block diagram of a system for image acquisition and image analysis of the surroundings of a motor vehicle; Fig. 2 a schematic representation of a section of a two-lane roadway with an exit; and Fig. 3 A schematic representation of a section of a three-lane roadway with a sign bridge for displaying lane-specific traffic signs.

[0021] In Fig. Figure 1 shows a block diagram of an image acquisition and evaluation system 10 integrated into a motor vehicle, in particular a car or truck. The image acquisition and evaluation system 10 has a camera system 12 that captures a sequence of images depicting at least one area in front of the vehicle and generates corresponding image data for each image. This image data is transmitted to an image processing unit 14 of the system 10, which analyzes and processes the image data using control and processing modules 16a to 16d.

[0022] The image processing unit 14 is connected via a vehicle bus 18 to other control and processing units 20 to 26 of the vehicle, such as the navigation system, the speed detection and / or control system, and the image processing unit 14 can exchange data with these other control units 20 to 26 via the vehicle bus 18. The image processing unit 14 is also connected via a suitable data line to a display and warning unit 28 and to a speed control and / or limiter 30. Furthermore, the image processing unit 14 is connected via a data line to a permanent storage device 32. Such a permanent storage device 32 can be, for example, a hard disk, flash memory, or another control unit. The permanent storage device 32 can also be provided by a vehicle database system.The camera system 12 can comprise a monocular camera, several monocular cameras, a stereo camera system, and / or several stereo camera systems, with the individual cameras producing grayscale or color images. Commonly known bus systems, such as a LIN bus, a CAN bus, and / or a FlexRay bus, can be used as the vehicle bus 18. The optics of the camera(s) of the camera system 12 can, in particular, have different fixed focal lengths or a focal length that is adjustable, especially via a turret lens system or a varifocal lens system.

[0023] The image processing unit 14 also communicates with other sensors and receivers via the vehicle bus 18, and in particular transmits information about signal states of individual actuators and sensors, especially about the activation of the vehicle's direction indicators.

[0024] Using the image sequence captured by the camera system 12, the image data is analyzed by the image processing unit 14 to identify and classify the traffic signs within the camera system's detection range. By classifying the traffic signs using a suitable recognition method, the depicted traffic sign is identified as a specific traffic sign. The image of the traffic sign is displayed via the display and warning unit 28. Optionally, a symbol corresponding to this traffic sign is also displayed.

[0025] Additionally, text and / or audible information can be output. Furthermore, traffic information, particularly regarding speed limits or minimum speeds, is transmitted to the speed control and / or limiter 30, which can issue a warning signal if the maximum permitted speed is exceeded and / or a minimum speed is undershot. Alternatively, the speed limits can be used as the maximum limit values ​​for a speed control and / or limiter, possibly with an offset.

[0026] A further processing and control module 16a to 16d of the image processing unit 14 determines the vehicle's current lane based on the images in the image sequence. Another processing module 16a to 16d assigns the traffic signs that can be uniquely assigned to one of the determined lanes only to that lane, and these traffic signs assigned to only one lane, along with the lane information, are stored in permanent memory 32. Furthermore, comparison data for pattern recognition of traffic signs, in particular country-specific patterns of traffic signs, symbols that can be displayed on the display unit 28 instead of the images of the recognized traffic signs, as well as text and / or acoustic information to be displayed, can be stored in permanent memory 32.

[0027] In Fig. Figure 2 is a schematic representation of a section of a two-lane carriageway, showing two traffic signs 46 and 48 limiting the maximum speed to 120 km / h, positioned at approximately the same height on the right and left edges of the carriageway. By placing two traffic signs 46 and 48 on the right and left sides of the carriageway, it is prevented that a traffic sign 46 or 48 could be overlooked because it is obscured by another vehicle. This is particularly likely to occur if, on a multi-lane carriageway, only one traffic sign 48 is positioned on the right edge of the carriageway and a vehicle is traveling in the right lane that obscures the sign. This is especially true if a truck is traveling in the right lane, thus obscuring the view of the traffic sign 48 on the right edge of the carriageway for a vehicle traveling in the left lane.Traffic signs 46 and 48 can be designed, in particular, as permanent traffic signs or as traffic signal systems that can be used to generate various traffic signs. The [details of the traffic sign system] are as follows: Fig. Figure 2 shows a two-lane carriageway with an exit lane 44 on the right-hand side of the carriageway, where the maximum speed in exit lane 44 is limited to 60 km / h. Two traffic signs 50 and 52 are provided to limit the maximum speed in the area of ​​exit lane 44, with the traffic signs 50 and 52 being arranged opposite each other at approximately the same height on exit lane 44. According to the invention, lane detection is carried out using the image sequence recorded by the camera system 12 in order to distinguish between the two driving lanes 40 and 42 and the exit lane 44. Furthermore, traffic signs 46 to 52 are detected via the traffic sign recognition module 16a to 16d, whereby traffic signs 46 and 48 are assigned to lanes 40 and 42, respectively, and traffic signs 50 and 52 to exit lane 44. Lanes 40 and 42 are lanes of the carriageway, and exit lane 44 can also be referred to as a lane.

[0028] In Fig. Figure 3 is a schematic representation of a section of a three-lane roadway with lanes 54 to 58. A sign bridge 60 is positioned above lanes 54 to 58, capable of displaying lane-specific variable traffic signs. Currently, the sign bridge displays traffic signs 62 to 66. Traffic sign 62 prohibits drivers of motor vehicles with a permissible total weight exceeding 3.5 metric tons, including their trailers, and of tractors (excluding passenger cars and buses), from overtaking multi-track vehicles and motorcycles with sidecars. This traffic sign 62 applies to all lanes 58 to 62. Furthermore, traffic sign 64 is assigned to the right and center lanes 56 and 58, prohibiting traffic on lanes 56 and 58 from exceeding a speed of 90 km / h.Traffic sign 66 is assigned only to lane 54 and prohibits driving faster than 120 km / h on lane 54. Traffic signs 64 and 66 are therefore not assigned to all lanes 54 to 58. According to the invention, a system 10 integrated into a motor vehicle for image acquisition and evaluation of the vehicle's surroundings detects the lanes 40 to 44 and 54 to 58 in the area in front of the vehicle, as well as the traffic signs 46 to 52 and 62 to 66 located in and alongside the lanes. The area in which traffic signs 46 to 52 and 62 to 66 and the lanes are detected is preferably defined by means of a so-called "driving tube," which includes at least the lanes 46 to 52 and 62 to 66 in the direction of travel of the vehicle, as well as the areas alongside the roadway where traffic signs are usually located.

[0029] Lane detection determines which lane (40-44 or 54-58) the vehicle with integrated system 10 is currently in. If the vehicle is a passenger car and is in lane 58, the driver will not receive any information about traffic sign 62, as this sign does not apply to this vehicle. However, if the vehicle is a truck, the driver will receive information about traffic sign 62. Specifically, a symbol of traffic sign 62 will be displayed on an output unit. Additionally or alternatively, an acoustic signal, particularly a voice announcement with information about the overtaking prohibition, can be provided.

[0030] If the vehicle is currently in lane 56, the driver will be informed of the speed limit of 90 km / h.

[0031] If the vehicle is a truck, the driver will change lanes. If the vehicle is currently in lane 54, the system will determine the speed limit for that lane, which is set to 120 km / h, as it is determined that traffic sign 66 is assigned to lane 54. The driver will then be informed of the 120 km / h speed limit. If the vehicle is a truck, the driver will also be notified of the applicable overtaking ban. The driver may then be instructed to move to the right-hand lane 58. This ensures that only the traffic signs relevant to the specific vehicle are processed and / or displayed. For example, a filter module 16a to 16d can be used to filter out traffic signs that are not relevant to the respective vehicle.

[0032] In the present embodiment, traffic sign 62 applies to all three lanes, and traffic sign 64 applies to lanes 58 and 56, since no further speed limit is specified for the right-hand lane 58 in this embodiment. However, traffic sign 62 is only displayed to the driver if the vehicle is a truck over 3.5 t covered by the prohibition. Otherwise, this traffic sign is considered irrelevant to the vehicle and filtered out. Regardless of the vehicle type, however, every driver in the right-hand lane 58 or middle lane 56 is informed of the speed limit of 90 km / h, and in the left-hand lane of the speed limit of 120 km / h.If a driver changes lanes from the right lane 58 or the middle lane 56 to the left lane 54 before or after overhead sign 60, they will no longer be shown the 90 km / h speed limit but rather the applicable speed limit of 120 km / h. Information about the speed limits indicated by traffic signs 64 and 66, or the traffic signs themselves, is stored lane-specifically. Traffic sign 64 is thus stored as being assigned to lanes 58 and 56, and traffic sign 66 is stored as being assigned to lane 54.

[0033] Similarly, in the schematic representation according to Fig.The traffic signs 50 and 52 are stored and assigned to exit lane 44, and traffic signs 46 and 48 are stored and assigned to lanes 40 and 42. This allows drivers to receive situationally relevant information, as the information currently relevant to the vehicle can be selected. This is made possible by the integration of lane detection and traffic sign recognition systems, combining them in such a way that traffic signs applicable only to a portion of the lanes are detected and preferably stored on a lane-specific basis.

[0034] In particular, a navigation system can be used to determine navigation information about the lane or road layout, enabling lane detection or improving and / or verifying lane detection performed using a camera and image processing system. This allows traffic signs generated by overhead sign gantries (e.g., traffic lights) to be detected lane-specifically. These traffic lights are also known as variable message signs and serve as a traffic control system, preferably coupled with a traffic management system.

[0035] The lanes are preferably identified using a lane model to determine which lane (40 to 44, 54 to 58) is currently being used by the vehicle. Furthermore, the presence and course of adjacent lanes are determined. Information about on- and off-ramps is also gathered. Preferably, only the relevant traffic signs, in particular prohibition, warning, and mandatory signs, that are currently relevant to the vehicle or driver are processed and / or displayed. For example, traffic signs with regional information are not further classified.

[0036] If the traffic signs identified for individual lanes are stored lane-specifically, this information can then be used in the event of a subsequent lane change, particularly as information provided to the driver. When a driver intends to change direction, information about the lane they plan to move into can be displayed. Alternatively, the driver can be given information about the prohibitions and requirements of the current lane and the lane they intend to move into. The intended lane change can be determined, for example, by an activated turn signal. The identified traffic signs assigned to a lane can be checked for plausibility.Furthermore, identified traffic signs can only be assigned to a specific lane if the unambiguous assignment to that lane has been determined with a high degree of probability. Additionally, navigation information from the vehicle's navigation system can provide further details about the road type, such as motorway or rural road, which can be taken into account when identifying the lanes. If information about the country where the vehicle is currently located is also available, this country-specific information can be used to identify the lane-specific traffic signs. In particular, this country-specific information includes details about whether traffic drives on the right or left.

Claims

[1] Method for determining the traffic signs relevant to a vehicle, wherein the lanes (40 to 44, 54 to 58) are recorded at least in the direction of travel of the vehicle; at least the traffic signs relevant to the direction of travel of the vehicle (46 to 52, 62 to 66) are recorded; wherein the traffic signs (46 to 52, 62 to 66) that can be assigned to one of the identified lanes (40 to 44, 54 to 58) are recorded lane-specifically, wherein The lane (40 to 44, 54 to 58) is determined using a navigation system and a sequence of images captured by a camera system (12) of the area in front of the vehicle, wherein a lane (40 to 44, 54 to 58) determined using the sequence of images captured by the camera system (12) is checked using the lane (40 to 44, 54 to 58) determined using the navigation system or where the lane (40 to 44, 54 to 58) detected by the navigation system is checked against the lane (40 to 44, 54 to 58) determined by the image sequence detected by the camera system, where the traffic signs (46 to 52, 62 to 66) assigned to a determined lane (40 to 44, 54 to 58) are stored lane-related and the identified lane (40 to 44, 54 to 58) is selected as the current lane (40 to 44, 54 to 58) of the vehicle to which a lane change is intended, and information about the identified traffic signs (46 to 52, 62 to 66) relating to the current lane (40 to 44, 54 to 58) is displayed, the intended lane change (40 to 44, 54 to 58) being determined by an activated turn signal, and the information about the identified traffic signs is given to a driver by displaying a recording of the traffic sign (46 to 52, 62 to 66) taken with the help of the vehicle's camera system (12), and Additional information will be provided about the identified traffic signs (46 to 52, 62 to 66) that could not be clearly assigned to the current lane (40 to 44, 54 to 58). [2] Method according to claim 1, characterized by , that the lanes (40 to 44, 54 to 58) are detected by taking an image using the vehicle's camera system (12) in the area in front of the vehicle, whereby the lanes (40 to 44, 54 to 58) are identified using a lane model. [3] Method according to any one of the preceding claims, characterized by, that both permanently installed traffic signs (46 to 52, 62 to 66) and traffic signs (46 to 52, 62 to 66) generated by means of traffic signal systems are recorded, whereby the traffic signs generated by means of traffic signal systems (46 to 52, 62 to 66) are processed as having priority over the permanently installed traffic signs (46 to 52, 62 to 66). [4] Device for determining the traffic signs relevant to a vehicle, comprising a first detection module (16a) for detecting the lanes (40 to 44, 54 to 58) at least in the direction of travel of the vehicle; comprising a second detection module (16b) for detecting at least the traffic signs (46 to 52, 62 to 66) relevant to this direction of travel; comprising a processing module (16c) which assigns traffic signs (46 to 52, 62 to 66) that can be assigned to one of the detected lanes (40 to 44, 54 to 58) to that lane (40 to 44, 54 to 58), where the processing module (16c) determines the lane (40 to 44, 54 to 58) using a navigation system and using an image sequence of the area in front of the vehicle captured by a camera system (12), wherein a lane (40 to 44, 54 to 58) determined using the image sequence captured by the camera system (12) is checked using the lane (40 to 44, 54 to 58) determined using the navigation system, or wherein the lane (40 to 44, 54 to 58) determined using the navigation system is checked using the image sequence captured by the camera system. where the device is configured to store the traffic signs (46 to 52, 62 to 66) assigned to a determined lane (40 to 44, 54 to 58) in a lane-specific manner and to select the determined lane (40 to 44, 54 to 58) as the current lane (40 to 44, 54 to 58) of the vehicle to which a change of lane is intended, and to output information about the determined traffic signs (46 to 52, 62 to 66) relating to the current lane (40 to 44, 54 to 58), wherein the device is configured to determine the intended change of lane (40 to 44, 54 to 58) by means of an activated direction indicator, the device is designed to output the information about the identified traffic signs to a driver by displaying a recording of the traffic sign (46 to 52, 62 to 66) taken with the help of the vehicle's camera system (12), and the device is set up to output additional information about the identified traffic signs (46 to 52, 62 to 66) that could not be clearly assigned to the current lane (40 to 44, 54 to 58).

Images