Traffic sign recognition

Abstract

Method (300) for driver assistance on board a motor vehicle (105), comprising the following steps: - determining (320) an impending change of direction of the motor vehicle (105) on the basis of information from a navigation system (115), wherein probabilities for several possible changes of direction are determined (320) and the change of direction with the highest probability is used (320); - selecting (340, 345, 350) a camera image of a front camera (125) or a side camera (130, 135) depending on the determined change in direction; - Determining (355) a traffic sign (210, 220, 230) in the surroundings of the motor vehicle (105) on the basis of the selected camera image.

Description

[0001] The invention relates to traffic sign recognition on board a motor vehicle. In particular, the invention relates to traffic sign recognition using several different cameras. State of the art

[0002] A traffic sign recognition system on board a motor vehicle captures an image from a forward-facing camera and attempts to identify a traffic sign in the captured image. The traffic sign can be a traffic light or a sign, for example. In the case of a traffic light, a current status can also be evaluated, i.e. it can be determined which signal is currently being given by the traffic light. The scanned information is usually presented to the driver of the motor vehicle visually. Other notification options are also possible, for example a haptic or acoustic warning can be given if the motor vehicle violates a rule such as a valid speed limit that was derived from a captured traffic sign.

[0003] The field of view of the camera that provides the image for traffic signs is usually limited to a predetermined angle. If the vehicle changes direction, for example, during a turn, a traffic sign may be difficult to detect for both the driver and the recognition system. As a result, the driver cannot be aware of the applicable traffic regulations, so there is a risk that the driver will drive the vehicle illegally despite the use of the recognition system.

[0004] WO 2011 / 105710 A3 relates to a display system with one forward-facing and two side-facing cameras. If a turn signal is detected that indicates a change in direction of the motor vehicle, the image from the corresponding side-facing camera is displayed on a monitor; otherwise, the image from the forward-facing camera is displayed.

[0005] US 2010 / 0033571 A1 shows a technique for determining an artificial horizon in order to be able to detect a traffic sign even if it is located on a curve.

[0006] US 5,530,421 A shows a monitoring system with multiple cameras, whose images are displayed on a monitor depending on a specific driving situation. In particular, a turn signal is used to select the image from a camera facing right or left.

[0007] From DE 10 2008 028 373 A1 a method for the combined output of an image and driving information is known.

[0008] From DE 10 2004 013 440 A1 a method and a device for specifying the driving instructions of a navigation system are known.

[0009] From DE 198 42 176 A1 a method and a device for traffic sign recognition and navigation is known.

[0010] The invention is based on the object of specifying a method, a computer program product, a device and a system for improved traffic sign recognition.

[0011] The invention achieves these objects by means of a method, a computer program product, a device, and a system having the features of the independent claims. Subclaims specify preferred embodiments. Disclosure of the invention

[0012] A method according to the invention for driver assistance on board a motor vehicle comprises steps of determining an impending change of direction of the motor vehicle on the basis of information from a navigation system, selecting a camera image from a front camera or a side camera depending on the determined change of direction and determining a traffic sign in the surroundings of the motor vehicle on the basis of the selected camera image.

[0013] By taking the information from the navigation system into account, it is possible to determine in an improved manner whether and, if so, in which direction a change in the vehicle's movement is imminent. By selecting the camera with the corresponding viewing angle, an image can be provided in which a traffic sign relevant to the vehicle's onward journey is located. The method can particularly assist the driver when the traffic situation is confusing at an intersection or when several traffic signs are visible after a change of direction, only a few of which need to be observed. The driver can be relieved of the burden of identifying a locally applicable traffic sign in addition to the road layout and the prevailing traffic, especially when turning or changing lanes.

[0014] In a preferred embodiment, the change of direction is additionally determined based on a turn signal. The turn signal can provide a good approximation of the driver's intentions, and the information from the turn signal can be correlated with the information from the navigation system, for example, to distinguish a stopping maneuver where no traffic sign is expected from a turning maneuver that may lead to a route with a traffic sign.

[0015] The navigation system can be configured to guide the driver to a destination, with the change of direction being determined based on a predetermined route from the current position of the vehicle to the destination. If the driver follows the predetermined route, the determination of changes of direction can be particularly simple and efficient. Traffic sign recognition can thus be performed even more efficiently.

[0016] In one embodiment, the change of direction is determined based on the position of the motor vehicle on a road network and information about permitted turns. This allows for an improved determination of which changes of direction are possible. In one embodiment, multiple changes of direction can be speculatively considered, allowing traffic signs to be detected even before the motor vehicle's change of direction is finally determined.

[0017] According to the invention, several possible changes of direction are determined, and for each of the changes of direction, a probability is determined with which it will be carried out. The change of direction underlying the selection of the camera image can be determined as the one with the highest probability. This embodiment can be combined, in particular, with the speculative consideration of several changes of direction. This can contribute to determining the correct change of direction early on, so that more time is available for determining applicable traffic signs or the detection of traffic signs can be carried out with improved accuracy.

[0018] In a further preferred embodiment, the navigation system is configured to determine an electronic horizon that represents a locally visible or relevant area of ​​an upcoming road. The change of direction is determined within the range of the electronic horizon. This can contribute to prioritizing upcoming opportunities for changing direction, thus enabling an improved determination of the change of direction.

[0019] In one embodiment, the output of a signal indicating a detected traffic sign is delayed until the motor vehicle has completed the specified change of direction. This prevents an incorrectly determined impending change of direction from leading to a signal indicating a traffic sign that is not relevant to the actual direction taken by the motor vehicle.

[0020] A computer program product comprises program code means for implementing the described method when the computer program product runs on a processing device or is stored on a computer-readable data carrier. In particular, the computer program product can be used to link a conventional traffic sign recognition system and a conventional navigation system in the manner described above. The computer program product can alternatively be executed on a processing device of the navigation system, the recognition system, or on a dedicated processing device.

[0021] A device according to the invention on board a motor vehicle comprises a first interface for obtaining information from a navigation system, a second interface for connecting to a recognition system for a traffic sign in the area of ​​the motor vehicle on the basis of a camera image and a processing device for determining an impending change of direction of the motor vehicle on the basis of the information obtained from the navigation system and for selecting a camera image of a front camera or a side camera for the recognition system on the basis of the determined change of direction.

[0022] The device can utilize elements or subsystems already installed on board a known motor vehicle to provide improved traffic sign recognition. Development and installation costs for the device can thus be reduced.

[0023] A system according to the invention for driver assistance on board a motor vehicle comprises a front camera, a side camera, a navigation system, and a recognition system configured to determine a traffic sign in the surroundings of the motor vehicle based on a camera image. Furthermore, a processing device configured to determine an impending change of direction of the motor vehicle based on information from the navigation system and to select a camera image from one of the cameras based on the determined change of direction. Short description of the characters

[0024] The invention will now be described in more detail with reference to the accompanying figures, in which: Fig. 1 a traffic sign recognition system on board a motor vehicle; Fig. 2 Situations in which the motor vehicle is Fig. 1 changes direction, and Fig. 3 a flowchart of a method for traffic sign recognition on board the motor vehicle of Fig. 1represents. Detailed description of embodiments

[0025] Fig. 1 shows a recognition system 100 for traffic signs on board a motor vehicle 105. The system 100 comprises a recognition system 110, a navigation system 115, and a selection device 120. The recognition system 110 is configured to recognize a traffic sign in a camera image from a camera on board the motor vehicle 105. A front camera 125, a left side camera 130, and a right side camera 135 are mounted on board the motor vehicle 105. The cameras 125 to 135 can serve additional purposes on board the motor vehicle 105, for example, to assist the driver when parking or leaving a parking space or to provide a panoramic image for the driver. An indication of a traffic sign recognized by the recognition system 110 can be output by means of an output device 140, wherein the output device 140 can provide an acoustic, optical, or haptic signal.The output device 140 can also be shared by other systems or assistants on board the motor vehicle 105; in another embodiment, the output device 140 is exclusively assigned to the recognition system 110.

[0026] The selection device 120 is configured to select which of the cameras 125, 130, or 135 an image should form the basis for the evaluation of the recognition system 110. In the illustrated embodiment, a switching device 145 is used to forward one of the images from the cameras 125 to 135 to the recognition system 110 based on the selection made by the selection device 120. The switching device 145 can be independent or, in another embodiment, can be included in the recognition system 110. The switching device 145 is connected to the selection device 120 via a first interface 150. A second interface 155 is provided for connecting the selection device 120 to the recognition system 110. A third interface 160 serves to connect the selection device 120 to the navigation system 115.

[0027] In another embodiment, the switching device 145 is included in the selection device 120, wherein the cameras 125 to 135 are directly connected to the selection device 120, and the selection device 120 provides the selected image to the recognition system 110. In yet another embodiment, the switching device 145 is included in the recognition system 110.

[0028] The navigation system 115 preferably comprises a satellite-based navigation system such as GPS, GLONASS, or GALILEO. It is also preferred that the navigation device 115 is configured to determine a route between a current position of the motor vehicle 105 and a destination and to output instructions for following the determined route to the driver of the motor vehicle 105. It is further preferred that the navigation system 115 has a memory for storing map data, which in particular includes information about a road network in the vicinity of the motor vehicle 105. The map data can also include information about route restrictions, for example, one-way streets or no-turn zones. Information from the navigation system 115 can be made accessible to the selection device 120 via the third interface 160.

[0029] In a preferred embodiment, the selection device 120 is additionally connected to a turn signal transmitter 165, which carries a turn signal which the driver of the motor vehicle 105 can set in order to indicate a change of direction or a turning maneuver to the outside.

[0030] Fig. 2 shows several exemplary situations in which the motor vehicle 105 from Fig. 1 changes direction. Fig. 2A shows motor vehicle 105 on a turning road 205 without any further branches. Beyond a curve is a traffic sign 210 indicating a route restriction or other traffic regulation applicable to road 205, such as a speed limit or a no-overtaking rule.

[0031] The course of the route 205 can be obtained from the map data of the navigation system 115 via the third interface 160 by the selection device 120. Based on this information, a change of direction to the right is determined in the situation shown, whereupon the right side camera 135 in Fig. 1 is connected to the detection system 110 by means of the switching device 145. The right side camera can already detect the traffic sign 210 before the change of direction of the motor vehicle 105 is completed.

[0032] As a result, the traffic sign 210 can be detected and evaluated by the recognition system 110 much earlier than would be possible with the front camera 125 alone.

[0033] Fig. Figure 2B shows another exemplary situation in which a second road 215 branches off to the right from road 205, which motor vehicle 105 is currently using. If motor vehicle 105 follows road 205 straight ahead, traffic sign 210 applies to motor vehicle 105. However, if it turns right onto the second road 215, a second traffic sign 220 applies there.

[0034] If a route is stored in the navigation system 115 that indicates to the driver of the motor vehicle 105 which of the roads 205 or 215 he or she should follow, this road 205, 215 can be determined as the upcoming direction of the motor vehicle 105 by the selection device 120. If the motor vehicle 105 is to follow road 215 according to the route, the right side camera 135 is selected; otherwise, the front camera 125 is selected.

[0035] If no route is provided in the navigation system, the probability of using routes 205, 215 can be determined based on additional information. The route 205, 215 with the highest probability then forms the basis for the change of direction determined by the selection device 120. Route 205 can be determined as probable, for example, if it has a higher traffic importance than route 215 or if the speed of motor vehicle 105 in the area of ​​the intersection of routes 205 and 215 makes a turn unlikely.

[0036] Fig. Figure 2C shows another situation similar to that of Fig. 2B, but with an additional third road 225 to the left of the motor vehicle 105. If the motor vehicle 105 follows the third road 225, a third traffic sign 230 applies there. The upcoming road 205, 215 or 225 can be determined, for example, on the basis of a set turn signal of the turn signal transmitter 165 and a change of direction of the motor vehicle 105 can be derived from the upcoming road 205, 215 or 225.

[0037] In situations with more than one possibility for a change of direction (cf. Fig. 2B, Fig. 2C) several changes of direction can also be determined and an evaluation of the corresponding camera 125 to 135 can be carried out speculatively.

[0038] Fig. Figure 3 shows a flowchart of a method 300 for improved traffic sign recognition. In an optional step 305, an electronic horizon is determined based on information from the navigation system 115, with only information in the range of the electronic horizon being considered below.

[0039] In a subsequent step 310, it is determined whether a course change is possible. This determination is preferably made based on map information from the navigation system 115. In one embodiment, prohibitions such as a no-turn rule are taken into account in this step.

[0040] In a subsequent step 315, probabilities are determined for all possible changes of direction from step 310. In a subsequent step 320, an impending change of direction of the motor vehicle 105 is determined on the basis of the available information. Preferably, the change of direction for which the highest probability was determined is selected. The selection in step 320 can additionally be determined on the basis of a step 325 by evaluating a turn signal of the turn signal transmitter 165, as described above with reference to Fig. 2. Furthermore, in a step 330, a route predetermined in the navigation system 115 can additionally be used as the basis for the selection. It is then assumed that the motor vehicle 105 is following the determined route, from which an impending change of direction is deduced.

[0041] In a step 335, the impending directional change is determined. If a leftward directional change is imminent, the left side camera 130 is selected in a step 340. If a rightward directional change is imminent, the right side camera 135 is selected in a step 345. If there is no rightward or leftward directional change, the front camera 125 is selected in a step 350.

[0042] Subsequently, in a step 355, the traffic sign 210, 220, or 230 is recognized by the recognition system 110, and, if appropriate, a message is displayed to the driver of the motor vehicle 105 by means of the output device 140. The output of the message can be delayed by a corresponding control signal via the second interface 155 by the selection device 120 until the motor vehicle 105 has already performed the determined change of direction to a predetermined degree. This can be particularly useful if, in step 320, several changes of direction were determined to be probable and a speculative execution of steps 325 to 350 was performed for different scenarios.

Claims

[1] Method (300) for driver assistance on board a motor vehicle (105), comprising the following steps: - determining (320) an impending change of direction of the motor vehicle (105) on the basis of information from a navigation system (115), wherein probabilities for several possible changes of direction are determined (320) and the change of direction with the highest probability is used (320); - selecting (340, 345, 350) a camera image of a front camera (125) or a side camera (130, 135) depending on the determined change in direction; - Determining (355) a traffic sign (210, 220, 230) in the surroundings of the motor vehicle (105) on the basis of the selected camera image. [2] Method (300) according to claim 1, wherein the change of direction is additionally determined (325) on the basis of a turn signal (165). [3] Method (300) according to claim 1 or 2, wherein the navigation system (115) is configured to guide to a destination and the change of direction is determined (330) on the basis of a predetermined route from a current position of the motor vehicle (105) to the destination. [4] Method (300) according to one of the preceding claims, wherein the change of direction is determined (320) on the basis of a position of the motor vehicle (105) on a road network (205, 215, 225) and information on permitted turning maneuvers. [5] Method (300) according to one of the preceding claims, wherein the navigation system (115) is configured to determine an electronic horizon (305) and the change in direction is determined in the region of the electronic horizon (320). [6] Method (300) according to one of the preceding claims, wherein an output (355) of a signal indicating a recognized traffic sign is delayed until the motor vehicle (105) has performed the determined change of direction. [7] Computer program product with program code means for carrying out the method (300) according to one of the preceding claims, if the computer program product runs on a processing device (110, 115, 120) or is stored on a computer-readable data carrier. [8] Device (110) on board a motor vehicle (115), comprising: - a first interface (160) for obtaining information from a navigation system (115); - a second interface (155) for connection to a recognition system (110) for a traffic sign (210, 220, 230) in the area of ​​the motor vehicle (105) on the basis of a camera image; - a processing device (120) for determining an impending change of direction of the motor vehicle (105) on the basis of the information obtained from the navigation system and for selecting a camera image from a front camera (125) or a side camera (130, 135) for the recognition system (110) on the basis of the determined change of direction, wherein probabilities for several possible changes of direction are determined (320) and the change of direction with the highest probability is used (320). [9] A system (100) for driver assistance on board a motor vehicle (105), comprising: - a front camera (125); - a side camera (130, 135); - a navigation system (115); - a recognition system (110) configured to determine a traffic sign (210, 220, 230) in the surroundings of the motor vehicle (105) on the basis of a camera image; characterized by - a processing device for determining an impending change of direction of the motor vehicle on the basis of information from the navigation system, wherein probabilities for several possible changes of direction are determined (320) and the change of direction with the highest probability is used (320), and - to select a camera image from one of the cameras based on the determined change in direction.

Images