Device and method for recognizing traffic signs

Abstract

Device for detecting a traffic sign located on a road, comprising: an information acquisition unit that captures a front image received from a front camera of a vehicle as a single image unit; an image processing unit (215) that detects traffic sign information, which detects a motion vector value and a motion speed value of the traffic sign contained in the front image by means of an object detection algorithm from a difference between a previous frame and a current frame; and a filter (231,232) which filters out an irrelevant traffic sign by comparing vehicle driving information supplied by an electronic control unit arranged in the vehicle, which includes a yaw rate value and a vehicle speed value of the vehicle, with the traffic sign information.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

[0001] This application claims priority under 35 USC § 119 over Korean patent application No. 10-2013-0107862, filed on September 9, 2013 with the Korean Intellectual Property Office, the contents of which are hereby incorporated in their entirety by reference into the present application. TECHNICAL AREA

[0002] The present invention relates to a device and a method for detecting a traffic sign on a road and in particular a device and a method for detecting traffic signs that display information relevant to a driver, such as several speed limit signs on a road, and for transmitting the information displayed on the traffic sign to a vehicle. BACKGROUND

[0003] More recently, various functionalities have been developed that use a front camera installed on the vehicle to provide warning functions, which indicate the possibility of lane deviation, vehicle collision, or similar situations, in order to make things easier for the driver.

[0004] Furthermore, there is a speed limit sign recognition function which, while the vehicle is driving, uses the vehicle's front camera to detect speed limit signs on the road and transmits the relevant information to the driver to avoid dangerous situations such as excessive speed. This speed limit sign recognition function is referred to as traffic signal recognition (TSR) technology.

[0005] The TSR technology is currently commercially available, and the existing speed limit traffic sign recognition algorithm has a general recognition functionality that detects an area in which the traffic sign is located from a basic image using a pattern recognition algorithm, recognizes the content of the corresponding traffic sign in the detected area, and transmits the content of the corresponding traffic sign to the driver when the vehicle passes the corresponding traffic sign while driving.

[0006] On a real road, however, different traffic signs are assigned to the respective lanes, and in the case of an intersection, the front camera, in addition to the speed limit traffic sign corresponding to a real-time direction in which the vehicle is currently moving, also detects traffic signs installed for vehicles moving in other directions, and consequently, several speed limit traffic signs are often included in a single image.

[0007] As in Fig. As shown in Figure 1, a sticker 10 indicating a maximum speed limit of a vehicle may be affixed to a rear area of ​​the vehicle, such as a bus or a large truck.

[0008] However, since the sticker affixed to the rear of the vehicle has the same shape as the traffic sign on the road, there is a possibility that the sticker will be mistakenly recognized as a typical traffic sign.

[0009] As listed above, traffic sign recognition technology, according to the relevant prior art, is very useful when a single traffic sign is located on a real road. However, in cases where several traffic signs are located on a real road, it is difficult to determine which traffic sign displays the most necessary information for the driver and to provide the driver with the driving information required for safe driving. DE 102 00 784 A discloses a device for recognizing a traffic sign located on a road, wherein front images are created by a vehicle's front camera, traffic signs are recognized in successive front images, and the change in their size is evaluated together with the vehicle speed to infer whether the traffic signs are moving.US 2010 / 0067805A1, US 8050460B2, and DE 102007021580A1 disclose further devices for detecting traffic signs. DE 102007021576A1 discloses a device for detecting a traffic sign located on a road, wherein a plausibility check of the determined position of a traffic sign is performed in successive images captured by a camera system by comparing the determined change in position in the image sequence with the movement of the vehicle. In particular, it is checked whether the determined position is actually within the recording area of ​​the camera system or whether the determined position is a virtual position outside the image recording area of ​​the camera system. OVERVIEW

[0010] The present invention was conceived with the aim of providing a device and a method for recognizing a traffic sign on a road, with which irrelevant traffic signs are filtered out when, during a process for recognizing a traffic sign pattern present in an image, this image contains several traffic signs.

[0011] According to an exemplary embodiment of the present invention, a device for detecting a traffic sign located on a road is provided, comprising: an information acquisition unit that captures a front image obtained from a front camera of a vehicle as a single image unit; an image processing unit that detects traffic sign information, which uses an object detection algorithm to determine a motion vector value and a motion speed value of the traffic sign contained in the front image from a difference between a previous single image and a current single image; and a filter that filters out an irrelevant traffic sign by comparing vehicle driving information supplied by an electronic control unit arranged in the vehicle, which includes a yaw rate value and a vehicle speed value of the vehicle, with the traffic sign information.

[0012] According to a further exemplary embodiment of the present invention, a method for detecting a traffic sign located on a road is provided, comprising the following steps: capturing a front image obtained from a front camera of a vehicle as a single image unit; detecting a motion vector value and a motion speed value of the traffic sign contained in the front image from a difference between a previous single image and a current single image using an object detection algorithm; and filtering out an irrelevant traffic sign by comparing vehicle driving information, which includes a yaw rate value and a vehicle speed value of the vehicle, with the traffic sign information.

[0013] The present invention prevents a situation in which, in a state where a traffic sign recognition (TSR) system is installed on the vehicle, the driver may receive incorrect information due to various misrecognitions of a traffic sign, and the invention improves the reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 shows a view of a sticker image on the rear of a large-scale vehicle, according to an exemplary embodiment of the present invention. Fig. Figure 2 shows a block diagram of a device for detecting a traffic sign, according to the exemplary embodiment of the present invention. Fig. Figure 3 shows a view of a situation in which the possibility of a problem exists on an access road, according to the exemplary embodiment of the present invention. Fig. Figure 4 is a flowchart of an operation for recognizing a traffic sign according to the exemplary embodiment of the present invention. DETAILED DESCRIPTION OF EXECUTION FORMS

[0014] The advantages and features of the present invention and methods for realizing these advantages and features will become apparent with reference to the exemplary embodiments described in detail below in conjunction with the accompanying drawings. However, the present invention is not limited to the exemplary embodiments listed here, but can be implemented in various forms. The exemplary embodiments described here are intended solely to fully illustrate the disclosed content and to provide those skilled in the field with a comprehensive and sufficient understanding of the substance of the present invention. Thus, the present invention is defined only by the wording of the claims.

[0015] The terms used in this description serve to illustrate the exemplary embodiments and are not to be understood as limiting the present invention. Unless expressly stated otherwise in this description, a singular form also includes the plural form. The words "feature" and / or "with" in this description do not exclude the presence or addition of the listed components, steps, processes, and / or devices and / or one or more other components, steps, processes, and / or devices. An exemplary embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.

[0016] The present invention serves to select, in a real vehicle driving situation, from a number of traffic signs arranged on a road, a traffic sign that displays the most relevant information for a driver, and to transmit the information displayed on the traffic sign to a driver and a vehicle.

[0017] In particular, according to the present invention, to prepare for a scenario where incorrect traffic sign information is transmitted and to improve the reliability of a traffic sign recognition (TSR) result, an unnecessary traffic sign pattern is first filtered out in an image using dynamic information from the vehicle. Furthermore, a second filtering process is performed in which shapes of figures are identified in a traffic sign pattern that remains in the image after the first filtering process. For the purposes of the following description, a speed limit traffic sign was selected from among the traffic signs.

[0018] Fig. Figure 2 shows a block diagram of a device for recognizing a traffic sign according to the exemplary embodiment of the present invention.

[0019] According to Fig. Figure 2 comprises a device for recognizing a traffic sign, designed according to the exemplary embodiment of the present invention: a traffic sign information acquisition unit 210, which processes an image obtained from a front camera of the vehicle and generates a list of traffic signs located on a road; a driving information receiving unit 220, which receives driving information relating to the vehicle; a first filter 231, which calculates a relative speed of the vehicle relative to a traffic sign contained in the list of traffic signs and checks whether the traffic sign is installed on the road; a second filter 232, which determines whether the traffic sign, which has been recognized as installed on the road by means of the first filter 231, is located in front of the vehicle; and a traffic information extraction unit 240, which extracts the information from the traffic signs.The system identifies traffic signs located in front of the vehicle, recognizes the nearest traffic sign, and extracts the speed limit displayed on that sign. Furthermore, the Traffic Sign Information Acquisition Unit 210 can compile a list of various signs installed on the road the vehicle is currently traveling on. This list can be generated according to an intelligent traffic network system implemented throughout the city. To this end, the Traffic Sign Information Acquisition Unit 210 can generate a list of all traffic signs installed on the road using a near-field communication (NFC) module capable of short-range communication.and / or an infrared communication module and / or a Zigbee module and / or a dedicated short range communication (DSRC) device and / or an ultra-wideband (UWB) module and / or a wireless LAN (WLAN) module.

[0020] Fig. Figure 3 shows a view to illustrate a problem that may occur on an access road in connection with the exemplary embodiment of the present invention.

[0021] According to Fig. 3. When a vehicle is traveling straight ahead, a speed limit traffic sign, which is a speed limit traffic sign (B) 320 installed at an intersection and on an approach road, may be shown in an image from the vehicle's front camera while the vehicle is traveling.

[0022] Assuming that a speed limit sign (A) 310, indicating a speed limit on a motorway on which the driver is traveling, is installed together with the speed limit sign (B) 320 on an approach road to the intersection, it cannot be determined which information should be transmitted under the information of traffic sign (A) 310 and the information of traffic sign (B) 320 according to the existing algorithm.

[0023] For example, when vehicle 330 is going straight ahead, it needs the information from traffic sign (A) 310, and when vehicle 330 is exiting the motorway, it needs the information from traffic sign (B) 320.

[0024] However, in a pattern recognition process according to the relevant state of the art, it may not be possible to distinguish between traffic sign (A) 310 and traffic sign (B) 320.

[0025] Therefore, according to the exemplary embodiment of the present invention, the usability of the traffic sign is determined by a filtering process, which is carried out by means of the in Fig. The first and second filters 231 and 232 shown are used.

[0026] To determine the usability of the traffic sign, the traffic sign information acquisition unit 210 records a front image, which is taken by the front camera mounted on the vehicle, as a single image unit.

[0027] An image processing unit 215 receives the front image captured by the traffic sign information acquisition unit 210 as a single image unit and obtains information about the speed limit sign contained in each frame using an object recognition technique such as an object extraction algorithm. This information about the speed limit sign can include a pixel value (or pixel coordinate) that identifies the speed limit sign in each single image, and a motion vector value and a motion speed value of the speed limit sign, which are calculated from a difference between a previous single image and the current single image.

[0028] The vehicle information receiver unit 220 is connected via a controller area network (CAN) communication network to various electronic control units in the vehicle, which are not shown in the drawings, and receives vehicle driving information from the various electronic control units, such as vehicle speed and yaw rate.

[0029] The first filter 231 filters out irrelevant traffic signs using the motion vector value and the motion speed value of the speed traffic sign transmitted by the image processing unit 215.

[0030] The first filter 231 can filter out patterns of stickers attached to the rear of a vehicle in front (a large-scale vehicle).

[0031] The speed limit sign installed on a real road is firmly fixed to the ground. Therefore, it can be confirmed that speed limit signs in the image move in a direction opposite to the direction of travel of the vehicle, and at the same speed as the vehicle. However, in the case of the vehicle ahead, the corresponding speed and direction cannot be presented as a value opposite to that of the vehicle traveling ahead.

[0032] By using the aforementioned features, the first filter can filter out 231 unnecessary sticker patterns affixed to the rear of the vehicle in front.

[0033] For example, if a motion vector value (or motion direction value) calculated by image processing is opposite to a vehicle speed direction value of the currently moving vehicle and has the same motion velocity value, it is recognized that the front camera mounted on the moving vehicle is detecting a fixed speed limit sign, and the first filter 231 classifies the speed limit sign as a relevant traffic sign and accepts it. The motion velocity value can be defined here as a value generated by dividing the difference (distance traveled in the image) between a coordinate value of a traffic sign displayed in the previous frame and a coordinate value of a traffic sign displayed in the current frame by the frame frequency.

[0034] However, if the direction of movement value (movement vector value) of the speed limit sign attached to the rear of the vehicle ahead is equal to the vehicle speed direction value of the currently moving vehicle, and the speed of movement value of the speed limit sign attached to the rear of the vehicle ahead is equal to or less than the vehicle speed value of the currently moving vehicle, it is recognized that the front camera of the currently moving vehicle is detecting a vehicle ahead that is moving in a direction identical to the direction of travel of the vehicle, and the first filter 231 classifies the speed limit sign as an irrelevant traffic sign and filters it out.

[0035] The first filter 231 as such can filter out irrelevant traffic sign information, such as that of the sticker attached to the rear of the large-scale vehicle, by using the motion vector value of the speed traffic sign, which is calculated by processing the front image, and by using the speed value of the speed traffic sign, which is calculated from a difference between the previous single image and the current single image.

[0036] Even after the irrelevant speed limit traffic sign pattern has been filtered out as described above, information about a number of traffic signs, such as the traffic sign arranged along a straight road and the traffic sign arranged on the approach road to the intersection, can still be obtained. Fig. Three are shown, and therefore it is necessary to select the traffic sign required by the vehicle. This selection can be made using the second filter, number 232.

[0037] When the second filtering process is carried out by the second filter 232, this filtering process takes into account the fact that the speed limit traffic sign, which is relevant for the vehicle actually driving, is installed vertically on the road.

[0038] According to Fig. 3, while vehicle 330 is traveling straight ahead, the speed traffic sign (A) 310 is installed vertically on the straight roadway, and the speed limit traffic sign (B) 320 is installed vertically on a curved section of the access road.

[0039] In the actual image, the aspect ratio of the detected traffic sign is calculated, and it is determined whether the shape of the detected traffic sign is approximately a circle. Consequently, the traffic sign installed on the straight roadway can be identified. For example, a circular traffic sign can be recognized as such if its horizontal and vertical dimensions are equal, and it can be recognized as elliptical if its horizontal and vertical dimensions differ.

[0040] The speed limit sign installed on the road travelled by the vehicle is detected as approximately circular because it is captured in front of the front camera mounted on the vehicle; however, the speed limit sign installed on a peripheral road, such as the access road, is detected as approximately elliptical because it is captured at an angle relative to the front of the front camera.

[0041] Therefore, the aspect ratio of the speed limit sign detected in the front image can be used to determine whether the sign is the speed limit sign installed on the road or the sign installed on the access road. This means that if the vehicle is to recognize the traffic sign installed on the road it is currently traveling on, the sign with the detected difference between its horizontal and vertical dimensions will be filtered out.

[0042] The information of traffic sign (B) 320 is also removed by means of the second filter 232 in the case that, while the vehicle 330 is driving straight ahead, traffic signs (A and B) 310 and 320 are not simultaneously in the image capture area of ​​the front camera, but only traffic sign (B) 320 is in the image capture area; this prevents the vehicle speed limit displayed on the irrelevant traffic sign (B) 320 from being communicated to the driver driving on the motorway when the vehicle is driving straight ahead.

[0043] Even after the second filtering process has been carried out by the second filter 232 as described above, there may still be a number of vertically installed traffic signs on the road.

[0044] For example, there may be different speed limits for the respective lanes on the motorway, such as 100 km / h for a first lane and 80 km / h for a second lane, and thus it may happen that information from a number of traffic signs is detected even though the second filtering process is also carried out.

[0045] In this case, for the traffic information extraction unit 240 of the present invention, the most relevant sign may be the traffic sign that is detected at the shortest distance to a center line running in a vertical direction which crosses the center of the front image.

[0046] Fig. Figure 4 shows a flowchart to illustrate an operation for recognizing a traffic sign according to the exemplary embodiment of the present invention.

[0047] According to Fig.4. In order to identify, from among a number of traffic signs arranged on a road, a traffic sign that displays relevant speed limit information for a driver, a front-facing image is first captured by the vehicle's front camera, and this image is then subjected to an identification process (S410). This means that a traffic sign contained in the front-facing image, detected using object detection technology, is identified (S410). Additionally, the intelligent traffic network system of a "smart city" generates a list of all traffic signs installed on the road (S410).

[0048] Next, various electronic control units of the vehicle receive vehicle driving information (e.g. vehicle speed, yaw rate, etc.) via CAN communication (S421), and it is checked whether the traffic sign is firmly attached to the road by comparing the traffic sign included in the list of traffic signs with a relative speed of the vehicle (S420).

[0049] Since the traffic sign is a fixed object on the ground, in a situation where the vehicle passes the speed limit sign positioned on the road while driving, a movement is detected whose direction is opposite to the direction of travel of the currently moving vehicle and whose speed is equal to the speed of the moving vehicle.

[0050] However, in the case of the sticker affixed to the large vehicle moving along with the vehicle, the corresponding speed and direction values ​​cannot be opposite to the values ​​of the moving vehicle, except in a situation where the large vehicle stops.

[0051] In the present invention, during the journey of the vehicle, only the information relating to the traffic sign is obtained from the traffic sign information which is entered on the basis of the traffic sign and the relative speed of the vehicle (S420).

[0052] The system then determines whether the speed limit traffic sign, which is identified as a stationary traffic sign on the road using the first filter 231, is present in front of the vehicle (S430).

[0053] To determine the speed limit traffic sign in front of the vehicle, the traffic sign that is installed vertically on the road is identified.

[0054] For this purpose, according to an exemplary embodiment of the present invention, the filtering process is carried out using the aspect ratio of the traffic sign designated as a speed limit sign. This means that a process is performed in which the traffic sign whose shape has been identified as elliptical with a vertical dimension equal to or greater than a horizontal length by a predetermined ratio is filtered out.

[0055] The speed limit sign positioned directly in front of the vehicle is displayed as a circle because it is captured by the front-facing camera mounted on the moving vehicle. In contrast, a speed limit sign positioned on the access road or similar is displayed as an elliptical shape because it is captured at an angle by the front camera. The irrelevant signal can be filtered out using the aforementioned methods.

[0056] Among the speed limit signs identified by the second filter 232 as being in front of the vehicle, the speed limit sign located at the shortest distance from the center line vertically intersecting the center of the front image is determined (S440).

[0057] The speed information indicated on the circular traffic sign located at the shortest distance from the vehicle is extracted as such through the first filtering operation, the second filtering operation and a step to determine a distance, and the speed information is transmitted to the vehicle (S450).

[0058] With the present invention, if in a situation where the vehicle is actually driving on the road a number of traffic information signs are present for the purpose of safe driving, the traffic sign which displays the most useful information for the driver can be selected, and the information displayed on the selected traffic sign can be transmitted to the driver and a speed control system of the vehicle.

[0059] Although an exemplary embodiment of the present invention has been described here for illustrative purposes, it will be apparent to those skilled in the field that various modifications and alterations are possible without deviating from the scope and spirit of the invention. Thus, the disclosed exemplary embodiments of the present invention are not intended to limit its scope, but rather to illustrate its technical content, and the scope of the technical content of the present invention is not limited by these exemplary embodiments. The scope of protection of the present invention is determined on the basis of the appended claims, and it should be evident that the present invention encompasses technical content that is identical or equivalent to that of the claims.

Claims

[1] Device for detecting a traffic sign located on a road, comprising: an information acquisition unit that captures a front image received from a front camera of a vehicle as a single image unit; an image processing unit (215) that detects traffic sign information, which detects a motion vector value and a motion speed value of the traffic sign contained in the front image by means of an object detection algorithm from a difference between a previous frame and a current frame; and a filter (231,232) which filters out an irrelevant traffic sign by comparing vehicle driving information supplied by an electronic control unit arranged in the vehicle, which includes a yaw rate value and a vehicle speed value of the vehicle, with the traffic sign information. [2] Device according to claim 1, characterized by, that the filter (231) allows the traffic sign to pass as a relevant traffic sign permanently attached to the road if a direction of movement value of the traffic sign, which depends on the motion vector value, is opposite to a vehicle speed direction value of the vehicle and a speed of movement value of the traffic sign contained in the front image is equal to a speed value of the vehicle. [3] Device according to claim 1, characterized by, that the filter filters out the traffic sign as an irrelevant traffic sign attached to the rear of a vehicle ahead if a direction of movement value of the traffic sign, which depends on the motion vector value, is equal to a vehicle speed direction value of the vehicle, which depends on a yaw rate value of the vehicle, and a speed of movement value of the traffic sign contained in the front image differs from a speed value of the vehicle. [4] Device according to claim 1, characterized by , that the image processing unit (215) detects information about the traffic sign, which furthermore includes a ratio between the horizontal and vertical dimensions of the traffic sign. [5] Device according to claim 4, characterized by, that the filter (231,232) filters out an irrelevant traffic sign based on the ratio between the horizontal and vertical dimensions of the traffic sign. [6] Device according to claim 5, characterized by , that the filter (231,232) allows the traffic sign to pass as a relevant traffic sign if the horizontal and vertical dimensions of the traffic sign are equal. [7] Device according to claim 5, characterized by , that the filter (231,232) filters out the traffic sign as an irrelevant traffic sign if the horizontal and vertical dimensions of the traffic sign differ from each other. [8] Device according to claim 7, characterized by , that the traffic sign whose horizontal and vertical dimensions differ from each other is an irrelevant traffic sign installed vertically on a curved road. [9] Method for recognizing a traffic sign located on a road, comprising: Capturing a front image obtained from a vehicle's front camera as a single image unit; Detecting a motion vector value and a motion velocity value of the traffic sign contained in the front image from a difference between a previous frame and a current frame using an object detection algorithm; and Filtering out an irrelevant traffic sign by comparing vehicle driving information, which includes a yaw rate value and a vehicle speed value of the vehicle, with the traffic sign information. [10] Method according to claim 9, characterized by, that when filtering out the irrelevant traffic sign, the traffic sign is allowed to pass as a relevant traffic sign permanently attached to the road if a direction of movement value of the traffic sign, which depends on the movement vector value, is opposite to a vehicle speed direction value of the vehicle and a speed value of the traffic sign contained in the front image is equal to a speed value of the vehicle. [11] Method according to claim 9, characterized by, that when filtering out the irrelevant traffic sign, the traffic sign is filtered out as an irrelevant traffic sign attached to the rear of a vehicle in front if a direction of movement value of the traffic sign, which depends on the motion vector value, is equal to a vehicle speed direction value of the vehicle, which depends on a yaw rate value of the vehicle, and a speed of movement value of the traffic sign contained in the front image differs from a speed value of the vehicle. [12] Method according to claim 9, characterized by , that detecting the motion vector value and the motion speed value includes detecting a ratio between the horizontal and vertical dimensions of the traffic sign. [13] Method according to claim 12, characterized by, that filtering out the irrelevant traffic sign includes filtering out the irrelevant traffic sign based on the ratio between the horizontal and vertical dimensions of the traffic sign. [14] Method according to claim 13, characterized by , that filtering out the irrelevant traffic sign includes allowing the traffic sign to pass as a relevant traffic sign if the horizontal and vertical dimensions of the traffic sign are equal. [15] Method according to claim 13, characterized by, that when filtering out the irrelevant traffic sign, the traffic sign is filtered out as an irrelevant traffic sign if the horizontal and vertical dimensions of the traffic sign differ from each other, and the traffic sign whose horizontal and vertical dimensions differ from each other is an irrelevant traffic sign installed vertically on a curved road.

Images