Traffic light recognition method and traffic light recognition device

Abstract

To improve recognition accuracy by correctly recognizing a lighted lamp of a traffic signal even when the lighted lamp of the traffic signal has a dark portion due to long-term deterioration.SOLUTION: A traffic signal recognition device 1 is configured to: select a lamp region indicating a position of a lighted lamp of a traffic signal from a captured image; perform circle detection processing on the lamp region; generate a plurality of mask images having a circular region set on the basis of a circle detected in the circle detection processing and a mask region set on a portion other than the circular region, the plurality of mask images with the circular regions of different shapes; perform color extraction processing on the lamp region to generate a color extraction image having a color extraction region in which a color of a lighted lamp is extracted; and perform mask processing on the color extraction image by using the plurality of mask images to recognize the lighted lamp of the traffic signal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a traffic light recognition method and device for recognizing lights of a traffic light that is turned on in front of a vehicle from an image captured by an imaging means mounted on the vehicle. [Background technology]

[0002] Conventionally, a traffic light detection device for detecting traffic lights in an image is disclosed in Patent Document 1. The traffic light detection device disclosed in Patent Document 1 extracts red, blue, and yellow parts from a color image and derives the circularity of each color part to detect the traffic light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-301518 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional traffic light detection device described above, the circularity of each color portion is derived to detect the traffic light, so if a dark portion occurs in the light of the lit traffic light due to aging, the circularity cannot be derived accurately. For example, if the light of the traffic light is partially dark due to aging or if the center of the light is burned by lens burn, the circularity decreases. Therefore, in the conventional device, if a dark portion occurs in the light of the lit traffic light due to aging, the light of the lit traffic light cannot be accurately recognized, and the recognition accuracy decreases.

[0005] Therefore, the present invention has been proposed in consideration of the above-mentioned situation, and aims to provide a traffic light recognition method and device therefor that can accurately recognize traffic light lights and improve recognition accuracy even if dark areas appear in the lights of a lit traffic light due to deterioration over time. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a traffic light recognition method and an apparatus thereof according to one aspect of the present invention acquire a captured image, and select a light area indicating the position of a light of a lit traffic light from the acquired image. Then, a circle detection process is performed on the light area, and a plurality of mask images are generated having a circular area set based on the circle detected by the circle detection process and a mask area set in a portion other than the circular area, and the circular areas of the plurality of mask images have different shapes. Furthermore, the traffic light recognition method and an apparatus thereof perform a color extraction process on the light area, and generate a color extraction image having a color extraction area in which the color of the lit light is extracted, and perform a mask process on the color extraction image using the plurality of mask images to recognize the lights of the lit traffic lights. Effect of the Invention

[0007] According to the present invention, even if dark areas appear in the lights of a lit traffic signal due to deterioration over time, the lights of the lit traffic signal can be accurately recognized, thereby improving recognition accuracy. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of a vehicle system equipped with a traffic light recognition device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of a traffic light in which a dark area occurs in the light. [Diagram 3] FIG. 3 is a diagram showing a result of detection of a traffic light area by a traffic light recognition device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a result of a circle detection process performed by a traffic light recognition device according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing a mask image generated by a traffic light recognition device according to an embodiment of the present invention. [Figure 6]FIG. 6 is a diagram showing a color-extracted image generated by a traffic light recognition device according to an embodiment of the present invention. [Figure 7] FIG. 7 is a flowchart showing a processing procedure of a traffic light recognition process performed by the traffic light recognition device according to one embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart showing the procedure of the light recognition process performed by the traffic light recognition device according to one embodiment of the present invention. [Figure 9] FIG. 9 is a diagram for explaining the masking process using the first mask image by the traffic light recognition device according to one embodiment of the present invention. [Figure 10] FIG. 10 is a diagram for explaining the masking process using the second mask image by the traffic light recognition device according to one embodiment of the present invention. [Figure 11] FIG. 11 is a diagram for explaining the masking process using the third mask image by the traffic light recognition device according to one embodiment of the present invention. [Figure 12] FIG. 12 is a diagram for explaining mask processing performed by a traffic light recognition device according to an embodiment of the present invention when the threshold value is changed using a first mask image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same parts are given the same reference numerals and detailed description will be omitted.

[0010] [Vehicle system configuration] Fig. 1 is a block diagram showing the configuration of a vehicle system of a vehicle equipped with a traffic light recognition device according to this embodiment. As shown in Fig. 1, the vehicle system 100 includes a traffic light recognition device 1, a camera 3, a vehicle sensor 5, and a vehicle control device 7, and is installed in a vehicle capable of automatic driving or a vehicle that provides driving assistance to a driver.

[0011] The traffic light recognition device 1 is a device that recognizes the lights of a traffic light that is turned on in front of the vehicle from an image captured by the camera 3. Therefore, the traffic light recognition device 1 processes the image captured by the camera 3 to recognize whether the light of the traffic light currently turned on in front of the vehicle is green, red, or yellow, and outputs the recognition result to the vehicle control device 7.

[0012] In particular, traffic light may become darker due to deterioration over time. For example, as shown in Fig. 2, before deterioration, both LED traffic light 23 and bulb-type traffic light 24 have bright green lights, but in bulb-type traffic light 25 that has deteriorated over time, the center of the blue light is bright and the surroundings are dark. In bulb-type traffic light 26 that has deteriorated over time, the center of the blue light has become dark due to lens burn, and in bulb-type traffic light 27, the blue light has become partially dark due to deterioration over time.

[0013] Therefore, the traffic light recognition device 1 is capable of determining the state of deterioration due to aging and accurately recognizing the lights of lit traffic lights, even if dark areas appear in the lights of lit traffic lights due to aging.

[0014] The camera 3 is mounted on the vehicle as an image processing sensor, and is an imaging means for capturing images of the surroundings of the vehicle, particularly the area in front of the vehicle, and outputs the captured color images to the traffic light recognition device 1. The number of cameras 3 does not need to be limited to one, and multiple cameras may be mounted.

[0015] The vehicle sensor 5 is a group of sensors mounted on the vehicle for detecting the position information and driving state of the vehicle, and includes, for example, a GPS receiver 51, a vehicle speed sensor 53, an acceleration sensor 55, a gyro sensor 57, and the like.

[0016] The vehicle control device 7 is a device that acquires the recognition result of the traffic light from the traffic light recognition device 1 and controls the traveling of the vehicle. Specifically, when the vehicle is autonomous, the vehicle control device 7 acquires the recognition result of the traffic light in front of the vehicle from the traffic light recognition device 1, and when the traffic light is red, the vehicle control device 7 controls the brakes of the vehicle to stop the vehicle, and when the traffic light is green, the vehicle control device 7 controls the accelerator to pass the traffic light. In addition, when the vehicle performs driving assistance for the driver, the vehicle control device 7 notifies the driver of the recognition result of the traffic light in front of the vehicle acquired from the traffic light recognition device 1.

[0017] Here, as shown in Figure 1, the traffic light recognition device 1 includes an image acquisition unit 11, a traffic light area detection unit 13, a light area selection unit 15, a mask image generation unit 17, a color extraction image generation unit 19, and a light recognition unit 21.

[0018] The image acquisition unit 11 acquires color images captured by the camera 3 via a communication line such as a Controller Area Network-BUS (CAN-BUS) or Ethernet, and records the images in a database or memory (not shown).

[0019] The traffic light area detection unit 13 detects a traffic light area in which the main body of a traffic light is captured from the image acquired by the image acquisition unit 11. For example, as shown in Fig. 3, a traffic light area 31 is detected from an image captured by the camera 3 using a method such as template matching or machine learning. The traffic light area 31 is an area in which the main body of a traffic light is captured, and Fig. 3 shows the detection result when the main body of a three-light traffic light is detected.

[0020] The light area selection unit 15 selects a light area indicating the position of a light that is turned on from among the traffic light areas detected by the traffic light area detection unit 13. For example, as shown in Fig. 3, the light area selection unit 15 detects lights that are turned on from within a traffic light area 31, and when a green light is turned on, it divides the width of the three-light traffic light area 31 into thirds and selects the leftmost area as the light area 33. When a yellow light is turned on, it selects the center area of ​​the traffic light area 31, which is divided into three, as the light area, and when a red light is turned on, it selects the rightmost area as the light area.

[0021] As described above, in this embodiment, a traffic light area in which the main part of the traffic light is captured is detected from the image acquired by the image acquisition unit 11, and a light area indicating the position of a light that is turned on is selected from the detected traffic light area. However, the method of selecting (extracting) the light area is not limited to this.

[0022] For example, the approximate shape of the light area of ​​a traffic light, including its size, can be estimated based on the distance between the vehicle and the traffic light. Therefore, by detecting the distance between the vehicle and the traffic light, estimating the shape of the light area of ​​the traffic light based on this distance, and further extracting an area corresponding to the estimated shape from the image acquired by the image acquisition unit 11, it is possible to select (extract) the light area directly from the image acquired by the image acquisition unit 11 without detecting the traffic light area.

[0023] However, in order to achieve highly accurate selection (extraction) of the light area, as described above, it is preferable to detect the traffic light area in which the main part of the traffic light is captured from the image acquired by the image acquisition unit 11, and select the light area indicating the position of the lit light from among the detected traffic light areas.

[0024] The mask image generating unit 17 performs a circle detection process on the light region selected by the light region selecting unit 15, and generates a plurality of mask images based on the circle detected by the circle detection process. The plurality of mask images have a circular region set based on the circle detected by the circle detection process, and a mask region set in a portion other than the circular region, and the shapes of the circular regions are different from each other.

[0025] First, the mask image generating unit 17 performs circle detection processing on the light region. As shown in Fig. 4, the mask image generating unit 17 performs circle detection on the light region 33 using a method such as the Hough transform to detect the circumscribing circle 35 of the light. Note that any method other than the Hough transform may be used as long as it can detect the circumscribing circle of the light.

[0026] When circle detection is performed in this manner, mask image generation unit 17 generates multiple mask images having a circular area set based on the circle detected in the circle detection process and a mask area set in the area other than the circular area. The mask images generated here are binary images of black and white (1, 0), with the circular area becoming a white area (1) and the mask area becoming a black area (0). Therefore, when mask processing is performed, the area corresponding to the circular area is displayed and the area corresponding to the mask area is processed so that it is not displayed. The multiple mask images generated represent a deterioration model of traffic light using masks, and each of the circular areas has a different shape.

[0027] For example, as shown in Fig. 5, the mask image generating unit 17 generates first to third mask images. The first mask image 41 has a circular region 41A and a mask region 41B, the circular region 41A has a radius r1 equal to the radius of the circumscribing circle 35 detected in the circle detection process, and the mask region 41B is set in a portion other than the circular region 41A. This first mask image 41 is a model of the traffic lights 23 and 24 shown in Fig. 2 that have not deteriorated with age, and is used to detect lights that have not deteriorated with age and whose entire lights are brightly lit.

[0028] The second mask image 43 has a circular region 43A and a mask region 43B, the circular region 43A has a radius r2 smaller than the radius of the circumscribing circle 35 detected in the circle detection process, and the mask region 43B is set in a portion other than the circular region 43A. This second mask image 43 is a model of the traffic light 25 shown in Fig. 2, which is bright in the center and dark around, and is used to detect lights that have become bright in the center and dark around due to aging. The radius r2 is set to a value of about 80% of the radius of the circumscribing circle 35 detected in the circle detection process.

[0029] The third mask image 45 has a circular region 45A and a mask region 45B. The circular region 45A has a radius r1 equal to the radius of the circumscribing circle 35 detected in the circle detection process, and a circular mask region 45C is formed in the center. The mask region 45B is set in a portion other than the circular region 45A. The mask region 45C is set in the center of the circular region 45A and is a circular region having a radius r3 smaller than the radius of the circumscribing circle 35 detected in the circle detection process. This third mask image 45 is a model of the traffic light 26 shown in FIG. 2, the center of which has become dark due to lens burn, and is used to detect a light whose center has become dark due to lens burn caused by aging. The radius r3 is set to a value of about 20% of the radius of the circumscribing circle 35 detected in the circle detection process.

[0030] The color extraction image generating unit 19 performs color extraction processing on the light region selected by the light region selecting unit 15 using the RGB color space or the HSV color space, and generates a color extraction image having a color extraction region where the color of the light that is turned on is extracted. For example, as shown in Fig. 6, a color extraction image 61 in the case of a green light has a color extraction region 61A where the blue color that is the color of the light that is turned on is extracted. The color extraction image 61 was generated from the images of the traffic lights 23 and 24 shown in Fig. 2 that have not deteriorated with age, and since the lights of the traffic lights 23 and 24 are all brightly lit, the color extraction region 61A is approximately equal in size to the size of the blue light on the image.

[0031] In addition, the color extraction image 63 was generated from an image of a traffic light 25 in which the center of the light is bright and the periphery is dark due to deterioration over time, as shown in FIG. 2. Since the periphery of the light is dark, the color extraction area 63A is smaller than the size of the blue light on the image.

[0032] Furthermore, the color extraction image 65 was generated from an image of a traffic light 26 in which the center of the light has become dark due to lens burn caused by aging, as shown in Figure 2. Since the center of the light is dark, the color extraction area 65A has a donut shape with a missing center.

[0033] In addition, the color extraction image 67 was generated from an image of a traffic light 27 whose light has become partially dark due to deterioration over time, as shown in Figure 2. Since the light is partially dark, the color extraction region 67A has a shape including a missing portion 69.

[0034] The light recognition unit 21 uses the multiple mask images generated by the mask image generation unit 17 to perform mask processing on the color extraction image generated by the color extraction image generation unit 19, and recognizes the lights of lit traffic lights. The recognized lights are output to the vehicle control device 7 as traffic light candidates. A specific method for recognizing lights will be described later.

[0035] The traffic light recognition device 1 is a controller that includes general-purpose electronic circuits, including a microcomputer, a microprocessor, and a CPU, and peripheral devices, such as a memory, and is installed with a computer program for executing traffic light recognition processing. Each function of the traffic light recognition device 1 can be implemented by one or more processing circuits. The processing circuits include, for example, programmed processing devices including electrical circuits, and also include devices such as application specific integrated circuits (ASICs) and conventional circuit components arranged to perform the functions described in the embodiments.

[0036] [Traffic signal recognition processing] Next, a description will be given of the traffic light recognition processing executed by the traffic light recognition device 1 according to this embodiment. Fig. 7 is a flowchart showing the processing procedure of the traffic light recognition processing executed by the traffic light recognition device 1 according to this embodiment.

[0037] As shown in FIG. 7, in step S101, the image acquisition unit 11 acquires a color image captured by the camera 3.

[0038] In step S103, the traffic light area detection unit 13 judges whether or not the traffic light area in which the main part of the traffic light is captured has been detected from the image acquired in step S101. For example, it judges whether or not the traffic light area 31 shown in Fig. 3 has been detected, and if it has been detected, the process proceeds to step S105. On the other hand, if the traffic light area has not been detected, the traffic light recognition process according to this embodiment is terminated.

[0039] In step S105, the light area selection unit 15 selects a light area indicating the position of a light that is turned on from among the traffic light areas detected in step S103. For example, when a blue light is turned on, the light area selection unit 15 selects the light area 33 shown in FIG. 3.

[0040] In step S107, the mask image generating unit 17 performs a circle detection process on the light region selected in step S105, and judges whether or not a circle has been detected. For example, as shown in Fig. 4, it judges whether or not a circumscribing circle 35 of a blue light has been detected, and if it has been detected, the process proceeds to step S109. On the other hand, if the circumscribing circle 35 has not been detected, the traffic light recognition process according to this embodiment is terminated.

[0041] In step S109, the mask image generating unit 17 generates a plurality of mask images based on the circular shape detected in the circle detection process in step S107. For example, as shown in Fig. 5, the mask image generating unit 17 generates a first mask image 41, a second mask image 43, and a third mask image 45.

[0042] In step S111, the color extraction image generating unit 19 performs color extraction processing on the light area selected in S105 to generate a color extraction image having a color extraction area in which the color of the lit light is extracted. For example, color extraction images 61 to 67 shown in FIG. 6 are generated.

[0043] In step S113, the light recognition unit 21 uses the multiple mask images generated in step S109 to perform mask processing on the color extraction image generated in step S111, and recognizes the lights of the lit traffic lights. Specifically, the light recognition unit 21 recognizes the lit lights by executing the flowchart shown in FIG.

[0044] Fig. 8 is a flowchart showing the processing procedure of the light recognition processing by the traffic light recognition device 1 according to this embodiment. As shown in Fig. 8, in step S201, the light recognition unit 21 performs mask processing on the color extraction image using the first mask image. For example, as shown in Fig. 9, the first mask image 41 and the color extraction image 61 are multiplied by AND processing to calculate a masked image 91.

[0045] In step S203, the light recognition unit 21 judges whether or not the area of ​​the color region 91A of the masked image 91 shown in Fig. 9 is equal to or larger than a first predetermined value. That is, it judges whether or not the ratio of the color extraction region 61A to the circular region 41A of the first mask image 41 is equal to or larger than a first predetermined threshold. For example, when the first threshold is set to 0.9, if the ratio of the color extraction region 61A to the circular region 41A is equal to or larger than 0.9, the process proceeds to step S205.

[0046] In step S205, the light recognition unit 21 recognizes the color extraction region 61A as a light of a lit traffic light. At this time, the light recognition unit 21 can recognize the light by the first mask image 41 that models a traffic light whose entire light is brightly lit, and therefore determines that the color extraction region 61A is a light whose entire light is brightly lit and has not deteriorated over time.

[0047] On the other hand, if the area of ​​the color region 91A of the masked image 91 shown in FIG. 9 is less than the first predetermined value, that is, if the ratio of the color extraction region 61A to the circular region 41A is less than the first threshold, the process proceeds to step S207.

[0048] In step S207, the light recognition unit 21 performs a mask process on the color extraction image using the second mask image. For example, as shown in Fig. 10, the second mask image 43 and the color extraction image 63 are multiplied by AND processing to calculate a masked image 93.

[0049] In step S209, the light recognition unit 21 judges whether the area of ​​the color region 93A of the masked image 93 shown in Fig. 10 is equal to or larger than a second predetermined value. That is, it judges whether the ratio of the color extraction region 63A to the circular region 43A of the second mask image 43 is equal to or larger than a second predetermined threshold value. For example, when the second threshold value is set to 0.9, if the ratio of the color extraction region 63A to the circular region 43A is equal to or larger than 0.9, the process proceeds to step S205.

[0050] In step S205, the light recognition unit 21 recognizes the color extraction region 63A as a light of a lit traffic light. At this time, the light recognition unit 21 can recognize the light by the second mask image 43 that models a traffic light in which the center of the light is bright and the surroundings are dark, and therefore determines that the color extraction region 63A is a light in which the center of the light is bright and the surroundings are dark due to aging.

[0051] On the other hand, if the area of ​​the color region 93A of the masked image 93 shown in FIG. 10 is less than the second predetermined value, that is, if the ratio of the color extraction region 63A to the circular region 43A is less than the second threshold value, the process proceeds to step S211.

[0052] In step S211, the light recognition unit 21 performs a mask process on the color extraction image using the third mask image. For example, as shown in Fig. 11, the third mask image 45 and the color extraction image 65 are multiplied by AND processing to calculate a masked image 95.

[0053] In step S213, the light recognition unit 21 judges whether or not the area of ​​the color region 95A of the masked image 95 shown in Fig. 11 is equal to or larger than a third predetermined value. That is, it judges whether or not the ratio of the color extraction region 65A to the circular region 45A of the third mask image 45 is equal to or larger than a predetermined third threshold. For example, when the third threshold is set to 0.9, if the ratio of the color extraction region 65A to the circular region 45A is equal to or larger than 0.9, the process proceeds to step S205.

[0054] In step S205, the light recognition unit 21 recognizes the color extraction region 65A as a light of a lit traffic light. At this time, the light recognition unit 21 can recognize the light by the third mask image 45 that models a traffic light whose center has become dark due to lens burn, and therefore determines that the color extraction region 65A is a light whose center has become dark due to aging deterioration due to lens burn.

[0055] On the other hand, if the area of ​​the color region 95A of the masked image 95 shown in FIG. 11 is less than the third predetermined value, that is, if the ratio of the color extraction region 65A to the circular region 45A is less than the third threshold, the process proceeds to step S215.

[0056] In step S215, the light recognition unit 21 performs a mask process on the color extraction image using the first mask image. For example, as shown in Fig. 12, the first mask image 41 and the color extraction image 67 are multiplied by AND processing to calculate a masked image 97.

[0057] In step S217, the light recognition unit 21 judges whether the area of ​​the color region 97A of the masked image 97 shown in Fig. 12 is equal to or larger than a fourth predetermined value. That is, it judges whether the ratio of the color extraction region 67A to the circular region 41A of the first mask image 41 is equal to or larger than a predetermined fourth threshold. For example, when the fourth threshold is set to 0.6, if the ratio of the color extraction region 67A to the circular region 41A is equal to or larger than 0.6, the process proceeds to step S205. Note that the fourth threshold is set to a value smaller than the first threshold so that it can prevent the light from being overlooked even if the dark areas of the light increase due to aging.

[0058] In step S205, the light recognition unit 21 recognizes the color extraction region 67A as a light of a lighted traffic light. At this time, the light recognition unit 21 cannot recognize the light by the first to third mask images, but can recognize the light by reducing the threshold value, and therefore determines that the color extraction region 67A is a light that has become partially dark due to aging. When the color extraction region is thus recognized as a light of a lighted traffic light, the light recognition process shown in FIG. 8 is terminated, and the process returns to step S113 in FIG. 7.

[0059] On the other hand, when the area of ​​the color region 97A of the masked image 97 shown in Fig. 12 is less than the fourth predetermined value, that is, when the ratio of the color extraction region 67A to the circular region 41A is less than the fourth threshold value, the light recognition process shown in Fig. 8 is terminated without recognizing the traffic light. Then, when the light recognition process is terminated, the process returns to step S113 in Fig. 7.

[0060] In the light recognition process shown in FIG. 8, the first mask image is used first, followed by the second mask image and the third mask image in this order for mask processing. In normal traffic lights, traffic lights with all lights lit brightly occur most frequently, so the first mask image is used first. And traffic lights with dark edges occur more frequently than traffic lights with dark centers, so the second mask image is used before the third mask image. By using the mask images in this order, from the first mask image to the second mask image and then the third mask image, processing can be performed in order of frequency of occurrence, so there is no need to process traffic lights with a low frequency of occurrence, and it is possible to reduce the calculation load.

[0061] 7, in step S115, the light recognition unit 21 outputs the recognition result of the traffic light to the vehicle control device 7, and ends the traffic light recognition process according to this embodiment. Note that, although this embodiment has been described using the example of a green light, the same process can be performed in the case of a yellow light or a red light by selecting the light area of ​​the yellow light or red light in step S105.

[0062] [Effects of the embodiment] As described above in detail, the traffic light recognition device 1 according to this embodiment generates a plurality of mask images having a circular area set based on the circle detected in the circle detection process and a mask area set in the area other than the circular area. The generated plurality of mask images each have a different shape for the circular area, and the plurality of mask images are used to perform mask processing on the color extraction image to recognize the lights of a lit traffic light. This makes it possible to accurately recognize the lights of a lit traffic light, improving recognition accuracy, even if dark areas appear in the lights of a lit traffic light due to deterioration over time.

[0063] In addition, in the traffic light recognition device 1 according to the present embodiment, the multiple mask images include a first mask image having a circular area having a radius equal to the radius of the circle detected in the circle detection process and a mask area set in the portion other than the circular area. This allows accurate recognition of a traffic light whose entire light is bright. Furthermore, the multiple mask images include a second mask image having a circular area having a radius smaller than the radius of the circle detected in the circle detection process and a mask area set in the portion other than the circular area. This allows accurate recognition of a traffic light whose center is bright and whose periphery is dark. Furthermore, the multiple mask images include a third mask image having a circular area having a radius equal to the radius of the circle detected in the circle detection process, in which a circular mask area is formed in the center, and a mask area set in the portion other than the circular area. This allows accurate recognition of a traffic light whose center is dark due to lens burn. Therefore, it is possible to accurately recognize the light of a lit traffic light according to the state of aging deterioration.

[0064] Furthermore, in the traffic light recognition device 1 according to this embodiment, a first mask image is used to perform mask processing on the color extraction image, and when the ratio of the color extraction region to the circular region is equal to or greater than a predetermined first threshold, the color extraction region is recognized as a light of a lit traffic light. This makes it possible to accurately recognize a traffic light with all of its lights brightly lit. In addition, since the traffic light with the most frequent occurrence of all of its lights brightly lit can be recognized first, it is possible to reduce subsequent processing and reduce the calculation load.

[0065] Moreover, in the traffic light recognition device 1 according to this embodiment, a first mask image is used to perform mask processing on the color extraction image, and if the ratio of the color extraction region to the circular region is less than a first threshold value, a second mask image is used to perform mask processing on the color extraction image. Then, if the ratio of the color extraction region to the circular region is equal to or greater than a predetermined second threshold value, the color extraction region is recognized as a light of a lit traffic light. This makes it possible to accurately recognize a traffic light whose center is bright and whose surroundings are dark. Also, since the traffic light whose second most frequently occurring light has a bright center and whose surroundings are dark can be recognized second, by processing in order of frequency of occurrence, subsequent processing can be reduced and the calculation load can be reduced.

[0066] Furthermore, in the traffic light recognition device 1 according to this embodiment, a mask process is performed on the color extraction image using a second mask image, and when the ratio of the color extraction region to the circular region is less than a second threshold value, a mask process is performed on the color extraction image using a third mask image. Then, when the ratio of the color extraction region to the circular region is equal to or greater than a predetermined third threshold value, the color extraction region is recognized as a light of a lit traffic light. This makes it possible to accurately recognize traffic lights whose centers are darkened by lens burn. In addition, since traffic lights whose centers occur less frequently are recognized last, the calculation load can be reduced by reducing the processing of traffic lights whose occurrence is less frequent.

[0067] In addition, in the traffic light recognition device 1 according to this embodiment, a third mask image is used to perform mask processing on the color extraction image, and when the ratio of the color extraction region to the circular region is less than a third threshold, a first mask image is used to perform mask processing on the color extraction image. Then, when the ratio of the color extraction region to the circular region is equal to or greater than a fourth threshold smaller than the first threshold, the color extraction region is recognized as a light of a lit traffic light. This makes it possible to accurately recognize a partially dark traffic light. In addition, since the fourth threshold is set smaller than the first threshold, it is possible to prevent the light from being overlooked even if the dark parts of the light have increased due to deterioration over time.

[0068] The above-described embodiment is merely an example of the present invention, and the present invention is not limited to the above-described embodiment, and various modifications can be made to the design and other aspects of the present invention without departing from the technical concept of the present invention. [Explanation of symbols]

[0069] 1 Traffic light recognition device 3. Camera 5 Vehicle Sensors 7 Vehicle control device 11 Image acquisition section 13 Traffic light area detection unit 15 Light area selection section 17 Mask image generation unit 19 Color extraction image generation section 21 Light recognition section 41 First mask image 43 Second mask image 45 3rd mask image 51 GPS receiver 53 Vehicle speed sensor 55 Acceleration Sensor 57 Gyro sensor 61~67 Color Extraction Images 91~97 Masked images 100 Vehicle Systems

Claims

1. A traffic light recognition method for recognizing a light of a traffic light that is turned on in front of a vehicle from an image captured by an imaging means mounted on the vehicle, comprising: Acquire an image captured by the imaging means, Selecting a light area indicating the position of a light of a lit traffic light from the acquired image; performing a circle detection process on the light region, and generating a plurality of mask images having a circular region set based on the circle detected by the circle detection process and a mask region set in a portion other than the circular region, the plurality of mask images each having a different shape of the circular region; A color extraction process is performed on the light area to generate a color extraction image having a color extraction area in which the color of the light that is turned on is extracted; A traffic light recognition method, comprising: performing mask processing on the color extraction image using the plurality of mask images; and recognizing lights of lit traffic lights.

2. The plurality of mask images are a first mask image having a circular area having a radius equal to the radius of the circle detected by the circle detection process and a mask area set in a portion other than the circular area; a second mask image having a circular area having a radius smaller than the radius of the circle detected by the circle detection process and a mask area set in a portion other than the circular area; a third mask image having a circular region having a radius equal to the radius of the circle detected by the circle detection process and a circular mask region formed in the center, and a mask region set in a portion other than the circular region; The method for recognizing a traffic signal according to claim 1, further comprising:

3. The traffic light recognition method according to claim 2, characterized in that a mask process is performed on the color extraction image using the first mask image, and when a ratio of the color extraction region to the circular region is equal to or greater than a predetermined first threshold value, the color extraction region is recognized as a light of a lit traffic light.

4. 4. The traffic light recognition method according to claim 3, further comprising the steps of: performing a masking process on the color extraction image using the first mask image; and, if a ratio of the color extraction region to the circular region is less than the first threshold value, performing a masking process on the color extraction image using the second mask image; and, if a ratio of the color extraction region to the circular region is equal to or greater than a predetermined second threshold value, recognizing the color extraction region as a light of a lit traffic light.

5. 5. The traffic light recognition method according to claim 4, further comprising: performing a masking process on the color extraction image using the second mask image; and, if a ratio of the color extraction region to the circular region is less than the second threshold value, performing a masking process on the color extraction image using the third mask image; and, if a ratio of the color extraction region to the circular region is equal to or greater than a predetermined third threshold value, recognizing the color extraction region as a light of a lit traffic light.

6. 6. The traffic light recognition method according to claim 5, further comprising the steps of: performing a masking process on the color extraction image using the third mask image; and, if a ratio of the color extraction region to the circular region is less than the third threshold value, performing a masking process on the color extraction image using the first mask image; and, if a ratio of the color extraction region to the circular region is equal to or greater than a fourth threshold value that is smaller than the first threshold value, recognizing the color extraction region as a light of a lit traffic light.

7. A traffic light recognition device that recognizes a light of a traffic light that is turned on in front of a vehicle from an image captured by an imaging means mounted on the vehicle, an image acquisition unit that acquires an image captured by the imaging means; a light area selection unit that selects a light area indicating a position of a light of a lighted traffic signal from the image acquired by the image acquisition unit; a mask image generating unit that performs a circle detection process on the light area selected by the light area selecting unit, and generates a plurality of mask images having a circular area set based on the circle detected by the circle detection process and a mask area set in a portion other than the circular area; a color extraction image generating unit that performs a color extraction process on the light area selected by the light area selecting unit and generates a color extraction image having a color extraction area in which the color of a light that is turned on is extracted; a light recognition unit that uses the plurality of mask images generated by the mask image generation unit to perform a mask process on the color extraction image generated by the color extraction image generation unit and recognizes lights of a lit traffic light, The traffic light recognition device is characterized in that the mask image generation unit generates a plurality of mask images, each of which has a different shape of the circular region.

Images