Traffic light recognition device
Through color shooting and image processing technology, the light area of the signal machine and the search area are detected, which solves the problem of low recognition efficiency caused by unclear edges of the signal machine in the prior art, and realizes efficient search and accurate identification of arrow lamps.
Patent Information
- Application Number
- CN202410460871.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-02
- Filing Date
- 2024-04-17
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, when setting the signal area by template matching, it is difficult to identify the signal with unclear edges, especially in the case of long distances, it is difficult to narrow the search area of the arrow lamp, resulting in low recognition efficiency.
Using color shooting and image processing technology, the lighting area of the main body of the signal machine is detected, the search area of the same size as the lighting fixture is set, and the lighting fixture is turned off through edge and center brightness evaluation to determine the horizontal or vertical configuration of the signal machine, thereby Accurately locate the search area of the arrow fixture.
An efficient search for arrow lamp lighting is achieved, improving the accuracy and efficiency of signal recognition, especially when the edges are unclear or long distances.
Smart Images

Figure CN119068450A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a signal recognition device for recognizing a signal based on image information obtained by photographing a driving environment. Background Art
[0002] In recent years, many vehicles such as automobiles are equipped with driving assistance devices. Driving assistance devices basically implement driving assistance control by having functions such as following vehicle distance control (ACC: Adaptive Cruise Control) and lane centering control (ALKC: Active Lane Keep Centering). Through such driving assistance control, driving assistance devices can reduce the burden of driving operations on the driver and improve driving safety.
[0003] In addition, various technologies have been proposed in driving assistance devices for the purpose of further improving convenience, etc. For example, the following technology has been proposed: a technology for identifying a traffic light based on image data obtained by photographing the area in front of the vehicle, and automatically stopping or starting the vehicle when passing through an intersection, etc. according to the lighting state of the identified traffic light.
[0004] For example, Patent Document 1 discloses a signal recognition device for identifying a signal based on image data. The signal recognition device uses a template showing a standard signal shape to perform template matching. Thus, the signal recognition device sets a signal area. In addition, the signal recognition device extracts an area whose brightness value is greater than a judgment brightness value in the signal area as a light area. Furthermore, when there are multiple light areas, the signal recognition device determines whether there is a saturated area, and the outer edge shape of the light area or the outer edge shape of the saturated area for each light area. Through these determinations, the signal recognition device determines the light (lighting) of the three-light signal body and the arrow lamp.
[0005] Prior art literature
[0006] Patent Literature
[0007] Patent Document 1: Japanese Patent Application Publication No. 2022-108518 Summary of the invention
[0008] Technical issues
[0009] However, in order to set the signal area by template matching as in the technique disclosed in the above-mentioned patent document 1, it is necessary to project the edge showing the shape of the signal on the image. Therefore, in the technique disclosed in the patent document 1, it is difficult to set the signal area for a signal whose edge is not clear on the image, such as a signal that exists far away from the vehicle. Therefore, in the technique of the patent document 1, there is a risk that it is difficult to narrow the search area of the arrow lamp, etc., even if the lighting of the signal body is recognized.
[0010] An object of the present invention is to provide a signal recognition device capable of efficiently searching for lighting of arrow lamps.
[0011] Technical Solution
[0012] A traffic light recognition device according to one embodiment of the present invention comprises: a photographing unit for photographing the driving environment ahead of a vehicle in color; a light area detection unit for detecting the light area of a traffic light body on an image photographed by the photographing unit; a first lamp detection unit for detecting, based on the light area, the light that is on among a plurality of lamps arranged on the traffic light body; a first search area setting unit for setting search areas of the same size as the lamps at a plurality of positions on the image connected to the lighted lamps in the horizontal and vertical directions and at specific positions corresponding to the light colors of the light area; a second lamp detection unit for evaluating the brightness of the edge of the search area and the brightness of the center of the search area for each search area and detecting the light that is off and arranged on the traffic light body; and a horizontal and vertical determination unit for determining whether the traffic light body is arranged in a horizontal or vertical configuration based on the configuration of the lighted lamps and the light that is off.
[0013] In addition, another embodiment of the signal recognition device of the present invention comprises: a camera for taking color photos of the driving environment in front of the vehicle; and a processor, which performs the following actions: detecting the light area of the signal body on the image taken by the camera; detecting the lit lamps among the multiple lamps arranged on the signal body based on the light area; setting search areas of the same size as the lamps at multiple positions on the image connected to the lit lamps in the horizontal and vertical directions of the lit lamps and as specific positions corresponding to the light colors of the light area; evaluating the brightness of the edge of the search area and the brightness of the center of the search area for each search area, and detecting the turned-off lamps arranged on the signal body; and determining whether the signal body is arranged in a horizontal or vertical configuration based on the configuration of the lit lamps and the turned-off lamps.
[0014] Technical Effects
[0015] According to the signal recognition device of the present invention, it is possible to efficiently search for lighting of arrow lamps. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 This is a schematic diagram of the driving assistance device.
[0017] Figure 2 is a flow chart showing a signal recognition routine.
[0018] Figure 3 is a flowchart showing a black area determination subroutine.
[0019] Figure 4 2 is a flowchart showing the first horizontal and vertical determination subroutine.
[0020] Figure 5 2 is a flowchart showing the second horizontal and vertical determination subroutine.
[0021] Figure 6 It is a schematic diagram illustrating an image of the driving environment in front of the vehicle.
[0022] Figure 7 This is a plan view showing an example of a horizontal traffic signal with a blue lamp lit.
[0023] Figure 8 This is a plan view showing an example of a vertical traffic light with a blue lamp lit.
[0024] Fig. 9 It is an explanatory diagram showing the search areas of the yellow lamp and the red lamp when the blue lamp is turned on.
[0025] Fig.10 This is a plan view showing an example of a horizontal traffic signal with a yellow lamp lit.
[0026] Fig.11 This is a plan view showing an example of a vertical traffic light with a yellow lamp lit.
[0027] Fig.12 1 is an explanatory diagram showing the search areas of the blue lamp and the red lamp when the yellow lamp is turned on.
[0028] Fig.13 This is a plan view showing an example of a horizontal traffic light with a red lamp lit.
[0029] Fig.14 This is a plan view showing an example of a vertical traffic light with a red lamp lit.
[0030] Fig.15 It is an explanatory diagram showing the search areas of the blue lamp and the yellow lamp when the red lamp is turned on.
[0031] Fig.16 It is an explanatory diagram showing coordinates for black area determination set in each search area.
[0032] Fig.17 It is an explanatory diagram showing an arrow lamp search area for a traffic signal determined to be a horizontal type.
[0033] Fig.18 It is an explanatory diagram showing an arrow lamp search area for a traffic signal determined to be of a longitudinal type.
[0034] Fig.19 This is an explanatory diagram showing an arrow lamp search area for a traffic signal that cannot perform lateral or longitudinal determination.
[0035] Fig. 20 This is a plan view showing an example of a horizontal signal with arrow lamps lit.
[0036] Fig.21 This is a plan view showing an example of a vertical signal with arrow lamps lit.
[0037] Explanation of symbols
[0038] 1…Driving assistance devices
[0039] 10…Camera unit
[0040] 11…Stereo Camera
[0041] 11a…Main Camera
[0042] 11b…Secondary camera
[0043] 12…IPU
[0044] 13…Image recognition_ECU
[0045] 14…Driving_ECU
[0046] 22…E / G_ECU
[0047] 23…T / M_ECU
[0048] 24…BK_ECU
[0049] 25…PS_ECU
[0050] 32…Throttle valve actuator
[0051] 33…Hydraulic control circuit
[0052] 34…Brake actuator
[0053] 35…Electric power steering motor
[0054] 50…Signal
[0055] 50H…horizontal type signal
[0056] 50V...Signal with vertical configuration
[0057] 51…Signal body
[0058] 52…Lighting
[0059] 52B...Lighting (blue lighting)
[0060] 52R... lamp (red lamp)
[0061] 52Y... lamp (yellow lamp)
[0062] 53…Arrowhead lamp
[0063] 55…Lighting search area
[0064] 56…Arrow search area
[0065] M...Vehicle (this vehicle) DETAILED DESCRIPTION
[0066] Hereinafter, the mode of the present invention will be described with reference to the accompanying drawings. The accompanying drawings relate to one embodiment of the present invention. Figure 1 This is a diagram of the overall structure of the driving assistance device.
[0067] like Figure 1 As shown, the driving assistance device 1 is configured to include a camera unit 10 fixed to the upper center of the front portion of the cabin of a vehicle (host vehicle) M, for example.
[0068] The camera unit 10 includes a stereo camera 11 as an imaging unit, an image processing unit (IPU) 12 , an image recognition unit (image recognition_ECU) 13 , and a travel control unit (travel_ECU) 14 .
[0069] The stereo camera 11 includes a main camera 11a and a sub-camera 11b. The main camera 11a and the sub-camera 11b are arranged at left-right symmetrical positions, for example, sandwiching the center in the vehicle width direction. In addition, the main camera 11a and the sub-camera 11b each include an imaging element such as a CMOS and a color filter. Thus, the main camera 11a and the sub-camera 11b perform color imaging of the driving environment outside the vehicle from different viewpoints at a predetermined imaging cycle synchronized with each other.
[0070] The IPU 12 performs image processing on the driving environment image captured by the stereo camera 11 in a predetermined manner, and detects the edges of various objects such as the three-dimensional object and / or the dividing line on the road surface shown in the image. In addition, the IPU 12 obtains distance information based on the positional offset of the corresponding edges on the left and right images. Thus, the IPU 12 generates image information (distance image information) including the distance information.
[0071] Image recognition_ECU13 obtains information about the road on which the vehicle M is traveling (the road on which the vehicle M is traveling) based on the distance image information received from IPU12. For example, image recognition_ECU13 obtains the road curvature [1 / m] of the dividing line that divides the road on which the vehicle M is traveling to the left and right and the width between the left and right dividing lines (lane width). In addition, image recognition_ECU13 calculates the lane width based on the difference in curvature between the left and right dividing lines.
[0072] In addition, the image recognition_ECU 13 performs predetermined pattern matching on the distance image information, and recognizes three-dimensional objects such as guardrails, curbstones, and surrounding vehicles extending along the road. Here, in the recognition of three-dimensional objects in the image recognition_ECU 13, for example, the type of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, and the relative speed between the three-dimensional object and the vehicle M are recognized.
[0073] In addition, the image recognition_ECU 13 recognizes the traffic light in front of the vehicle M. In the traffic light recognition, the image recognition_ECU 13 recognizes the lighting state of each lamp (blue lamp, yellow lamp, and red lamp) arranged in the traffic light body. In addition, when an arrow lamp is added to the traffic light body, the image recognition_ECU 13 also recognizes the lighting state of the arrow lamp.
[0074] As described above, in the present embodiment, the stereo camera 11 , the IPU 12 , and the image recognition_ECU 13 correspond to a specific example of a traffic signal recognition device.
[0075] Here, for example, Figure 7 , 8 As shown, for the traffic lights installed on the road, there are traffic lights 50H in which the traffic light main body 51 is arranged in a horizontal type, and traffic lights 50V in which the traffic light main body 51 is arranged in a vertical type. In the traffic light main body 51 arranged in a horizontal type, the blue lamp 52B, the yellow lamp 52Y, and the red lamp 52R constituting the traffic light main body 51 are arranged in a horizontal direction (horizontal direction). In addition, in the traffic light main body 51 arranged in a vertical type, the blue lamp 52B, the yellow lamp 52Y, and the red lamp 52R constituting the traffic light main body 51 are arranged in a vertical direction (vertical direction).
[0076] Furthermore, among the traffic lights 50, there are traffic lights in which arrow lamps 53 are added to the traffic light main body 51. For example, when the red lamp 52R is lit, the arrow lamp 53 allows vehicles to pass only in the specific direction indicated by the arrow. In such traffic lights 50 in which arrow lamps 53 are added, the arrangement of the arrow lamps 53 with the red lamp 52R as the reference position is different between the traffic light main body 51 in the horizontal arrangement and the traffic light main body 51 in the vertical arrangement.
[0077] It should be noted that the recognition of the traffic signal 50 including the arrow lamp 53 will be described in detail later.
[0078] The various information recognized by the image recognition_ECU 13 is output to the driving_ECU (driving_ECU) 14 as driving environment information.
[0079] The travel_ECU 14 is a control unit for comprehensively controlling the driving assistance device 1 .
[0080] The driving_ECU 14 is connected to various control units such as an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, and a power steering control unit (PS_ECU) 25 via an in-vehicle communication line such as a CAN (Controller Area Network).
[0081] A throttle valve actuator 32 of an electronically controlled throttle valve and the like are connected to the output side of the E / G_ECU 22. Various sensors such as an acceleration sensor (not shown) are connected to the input side of the E / G_ECU 22.
[0082] The E / G_ECU 22 controls the driving of the throttle valve actuator 32 based on the control signal from the driving_ECU 14 or the detection signals from various sensors. Thus, the E / G_ECU 22 adjusts the intake air volume of the engine to generate the desired engine output. In addition, the E / G_ECU 22 outputs the signals such as the accelerator opening detected by various sensors to the driving_ECU 14.
[0083] The output side of T / M_ECU23 is connected to a hydraulic control circuit 33. In addition, various sensors such as a gear position sensor (not shown) are connected to the input side of T / M_ECU23. T / M_ECU23 performs hydraulic control of the hydraulic control circuit 33 based on the engine torque signal estimated in E / G_ECU22 and / or detection signals from various sensors. Thus, T / M_ECU23 activates the friction engagement element, pulley, etc. provided in the automatic transmission to change the engine output speed at the desired speed ratio. In addition, T / M_ECU23 outputs signals such as the gear position detected by various sensors to driving_ECU14.
[0084] A brake actuator 34 for adjusting the brake fluid pressure output to the wheel cylinders provided at each wheel is connected to the output side of the BK_ECU 24. In addition, various sensors such as a brake pedal sensor, a yaw rate sensor, a longitudinal acceleration sensor, and a vehicle speed sensor (not shown) are connected to the input side of the BK_ECU 24.
[0085] The BK_ECU 24 controls the driving of the brake actuator 34 based on the control signal from the driving_ECU 14 or the detection signals from various sensors. Thus, the BK_ECU 24 appropriately generates a braking force for each wheel for performing forced braking control and / or yaw rate control on the host vehicle M. In addition, the BK_ECU 24 outputs signals such as the braking operation state, yaw rate, longitudinal acceleration, and vehicle speed (host vehicle speed) detected by various sensors to the driving_ECU 14.
[0086] An electric power steering motor 35 that applies steering torque generated by the rotational force of the motor to the steering mechanism is connected to the output side of the PS_ECU 25. Various sensors such as a steering torque sensor and a steering angle sensor are connected to the input side of the PS_ECU 25.
[0087] The PS_ECU 25 controls the driving of the electric power steering motor 35 based on the control signal from the driving_ECU 14 or the detection signals from various sensors. Thus, the PS_ECU 25 generates a steering torque for the steering mechanism. In addition, the PS_ECU 25 outputs signals such as the steering torque and the steering angle detected by various sensors to the driving_ECU 14.
[0088] A manual driving mode, a first driving control mode and a second driving control mode, which are modes for driving control, and an evacuation mode are set as driving modes in the driving ECU 14. These driving modes can be selectively switched in the driving ECU 14.
[0089] Here, the manual driving mode is a driving mode that requires control by the driver. The manual driving mode is a driving mode in which the host vehicle M is driven according to driving operations such as steering, acceleration, and braking performed by the driver.
[0090] In addition, the first driving control mode is also a driving mode that requires control by the driver. That is, the first driving control mode is a so-called semi-automatic driving mode that appropriately combines, for example, the preceding vehicle following control (ACC: Adaptive Cruise Control) and the lane centering control (ALKC: Active Lane Keep Centering) and the lane departure prevention (Active Lane Keep Bouncing) control while reflecting the driving operation performed by the driver.
[0091] Here, the preceding vehicle following control is basically performed based on the running environment information input from the image recognition_ECU 13. That is, the preceding vehicle following control is performed based on, for example, the preceding vehicle information included in the running environment information.
[0092] In addition, the lane centering control and the lane departure suppression control are performed based on, for example, the running environment information input from the image recognition_ECU 13. That is, the lane centering control and the lane departure suppression control are performed based on the lane dividing line information and the like included in the running environment information.
[0093] The second driving control mode is an automatic driving mode that does not require the driver to perform steering, acceleration and braking operations, and makes the vehicle M travel by appropriately combining, for example, the preceding vehicle following control with the lane centering control and the lane departure suppression control.
[0094] The retreat mode is a mode for automatically stopping the vehicle M on the roadside strip, etc. when, for example, driving based on the second driving control mode becomes impossible to continue and the driving operation cannot be handed over to the driver (i.e., it is impossible to switch to the manual driving mode or the first driving control mode).
[0095] Furthermore, in each of the above-mentioned driving modes, the driving_ECU 14 appropriately performs emergency braking (AEB (Autonomous Emergency Braking): collision damage reduction braking) control for obstacles such as vehicles that are highly likely to collide with the host vehicle M.
[0096] Furthermore, when determining that it is difficult to avoid a collision with an obstacle by emergency braking control, the driving_ECU 14 can perform emergency steering control for avoiding a collision with an obstacle instead of emergency braking control or in combination with emergency braking control.
[0097] Here, emergency braking control and emergency steering control are basically performed based on the running environment information input from the image recognition_ECU 13. That is, emergency braking control and emergency steering control are performed based on obstacle information such as the preceding vehicle and / or the stopped vehicle included in the running environment information.
[0098] Furthermore, for example, when the second travel control mode is selected, the travel_ECU 14 can also perform automatic stop control and automatic start control of the host vehicle M according to the lighting state of the traffic signal.
[0099] Here, the automatic stop control and automatic start control for the traffic signal are basically performed based on the running environment information input from the image recognition_ECU 13. That is, the running_ECU 14 performs the automatic stop control and automatic start control for the traffic signal based on the identification information of the traffic signal included in the running environment information.
[0100] It should be noted that, when a mode other than the second travel control mode is selected, the travel_ECU 14 can also notify the driver of information such as the lighting state of the traffic signal.
[0101] Next, the specific contents of the traffic signal recognition performed by the image recognition_ECU 13 will be described.
[0102] In the signal recognition process, the image recognition_ECU 13, for example, uses the color image (for example, referring to Figure 6 ) detects the light area in the signal body. For example, the image recognition_ECU 13 detects the blue, yellow or red high-brightness area as the light area on the color image.
[0103] When detecting the light area, the image recognition_ECU 13 can refer to the distance image information generated by the IPU 12. By referring to the distance image information, the image recognition_ECU 13 can detect a high brightness area at a predetermined height above the road surface as the light area of the signal body.
[0104] Furthermore, the image recognition_ECU 13 detects a lighting lamp among a plurality of (for example, three) lamps arranged in the signal main body.
[0105] Such detection of lamps is performed, for example, based on the light area. For example, the image recognition_ECU13 detects the light part that is lit in the signal body, assuming that the light area detected on the image is consistent with the light part of the lamp. Furthermore, the image recognition_ECU13 estimates the size of the entire lamp based on the detected light part. For example, the image recognition_ECU13 calculates the size of the rectangular frame part surrounding the light part based on the size ratio of the light part to the frame part (frame body) in a general lamp. Thus, the image recognition_ECU13 estimates the size of the entire lamp including the light part and the frame part.
[0106] In addition, the image recognition_ECU13 sets a plurality of lamp search areas at a plurality of positions on the image connected to the detected lamp in the horizontal direction (lateral direction) and the vertical direction (longitudinal direction). These lamp search areas are areas for searching other lamps that are turned off in the main body of the signal. Therefore, the image recognition_ECU13 sets the size of each lamp search area on the image to the same size as the lamp that is lit. In addition, the image recognition_ECU13 determines the relative position of each lamp search area with respect to the lamp that is lit according to the light color of the light area.
[0107] For example, Figure 7 As shown, in the signal main body 51 of the horizontal type configuration, when the blue lamp 52B is lit, the other lamps (yellow lamp 52Y and red lamp 52R) that are turned off are arranged on the right side of the blue lamp 52B. Figure 8 As shown, in the vertically arranged signal main body 51, when the blue lamp 52B is on, the other lamps (yellow lamp 52Y and red lamp 52R) that are turned off are arranged above the blue lamp 52B. Based on the legal layout of each lamp 52 in each signal main body 51, the image recognition_ECU 13 sets two lamp search areas 55 on the right and above the blue lamp 52B in the image (refer to Fig. 9 ).
[0108] In addition, for example, Fig.10 As shown, in the signal main body 51 of the horizontal type arrangement, when the yellow lamp 52Y is lit, the other lamps (blue lamp 52B and red lamp 52R) that are turned off are arranged on the left and right of the yellow lamp 52Y. Fig.11As shown, in the vertically arranged signal main body 51, when the yellow lamp 52Y is on, the other lamps (blue lamp 52B and red lamp 52R) that are turned off are arranged above and below the yellow lamp 52Y. Based on the legal layout of each lamp 52 in each signal main body 51, the image recognition_ECU 13 sets a lamp search area 55 on the left, right, top and bottom of the yellow lamp 52Y in the image (refer to Fig.12 ).
[0109] In addition, for example, Fig.13 As shown, in the signal main body 51 of the horizontal type configuration, when the red lamp 52R is lit, the other lamps (blue lamp 52B and yellow lamp 52Y) that are turned off are arranged on the left side of the red lamp 52R. Fig.14 As shown, in the vertically arranged signal main body 51, when the red lamp 52R is on, the other lamps (blue lamp 52B and yellow lamp 52Y) that are turned off are arranged below the red lamp 52R. Based on the legal layout of each lamp 52 in each signal main body 51, the image recognition_ECU 13 sets two lamp search areas 55 on the left and below the red lamp 52R in the image (refer to Fig.15 ).
[0110] Furthermore, the image recognition_ECU 13 sets a plurality of black region determination coordinates for each of the set lamp search regions 55 .
[0111] Here, generally, the frame portion of each lamp 52 is painted with a relatively bright color such as light gray or white. On the other hand, the light portion of each lamp 52 is darker than the frame portion when the light is turned off, and becomes a color close to black. Therefore, the brightness of the frame portion on the image is relatively higher than the brightness of the light portion when the light is turned off. In other words, the light portion when the light is turned off forms a black area on the image that is darker than the frame portion. Therefore, in order to determine such a black area, the image recognition_ECU13 sets coordinates for black area determination at the edge of the lamp search area 55 and the center of the lamp search area 55.
[0112] More specifically, the image recognition_ECU 13 in this embodiment sets the black area determination coordinates c0, c1, c2, and c3, for example, near the four corners of the lamp search area 55. In addition, the image recognition_ECU 13 sets the black area determination coordinates c4, c5, c6, c7, and c8, for example, at five positions in the center of the lamp search area 55.
[0113] For example, Fig.16As shown in FIG. 1 , when the coordinate in the lateral direction of the lamp search area 55 is defined as i and the coordinate in the longitudinal direction of the lamp search area is defined as j, the coordinates of each lamp search area 55 can be expressed as follows.
[0114] c0: (is+α, je-α)
[0115] c1: (is+α, js+α)
[0116] c2: (ie-α, js+α)
[0117] c3: (ie-α, je-α)
[0118] c4: (ic, jc)
[0119] c5: (ic-α, jc+α)
[0120] c6: (ic-α, jc-α)
[0121] c7: (ic+α, jc-α)
[0122] c8: (ic+α, jc+α)
[0123] Here, is represents the starting point of the lamp search area 55 in the i-axis direction. In addition, ie represents the end point of the lamp search area 55 in the i-axis direction. In addition, ic represents the center point of the lamp search area 55 in the i-axis direction. In addition, js represents the starting point of the lamp search area 55 in the j-axis direction. In addition, je represents the end point of the lamp search area 55 in the j-axis direction. In addition, jc represents the center point of the lamp search area 55 in the j-axis direction. In addition, α represents a predetermined number of pixels on the image (for example, small pixels of more than 1 pixel).
[0124] When the black area determination coordinates are set, the image recognition_ECU 13 determines whether or not a black area exists in the center of the lamp search area 55 based on, for example, the following (first condition) to (fourth condition).
[0125] (First condition) The peripheral brightness Ln exceeds the first threshold value Lth1. That is, the image recognition_ECU13 calculates the average value of the brightness at the coordinates c0 to c3 for black area determination as the peripheral brightness Ln. Then, when the peripheral brightness Ln is below the first threshold value Lth1, the image recognition_ECU13 excludes the lamp search area 55 currently being processed from the lamp search object. Thus, the image recognition_ECU13 avoids performing lamp searches under overly dark conditions such as under an elevated road and / or at night. This is because when the peripheral brightness is too low, it is difficult to use brightness to distinguish between the peripheral portion formed by the frame portion and the central portion formed by the light portion.
[0126] (Second condition) The center brightness Lc does not exceed the second threshold value Lth2 (where Lth2 ≤ Lth1). That is, the image recognition_ECU 13 calculates the center brightness Lc by, for example, the average value of the brightness at the black area determination coordinates c4 to c8. Then, when the center brightness Lc is greater than the second threshold value Lth2, the image recognition_ECU 13 determines that there is no black area in the center of the lamp search area 55.
[0127] (Third condition) The peripheral brightness Ln is higher than the central brightness Lc. That is, as described above, in most signal bodies, the frame of the lamp has a bright color such as white, and the light of the lamp has a dark color such as black when the light is turned off. Therefore, when the peripheral brightness Ln is less than the central brightness Lc, the image recognition_ECU 13 determines that there is no black area in the center of the lamp search area 55.
[0128] (Fourth condition) The RGB brightness ratio of the center portion is within the range of the RGB brightness ratio set according to the camera specifications, etc. That is, the range of the RGB brightness ratio when, for example, photographing the light portion when the light is turned off is pre-set in the image recognition_ECU 13 based on the camera specifications, etc. Therefore, when the RGB brightness ratio of the center portion is outside the range of the RGB brightness ratio set according to the camera specifications, etc., the image recognition_ECU 13 determines that there is no black area in the center portion of the lamp search area 55.
[0129] It is desirable to perform such black area determination of each lamp search area 55 on the image for a set frame or more (for example, 3 frames or more).
[0130] Then, the image recognition_ECU 13 detects the lamp search area 55 where all the first to fourth conditions are satisfied at a predetermined ratio or more in the judgment of more than a plurality of frames as an area where a black area exists. In other words, the image recognition_ECU 13 detects the lamp search area 55 where a black area is determined to exist as an area where a lamp that is turned off exists.
[0131] Furthermore, the image recognition_ECU13 determines the configuration direction of the signal body including the lamps being lit based on the configuration of the detected lamps being turned off. In this determination, the image recognition_ECU13 compares, for example, the frequency of the lamps being turned off in the lateral direction of the lamps being lit with the frequency of the lamps being turned off in the longitudinal direction of the lamps being lit. Then, when the frequency detected in the lateral direction is higher than the frequency detected in the longitudinal direction, the image recognition_ECU13 determines that the signal body including the lamps being lit is in a lateral configuration. On the other hand, when the frequency detected in the lateral direction is lower than the frequency detected in the longitudinal direction, the image recognition_ECU13 determines that the signal body including the lamps being lit is in a longitudinal configuration.
[0132] It should be noted that such a horizontal and vertical determination of the signal main body can also be performed by the movement of the light area when the lighting of the lamp is switched. For example, when the light area moves horizontally with the switching of the lighting lamp, the image recognition_ECU13 determines that the signal main body including the lighting lamp is in a horizontal configuration. On the other hand, for example, when the light area moves vertically with the switching of the lighting lamp, the image recognition_ECU13 determines that the signal main body including the lighting lamp is in a vertical configuration.
[0133] If the lateral and longitudinal determination of the main body of the signal is performed, the image recognition_ECU13 sets the search area (arrow search area) of the arrow lamp attached to the main body of the signal. The arrow search area is set, for example, based on the red lamp 52R. For example, when the red lamp 52R is detected as a lamp being lit, the image recognition_ECU13 sets the arrow search area 56 based on the red lamp 52R being lit. In addition, for example, when the blue lamp 52B or the yellow lamp 52Y is detected as a lamp being lit, the image recognition_ECU13 sets the arrow search area 56 at the moment when the red lamp 52R is lit.
[0134] Here, in the case where the determination is not made as to whether the signal main body 51 is arranged in a horizontal type or in a vertical type, for example, Fig.19 As shown, the image recognition_ECU 13 sets arrow search areas 56 at nine locations around the red lamp 52R.
[0135] In this regard, when it is determined that the signal main body 51 is arranged in a horizontal type, for example, Fig.17As shown, the image recognition_ECU 13 sets three arrow search areas 56 arranged in the horizontal direction at the lower side of the red lamp 52R. That is, the image recognition_ECU 13 reduces the position of the arrow search area 56 from the nine arrow search areas 56 around the red lamp 52R. This is because it is difficult to assume that the arrow lamps 53 are arranged on the left and right sides and the upper side of the red lamp 52R when the signal main body 51 is arranged in the horizontal direction.
[0136] In addition, when it is determined that the signal main body 51 is arranged in a longitudinal type, for example, Fig.18 As shown, the image recognition_ECU 13 sets four arrow search areas arranged in the vertical direction to the left of the red lamp 52R. Furthermore, the image recognition_ECU 13 sets three arrow search areas arranged in the vertical direction to the right of the red lamp 52R. That is, the image recognition_ECU 13 reduces the position of the arrow search area 56 from the nine arrow search areas 56 around the red lamp 52R. This is because it is difficult to imagine arranging arrow lamps 53 above and below the red lamp 52R when the signal main body 51 is arranged in the vertical direction.
[0137] Furthermore, the image recognition_ECU 13 searches for the arrow lamp 53 of the signal main body 51. That is, the image recognition_ECU 13 searches for the arrow lamp 53 that lights up as the red lamp 52R lights up (for example, refer to Fig. 20 , 21 The search of the arrow lamp 53 is performed for each set arrow search area 56.
[0138] Thus, in this embodiment, the image recognition_ECU 13 corresponds to a specific example of a light area detection unit, a first lamp detection unit, a first search area setting unit, a second lamp detection unit, a horizontal and vertical determination unit, and a second search area setting unit.
[0139] Next, follow Figure 2 The flowchart of the traffic signal recognition routine shown in FIG. 1 is used to explain the traffic signal recognition processing performed in the image recognition_ECU 13. This routine is repeatedly executed in the image recognition_ECU 13 at every set time.
[0140] When the routine starts, the image recognition_ECU 13 checks whether a lighted light is detected ahead of the road on which the vehicle is traveling. That is, the image recognition_ECU 13 checks whether a light region (ie, light) is detected in the color image captured by the main camera 11 a , for example.
[0141] Then, in step S101 , when it is determined that the light part is not detected (step S101 : No), the image recognition_ECU 13 directly exits the routine.
[0142] On the other hand, when it is determined in step S101 that the light portion is detected (step S101 : Yes), the image recognition_ECU 13 proceeds to step S102 .
[0143] In step S102 , the image recognition_ECU 13 checks whether the light unit detected this time and the light unit detected last time are installed in the same signal.
[0144] Here, the determination of whether the current lighting unit and the previous lighting unit are the lighting units installed at the same signal can be determined, for example, based on the distance image generated in the IPU 12. That is, the image recognition_ECU 13 estimates the relative distance between the current lighting unit and the previous lighting unit in the actual space based on the distance image of each frame. Thus, when the relative distance between the lighting units in the actual space is less than the set distance (for example, 1m), the image recognition_ECU 13 can determine that the lighting units are installed at the same signal.
[0145] Then, in step S102 , when it is determined that the current light unit and the previous light unit are not installed in the same light unit of the signal (step S102 : No), the image recognition_ECU 13 proceeds to step S104 .
[0146] On the other hand, when it is determined in step S102 that the current light unit and the previous light unit are light units installed at the same signal (step S102 : Yes), the image recognition_ECU 13 proceeds to step S103 .
[0147] In step S103 , the image recognition_ECU 13 checks whether the light color of the light part detected this time has changed from the light color of the light part detected last time.
[0148] Then, in step S103 , when it is determined that the light color of the light part detected this time has changed from that detected last time (step S103 : Yes), the image recognition_ECU 13 proceeds to step S106 .
[0149] On the other hand, when it is determined in step S103 that the light color of the light part detected this time has not changed from that detected last time (step S103 : No), the image recognition_ECU 13 proceeds to step S104 .
[0150] If the process proceeds from step S102 or step S103 to step S104, the image recognition ECU 13 determines the black area of the currently detected light portion relative to the surrounding area. The black area determination is performed, for example, according to Figure 3 The black area determination subroutine is executed according to the flowchart of the black area determination subroutine shown.
[0151] When the subroutine starts, the image recognition_ECU 13 estimates the size of the lamp on the image based on the currently detected light part in step S201. That is, the image recognition_ECU 13 calculates the size of the frame part on the image based on the size ratio between the light part and the frame part in a general lamp. Thus, the image recognition_ECU 13 estimates the size of the lamp on the image.
[0152] In the next step S202, the image recognition_ECU 13 sets a search area (black area search area) corresponding to the light color of the currently detected light part on the image. For example, when the currently detected light fixture is a blue light fixture 52B, the image recognition_ECU 13 sets two light fixture search areas 55 on the right side and the upper side of the blue light fixture 52B (see FIG. Fig. 9 In addition, for example, when the currently detected lamp is a yellow lamp 52Y, the image recognition ECU 13 sets a lamp search area 55 on the left side, right side, top side and bottom side of the yellow lamp 52Y respectively (refer to Fig.12 ). In addition, for example, when the currently detected lamp is a red lamp 52R, the image recognition_ECU 13 sets two lamp search areas 55 on the left and bottom sides of the red lamp 52R (see Fig.15 ).
[0153] In the next step S203 , the image recognition_ECU 13 extracts one lamp search area 55 that has not been extracted yet from the lamp search areas 55 set in step S202 .
[0154] In the next step S204, the image recognition_ECU 13 sets the coordinates c0 to c8 for black area determination in the extracted lamp search area 55 (see Fig.16 ).
[0155] In the next step S205, the image recognition_ECU 13 performs black area determination based on the brightness values of the coordinates c0 to c8 on the image for each black area determination. That is, the image recognition_ECU 13 performs black area determination using the first to fourth conditions described above. Here, in this embodiment, the black area determination is a determination of whether a black area exists in the center of the currently extracted lamp search area 55.
[0156] In the next step S206 , the image recognition_ECU 13 checks whether the brightness value at each of the coordinates c0 to c8 in the currently extracted lamp search area 55 satisfies the first to fourth conditions.
[0157] Then, in step S206 , when it is determined that the brightness value at each of the coordinates c0 to c8 satisfies all of the first to fourth conditions (step S206 : Yes), the image recognition_ECU 13 proceeds to step S207 .
[0158] When the process proceeds from step S206 to step S207 , the image recognition_ECU 13 sets a black region candidate flag F to “1”, indicating that there is a high possibility that a black region exists in the currently extracted lamp search region.
[0159] On the other hand, when it is determined in step S206 that the brightness value at each of the coordinates c0 to c8 does not satisfy at least any one of the first to fourth conditions (step S207 : No), the image recognition_ECU 13 proceeds to step S208 .
[0160] When the process proceeds from step S206 to step S208 , the image recognition_ECU 13 clears the black area candidate flag F to “0”.
[0161] When the process proceeds from step S207 or step S208 to step S209 , the image recognition_ECU 13 checks whether all lamp search areas have been extracted.
[0162] Then, in step S209 , when it is determined that all lamp search areas have not been extracted (step S209 : No), the image recognition_ECU 13 returns to step S203 .
[0163] On the other hand, when it is determined in step S209 that all lamp search areas have been extracted (step S209 : Yes), the image recognition_ECU 13 exits the subroutine.
[0164] exist Figure 2 In the main routine of FIG. 104, if the process proceeds from step S104 to step S105, the image recognition ECU 13 performs a lateral and longitudinal determination of the signal body including the currently detected lamp. The lateral and longitudinal determination is performed, for example, according to Figure 4 The flowchart of the first horizontal and vertical determination subroutine is shown.
[0165] When the subroutine starts, in step S301 , the image recognition_ECU 13 checks whether the current signal has been changed to a signal different from the signal in the previous frame.
[0166] Here, the current signal is a signal that should be identified in correspondence with the nearest lamp currently detected in front of the road on which the vehicle is traveling. In addition, the signal until the previous frame is a signal that should be identified in correspondence with the nearest lamp detected in front of the road on which the vehicle is traveling until the previous frame. The nearest lamp in front of the road on which the vehicle is traveling is switched every time the vehicle M passes through the signal. Therefore, the judgment of whether there is a change in these signals can be made, for example, based on whether the nearest lamp in front of the road on which the vehicle is traveling is switched.
[0167] Then, in step S301 , when it is determined that there is no change in the traffic signal (step S301 : No), the image recognition_ECU 13 proceeds to step S303 .
[0168] On the other hand, when it is determined in step S301 that the traffic signal has been changed (step S301 : Yes), the image recognition_ECU 13 proceeds to step S302 .
[0169] In step S302, the image recognition_ECU 13 clears both the horizontal determination score P1 and the vertical determination score P2 for the current traffic light to "0". Here, the horizontal determination score P1 is a score indicating that the current traffic light is likely to have the lamps arranged horizontally. In addition, the vertical determination score P2 is a score indicating that the current traffic light is likely to have the lamps arranged vertically.
[0170] When the process proceeds from step S301 or step S302 to step S303 , the image recognition_ECU 13 checks whether all black area candidate flags F of the lamp search areas set in the lateral direction of the currently detected lamp are “1”.
[0171] Then, in step S303 , when it is determined that at least one of the horizontal black area candidate flags F is not “1” (step S303 : No), the image recognition_ECU 13 proceeds to step S305 .
[0172] On the other hand, when it is determined in step S303 that all the horizontal black area candidate flags F are “1” (step S303 : Yes), the image recognition_ECU 13 proceeds to step S304 .
[0173] When the process proceeds from step S303 to step S304, the image recognition_ECU 13 increases the lateral determination score P1 (P1←P1+1), and then proceeds to step S305. It should be noted that the point by which the lateral determination score P1 is increased is not limited to "1", and may be changed according to the reliability of the information, for example.
[0174] When the process proceeds from step S303 or step S304 to step S305 , the image recognition_ECU 13 checks whether all black area candidate flags F of the lamp search areas set in the longitudinal direction of the currently detected lamp are “1”.
[0175] Then, in step S305 , when it is determined that at least one of the vertical black area candidate flags F is not “1” (step S305 : No), the image recognition_ECU 13 proceeds to step S307 .
[0176] On the other hand, when it is determined in step S305 that all the vertical black area candidate flags F are “1” (step S305 : Yes), the image recognition_ECU 13 proceeds to step S306 .
[0177] When the process proceeds from step S305 to step S306, the image recognition_ECU 13 increases the longitudinal determination score P2 (P2←P2+1), and then proceeds to step S307. It should be noted that the point by which the longitudinal determination score P2 is increased is not limited to "1", and may be changed according to the reliability of the information, for example.
[0178] When the process proceeds from step S305 or step S306 to step S307 , the image recognition_ECU 13 checks whether the absolute value |P1-P2| of the difference between the lateral determination score P1 and the longitudinal determination score P2 is greater than a preset threshold value (eg, a difference of 3 points).
[0179] Then, in step S307 , when it is determined that the absolute value of the score difference |P1−P2| is smaller than the threshold value (step S307 : No), the image recognition_ECU 13 proceeds to step S308 .
[0180] When the process proceeds from step S307 to step S308 , the image recognition_ECU 13 exits the subroutine after determining that the lateral and longitudinal determination of the signal to be currently recognized cannot be performed.
[0181] On the other hand, when it is determined in step S307 that the absolute value of the score difference |P1-P2| is equal to or larger than the threshold value (step S307: Yes), the image recognition_ECU 13 proceeds to step S309.
[0182] In step S309 , the image recognition_ECU 13 checks whether the lateral determination score P1 is greater than the longitudinal determination score P2 .
[0183] Then, in step S309 , when it is determined that the lateral determination score P1 is greater than the longitudinal determination score P2 (step S309 : Yes), the image recognition_ECU 13 proceeds to step S310 .
[0184] When the process proceeds from step S309 to step S310 , the image recognition_ECU 13 exits the subroutine after determining that the signal to be currently recognized is a transverse signal.
[0185] On the other hand, when it is determined in step S309 that the lateral determination score P1 is equal to or less than the longitudinal determination score P2 (step S309 : No), the image recognition_ECU 13 proceeds to step S311 .
[0186] When the process proceeds from step S309 to step S311 , the image recognition_ECU 13 exits the subroutine after determining that the signal to be currently recognized is a longitudinal signal.
[0187] exist Figure 2 In the main routine of FIG. 100 , if the process proceeds from step S103 to step S106 , the image recognition ECU 13 performs a lateral and longitudinal determination of the signal body including the currently detected lamp. The lateral and longitudinal determination is performed, for example, according to Figure 5 The flowchart of the second horizontal and vertical determination subroutine is shown.
[0188] If the subroutine starts, in step S401, the image recognition_ECU 13 checks whether the horizontal and vertical determination results for the current signal already exist. Here, the situation where the horizontal and vertical determination results for the current signal already exist refers to, for example, the situation where the horizontal and vertical determination results based on the above-mentioned steps S104 and S105 exist before the color of the light being lit changes.
[0189] Then, in step S401 , when it is determined that there is a lateral and longitudinal determination result for the current signal (step S401 : Yes), the image recognition_ECU 13 proceeds to step S402 .
[0190] In step S402 , the image recognition_ECU 13 exits the subroutine while retaining the current horizontal and vertical determination result.
[0191] On the other hand, when it is determined in step S401 that there is no lateral or longitudinal determination result for the current signal (step S401 : No), the image recognition_ECU 13 proceeds to step S403 .
[0192] In step S403, the image recognition_ECU 13 checks whether the lighting position of the lamp in the actual space changes horizontally due to the change of the light color. In other words, the image recognition_ECU 13 checks whether the lighting position of the lamp changes horizontally due to the switching of the lighting lamp.
[0193] Then, in step S403 , when it is determined that the lighting position changes in the lateral direction (step S403 : Yes), the image recognition_ECU 13 proceeds to step S404 .
[0194] In step S404 , the image recognition_ECU 13 determines that the signal to be currently recognized is a transverse type signal, and then exits the subroutine.
[0195] On the other hand, when it is determined in step S403 that the lighting position has not changed in the lateral direction (step S403 : No), the image recognition_ECU 13 proceeds to step S405 .
[0196] In step S405, the image recognition_ECU 13 checks whether the lighting position of the lamp in the actual space changes in the vertical direction according to the change of the light color. In other words, the image recognition_ECU 13 checks whether the lighting position of the lamp changes in the vertical direction according to the switching of the lighting lamp.
[0197] Then, in step S405 , when it is determined that the lighting position changes in the longitudinal direction (step S405 : Yes), the image recognition_ECU 13 proceeds to step S406 .
[0198] In step S406 , the image recognition_ECU 13 determines that the signal to be currently recognized is a longitudinal signal, and then exits the subroutine.
[0199] On the other hand, when it is determined in step S405 that the lighting position has not changed in the longitudinal direction (step S405 : No), the image recognition_ECU 13 directly exits the subroutine.
[0200] exist Figure 2 In the main routine of , if the process proceeds from step S105 or step S106 to step S107, the image recognition_ECU 13 checks whether the light part currently lit is red. That is, the image recognition_ECU 13 checks whether the lamp currently lit is a red lamp.
[0201] Then, in step S107 , when it is determined that the currently lit light portion is not red (step S107 : No), the image recognition_ECU 13 exits the routine.
[0202] On the other hand, when it is determined in step S107 that the currently lit light portion is red (step S107 : Yes), the image recognition_ECU 13 proceeds to step S108 .
[0203] In step S108 , the image recognition ECU 13 checks whether there is a horizontal or vertical determination result for the current signal. In other words, the image recognition ECU 13 checks whether a horizontal or vertical determination result for the current signal has been obtained in at least one of the processes of step S105 and step S106 .
[0204] Then, in step S108 , when it is determined that the lateral and longitudinal determination result for the current signal has not been obtained (step S108 : No), the image recognition_ECU 13 proceeds to step S109 .
[0205] In step S109, the image recognition ECU 13 sets all arrow search areas around the currently lit red lamp (refer to Fig.19 ).
[0206] On the other hand, when it is determined in step S108 that the lateral and longitudinal determination result of the current signal has been obtained (step S108 : Yes), the image recognition_ECU 13 proceeds to step S110 .
[0207] In step S110, the image recognition ECU 13 reduces the arrow search area set around the red lamp. That is, the image recognition ECU 13 sets an arrow search area around the red lamp corresponding to the horizontal and vertical determination results of the traffic light (refer to Fig.17 , 18 ).
[0208] When the routine proceeds from step S109 or step S110 to step S111 , the image recognition_ECU 13 searches for the arrow lamps that are lit in each set arrow search area, and then exits the routine.
[0209] In such an embodiment, the image recognition_ECU13 detects the lighting area of the main body of the signal on the image obtained by color shooting in front of the vehicle M, thereby detecting the lighting lamps among the multiple lamps arranged in the main body of the signal. In addition, the image recognition_ECU13 sets the lamp search area of the same size as the lighting lamp at multiple positions on the image connected to the lighting lamp in the horizontal direction and the vertical direction and corresponding to the light color of the light area. In addition, the image recognition_ECU13 evaluates the brightness of the edge of the lamp search area and the brightness of the center of the lamp search area for each lamp search area, and detects the lamps arranged in the main body of the signal that are turned off. Furthermore, the image recognition_ECU13 determines whether the main body of the signal is arranged in a horizontal type or a vertical type based on the configuration of the lighting lamps and the turned-off lamps. Then, the image recognition_ECU13 sets the search area of the arrow lamps attached to the main body of the signal according to the determined configuration of the main body of the signal. In this way, the lighting of the arrow lamps can be searched efficiently.
[0210] That is, the image recognition_ECU 13 detects the lighting lamp by assuming that the light area on the color image is consistent with the light part of the lamp. Thus, even if the boundaries of the lamps arranged in the main body of the signal are not clear in the image, the lighting lamp can be accurately detected.
[0211] In addition, the image recognition_ECU 13 sets the lamp search area at the position connected to the lit lamp in the horizontal direction and the vertical direction of the lit lamp and at the specific position corresponding to the lighting color of the lit lamp. Such processing is performed with a view to the fact that the arrangement order of the lamps of different colors in the horizontally configured signal machine body and the vertically configured signal machine body is known. In this way, the lamps included in the signal machine body that are turned off can be searched efficiently.
[0212] In addition, the image recognition_ECU 13 evaluates the brightness of the edge of the lamp search area and the brightness of the center of the lamp search area for each lamp search area, and detects the lamps set in the main body of the signal that are turned off based on the evaluation results. Such processing is performed with an eye on the fact that the light part of the lamp that is turned off is a black area with a lower brightness than the frame part. As a result, even if the boundaries of the lamps arranged in the main body of the signal are not clear in the image, the lamps that are turned off can be accurately detected.
[0213] Furthermore, the image recognition_ECU 13 determines whether the signal main body is arranged in a horizontal or vertical configuration based on the configuration of the lighting lamps and the lighting lamps in the off state. Then, by setting the arrow search area according to the configuration in the signal main body, unnecessary arrow search areas can be excluded. Therefore, the lighting of arrow lamps can be searched efficiently.
[0214] Here, in the above-mentioned embodiment, the image recognition_ECU13, the driving_ECU14, the E / G_ECU22, the T / M_ECU23, the BK_ECU24 and the PS_ECU25 are composed of a well-known microcomputer having a CPU, a RAM, a ROM, a non-volatile storage unit and the like and its peripheral devices, and the ROM pre-stores a program executed by the CPU and / or fixed data such as a data table, etc. It should be noted that all or part of the functions of the processor may be composed of a logic circuit or an analog circuit, and in addition, the processing of various programs may be realized by an electronic circuit such as an FPGA.
[0215] The invention described in the above embodiments is not limited to these modes, and various modifications can be implemented in the implementation stage without departing from the scope of the gist thereof. In addition, the above modes include inventions at various stages, and various inventions can be extracted by appropriate combinations of the disclosed multiple constituent elements.
[0216] For example, when the above-described problem can be solved and the above-described effect can be obtained even if some of the above-described components are deleted from all the components, the configuration without the components can be extracted as an invention.
Claims
1. A signal recognition device, It is characterized in that have: A photographing unit for taking color photographs of the driving environment in front of the vehicle; A light area detection unit detects the light area of the signal main body on the image captured by the capturing unit; a first lamp detection unit, for detecting the lighted lamp among the plurality of lamps arranged in the signal main body based on the light area; A first search area setting unit sets search areas of the same size as the lamp at a plurality of positions on the image connected to the lamp in the horizontal direction and the vertical direction of the lamp and corresponding to the light color of the light area, respectively; a second lamp detection unit, for each of the search areas, evaluating the brightness of the edge of the search area and the brightness of the center of the search area, and detecting the lamps that are turned off and arranged on the main body of the signal; as well as The horizontal and vertical determination unit determines whether the signal main body is arranged in a horizontal type or a vertical type based on the arrangement of the lighted lamps and the lighted lamps.
2. The signal recognition device according to claim 1, It is characterized in that The signal recognition device further includes a second search area setting unit that sets a search area of an arrow lamp attached to the signal main body according to the arrangement of the signal main body.
3. The signal recognition device according to claim 1, It is characterized in that The horizontal / vertical determination unit determines whether the signal main body is arranged in a horizontal or vertical configuration based on the configuration of the light fixtures in the turned-off state detected on the images of a plurality of frames.
4. A signal recognition device, It is characterized in that have: A camera that takes color photos of the driving environment in front of the vehicle; and processor, The processor performs the following actions: Detecting the light area of the main body of the signal light on the image captured by the camera; Detecting the lighting lamp among the plurality of lamps arranged in the signal main body based on the light area; At a plurality of positions on the image connected to the lit lamp in the horizontal direction and the vertical direction of the lit lamp and corresponding to the light color of the light area, respectively set a search area of the same size as the lamp; For each of the search areas, the brightness of the edge of the search area and the brightness of the center of the search area are evaluated, and the lamps that are turned off and arranged in the main body of the signal are detected; as well as Based on the arrangement of the lighting fixtures and the lighting fixtures being turned off, it is determined whether the signal main body is arranged in a horizontal type or a vertical type.
Citation Information
Patent Citations
Traffic light recognition method and traffic light recognition apparatus
JP2022108518A
