Light status recognition device

Abstract

To provide a lamp state recognition device capable of continuing recognition in a lighting state of arrow lamp light when the arrow lamp light in the lighting state is visually cut off, at a signal apparatus provided with the arrow lamp light.SOLUTION: A lamp state recognition device is provided with: a signal apparatus recognition unit for recognizing a signal apparatus on an imaged image at each cycle of a fixed time; an arrow lamp light determination unit for determining whether or not, on the imaged image, arrow lamp light of the signal apparatus provided with the arrow lamp light is turned on when the signal apparatus is recognized by the signal apparatus recognition unit; a visual cut-off determination area setting unit for setting a visual cut-off determination area including the signal apparatus provided with the arrow lamp light on the imaged image of a front camera when it is determined that the arrow lamp light is turned on; an arrow lamp light visual cut-off determination area for determining whether or not the arrow lamp light is present in the visual cut-off determination area at each cycle; and a lighting state recognition unit for recognizing that a lighting state of the arrow lamp light is continued for a preset first setting time, when it is determined that no arrow lamp light is present in the visual cut-off determination area, at the following cycle to which it is determined that the arrow lamp light is present in the visual cut-off determination area.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a light condition recognition device. [Background technology]

[0002] Conventionally, JP 2021-002275 A is known as a technical document related to a light state determination device. This publication describes a traffic light recognition system that improves the recognition accuracy of traffic lights around a vehicle by comparing a predetermined light pattern of a light part of a traffic light imaged by a camera with signal detection information of the imaged traffic light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-002275 Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is difficult for a camera to recognize the lit arrow lights of a traffic light equipped with arrow lights compared to the main lights. If the arrow lights in front are not visible due to the distance between the vehicle and the traffic light or the camera's angle of view, and only the main lights are recognized, there is a problem that the light state of the traffic light will be recognized incorrectly. [Means for solving the problem]

[0005] One aspect of the present invention is a light state recognition device that recognizes the light state of a traffic light with an arrow light in front of a vehicle, the light state recognition device including: a traffic light recognition unit that recognizes a traffic light on an image captured by a front camera of the vehicle at a fixed time cycle based on an image captured by the front camera of the vehicle; an arrow light determination unit that determines whether an arrow light of the traffic light with an arrow light is on or not on the captured image when the traffic light recognition unit recognizes the traffic light; a cut-off determination area setting unit that sets an out-of-view determination area including the traffic light with an arrow light on the image captured by the front camera when the arrow light determination unit determines that the arrow light is on; an arrow light cut-off determination unit that determines whether an arrow light is present in the cut-off determination area for each cycle after setting the cut-off determination area; and a light state recognition unit that recognizes that the turn-on state of the arrow light in the previous cycle in which the arrow light was determined to be present continues for a first set time when the arrow light cut-off determination unit determines that the arrow light is not present in the cut-off determination area in the next cycle in which the arrow light cut-off determination unit determines that the arrow light is present in the cut-off determination area. Based on past changes in the lighting state of the traffic light with an arrow light, the timing of the change in the lighting state of the traffic light with an arrow light is grasped, and if it is determined that the arrow light is not present in the cut-off judgment area at a timing later than the timing of the change in the lighting state of the arrow light after it is determined that the arrow light in the traffic light with an arrow light is on, the arrow light is not recognized as being continuously on even if it is determined that the arrow light is not present in the cut-off judgment area in the next cycle in which it is determined that the arrow light is present in the cut-off judgment area. .

[0006] According to one embodiment of the light state recognition device of the present invention, when the arrow light out-of-view judgment unit judges that an arrow light is present within the out-of-view judgment area in the next cycle, but judges that an arrow light is not present within the out-of-view judgment area, it recognizes that the arrow light's lit state in the previous cycle in which it was judged that an arrow light was present will continue for a predetermined first set time, so that the recognition of the arrow light's lit state can continue even if the arrow light is out of view.

[0007] In one embodiment of the light state recognition device of the present invention, after the out-of-view judgment area is set, the device further includes a main light out-of-view judgment unit which judges whether or not the main light of a traffic light with an arrow light is present within the out-of-view judgment area for each cycle, and when it is determined that the arrow light is not present within the out-of-view judgment area in the next cycle after the main light out-of-view judgment unit has determined that the arrow light is present within the out-of-view judgment area, the light state recognition unit may recognize that the lit state of the arrow light in the previous cycle in which it was determined that the arrow light was present will continue for a predetermined second set time. According to this light state recognition device, if the main light out-of-view judgment unit judges that the arrow light is present within the out-of-view judgment area in the next cycle, but judges that the arrow light is not present within the out-of-view judgment area, the arrow light's lit state in the previous cycle in which it was judged that the arrow light was present continues for a preset second set time, so that the recognition of the arrow light's lit state can be continued even if the main light is out of view.

[0008] In a light state recognition device according to one embodiment of the present invention, the cut-off judgment area setting unit may calculate a predicted position of the traffic light with an arrow light on the captured image in the next cycle based on the position of the traffic light with an arrow light on the captured image in the past cycle, and set the cut-off judgment area to include the predicted position of the traffic light with an arrow light on the captured image. According to this light state recognition device, the predicted position of the traffic light with an arrow light on the captured image in the next cycle is calculated based on the position of the traffic light with an arrow light on the captured image in the past cycle, and the cut-off judgment area is set to include the predicted position of the traffic light with an arrow light on the captured image, so that the cut-off judgment area can be set to a more appropriate position. Effect of the Invention

[0009] According to one aspect of the present invention, when a lit arrow light in a traffic light equipped with an arrow light is no longer visible, it is possible to continue to recognize the lit state of the arrow light. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a light state recognition device according to an embodiment; [Diagram 2] FIG. 13 is a diagram for explaining the setting of an out-of-view determination area. [Diagram 3] 13 is a flowchart showing an example of a cut-off determination area setting process. [Figure 4] 13A is a flowchart showing an example of an arrow light out-of-sight process, and FIG. 13B is a flowchart showing an example of a main light out-of-sight process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] A driving assistance device (light state recognition device) 100 shown in FIG. 1 is mounted on a vehicle (host vehicle) such as a passenger car, and is a device that assists the driver in driving the host vehicle. The driving assistance device 100 provides driving assistance to the driver according to the light state of a traffic light in front of the host vehicle. The light state of a traffic light includes a pass permission state (e.g., green light) that permits the vehicle to pass, a pass prohibition state (e.g., red light) that prohibits the vehicle from passing, and a signal transition state (e.g., yellow light) that transitions from the pass permission state to the pass prohibition state. The signal transition state does not necessarily have to be included in the light state of the traffic light. The light state of a traffic light with an arrow light also includes the light state of the arrow light.

[0013] [Configuration of driving assistance device (light status recognition device)] The configuration of the driving assistance device 100 will be described below with reference to the drawings. As shown in FIG. 1, the driving assistance device 100 includes a driving assistance ECU (Electronic Control Unit) 10 that performs overall management of the device. The driving assistance ECU 10 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. In the driving assistance ECU 10, for example, various functions are realized by the CPU executing a program stored in the ROM. The driving assistance ECU 10 may be composed of multiple electronic units.

[0014] The driving assistance ECU 10 is connected to a front camera 1, an actuator 2, and an HMI [Human Machine Interface] 3. The front camera 1 is a camera for capturing an image of the area in front of the vehicle. The front camera 1 is provided, for example, on the back side of the windshield of the vehicle. The front camera 1 transmits a captured image of the area in front of the vehicle to the driving assistance ECU 10. The front camera 1 may be a monocular camera or a stereo camera.

[0015] The actuator 2 is a device used to control the host vehicle. The actuator 2 includes at least a drive actuator and a brake actuator. The actuator 2 may include a steering actuator. The drive actuator controls the amount of air supplied to the engine (throttle opening) in response to a control signal from the driving assistance ECU 10, thereby controlling the driving force of the host vehicle. If the host vehicle is a hybrid vehicle, in addition to the amount of air supplied to the engine, a control signal from the driving assistance ECU 10 is input to a motor serving as a power source to control the driving force. If the host vehicle is an electric vehicle, a control signal from the driving assistance ECU 10 is input to a motor serving as a power source to control the driving force. The motor serving as a power source in these cases constitutes the actuator 2.

[0016] The brake actuator controls a brake system in response to a control signal from the driving assistance ECU 10, and controls the braking force applied to the wheels of the vehicle. For example, a hydraulic brake system can be used as the brake system. The steering actuator controls the drive of an assist motor that controls the steering torque of the electric power steering system in response to a control signal from the driving assistance ECU 10. In this way, the steering actuator controls the steering torque of the vehicle.

[0017] The HMI 3 is an interface for inputting and outputting information between the driving assistance device 100 and the driver. The HMI 3 includes, for example, a display, a speaker, and the like. The HMI 3 outputs images from the display and sounds from the speaker in response to a control signal from the driving assistance ECU 10. The display may be a HUD [Head Up Display] or a multi-information display provided on an instrument panel.

[0018] Next, a functional configuration of the driving assistance ECU 10 will be described. As shown in Fig. 1, the driving assistance ECU 10 has a traffic light recognition unit 11, an arrow light determination unit 12, a view cut determination area setting unit 13, an arrow light view cut determination unit 14, a main light view cut determination unit 15, a light state recognition unit 16, a light state storage unit 17, and a driving assistance unit 18. Note that some of the functions of the driving assistance ECU 10 described below may be executed by a server capable of communicating with the vehicle.

[0019] The traffic light recognition unit 11 recognizes a traffic light in front of the vehicle based on an image captured by the front camera 1. The traffic light recognition unit 11 recognizes the casing of the traffic light on the captured image. The traffic light recognition unit 11 repeats the recognition of the casing of the traffic light on the captured image at regular time cycles. The cycle is, for example, a processing cycle of the driving assistance ECU 10. The cycle of the traffic light recognition unit 11 may correspond to the transmission timing of a signal related to the image captured by the front camera 1.

[0020] The method of recognizing a traffic light by the traffic light recognition unit 11 is not particularly limited, and the casing of the traffic light in the captured image can be recognized by image processing such as edge extraction processing, noise removal processing, pattern matching, deep learning from feature extraction, etc. The image processing used in the following description includes at least one of edge extraction processing, noise removal processing, pattern matching, and deep learning.

[0021] The traffic light recognition unit 11 may recognize that the traffic light is equipped with an arrow light from the shape of the traffic light's housing. The traffic light recognition unit 11 is not limited to recognizing the housing as long as it can recognize the presence of the traffic light. The traffic light recognition unit 11 may detect the lighting state of the main light.

[0022] The traffic light recognition unit 11 also tracks the traffic light on the captured image. The traffic light recognition unit 11 tracks the traffic light by comparing the position of the traffic light on the captured image in the past cycle with the position of the traffic light on the captured image in the current cycle, thereby determining the identity of the traffic light.

[0023] The traffic light recognition unit 11 may determine whether or not a traffic light ahead of the host vehicle is a traffic light intended for the host vehicle. A traffic light intended for the host vehicle is a traffic light corresponding to the lane in which the host vehicle is traveling. The host vehicle needs to determine whether to proceed or stop depending on the lighting state of the traffic light intended for the host vehicle. The traffic light recognition unit 11 determines whether or not a traffic light ahead of the host vehicle is a traffic light intended for the host vehicle by recognizing the lane in which the host vehicle is traveling and the traffic light corresponding to the lane, for example, based on an image captured by the front camera 1.

[0024] The arrow light determination unit 12 determines whether or not the arrow light of the traffic light with arrow light is on in the captured image. The arrow light determination unit 12 determines whether or not the arrow light is on by image processing. The arrow light determination unit 12 may determine the arrow light only when the traffic light ahead of the vehicle is a traffic light with arrow light, or may determine the arrow light for all traffic lights.

[0025] When the arrow light determination unit 12 determines that the arrow light is on, the cut-out determination area setting unit 13 sets a cut-out determination area including the traffic light with an arrow light on the image captured by the front camera 1. The cut-out determination area is an area for determining whether the arrow light and main light of the traffic light with an arrow light are cut-out. The cut-out determination area is set on the captured image so as to include the traffic light with an arrow light. The cut-out determination area is set on the captured image across multiple cycles.

[0026] Fig. 2 is a diagram for explaining the setting of the cut-off judgment area. Fig. 2 shows an image G captured by the front camera 1, an intersection P, a traffic light T1 with an arrow light in front of the intersection P, a traffic light T2 with an arrow light at the back of the intersection P, a main light M of the traffic light T1 with an arrow light, an arrow light Y of the traffic light T1 with an arrow light, and a cut-off judgment area C. As an example, in the situation shown in Fig. 2, the cut-off judgment area setting unit 13 sets the cut-off judgment area C at the upper left of the captured image G at a position including the traffic light T1 with an arrow light.

[0027] The cut-off judgment area setting unit 13 may set a certain area on the captured image centered on the traffic light with an arrow lamp as the cut-off judgment area. The cut-off judgment area setting unit 13 may set the cut-off judgment area so that each side of the rectangular cut-off judgment area is located at a certain distance from the top, bottom, right and left ends of the casing of the traffic light.

[0028] The cut-off judgment area setting unit 13 may calculate a predicted position of the traffic light with an arrow lamp on the captured image in the next cycle based on the position of the traffic light with an arrow lamp on the captured image in the past cycle, and set the cut-off judgment area so as to include the predicted position of the traffic light with an arrow lamp on the captured image. The position of the traffic light with an arrow lamp on the captured image in the past cycle may include the position of the traffic light with an arrow lamp on the captured image in at least the cycle immediately preceding the one before.

[0029] The cut-off judgment area setting unit 13 calculates a predicted position of the traffic light with an arrow light on the captured image in the next cycle, taking into account a change in the position of the traffic light on the captured image due to the movement of the vehicle. The cut-off judgment area setting unit 13 may calculate the predicted position of the traffic light with an arrow light on the captured image in the next cycle, taking into account the speed of the vehicle. The cut-off judgment area setting unit 13 may calculate the predicted position of the traffic light with an arrow light on the captured image in the next cycle, taking into account the yaw rate or steering angle of the vehicle. The cut-off judgment area setting unit 13 sets the cut-off judgment area to include the predicted position of the traffic light with an arrow light. The cut-off judgment area setting unit 13 may set a certain range centered on the predicted position of the traffic light with an arrow light as the cut-off judgment area, for example.

[0030] The cut-off judgment area setting unit 13 may have a data table that associates the position of the vehicle on the map with the position of the cut-off judgment area on the captured image based on past statistical data, and may use the data table to determine the position of the cut-off judgment area on the captured image from the position of the vehicle on the map.

[0031] After setting the cut-off judgment area, the arrowhead cut-off judgment unit 14 judges whether an arrowhead exists within the cut-off judgment area for each cycle. The arrowhead cut-off judgment unit 14 judges whether an arrowhead exists within the cut-off judgment area by performing image processing within the cut-off judgment area on the captured image.

[0032] After setting the out-of-view judgment area, the main light out-of-view judgment unit 15 judges whether or not the main light is present within the out-of-view judgment area for each cycle. The main light out-of-view judgment unit 15 judges whether or not the main light is present within the out-of-view judgment area by performing image processing within the out-of-view judgment area on the captured image.

[0033] The main light cut-off determination unit 15 may perform cut-off determination of the main light only when it is determined that no arrow light exists within the cut-off determination area by the arrow light cut-off determination unit 14. It is known that cut-off occurs first for arrow lights because arrow lights are more difficult to recognize by a camera than main lights.

[0034] The light state recognition unit 16 recognizes the light state of the traffic light. In the case of a traffic light with an arrow light, the light state recognition unit 16 recognizes the light state including the determination result of the arrow light by the arrow light determination unit 12. The light state recognition unit 16 may track the light state by comparing the light state of the traffic light in the past cycle stored in a light state storage unit 17 described later with the light state of the traffic light in the current cycle.

[0035] When the arrow light out-of-view determination unit 14 determines that no arrow light is present within the out-of-view determination area, the light state recognition unit 16 recognizes the light state of the traffic light with the arrow light assuming that the light state continues for a certain period of time when it is determined that the arrow light is present.

[0036] Specifically, when it is determined that the arrow light does not exist in the out-of-view judgment area in the next cycle after the arrow light out-of-view judgment unit 14 has determined that the arrow light exists in the out-of-view judgment area, the light state recognition unit 16 recognizes that the lighting state of the arrow light in the previous cycle in which it was determined that the arrow light exists continues for a first set time that is set in advance. The first set time may be 1 second, 1.8 seconds, 2.5 seconds, or 3 seconds. The first set time may be counted using the above-mentioned cycle. The counting of the first set time starts, for example, when it is determined that the arrow light does not exist in the out-of-view judgment area.

[0037] Similarly, when the main light out-of-view determination unit 15 determines that there is no main light within the out-of-view determination area, the light state recognition unit 16 recognizes the light state of the traffic light with arrow light as continuing for a certain period of time when it was determined that there was a main light.

[0038] Specifically, when the main light out-of-view determination unit 15 determines that the main light does not exist in the out-of-view determination area in the next cycle after the main light out-of-view determination unit 15 determines that the main light is present in the out-of-view determination area, the light state recognition unit 16 recognizes that the main light is turned on in the previous cycle in which the main light was present, for a second preset time. The second preset time may be 1 second, 1.8 seconds, 2.5 seconds, or 3 seconds. The second preset time may be counted using the above-mentioned cycle. The second preset time is started to be counted when it is determined that the main light does not exist in the out-of-view determination area, for example. The second preset time may be the same as the first preset time, or may be shorter than the first preset time.

[0039] The light state storage unit 17 stores the light state of the traffic light recognized by the light state recognition unit 16. The light state storage unit 17 stores the light state of the traffic light for each cycle. The light state storage unit 17 retains the memory of the past light states of the traffic light for, for example, a predetermined number of cycles. The light state storage unit 17 may store the position of the casing of the traffic light on the captured image.

[0040] The driving assistance unit 18 performs driving assistance for assisting the driver of the vehicle. The driving assistance unit 18 performs various driving assistance depending on the lighting state of the traffic light for the vehicle recognized by the lighting state recognition unit 16. The driving assistance may include a warning to the driver, or an automatic deceleration control of the vehicle.

[0041] The driving support unit 18 issues a warning to inform the driver of the light state of the traffic light, for example, when the light state of the traffic light is a no-passing state or a signal transition state. The driving support unit 18 issues the warning by transmitting a control signal to the HMI 3. The warning may be, for example, an output of a buzzer sound. The warning may be an audio output or an image display on a multi-information display of an instrument panel. The warning may be a projected display on the windshield of the vehicle by the HUD.

[0042] The driving support unit 18 may perform automatic deceleration control of the host vehicle when the lighting state of the traffic light is a passage prohibition state. The driving support unit 18 executes the automatic deceleration control by transmitting a control signal to the brake actuator of the actuator 2. The driving support unit 18 performs the automatic deceleration control so that the host vehicle will be at or below a certain speed before reaching the traffic light.

[0043] [Processing by driving assistance device (light status recognition device)] Next, the processing of the driving support device 100 will be described with reference to the drawings. Fig. 3 is a flowchart showing an example of the cut-off determination area setting processing.

[0044] 3, the driving assistance ECU 10 of the driving assistance device 100 recognizes the housing of a traffic light ahead of the vehicle by the traffic light recognition unit 11 at S10. The traffic light recognition unit 11 recognizes the housing of the traffic light based on an image captured by the front camera 1.

[0045] In S11, the driving assistance ECU 10 uses the traffic light recognition unit 11 to determine whether or not the traffic light ahead of the vehicle is intended for the vehicle. The traffic light recognition unit 11 determines whether or not the traffic light ahead of the vehicle is intended for the vehicle by recognizing the lane in which the vehicle is traveling and the traffic light corresponding to that lane, for example, based on an image captured by the front camera 1. If the driving assistance ECU 10 determines that the traffic light is intended for the vehicle (S11: YES), it proceeds to S12. If the driving assistance ECU 10 does not determine that the traffic light is intended for the vehicle (S11: NO), it ends the current cut-off determination area setting process.

[0046] In S12, the driving assistance ECU 10 recognizes the light state of the traffic light by the light state recognition unit 16. The light state recognition unit 16 recognizes the light state of the traffic light by image processing based on the image captured by the front camera 1. Then, in S13, the driving assistance ECU 10 stores the light state of the traffic light in the light state storage unit 17.

[0047] In S14, the driving assistance ECU 10 judges whether or not the arrow light of the traffic light with arrow lights is on in the captured image by the arrow light judgment unit 12. If the driving assistance ECU 10 judges that the arrow light of the traffic light with arrow lights is on in the captured image (S14: YES), the driving assistance ECU 10 proceeds to S15. If the driving assistance ECU 10 does not judge that the arrow light of the traffic light with arrow lights is on in the captured image (S14: NO), the driving assistance ECU 10 ends this cut-off judgment area setting process. Note that if the traffic light is not a traffic light with arrow lights, it is not judged that the arrow light is on.

[0048] In S15, the driving assistance ECU 10 sets a cut-off determination area including the traffic light with an arrow lamp on the image captured by the front camera 1 by the cut-off determination area setting unit 13. For example, the cut-off determination area setting unit 13 sets an area of ​​a certain range centered on the traffic light with an arrow lamp on the captured image as the cut-off determination area. After that, the driving assistance ECU 10 ends the current cut-off determination area setting process.

[0049] 4A is a flow chart showing an example of the arrowhead cut-off process, which is executed when the cut-off determination area is set in the cut-off determination area setting process of FIG.

[0050] As shown in FIG. 4(a), the driving assistance ECU 10 determines whether an arrow exists in the cut-off judgment area by the arrow cut-off judgment unit 14 in S20. The arrow cut-off judgment unit 14 determines whether an arrow exists in the cut-off judgment area by performing image processing in the cut-off judgment area on the captured image. If it is determined that an arrow does not exist in the cut-off judgment area (S20: YES), the driving assistance ECU 10 proceeds to S21. If it is not determined that an arrow does not exist in the cut-off judgment area (S20: NO), the driving assistance ECU 10 ends the current arrow cut-off processing. After that, the driving assistance ECU 10 repeats the processing from S20 again after a certain time has elapsed.

[0051] In S21, the driving assistance ECU 10 recognizes that the arrow light in the previous cycle has been on for the first set time by the light state recognition unit 16. Even if the light state recognition unit 16 cannot detect the arrow light, it recognizes the light state of the traffic light as the arrow light has been on for the first set time. After that, the driving assistance ECU 10 ends the current arrow light out of sight process.

[0052] 4B is a flowchart showing an example of a main light out-of-view process. The main light out-of-view process is executed when an out-of-view determination area is set in the out-of-view determination area setting process of FIG.

[0053] As shown in FIG. 4(b), the driving assistance ECU 10 determines whether or not a main light is present within the cut-off judgment area by the main light cut-off judgment unit 15 in S30. The main light cut-off judgment unit 15 determines whether or not a main light is present within the cut-off judgment area by performing image processing within the cut-off judgment area on the captured image. If it is determined that a main light is not present within the cut-off judgment area (S30: YES), the driving assistance ECU 10 proceeds to S31. If it is not determined that a main light is not present within the cut-off judgment area (S30: NO), the driving assistance ECU 10 ends this main light cut-off processing. After that, the driving assistance ECU 10 repeats the processing from S30 again after a certain time has elapsed.

[0054] In S31, the driving assistance ECU 10 recognizes that the main light on state in the previous cycle continues for the second set time by the light state recognition unit 16. Even if the light state recognition unit 16 cannot detect the main light, it recognizes the light state of the traffic light as the main light on state continues for the second set time. After that, the driving assistance ECU 10 ends the current main light out processing.

[0055] According to the driving support device (light state recognition device) 100 of the present embodiment described above, when it is determined that the arrow light does not exist in the cut-off judgment area in the next cycle after the arrow light cut-off judgment unit 14 has determined that the arrow light exists in the cut-off judgment area, it recognizes that the arrow light's lighting state in the previous cycle in which it was determined that the arrow light exists continues for a preset first set time, so that it is possible to continue to recognize the lighting state of the arrow light even if the arrow light is cut off. Therefore, the driving support device 100 can avoid performing erroneous driving support due to the cut-off of the arrow light.

[0056] In addition, according to the driving support device 100, the main light outage determination unit 15 determines whether the main light outage determination area is within the outage determination area. main In the next cycle after it is determined that a light is present, main If it is determined that no light is present, main In the previous cycle, when a light was determined to be present, main Since the lighting state of the light continues for a second preset time period, the lighting state of the arrow light can be continuously recognized even if the main light is out of sight.

[0057] Furthermore, according to the driving assistance device 100, the predicted position of the traffic light with an arrow light on the captured image in the next cycle can be calculated based on the position of the traffic light with an arrow light on the captured image in the past cycle, and the cut-off judgment area can be set to a more appropriate position by setting the cut-off judgment area to include the predicted position of the traffic light with an arrow light on the captured image.

[0058] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment. The present invention can be embodied in various forms including the above-mentioned embodiment and various modifications and improvements based on the knowledge of those skilled in the art.

[0059] The light state recognition device does not necessarily have to be a part of the driving assistance device 100. The light state recognition device may be a separate device independent of the driving assistance device. The light state recognition device does not necessarily have to perform main light outage determination. In this case, the ECU does not necessarily have to include the main light outage determination unit 15.

[0060] The light state recognition unit 16 may grasp the timing of the change of the light state of the traffic light with an arrow light based on the past changes of the light state of the traffic light with an arrow light. Alternatively, the light state recognition unit 16 may grasp the timing of the change of the light state of the traffic light with an arrow light as a certain time. The light state recognition unit 16 may determine that the arrow light has been cut off only when it is determined that the arrow light does not exist in the cut-off judgment area at a timing earlier than the timing of the change of the light state after recognizing the lighting state of the arrow light in the traffic light with an arrow light (after the arrow light judgment unit 12 judges that the arrow light is on). In other words, when it is determined that the arrow light does not exist in the cut-off judgment area at the same timing as or later than the timing of the change of the light state of the traffic light with an arrow light, the light state recognition unit 16 may not continue to recognize the lighting state of the arrow light because there is a possibility that the arrow light has been turned off by the change of the light state. [Explanation of symbols]

[0061] 1...forward camera, 10...driving assistance ECU, 11...traffic light recognition unit, 12...arrow light judgment unit, 13...judgment area setting unit, 14...arrow light out-of-view judgment unit, 15...main light out-of-view judgment unit, 16...light state recognition unit, 100...driving assistance device (light state recognition device), C...out-of-view judgment area, G...captured image, M...main light, T1, T2...traffic light with arrow light, Y...arrow light.

Claims

1. A light state recognition device that recognizes the light state of a traffic light with an arrow light ahead of a vehicle, a traffic light recognition unit that recognizes a traffic light in an image captured by a front camera of the host vehicle at regular intervals based on the captured image; an arrow light determination unit that determines whether or not an arrow light of the arrow light-equipped traffic light is turned on in the captured image when the traffic light is recognized by the traffic light recognition unit; a cut-off determination area setting unit that sets a cut-off determination area including the traffic light with the arrow light on the image captured by the front camera when the arrow light determination unit determines that the arrow light is on; a light-out-of-view determination unit that determines whether the light-out-of-view exists within the light-out-of-view determination area for each cycle after setting the light-out-of-view determination area; A lighting state recognition unit that recognizes that the lighting state of the arrow light in the previous cycle in which it was determined that the arrow light exists continues for a first set time when it is determined that the arrow light does not exist in the cut-off judgment area in the next cycle in which the cut-off judgment unit determines that the arrow light exists in the cut-off judgment area; Equipped with The light state recognition unit grasps the timing of the change in the light state of the traffic light with an arrow light based on past changes in the light state of the traffic light with an arrow light, and if it is determined that the arrow light is not present within the out-of-view judgment area at a timing later than the timing of the change in the light state of the arrow light after it is determined that the arrow light is on at the traffic light with an arrow light, the light state recognition device does not recognize that the arrow light will continue to be on even if it is determined that the arrow light is not present within the out-of-view judgment area in the next cycle after it is determined that the arrow light is present within the out-of-view judgment area.

2. A main light out-of-view determination unit is further provided for determining whether or not a main light of the arrow-equipped traffic signal is present within the out-of-view determination area for each cycle after the out-of-view determination area is set, The light state recognition device according to claim 1, wherein when it is determined that the main light is not present within the out-of-view judgment area in the next cycle after the main light out-of-view judgment unit has determined that the main light is present within the out-of-view judgment area, the light state recognition unit recognizes that the main light's on state in the previous cycle in which it was determined that the main light was present continues for a second preset time.

3. 3. The light state recognition device according to claim 1, wherein the out-of-view determination area setting unit calculates a predicted position of the traffic light with an arrow light on the captured image in the next cycle based on the position of the traffic light with an arrow light on the captured image in the past cycle, and sets the out-of-view determination area so as to include the predicted position of the traffic light with an arrow light on the captured image.

Images