Lighting status recognition device

Abstract

To appropriately recognize an arrow lamp lighting state of a traffic signal equipped with an arrow lamp.SOLUTION: A lighting state recognition device (20) comprises: determination means (22) for determining, for each predetermined processing cycle, a lighting state of each of a first traffic signal (S1) as a traffic signal equipped with an arrow lamp that is installed at an intersection ahead of a self-vehicle (1) and is on the closer side when viewed from the self-vehicle, and a second traffic signal (S2) as a traffic signal equipped with an arrow lamp that is installed at the intersection and is on the farther side when viewed from the self-vehicle on the basis of an image capturing the first traffic signal and the second traffic signal; and estimation means (29) for, if at least part of the first traffic signal is not captured in the image in the current processing cycle, estimating a current lighting state on the basis of a lighting state of the first traffic signal that was determined by the determination means in a past processing cycle before the current processing cycle and a lighting state of the second traffic signal that is determined by the determination means in the current processing cycle.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technical field of a lighting state recognition device.

Background Art

[0002] As this type of device, for example, based on the direction and position of the arrow light of a traffic signal existing in an image obtained by imaging the front of a vehicle, a reference area where a red light of the traffic signal should exist is set, and when a red light exists in the reference area, a device that recognizes the direction of the detected arrow light as the current indication of the traffic signal has been proposed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a traffic signal with arrow lights, the arrow lights in the lit state are more difficult to recognize by a camera than the main lights. In addition, due to at least one of the distance between the host vehicle and the traffic signal and the viewing angle of the camera, the entire traffic signal may not be imaged, and there is a possibility that the arrow lights may not be properly recognized.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide a lighting state recognition device capable of appropriately recognizing the state of arrow lights.

Means for Solving the Problems

[0006] A lighting state recognition device according to one aspect of the present invention comprises: determination means for determining the lighting state of the first and second traffic lights based on an image that shows a first traffic light, which is a traffic light with an arrow light located at an intersection in front of the vehicle and is on the near side from the vehicle's perspective, and a second traffic light, which is a traffic light with an arrow light located at the intersection and is on the far side from the vehicle's perspective, at each predetermined processing cycle; and estimation means for estimating the current lighting state based on the lighting state of the first traffic light determined by the determination means in a previous processing cycle prior to the current processing cycle and the lighting state of the second traffic light determined by the determination means in the current processing cycle, if at least a part of the first traffic light is not visible in the image during the current processing cycle. The first and second traffic lights are traffic lights that restrict the movement of the vehicle. That is the case. [Brief explanation of the drawing]

[0007] [Figure 1] Block diagram showing the configuration of a vehicle according to the embodiment. [Figure 2] This is a diagram illustrating the state of traffic light illumination. [Figure 3] This is a diagram illustrating an example of the state of traffic light illumination. [Figure 4] This is an example of an image showing an intersection with a traffic light equipped with an arrow-shaped indicator. [Modes for carrying out the invention]

[0008] Embodiments relating to the lighting state recognition device will be described with reference to Figures 1 to 4. A vehicle 1 equipped with the lighting state recognition device 20 according to this embodiment will be described with reference to Figure 1.

[0009] In Figure 1, Vehicle 1 is equipped with a front camera 11, a storage device 12, an HMI (Human Machine Interface) 13, a driver assistance device 14, an actuator 15, and a lighting status recognition device 20. The front camera 11, storage device 12, HMI (Human Machine Interface) 13, driver assistance device 14, and lighting status recognition device 20 are connected. The driver assistance device 14 and the actuator 15 are connected. The front camera 11 is a camera for capturing images of the area in front of the vehicle 1. The front camera 11 may be installed on the back of the windshield of the vehicle 1. The front camera 11 transmits the captured images of the area in front of the vehicle 1 to the light status recognition device 20. The front camera 11 may be a single monocular camera or a stereo camera.

[0010] The driver assistance device 14 controls the actuator 15. The actuator 15 is a device used to control the vehicle 1. The actuator 15 may include at least one of a drive actuator, a brake actuator, and a steering actuator.

[0011] If vehicle 1 is equipped with an engine, the drive actuator controls the amount of air supplied to the engine (e.g., throttle opening) in response to a control signal from the driver assistance device 14, thereby controlling the driving force of vehicle 1. If vehicle 1 is a hybrid vehicle, the control signal from the driver assistance device 14 is input to the motor as a power source in addition to the drive actuator. As a result, the driving force of the motor is controlled. If vehicle 1 is an electric vehicle, the control signal from the driver assistance device 14 is input to the motor as a power source. In this case, actuator 15 does not necessarily have to include a drive actuator. However, actuator 15 may include a motor as a power source.

[0012] The brake actuator controls the brake system in response to a control signal from the driver assistance device 14, and controls the braking force applied to the wheels of the vehicle 1. The steering actuator controls the drive of the assist motor that controls the steering torque in the electric power steering system in response to a control signal from the driver assistance device 14. As a result, the steering actuator controls the steering torque of the vehicle 1.

[0013] The driver assistance device 14 may cause the vehicle 1 to travel along a target trajectory by controlling the actuators 15 (specifically, at least one of a drive actuator, a brake actuator, and a steering actuator). In other words, as one form of driver assistance, the driver assistance device 14 may automatically drive the vehicle 1.

[0014] HMI13 is an interface for inputting and outputting information between the occupant of vehicle 1 (e.g., the driver) and the systems of vehicle 1 (e.g., the driver assistance device 14 and the lighting status recognition device 20). HMI13 may have a display and speakers, etc. The display of HMI13 may be a HUD (Head Up Display) or a multi-information display provided on the instrument panel.

[0015] The lighting state recognition device 20 includes, as a logically configured logical block or as a physically implemented processing circuit, a housing area detection unit 21, a lighting state determination unit 22, a vehicle target determination unit 23, a tracking unit 24, a visibility obstruction determination area setting unit 25, an arrow light visibility obstruction determination unit 26, a main light visibility obstruction determination unit 27, a lighting state holding unit 28, and a lighting state estimation unit 29.

[0016] The housing region extraction unit 21 recognizes the traffic light in front of the vehicle 1 based on the image captured by the front camera 11. The housing region extraction unit 21 extracts the housing region corresponding to the housing of the traffic light that is captured in the image. The housing region extraction unit 21 may recognize the traffic light by applying image processing (for example, at least one of edge extraction processing, noise reduction processing, and pattern matching processing) to the image. The housing region extraction unit 21 may also extract the housing region of the traffic light using a computational model that extracts the housing region of the traffic light when it receives the image as input. An example of such a computational model is a computational model using a neural network (for example, a Convolutional Neural Network: CNN).

[0017] The lighting state determination unit 22 may recognize the shape of the traffic signal housing based on the housing area extracted by the housing area extraction unit 21. At this time, the lighting state determination unit 22 may recognize that the traffic signal is a traffic signal with arrow lights based on the shape of the traffic signal housing. If the traffic signal is a traffic signal with arrow lights, the lighting state determination unit 22 may determine the lighting state of the main lights and the lighting state of the arrow lights. If the traffic signal is a traffic signal without arrow lights, the lighting state determination unit 22 may determine the lighting state of the main lights.

[0018] The illumination states of the main lights may include a pass-permit state that allows vehicles to pass (e.g., green light), a pass-prohibition state that prohibits vehicles from passing (e.g., red light), and a signal transition state that transitions from a pass-permit state to a pass-prohibition state (e.g., yellow light). Note that the signal transition state does not have to be included in the illumination states of the main lights. The illumination states of the arrow lights may include a first state in which the arrow lights are not illuminated, a second state in which the arrow lights are illuminated and the direction of pass-permitted traffic indicated by the arrow lights is specified, and a third state in which the arrow lights are illuminated and the direction of pass-permitted traffic indicated by the arrow lights is unknown.

[0019] Further, the lighting state determination unit 22 may generate main lighting color information indicating the color of the main lighting as lighting state information indicating the lighting state of the main lighting. The lighting state determination unit 22 may generate arrow lighting direction information indicating the passing permission direction indicated by the arrow lighting as lighting state information indicating the lighting state of the arrow lighting. When the lighting state of the arrow lighting is the first state, the lighting state determination unit 22 may not generate the arrow lighting direction information. When the lighting state of the arrow lighting is the third state, the lighting state determination unit 22 may generate arrow lighting direction information indicating that the direction is unknown.

[0020] The lighting state will be further described with reference to FIGS. 2 and 3. The signal device with arrow lighting shown in FIG. 2 has a blue main lighting G, a yellow main lighting Y, and a red main lighting R as the main lighting. The signal device with arrow lighting also has a left arrow lighting LA, an upward arrow lighting UA, and a right arrow lighting RA as the arrow lighting.

[0021] When the main lighting G is lit, the main lighting Y and R, as well as the arrow lighting LA, UA, and RA, are turned off. In this case, the lighting state determination unit 22 may generate lighting state information indicating a passing permission state. In this case, the lighting state determination unit 22 may generate main lighting color information indicating a green signal as the lighting state information. Further, the lighting state determination unit 22 may not generate the arrow lighting direction information.

[0022] When the main lighting Y is lit, the main lighting G and R, as well as the arrow lighting LA, UA, and RA, are turned off. In this case, the lighting state determination unit 22 may generate lighting state information indicating a signal transition state. In this case, the lighting state determination unit 22 may generate main lighting color information indicating a yellow signal as the lighting state information. Further, the lighting state determination unit 22 may not generate the arrow lighting direction information.

[0023] If the main light R is illuminated, and the main lights G and Y, as well as the arrow lights LA, UA, and RA, are off, the light state determination unit 22 may generate light state information indicating a no-passing state. In this case, the light state determination unit 22 may generate main light color information indicating a red signal as light state information. The light state determination unit 22 does not need to generate arrow light direction information.

[0024] If the main light R and at least one of the arrow lights LA, UA, and RA are lit, and the main lights G and Y are off, the lighting state determination unit 22 may generate lighting state information indicating a no-passing state and a second state. In this case, the lighting state determination unit 22 may generate main light color information indicating a red signal, as well as arrow light direction information indicating the direction in which passage is permitted, as indicated by at least one of the lit arrow lights LA, UA, and RA.

[0025] The longer the distance from vehicle 1 to the traffic light, the smaller the area occupied by the traffic light in the captured image. In other words, the longer the distance from vehicle 1 to the traffic light, the fewer pixels are included in the area occupied by the traffic light in the captured image. For example, when the main light R and arrow light RA of the traffic light are illuminated, and the distance from vehicle 1 to the traffic light is relatively long, the arrow light may become unclear in the captured image, as shown in Figure 4.

[0026] Therefore, the light state determination unit 22 may not be able to determine the permitted direction of passage indicated by the arrow light from the captured image. On the other hand, when the arrow light of the traffic signal is lit, the brightness value of the area corresponding to the lit arrow light (the area corresponding to arrow light RA in Figure 4) is higher than the brightness value of the surrounding area, so the light state determination unit 22 can determine that the arrow light is lit and the position of the lit arrow light (corresponding to the third state described above). In such a case, the light state determination unit 22 may generate main light color information indicating a red signal, as well as arrow light direction information indicating that the permitted direction of passage is unknown.

[0027] Returning to Figure 1, the vehicle target determination unit 23 estimates the position of the traffic signal housing (e.g., height, angle, and lateral position) based on the housing region extracted by the housing region extraction unit 21. Based on the estimated position of the traffic signal housing, the vehicle target determination unit 23 determines whether or not the traffic signal is intended for vehicle 1 (i.e., the vehicle itself) (in other words, whether or not it restricts the movement of vehicle 1).

[0028] The tracking unit 24 acquires the light status information of the traffic signal determined by the light status determination unit 22 (for example, at least one of the main light color information and the arrow light direction information) and information indicating the determination result by the vehicle target determination unit 23. The tracking unit 24 stores in the storage device 12 the light status information related to the traffic signal that the vehicle target determination unit 23 has determined to be a traffic signal targeting vehicle 1 from the light status information acquired from the light status determination unit 22. The information indicating the determination result by the vehicle target determination unit 23 may include housing position information indicating the location of the housing of the traffic signal targeting vehicle 1. In addition to the light status information, the tracking unit 24 may store the housing position information in the storage device 12.

[0029] Furthermore, the tracking unit 24 may discard the lighting status information obtained from the lighting status determination unit 22 that pertains to traffic signals that the vehicle target determination unit 23 has determined not to be traffic signals targeting vehicle 1. Furthermore, the lighting status determination unit 22 may determine the lighting status only for traffic signals that the vehicle target determination unit 23 has determined to be traffic signals targeting vehicle 1.

[0030] The housing area extraction unit 21, the lighting state determination unit 22, the vehicle target determination unit 23, and the tracking unit 24 each repeatedly perform the operations described above at predetermined processing cycles. The processing cycle may be the processing cycle of the lighting state recognition device 20. The processing cycle may also be a cycle corresponding to the transmission timing of signals related to the image captured by the front camera 11.

[0031] The tracking unit 24 further tracks the traffic signals captured in the image. Specifically, the tracking unit 24 determines the identity of the traffic signals by comparing the position of the traffic signal housing, indicated by the housing position information in past processing cycles (e.g., the previous processing cycle) stored in the storage device 12, with the position of the traffic signal housing, indicated by the housing position information in the current processing cycle. The tracking unit 24 tracks the transition of the traffic signal's lighting state by comparing the lighting state information in past processing cycles (e.g., the previous processing cycle) stored in the storage device 12 with the lighting state information in the current processing cycle.

[0032] The tracking unit 24 may transmit the light status information related to the traffic signal for vehicle 1 to the driver assistance device 14. The driver assistance device 14 may provide driver assistance to the driver of vehicle 1 based on the light status information. If the light status indicated by the light status information is a no-passing state or a signal transition state, the driver assistance device 14 may provide a warning to the driver via the HMI 13 to inform them of the light status of the traffic signal. If the light status indicated by the light status information is a no-passing state or a signal transition state, the driver assistance device 14 may perform automatic deceleration control of vehicle 1 by transmitting a control signal to the brake actuator included in the actuator 15.

[0033] The cropping detection area setting unit 25 sets a cropping detection area on the captured image when the light state determination unit 22 recognizes that the traffic light is a traffic light with arrow lights. The cropping detection area is an area for determining whether the arrow lights and main lights of a traffic light with arrow lights are cropped. Here, "cropped" means that at least a part of the traffic light is not captured in the captured image. The cropping detection area is set on the captured image to include the traffic light with arrow lights. The cropping detection area is set on the captured image across multiple processing cycles.

[0034] The obstruction detection area will be explained with reference to Figure 4. Figure 4 is an example of an image (corresponding to the image captured by the front camera 11) showing an intersection with a traffic light equipped with an arrow light. Assume that vehicle 1 is traveling from the bottom to the top of Figure 4. Traffic lights S1 and S2, which target vehicle 1 (in other words, restrict the movement of vehicle 1), are installed at intersection IS. Traffic light S1 is the traffic light closer to vehicle 1. Traffic light S2 is the traffic light further away from vehicle 1.

[0035] The cropping detection area setting unit 25 may set a cropping detection area C at a position including the traffic light S1 on the image shown in Figure 4. The cropping detection area setting unit 25 may also set a certain range of area centered on the traffic light with arrow lights on the captured image as the cropping detection area. The cropping detection area setting unit 25 may also set the cropping detection area such that each side of the rectangular frame-shaped cropping detection area is located at a certain distance from the upper, lower, right, and left edges of the housing region extracted by the housing region extraction unit 21.

[0036] Returning to Figure 1, the arrow light obstruction detection unit 26 determines whether or not an arrow light exists within the obstruction detection area for each processing cycle. The arrow light obstruction detection unit 26 may also determine whether or not an arrow light exists within the obstruction detection area by performing image processing on the region corresponding to the obstruction detection area of ​​the captured image.

[0037] The main light obstruction detection unit 27 determines whether or not a main light is present within the obstruction detection area for each processing cycle. The main light obstruction detection unit 27 may also determine whether or not a main light is present within the obstruction detection area by performing image processing on the region of the captured image that corresponds to the obstruction detection area. Furthermore, the main light obstruction detection unit 27 may only determine whether or not a main light is present within the obstruction detection area if the arrow light obstruction detection unit 26 has determined that no arrow lights are present within the obstruction detection area.

[0038] The light state holding unit 28 temporarily holds light state information from past processing cycles (for example, the previous processing cycle) used by the tracking unit 24 in order to track the transition of the lighting state of the traffic signals. The light state holding unit 28 may temporarily hold only the lighting state information related to the arrow-shaped traffic signal on the near side from the perspective of vehicle 1 (for example, traffic signal S1 in Figure 4) as lighting state information from past processing cycles.

[0039] As shown in Figure 4, when traffic lights S1 and S2 targeting vehicles 1 are installed at a single intersection IS, the lighting state determination unit 22 determines the lighting state of each traffic light S1 and S2. The lighting state information indicating the lighting state of each traffic light S1 and S2 is then stored in the storage device 12.

[0040] Now, from the situation shown in Figure 4, even if traffic light S1 is cut off from view (i.e., at least a part of traffic light S1 is no longer captured in the image), it can be expected that the entire traffic light S2 will still be captured in the image as vehicle 1 moves forward (i.e., approaches intersection IS).

[0041] In the current processing cycle, if the arrow light obstruction determination unit 27 determines that there are no arrow lights within the obstruction determination area, and if the main light obstruction determination unit 28 determines that there are no main lights within the obstruction determination area, at least one of these cases, the light state estimation unit 29 estimates the current light state based on the light state information (hereinafter referred to as "past light state of traffic light S1") indicating the light state of traffic light S1 in a past processing cycle (for example, the previous processing cycle) temporarily held in the light state holding unit 28, and the light state information (hereinafter referred to as "current light state of traffic light S2") indicating the light state of traffic light S2 determined by the light state determination unit 22 in the current processing cycle. Since the lighting states of traffic lights S1 and S2 installed at the intersection IS are synchronized, the above "current light state" refers to the current lighting state of traffic light S1 and the current lighting state of traffic light S2. "Current lighting status" can be rephrased as "current driving restriction status for vehicle 1."

[0042] The operation of the lighting state estimation unit 29 will be explained with a specific example. If the past lighting state of traffic light S1 and the current lighting state of traffic light S2 match, the lighting state estimation unit 29 estimates the lighting state that is the past lighting state of traffic light S1 and the current lighting state of traffic light S2 as the current lighting state.

[0043] If the past lighting state of signal S1 and the current lighting state of signal S2 are different, the lighting state estimation unit 29 may estimate the current lighting state of signal S2 as the current lighting state. For example, if the past lighting state of signal S1 was when the main light R and arrow light RA were lit, and the current lighting state of signal S2 is when only the main light R is lit, the lighting state estimation unit 29 may estimate the state where only the main light R is lit as the current lighting state based on the current lighting state of signal S2.

[0044] However, even if the past lighting state of signal S1 and the current lighting state of signal S2 are different, if the current lighting state of signal S2 indicates that the direction of passage permitted by the arrow light is unknown (the third state described above), the lighting state estimation unit 29 may estimate the direction of passage permitted by the arrow light based on the past lighting state of signal S1, and then estimate the current lighting state of signal S2 based on the current lighting state.

[0045] For example, if the past lighting state of traffic light S1 was that the main light R and arrow light LA ​​were lit, and the current lighting state of traffic light S2 is that the main light R is lit and the area corresponding to the arrow light LA ​​is lit (however, the permitted direction of passage is unknown), then the lighting state estimation unit 29 may estimate the state in which the main light R and arrow light LA ​​are lit as the current lighting state.

[0046] The lighting state estimation unit 29 transmits lighting state information indicating the estimated current lighting state to the tracking unit 24. In this processing cycle, if the arrow light obstruction determination unit 27 determines that there are no arrow lights within the obstruction determination area, and if the main light obstruction determination unit 28 determines that there are no main lights within the obstruction determination area, at least one of these cases is detected, the tracking unit 24 transmits lighting state information indicating the current lighting state estimated by the lighting state estimation unit 29 in this processing cycle to the driving support device 14.

[0047] (Technical effects) A signal light remains lit for a certain period of time (for example, several seconds). Therefore, in the current processing cycle, if the light obstruction determination unit 27 determines that there is no arrow light within the obstruction determination area, and if the main light obstruction determination unit 28 determines that there is no main light within the obstruction determination area, the current lighting state of the signal light may be the same as the lighting state of the obstructed signal light in a previous processing cycle (for example, the last processing cycle). On the other hand, the lighting state of the obstructed signal light in a previous processing cycle may be different from the current lighting state of the signal light. In the latter case, if the driver assistance device 14 provides driver assistance based on the lighting state of the obstructed signal light in a previous processing cycle, unnecessary driver assistance (for example, at least one of warning and automatic deceleration processing) may be performed, or necessary driver assistance (for example, at least one of warning and automatic deceleration processing) may not be performed.

[0048] In the aforementioned light state recognition device 20, the current light state is estimated based on, for example, the light state of the partially obscured traffic light S1 (in other words, traffic light S1 whose part is not captured in the image) in a past processing cycle (for example, the previous processing cycle) and the light state of traffic light S2 in the current processing cycle. The lighting state of traffic light S2 may be unclear in the image, but the lighting state of traffic light S2 indicates the lighting state from the most recent past. Here, the light transition pattern of traffic light S1 and the light transition pattern of traffic light S2 are the same. Therefore, even if the lighting state of traffic light S2 is unclear in the image, the lighting state of traffic light S2 can be estimated by referring to the lighting state of the partially obscured traffic light S1 in a past processing cycle. Therefore, the lighting state recognition device 20 can appropriately estimate the current lighting state (especially the lighting state of the arrow light) from the lighting state of the partially obscured traffic signal S1 in past processing cycles and the lighting state of the traffic signal S2 in the current processing cycle. As a result, the lighting state recognition device 20 can prevent the driver assistance device 14 from performing unnecessary driver assistance (for example, at least one of warning and automatic deceleration processing) or failing to perform necessary driver assistance (for example, at least one of warning and automatic deceleration processing).

[0049] (modified version) When estimating the current lighting state, the lighting state estimation unit 29 may predict the next lighting state of signal S1 indicated by the lighting state information, based on the lighting state information indicating the lighting state of signal S1 in a past processing cycle (for example, the previous processing cycle) temporarily held in the lighting state holding unit 28. If the predicted next lighting state is a yellow light (i.e., a signal transition state), the lighting state estimation unit 29 may predict the lighting state following the yellow light. If the lighting state of signal S2 in the current processing cycle is different from the predicted lighting state, the lighting state estimation unit 29 may discard the lighting state information indicating the lighting state of signal S2 in the current processing cycle. In this case, the lighting state estimation unit 29 may estimate the current lighting state based on the lighting state of signal S1 in a past processing cycle.

[0050] When estimating the current lighting state, the lighting state estimation unit 29 may determine whether or not the lighting state of signal S2 in the current processing cycle is different from the lighting state of signal S1 in a past processing cycle (for example, the previous processing cycle). If it is determined that the lighting state is not a transition from a yellow light, the lighting state estimation unit 29 may discard the lighting state information indicating the lighting state of signal S2 in the current processing cycle. In this case, the lighting state estimation unit 29 may estimate the current lighting state based on the lighting state of signal S1 in a past processing cycle.

[0051] The housing region extraction unit 21 may determine whether or not the traffic signal has a housing for the arrow light when extracting the housing region from the captured image. If it is determined that there is a housing for the arrow light, the housing region extraction unit 21 may estimate the direction in which the arrow light can be illuminated by the traffic signal (i.e., the direction in which passage is permitted). The direction in which the arrow light can be illuminated by the traffic signal, as estimated by the housing region extraction unit 21, may be output to the light state determination unit 22. With this configuration, the light state of the traffic signal can be determined more appropriately.

[0052] Aspects of the invention derived from the embodiments and modifications described above are described below.

[0053] A lighting state recognition device according to one aspect of the invention includes: determination means for determining the lighting state of the first and second traffic lights based on an image that shows the first traffic light, which is a traffic light with an arrow light located at an intersection in front of the vehicle and is on the near side from the vehicle's perspective, and the second traffic light, which is a traffic light with an arrow light located at the intersection and is on the far side from the vehicle's perspective, at each predetermined processing cycle; and estimation means for estimating the current lighting state based on the lighting state of the first traffic light determined by the determination means in a previous processing cycle prior to the current processing cycle and the lighting state of the second traffic light determined by the determination means in the current processing cycle, if at least a part of the first traffic light is not visible in the image during the current processing cycle.

[0054] In the above-described embodiment, the light state determination unit 22 corresponds to an example of determination means, and the light state estimation unit 29 corresponds to an example of estimation means. In Figure 4, traffic light S1 corresponds to an example of a first traffic light, and traffic light S2 corresponds to an example of a second traffic light. The first traffic light and the second traffic light may be traffic lights that restrict the movement of the vehicle.

[0055] If the lighting state of the first signal determined by the determination means in the previous processing cycle is different from the lighting state of the second signal determined by the determination means in the current processing cycle, the estimation means may estimate the current lighting state based on the lighting state of the second signal determined by the determination means in the current processing cycle.

[0056] The lighting state of the first and second traffic signals may include a state of unknown direction where the arrow light is illuminated and the direction of the illuminated arrow light is unknown. If the lighting state of the second traffic signal determined by the determination means in the current processing cycle is the state of unknown direction, the estimation means may estimate the direction indicated by the arrow light based on the illuminated position of the arrow light in the second traffic signal.

[0057] The present invention is not limited to the embodiments described above, and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be read from the claims and specification as a whole. A lighting state recognition device with such modifications is also included within the technical scope of the present invention. [Explanation of symbols]

[0058] 1...Vehicle, 14...Driving support system, 20...Lighting state recognition device, 21...Housing area extraction unit, 22...Lighting state determination unit, 23...Self-vehicle target determination unit, 24...Tracking unit, 25...Obstruction detection area setting unit, 26...Arrow light obstruction detection unit, 27...Main light obstruction detection unit, 28...Lighting state holding unit, 29...Lighting state estimation unit

Claims

1. A determination means determines the lighting status of the first and second traffic lights based on an image that shows a first traffic light with an arrow light located at an intersection in front of the vehicle and closer to the vehicle, and a second traffic light with an arrow light located at the intersection and further away from the vehicle, at each predetermined processing cycle. In the current processing cycle, if at least a portion of the first traffic light is not visible in the image, an estimation means estimates the current lighting state based on the lighting state of the first traffic light determined by the determination means in a previous processing cycle prior to the current processing cycle and the lighting state of the second traffic light determined by the determination means in the current processing cycle. Equipped with, The first and second traffic lights are traffic lights that regulate the movement of the vehicle in question. A lighting status recognition device characterized by the following features.

2. If the lighting state of the first traffic signal determined by the determination means in the previous processing cycle is different from the lighting state of the second traffic signal determined by the determination means in the current processing cycle, the estimation means estimates the current lighting state based on the lighting state of the second traffic signal determined by the determination means in the current processing cycle, characterized in that the lighting state recognition device according to claim 1.

3. The lighting state recognition device according to claim 1, characterized in that the lighting states of the first traffic light and the second traffic light each include a state of unknown direction in which the arrow light is illuminated and the direction of the illuminated arrow light is unknown.

4. If the lighting state of the second traffic light determined by the determination means in the current processing cycle is the direction unknown state, the estimation means estimates the direction indicated by the arrow light based on the lighting position of the arrow light in the second traffic light, according to claim 3, the lighting state recognition device.

Images