Driving Evaluation System

Abstract

To provide a drive evaluation system capable of determining whether or not a driver complies with traffic regulations related to a traffic signal for a vehicle by an easy method.SOLUTION: A drive evaluation system 1 includes: a lighting color recording unit 23 for making lighting color of a first lighting unit 41A reflected in a frame 50C before first time t1 first lighting color 42A and lighting color of a second lighting unit 41B reflected in a frame 50D before second time t2 second lighting color 42B, among a plurality of frames constituting a moving image captured by a front camera 2 to record them; a combination determination unit 12 for determining a combination of the first lighting color 42A and the second lighting color 42B; and an observance determination unit 13 for determining whether or not a car 100 observes traffic regulations defined by the second lighting color 42B of the second light unit 41B based on the combination of the first lighting color 42A and the second lighting color 42B and car speed v recorded in a car speed recording unit 25.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving evaluation system. [Background technology]

[0002] Conventionally, there is an analysis device that determines whether a driver of a vehicle has run a red traffic light by detecting a red light of a traffic signal based on an image captured by an in-vehicle camera (see, for example, Patent Document 1). Also, Patent Documents 2 to 6 propose various systems and devices that accurately detect the lighting colors of traffic indicators and traffic signals from image data in front of the vehicle.

[0003] Conventional driving evaluation systems capture video using an on-board camera and perform image recognition processing on the video to determine whether the driver is complying with traffic rules. In this case, the on-board device detects the color of the traffic light using image recognition processing, and also detects the stop line in front of the traffic light (see, for example, Patent Document 7). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-238779 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-241469 [Patent Document 3] Japanese Patent Application Laid-Open No. 2009-129290 [Patent Document 4] Japanese Patent Application Laid-Open No. 2008-293277 [Patent Document 5] Japanese Patent Application Laid-Open No. 2016-42226 [Patent Document 6] Japanese Patent Application Publication No. 2018-63680 [Patent Document 7] Patent Publication No. 2021-33772 Summary of the Invention [Problem to be solved by the invention]

[0005] As such, driving evaluation systems contribute to improving drivers' awareness of safe driving by determining whether they obey traffic lights when driving a vehicle, so an easier method of determination is desired.

[0006] An object of the present invention is to provide a driving evaluation system that can easily determine whether a driver is complying with traffic rules related to vehicular traffic signals. [Means for solving the problem]

[0007] In order to achieve the above object, a driving evaluation system according to the present invention is mounted on a vehicle and includes an imaging unit that captures a moving image ahead of the vehicle; a signal light detection unit that detects lights of a vehicular traffic signal included in the moving image captured by the imaging unit; a time recording unit that records, as a first time, a time when one of the two lights detected by the signal light detection unit, a first light, becomes undetectable from the moving image, and as a second time, a time when the other of the two lights detected by the signal light detection unit, a second light, becomes undetectable from the moving image; and a time recording unit that records, as a first lighting color, a lighting color of the first light that appears in a frame prior to the first time among a plurality of frames that make up the moving image, and a time recording unit that records, as a second lighting color, a lighting color of the second light that appears in a frame prior to the first time among a plurality of frames that make up the moving image. The vehicle is characterized by comprising: a lighting color recording unit that records the lighting color of the second lighting unit that appears in a frame before the time as a second lighting color; a combination determination unit that determines the combination of the first lighting color and the second lighting color recorded by the lighting color recording unit; a vehicle speed acquisition unit that acquires the vehicle speed; a compliance determination unit that determines whether the vehicle has complied with the traffic rules specified by the second lighting color of the second lighting unit based on the combination of the first lighting color and the second lighting color determined by the combination determination unit and the vehicle speed acquired by the vehicle speed acquisition unit; and a determination result recording unit that records the determination result by the compliance determination unit. [Effects of the Invention]

[0008] The driving evaluation system according to the present invention has the advantage of being able to determine in an easy manner whether or not a driver is complying with traffic rules regarding vehicular traffic signals. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a driving evaluation system according to an embodiment. [Figure 2] FIG. 2 is a flowchart showing an example of evaluation processing by the driving evaluation system according to the embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of evaluation processing by the driving evaluation system according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing an example of evaluation processing by the driving evaluation system according to the embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of evaluation processing by the driving evaluation system according to the embodiment. [Figure 6] FIG. 6 is a schematic diagram showing the positional relationship between a traveling vehicle and a traffic light at time t0 in the embodiment. [Figure 7] FIG. 7 is a schematic diagram showing the positional relationship between a traveling vehicle and a traffic light at time t1 in the embodiment. [Figure 8] FIG. 8 is a schematic diagram showing the positional relationship between a traveling vehicle and a traffic light at time t2 in the embodiment. [Figure 9] FIG. 9 is a diagram showing an example of frames at time t0, after Tr time has elapsed since time t0, at time t1, and at time t2 in the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a combination of lighting colors in the embodiment. [Figure 11] FIG. 11 is a schematic diagram showing an example of a crosswalk with traffic lights on a two-lane road according to an embodiment. [Figure 12] FIG. 12 is a block diagram showing a schematic configuration of a driving evaluation system according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the following embodiments. That is, the components in the following embodiments include those that can be easily imagined by a person skilled in the art or those that are substantially the same, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention.

[0011] [Embodiment] The driving evaluation system 1 according to this embodiment shown in FIG. 1 is mounted on a vehicle 100 such as an automobile, and monitors and evaluates the surrounding environment and driving conditions of the vehicle 100. The driving evaluation system 1 is introduced as equipment for businesses such as freight truck transport businesses and passenger truck transport businesses. The driving evaluation system 1 operates, for example, according to a vehicle operation management program. The driving evaluation system 1 according to this embodiment has the function of recording the surrounding environment and driving conditions of the vehicle 100 and evaluating the driving of the driver (crew) of the vehicle 100.

[0012] Here, the vehicle 100 to which the driving evaluation system 1 is applied has, in principle, a function and structure for transporting people, luggage, etc., and includes, for example, trucks and buses.

[0013] 1, the driving evaluation system 1 includes a front camera 2, a vehicle speed sensor 3, and a main body device 4. The driving evaluation system 1 may also include a rear camera that captures images of the area behind the vehicle 100.

[0014] The front camera 2 is an example of an imaging unit, and captures moving images ahead of the vehicle 100 to generate moving image data. The front camera 2 is installed, for example, in front of the vehicle 100, and captures the scenery ahead of the vehicle as a subject to be captured. This subject to be captured includes one or more vehicular traffic lights arranged along the road. The front camera 2 captures an image of a capture range (for example, a field angle θ) set ahead of the vehicle 100.

[0015] The front camera 2 generates color moving image data so that it can detect at least the lighting units provided on the vehicle traffic lights and the lighting colors of the lighting units. The front camera 2 is electrically connected to the main unit 4 by wiring materials such as electric wires and cables that are laid on the vehicle 100. The front camera 2 outputs the generated moving image data to the main unit 4 via the wiring materials.

[0016] Here, the vehicular traffic lights to be photographed by the forward camera 2 are, for example, vehicular traffic lights 40A, 40B located at an intersection 111 on a two-lane road 110 shown in FIGS. 6 to 8 or at a crosswalk 113 with traffic lights shown in FIG. 11. The vehicular traffic lights 40A, 40B are general traffic lights that display green, yellow, and red lights for vehicles and pedestrians crossing at grade. The vehicular traffic lights 40A, 40B are not so-called time-shifted traffic lights, but rather display two lights at the same time. The vehicular traffic lights 40A, 40B are double-sided traffic lights that light up one side facing the other and have a pair of lighting units. The vehicular traffic lights 40A, 40B are configured to include, in addition to the lighting units, for example, a signal controller, a signal pole, etc., and the lighting color of the lighting units automatically changes based on parameters preset in the signal controller. The vehicular traffic lights 40A, 40B may be of a sensory (perceptual) type.

[0017] The vehicular traffic light 40A is installed, for example, at an intersection 111, at an intersection exit 112A on the first lane 110A side. The first lane 110A is the lane opposite to the second lane 110B in which the vehicle 100 is traveling. The vehicular traffic light 40A is also installed, for example, at a crosswalk 113, at a crosswalk exit 113A on the first lane 110A side. The vehicular traffic light 40A of this embodiment is located in front of the vehicular traffic light 40B in the traveling direction of the vehicle 100 traveling in the second lane 110B. The vehicular traffic light 40A has a first lamp 41A on a surface facing the vehicle 100 traveling in the second lane 110B toward the intersection 111. The first lamp 41A has an identical first lamp 41A provided on the back side.

[0018] The vehicular traffic light 40B is installed, for example, at the intersection 111, at the intersection exit 112B on the second lane 110B side. The vehicular traffic light 40B is installed, for example, at the crosswalk exit 113B on the second lane 110B side at the crosswalk 113. The vehicular traffic light 40B of this embodiment is located on the far side of the vehicular traffic light 40A in the traveling direction of the vehicle 100 traveling on the second lane 110B. The vehicular traffic light 40B has a second lamp 41B on a surface facing the vehicle 100 traveling on the second lane 110B toward the intersection 111. The second lamp 41B has an identical second lamp 41B on its back side. In this way, both the first lamp 41A and the second lamp 41B are visible to the driver of the vehicle 100 approaching the intersection 111 or the crosswalk 113 with a traffic light.

[0019] The first lighting unit 41A and the second lighting unit 41B each have a blue signal light that emits blue light, a yellow signal light that emits yellow light, and a red signal light that emits red light. In this embodiment, the first lighting unit 41A and the second lighting unit 41B are both horizontal, and the signal lights are arranged in the order of blue, yellow, and red from the left as you face them, in accordance with the Enforcement Ordinance of the Road Traffic Act.

[0020] The vehicle speed sensor 3 is an example of a vehicle speed acquisition unit, and is mounted on the vehicle 100 to acquire the vehicle speed v of the vehicle 100. The vehicle speed sensor 3 detects the vehicle speed v, for example, at predetermined time intervals. The vehicle speed sensor 3 is electrically connected to the main device 4 by wiring materials such as electric wires and cables that are wired in the vehicle 100. The vehicle speed sensor 3 outputs data on the acquired vehicle speed v and the time when the vehicle speed v was acquired to the main device 4.

[0021] The main unit 4 is configured to include a control unit 10, a memory 20, and a warning unit 30. The main unit 4 is, for example, a digital tachograph, a drive recorder, a taximeter, or the like that records the driving state of the vehicle 100. The main unit 4 records various information related to the driving and driving of the vehicle 100 while the vehicle 100 is traveling, including video image data acquired by the front camera 2 and the vehicle speed v acquired by the vehicle speed sensor 3. In addition to the control unit 10 and the memory 20, the main unit 4 also has a speed interface, an engine interface, a GPS receiving unit, an analog interface, an external input interface, a wide-area communication unit, a card interface, switches, and a display unit, which are components of a digital tachograph, but these will not be shown or described.

[0022] The main unit 4 determines whether the driver of the vehicle 100 has complied with the traffic rules stipulated by the traffic lights, based on the video image acquired from the front camera 2 and the vehicle speed v of the vehicle 100 acquired by the vehicle speed sensor 3, and records the determination result. Furthermore, the main unit 4 issues a warning to the driver of the vehicle 100 and evaluates the driver's driving based on the determination result.

[0023] The control unit 10 is a central processing unit (CPU) that controls each unit of the driving evaluation system 1. The control unit 10 is configured to include an electronic circuit mainly composed of, for example, a well-known microcomputer. The control unit 10 is electrically connected to the forward camera 2, the vehicle speed sensor 3, the memory 20, the warning unit 30, etc. via various interfaces, and inputs and outputs various electrical signals to and from these units. The control unit 10 executes a predetermined program recorded in the memory 20 and performs various processes necessary to realize the functions of the driving evaluation system 1. For example, when the control unit 10 receives an IGN-on signal output from an ignition switch provided in the vehicle 100 while the vehicle 100 is stopped, the control unit 10 activates and puts the driving evaluation system 1 into an activated state. On the other hand, when the control unit 10 receives an IGN-off signal output from the ignition switch while the vehicle 100 is started, the control unit 10 stops operating and puts the driving evaluation system 1 into a stopped state. Functionally, the control unit 10 conceptually includes a signal light detection unit 11, a combination determination unit 12, a compliance determination unit 13, a driving evaluation unit 14, etc.

[0024] The signal light detection unit 11 detects the first lighting unit 41A and the second lighting unit 41B of the vehicular traffic lights 40A, 40B included in the video captured by the forward camera 2. The signal light detection unit 11 detects the lighting units of the vehicular traffic lights, which are the target objects, from each frame constituting the video using known image recognition. The signal light detection unit 11 records the time when the lighting unit is detected and the time when the detected lighting unit becomes undetectable in the time recording unit 21. When the signal light detection unit 11 detects a lighting unit, it detects the lighting color of the lighting unit and records the lighting color of the lighting unit that appears in the frame before the time when the detected lighting unit becomes undetectable in the lighting color recording unit 23. In other words, the signal light detection unit 11 returns one frame from the time when the detected lighting unit becomes undetectable and records the lighting color of the lighting unit that appears in the frame in question in the lighting color recording unit 23.

[0025] When the signal light detection unit 11 detects one lamp from one frame out of multiple frames that are consecutive in time series, it designates the detected lamp as the first lamp 41A. The signal light detection unit 11 tracks the position of the first lamp 41A in the frame after the first lamp 41A is detected until the first lamp 41A becomes undetectable from the frame after that. In this case, the signal light detection unit 11 records in the time recording unit 21 the time t0 when the first lamp 41A is detected from the frame and the first time t1 when the first lamp 41A becomes undetectable from the subsequent frame. Then, the signal light detection unit 11 records in the lighting color recording unit 23 the lighting color of the first lamp 41A that appears in the frame that is one frame earlier than the time when the detected first lamp 41A became undetectable.

[0026] The signal light detection unit 11 determines whether a light has been detected from a frame within a range of a certain distance x1 to x2 [m] that the vehicle 100 has traveled from the first time t1. Here, x1 to x2 [m] is set to an appropriate value, for example, as the distance between two vehicular traffic lights 40A, 40B at an intersection. The signal light detection unit 11 calculates the elapsed time after the first time t1 based on the distance x1 to x2 [m] and the vehicle speed v of the vehicle 100 at the first time t1, and determines whether a light has been detected from multiple chronologically consecutive frames within that elapsed time. If the signal light detection unit 11 determines that a light has been detected, it designates the detected light as the second light 41B. The signal light detection unit 11 tracks the position of the second light 41B in the frame after detecting the second light 41B until the second light 41B can no longer be detected. In this case, the signal light detection unit 11 detects the second lamp 41B and then records the second time t2 at which the second lamp 41B becomes undetectable in the time recording unit 21. Then, the signal light detection unit 11 goes back one frame from the time at which the detected second lamp 41B became undetectable and records in the lighting color recording unit 23 the lighting color of the second lamp 41B that appears in that frame.

[0027] On the other hand, if the signal light detection unit 11 is unable to detect the light unit, it is highly likely that the vehicle 100 will not pass through an intersection 111 or a crosswalk with traffic lights 113, etc., so the signal light detection unit 11 resets the detection process and detects the light unit of the vehicle traffic light from a new frame.

[0028] When the signal light detection unit 11 detects two lamps from one frame out of a plurality of frames that are consecutive in time series, it designates the lamp that is closest to the vehicle 100 as the first lamp 41A and the other as the second lamp 41B. Here, being closest to the vehicle 100 means, for example, that in frames 50A and 50B shown in Fig. 9, the lamp that is located lower relative to the second lamp 41B and is larger in size. In this case, the signal light detection unit 11 tracks the position of the first lamp 41A on the frame from the frame after detecting the first lamp 41A until the first lamp 41A can no longer be detected, and tracks the position of the second lamp 41B on the frame until the second lamp 41B can no longer be detected.

[0029] The signal light detection unit 11 then records in the time recording unit 21 the time t0 when the first lamp 41A was detected in the frame, the first time t1 when the first lamp 41A could no longer be detected in the subsequent frame, and the second time t2 when the second lamp 41B could no longer be detected in the frame. Furthermore, the signal light detection unit 11 records in the lighting color recording unit 23 the lighting color of the first lamp 41A that appears in the frame that is one frame back from the first time t1 when the detected first lamp 41A could no longer be detected, and the lighting color of the second lamp 41B that appears in the frame that is one frame back from the second time t2 when the detected second lamp 41B could no longer be detected.

[0030] The combination determination unit 12 determines the combination of the first lighting color 42A and the second lighting color 42B recorded in the lighting color recording unit 23. The combination determination unit 12 determines whether there is an erroneous detection of the lighting color and the lighting colors of the vehicular traffic lights 40A, 40B by referring to table information shown in FIG. 10, for example. The combination determination unit 12 determines that the vehicular traffic lights 40A, 40B are green signals when the first lighting color 42A is "green" and the second lighting color 42B is "green," "yellow," or "red." On the other hand, the combination determination unit 12 determines that the vehicular traffic lights 40A, 40B are yellow signals when the first lighting color 42A is "yellow" and the second lighting color 42B is "yellow" or "red." On the other hand, when the first lighting color 42A is "red" and the second lighting color 42B is "green," "yellow," or "red," the combination determination unit 12 determines that the vehicular traffic lights 40A, 40B are red lights. In this embodiment, when the first lighting color 42A is "yellow" and the second lighting color 42B is "green," the combination determination unit 12 determines that there is a possibility of erroneous detection of the lighting colors.

[0031] The compliance determination unit 13 determines whether the vehicle 100 has complied with the traffic rules stipulated by the second lighting color 42B of the second lighting unit 41B, based on the combination of the first lighting color 42A and the second lighting color 42B determined by the combination determination unit 12 and the vehicle speed v acquired by the vehicle speed sensor 3. The compliance determination unit 13 records the determination result in the determination result recording unit 24, which will be described later. For example, the compliance determination unit 13 determines that the driver of the vehicle 100 has not complied with the traffic rules when the combination of the first lighting color 42A is "red" and the second lighting color 42B is "yellow" or "red" and there is no period during which the vehicle speed v of the vehicle 100 is 0 between the first time t1 and the second time t2. In this case, the combination of the first lighting color 42A and the second lighting color 42B is determined to be a red light, and since the vehicle speed v of the vehicle 100 is not 0, the compliance determination unit 13 determines that the driver of the vehicle 100 has ignored a traffic light when passing through an intersection 111 or a traffic-signalized crosswalk 113, etc.

[0032] On the other hand, the compliance determination unit 13 determines that the driver of the vehicle 100 is complying with traffic rules when the first lighting color 42A is a combination of "red" and the second lighting color 42B is "yellow" or "red" and there is a period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0. In this case, the combination of the first lighting color 42A and the second lighting color 42B is determined to be a red light, and the vehicle speed v of the vehicle 100 is 0, so the compliance determination unit 13 determines that the driver of the vehicle 100 is not running a red light when passing through the intersection 111 or the signalized crosswalk 113, etc.

[0033] Furthermore, the compliance determination unit 13 determines that the driver of the vehicle 100 is complying with traffic rules when the first lighting color 42A is a combination of "green" and the second lighting color 42B is "yellow" or "red" and there is no period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0. In this case, since the combination of the first lighting color 42A and the second lighting color 42B is determined to be a green light, the compliance determination unit 13 determines that the driver of the vehicle 100 is not running a red light when passing through the intersection 111, the signalized crosswalk 113, or the like, even if the vehicle speed v of the vehicle 100 does not become 0.

[0034] If the combination of the first lighting color 42A is "yellow" and the second lighting color 42B is "green" and there is no period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 becomes 0, the compliance determination unit 13 determines that the first lighting color 42A and the second lighting color 42B have been erroneously detected. In this case, because the combination of the first lighting color 42A and the second lighting color 42B is an extremely rare case, the compliance determination unit 13 determines that there is a possibility of erroneous detection of the lighting colors even if the vehicle speed v of the vehicle 100 does not become 0.

[0035] If the first lighting color 42A is a combination of "yellow" and the second lighting color 42B is "green" and there is a period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0, the compliance determination unit 13 determines that the first lighting color 42A and the second lighting color 42B are not erroneously detected. In this case, since there is a period during which the vehicle speed v of the vehicle 100 is 0, it is highly likely that the driver has stopped the vehicle 100 before the intersection 111 or the crosswalk 113. Therefore, the compliance determination unit 13 determines whether the time from the time t0 to the first time t1 is shorter than the time Ty. Here, the time Ty is the time required from when the driver of the vehicle 100 recognizes the lighting color of the lamp unit until the vehicle 100 passes through the intersection 111 or the like. If the compliance determination unit 13 determines that the time from time t0 to the first time t1 is less than time Ty, it determines that there was not enough time between the time the driver of the vehicle 100 recognized the lighting color of the lighting unit and the time the vehicle 100 passed through the intersection 111, etc., that is, stopping would result in sudden braking, and therefore it determines that the vehicle 100 can pass through the intersection 111, etc.

[0036] On the other hand, if the compliance determination unit 13 determines that the time from time t0 to the first time t1 is equal to or greater than the time Ty, it determines that there was sufficient time between the time the driver of the vehicle 100 recognized the lighting color of the lamps and the time the vehicle 100 passed through the intersection 111, etc., that is, that there is little possibility that the vehicle 100 will exceed the stopping position even if the driver applies the brakes, and references the vehicle speed v after the time Tr has elapsed from the first time t1 in the vehicle speed recording unit 22, which will be described later. In this case, the compliance determination unit 13 determines whether the driver stopped the vehicle 100 before the intersection 111, etc., based on the vehicle speed v after the time Tr has elapsed from the first time t1. If the vehicle speed v after the time Tr has elapsed from the first time t1 is 0, the compliance determination unit 13 determines that the driver stopped the vehicle 100 before the intersection, and if the vehicle speed v is not 0, it determines that the driver did not stop the vehicle 100 before the intersection.

[0037] The driving evaluation unit 14 evaluates the driving of the driver of the vehicle 100 based on the judgment results recorded in the judgment result recording unit 24, which will be described later. If the compliance judgment unit 13 judges that the driver of the vehicle 100 has not run a red light when passing through an intersection 111, etc., the driving evaluation unit 14 does not lower the driving evaluation. On the other hand, if the compliance judgment unit 13 judges that the driver has run a red light, the driving evaluation unit 14 lowers the driving evaluation, for example.

[0038] The memory 20 is composed of a non-volatile memory, a volatile memory, a recording medium, etc. The non-volatile memory records processing programs executed by the control unit 10 and various data. The volatile memory records, for example, calculation results by the control unit 10. The recording medium records information acquired by the driving evaluation system 1. The recording medium is configured to be detachable from the main unit 4 via, for example, various interfaces. For example, various types of memory cards can be used as the recording medium. The recording medium records data of moving images captured by the forward camera 2, data of the vehicle speed v acquired by the vehicle speed sensor 3, and the time when the vehicle speed v was acquired. Functionally, the memory 20 is configured to include a time recording unit 21, a vehicle speed recording unit 22, a lighting color recording unit 23, a judgment result recording unit 24, etc.

[0039] The time recording unit 21 records the time t0 when the first lamp 41A is detected from the frame by the signal light detection unit 11. The time recording unit 21 also records the time when the first lamp 41A detected by the signal light detection unit 11 can no longer be detected from the moving image as a first time t1, and the time when the other, second lamp 41B can no longer be detected from the moving image as a second time t2.

[0040] The vehicle speed recording unit 22 records data on the vehicle speed v acquired by the vehicle speed sensor 3 and the time at which the vehicle speed v was acquired.

[0041] The lighting color recording unit 23 records, among the multiple frames constituting the moving image, the lighting color of the first lamp 41A that appears in frame 50C that is a first time before t1 as the first lighting color 42A, and the lighting color of the second lamp 41B that appears in frame 50D that is a second time before t2 as the second lighting color 42B. In this case, as shown in FIG. 9 , the first lamp 41A that appears in frame 50C that is a first time before t1 is at position A. Meanwhile, the position of the first lamp 41A at the first time t1 is outside the frame at position B. Meanwhile, the second lamp 41B that appears in frame 50D that is a second time before t2 is at position C. Meanwhile, the position of the second lamp 41B at the second time t2 is outside the frame at position D.

[0042] The judgment result recording unit 24 records the judgment result by the compliance judgment unit 13. The judgment result recording unit 24 records the judgment result that the driver of the vehicle 100 is complying with traffic rules, or the judgment result that the driver of the vehicle 100 is not complying with traffic rules. In addition, the judgment result recording unit 24 records the judgment result that the lighting color is being erroneously detected, or the judgment result that the lighting color is not being erroneously detected.

[0043] The warning unit 30 issues a warning to the driver of the vehicle 100. The warning unit 30 issues a warning to the driver of the vehicle 100 according to the determination result by the compliance determination unit 13. The warning unit 30 may, for example, emit a warning sound or vibrate the driver's seat to alert the driver. Note that the warning method used by the warning unit 30 is not limited to these, and may be, for example, emitting an odor or applying an electrical or physical shock, as long as it alerts the driver.

[0044] Next, the driving evaluation process executed in the driving evaluation system 1 will be described with reference to the flowcharts shown in Figures 2 to 5. The driving evaluation system 1 starts up, for example, when the ACC power supply or the like of the vehicle 100 is turned on, and repeats the process described below until the power supply or the like is turned off.

[0045] In FIG. 2, in step S1, the control unit 10 performs image recognition processing on each of a plurality of frames constituting a moving image, and the signal light detection unit 11 detects the light units of a vehicular traffic signal from each frame.

[0046] In step S2, the control unit 10 determines whether or not one or more lighting units have been detected by the signal light detection unit 11. If the result of this determination is that one or more lighting units have been detected, the process proceeds to step S3. On the other hand, if the signal light detection unit 11 has detected three or more lighting units or if no lighting units have been detected, the process returns to step S1 and repeats the process.

[0047] In step S3, the control unit 10 designates the lamp detected by the signal lamp detection unit 11 as the first lamp 41A.

[0048] In step S4, the control unit 10 records the time t0 at which the first lighting unit 41A is detected by the signal light detection unit 11 in the time recording unit 21, and tracks the position of the first lighting unit 41A on the frame after the first lighting unit 41A is detected until the first lighting unit 41A can no longer be detected.

[0049] In step S5, the control unit 10 determines whether or not a lamp other than the first lamp 41A has been detected by the signal lamp detection unit 11. If the result of this determination is that a lamp other than the first lamp 41A has been detected, the process proceeds to step S6. On the other hand, if a lamp other than the first lamp 41A has not been detected, the process proceeds to step S8.

[0050] In step S6, the control unit 10 designates the lamp detected by the signal lamp detection unit 11 as the second lamp 41B.

[0051] In step S7, the control unit 10 tracks the position of the second lamp 41B on the frame from the frame after the signal lamp detection unit 11 detects the second lamp 41B until the second lamp 41B becomes undetectable.

[0052] In step S8, the control unit 10 determines whether or not the first lamp 41A can no longer be detected by the signal lamp detection unit 11. If the result of this determination is that the first lamp 41A can no longer be detected, the process proceeds to step S9. On the other hand, if the first lamp 41A can be detected, step S8 is repeated.

[0053] In step S9, the control unit 10 records in the time recording unit 21 the time at which the signal lamp detection unit 11 is no longer able to detect the first lamp unit 41A as the first time t1.

[0054] In step S10, the control unit 10 causes the signal lamp detection unit 11 to go back one frame from the first time t1, and records in the lighting color recording unit 23 the lighting color of the first lamp unit 41A from that frame.

[0055] 3, in step S11, the control unit 10 determines whether or not the second lamp 41B has been detected by the signal light detection unit 11. If the result of this determination is that the second lamp 41B has not been detected by the signal light detection unit 11, the process proceeds to step S15. On the other hand, if the second lamp 41B has been detected by the signal light detection unit 11, the process proceeds to step S12.

[0056] In step S12, the control unit 10 determines whether or not the second lamp 41B can no longer be detected by the signal lamp detection unit 11. If the result of this determination is that the second lamp 41B can no longer be detected, the process proceeds to step S13. On the other hand, if the second lamp 41B can be detected, step S12 is repeated.

[0057] In step S13, the control unit 10 records in the time recording unit 21 the time at which the signal lamp detection unit 11 is no longer able to detect the second lamp unit 41B as the second time t2.

[0058] In step S14, the control unit 10 causes the signal lamp detection unit 11 to go back one frame from the second time t2, records the lighting color of the second lamp unit 41B from that frame in the lighting color recording unit 23, and proceeds to step S20 in FIG.

[0059] 3, the control unit 10 determines whether the signal light detection unit 11 has detected a lamp other than the first lamp 41A within a certain distance x1 to x2 [m] from time t1. If the result of this determination is that the signal light detection unit 11 has detected a lamp other than the first lamp 41A within the certain distance x1 to x2 [m] from time t1, the process proceeds to step S16. On the other hand, if the signal light detection unit 11 has not detected a lamp other than the first lamp 41A within the certain distance x1 to x2 [m] from time t1, the process ends.

[0060] In step S16, the control unit 10 designates the lamp detected by the signal lamp detection unit 11 as the second lamp 41B.

[0061] In step S17, the control unit 10 tracks the position of the second lighting unit 41B on the frame after the signal light detection unit 11 detects the second lighting unit 41B until the second lighting unit 41B can no longer be detected, and then returns to step S12.

[0062] In step S20 of FIG. 4, the control unit 10 causes the combination determination unit 12 to determine the combination of light colors described above.

[0063] In step S21, the control unit 10 determines whether or not there is a possibility of erroneous detection by the combination determination unit 12. If the result of this determination indicates that there is a possibility of erroneous detection, the process proceeds to step S22. On the other hand, if there is no possibility of erroneous detection, the process proceeds to step S24.

[0064] In step S22, the control unit 10 determines whether there is a period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0, using the combination determination unit 12. If the result of this determination shows that there is a period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0, the process proceeds to step S27. On the other hand, if there is no period between the first time t1 and the second time t2 during which the vehicle speed v of the vehicle 100 is 0, the process proceeds to step S23.

[0065] In step S23, the control unit 10 determines that the signal lamp detection unit 11 has erroneously detected the lighting color, and ends this process.

[0066] In step S24, the control unit 10 determines whether the lighting color is blue, yellow, or red based on the light color combination determination in step S20. If the lighting color is blue, proceed to step S25. If the lighting color is yellow, proceed to step S27. If the lighting color is red, proceed to step S30 in FIG. 5.

[0067] In step S25, the control unit 10 determines, by the compliance determination unit 13, that the vehicle 100 can pass through the intersection 111 or the like.

[0068] In step S26, the control unit 10 does not lower the evaluation of the driver of the vehicle 100 by the driving evaluation unit 14, and does not issue a warning to the driver by the warning unit 30.

[0069] In step S27, the control unit 10 determines whether the time from time t0 to the first time t1 is shorter than time Ty using the compliance determination unit 13. If the result of this determination is that the time from time t0 to the first time t1 is shorter than time Ty, the process proceeds to step S28. On the other hand, if the time from time t0 to the first time t1 is not shorter than time Ty, the process proceeds to step S29.

[0070] In step S28, the control unit 10 determines that the vehicle 100 can pass through the intersection 111, etc., because, as described above, there was not enough time between the time the driver of the vehicle 100 recognized the lighting color of the lighting unit and the time the vehicle 100 passed through the intersection 111, etc., and proceeds to step S25.

[0071] In step S29, the control unit 10 determines, as described above, that there is sufficient time between when the driver of the vehicle 100 recognizes the lighting color of the lighting unit and when the vehicle 100 passes through the intersection 111, etc., and that there is little possibility that the vehicle 100 will go beyond the stopping position even if the driver applies the brakes, and proceeds to step S30 in Figure 5.

[0072] In step S30, the control unit 10 refers to the vehicle speed v after the time Tr has elapsed from the first time t1 in the vehicle speed recording unit 22, which will be described later.

[0073] In step S31, the control unit 10 determines whether the vehicle speed v after the time Tr has elapsed since the first time t1 is 0 using the compliance determination unit 13. If the result of this determination is that the vehicle speed v after the time Tr has elapsed since the first time t1 is 0, the process proceeds to step S32. On the other hand, if the vehicle speed v after the time Tr has elapsed since the first time t1 is not 0, the process proceeds to step S33.

[0074] In step S32, the control unit 10 determines, by the compliance determination unit 13, that the driver has stopped the vehicle 100 before the intersection, and proceeds to step S26 in FIG.

[0075] In step S33, the control unit 10 determines, by the compliance determination unit 13, that the driver did not stop the vehicle 100 before the intersection.

[0076] In step S34, the control unit 10 causes the driving evaluation unit 14 to lower the evaluation of the driver of the vehicle 100, and also causes the warning unit 30 to issue a warning to the driver, and then ends this process.

[0077] The driving evaluation system 1 according to the present embodiment described above detects the lights of a vehicle traffic signal included in a moving image captured by the forward camera 2 using the signal light detection unit 11. The driving evaluation system 1 records in the time recording unit 21 the point in time at which one of the two detected lights, the first light 41A, becomes undetectable from the moving image as a first time t1, and the point in time at which the other, the second light 41B, becomes undetectable from the moving image as a second time t2. The driving evaluation system 1 records in the lighting color recording unit 23 the lighting color of the first light 41A that appears in frame 50C before the first time t1, of the multiple frames that make up the moving image, as a first lighting color 42A, and the lighting color of the second light 41B that appears in frame 50D before the second time t2 as a second lighting color 42B.

[0078] The driving evaluation system 1 of this embodiment judges the recorded combination of the first lighting color 42A and the second lighting color 42B, and based on the judged combination of the first lighting color 42A and the second lighting color 42B and the vehicle speed v, judges whether or not the traffic rules stipulated by the lighting color corresponding to the combination of the first lighting color 42A and the second lighting color 42B have been complied with, and records the judgment result in the judgment result recording unit 24.

[0079] With the above configuration, the driving evaluation system 1 can, for example, when the vehicle 100 passes through an intersection 111 or the like, easily determine whether the vehicle 100 is complying with traffic rules related to the vehicular traffic signals 40A, 40B installed at the intersection 111 or the like. As a result, the driving evaluation system 1 can reduce the amount of image processing required on the driving evaluation system 1, without using stop lines or the like detected by image recognition from moving images in front of the vehicle, as in conventional driving evaluation systems.

[0080] Furthermore, in the driving evaluation system 1 according to this embodiment, the compliance determination unit 13 determines that the driver of the vehicle 100 is not complying with traffic rules if the first lighting color 42A is red and the second lighting color 42B is yellow or red, and there is no period during which the vehicle speed v is 0 between the first time t1 and the second time t2. Furthermore, the compliance determination unit 13 determines that the driver of the vehicle 100 is complying with traffic rules if the first lighting color 42A is red and the second lighting color 42B is yellow or red, and there is a period during which the vehicle speed v is 0 between the first time t1 and the second time t2. This allows the driving evaluation system 1 to easily determine whether the driver is complying with traffic rules regarding vehicular traffic signals 40A, 40B installed at intersections 111, etc.

[0081] Furthermore, in the driving evaluation system 1 according to this embodiment, the compliance determination unit 13 determines that the driver of the vehicle 100 is complying with traffic rules if the first lighting color 42A is a combination of blue and the second lighting color 42B is yellow or red, and there is no period between the first time t1 and the second time t2 during which the vehicle speed v is 0. In this way, a method of determining red light ignorance based solely on an illuminated image obtained by capturing an image of one traffic light on the road 110 would normally count a passing pattern that does not constitute red light ignorance (such as when the vehicle 100 passes the traffic light just before it changes from green to yellow or red) as red light ignorance. However, the above determination method makes it possible to determine that such a passing pattern also constitutes compliance with traffic rules.

[0082] Furthermore, in the driving evaluation system 1 according to this embodiment, the compliance determination unit 13 determines that the first lighting color 42A and the second lighting color 42B are not erroneously detected if the first lighting color 42A is yellow and the second lighting color 42B is blue, and there is a period between the first time t1 and the second time t2 when the vehicle speed v of the vehicle 100 is 0. Furthermore, the compliance determination unit 13 determines that the first lighting color 42A and the second lighting color 42B are not erroneously detected if the first lighting color 42A is yellow and the second lighting color 42B is blue, and there is a period between the first time t1 and the second time t2 when the vehicle speed v is 0. This enables the driving evaluation system 1 to suppress a decrease in the image recognition accuracy of the lighting colors of the vehicular traffic lights 40A, 40B.

[0083] The driving evaluation system 1 according to this embodiment further includes a warning unit 30 that issues a warning to the driver of the vehicle 100, and a driving evaluation unit 14 that evaluates the driver's driving based on the judgment result. This enables the driving evaluation system 1 to warn drivers who do not obey traffic lights in real time. Furthermore, the driving evaluation system 1 allows, for example, the operation manager of the vehicle 100 to easily evaluate the driver's compliance with traffic rules regarding traffic lights.

[0084] In the driving evaluation system 1 in the above embodiment, the main body device 4 on the vehicle 100 side has the driving evaluation unit 15, but this is not limited to this. Fig. 12 is a block diagram showing a schematic configuration of a driving evaluation system according to a modified example of the present embodiment. As shown in Fig. 12, the driving evaluation system 1A according to the modified example of the present embodiment differs from the above driving evaluation system 1 in that a driving evaluation device 5 provided outside the vehicle 100 has the driving evaluation unit 15.

[0085] The driving evaluation device 5 is, for example, a PC (Personal Computer) that is installed in an office that manages the vehicle 100 and the driver of the vehicle 100, and is capable of exchanging data with the main device 4. The driving evaluation device 5 can exchange data with the main device 4 via a recording medium such as an SD card. Note that the main device 4 and the driving evaluation device 5 may exchange data via direct communication via a USB cable or the like, or wireless communication such as Wi-Fi.

[0086] The driving evaluation device 5 has the above-mentioned driving evaluation unit 14. The driving evaluation device 5 receives data including the judgment result recorded in the judgment result recording unit 24, and the driving evaluation unit 14 that has received the data performs a driving evaluation of the driver who drives the vehicle 100.

[0087] Furthermore, in the above embodiment, the first lighting unit 41A and the second lighting unit 41B are both horizontal, but this is not limited to this and they may be vertical, or one may be horizontal and the other vertical.

[0088] In the above embodiment, the driving evaluation system 1, 1A is, for example, a system in which a traffic management manager provides safe driving guidance to a driver based on the evaluation results recorded in the main device 4 or the like, but is not limited to this. For example, the driving evaluation system 1, 1A may accumulate and analyze data obtained from multiple vehicles 100 to extract routes on which drivers of the vehicles 100 are likely to run red lights. In this case, the traffic management manager can issue warnings to drivers of vehicles traveling on routes on which the incidents are likely to occur. In addition, the traffic management manager may analyze the evaluation results recorded in the main device 4 or the like using analytical evaluation software or the like, and assign points to drivers based on the number of times they run red lights.

[0089] In the above embodiment, the signal light detection unit 11 has been described as detecting two or less lights from one frame among a plurality of frames that are consecutive in time series. On the other hand, if the signal light detection unit 11 detects three or more lights from one frame among a plurality of frames that are consecutive in time series, the signal light detection unit 11 may designate the light closest to the vehicle 100 as the first light 41A and the other as the second light 41B, or may determine that the lighting color has been erroneously detected and reset the processing.

[0090] In the modified example of the above embodiment, the driving evaluation device 5 is configured to include the driving evaluation unit 14, but the present invention is not limited to this. In this case, the driving evaluation device 5 may be configured to include, in addition to the driving evaluation unit 14, a signal light detection unit 11, a combination determination unit 12, a compliance determination unit 13, a time recording unit 21, a vehicle speed recording unit 22, a lighting color recording unit 23, and a determination result recording unit 24. [Explanation of symbols]

[0091] 1. Driving evaluation system 2. Front camera 3 Vehicle speed sensor 4 Main unit 5. Driving evaluation device 10 Control Unit 11 Signal light detector 12 Combination judgment unit 13 Compliance Judgment Department 14 Driving Evaluation Department 20 memory 21 Time Recording Unit 22 Vehicle speed recorder 23 Lighting color recording unit 24 Judgment result recording section 30 Warning section 40A, 40B Vehicle Signals 41A 1st light section 41B 2nd light section 42A 1st lighting color 42B Second lighting color 50A, 50B, 50C, 50D Frame 100 vehicles 110 Road 111 Intersection

Claims

1. an imaging unit mounted on a vehicle and configured to capture a moving image of an area in front of the vehicle; a signal light detection unit that detects a light portion of a vehicular traffic signal included in the moving image captured by the imaging unit; a time recording unit that records, as a first time, a time when one of the two lamps detected by the signal lamp detection unit, a first lamp, becomes undetectable from the moving image, and a time when the other, a second lamp, becomes undetectable from the moving image, as a second time; a lighting color recording unit that records, among a plurality of frames constituting the moving image, the lighting color of the first lamp unit that appears in the frame before the first time as a first lighting color and the lighting color of the second lamp unit that appears in the frame before the second time as a second lighting color; a combination determination unit that determines a combination of the first lighting color and the second lighting color recorded by the lighting color recording unit; a vehicle speed acquisition unit that acquires a vehicle speed of the vehicle; a compliance determination unit that determines whether the vehicle has complied with a traffic rule defined by the second lighting color of the second lamp unit, based on the combination of the first lighting color and the second lighting color determined by the combination determination unit and the vehicle speed acquired by the vehicle speed acquisition unit; and a judgment result recording unit that records the judgment result by the compliance judgment unit, A driving evaluation system characterized by:

2. The compliance determination unit if the first lighting color is red, the second lighting color is a combination of yellow or red, and there is no period during which the vehicle speed of the vehicle is 0 between the first time point and the second time point, it is determined that the driver of the vehicle is not complying with the traffic rules; When the first lighting color is red, the second lighting color is a combination of yellow or red, and there is a period during which the vehicle speed is 0 between the first time point and the second time point, it is determined that the driver of the vehicle is complying with the traffic rules. The driving evaluation system according to claim 1 .

3. The compliance determination unit When the first lighting color is a combination of blue and the second lighting color is yellow or red, and there is no period during which the vehicle speed is 0 between the first time point and the second time point, it is determined that the driver of the vehicle is complying with the traffic rules. The driving evaluation system according to claim 1 .

4. The compliance determination unit if the first lighting color is yellow and the second lighting color is blue, and there is no period during which the vehicle speed is 0 between the first time point and the second time point, it is determined that the first lighting color and the second lighting color are erroneously detected; If the first lighting color is a combination of yellow and the second lighting color is a combination of blue, and there is a period during which the vehicle speed is 0 between the first time point and the second time point, it is determined that the first lighting color and the second lighting color are not erroneously detected. The driving evaluation system according to any one of claims 1 to 3.

5. a warning unit that issues a warning to the driver of the vehicle in accordance with the determination result by the compliance determination unit; A driving evaluation unit that evaluates the driving of the driver of the vehicle based on the judgment result recorded in the judgment result recording unit, The driving evaluation system according to any one of claims 1 to 3.

Images