Onboard apparatus and program

Abstract

To inhibit an alarm notification from being made through erroneous determination of red-light ignorance.SOLUTION: An onboard apparatus according to an embodiment includes a controller. The controller executes red-light ignorance determination processing on the basis of an image taken by a camera installed in a vehicle. When determining that red-light ignorance has been observed, the controller gives an alarm to a driver. The controller detects a plurality of traffic lights from the image, and calculates distances between the detected traffic lights. When the calculated distances are shorter than a predesignated threshold, the controller excludes the traffic lights from the red-light ignorance determination processing.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The disclosed embodiments relate to an in-vehicle device and a program. [Background technology]

[0002] A technology has been proposed in the past that recognizes traffic lights and their illumination status from an image in front of the vehicle captured by an onboard camera, determines whether a traffic light in the vehicle's lane is red, and issues a warning to the driver if a traffic light is ignored (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-199148 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when using the above-mentioned conventional technology, for example, at an intersection where roads intersect diagonally and multiple traffic lights are located close together, there is a risk that a red light at a traffic light other than the vehicle's own lane may be mistakenly recognized as a red light for the vehicle's own lane, resulting in an erroneous judgment that the vehicle has ignored the red light.

[0005] Furthermore, even if a warning signal is installed before an intersection, there is a risk that when passing through a red warning signal, the system may erroneously determine that the driver has run a red light even though they have not. If such an erroneous determination is made, an erroneous warning will be sent to the driver.

[0006] One aspect of the embodiment has been made in view of the above, and aims to provide an in-vehicle device and a program that can suppress the issuance of an alarm due to an erroneous determination of red light ignoring. [Means for solving the problem]

[0007] According to one aspect of the embodiment, an in-vehicle device includes a controller that executes a red light violation determination process based on an image captured by a camera mounted on the vehicle, and issues a warning to the driver when a red light violation is determined to have occurred. The controller also detects multiple traffic signals from the image, calculates distances between the detected traffic signals, and excludes the multiple traffic signals from the red light violation determination process when the calculated distances are shorter than a predetermined threshold. [Effects of the Invention]

[0008] According to one aspect of the embodiment, the distances between multiple traffic lights detected from an image are calculated, and multiple traffic lights that are too close are excluded from the red light running judgment process. If the distances are too close, false detections between multiple traffic lights are likely to occur, and false warnings are likely to be issued. However, by excluding traffic lights that are close from the red light running judgment process in the first place, it is possible to suppress the issuance of warnings regarding red light running. Therefore, the in-vehicle device according to the embodiment can suppress the issuance of warnings due to false red light running judgments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a first control method for a drive recorder according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a second control method for the drive recorder according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the drive recorder according to the embodiment. [Figure 4] FIG. 4 is a block diagram showing a basic processing flow of the red light ignoring detection processing according to the embodiment. [Figure 5] FIG. 5 is an explanatory diagram of the processing procedure of the first traffic light ignorance determination processing. [Figure 6] FIG. 6 is an explanatory diagram of the first inter-signal distance calculation process and the execution determination process. [Figure 7]FIG. 7 is an explanatory diagram of the second inter-signal distance calculation process and the execution determination process. [Figure 8] FIG. 8 is an explanatory diagram of the implementation determination process according to the modified example. [Figure 9] FIG. 9 is a flowchart showing the procedure of the first red light ignorance determination process. [Figure 10] FIG. 10 is a flowchart showing the processing procedure of the first inter-signal distance calculation processing. [Figure 11] FIG. 11 is a flowchart showing the processing procedure of the second inter-signal distance calculation processing. [Figure 12] FIG. 12 is a flowchart showing the processing procedure of the execution determination process. [Figure 13] FIG. 13 is a flowchart showing the processing procedure of the implementation determination process according to the modified example. [Figure 14] FIG. 14 is an explanatory diagram of the processing procedure of the second red light ignorance determination processing. [Figure 15] FIG. 15 is an explanatory diagram (part 1) of the other signal detection determination process. [Figure 16] FIG. 16 is an explanatory diagram (part 2) of the other signal detection determination process. [Figure 17] FIG. 17 is a flowchart showing the procedure of the second red light ignorance determination process. [Figure 18] FIG. 18 is a flowchart showing the procedure of the other signal detection determination process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an in-vehicle device and a program disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.

[0011] In the following, the in-vehicle device according to the embodiment is assumed to be a drive recorder 10 (see FIG. 1) mounted on a vehicle V. In the following, the control method for the in-vehicle device according to the embodiment is assumed to be a control method for the drive recorder 10 executed by a controller 15 (see FIG. 3) of the drive recorder 10.

[0012] In the following, when it is necessary to distinguish between multiple identical elements, a number in the form "-n" (n is a natural number) may be added after the symbol indicating the element. When there is no particular need to distinguish between them, this numbering will not be used.

[0013] First, an overview of a control method for the drive recorder 10 according to the embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is an explanatory diagram illustrating an overview of a first control method for the drive recorder 10 according to the embodiment. Fig. 2 is an explanatory diagram illustrating an overview of a second control method for the drive recorder 10 according to the embodiment.

[0014] In the first control method, as shown in Figure 1, a scene is assumed in which a road intersects diagonally with the lane in which vehicle V is traveling, and traffic light 300-1 of the vehicle's lane and traffic light 300-2 of the other lane are installed close to each other.

[0015] In this case, in the first control method, the controller 15 of the drive recorder 10 calculates the inter-signal distance, which is the distance between the traffic lights 300-1 and 300-2, based on the image recognition results of the image ahead of the vehicle captured by the camera 12a (see Figure 3) (step S1-1).

[0016] Then, the controller 15 determines whether or not to perform a red light ignoring determination based on the calculated distance between the signals (step S2). Specifically, the controller 15 determines whether or not the calculated distance between the signals is shorter than a threshold value.

[0017] 1, the controller 15 should always treat the traffic light 300-1 as the target for determining whether the driver has run a red light (hereinafter referred to as the "target signal"), but if the traffic light 300-2 is located nearby, the controller 15 may erroneously detect the traffic light 300-2 as the target signal. This erroneous detection is likely to occur, for example, when the traffic light 300-2 is flashing at night.

[0018] Therefore, in the first control method, the controller 15 determines whether or not to perform a red light disregarding determination based on the distance between the signals calculated in step S1-1. If the distance between the signals is shorter than a threshold, the controller 15 determines that the detection accuracy of the traffic light 300 is not being maintained, and therefore does not perform the red light disregarding determination process described below at the intersection shown in FIG. 1. Note that, hereinafter, this case of not performing the process will be referred to as "canceling" where appropriate. This can also be rephrased as excluding the intersection from the red light disregarding determination process.

[0019] On the other hand, when the distance between the signals is longer than the threshold, the controller 15 can assume that the detection accuracy of the traffic light 300 is maintained, and therefore performs the red light running judgment process. In this way, the controller 15 can prevent the erroneous red light running judgment itself by canceling the red light running judgment process in a scene such as that shown in FIG. 1 where an erroneous red light running judgment may occur. Therefore, according to the first control method, it is possible to suppress the issuance of a warning due to an erroneous red light running judgment.

[0020] In the second control method, as shown in FIG. 2, a scene is assumed in which a warning signal 300-3 is installed before an intersection in the lane in which the vehicle V is traveling.

[0021] In this case, in the second control method, the controller 15 of the drive recorder 10 determines whether another signal has been detected based on the image recognition result of the image ahead of the vehicle captured by the camera 12a (step S1-2).

[0022] Specifically, "determining whether other traffic signals are detected" refers to the controller 15 determining whether or not it has detected not only the warning signal 300-3 but also other traffic signals 300. At a typical intersection, there are multiple traffic signals 300, and multiple traffic signals 300 are often detected as a result of image recognition.

[0023] Therefore, the detection of multiple traffic lights 300 is likely to indicate an intersection. On the other hand, when the warning signal 300-3 is red, it is unlikely that other traffic lights 300 are detected in the vicinity when the vehicle V passes the determination line L1, which is the criterion for determining that the vehicle V has run a red light.

[0024] Therefore, in the second control method, the controller 15 determines whether or not another traffic light 300 has been detected in step S1-2, and determines whether to perform a traffic light violation determination based on the result of the determination (step S2). Specifically, if only one traffic light 300 has been detected, the controller 15 cancels the traffic light violation determination process because the red light can be considered to be the warning signal 300-3.

[0025] On the other hand, when multiple traffic lights 300 are detected, the controller 15 can determine that the intersection is an intersection and therefore performs a red light disregarding determination process. In this way, even in a scene such as that shown in FIG. 2 where a red light disregarding error may occur, the controller 15 can cancel the red light disregarding determination process, thereby preventing the red light disregarding error itself. Therefore, according to the second control method, it is possible to suppress the issuance of a warning due to a red light disregarding error.

[0026] In the first and second control methods, the controller 15 performs control to turn on the cancel flag when canceling the traffic light running judgment process. On the other hand, the controller 15 turns off the cancel flag when performing the traffic light running judgment process. Turning the cancel flag on refers to setting the flag value of the cancel flag to, for example, "1." Conversely, turning the cancel flag off refers to setting the flag value of the cancel flag to, for example, "0."

[0027] The first traffic light disregarding determination process using the first control method or the second traffic light disregarding determination process using the second control method will be described in detail later with reference to FIG. 5 and subsequent figures.

[0028] An example of the configuration of the drive recorder 10 to which the control method according to the embodiment described above is applied will be described in more detail below.

[0029] 3 is a diagram showing an example of the configuration of a drive recorder 10 according to an embodiment. As shown in FIG. 3, the drive recorder 10 includes an HMI (Human Machine Interface) unit 11, a sensor unit 12, a communication unit 13, a storage unit 14, and a controller 15.

[0030] The HMI unit 11 is a component that provides interface components related to input and output for each user, including the driver who operates the drive recorder 10. The HMI unit 11 includes an input interface that accepts input operations from the user. The input interface is realized by, for example, a touch panel. The input interface may also be realized by a microphone or the like. The input interface may also be realized by software components.

[0031] The HMI unit 11 also includes an output interface that presents visual and audio information to the user. The output interface is realized by, for example, a display, a speaker, etc. The HMI unit 11 may also provide the user with an input interface and an output interface integrated into one, for example, by a touch panel display.

[0032] In this embodiment, the output interface of the HMI unit 11 notifies the user of an alert when it is determined by the red light running judgment process that a red light has been run.

[0033] The sensor unit 12 is a group of various sensors mounted on the drive recorder 10. The sensor unit 12 includes, for example, a camera 12a, a GPS (Global Positioning System) sensor 12b, and a G sensor 12c.

[0034] The camera 12a is provided so as to be able to capture at least an image ahead of the vehicle V. The camera 12a is attached near the windshield, near the dashboard, or the like.

[0035] The GPS sensor 12b measures the GPS position of the vehicle V. The G sensor 12c measures the G value, which is the acceleration value applied to the vehicle V.

[0036] In addition to the sensor unit 12, the drive recorder 10 is also connected to an on-vehicle sensor 5, which is a group of various sensors mounted on the vehicle V. The on-vehicle sensor 5 includes, for example, a vehicle speed sensor, an accelerator sensor, a brake sensor, etc. The on-vehicle sensor 5 is connected to the drive recorder 10 via an on-vehicle network such as a CAN (Controller Area Network).

[0037] The communication unit 13 is realized by a network adapter or the like. The communication unit 13 is wirelessly connected to a network such as a mobile phone network or a C-V2X (Cellular Vehicle to Everything) communication network, and performs data communication with other devices via the network. The communication unit 13 performs data communication with, for example, a center device in a data center.

[0038] The storage unit 14 is realized by a storage device such as a read-only memory (ROM), a random access memory (RAM), a flash memory, etc. In the example of Fig. 3, the storage unit 14 stores program information 14a and an image recognition model 14b.

[0039] The program information 14a stores various programs including a program according to an embodiment executed by the controller 15. The image recognition model 14b is an AI (Artificial Intelligence) model for image recognition. The image recognition model 14b is, for example, a DNN (Deep Neural Network) model trained using a deep learning algorithm.

[0040] Image recognition model 14b is loaded into controller 15 as a DNN model, and is then configured to be able to detect traffic light 300 appearing in an image captured by camera 12a when the image is input to controller 15. Image recognition model 14b is also configured to be able to detect the signal state of traffic light 300, including the signal color. In other words, controller 15 that has loaded image recognition model 14b operates as an image recognition AI that detects at least traffic light 300 and the signal state of traffic light 300.

[0041] Although not shown, the storage unit 14 stores image data of the image captured by the camera 12a, camera parameters, and the like.

[0042] The controller 15 corresponds to a so-called processor. The controller 15 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or the like. The controller 15 executes a program according to the embodiment stored in the program information 14a of the storage unit 14, using RAM as a work area. The controller 15 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0043] The controller 15 executes various information processes in the red light running detection process according to the embodiment shown in Fig. 4 and subsequent figures. Next, the various information processes in the red light running detection process will be explained one by one.

[0044] 4 is a block diagram showing a basic process flow of the red light running detection process according to the embodiment. Note that before the red light running detection process, the controller 15 has already detected the traffic light 300 as an image recognition AI based on the image captured by the camera 12a.

[0045] The controller 15 extracts the detection result of the traffic light 300 as a bounding box (hereinafter referred to as "Bbox" where appropriate). The Bbox is a rectangular area that includes the traffic light 300. The Bbox is an example of a "detection area." The Bbox includes at least the coordinates of the Bbox in the image, an identifier of the traffic light 300 (hereinafter referred to as "ID" where appropriate), and the probability (certainty) that it is the traffic light 300.

[0046] As shown in Fig. 4, image data, Bbox, camera parameters, and a red light running detection time are input to the red light running detection process. First, in step S11, preprocessing is performed. In the preprocessing, the extracted Bbox is corrected for its position in the image. In the preprocessing, Bboxes with low accuracy are excluded.

[0047] After the pre-processing, frame interpolation processing is performed in step S12. After the frame interpolation processing is performed, trajectory calculation processing of the traffic light 300 is performed in step S13. Furthermore, signal state determination processing is performed in step S14 to determine the signal state of the traffic light 300.

[0048] Then, in step S15, a signal ignore determination process is performed, which outputs a signal ignore flag and a probability of signal ignore.

[0049] If the red light ignoring flag is ON (for example, "1") and the probability of red light ignoring exceeds the threshold, the controller 15 notifies the driver of the vehicle V via the HMI unit 11 of a warning that a red light has been ignored.

[0050] At this time, for example, the controller 15 transmits various data of the vehicle V, including an image of the vehicle V when the red light was ignored, to the center device via the communication unit 13.

[0051] Furthermore, if the red light running flag is ON and the probability of red light running is equal to or less than the threshold, the controller 15 determines that a red light has possibly been run, and transmits various data of the vehicle V at that time to the center device via the communication unit 13. If the red light running flag is ON and the probability of red light running is equal to or less than the threshold, the controller 15 does not need to issue an alert to the driver.

[0052] When the red light ignoring flag is OFF (for example, "0"), the controller 15 does not transmit data to the center device or issue a warning to the driver.

[0053] (First traffic light ignorance judgment process) Next, the details of the signal neglect determination process in step S15 will be explained. First, the details of the first signal neglect determination process (step S15-1) will be explained. Fig. 5 is an explanatory diagram of the processing procedure of the first signal neglect determination process. Note that Fig. 5 shows an example in which inputs are made regarding two traffic lights, the first and second 300 (i.e., inputs regarding two Bboxes).

[0054] In the first signal neglect determination process (step S15-1), first, a target signal determination process is performed to determine whether the first traffic light is a target signal using the trajectory calculation result and signal state determination result of the first traffic light as input (step S151-1). In parallel with this, a target signal determination process is performed using the trajectory calculation result and signal state determination result of the second traffic light as input (step S151-2).

[0055] At the same time, a signal distance calculation process is performed based on the trajectory calculation results of both the first and second traffic lights (step S152). Then, an implementation determination process is performed to determine whether or not to cancel the red light disregard determination process based on the signal distance calculated by the signal distance calculation process (step S153).

[0056] Here, the inter-signal distance calculation process and the execution determination process will be described in more detail. Fig. 6 is an explanatory diagram of the first inter-signal distance calculation process and the execution determination process. Fig. 7 is an explanatory diagram of the second inter-signal distance calculation process and the execution determination process. Fig. 8 is an explanatory diagram of the execution determination process according to a modified example.

[0057] For example, as the first inter-signal distance calculation process, the controller 15 expands each Bbox and calculates the overlapping area of ​​the expanded Bboxes as the inter-signal distance. Specifically, as shown in Fig. 6, the controller 15 expands the Bbox of "ID: 8" and the Bbox of "ID: 4" extracted as the traffic light 300 by a preset expansion rate.

[0058] The controller 15 then calculates the area O1 where the expanded detection areas R1 and R2 overlap as the inter-signal distance. If the calculated area O1 is equal to or greater than a preset area threshold (i.e., the inter-signal distance is shorter than a preset distance threshold), the controller 15 turns on the cancel flag.

[0059] In this way, by expanding each Bbox and calculating the overlapping area of ​​the expanded Bboxes as the inter-signal distance, it becomes possible to reliably determine whether the distance between traffic lights 300 detected as small in the image is close. Furthermore, based on the determination result, traffic lights 300 that are close in distance are excluded from the target of the red light ignition determination process, thereby suppressing the issuance of a warning.

[0060] Although the Bbox is expanded here, it is not necessary to expand it. For example, for overlapping Bboxes that do not require expansion, it is possible to determine whether the distance is close by calculating the area of ​​the overlap as the inter-signal distance.

[0061] Furthermore, as the second inter-signal distance calculation process, for example, the controller 15 calculates the bird's-eye view position of the traffic light 300 when viewed from the midpoint of each Bbox using perspective projection transformation (viewpoint transformation) and the distance from the vehicle V. Then, the controller 15 calculates a predetermined range from the bird's-eye view position of each traffic light 300, and calculates the area where the calculated ranges overlap as the inter-signal distance. Specifically, as shown in FIG. 7, the controller 15 calculates the bird's-eye view positions of "ID: 8" and "ID: 4" extracted as traffic lights 300.

[0062] Then, the controller 15 calculates preset ranges R3 and R4 centered on each bird's-eye view position, and calculates the area O2 where the ranges R3 and R4 overlap as the distance between the signals. If the calculated area O2 is equal to or greater than the preset area threshold (i.e., the distance between the signals is shorter than the preset distance threshold), the controller 15 turns on the cancel flag.

[0063] In this way, by calculating the bird's-eye view position of each traffic light 300 and calculating the area of ​​overlap between preset ranges from the bird's-eye view position as the inter-signal distance, it becomes possible to reliably determine whether the traffic lights 300 are close to each other. Furthermore, based on the determination result, traffic lights 300 that are close to each other are excluded from the signal ignorance determination process, thereby suppressing the issuance of warnings.

[0064] The controller 15 may be configured to execute either one of the first and second inter-signal distance calculation processes, or may be configured to execute both of them.

[0065] Furthermore, in the implementation determination process, if it is determined that the inter-signal distance is shorter than the threshold value after one or both of the first and second inter-signal distance calculation processes, the signal state of each traffic light 300 may be added as a condition. Specifically, as shown in Fig. 8, in the implementation determination process, the controller 15 may compare the signal colors of each traffic light 300, and turn on a cancel flag if the signal colors are different.

[0066] In this way, for example, if the signal colors are the same red and an erroneous judgment is unlikely to occur, the signal ignoring judgment process is carried out, but if the signal colors are different and an erroneous judgment is likely to occur, the signal ignoring judgment process is canceled, thereby suppressing the issuance of an alarm due to an erroneous judgment.

[0067] Returning to the description of Fig. 5, a final determination process is then performed (step S154) based on the determination results of the two target signal determination processes (steps S151-1 and S151-2) and the cancel flag output by the implementation determination process (step S153).

[0068] In the final judgment process, target signals are distinguished from non-target signals, and if it is determined that a target signal has been ignored, a signal ignore flag set to ON and a probability are output, and if it is determined that a target signal has not been ignored, a signal ignore flag set to OFF is output. However, this only applies when the cancel flag is OFF. If the cancel flag is ON, the signal ignore flag is unconditionally set to OFF and output in the final judgment process. This makes it possible to reliably prevent an alert from being issued when the cancel flag is ON, i.e., when the signal ignore is not subject to the signal ignore judgment process.

[0069] The processing procedures for various information processes in the first red light disregarding determination process explained above will be explained using Fig. 9 to Fig. 13. Fig. 9 is a flowchart showing the processing procedures for the first red light disregarding determination process. Note that Fig. 9 shows the processing procedures for one image frame, which are repeated while the image frame is input.

[0070] Fig. 10 is a flowchart showing the processing procedure of a first inter-signal distance calculation process. Fig. 11 is a flowchart showing the processing procedure of a second inter-signal distance calculation process. Fig. 12 is a flowchart showing the processing procedure of an implementation determination process. Fig. 13 is a flowchart showing the processing procedure of an implementation determination process according to a modified example.

[0071] 9, in the first traffic light disregard determination process, the controller 15 acquires the trajectory calculation results and the signal state determination results of each traffic light 300 (step S101). Then, the controller 15 determines whether each traffic light 300 is a target signal (step S102). After executing the inter-signal distance calculation process (step S200), the controller 15 executes the implementation determination process (step S300).

[0072] Then, the controller 15 determines whether the cancel flag output by the execution determination process is ON (step S103). If the cancel flag is ON (step S103, Yes), the controller 15 turns the signal ignore flag OFF (step S104), outputs the signal ignore flag (step S105), and ends the process.

[0073] On the other hand, if the cancel flag is OFF (step S103, No), the controller 15 determines whether or not a traffic light has been ignored (step S106). If a traffic light has been ignored (step S106, Yes), the controller 15 turns the traffic light ignore flag ON (step S107), outputs the traffic light ignore flag and the accuracy (step S108), and ends the process.

[0074] If the traffic signal has not been ignored (step S106, No), the controller 15 turns off the traffic signal ignore flag (step S104), outputs the traffic signal ignore flag (step S105), and ends the process.

[0075] Regarding the signal distance calculation process in step S200, in the first signal distance calculation process, as shown in FIG. 10, the controller 15 expands the Bbox of each traffic light 300 (step S201) and calculates the overlapping area of ​​the expanded Bboxes (step S202).

[0076] Then, the controller 15 outputs the calculated area as the distance between the signals (step S203), and ends the process.

[0077] Similarly, in the inter-signal distance calculation process of step S200, in the second inter-signal distance calculation process, as shown in Fig. 11, the controller 15 calculates the overhead position of each traffic light 300 (step S211). Next, the controller 15 calculates a preset range centered on each overhead position (step S212).

[0078] Then, the controller 15 calculates the area where the calculated ranges overlap (step S213), outputs the calculated area as the distance between the signals (step S214), and ends the process.

[0079] 12, in the execution determination process of step S300, the controller 15 determines whether the inter-signal distance output by the inter-signal distance calculation process of step S200 is shorter than a threshold (step S301). If the inter-signal distance is an area, as described above, a case where the inter-signal distance is shorter than the threshold corresponds to a case where this area is equal to or greater than the area threshold.

[0080] If the distance between the signals is shorter than the threshold (step S301, Yes), the controller 15 turns the cancel flag ON (step S302). On the other hand, if the distance between the signals is longer than the threshold (step S301, No), the controller 15 turns the cancel flag OFF (step S303). Then, the controller 15 outputs the cancel flag (step S304) and ends the process.

[0081] In the execution determination process according to the modified example, as shown in FIG. 13, the controller 15 determines whether the inter-signal distance output by the inter-signal distance calculation process in step S200 is shorter than a threshold value (step S311).

[0082] If the distance between the signals is shorter than the threshold (Yes in step S311), the controller 15 determines whether the signal colors are different (step S312). If the signal colors are different (Yes in step S312), the controller 15 turns on the cancel flag (step S313).

[0083] On the other hand, if the signal colors are the same (step S312, No), or if the distance between the signals is longer than the threshold (step S311, No), the controller 15 turns off the cancel flag (step S314).Then, the controller 15 outputs the cancel flag (step S315), and ends the process.

[0084] (Second traffic light ignorance judgment process) Next, regarding the traffic light disregarding determination process of step S15, the second traffic light disregarding determination process (step S15-2) will be described in detail. Figure 14 is an explanatory diagram of the processing procedure of the second traffic light disregarding determination process. Note that since Figure 14 corresponds to Figure 5 already shown, the explanation using Figure 14 will mainly focus on the differences from Figure 5.

[0085] As shown in Figure 14, the second signal ignorance judgment process (step S15-2) differs from Figure 5 in that an other signal detection judgment process (step S155) is performed instead of the inter-signal distance calculation process (step S152) and implementation judgment process (step S153) shown in Figure 5.

[0086] In the other signal detection determination process, the controller 15 determines whether or not another traffic light 300 is present around the detected traffic light 300. The other signal detection determination process will now be described in more detail. Fig. 15 is an explanatory diagram (part 1) of the other signal detection determination process. Fig. 16 is an explanatory diagram (part 2) of the other signal detection determination process.

[0087] As shown in Figure 15, at a normal intersection, there are multiple traffic lights 300, and the image recognition AI often detects multiple traffic lights 300. Therefore, if multiple traffic lights 300 are detected, there is a high possibility that the intersection is the location.

[0088] On the other hand, as shown in FIG. 16, when the warning signal 300-3 passes the determination line L1 in its trajectory calculated for each image frame, there is a low possibility that other traffic signals 300 will be detected in the vicinity.

[0089] Therefore, in the second signal ignorance judgment process, the controller 15 judges whether or not another traffic light 300 is detected within a predetermined periphery R5 of the traffic light 300 when the traffic light 300 being detected passes through the judgment line L1 in the other signal detection judgment process.

[0090] If there are no other traffic lights 300 in the surrounding area R5, i.e., if only one traffic light 300 is detected, the controller 15 regards the detected traffic light 300 as the warning signal 300-3 and turns the cancel flag ON. On the other hand, if another traffic light 300 is detected in the surrounding area R5, the controller 15 regards the detected traffic light 300 as an intersection and turns the cancel flag OFF to perform the red light disregard determination process.

[0091] Furthermore, when only one traffic light 300 is detected in the image, the detected traffic light 300 may be regarded as the warning signal 300-3 without searching the surroundings R5.

[0092] This prevents the vehicle V from being mistakenly judged to have run a red light when passing through the warning signal 300-3, and from receiving a warning notification to the driver.

[0093] Returning to the explanation of Fig. 5, the final determination process (step S154) is then performed in the same manner as in the first signal ignorance determination process, based on the determination results of the two target signal determination processes (steps S151-1 and S151-2) and the cancel flag output by the other signal detection determination process (step S155).

[0094] In addition, if no other traffic lights 300 are detected in the surrounding area R5 during the other signal detection and judgment process, the trajectory calculation results and signal state judgment results will be one input of either the first traffic light or the second traffic light, rather than two inputs of the first traffic light and the second traffic light.

[0095] The processing procedures for various information processes in the second signal neglect determination process explained above will be explained using Fig. 17 and Fig. 18. Fig. 17 is a flowchart showing the processing procedures for the second signal neglect determination process. Fig. 18 is a flowchart showing the processing procedures for other signal detection determination process.

[0096] 17 corresponds to FIG. 9 already shown, the explanation using FIG. 17 will mainly focus on the differences from FIG.

[0097] As shown in Figure 17, the second signal neglect judgment process differs from Figure 9 in that an other signal detection judgment process (step S400) is performed instead of the inter-signal distance calculation process (step S200) and implementation judgment process (step S300) shown in Figure 9.

[0098] 17 differs from FIG. 9 in that in step S103 shown in FIG. 17, the controller 15 determines whether or not the cancel flag output by the other signal detection determination process is ON.

[0099] In the other signal detection determination process of step S400, as shown in Fig. 18, the controller 15 determines whether or not one traffic light 300 has been detected (step S401). If one traffic light 300 has been detected (step S401, Yes), the controller 15 turns on the cancel flag (step S402).

[0100] On the other hand, if multiple traffic lights 300 are detected (step S401, No), the controller 15 turns off the cancel flag (step S403). Then, the controller 15 outputs the cancel flag (step S404) and ends the process.

[0101] (Conclusion) As described above, the drive recorder 10 according to the embodiment includes the controller 15. The controller 15 executes a red light disregarding determination process based on an image captured by the camera 12a mounted on the vehicle V, and issues a warning to the driver if it is determined that a red light has been disregarded. The controller 15 also detects multiple traffic lights 300 from the image, calculates the distance between the detected multiple traffic lights 300, and excludes the multiple traffic lights 300 from the red light disregarding determination process if the calculated distance is shorter than a preset threshold.

[0102] That is, the drive recorder 10 according to the embodiment calculates the distances between multiple traffic lights 300 detected from an image, and multiple traffic lights 300 that are too close are excluded from the red light running judgment process by setting a cancel flag. When the distances are too close, false detections between multiple traffic lights 300 are likely to occur, and false alarms are likely to be issued. However, by excluding traffic lights 300 that are close from the red light running judgment process in the first place, it is possible to suppress the issuance of warnings regarding red light running. Therefore, the drive recorder 10 according to the embodiment can suppress the issuance of warnings due to false red light running judgments.

[0103] In the above-described embodiment, the in-vehicle device is the drive recorder 10, but the in-vehicle device is not limited to this example. For example, the in-vehicle device may be a car navigation device or the like.

[0104] In the above-described embodiment, the controller 15 executes the control method for the drive recorder 10 according to the embodiment based on the image recognition result of the image captured by the camera 12a. However, the controller 15 may execute the control method for the drive recorder 10 according to the embodiment using sensor data from other sensors in addition to the camera 12a, information previously stored in the storage unit 14, and the like.

[0105] For example, the controller 15 may use location information from the GPS sensor 12b, map information stored in the memory unit 14, etc. When using this location information or map information, the controller 15 calculates, for example, the depth of the intersection that the vehicle V is about to enter based on the location information and map information. Then, if the distance between the signals of the multiple traffic lights 300 is shorter than the depth of the intersection, the controller 15 excludes the multiple traffic lights 300 from the red light disregard determination process.

[0106] This method also makes it possible to exclude traffic lights 300 that are close from the target of the red light ignition determination process, thereby suppressing the issuance of a warning due to an erroneous determination of red light ignition.

[0107] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0108] 5. In-vehicle sensors 10 Drive Recorder 11 HMI section 12 Sensor section 12a Camera 12b GPS sensor 12c G sensor 13 Communications Department 14 Storage section 14a Program Information 14b Image Recognition Model 15 Controller 300 Traffic Light

Claims

1. a controller that executes a signal violation determination process based on an image captured by a camera mounted on the vehicle, and issues an alarm to the driver when it is determined that a signal violation has occurred; The controller Detecting a plurality of traffic lights from the image; Calculating distances between the detected traffic lights; If the calculated distance is shorter than a preset threshold, the plurality of traffic lights are excluded from the target of the traffic light ignorance determination process. In-vehicle device.

2. The controller calculating an overlapping area between detection areas of the plurality of traffic lights on the image as a distance between the plurality of traffic lights; The in-vehicle device according to claim 1 .

3. The controller calculating bird's-eye positions of the plurality of traffic lights in a bird's-eye view state from the midpoints of the detection areas of the plurality of traffic lights by perspective projection transformation; calculating a predetermined range from each of the bird's-eye views; calculating an overlapping area between the calculated ranges as the distance between the plurality of traffic lights; The in-vehicle device according to claim 1 .

4. the controller excludes the plurality of traffic lights from the target of the signal neglect determination process when the overlapping area is equal to or greater than a predetermined threshold. The in-vehicle device according to claim 2 or 3.

5. the controller excludes the plurality of traffic lights from the target of the signal neglect determination process when the overlapping area is equal to or larger than a predetermined threshold and the signal colors of the plurality of traffic lights are different. The in-vehicle device according to claim 2 or 3.

6. The controller When notifying the driver of an alarm, a flag value indicating an ON state is output from the signal ignorance determination process; If the plurality of traffic lights are excluded from the target of the traffic light neglect determination process, the flag value indicating the OFF state is unconditionally output from the traffic light neglect determination process. The in-vehicle device according to claim 1 .

7. The controller Obtain location and map information, calculating a depth of an intersection into which the vehicle is to enter based on the position information and the map information; If the distance between the plurality of traffic lights is shorter than the calculated depth, the plurality of traffic lights are excluded from the target of the signal neglect determination process. The in-vehicle device according to claim 1 .

8. a controller that executes a signal violation determination process based on an image captured by a camera mounted on the vehicle, and issues an alarm to the driver when it is determined that a signal violation has occurred; The controller Detecting a plurality of traffic lights from the image; Expanding detection areas of the plurality of traffic lights on the image by a preset expansion rate; When the expanded detection areas overlap with each other, the plurality of traffic lights are excluded from the target of the signal ignorance determination process. In-vehicle device.

9. a controller that executes a signal violation determination process based on an image captured by a camera mounted on the vehicle, and issues an alarm to the driver when it is determined that a signal violation has occurred; The controller Detecting a traffic light from the image; If the number of detected traffic lights is one, the traffic light is excluded from the target of the traffic light ignorance determination process. In-vehicle device.

10. A computer that executes a signal violation determination process based on an image captured by a camera mounted on a vehicle and issues a warning to a driver when it is determined that a signal violation has occurred. detecting a plurality of traffic lights from the image; Calculating distances between the detected traffic lights; If the calculated distance is shorter than a preset threshold, the plurality of traffic lights are excluded from the target of the traffic light ignorance determination process; A program that executes the following.

Images