In-vehicle devices and control programs

Abstract

To provide an in-vehicle device and control program that can reduce the annoyance of notifications. [Solution] The in-vehicle device according to the embodiment is an in-vehicle device that notifies the driver of a vehicle when the vehicle does not start moving after the traffic light's illumination state, as recognized from the image captured by the camera while the vehicle is stopped, changes from red to blue. The controller is configured to notify the driver of a vehicle. If the vehicle remains stopped and does not start moving after the notification has been made, the controller will not re-notify the driver when the traffic light's illumination state changes from red to blue.

Description

Technical Field

[0001] The present invention relates to an in-vehicle device and a control program.

Background Art

[0002] Conventionally, an in-vehicle device that records images of the vehicle surroundings, so-called a drive recorder, is known. Further, by analyzing the captured images, it is possible to detect that the traffic signal has changed from red to blue, and when the vehicle remains stopped despite the traffic signal being blue, a technique for giving a notification prompting the vehicle to start is disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, since the notification is repeated until the vehicle starts, when parked on the road near a traffic signal, the notification is given each time the traffic signal changes to blue. That is, in the prior art, if the notification is repeated even though the driver has no intention of starting due to parking on the road or the like, there is a risk that the driver will find the notification annoying.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an in-vehicle device and a control program capable of reducing the annoyance with respect to the notification.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the in-vehicle device according to the present invention is an in-vehicle device that notifies the driver of a vehicle when the vehicle does not start moving after the state of a traffic light, as recognized from an image captured by a camera while the vehicle is stopped, has changed from red to blue, and includes a controller. If the vehicle remains stopped and does not start moving even after the notification has been made, the controller does not re-notify the driver when the state of the traffic light changes from red to blue. [Effects of the Invention]

[0007] According to the present invention, if the traffic light changes from red to green while the vehicle remains stopped after the initial notification, the re-notification is stopped, thereby preventing repeated notifications from being sent even though the driver has no intention of moving. In other words, the present invention can reduce the annoyance of notifications for the driver. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram showing an overview of the processes performed by the drive recorder according to this embodiment. [Figure 2] Figure 2 shows an example of the configuration of a drive recorder according to the embodiment. [Figure 3] Figure 3 shows an example of the operation in Scene (1). [Figure 4] Figure 4 shows an example of the operation in Scene (2). [Figure 5] Figure 5 shows an example of the operation in Scene (3). [Figure 6] Figure 6 shows an example of the operation in Scene (4). [Figure 7] Figure 7 is a flowchart showing the processing procedure for the notification process of the drive recorder according to the embodiment. [Modes for carrying out the invention]

[0009] The in-vehicle device and control program according to the embodiment will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the embodiments described below.

[0010] Furthermore, in the following description, the in-vehicle device according to the embodiment is assumed to be a drive recorder 1 mounted on a vehicle. Also, in the following description, the control program of the in-vehicle device according to the embodiment is assumed to be a control program executed by the controller 3 (see Figure 2) of the drive recorder 1.

[0011] First, an overview of the processes performed by the drive recorder 1 according to this embodiment will be explained using Figure 1. Figure 1 is a diagram illustrating an overview of the processes performed by the drive recorder 1 according to this embodiment. In Figure 1, the green light violation detection process, which is one of the processes performed by the drive recorder 1, will be explained. A green light violation is a situation in which the vehicle has not started moving even though the traffic light has changed from red to green.

[0012] The drive recorder 1 described in this disclosure will not issue any further notifications after the initial notification prompting the vehicle to start moving when the traffic light changes from red to green. Specifically, after the initial notification prompting the vehicle to start moving, if the traffic light changes from red to green again while the vehicle remains stationary, the drive recorder 1 will stop issuing further notifications prompting the vehicle to start moving (will not issue any further notifications). Alternatively, after the initial notification prompting the vehicle to start moving, the drive recorder 1 will stop issuing further notifications until the vehicle starts moving.

[0013] This means that if the driver intentionally stops after the initial notification, no further notifications will be sent. In other words, Drive Recorder 1 does not repeatedly send notifications when the driver has no intention of moving, such as when parked illegally on the street, thus reducing the annoyance of notifications for the driver.

[0014] The above will be explained in detail using Figure 1.

[0015] Specifically, the drive recorder 1 first acquires forward vehicle detection information, position information, and color information. The forward vehicle detection information is information regarding other vehicles (forward vehicles) present in front of the host vehicle, and includes information on the presence or absence of forward vehicles and the distance to the forward vehicles. The forward vehicle detection information may be, for example, information obtained by analyzing an image captured by the drive recorder 1, or may be information detected by a vehicle detection sensor (such as a distance measurement sensor). The position information is information indicating the positions of traffic signals present around the host vehicle. The color information is obtained by analyzing an image captured by the drive recorder 1. For example, the drive recorder 1 performs template matching on the captured image and extracts a traffic signal area where a traffic signal exists from the image. The color information is information indicating the lighting state of the traffic signal indicated by the position information. The drive recorder 1 specifies the currently lit color (red, blue, yellow) as the color information by performing image analysis on the above traffic signal area.

[0016] Subsequently, the drive recorder 1 performs a green signal violation determination process using the acquired forward vehicle detection information, position information, and color information. Specifically, when the drive recorder 1 satisfies the conditions 1 to 6 shown in FIG. 1, in the flag determination process, it determines that a green signal violation has occurred (sets a green signal violation flag).

[0017] In the present disclosure, conditions 1 to 4 are conditions for determining a green signal violation, and conditions 5 to 6 are conditions for preventing re-notification. In the present disclosure, particularly, by providing conditions 5 to 6, re-notification can be prevented, so that the annoyance of the driver with respect to the notification can be reduced. Hereinafter, the outline of each condition will be described, and the details of each condition will be described later.

[0018] (Condition 1) The process of condition 1 is a process for determining whether the host vehicle is moving or stopped (i.e., parked). In the process of condition 1, the drive recorder 1 determines whether the host vehicle is moving or stopped, sets a "movement flag" when it determines that the vehicle is moving, and sets a "stop flag" when it determines that the vehicle is stopped.

[0019] (Condition 2) The process of Condition 2 is a process for determining that the lighting state of the traffic signal has changed. The drive recorder 1 determines the presence or absence of a change in the lighting state and the color information after the change in the process of Condition 2. When the "stop flag" is set and the traffic signal changes from red to blue, the drive recorder 1 sets the "color change flag".

[0020] (Condition 3) The process of Condition 3 is a process for measuring the elapsed time from the timing when the lighting state of the traffic signal changed in the process of Condition 2 and determining that the elapsed time has reached the threshold time. The drive recorder 1 determines whether or not the elapsed time from the timing when the lighting state of the traffic signal changed has elapsed for the threshold time or more in the process of Condition 3. The drive recorder 1 sets the "elapsed flag" at the timing when the threshold time has elapsed since the "color change flag" was set. Further, when the "move flag" of Condition 1 is set while the "elapsed flag" is set, the drive recorder 1 removes the set "elapsed flag".

[0021] (Condition 4) The process of Condition 4 is a process for determining whether or not there is a preceding vehicle within the threshold distance. The drive recorder 1 determines whether or not there is a preceding vehicle within the threshold distance in the process of Condition 4. When there is a preceding vehicle within the threshold distance, the drive recorder 1 sets the "preceding vehicle present flag".

[0022] (Condition 5) The process for Condition 5 is the process of counting the number of times the traffic light color changes in the process for Condition 2. Drive recorder 1 counts the number of times the traffic light changes from red to blue (number of blue changes) and the number of times it changes from blue to red (number of red changes). The number of red changes is the number of times the light changes from blue to red without going through yellow; in other words, it is the number of times the lighting state changes from blue (actually red) to red (actually blue) when red is misdetected as blue (or blue as red). Drive recorder 1 counts the number of blue changes and the number of red changes while the "stop flag" is set, and resets the respective count values ​​when the "move flag" is set.

[0023] (Condition 6) The process for condition 6 is to count the number of times the system determines that the vehicle ignored a green light while stopped (number of determinations). In the process for condition 6, drive recorder 1 counts the number of times the flag determination process determined that the vehicle ignored a green light (number of green light ignore flag outputs). Drive recorder 1 counts the number of determinations while the "stop flag" is set. Drive recorder 1 resets the count value when the "stop flag" changes to the "move flag" and then the "stop flag" is set again.

[0024] (Flag check) In the flag determination process, the system determines whether the vehicle ignored a green light based on the processing results of conditions 1 to 6. Specifically, drive recorder 1 determines that conditions 1 to 4 are met if the "stop flag," "color change flag," and "elapsed time flag" are set and the "vehicle ahead flag" is not set. Drive recorder 1 also determines that condition 5 is met if the number of green light changes is 1 or less and the number of red light changes is 0. Drive recorder 1 also determines that condition 6 is met if the number of determinations for condition 6 is 0. Finally, if all conditions 1 to 6 are met, drive recorder 1 determines that the vehicle ignored a green light and outputs a green light ignored flag to indicate that a green light was ignored.

[0025] When the drive recorder 1 outputs a green light violation flag, it notifies the driver to start moving. On the other hand, if at least one of conditions 1 to 6 is not met, the drive recorder 1 does not output a green light violation flag. In other words, the drive recorder 1 does not notify the driver to start moving.

[0026] For example, even if drive recorder 1 meets conditions 1 to 4 for determining whether a green light has been ignored, in the processing of condition 5, if the number of times the light changes from red to green is two or more, condition 5 is not met, and therefore the notification to the driver is stopped. As an example of this, if the vehicle is parked on the road near a traffic light, a notification will be given the first time the light changes from red to green, but no notifications will be given from the second time onward.

[0027] Thus, according to this disclosure, if the traffic light changes from red to green while the vehicle remains stopped after the initial notification, the system can stop sending further notifications, thereby preventing repeated notifications from being sent even when the driver has no intention of moving. In other words, this disclosure can reduce the annoyance drivers experience with notifications.

[0028] Furthermore, if the vehicle starts moving after the initial notification, the count values ​​for conditions 5 and 6 are reset, and the next time the vehicle stops, it will determine whether or not to issue the initial notification (whether or not conditions 1-6 are met). In other words, after the initial notification, drive recorder 1 will refrain from issuing further notifications until the vehicle starts moving again.

[0029] Thus, according to this disclosure, by suspending re-notifications after the initial notification until the vehicle starts moving, re-notifications are suspended while the vehicle is stopped, thus avoiding repeated notifications even when the driver has no intention of moving. In other words, according to this disclosure, the annoyance of notifications for the driver can be reduced.

[0030] In addition, the example shown demonstrates that after the initial notification, drive recorder 1 determines whether conditions 1-6 are met again and decides whether to issue another notification. On the other hand, since condition 6 will not be met until the vehicle starts moving after the initial notification, drive recorder 1 may stop the process of determining conditions 1-6 altogether.

[0031] If drive recorder 1 detects a vehicle running a green light, it may associate the image at the time of detection with the fact that a green light violation was detected and store that information. In addition, if drive recorder 1 detects a vehicle running a green light, it may also associate the above information (initial notification or re-notification) with whether or not it issued the notification. Furthermore, if drive recorder 1 detects a vehicle running a green light, it may associate the image at the time of detection with the fact that a green light violation was detected and transmit that information to an external source.

[0032] Next, an example of the configuration of the drive recorder 1 will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the drive recorder 1 according to the embodiment. As shown in Figure 2, the drive recorder 1 has a camera 100, a communication unit 2, a controller 3, and a storage unit 4.

[0033] The camera 100 is configured to capture at least an image of the area in front of the vehicle. The camera 100 is mounted near the windshield, near the dashboard, or the like.

[0034] Furthermore, the drive recorder 1 is connected to various sensors (not shown) mounted on the vehicle. These sensors include, for example, a vehicle speed sensor that detects the vehicle's speed and a GPS (Global Positioning System) sensor that detects the vehicle's position.

[0035] Communication unit 2 is implemented by a network adapter or the like. Communication unit 2 is wirelessly connected to a network such as a mobile phone network or a C-V2X (Cellular Vehicle to Everything) communication network, and performs data communication with other devices via the network. For example, communication unit 2 performs data communication with a data center device that collects captured images.

[0036] The memory unit 4 is implemented using storage devices such as ROM (Read Only Memory), RAM (Random Access Memory), and flash memory. In the example shown in Figure 2, the memory unit 4 stores the image recognition AI 41.

[0037] Image Recognition AI41 is an AI (Artificial Intelligence) model for image recognition. Image Recognition AI41 is a Deep Neural Network (DNN) model trained using, for example, a deep learning algorithm.

[0038] The image recognition AI 41 is loaded into the controller 3 as a DNN model, and then, when an image captured by the camera 100 is input to the controller 3, it is configured to detect traffic lights in the input image. Furthermore, the image recognition AI 41 is configured to detect the lighting state of the traffic lights, including the signal color. In other words, the controller 3, after loading the image recognition AI 41, operates as an image recognition AI that can detect at least the traffic lights and the lighting state of the traffic lights.

[0039] Although not shown in the diagram, the memory unit 4 stores image data of the images captured by the camera 100, as well as camera parameters, etc.

[0040] Controller 3 corresponds to a so-called processor. Controller 3 is implemented by a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphical Processing Unit), etc. Controller 3 executes the control program according to the embodiment stored in the memory unit 4, using RAM as the working area. Controller 3 can also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0041] Controller 3 acquires forward vehicle detection information, position information, and color information, and performs a green light violation determination process based on the acquired information. Specifically, Controller 3 determines whether the above conditions 1 to 6 are met based on the acquired information, and if all conditions are met, it determines that a green light violation has occurred.

[0042] The controller 3 then detects that a green light has been ignored and notifies the driver to start moving. The notification can be sent using any method, such as a screen display, light emission, voice notification, or warning sound.

[0043] Furthermore, if Controller 3 stops a re-notification because it does not meet any of the conditions 1 to 6, it may also stop some of the notification methods described above. For example, if Controller 3 provides an initial notification via screen display and audio notification (e.g., a display or voice message such as "Please check ahead"), it may stop the audio notification for the re-notification and only provide a screen display. In other words, if Controller 3 does not meet any of the conditions 1 to 6, it may limit the notification methods by stopping some of the notification methods rather than stopping all of them. Specifically, if Controller 3 provides an initial notification via display and audio, and the vehicle remains stopped and does not start after the initial notification, it may choose not to provide either a display or audio notification when providing a re-notification. This reduces the driver's annoyance with re-notifications.

[0044] Furthermore, if Controller 3 detects that its own vehicle has started moving while it is in a state where it has stopped re-notifying (a state in which the process of not re-notifying is continuing) because it does not meet any of the conditions 1 to 6, it resets the state in which it has stopped re-notifying. In other words, Controller 3 resets the processing results of conditions 1 to 6 and, when its own vehicle stops again, performs the judgment process for conditions 1 to 6 to determine whether or not to send a notification. This allows Controller 3 to avoid a situation where no notification is sent even after its own vehicle has started moving.

[0045] Next, we will explain the green light disregard detection process in actual scenarios using Figures 3 to 6. Figures 3 to 6 show examples of operation in scenarios (1 to 4).

[0046] First, let's explain the scenes in Figures 3 and 4 (Scenario 1 and 2). Figure 3 shows a scene where the traffic light status is correctly detected (blue is correctly detected as blue), and Figure 4 shows a scene where the traffic light status is incorrectly detected (blue is incorrectly detected as red). In Figures 3 and 4, it is assumed that the vehicle is stopped at the front position (a position where there are no vehicles in front) before the traffic light. In other words, in Figures 3 and 4, the "stop flag" of condition 1 is set, and the "vehicle ahead flag" of condition 4 is not set (a state that satisfies conditions 1 and 4).

[0047] As shown in Figure 3, in a positive detection scenario, suppose at time t1, it is detected that the traffic light's illumination status has changed from red to blue. In this case, at time t1, controller 3 sets the "color change flag" for condition 2. Subsequently, at time t2, after the threshold time TH has elapsed from time t1, controller 3 sets the "elapsed time flag" for condition 3 because the threshold time TH has elapsed since the "color change flag" was set. In other words, at time t2, controller 3 determines that conditions 1 to 4 are satisfied. Also, as shown in Figure 3, since the number of blue changes in condition 5 is 1 and the number of red changes is 0, controller 3 determines that condition 5 is satisfied at time t2. Furthermore, at time t2, controller 3 has not issued any notifications since stopping, meaning the green light disregard judgment flag has not been output even once, so it determines that condition 6 is satisfied. As a result, at time t2, conditions 1 to 6 are satisfied, so controller 3 issues the first notification to the driver prompting them to start moving.

[0048] Next, let's assume that between time t3 and time t4, the traffic light changes from yellow to red, and then from red to blue at time t4. In this case, at time t4, controller 3 sets the "color change flag" for condition 2. Subsequently, at time t5, after the threshold time TH has elapsed from time t4, controller 3 sets the "elapsed flag" for condition 3 because the threshold time TH has elapsed since the "color change flag" was set. In other words, at time t5, controller 3 determines that conditions 1 to 4 are satisfied. Also, as shown in Figure 3, since the number of blue changes for condition 5 is 2 and the number of red changes is 0, controller 3 determines that condition 5 is not satisfied at time t5. Furthermore, at time 5, controller 3 has issued one notification since stopping, meaning the green light disregard judgment flag has not been output once, so it determines that condition 6 is not satisfied. As a result, at time t5, conditions 1-4 are met, but conditions 5 and 6 are not, so controller 3 does not send a notification. In other words, controller 3 does not send a follow-up notification after the initial notification.

[0049] In this way, when the controller 3 correctly detects the state of the traffic light, it does not send a re-notification for the second time or later when the traffic light changes from red to green, thereby reducing the driver's annoyance with notifications.

[0050] Next, using Figure 4, we will explain a scenario of erroneous detection of the traffic light's illumination state. Suppose that at time t1, it is detected that the traffic light's illumination state has changed from blue to red. In this case, at time t1, the controller 3 does not set the "color change flag" of condition 2 because the change is from blue to red. Also, the controller 3 increments the number of red changes in condition 5 to 1. Next, suppose that at time t2, it is detected that the traffic light's illumination state has changed from red to blue. In this case, at time t2, the controller 3 sets the "color change flag" of condition 2. Then, at time t3, after the threshold time TH has elapsed from time t2, the controller 3 sets the "elapsed time flag" of condition 3 because the threshold time TH has elapsed since the "color change flag" was set. In other words, at time t3, the controller 3 determines that conditions 1 to 4 are satisfied. Also, as shown in Figure 4, since the number of blue changes in condition 5 is 1 and the number of red changes is 1, the controller 3 determines that condition 5 is not satisfied at time t3. As a result, at time t3, conditions 1 to 4 are met, but condition 5 is not, so controller 3 does not send a notification. In other words, if the number of times the light has changed to red is 1, controller 3 determines that it has misdetected the state of the traffic light and does not send the initial notification at all. Furthermore, even if the traffic light changes from red to green from the second time onward, the number of times the light has changed to red will be 1 or more, so no notification is sent.

[0051] In this way, controller 3 can avoid issuing incorrect notifications due to false detection by not issuing an initial notification in scenes where the traffic light's illumination status is incorrectly detected.

[0052] Next, let's explain the scene (part 3) in Figure 5. Figure 5 shows a scene where controller 3 has detected multiple traffic lights 500a and 500b. In Figure 5, it is assumed that vehicle C is stopped at the front position (a position where there are no vehicles ahead) before the traffic lights. In other words, in Figure 5, the "stop flag" of condition 1 is set, and the "vehicle ahead flag" of condition 4 is not set (a state that satisfies conditions 1 and 4).

[0053] As shown in Figure 5, when multiple traffic signals 500a and 500b are detected, the controller 3 determines whether conditions 2, 3, and 5 are met for each of the multiple traffic signals 500a and 500b. In this case, the controller 3 determines whether conditions 2, 3, and 5 are met for each of the traffic signals 500a and 500b in turn.

[0054] Now, let's assume that both traffic lights 500a and 500b satisfy conditions 2, 3, and 5. In other words, let's assume that controller 3 has determined that conditions 1 to 5 are satisfied for traffic lights 500a and 500b, respectively.

[0055] In this case, for example, when processing the determination of conditions 1 to 5 for traffic light 500a, controller 3 determines that condition 6 for traffic light 500a is met because the output of the green light ignore determination flag is zero. In other words, controller 3 outputs the green light ignore flag and makes the initial notification because conditions 1 to 6 are met.

[0056] Next, after the judgment process for traffic light 500a, when the judgment process for conditions 1 to 5 is performed for traffic light 500b, the controller 3 determines that condition 6 for traffic light 500a is not met because the green light ignore judgment flag is output only once. In other words, the controller 3 satisfies conditions 1 to 5 but does not satisfy condition 6, so it does not output the green light ignore flag and stops the notification. That is, as in the scene in Figure 5, when multiple traffic lights are detected, the controller 3 sends a notification only for traffic light 500a, which was the first traffic light to be judged, and stops sending notifications for traffic light 500b, which was judged afterward. In other words, when the controller 3 detects multiple traffic lights 500a and 500b by image recognition, it counts the number of green light changes for each of the multiple traffic lights 500a and 500b, and stops sending notifications again when the cumulative value of the number of changes exceeds a threshold (2 times in Figure 5).

[0057] This allows for the avoidance of sending separate notifications to each of the multiple traffic lights 500a and 500b, for example, even if multiple traffic lights 500a and 500b change from red to green simultaneously, by only notifying the first traffic light 500a.

[0058] Furthermore, controller 3 resets the above accumulated value when vehicle C starts moving and then stops. This allows controller 3 to issue the initial notification again when vehicle C starts moving and stops at the next different traffic light.

[0059] Next, let's explain the scene (part 4) in Figure 6. Figure 6 shows a scene where the same traffic light 500 is detected the first time (first image frame), then not detected the second time, and then detected again the third time. In Figure 6, it is assumed that vehicle C is stopped at the front position (a position where there are no vehicles in front) before the traffic light. In other words, in Figure 6, the "stop flag" of condition 1 is set, and the "vehicle ahead flag" of condition 4 is not set (a state that satisfies conditions 1 and 4).

[0060] Even in the case shown in Figure 6, by ensuring that condition 6 is not met during the third detection, the re-detection of traffic light 500 is not mistakenly treated as the first notification.

[0061] Specifically, let's assume that when controller 3 detects a vehicle for the first time, conditions 1 to 5 are met, and the number of times the green light violation flag has been output is 0, thus meeting condition 6 and notifying the driver.

[0062] Subsequently, during the second detection process, assume that traffic light 500 was not detected. Then, during the third detection process, assume that controller 3 detected traffic light 500 again. In this case, controller 3 determines that conditions 1 to 5 are met, and since the green light disregard flag has been output only once, condition 6 is not met, and stops the re-notification.

[0063] This allows the system to avoid treating the initial notification as if traffic light 500 was not detected during an intermediate detection attempt (the second attempt), even if it is detected during the next detection attempt (the third attempt). This reduces the driver's annoyance with notifications.

[0064] Furthermore, when the controller 3 is stopped at traffic light 500 and there is a vehicle ahead, if the traffic light 500 changes from red to green while the vehicle ahead has not started moving, the controller 3 will not send a notification. Instead, it will send a notification after the traffic light 500 changes from red to green again and the vehicle ahead has started moving. This allows the controller 3 to prompt the vehicle to start moving by sending a notification if the vehicle ahead has started moving but the vehicle itself is ignoring the green light.

[0065] Furthermore, the controller 3 may also stop the initial notification if, for example, it detects that vehicle C is parked on the street before the notification is issued. This prevents notifications prompting the driver to start moving when parked on the street, further reducing the driver's annoyance with notifications.

[0066] On-street parking can be detected using various sensors and control states of the vehicle. For example, if controller 3 detects that the vehicle is parked on the road, such as when the vehicle's hazard lights are on, the vehicle's side mirrors are folded in, or the vehicle is straddling the shoulder of the road based on lane detection, it will determine that the vehicle is parked on the road and will stop the initial notification. In addition, if controller 3 detects that the vehicle is parked based on information from the vehicle speed sensor, images captured by camera 100, location information, etc., it will not issue an initial notification.

[0067] Furthermore, as a method for determining whether a vehicle is parked on the street, controller 3 may acquire the vehicle's shift information and determine that the vehicle is parked on the street (stopped) if the shift signal is in parking position. Alternatively, controller 3 may determine that the vehicle is parked on the street (remaining stopped) if it is in parking surveillance mode, where the drive recorder 1 is recording while the vehicle is parked.

[0068] The parking surveillance mode may be activated, for example, by the driver or other occupant operating a button on the drive recorder 1, or it may be activated automatically when the vehicle's ignition is turned off.

[0069] Furthermore, when the vehicle remains stationary without moving (such as when parked on the street), the controller 3 issues a notification regarding a parking violation based on the relative position of the vehicle to the traffic light 500. Specifically, the controller 3 analyzes the image captured by the camera 100 to determine the distance from the vehicle to the traffic light 500. If this distance is below a threshold (for example, less than 5m), the controller 3 issues a notification regarding a parking violation (for example, a display or voice message such as "This is a no-parking zone, please move immediately"), assuming that the vehicle is located in a no-parking zone around the traffic light 500. This allows the controller 3 to prompt the driver to immediately move their vehicle out of the no-parking zone.

[0070] Furthermore, controller 3 may issue a re-notification if the vehicle's stopping time exceeds a predetermined period during the period in which it continues to process not to issue a re-notification. This allows the driver to be prompted to start moving by issuing a re-notification if they have been ignoring a green light for an extended period.

[0071] Next, the processing procedure for the notification process of the drive recorder 1 according to the embodiment will be described using Figure 7. Figure 7 is a flowchart showing the processing procedure for the notification process of the drive recorder 1 according to the embodiment. The process shown in Figure 7 is repeated at regular intervals after the vehicle is started (ignition on).

[0072] First, the controller 3 acquires information about the vehicle ahead and traffic light information (location information and color information) (step S101).

[0073] Next, the controller 3 determines whether condition 1 is met (step S102). Specifically, in processing condition 1, the controller 3 determines whether its own vehicle is moving or stationary.

[0074] If condition 1 is met (step S102: Yes), i.e., the vehicle is stopped, controller 3 determines whether condition 2 is met (step S103). Specifically, in step S103, controller 3 determines whether the traffic light has changed from red to blue. On the other hand, if condition 1 is not met (step S102: No), i.e., the vehicle is moving, controller 3 terminates the process. In other words, controller 3 stops the notification process without sending a notification.

[0075] In step S103, if condition 2 is met (step S103: Yes), that is, if the traffic light's illumination state changes from red to blue, the controller 3 determines whether condition 3 is met (step S104). Specifically, the controller 3 determines whether a threshold time TH or more has elapsed since the traffic light's illumination state changed from red to blue. On the other hand, if condition 2 is not met (step S103: No), that is, if the traffic light's illumination state has not changed from red to blue, the controller 3 terminates the process.

[0076] In step S104, if condition 3 is met (step S104: Yes), that is, if a threshold time TH or more has elapsed since the traffic light changed from red to blue, the controller 3 determines whether condition 4 is met (step S105). Specifically, the controller 3 determines whether or not there is a vehicle ahead within a threshold distance. On the other hand, if condition 3 is not met (step S104: No), that is, if a threshold time TH or more has not elapsed since the traffic light changed from red to blue, the controller 3 terminates the process.

[0077] In step S105, if condition 4 is met (step S105: Yes), that is, if there is no vehicle ahead within the threshold distance, the controller 3 determines whether condition 5 is met (step S106). Specifically, the controller 3 determines whether the number of times the traffic light has changed to green is 1 or less, and the number of times it has changed to red is 0. On the other hand, if condition 4 is not met (step S105: No), that is, if there is a vehicle ahead within the threshold distance, the controller 3 terminates the process.

[0078] In step S106, if condition 5 is met (step S106: Yes), that is, if the number of times the traffic light has turned green is 1 or less and the number of times it has turned red is 0, the controller 3 determines whether condition 6 is met (step S107). Specifically, the controller 3 determines whether the number of times the green light disregard judgment flag has been output is 0. On the other hand, if condition 5 is not met (step S106: No), that is, if the number of times the traffic light has turned green is 2 or more, or if the number of times it has turned red is 1 or more, the controller 3 terminates the process.

[0079] In step S107, if condition 6 is met (step S107: Yes), that is, if the number of times the green light violation judgment flag is output is 0, the controller 3 notifies the driver to start moving (step S108) and terminates the process.

[0080] On the other hand, if condition 6 is not met (step S107: No), that is, if the number of times the green light disregard judgment flag is output is one or more, the controller 3 terminates the process.

[0081] As described above, the in-vehicle device (drive recorder 1) according to the embodiment is an in-vehicle device that notifies the driver of a vehicle when the vehicle does not start moving after the traffic light's illumination status, as recognized from the image captured by the camera while the vehicle is stopped, changes from red to blue, and includes a controller 3. If the vehicle remains stopped and does not start moving after the controller 3 has made a notification, the controller 3 will not re-notify the driver when the traffic light's illumination status changes from red to blue.

[0082] According to this disclosure, if the traffic light changes from red to green while the vehicle remains stopped after the initial notification, the system can stop sending further notifications, thus preventing repeated notifications from being sent even when the driver has no intention of moving. In other words, this disclosure can reduce the annoyance drivers experience from notifications.

[0083] Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents. [Explanation of Symbols]

[0084] 1. Dashcam 2 Communications Department 3 Controllers 4 Storage section 41 Image Recognition AI 100 Cameras 500 traffic lights

Claims

1. An in-vehicle device that notifies the driver of a vehicle if, while the vehicle is stopped, the traffic light status, as recognized from the image captured by the camera, changes from red to blue, but the vehicle does not start moving, and the device has a controller. The aforementioned controller, If the vehicle remains stopped and does not move after the aforementioned notification has been given, the notification will not be resent when the traffic light changes from red to green. In-vehicle device.

2. The aforementioned controller, When it is detected that the vehicle has started moving, the process of not sending the aforementioned re-notification is reset. The in-vehicle device according to claim 1.

3. The aforementioned controller, If the vehicle's parking time exceeds the specified time, the aforementioned re-notification will be issued. The in-vehicle device according to claim 1.

4. The aforementioned controller, The system determines that the vehicle is stopped if the shift signal of the vehicle is in the parking position. The in-vehicle device according to claim 1.

5. The aforementioned controller, If the system is in parking surveillance mode, which records while the vehicle is parked, It is determined that the vehicle remains stationary. The in-vehicle device according to claim 1.

6. The aforementioned controller, If the vehicle remains stationary without moving, a notice regarding a parking violation will be issued based on the relative positions of the traffic light and the vehicle. The in-vehicle device according to claim 1.

7. The aforementioned controller, If the traffic light changes from red to green while the vehicle ahead has not started moving, the aforementioned notification will not be given. After the traffic light changes from red to green again, and the vehicle ahead starts moving, the notification will be given. The in-vehicle device according to claim 1.

8. The aforementioned controller, The aforementioned notification is given both visually and audibly. If the vehicle remains stationary and does not start moving after the notification has been given, the subsequent notification will not be given visually or audibly. The in-vehicle device according to claim 1.

9. The aforementioned controller, During the period in which the vehicle remains stopped, the number of times the signal light changes from red to green is counted, and if the number of changes exceeds a threshold, the re-notification will not be issued. The in-vehicle device according to claim 1.

10. The aforementioned controller, If multiple traffic lights are detected by image recognition, the number of changes for each of the multiple traffic lights is counted, and if the cumulative value of the number of changes exceeds a threshold, the re-notification is not performed. The in-vehicle device according to claim 9.

11. The aforementioned controller, If the vehicle stops after starting, the accumulated value is reset. The in-vehicle device according to claim 10.

12. The aforementioned controller, If the traffic light changes from green to red while the vehicle remains stopped, the aforementioned re-notification will not be issued. The in-vehicle device according to claim 1.

13. The aforementioned controller, If, before issuing the aforementioned notification, the vehicle is detected to be parked based on information from the vehicle speed sensor installed on the vehicle, images captured by a camera, or location information, the aforementioned notification will not be issued. The in-vehicle device according to claim 1.

14. A control program executed by an in-vehicle device that notifies the driver of a vehicle if the vehicle does not start moving after the traffic light, as recognized from the image captured by the camera while the vehicle is stopped, changes from red to blue, If the vehicle remains stopped and does not move after the aforementioned notification has been given, the notification will not be resent when the traffic light changes from red to green. Control program.

15. An in-vehicle device that notifies the driver of a vehicle if, while the vehicle is stopped, the traffic light status, as recognized from the image captured by the camera, changes from red to blue, but the vehicle does not start moving, and the device has a controller. The aforementioned controller, After the aforementioned notification has been given, no further notification will be given until the vehicle has started moving. In-vehicle device.

16. A control program executed by an in-vehicle device that notifies the driver of a vehicle if the vehicle does not start moving after the traffic light, as recognized from the image captured by the camera while the vehicle is stopped, changes from red to blue, After the aforementioned notification has been given, no further notification will be given until the vehicle has started moving. Control program.

Images