In-vehicle devices and programs

Abstract

To provide an in-vehicle device and program that can suppress the use of mobile devices by drivers even when the vehicle is stopped. [Solution] The in-vehicle device according to the embodiment includes a controller. When the controller detects that the vehicle driver is distracted by a mobile device, it issues a warning to the driver. The controller determines whether the driver is distracted by a mobile device based on the distance from the driver's face to the mobile device, or the direction of the driver's gaze, as determined by image recognition from the camera image captured by the in-vehicle camera that captures the interior of the vehicle. The controller sets the detection sensitivity for when the vehicle is stopped to be higher than when the vehicle is in motion.

Description

Technical Field

[0006] , ,

[0005] , , , ,

[0007] ,

[0001] The disclosed embodiments relate to in-vehicle devices and programs.

Background Art

[0002] During vehicle operation, a driver's operation of a mobile terminal such as a smartphone can cause an accident. Therefore, except when the vehicle is stopped, the use of a mobile terminal by the driver is prohibited by law.

[0003] Therefore, a system has been proposed that issues a warning when it detects that a driver is looking at a smartphone while driving (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, since the prior art system limits the target of the warning against smartphone use to the driver during driving, a warning is not issued even if the driver is looking at a smartphone while stopped due to traffic congestion or at a traffic signal.

[0006] For this reason, a driver looking at a smartphone may release the brakes, for example, when a vehicle ahead has stopped but a vehicle on the side starts moving forward or a vehicle behind honks,

[0007] If a vehicle's brakes are released, an automatic transmission vehicle may move forward due to creep and potentially rear-end the vehicle in front. Therefore, even when stopped, the use of mobile devices by drivers is dangerous and should be discouraged.

[0008] One embodiment of the invention has been made in view of the above, and aims to provide an in-vehicle device and program that can suppress the driver's use of a mobile device even when the vehicle is stopped. [Means for solving the problem]

[0009] An in-vehicle device according to one embodiment includes a controller. When the controller detects that the driver of the vehicle is distracted by a mobile device, it issues a warning to the driver. The controller determines whether the driver is distracted by the mobile device based on the distance from the driver's face to the mobile device, which is recognized from the camera image captured by an in-vehicle camera that captures the interior of the vehicle, or based on the direction of the driver's gaze. The controller sets a higher sensitivity for detecting when the driver is distracted by the mobile device when the vehicle is stopped than when the vehicle is in motion. [Effects of the Invention]

[0010] In one embodiment of the in-vehicle device, the detection sensitivity for when the vehicle is stopped is set higher than when the vehicle is in motion to detect when the driver is distracted by the mobile terminal. Therefore, even when the vehicle is stopped, a warning can be issued to discourage the driver from using the mobile terminal. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is an explanatory diagram showing an example of the configuration of an in-vehicle device according to an embodiment. [Figure 2] Figure 2 is an explanatory diagram showing an example of detection sensitivity information according to the embodiment. [Figure 3] Figure 3 is an explanatory diagram showing an example of mobile device usage detection according to the embodiment. [Figure 4] Figure 4 is an explanatory diagram showing an example of a warning method according to the embodiment. [Figure 5] Figure 5 is an explanatory diagram showing another example of detection sensitivity information according to the embodiment. [Figure 6] Figure 6 is an explanatory diagram showing another example of mobile device usage detection according to the embodiment. [Figure 7] Figure 7 is a flowchart showing an example of the process performed by the controller according to this embodiment. [Modes for carrying out the invention]

[0012] The embodiments of the in-vehicle device and the program executed by the controller of the in-vehicle device will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the embodiments described below.

[0013] The in-vehicle device according to this embodiment is, for example, a drive recorder that records video of the surroundings of the vehicle on which the in-vehicle device is installed. In addition to its function as a drive recorder, the in-vehicle device according to this embodiment also has a function that warns the driver when it detects that the driver of the vehicle is distracted by a mobile device.

[0014] In the following explanation, "the driver of the vehicle being distracted by a mobile device" includes the driver operating the mobile device, the driver making a call using the mobile device, and the driver staring at the mobile device.

[0015] Figure 1 is an explanatory diagram showing an example of the configuration of an in-vehicle device 1 according to an embodiment. The in-vehicle device 1 is mounted on the inner surface of the vehicle's windshield. As shown in Figure 1, the in-vehicle device 1 includes a front camera 2, an interior camera 3, a display 4, a speaker 5, a storage unit 6, and a controller 7.

[0016] The front camera 2 is an imaging device that images the front of the host vehicle. The front camera 2 outputs the image information of the captured image to the controller 7. The in-vehicle camera 3 is an imaging device that images the interior of the host vehicle. The in-vehicle camera 3 outputs the image information of the captured image to the controller 7.

[0017] The display 4 is an information display device that displays various image information under the control of the controller 7. The speaker 5 is a sound output device that outputs various audio information under the control of the controller 7.

[0018] The storage unit 6 is an information storage device such as a data flash, for example, and stores recording information 61, detection sensitivity information 62, and the like. The recording information 61 is the image information of the images captured by the front camera 2 and the in-vehicle camera 3.

[0019] The detection sensitivity information 62 is information regarding the detection sensitivity used when the controller 7 detects the use of a mobile terminal by the driver. Here, referring to FIG. 2, an example of the detection sensitivity information 62 will be described. FIG. 2 is an explanatory diagram showing an example of the detection sensitivity information 62 according to the embodiment.

[0020] As shown in FIG. 2, the detection sensitivity information 62 is information in which the vehicle state and the mobile terminal use detection condition are associated. In the example shown in FIG. 2, for the vehicle state of "driving", the mobile terminal use detection condition of "continuously detecting the mobile terminal at a position within a predetermined distance from the driver's face for the first time (for example, driving detection sensitivity: 5 seconds)" is associated.

[0021] Also, for the vehicle state of "stopped (including slow driving)", the mobile terminal use detection condition of "continuously detecting the mobile terminal at a position within a predetermined distance from the driver's face for the second time (for example, stopped detection sensitivity: 1 second)" is associated.

[0022] In this context, "driving slowly" refers to a state where the vehicle is traveling at a speed that allows it to stop immediately (for example, 10 km / h or less). This detection sensitivity information 62 is used by the controller 7 when it detects that the vehicle driver is distracted by a mobile device.

[0023] Returning to the explanation of Figure 1, the controller 7 includes a microcomputer with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various circuits. The controller 7 includes an image processing unit 71 that functions by the CPU executing a program stored in ROM using RAM as a workspace, and an image recognition AI (Artificial Intelligence).

[0024] Furthermore, the image processing unit 71 and image recognition AI 72 of the controller 7 may be partially or entirely composed of hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0025] The image processing unit 71 and image recognition AI 72 of the controller 7 realize or execute the information processing functions described below. Note that the internal configuration of the controller 7 is not limited to the configuration shown in Figure 1; other configurations are acceptable as long as they perform the information processing described later. Furthermore, the program executed by the controller 7 may be stored in a storage device via an external communication line or the like.

[0026] The image processing unit 71 stores the image information input from the front camera 2 and the interior camera 3 as recording information 61 in the storage unit 6. The image processing unit 71 also performs various image processing on the image information input from the front camera 2 and the interior camera 3.

[0027] For example, the image processing unit 71 detects the vehicle's lane, vehicles ahead (including preceding and oncoming vehicles), and traffic lights by performing image processing such as edge detection and pattern matching on the image information input from the front camera 2. The image processing unit 71 also detects the vehicle's speed based on changes in the position of stationary objects in multiple frames captured in succession.

[0028] The image processing unit 71 outputs information to the image recognition AI 72 indicating the detected driving lane, the vehicle ahead, traffic lights, the vehicle's speed, and the distance to the vehicle ahead. The vehicle's speed may be input to the image recognition AI 72 from a speed sensor mounted on the vehicle. The distance to the vehicle ahead may be input to the image recognition AI 72 from a radar device or the like mounted on the vehicle.

[0029] The image recognition AI 72 is pre-trained to determine, based on information input from the front camera 2 and the image processing unit 71, whether there is a vehicle ahead, whether the vehicle is moving or stopped (including driving slowly), and the driver's status.

[0030] The controller 7, when it detects that the driver is distracted by a mobile device while the vehicle is in motion, issues a warning to the driver. Furthermore, the controller 7 issues a warning not only while the vehicle is in motion, but also when the vehicle is stopped (including when driving slowly), if it detects that the driver is distracted by a mobile device. In this case, that is, when the vehicle is stopped, the controller 7 sets a higher sensitivity to detect when the driver is distracted by a mobile device than when the vehicle is in motion.

[0031] In other words, when the vehicle is in motion, the controller 7 refers to the detection sensitivity information 62 and sets the mobile device usage detection conditions associated with the vehicle being in motion. Also, when the vehicle is stopped, including when it is moving slowly, the controller 7 refers to the detection sensitivity information 62 and sets the mobile device usage detection conditions associated with the vehicle being stopped (including when it is moving slowly).

[0032] Therefore, when the vehicle is in motion, if the controller 7 detects a mobile device within a predetermined distance from the driver's face for a continuous period of one hour, it detects that the driver is distracted by the mobile device.

[0033] When the vehicle is stopped, including when it is moving slowly, the controller 7 detects that the driver is distracted by the mobile device if it detects the mobile device for a period of time shorter than the first hour, within a predetermined distance from the driver's face, for a period of two hours.

[0034] Here, with reference to Figure 3, an example of a method for detecting when a vehicle driver is distracted by a mobile device will be described. Figure 3 is an explanatory diagram showing an example of mobile device usage detection according to an embodiment.

[0035] As shown in Figure 3, the controller 7 of the in-vehicle device 1 captures images of the driver's condition using the in-vehicle camera 3. Furthermore, the controller 7 measures the distance D from the driver 10 to the mobile terminal 20 by performing image recognition on the captured images.

[0036] Then, if the vehicle is in motion, the controller 7 detects that the driver 10 is distracted by the mobile device 20 if the distance D remains within a predetermined distance (for example, 50 cm) for 1 hour, and issues a warning to the driver 10.

[0037] Furthermore, if the vehicle is stopped, including when it is moving slowly, the controller 7 detects that the driver 10 is distracted by the mobile device 20 if the distance D remains within a predetermined distance (for example, 50 cm) for a period shorter than the first hour (2 hours), and issues a warning to the driver 10.

[0038] In this way, the controller 7 uses image recognition to detect that the driver 10 is distracted by the mobile device 20 from the camera image captured by the in-vehicle camera 3, thus accurately detecting that the driver 10 is distracted by the mobile device 20.

[0039] Next, an example of a warning method by the controller 7 will be described with reference to Figure 4. Figure 4 is an explanatory diagram showing an example of a warning method according to the embodiment.

[0040] As shown in Figure 4, if the controller 7 detects, for example, that the driver 10 is distracted by the mobile device 20 while the vehicle is in motion, it displays the message "Warning! Dangerous! Stop using the mobile device!" on the display 4. Furthermore, the controller 7 emits a warning sound B from the speaker 5.

[0041] As a result, the in-vehicle device 1 can suppress the occurrence of accidents caused by distracted driving due to the use of the mobile terminal 20, not only while driving but also while stopped, including while driving slowly.

[0042] Furthermore, the in-vehicle device 1 sets a higher sensitivity for detecting when the driver 10 is distracted by the mobile device 20 when the vehicle is stopped, including when the vehicle is moving slowly, compared to when the vehicle is in motion. As a result, the in-vehicle device 1 becomes more sensitive to detecting mobile device use, for example, during traffic congestion where the vehicle repeatedly starts and stops at low speeds, and is more likely to issue a warning, thereby suppressing the use of the mobile device 20.

[0043] Furthermore, the in-vehicle device 1 warns the driver 10 to stop using the mobile terminal 20 if they operate it for a shorter period of time than when the vehicle is moving, including when the vehicle is stopped, thus reducing the occurrence of accidents caused by distracted driving in traffic jams.

[0044] Furthermore, the controller 7 detects from the image of the driver 10 captured by the in-vehicle camera 3 that the driver 10 has stopped using the mobile device 20, and terminates the warning. This prevents the in-vehicle device 1 from causing discomfort to the driver 10 by continuing to display a warning even though the driver 10 has stopped using the mobile device 20.

[0045] Note that the detection sensitivity information 62 stored in the memory unit 6 is not limited to the detection sensitivity information 62 shown in Figure 2, but may be, for example, the detection sensitivity information 62 shown in Figure 5. Figure 5 is an explanatory diagram showing another example of the detection sensitivity information 62 according to the embodiment.

[0046] As shown in Figure 5, in another example, the detection sensitivity information 62 associates the vehicle state of "driving" with the mobile device usage detection condition "the driver's gaze was continuously directed towards the mobile device for 1 hour (for example, driving detection sensitivity: 5 seconds)."

[0047] Furthermore, the vehicle state of "stopped (including driving slowly)" is associated with the mobile device usage detection condition of "the driver's gaze is continuously directed towards the mobile device for two hours (for example, detection sensitivity while driving: 1 second)."

[0048] If the detection sensitivity information 62 shown in Figure 5 is stored in memory, the controller 7 will detect that the driver 10 is distracted by the mobile terminal 20 if the driver's gaze has been directed towards the mobile terminal 20 for a continuous period of time while the vehicle is in motion.

[0049] Furthermore, if the vehicle is stopped, including when it is moving slowly, the controller 7 detects that the driver 10's gaze has been directed at the mobile device 20 for a period of time shorter than the first hour (2 hours) and that the driver 10 is distracted by the mobile device 20.

[0050] Here, with reference to Figure 6, an example of a method for detecting that the vehicle driver 10 is distracted by the mobile device 20 will be described. Figure 6 is an explanatory diagram showing another example of mobile device usage detection according to the embodiment.

[0051] When the detection sensitivity information 62 shown in Figure 5 is stored in the storage unit 6, the controller 7 of the in-vehicle device 1 captures the state of the driver 10 using the in-vehicle camera 3, as shown in Figure 6. Furthermore, the controller 7 determines the direction of the driver 10's gaze L by performing image recognition on the captured image.

[0052] Furthermore, if the vehicle is in motion, the controller 7 detects that the driver's gaze L has been directed towards the mobile device 20 for a continuous period of time, and detects that the driver 10 is distracted by the mobile device 20, and issues a warning to the driver 10.

[0053] Furthermore, if the vehicle is stopped, including when it is moving slowly, the controller 7 detects that the driver's gaze L has been directed towards the mobile device 20 for a period of time shorter than the first hour (2 hours), and detects that the driver 10 is distracted by the mobile device 20, and issues a warning to the driver 10.

[0054] In this way, the controller 7 uses image recognition to detect that the driver 10 is distracted by the mobile device 20 from the camera image captured by the in-vehicle camera 3, thus accurately detecting that the driver 10 is distracted by the mobile device 20.

[0055] Furthermore, when the vehicle is stopped, including when it is moving slowly, the in-vehicle device 1 will warn the driver 10 to stop using the mobile terminal 20 even if they are not operating it, simply by directing their gaze L at the mobile terminal 20 for a shorter period than when the vehicle is moving. This makes the in-vehicle device 1 more likely to issue warnings, thereby discouraging the use of the mobile terminal 20.

[0056] Next, with reference to Figure 7, the processes performed by the controller 7 will be described. Figure 7 is a flowchart showing an example of the processes performed by the controller 7 according to this embodiment. The controller 7 repeatedly performs the processes shown in Figure 7 during the period when the ignition switch of its own vehicle is turned on.

[0057] As shown in Figure 7, when the ignition switch of the vehicle is turned on, the controller 7 first determines whether the vehicle is stopped (including driving slowly) or not (step S101). The controller 7 estimates the vehicle's speed from the images continuously captured by the front camera 2, and determines that the vehicle is stopped if its speed is 10 km / h or less. The controller 7 also determines that the vehicle is not stopped, i.e., is moving, if its speed is greater than 10 km / h.

[0058] If the controller 7 determines that the vehicle is not stationary (step S101, No), that is, that the vehicle is in motion, it sets the mobile device detection sensitivity to the motion detection sensitivity (see Figure 2) (step S102) and proceeds to step S104.

[0059] Furthermore, if the controller 7 determines that the vehicle is stopped (step S101, Yes), it sets the mobile device usage detection sensitivity to a stopped detection sensitivity that is higher than the driving detection sensitivity (step S103), and proceeds to step S104. In this way, the controller 7 sets the same detection sensitivity (stopped detection sensitivity) when the vehicle is completely stopped and when the vehicle is moving slowly.

[0060] Subsequently, in step S104, the controller 7 determines whether or not it has detected the driver 10 using the mobile terminal 20. The use of the mobile terminal 20 here includes operating the mobile terminal 20, making a call using the mobile terminal 20, and looking at the display of the mobile terminal 20.

[0061] If the controller 7 determines that it has not detected the driver 10 using the mobile terminal 20 (step S104, No), it proceeds to step S101. If the controller 7 determines that it has detected the driver 10 using the mobile terminal 20 (step S104, Yes), it issues a warning to the driver 10 (request to stop using the mobile terminal) as shown in Figure 4, for example (step S105).

[0062] Subsequently, the controller 7 determines whether the driver 10 has stopped using the mobile terminal 20 (step S106). If the controller 7 determines that the driver 10 has not stopped using the mobile terminal 20 (step S106, No), the process moves to step S105.

[0063] Furthermore, if the controller 7 determines that the driver 10 has stopped using the mobile terminal 20 (step S106, Yes), it terminates the current process and restarts the process from step S101.

[0064] In the embodiment described above, we explained a case where the detection sensitivity (hereinafter simply referred to as "detection sensitivity") for the driver 10 being distracted by the mobile terminal 20 is set higher when the vehicle is stationary than when it is in motion. However, this is just one example. The conditions under which the controller 7 sets the detection sensitivity to a higher level are not limited to the vehicle being stationary.

[0065] For example, the controller 7 may be configured to set the detection sensitivity higher than the detection sensitivity while driving when it recognizes a vehicle that is at a predetermined distance (e.g., 10m) or less from the camera image captured by the front camera 2 while the vehicle is stopped.

[0066] In this case, even when the vehicle is stopped, if there is no vehicle ahead within 10 meters, the controller 7 will not issue a warning even if the driver 10 is distracted by the mobile device 20. This prevents the in-vehicle device 1 from issuing unnecessary warnings in safe situations where the vehicle is completely stopped and there is no vehicle ahead within 10 meters.

[0067] Furthermore, the controller 7 may be configured to set the detection sensitivity higher than the detection sensitivity while driving when it recognizes an oncoming vehicle with a predetermined distance (for example, 10m) or less from the camera image captured by the front camera 2 while the vehicle is stopped.

[0068] As a result, even if there is no vehicle ahead of the vehicle, if there is an oncoming vehicle at a predetermined distance (10m) or less from the vehicle, the in-vehicle device 1 is more likely to issue a warning when the driver 10 is distracted by the mobile terminal 20, thereby discouraging the driver 10 from using the mobile terminal 20.

[0069] Furthermore, the controller 7 may be configured to set the detection sensitivity higher than the detection sensitivity during driving when it recognizes a vehicle stop line from the camera image captured by the front camera 2 while the vehicle is stopped.

[0070] As a result, the in-vehicle device 1 is more likely to issue a warning when the driver 10 is distracted by the mobile device 20 while stopped at the front of the line at a red light, before a pedestrian crossing, or at a stop sign, thereby discouraging the driver 10 from using the mobile device 20.

[0071] Furthermore, the image recognition AI 72 according to the above embodiment may be configured to be pre-trained to detect from camera images captured by the in-vehicle camera 3 that the driver 10 is distracted by the mobile terminal 20.

[0072] In this case, if the vehicle is in motion, the image recognition AI 72 detects that the driver 10 is distracted by the mobile device 20 if it determines that the probability of the driver 10 being distracted by the mobile device 20 is 95% or higher. Also, if the vehicle is stationary, the image recognition AI 72 detects that the driver 10 is distracted by the mobile device 20 if it determines that the probability is 70% or higher, which is lower than the first probability (95%).

[0073] With this configuration, the in-vehicle device 1 can suppress the use of the mobile device 20, for example, by increasing the sensitivity of its mobile device usage detection and issuing warnings more easily during traffic congestion where the vehicle repeatedly starts and stops at low speeds.

[0074] Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents. [Explanation of Symbols]

[0075] 1 In-vehicle device 2. Front camera 3. In-car camera 4 displays 5 speakers 6 Memory section 7 Controllers 10. Driver 20 Mobile devices 61 Recording Information 62 Detection Sensitivity Information 71 Image Processing Unit 72 Image Recognition AI B Warning tone D distance L line of sight

Claims

1. The vehicle is equipped with a controller that, when it detects that the driver is distracted by a mobile device, issues a warning to the driver. The aforementioned controller, Based on the distance from the driver's face to the mobile device, or the direction of the driver's gaze, as determined from the camera image captured by the in-vehicle camera that captures the interior of the vehicle, it is determined whether or not the driver is distracted by the mobile device. While the vehicle is stationary, the detection sensitivity for the driver being distracted by the mobile device is set higher than when the vehicle is in motion. In-vehicle device.

2. The aforementioned controller, If the vehicle is in motion, and the mobile device is detected for a continuous period of one hour at a predetermined distance from the driver's face, it is detected that the driver is distracted by the mobile device. If the vehicle is stationary, and the mobile device is detected for a second period of time shorter than the first period at a predetermined distance from the driver's face, it is detected that the driver is distracted by the mobile device. The in-vehicle device according to claim 1.

3. The aforementioned controller, If the vehicle is in motion, and it is detected that the driver's gaze has been directed at the mobile device for a continuous period of time (1 hour), it is detected that the driver is distracted by the mobile device. If the vehicle is stationary, and it is detected that the driver's gaze has been directed towards the mobile device for a second period of time shorter than the first period, it is detected that the driver is distracted by the mobile device. The in-vehicle device according to claim 1.

4. The aforementioned controller, During slow-speed operation of the vehicle, the detection sensitivity is set to be the same as that used when the vehicle is stopped. The in-vehicle device according to claim 1.

5. The aforementioned controller, If a vehicle ahead of the vehicle is detected by a forward-facing camera that captures images of the area in front of the vehicle while the vehicle is stopped, the detection sensitivity is set to a higher value than the detection sensitivity while the vehicle is in motion. The in-vehicle device according to claim 1.

6. The aforementioned controller, If the camera image captured by the forward camera recognizes an oncoming vehicle that is less than or equal to a predetermined distance from the vehicle while the vehicle is stopped, the detection sensitivity is set higher than the detection sensitivity while the vehicle is in motion. The in-vehicle device according to claim 5.

7. The aforementioned controller, If the vehicle stop line is recognized from the camera image captured by the forward camera while the vehicle is stopped, the detection sensitivity is set higher than the detection sensitivity while the vehicle is in motion. The in-vehicle device according to claim 5.

8. The aforementioned controller, The system includes an AI (Artificial Intelligence) that detects from the camera image captured by the in-vehicle camera whether the driver is distracted by the mobile device. The aforementioned AI is If the vehicle is in motion, and it is determined that the probability of the driver being distracted by the mobile device is greater than or equal to the first probability, the system detects that the driver is distracted by the mobile device. If the vehicle is stationary, and the probability is determined to be lower than the first probability (the second probability) or higher, it is detected that the driver is distracted by the mobile device. The in-vehicle device according to claim 1.

9. When the system detects that the vehicle driver is distracted by a mobile device, it sends a warning to the driver via a computer. A procedure for determining whether the driver is distracted by the mobile device, based on the distance from the driver's face to the mobile device, or the direction of the driver's gaze, as determined from the camera image captured by the in-vehicle camera that captures the interior of the vehicle, While the vehicle is stopped, the procedure is to set a higher sensitivity to detect when the driver is distracted by the mobile device than when the vehicle is in motion. program.

Images