Vehicle warning devices

Abstract

To combine safety and convenience by mitigating operation restrictions on a specific function when predetermined conditions are satisfied while issuing a warning when operation of portable equipment by a driver is detected while a vehicle is traveling.SOLUTION: A vehicle warning device issues a warning to a driver according to a state of the driver and a use state of a portable equipment while a vehicle is traveling, and includes a warning level determination unit and a warning unit. The warning level determination unit acquires line-of-sight information including a direction of a line-of-sight of the driver, motion information indicating motion of a specific portion of the driver, and screen information indicating a state of a screen of the portable equipment, determines a state of the driver and a use state of the portable equipment by the driver based on the information, and determines a warning level relative to the driver according to determination results. The warning unit gives stronger warning at a higher warning level according to the warning level. The warning level determination unit lowers the warning level when a function in use in the portable equipment is a permitted function permitted in advance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a vehicle warning device, particularly a vehicle warning device that warns an operator about operations on a portable device while the vehicle is in motion.

Background Art

[0002] With the spread of portable devices such as smartphones and tablet terminals, so-called "while-smartphone" cases, that is, cases where a driver stares at or operates a portable device while the vehicle is in motion, have become scattered. Since such actions by a driver may induce vehicle accidents, technologies for restricting operations on a portable device by a driver have been proposed.

[0003] For example, in Patent Document 1, based on the position information of a specific part of a driver and the difference in the reflectance of light near the specific part, it is determined whether the driver is operating a smartphone or the like, and when it is determined that the driver is operating a device such as a smartphone, an information processing device that prompts safe driving is disclosed. In addition, in Patent Document 2, when it is determined that a driver is using a portable device while the vehicle is in motion based on the heat distribution inside the vehicle measured by an infrared sensor and the shift position detected by a shift sensor, a warning device that warns the driver is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, in recent years, there has been an increase in drivers who obtain necessary driving information from their mobile devices, such as using the navigation function on their devices to check the route and estimated travel time to their destination. While the operation of mobile devices while driving should be strictly restricted, the operation and use of certain functions may be permitted only if safety can be guaranteed.

[0006] However, the conventional information processing or warning devices described above issue a uniform warning when they determine that a driver is operating a portable device. In effect, this treats the operation and use of portable devices as a completely prohibited act. Therefore, even functions necessary for obtaining driving information are not permitted to be used at all, which can be inconvenient.

[0007] The present invention aims to address such situations. Specifically, it aims to achieve a balance between safety and convenience by providing warnings to restrict the operation of portable devices by the driver while the vehicle is in motion, while relaxing restrictions on operation related to specific functions. [Means for solving the problem]

[0008] To solve these problems, the vehicle warning device according to the present invention has the following configuration. In other words, one aspect of the present invention provides a vehicle warning device that issues a warning to a driver while the vehicle is in motion according to the driver's condition and the driver's use of a portable device, comprising: a gaze information acquisition unit that acquires gaze information including the direction of the driver's gaze; a motion information acquisition unit that acquires motion information indicating the movement of a specific part of the driver; a device information acquisition unit that acquires screen information indicating the state of the screen of the portable device; a warning level determination unit that determines the driver's condition and the driver's use of the portable device based on the gaze information, motion information, and screen information, and determines a warning level to the driver according to the determination result; and a warning unit that issues a stronger warning the higher the warning level, wherein the warning level determination unit lowers the warning level when the function being used on the portable device is an authorized function that has been permitted to be used in advance. [Effects of the Invention]

[0009] A driver assistance system with these features can detect when the driver operates a portable device while the vehicle is in motion and issue a warning, while also relaxing restrictions on the operation of certain functions if predetermined conditions are met, thereby achieving both safety and convenience. [Brief explanation of the drawing]

[0010] [Figure 1] This is an explanatory diagram showing a schematic configuration of a vehicle control system including a vehicle warning device according to an embodiment of the present invention. [Figure 2] This is an explanatory diagram showing an example of an image taken inside a vehicle equipped with a vehicle warning device according to an embodiment of the present invention. [Figure 3] This is an explanatory diagram showing an example of an image taken inside a vehicle equipped with a vehicle warning device according to an embodiment of the present invention. [Figure 4] This is an explanatory diagram showing an example of the screen of a portable device used in a vehicle interior equipped with a vehicle warning device according to an embodiment of the present invention. [Figure 5]This is an explanatory diagram showing an example of a table stored in the memory unit of a vehicle warning device according to an embodiment of the present invention, which associates the driver's state, the driver's usage status of a portable device, and the corresponding warning level, and shows a table related to the determination of the warning level when using general functions. [Figure 6] This is an explanatory diagram showing an example of a table stored in the memory unit of a vehicle warning device according to an embodiment of the present invention, which associates the driver's status, the driver's usage status of a portable device, and the corresponding warning level, and shows a table related to the determination of the warning level when the permission function is used. [Figure 7] This is an explanatory diagram showing an example of a table in which warning levels and warning messages corresponding to the warning levels are associated, stored in the storage unit of a vehicle warning device according to an embodiment of the present invention. [Figure 8] This is a flowchart showing the flow of warning processing in a vehicle warning device according to an embodiment of the present invention. [Figure 9] This flowchart shows the flow of the process for determining the warning level when using a general function in a vehicle warning device according to an embodiment of the present invention. [Figure 10] This flowchart shows the flow of the warning level determination process when the permission function is used in a vehicle warning device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will now be described with reference to the drawings. In the following description, the same reference numerals in different figures indicate parts with the same function, and redundant explanations in each figure will be omitted as appropriate.

[0012] As shown in Figure 1, the vehicle warning device according to an embodiment of the present invention functions as part of a vehicle control system 1 and is installed in a vehicle 100. The vehicle control system 1 includes various electronic devices necessary for the operation of the vehicle 100, and a plurality of on-board ECUs (Electronic Control Units) that control these electronic devices. Each electronic device and each on-board ECU are connected to each other so as to be able to communicate with each other via an on-board network 3 such as CAN (Controller Area Network) or LIN (Local Interconnect Network), and are also connected to a central gateway (CGW) 4 as a relay device to constitute the vehicle control system 1.

[0013] Each in-vehicle ECU can be configured with, for example, a processor such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit), electrical circuits, and memory elements such as RAM (Random Access Memory) or ROM (Read Only Memory). Furthermore, some or all of the operations performed by the in-vehicle ECU can be implemented by hardware such as an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit). In the following description, detailed explanations and illustrations of electronic devices and the like that are not directly involved in the vehicle warning device (warning ECU 10) according to this embodiment will be omitted.

[0014] As shown in Figure 1, the vehicle control system 1 is configured as an in-vehicle ECU, including a warning ECU 10, an occupant monitoring ECU 20, an in-vehicle camera ECU 30, and a notification ECU 40. Each in-vehicle ECU is connected to the electronic device that it controls, and controls the operation of the connected electronic device based on information (data) acquired from the in-vehicle network 3. In addition, each in-vehicle ECU outputs information related to the operation of the connected electronic device to the in-vehicle network 3.

[0015] That is, in this embodiment, the warning ECU 10, the occupant monitoring ECU 20, the in-vehicle camera ECU 30, and the notification ECU 40 transmit and receive information to and from each other. The warning ECU 10 monitors the driver's state and the usage state of the mobile device by the driver based on the information received from the occupant monitoring ECU 20 and the in-vehicle camera ECU 30, and transmits a control signal regarding a warning according to the monitoring result to the notification ECU 40, thereby issuing a warning to the driver. The details of the warning ECU 10 will be described later.

[0016] The occupant monitoring ECU 20 constitutes a part of an occupant monitoring device (DMS: Driver Monitoring System) that monitors the occupants in the vehicle, and monitors the occupants by controlling electronic devices such as a camera, a microphone, a lighting device that illuminates the occupants, a seating sensor provided on the seat, and an opening / closing sensor of the door (none of which are shown in the figure) included in the occupant monitoring device.

[0017] For example, the occupant monitoring ECU 20 monitors the line of sight of the occupant based on the image captured by the camera 21, and outputs the line-of-sight information as the monitoring result to the in-vehicle network 3. Specifically, the occupant monitoring ECU 20 calculates the direction of the occupant's line of sight and the position coordinates of the target object at the end of the line of sight from the image captured by the camera 21, and outputs line-of-sight information including at least any one of information regarding the direction of the line of sight, the calculated position coordinates, and the displacement of the position coordinates to the in-vehicle network 3.

[0018] The in-vehicle camera ECU 30 is connected to the in-vehicle camera 31, controls the in-vehicle camera 31 to monitor the movement of specific parts such as the driver's hands and fingers, and monitors the state of the mobile device in the driver's seat and around the driver's seat. The in-vehicle camera 31 is provided at a position where it can image the driver's seat and the driver from a position above the driver's seat such as the ceiling or the pillar, and outputs the captured image to the in-vehicle camera ECU 30. The in-vehicle camera ECU 30 extracts movement information indicating the movement of specific parts of the driver and screen information indicating the state of the screen of the mobile device based on the image captured by the in-vehicle camera 31.

[0019] More specifically, the in-vehicle camera ECU 30 calculates the position coordinates of specific body parts, such as the driver's hands and fingers, based on the images captured by the in-vehicle camera 31, and outputs the displacement of these position coordinates as motion information of the specific body parts to the in-vehicle network 3. The in-vehicle camera ECU 30 also outputs screen information to the in-vehicle network 3, including at least illumination information indicating the illumination status of the mobile device's screen, based on the images captured by the in-vehicle camera 31. Furthermore, if the mobile device's screen displays an icon to identify the function (application) currently in use, or a two-dimensional code such as a QR code (registered trademark), the in-vehicle camera ECU 30 includes this as function identification information in the screen information and outputs it to the in-vehicle network 3. The position coordinates of the mobile device's screen and the displacement of the screen's position coordinates may also be included in the screen information.

[0020] Figures 2 and 3 show examples of images captured by the in-vehicle camera 31. The in-car camera 31 captures images of the driver's seat and the driver from a position above the driver's seat, such as on the ceiling or pillars. Therefore, it can capture images that include, for example, the steering wheel 5, the meter 32 in front of the driver's seat, the display device 42 and other in-car equipment, the dashboard 43, specific body parts 81 such as the driver's hands and fingers, and the screen of a portable device 80 held by the driver.

[0021] Figure 2 shows an example image of a driver holding a portable device 80 in their hand 81. Figure 3 shows an example image of the portable device 80 fixed in a predetermined position on the dashboard 43. By acquiring such images as a video or continuously acquiring still images, motion information about the driver's hand and finger movements, position coordinates indicating the position of the portable device's screen, and displacement of those position coordinates can be obtained. In Figures 2 and 3, a QR code (registered trademark) 82, which will be described later, is displayed on the screen of the portable device 80.

[0022] Figure 4 shows an example of the screen of the mobile device 80. In the example in Figure 4, the screen is shown when the navigation function is being used on the mobile device 80. In the example in Figure 4, a QR code (registered trademark) 82 is displayed at the edge of the screen as function identification information indicating that the navigation function is being used. If the image acquired by the in-car camera 31 contains such a QR code (registered trademark), the QR code (registered trademark) can be included in the screen information as function identification information. The screen of the mobile device 80 can display not only QR codes (registered trademark), but also other matrix-type or stack-type two-dimensional codes, icons, watermarks, etc., as function identification information.

[0023] The notification ECU 40 is connected, for example, to a speaker 41 and a display device 42, and controls the speaker 41 or the display device 42 to issue a warning to the driver of the vehicle 100 according to the warning level, in accordance with the control signal from the warning ECU 10.

[0024] Speaker 41 can output a warning message (described later) as voice, corresponding to the warning level, or a predetermined warning sound, in accordance with the notification instructions. Speaker 41 does not necessarily need to be provided for notification or warning purposes; for example, it can be used in conjunction with the speakers of the in-car audio system or navigation system.

[0025] The display device 42 consists of, for example, a display panel mounted on the dashboard or a HUD (Head-Up Display) projected onto the front windshield, and displays images that are visible to the occupants. If the display device 42 is a HUD, it can display messages or images corresponding to the warning level on the front windshield of the vehicle 100 to warn the occupants of the vehicle 100, especially the driver.

[0026] In addition to the above, the notification ECU40 can also control, for example, a seat vibrator that vibrates the driver's seat where the driver sits, and a steering wheel vibrator that vibrates the steering wheel, thereby issuing a warning to the driver through vibration according to the warning level.

[0027] Next, we will describe the warning ECU 10 as a vehicle warning device. As shown in Figure 1, the warning ECU 10 includes a CPU (Central Processing Unit) 111, ROM 112, RAM 113, and a storage unit 114. The CPU 111 performs various processes based on the program stored in the ROM 112. In this embodiment, the CPU 111 functions as the gaze information acquisition unit 11, motion information acquisition unit 12, screen information acquisition unit 13, warning level determination unit 14, and warning unit 15 shown in Figure 1 by reading the program stored in the ROM 112 into memory such as the RAM 113 and executing it.

[0028] Furthermore, the memory unit 114 stores information necessary for the CPU 111 to operate as the gaze information acquisition unit 11, the motion information acquisition unit 12, the screen information acquisition unit 13, the warning level determination unit 14, and the warning unit 15. The gaze information acquisition unit 11, the motion information acquisition unit 12, the screen information acquisition unit 13, the warning level determination unit 14, and the warning unit 15 will be described below.

[0029] The gaze information acquisition unit 11 acquires gaze information, including the direction of the driver's gaze. In this embodiment, the gaze information acquisition unit 11 acquires gaze information from the occupant monitoring ECU 20 via the in-vehicle network 3. The acquired gaze information includes at least one of the following: the direction of the occupant's gaze, the position coordinates of the target object in the line of sight, and information regarding the displacement of the position coordinates. By acquiring gaze information, the warning level determination unit 14, described later, can determine whether the driver's gaze is directed forward (whether or not they are driving distractedly), whether they are staring at the screen of a portable device, and so on.

[0030] The motion information acquisition unit 12 acquires motion information indicating the movement of specific body parts of the driver. In this embodiment, the motion information acquisition unit 12 acquires motion information from the in-vehicle camera ECU 30 via the in-vehicle network 3. The motion information includes at least the position coordinates of specific body parts such as the driver's hands and fingers, and information related to the displacement of those position coordinates. By acquiring motion information, the warning level determination unit 14, described later, can, in conjunction with screen information, determine whether there is any movement that suggests operation of a mobile device, such as the driver's fingers or hands touching the screen of a mobile device, and can determine the usage status of the mobile device.

[0031] The screen information acquisition unit 13 acquires screen information indicating the state of the mobile device's screen. In this embodiment, the screen information acquisition unit 13 acquires screen information from the in-vehicle camera ECU 30 via the in-vehicle network 3. The screen information includes at least illumination information indicating the illumination state of the mobile device's screen, and function identification information if an icon or a two-dimensional code such as a QR code (registered trademark) is displayed on the mobile device's screen to identify the function (application) currently in use. The screen information may also include the position coordinates of the mobile device's screen and the displacement of those position coordinates.

[0032] The screen illumination information of mobile devices includes information about the brightness level of the screen, such as whether it is brightly lit or dimly lit. The brightness level depends on the type of mobile device and its settings, but generally, the screen will be brightly lit when the mobile device is being used continuously, and will switch to a dimly lit state after a period of inactivity, and then turn off completely.

[0033] Furthermore, when the screen of a mobile device is brightly lit, there is a risk that the occupant's gaze will tend to be directed towards the screen, and that during nighttime driving, the difference in brightness may temporarily obscure dark areas (ahead). Therefore, including information indicating the degree of brightness in the lighting information provides useful information for determining the usage status of the mobile device and the warning level in the warning level determination unit 14, which will be described later.

[0034] The state of being brightly lit or dimly lit can be determined by a threshold value for the brightness of the mobile device's screen. On the other hand, screen brightness changes depending on the color of the screen. That is, for example, a black screen with the backlight on at maximum brightness on a mobile device will appear brighter than a white screen with the backlight on at low brightness. In other words, the degree of brightness of the screen may change depending on the color of the display. For this reason, the threshold value for brightness mentioned above may be changed according to the display color.

[0035] Using a mobile device while driving can lead to driver inattention due to staring at or operating the device's screen, potentially causing a vehicle accident. Therefore, it is necessary to warn (remind) the driver if they are using a mobile device while driving. For this reason, the screen information acquisition unit 13 acquires screen information, including illumination information indicating the screen's illumination status, as one of the pieces of information used to determine the driver's use of the mobile device.

[0036] On the other hand, drivers may obtain necessary driving information from their mobile devices, such as checking the route and estimated travel time to their destination using the navigation function installed on the device. When using specific functions such as navigation, and only if safety is guaranteed, a certain degree of use may be permitted instead of the same warning as when using other functions. Therefore, by including function identification information in the screen information, the warning level determination unit 14, described later, can determine whether the function being used on the mobile device is a permitted function that has been allowed to be used in advance.

[0037] The motion information acquisition unit 12 and the screen information acquisition unit 13 may both acquire images captured by the in-vehicle camera 31 from the in-vehicle camera ECU 30, and then acquire motion information and screen information, respectively, from those images through calculations.

[0038] The warning level determination unit 14 determines the driver's state and the driver's use of the mobile device based on gaze information, motion information, and screen information, and determines the warning level for the driver according to the determination result. The warning level determination unit 14 also acquires control information indicating whether the vehicle 100 is running under the control of the advanced driver assistance mode, as well as vehicle speed information, and can use this information to determine the warning level. The warning level determination process by the warning level determination unit 14 will be described later.

[0039] The warning unit 15 issues a warning according to the warning level determined by the warning level determination unit 14. Specifically, for example, the warning unit 15 outputs a control signal related to the warning according to the warning level to the notification ECU 40 via the in-vehicle network 3, and issues a warning to the driver via the notification ECU 40. Warnings made by the warning unit 15 include, for example, the output of a warning message, the display of a warning message, and the sounding of a warning sound.

[0040] The memory unit 114 stores information received from the gaze information acquisition unit 11, the motion information acquisition unit 12, and the screen information acquisition unit 13. The memory unit 114 also stores tables (see Figures 5 and 6) that associate the driver's status, the driver's usage status of portable devices, and the corresponding warning levels, as well as tables (see Figure 7) that associate warning levels with corresponding warning messages.

[0041] The warning processing in the warning ECU10 configured as described above will be explained below, following the flowcharts in Figures 8 to 10. Note that the following explanation will focus on the case where the mobile device is using the navigation function as an authorization function. Figure 8 is a flowchart showing the flow of warning processing by the warning ECU 10. In the warning ECU 10, the gaze information acquisition unit 11 acquires the driver's gaze information (step S101), the motion information acquisition unit 12 acquires motion information related to the driver's hand and finger movements (step S102), and the screen information acquisition unit 13 acquires screen information from a mobile device (step S103), and each of these is output to the warning level determination unit 14.

[0042] The warning level determination unit 14 determines whether the function being used on the mobile device is a navigation function based on the function identification information contained in the screen information (step S104). If the warning level determination unit 14 determines, based on the function identification information, that the function being used on the mobile device is not a navigation function, i.e., it is not an authorized function (step S104), it assumes that a function other than an authorized function is being used and performs the warning level determination process for general function use (step S105).

[0043] On the other hand, if it is determined that the function being used on the mobile device is a navigation function (step S104), a warning level determination process for when the permission function is used is performed (step S106). When the warning level is determined by the warning level determination process (step S105 or step S106), the warning unit 15 outputs a control signal related to the warning corresponding to the determined warning level to the notification ECU 40 and warns the driver.

[0044] (Warning level determination process by warning level determination unit 14) The warning level determination process in the warning ECU 10 performs the general function usage warning level determination process (step S105 in Figure 8) if a function other than an authorized function is being used on the mobile device, and performs the authorized function usage warning level determination process (step S106 in Figure 8) if an authorized function is being used, thus performing different processing depending on the function being used on the mobile device.

[0045] (1) Warning level determination process when using general functions The warning level determination process performed by the warning level determination unit 14 when using general functions is carried out according to the flowchart in Figure 9, and the warning level is determined according to the driver's condition and the driver's use of the portable device, for example, as shown in Figure 5. The specific process is as follows.

[0046] The warning level determination unit 14 first determines whether or not the driver is distracted based on the gaze information (step S201). If the driver is distracted (Yes in step S201) and the screen of a mobile device is lit (Yes in step S212), the warning level is set to 3.0 regardless of the brightness of the lit screen (step S215).

[0047] If the driver is distracted (Yes in step S201), the mobile device screen is not lit (Yes in step S201, No in step S212), but there are movements of the driver's hands or fingers that suggest operation of the mobile device (Yes in step S213), the warning level is set to 3.0 (step S215). If the driver's hands or fingers are not moving (No in step S213), the warning level is set to 2.5 (step S214).

[0048] If the driver is not distracted (No in step S201) and the screen of the mobile device is lit (Yes in step S202), then it is determined whether the light is dim or bright (step S206).

[0049] If the screen of the mobile device is brightly lit (Yes in step S206), and there is movement of the driver's hands or fingers that suggests operation of the mobile device (Yes in step S209), the warning level is set to 2.0 (step S211). If there is no movement of the driver's hands or fingers that suggests operation of the mobile device (No in step S209), the warning level is set to 1.5 (step S210).

[0050] If the screen of the mobile device is not brightly lit, i.e., dimly lit (No in step S206), and there is movement of the driver's hands or fingers that suggests operation of the mobile device (Yes in step S207), the warning level shall be set to 1.0 (step S208). If there is no movement of the driver's hands or fingers that suggests operation of the mobile device (No in step S207), the warning level shall be set to 0.5 (step S205).

[0051] If the driver is not distracted (No. of step S201) and the screen of the mobile device is not lit (No. of step S202), then it is determined whether there are any movements of the driver's hands or fingers that would suggest operation of the mobile device (step S203). If the driver's hands or fingers are moving (Yes in step S203), the warning level is set to 0.5 (step S205). If the driver's hands or fingers are not moving (No in step S203), the warning level is set to 0 (step S204).

[0052] The warning level determination unit 14 reduces the warning level if the vehicle 100 is running in advanced driver assistance mode (Yes in step S216). That is, the final determination result is a warning level that is one level lower than the warning level determined in steps S201 to S215 (step S217). The warning level determination unit 14 uses the warning level determined in steps S201 to S215 as the final determination result if the vehicle 100 is running in manual mode (No in step S216). If the warning level determination unit 14 has acquired vehicle speed information indicating the vehicle's speed, it can reflect this in the final determination result, for example, by increasing the warning level as the vehicle speed increases.

[0053] (2) Warning level determination process when using the permission function The warning level determination process when the permission function is used by the warning level determination unit 14 is performed according to the flowchart in Figure 10, and the warning level is determined according to the driver's condition and the driver's use of the portable device, for example, as shown in Figure 6. The specific process is as follows.

[0054] The warning level determination unit 14 first determines whether or not the driver is distracted based on the eye gaze information (step S301). If the driver is not distracted (No in step S301) and the screen position of the mobile device is fixed (Yes in step S302), it is determined whether or not the driver's hands or fingers are moving (step S303). In step S302, the determination is made based on whether or not there is a displacement in the position coordinates indicating the screen position of the mobile phone. For example, if the mobile device is placed in a fixed position on the dashboard as shown in Figure 3, it is determined that the screen position of the mobile device is fixed.

[0055] If the driver is not distracted (No. in step S301), the screen position of the mobile device is fixed (Yes in step S302), and the driver's hands or fingers are not moving (No. in step S303), the warning level is set to 0 (Step S304). If the driver's hands or fingers are moving (Yes in step S303), the warning level is set to 0.5 (Step S305).

[0056] If the driver is not distracted (Step S301 No.) and the screen position of the mobile device is not fixed (Step S302 No.), the warning level is set to 0.5 regardless of whether the driver's hands or fingers are moving or not (Step S305). However, if the screen position of the mobile device is not fixed, it is assumed that the driver is holding the mobile device in their hand, so the warning level may be uniformly set to the maximum level regardless of the driver's gaze or hand and finger movements.

[0057] If the driver is distracted (Yes in step S301) and the screen position of the mobile device is fixed (Yes in step S306), it is determined whether the driver's hands or fingers are moving (step S307). If the driver is distracted (Yes in step S301) and the screen position of the mobile device is fixed (Yes in step S306), and the driver's hands or fingers are not moving (No in step S307), the warning level is set to 2.5 (Step S308). If the driver's hands or fingers are moving (Yes in step S307), the warning level is set to 3.0 (Step S309). If the driver is distracted (Yes in step S301) and the screen position of the mobile device is not fixed (No in step S306), the warning level is set to 3.0 regardless of whether the driver's hands or fingers are moving (step S309).

[0058] The warning level determination unit 14 reduces the warning level if the vehicle 100 is running in advanced driver assistance mode (Yes in step S310). That is, the final determination result is a warning level that is one level lower than the warning level determined in steps S301 to S309 (step S311). The warning level determination unit 14 uses the warning level determined in steps S301 to S309 as the final determination result if the vehicle 100 is running in manual mode (No in step S310).

[0059] Furthermore, the warning level determination unit 14 can acquire vehicle speed information indicating the vehicle's speed from a vehicle speed sensor (not shown). In this case, the final determination result can be adjusted according to the vehicle speed information, such as increasing the warning level as the vehicle speed increases.

[0060] Thus, the warning level determination unit 14 performs a different process than the warning level determination process for general function use when the function being used on the portable device is a pre-approved authorized function, and selects a lower warning level than the warning level for general function use if predetermined conditions are met. In other words, when a portable device is used for a general function other than an authorized function while the vehicle is in motion, a warning is issued to restrict its use, and the restrictions on the use of specific authorized functions are relaxed to allow their use if predetermined conditions are met, thereby achieving both safety and convenience.

[0061] As shown in Figure 1, by equipping the vehicle 100 with a vehicle control system 1 including such a vehicle warning device (warning ECU), the vehicle 100 can provide appropriate warnings (attention alerts) to the driver's use of portable devices while driving, while relaxing restrictions on the use of predetermined permitted functions such as navigation functions, thereby achieving both safety and convenience.

[0062] In the above-described embodiment, the driver's condition and the driver's use of portable devices were determined using various information obtained from images captured by camera 21 (occupant monitoring ECU) and images captured by in-vehicle camera 31 (in-vehicle camera ECU). However, the system is not limited to this, and for example, a 360° camera can be installed in the vehicle, and various information can be obtained from the images captured by it.

[0063] As described above, the vehicle warning device according to this embodiment provides a warning to restrict the driver's operation of a portable device while the vehicle is in motion, while relaxing restrictions on operation of specific functions, thereby achieving both safety and convenience.

[0064] While embodiments of the present invention have been described in detail with reference to the drawings, the specific configurations are not limited to these embodiments, and any design changes, etc., that do not depart from the spirit of the present invention are also included. Furthermore, the above-described embodiments can be combined by utilizing each other's technologies, as long as there are no particular contradictions or problems in their purpose and configuration. [Explanation of Symbols]

[0065] 1: Vehicle control system, 2: Warning level, 3: In-vehicle network, 4: CGW, 10: Warning ECU, 11: Eye-tracking information acquisition unit, 12: Motion information acquisition unit, 13: Screen information acquisition unit, 14: Warning level determination unit, 15: Warning unit, 20: Occupant monitoring ECU, 21: Camera, 30: In-vehicle camera ECU, 31: In-vehicle camera, 40: Notification ECU, 41: Speaker, 42: Display device, 80: Portable device, 100: Vehicle, 111: CPU, 112: ROM, 113: RAM, 114: Memory unit

Claims

1. A vehicle warning device that issues a warning to the driver while the vehicle is in motion, depending on the driver's condition and the driver's use of a portable device, A gaze information acquisition unit that acquires gaze information including the direction of the driver's gaze, A motion information acquisition unit that acquires motion information indicating the movement of specific body parts of the driver, A device information acquisition unit that acquires screen information indicating the state of the screen of the aforementioned mobile device, A warning level determination unit determines the driver's state and the driver's usage status of the portable device based on the gaze information, motion information, and screen information, and determines the warning level for the driver according to the determination result. The system includes a warning unit that issues stronger warnings as the warning level increases, depending on the warning level. The warning level determination unit is a vehicle warning device that lowers the warning level when the function being used on the portable device is an authorized function that has been permitted to be used in advance.

2. The warning level determination unit determines whether the function being used by the portable device is an authorized function based on the screen information, as described in claim 1 for a vehicle warning device.

3. The vehicle warning device according to claim 1 or claim 2, wherein the permission function is a navigation function.

4. The vehicle warning device according to claim 1 or claim 2, wherein the screen information includes illumination information indicating the illumination status of the screen of the portable device and function identification information displayed on the screen while the permission function is in use.

5. The vehicle warning device according to claim 4, wherein the function identification information is an icon or a two-dimensional code.

6. The warning level determination unit lowers the warning level when the vehicle is driving in an advanced driver assistance mode, as described in claim 1 or claim 2.

7. The warning level determination unit acquires speed information indicating the vehicle's travel speed, and increases the warning level as the travel speed increases, as described in claim 1 or claim 2.

Images