Driver monitor device, driver monitor method, and computer program for driver monitor

Abstract

To provide a driver monitoring device capable of appropriately urging a driver to pay attention in accordance with a situation around a vehicle.SOLUTION: A driver monitoring device includes: a complexity calculation section 31 that calculates a degree of complexity of a situation around a vehicle 10 on the basis of at least one image generated by an imaging section 2 mounted on the vehicle 10 and representing the situation around the vehicle 10; a setting section 32 that sets an alert condition for urging a driver of the vehicle 10 to pay attention so that the alert condition is relaxed as the degree of complexity increases; a detection section 33 that detects a behavior of the driver; and a notification processing section 35 that notifies the driver of an alert for paying attention via a notification device 4 when the detected behavior of the driver satisfies the alert condition.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a driver monitoring device, a driver monitoring method, and a computer program for driver monitoring, which monitor a driver of a vehicle. [Background technology]

[0002] A technique has been proposed for alerting a vehicle driver based on a change in brightness of a monitoring area that includes an image portion of a connecting portion between a road and a side road (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-58999 Summary of the Invention [Problem to be solved by the invention]

[0004] The ease with which a driver can recognize the presence of an object that may cause an obstacle to the vehicle's travel, such as a pedestrian attempting to cross the roadway, varies depending on the situation around the vehicle.

[0005] Therefore, an object of the present invention is to provide a driver monitoring device that can appropriately alert the driver according to the situation around the vehicle. [Means for solving the problem]

[0006] According to one embodiment, there is provided a driver monitoring device including: a complexity calculation unit that calculates the complexity of a situation around the vehicle based on at least one image representing the situation around the vehicle generated by an imaging unit mounted on the vehicle; a setting unit that sets an attention alert condition such that the higher the complexity, the more relaxed the attention alert condition for alerting the driver of the vehicle; a detection unit that detects driver behavior; and a notification processing unit that notifies the driver of the attention alert via a notification device when the detected driver behavior satisfies the attention alert condition.

[0007] In this driver monitoring device, the complexity calculation unit calculates the complexity for each of multiple partial regions into which the image is divided, and it is preferable that the setting unit relaxes the warning conditions corresponding to partial regions among the multiple partial regions whose complexity is equal to or greater than a predetermined threshold more than the warning conditions corresponding to partial regions whose complexity is less than the predetermined threshold.

[0008] Alternatively, it is preferable that the complexity calculation unit calculates the spatial complexity of the situation around the vehicle and the complexity of the situation around the vehicle as the complexity, and the setting unit sets the warning condition based on the higher of the spatial complexity and the complexity of the situation around the vehicle as the complexity.

[0009] According to another embodiment, there is provided a driver monitoring method including: calculating a complexity of a situation around a vehicle based on at least one image representing the situation around the vehicle generated by an imaging unit mounted on the vehicle; setting an attention alert condition such that the higher the complexity, the more relaxed the attention alert condition for alerting the driver of the vehicle; detecting driver behavior; and notifying the driver of an attention alert via a notification device when the detected driver behavior satisfies the attention alert condition.

[0010] According to yet another embodiment, there is provided a computer program for driver monitoring, the computer program including instructions for causing a processor mounted on the vehicle to calculate a complexity of a situation around the vehicle based on at least one image representing the situation around the vehicle generated by an imaging unit mounted on the vehicle, set an attention alert condition such that the higher the complexity, the more relaxed the attention alert condition for alerting the driver of the vehicle, detect driver behavior, and notify the driver of an attention alert via a notification device when the detected driver behavior satisfies the attention alert condition. [Effects of the Invention]

[0011] The driver monitoring device according to the present disclosure has the effect of being able to appropriately alert the driver in accordance with the situation around the vehicle. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating a driver monitor system including a driver monitor device. [Figure 2] FIG. 2 is a hardware configuration diagram of an ECU that is an example of a driver monitoring device. [Figure 3] FIG. 2 is a functional block diagram of a processor of the ECU. [Figure 4] 10(a) and 10(b) are diagrams each showing an example of the degree of complexity of the situation around the vehicle. [Figure 5] 10 is an operation flowchart of a driver monitoring process. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following describes a driver monitoring device, a driver monitoring method executed by the driver monitoring device, and a computer program for driver monitoring, with reference to the drawings. The driver monitoring device calculates the complexity of the situation around the vehicle from an image showing the vehicle's surroundings, and sets conditions for issuing a warning based on the calculated complexity.

[0014] 1 is a schematic diagram of a driver monitor system including a driver monitor device. In this embodiment, a driver monitor system 1 is mounted on a vehicle 10 and controls the vehicle 10. The driver monitor system 1 includes a camera 2, a driver monitor camera 3, a notification device 4, and an electronic control unit (ECU) 5, which is an example of a driver monitor device. The camera 2, the driver monitor camera 3, the notification device 4, and the ECU 5 are connected to each other so that they can communicate with each other.

[0015] Camera 2 is an example of an imaging unit, and is attached so as to face a predetermined area around vehicle 10, such as the area ahead of vehicle 10. Note that vehicle 10 may be provided with a plurality of cameras with different imaging directions or focal lengths. Camera 2 captures an image of the predetermined area at each predetermined imaging cycle, thereby generating an image of the predetermined area, and outputs the generated image to ECU 5.

[0016] The driver monitor camera 3 is an example of an in-vehicle sensor, and is attached to the instrument panel or its vicinity, facing the driver seated in the driver's seat of the vehicle 10, so that the head of the driver is included in the imaging target area. The driver monitor camera 3 may have a light source such as an infrared LED. The driver monitor camera 3 captures an image of the driver at each predetermined imaging period to generate an image of the driver (hereinafter referred to as a driver image), and outputs the generated driver image to the ECU 5.

[0017] The notification device 4 is provided in the passenger compartment of the vehicle 10 and is a device that provides a predetermined notification to the driver by light, sound, vibration, text display, or image display. To achieve this, the notification device 4 has, for example, at least one of a speaker, a light source, a vibrator, or a display device. When the notification device 4 receives a notification indicating a warning to the driver from the ECU 5, it notifies the driver of the warning by sound from the speaker, light emission or flashing of the light source, vibration of the vibrator, or display of a warning message on the display device.

[0018] The ECU 5 sets an attention calling condition depending on the complexity of the situation around the vehicle 10, and when the set attention calling condition is satisfied, the ECU 5 notifies the driver of a warning via the notification device 4.

[0019] Fig. 2 is a hardware configuration diagram of the ECU 5. As shown in Fig. 2, the ECU 5 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may be configured as separate circuits, or may be configured integrally as a single integrated circuit.

[0020] The communication interface 21 has an interface circuit for connecting the ECU 5 to other devices in the vehicle. The communication interface 21 passes the image received from the camera 2 and the driver image received from the driver monitor camera 3 to the processor 23 via the in-vehicle network. The communication interface 21 also outputs the notification signal received from the processor 23 to the notification device 4 via the in-vehicle network.

[0021] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores various data used in the driver monitoring process executed by the processor 23.

[0022] The processor 23 includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit. The processor 23 executes driver monitoring processing.

[0023] 3 is a functional block diagram of the processor 23 relating to the driver monitoring process. The processor 23 has a complexity calculation unit 31, a setting unit 32, a detection unit 33, a determination unit 34, and a notification processing unit 35. Each of these units included in the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these units included in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23.

[0024] The complexity calculation unit 31 calculates the complexity of the situation around the vehicle 10, particularly the complexity related to how it appears to the driver, based on one or more images generated by the camera 2. In this embodiment, the complexity calculation unit 31 calculates both or either of the spatial complexity and the time-varying complexity.

[0025] The complexity calculation unit 31 calculates, as spatial complexity, an index representing the degree of visual confusion based on a target region set on the image, such as subband entropy or feature congestion. Note that the complexity calculation unit 31 may set the target region to the entire image, or may set a partial region on the image as the target region. Alternatively, the complexity calculation unit 31 may divide the image into multiple partial regions and set two or more of each partial region as the target region.

[0026] Specifically, the complexity calculation unit 31 calculates the subband entropy according to the following equation.

number

[0027] The complexity calculation unit 31 calculates feature congestion according to the method described in, for example, Ruth Rosenholtz et al., "Measuring Visual Clutter," Journal of Vision (2007), 7(2);17, 1-22. In this case, the complexity calculation unit 31 generates a contrast complexity map, a directional complexity map, and a color complexity map. Then, for each pixel, the complexity calculation unit 31 calculates the average value of that pixel in each map and further averages the values ​​over the entire target region to calculate feature congestion. Note that the higher the spatial complexity, the larger the value of feature congestion.

[0028] The complexity calculation unit 31 converts the target region to be represented in a predetermined color system, such as grayscale or the Lab color system, to generate maps of contrast, directionality, and color complexity. The complexity calculation unit 31 applies one or more downsampling processes to the converted target region, thereby multiplying the resolution of the converted target region. Furthermore, the complexity calculation unit 31 applies filtering processes, such as a Gaussian difference filter or a one-dimensional Gaussian filter with different directions, to each pixel in the resolution-multiplied target region to extract features related to contrast, directionality, or color. Furthermore, the complexity calculation unit 31 generates local maps representing the contrast, directionality, or color complexity at each resolution by calculating, for each pixel, a value representing the degree of variation, such as the covariance of values ​​obtained by filtering processes for that pixel and its neighboring pixels. The complexity calculation unit 31 then applies one or more upsampling processes to the local maps with different resolutions for contrast, directionality, and color, thereby unifying the resolutions of the local maps. Then, the complexity calculation unit 31 generates maps of the complexity of contrast, directionality, and color by finding the maximum value between the local maps for each corresponding pixel for each of contrast, directionality, and color.

[0029] Furthermore, the complexity calculation unit 31 calculates, as an index representing the complexity that changes over time, for example, structural similarity (SSIM) based on a plurality of images obtained in time series from the camera 2. In this case, the complexity calculation unit 31 may calculate the SSIM for each target region according to the following equation.

number

[0030] Alternatively, the complexity calculation unit 31 may calculate an index other than SSIM as an index representing the time-varying complexity. For example, the complexity calculation unit 31 may calculate the sum of squares of the differences in pixel values ​​between corresponding pixels for each pair of two temporally consecutive images included in the most recent predetermined period, and may use a value obtained by averaging the calculated sum of squares of the differences for each pair as an index representing the time-varying complexity. In this case, the value of this index increases as the time-varying complexity increases.

[0031] The complexity calculation unit 31 notifies the setting unit 32 of the calculated complexity. When the complexity calculation unit 31 calculates the complexity for each of two or more target regions, it notifies the setting unit 32 of the complexity for each target region. When the complexity calculation unit 31 calculates two or more types of complexity, it notifies the setting unit 32 of the complexity for each type.

[0032] The setting unit 32 sets an attention-calling condition for issuing a warning notification to prompt the driver to monitor the surroundings of the vehicle 10 in response to the driver's behavior, such as posture, gaze direction, or alertness. In this embodiment, the setting unit 32 relaxes the attention-calling condition so that the higher the complexity, the more likely it is that a warning notification to alert the driver will be issued.

[0033] Specifically, the setting unit 32 widens the range of driver behaviors that are the target of a warning as the complexity increases. For example, the setting unit 32 widens the range of postures and gaze directions that are the target of a warning as the complexity increases. Alternatively, the setting unit 32 increases the upper threshold of the alertness level that is the target of a warning (hereinafter referred to as the alertness threshold) as the complexity increases. Furthermore, the setting unit 32 may decrease the time threshold for the duration of the driver behavior that is the target of a warning as the complexity increases. Furthermore, when the complexity is calculated for each target region, the setting unit 32 may set the warning condition according to the highest complexity level among the calculated complexity levels. Similarly, when multiple types of complexity levels (e.g., spatial complexity level and time-varying complexity level) are calculated, the setting unit 32 may normalize each type of complexity level to fall within a predetermined value range, and then set the warning condition according to the highest complexity level. In this way, by using the maximum value of the complexity of each partial region or each type to set the attention condition, the setting unit 32 can set the attention condition more appropriately.

[0034] A reference table showing the relationship between the complexity and the attention condition may be stored in advance in the memory 22. The setting unit 32 can then set the attention condition corresponding to the calculated complexity by referring to the reference table.

[0035] 4(a) and 4(b) are diagrams each showing an example of the degree of complexity of the situation around the vehicle 10. In both the scene 400 shown in FIG. 4(a) and the scene 410 shown in FIG. 4(b), a person 401 in a wheelchair is about to cross the road in front of the vehicle 10. In the scene 400 shown in FIG. 4(a), the number of people, other vehicles, vegetation, and structures around the vehicle 10 is small, so the spatial complexity is relatively low. Therefore, the driver can easily perceive the person 401. Therefore, the attention alert condition is set to be relatively strict, i.e., relatively difficult to fulfill.

[0036] In contrast, in the scene 410 shown in Fig. 4(b), there are various people, other vehicles, vegetation, and structures around the vehicle 10, resulting in a relatively high degree of spatial complexity. Therefore, it is difficult for the driver to perceive the person 401. Therefore, the attention-calling conditions are set to be relatively lenient, i.e., relatively easy to fulfill.

[0037] The setting unit 32 notifies the determination unit 34 of the set alert condition.

[0038] The detection unit 33 detects driver behavior related to attention-raising for monitoring the surroundings of the vehicle 10. In this embodiment, the detection unit 33 detects at least one of the driver's posture, such as the direction or position of the driver's face, the direction of the driver's line of sight, and the level of alertness.

[0039] The detection unit 33, for example, inputs the driver image into a classifier that has been trained in advance to detect the driver's face from the image, thereby detecting the area in the driver image where the driver's face appears (hereinafter referred to as the facial area) and detecting multiple feature points of the driver's face, such as the outer corners and inner corners of the eyes, the upper eyelids, the lower eyelids, the tip of the nose, and the corners of the mouth. In this case, the detection unit 33 detects the facial area and facial feature points by inputting the driver image into a classifier that has been trained in advance to detect the facial area and facial feature points shown in the image. As such a classifier, the detection unit 33 may use, for example, a DNN with a CNN-type architecture, a support vector machine, or an AdaBoost classifier. The detection unit 33 may also detect the facial area and facial feature points from the driver image using other methods for detecting the facial area and facial feature points, such as template matching.

[0040] The detection unit 33 fits each of the detected facial feature points to a three-dimensional face model that represents the three-dimensional shape of the face. The detection unit 33 then detects the facial orientation of the three-dimensional face model when each feature point best fits the three-dimensional face model as the driver's facial orientation. The detection unit 33 may also detect the driver's facial orientation based on the driver image using other methods for determining the orientation of a face represented in an image. The detection unit 33 also detects, as the position of the driver's face, a position away from the driver monitor camera 3 by a distance corresponding to the position of the driver's seat in an orientation corresponding to the center of gravity of the face area from the driver monitor camera 3.

[0041] Furthermore, in order to detect the driver's level of alertness, the detection unit 33 detects the number and period of blinks, the degree of mouth opening, and the speed of movement in the direction of gaze based on multiple driver images obtained in time series.

[0042] To detect the number and cycle of blinks, the detection unit 33 estimates the degree of eye opening of the driver based on the distance between the upper and lower eyelids of each of the left and right eyes in multiple driver images acquired in time series. For example, the detection unit 33 may determine the average value of the distance between the upper and lower eyelids of each of the left and right eyes as the degree of eye opening. The detection unit 33 may also estimate the degree of eye opening using other methods that calculate the degree of eye opening from the upper and lower eyelids in the images. The detection unit 33 then calculates the time from one maximum value to the next maximum value as the duration of one blink of the driver based on the time-series change in the degree of eye opening in each of the series of driver images. The detection unit 33 then counts the number of blinks over a recent fixed period and calculates the average time between blinks as the blink cycle.

[0043] Furthermore, the detection unit 33 calculates the ratio of the vertical length to the horizontal length of the mouth for each driver image taken within a certain period of time and calculates the average value as the degree of mouth opening of the driver. Note that the detection unit 33 may also calculate the degree of mouth opening of the driver using another method for calculating the degree of mouth opening from the area where the mouth is represented on the image.

[0044] Furthermore, the detection unit 33 detects the driver's gaze direction from each driver image and calculates the movement speed of the gaze direction based on the detected gaze direction. For example, the detection unit 33 detects a corneal reflection image of a light source (hereinafter referred to as a Purkinje image) and a centroid of a pupil (hereinafter simply referred to as a pupil centroid) from an area surrounded by the upper and lower eyelids (hereinafter referred to as an eye area) for at least one of the driver's left and right eyes displayed on the driver image. In this case, the detection unit 33 detects the Purkinje image by template matching between a Purkinje image template and the eye area. Similarly, the detection unit 33 may detect the pupil by template matching between a pupil template and the eye area, and the centroid of the area in which the detected pupil is displayed may be set as the pupil centroid. The detection unit 33 then calculates the distance between the Purkinje image and the pupil centroid and detects the driver's gaze direction by referring to a table that indicates the relationship between the distance and the driver's gaze direction. Note that such a table may be stored in advance in the memory 22. The detection unit 33 then calculates the amount of movement in the gaze direction for each pair of two consecutive driver images within a certain recent period, and calculates the average amount of movement by dividing it by the interval between acquisitions of the driver images to determine the speed of movement in the gaze direction.

[0045] The detection unit 33 determines the level of alertness according to a combination of the number and frequency of blinks, the degree of mouth opening, and the speed of movement in the direction of gaze. For example, the detection unit 33 may detect the level of alertness of the driver by referring to a reference table that indicates the relationship between the combination of these factors and the level of alertness. The level of alertness is set to a larger value as the driver becomes more alert.

[0046] The detection unit 33 notifies the determination unit 34 of the detected driver behavior.

[0047] The determination unit 34 determines whether the driver's behavior detected by the detection unit 33 satisfies the attention condition set by the setting unit 32. Specifically, when the driver's alertness is detected as the driver's behavior, the determination unit 34 determines that the attention condition is satisfied when the period during which the alertness is equal to or less than a predetermined alertness threshold is equal to or greater than a predetermined time threshold. Furthermore, when the driver's facial orientation or gaze direction is detected as the driver's behavior, the determination unit 34 determines that the attention condition is satisfied when the period during which the facial orientation or gaze direction is not directed toward a monitoring target range including the front of the vehicle 10 is equal to or greater than a predetermined time threshold. Furthermore, when the driver's facial position is detected as the driver's behavior, the determination unit 34 determines that the attention condition is satisfied when the period during which the facial position is outside a predetermined allowable range is equal to or greater than a predetermined time threshold. As described above, the higher the complexity, the smaller the time threshold is set. Furthermore, the higher the complexity, the higher the alertness threshold may be set. Similarly, the higher the complexity, the narrower the monitoring range, i.e., the wider the range within the monitoring range that the driver is not looking at. Furthermore, the higher the complexity, the narrower the tolerance range. Therefore, the more complex the situation around the vehicle 10, the more likely it is that the attention-calling condition will be met.

[0048] The determination unit 34 notifies the notification processing unit 35 of the determination result regarding whether the alert condition is satisfied.

[0049] When the determination unit 34 notifies the notification processing unit 35 that the attention condition has been satisfied, the notification processing unit 35 issues a warning to the driver via the notification device 4 to request that the driver monitor the surroundings of the vehicle 10. For example, the notification processing unit 35 causes a speaker of the notification device 4 to emit an audio signal or a warning sound requesting the driver to monitor the surroundings. Alternatively, the notification processing unit 35 causes a display device of the notification device 4 to display a warning message or an icon requesting the driver to monitor the surroundings. Alternatively, the notification processing unit 35 vibrates a vibrator of the notification device 4, or turns on or flashes a light source.

[0050] After notifying the driver of a warning via the notification device 4, the notification processing unit 35 stops the notification of the warning when it receives a determination result from the determination unit 34 that the warning condition is no longer satisfied.

[0051] FIG. 5 is an operational flowchart of the driver monitoring process executed by the processor 23.

[0052] The complexity calculation unit 31 calculates the complexity of the situation around the vehicle 10 based on one or more images generated by the camera 2 (step S101). The setting unit 32 sets the attention calling conditions so that the higher the complexity, the more relaxed the attention calling conditions are (step S102).

[0053] The detection unit 33 detects the driver's behavior based on one or more driver images generated by the driver monitor camera 3 (step S103). Then, the determination unit 34 determines whether the detected driver's behavior satisfies an attention-calling condition (step S104).

[0054] If the attention-calling condition is met (step S104-Yes), the notification processing unit 35 issues an attention-calling warning to the driver via the notification device 4, requesting that the driver monitor the surroundings of the vehicle 10 (step S105).

[0055] After step S105, or if the attention-calling condition is not satisfied in step S104 (step S104-No), the processor 23 ends the driver monitoring process.

[0056] As explained above, this driver monitoring device sets the attention calling conditions according to the complexity of the situation around the vehicle. This allows the driver monitoring device to appropriately call the driver's attention regarding monitoring the vehicle's surroundings according to the situation around the vehicle. Furthermore, since the driver monitoring device does not call the driver's attention simply because the complexity has increased, but only when the attention calling conditions are met, this driver monitoring device can reduce the driver's feeling of annoyance.

[0057] If the attention condition remains satisfied even after a first grace period has elapsed since the start of the notification of the attention warning, the notification processing unit 35 may increase the intensity of the warning. For example, the notification processing unit 35 may increase the volume of the warning emitted from the speaker. Alternatively, the notification processing unit 35 may increase the number of devices issuing the warning. Furthermore, if the attention condition remains satisfied even after a second grace period, which is longer than the first grace period, has elapsed since the start of the notification of the attention warning, the processor 23 may control each unit of the vehicle 10 to slow down or stop the vehicle 10.

[0058] According to a modified example, when the complexity calculation unit 31 calculates the complexity for each of a plurality of partial regions (e.g., the right half and the left half) obtained by dividing an image as a target region, the setting unit 32 may set a corresponding warning condition for each partial region. In this case, the setting unit 32 may relax the warning condition for a partial region whose complexity is equal to or greater than a predetermined threshold compared to the warning condition for a partial region whose complexity is less than the predetermined threshold. Specifically, the setting unit 32 may set a monitoring target range and a time threshold according to the complexity of each partial region. The monitoring target range may be set to include the center of the corresponding partial region and to cover the range of directions from the camera 2 corresponding to the partial region. Hereinafter, the monitoring target range corresponding to a partial region whose complexity is equal to or greater than a predetermined threshold will be referred to as a first monitoring target range, and the monitoring target range corresponding to a partial region whose complexity is less than the predetermined threshold will be referred to as a second monitoring target range. The setting unit 32 may set a first time threshold for a period during which the driver's facial orientation or gaze direction is not included in the first monitoring range shorter than a second time threshold for a period during which the driver's facial orientation or gaze direction is not included in the second monitoring range. Similarly, the setting unit 32 may set the first monitoring range narrower than the second monitoring range. This makes it easier for a partial area with a higher degree of complexity to be alerted when the driver is not facing the direction corresponding to that partial area. Therefore, the setting unit 32 can appropriately set the alert condition depending on the driver's orientation. Note that if the driver's facial orientation or gaze direction has been included in the first monitoring range at least once within the most recent predetermined period, the setting unit 32 may set the first time threshold to be equal to the second time threshold.

[0059] According to another variation, the setting unit 32 may set an attention condition related to the alertness level according to the time-varying complexity, while setting an attention condition related to the driver's gaze direction or posture according to the spatial complexity. For example, when a driver is feeling drowsy, setting an attention condition based on the time-varying complexity rather than the spatial complexity may more appropriately alert the driver. Therefore, the setting unit 32 may set a higher alertness threshold or shorten the time threshold for the duration during which the alertness level remains below the alertness threshold as the time-varying complexity increases. Furthermore, the setting unit 32 may set a narrower monitoring range or shorten the time threshold for the duration during which the gaze direction or facial orientation remains outside the monitoring range as the spatial complexity increases. In this way, by setting different types of attention conditions according to the complexity characteristics, the setting unit 32 can more appropriately set the attention conditions.

[0060] In addition, a computer program that realizes the functions of the processor 23 of the ECU 5 according to the above embodiment or variant may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0061] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0062] 1 Driver Monitor System 2 Cameras 3 Driver monitor camera 4 Notification device 5. Electronic control unit (driver monitoring device) 10 vehicles 21 Communication Interface 22 Memory 23 processors 31 Complexity calculation unit 32 Setting section 33 Detection unit 34 Judgment section 35 Notification processing section

Claims

1. a complexity calculation unit that calculates the complexity of the situation around the vehicle based on at least one image that represents the situation around the vehicle and is generated by an imaging unit mounted on the vehicle; a setting unit that sets the attention calling condition so that the attention calling condition for calling the attention of the driver of the vehicle becomes more relaxed as the complexity level increases; a detection unit that detects the driver's behavior; a notification processing unit that notifies the driver of a warning to call attention via a notification device when the driver's behavior satisfies the attention calling condition; and the complexity calculation unit calculates, as the complexity, a spatial complexity of the situation around the vehicle and a time-varying complexity of the situation around the vehicle; The setting unit sets the attention condition based on the higher of the spatial complexity and the time-varying complexity. Driver monitoring device.

2. calculating a complexity of the situation around the vehicle based on at least one image representing the situation around the vehicle generated by an imaging unit mounted on the vehicle; The attention condition is set so that the attention condition for calling the attention of the driver of the vehicle becomes more relaxed as the complexity level increases; Detecting the driver's behavior; When the behavior of the driver satisfies the attention-calling condition, a warning is issued to the driver via a notification device. This includes: calculating the complexity includes calculating, as the complexity, a spatial complexity of the situation around the vehicle and a time-varying complexity of the situation around the vehicle; setting the attention condition includes setting the attention condition based on a higher one of the spatial complexity and the time-varying complexity. Driver monitoring method.

3. calculating a complexity of the situation around the vehicle based on at least one image representing the situation around the vehicle generated by an imaging unit mounted on the vehicle; The attention condition is set so that the attention condition for calling the attention of the driver of the vehicle becomes more relaxed as the complexity level increases; Detecting the driver's behavior; When the behavior of the driver satisfies the attention-calling condition, a warning is issued to the driver via a notification device. causing a processor mounted on the vehicle to execute the above steps; calculating the complexity includes calculating, as the complexity, a spatial complexity of the situation around the vehicle and a time-varying complexity of the situation around the vehicle; setting the attention condition includes setting the attention condition based on a higher one of the spatial complexity and the time-varying complexity. A computer program for driver monitoring.

Images