Driver monitoring device and monitoring program
A dual-camera system with controlled imaging timing effectively monitors driver conditions, reducing computational load and power consumption by using a narrow-angle camera for continuous face monitoring and a wide-angle camera only when needed, thus addressing the challenges of high-resolution image processing in in-vehicle systems.
Patent Information
- Application Number
- JP2022202280
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-12-19
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2042-12-19
AI Technical Summary
In-vehicle systems face challenges in accurately monitoring driver conditions such as drowsy or distracted driving due to the high computational load required for processing wide-area high-resolution images, leading to increased power consumption and resource utilization, which hinders the addition of new functions.
A dual-camera system with a wide-angle and narrow-angle camera configuration, controlled by a main control unit, where the narrow-angle camera continuously monitors the driver's face and switches to the wide-angle camera only when necessary for detailed observations, reducing processing load and power consumption.
This approach allows efficient monitoring of driver conditions while minimizing processing load, power consumption, and heat generation, enabling accurate detection of distracted driving and other unsafe behaviors without overburdening the system resources.
Smart Images

Figure 0007583009000001 
Figure 0007583009000002 
Figure 0007583009000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a driver monitoring device and a monitoring program. [Background technology]
[0002] In recent years, for example, due to amendments to the Road Traffic Act, new obligations are imposed on vehicle drivers and new devices that must be installed in vehicles are increasing. For example, since the driver is required to drive safely during automatic driving of a vehicle, it becomes increasingly necessary for the system on the vehicle to correctly grasp the driving situation of the driver to support the driver's safe driving and to correctly record the actual situation of safe driving.
[0003] For example, Patent Document 1 discloses a driver monitoring device that can detect dangerous driving factors outside the imaging range of a camera. Specifically, it shows that it detects distracted driving by judging the direction of the driver's face from the luminance distribution of the face, and detects distracted driving, drowsy driving, etc. from the position and movement of the pupils. It also shows that it detects a smartphone or the like reflected in the pupils of the face or glasses (spectacles). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2021-43637 Summary of the Invention [Problem to be solved by the invention]
[0005] When an in-vehicle system monitors the driver's condition while driving, it is required to have a basic function of detecting drowsy driving, distracted driving, abnormal posture, etc. Therefore, the in-vehicle system needs to grasp detailed information of the driver's eyes and face. In addition, the in-vehicle system needs to have a function to detect dangerous driving situations, such as when the driver is distracted while driving while operating a smartphone.
[0006] However, in order to realize the above-mentioned function, regardless of the actual driving situation, a wide area including the body of the driver sitting in the driver's seat must be constantly photographed with a high-resolution camera from a location where blind spots are unlikely to occur, and the photographed images must be constantly processed to monitor the condition in detail. This increases the load on the processing device that processes the image data, etc., and there are concerns about increased power consumption and heat generation.
[0007] In addition, the load on the processing device will increase, resulting in a situation in which most of the processing device's resources are constantly consumed, making it impossible to use the processing device's resources for any purpose other than monitoring the driver. This makes it difficult to make design changes to add new functions to existing processing devices.
[0008] The present invention has been made in consideration of the above-mentioned circumstances, and its object is to provide a driver monitoring device and a monitoring program that enable appropriate monitoring of the driver in various situations and reduce the load on a processing device that processes data such as images. [Means for solving the problem]
[0009] The above object of the present invention can be achieved by the following configuration.
[0010] A first imaging unit capable of imaging a driver of a vehicle and its surroundings; a second imaging unit having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the face of the driver; A control unit that controls imaging timing of the first imaging unit and the second imaging unit; A determination unit that determines whether or not the driver is performing at least an action unrelated to driving operation from images acquired from the first imaging unit and the second imaging unit; and when the determination unit detects that the driver is looking aside based on the image input from the second imaging unit while the input from the first imaging unit is stopped, the determination unit controls the control unit to start acquiring an image from the first imaging unit and determines whether or not the image acquired from the first imaging unit includes a detection target or a driver's posture that is to be detected. Driver monitoring device.
[0011] A monitoring program executable by a computer of a monitoring device including a first imaging unit capable of imaging a driver of a vehicle and the surroundings thereof, a second imaging unit having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the face of the driver, a control unit controlling imaging timings of the first imaging unit and the second imaging unit, and a determination unit determining whether or not the driver is performing at least an action unrelated to driving operation from images acquired from the first imaging unit and the second imaging unit, a step of detecting whether the driver is looking aside based on a second image input from the second imaging unit; a step of starting acquisition of a first image from the first imaging unit when it is detected based on the second image that the driver is looking aside while input from the first imaging unit is stopped, and determining whether or not the first image includes a detection target or a driver's posture that is to be detected; A monitoring program having Effect of the Invention
[0012] According to the driver monitoring device and monitoring program of the present invention, it is possible to appropriately monitor the driver in various situations and to reduce the load on a processing device that processes data such as images.
[0013] The present invention has been briefly described above. Furthermore, the details of the present invention will be further clarified by reading the following description of the preferred embodiment of the present invention (hereinafter, referred to as "embodiment") with reference to the accompanying drawings. [Brief description of the drawings]
[0014] [Figure 1] FIG. 1 is a block diagram showing an outline of the configuration of a driver monitoring device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a front view showing examples of two types of images captured by two cameras, respectively. [Diagram 3] FIG. 3 is a block diagram showing a configuration of main functions provided in the driver monitoring device of FIG. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the main control unit. [Diagram 5] FIG. 5 is a time chart showing Example 1 of the operation timing in the driver monitoring device of FIG. [Figure 6] FIG. 6 is a time chart showing Example 2 of the operation timing in the driver monitoring device of FIG. [Figure 7] FIG. 7 is a time chart showing Example 3 of the operation timing in the driver monitoring device of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Specific embodiments of the present invention will be described below with reference to the accompanying drawings.
[0016] FIG. 1 is a block diagram showing an outline of the configuration of a driver monitoring device 100 according to an embodiment of the present invention. The driver monitoring device 100 shown in FIG. 1 includes a first camera (camera 1) 10A, a second camera (camera 2) 10B, and a main control unit 20.
[0017] The installation position and angle of view of first camera 10A are adjusted in advance so that it can capture an image in shooting range A1 in Fig. 1. That is, first camera 10A can capture a relatively wide range of images, such as the face, upper body, arms, and hands, of driver 50 seated in the driver's seat.
[0018] For example, the first camera 10A is installed in any one of an overhead module, a room mirror, a center part of an instrument panel, a pillar, a meter hood, an inside of a meter, a steering column, and the like in the passenger compartment of the vehicle. The photographing range A1 includes an area in which a steering wheel 51, a seat belt 52, and other in-vehicle accessories of the vehicle are present.
[0019] The installation position and angle of view of second camera 10B are adjusted in advance so that it can capture an image in shooting range A2 in Fig. 1. That is, second camera 10B can capture only an image within a relatively narrow range compared to first camera 10A so as to capture detailed information of the facial parts of driver 50 seated in the driver's seat.
[0020] The installation position of second camera 10B may be the same as that of first camera 10A, but it is preferable to place second camera 10B in a position where it can capture an image from the front side of driver 50 so as to more easily obtain information such as the line of sight.
[0021] The main control unit 20 is an electronic control unit (ECU) having a function of monitoring the driver's situation based on images captured by the first camera 10A and the second camera 10B. The main control unit 20 can control the timing of capturing images, the timing of acquiring image data, the amount of exposure, gain, and the like for each of the first camera 10A and the second camera 10B. The main control unit 20 processes the image data acquired from at least one of the first camera 10A and the second camera 10B to acquire information representing the driving situation, automatically records and saves the resulting information, and outputs an alarm or the like according to the situation to support safe driving.
[0022] <Example of captured image> FIG. 2 shows specific examples of two types of captured images i1 and i2 captured by the first camera 10A and the second camera 10B, respectively.
[0023] In the captured image i1 shown in Fig. 2, sufficient information is shown to grasp the driving situation of the driver 50, such as the upper body, both hands, and both arms. In the captured image i2 shown in Fig. 2, information including the entire face and head of the driver 50 is shown in an enlarged state.
[0024] Therefore, for example, by analyzing the data of an image such as the captured image i2 by the main control unit 20, it becomes possible to detect the face direction and the line of sight of the driver 50 and to monitor changes therein. Also, by analyzing the data of an image such as the captured image i1 by the main control unit 20, it becomes possible to detect the state in which the driver 50 is holding the steering wheel and to detect a mobile terminal (such as a smartphone) that the driver 50 may be holding in his / her hand.
[0025] <Major functional configuration> FIG. 3 is a block diagram showing the configuration of main functions provided in the driver monitoring device 100 of FIG.
[0026] Each function of the main control unit 20 shown in FIG. 3 is realized, for example, by the operation of hardware of an electronic circuit mainly composed of a microcomputer (not shown) incorporated in the main control unit 20 and a program executed by this microcomputer.
[0027] The main control unit 20 includes feature detection functions 21 and 22, a timing control function 23, a monitoring function unit 24, an alarm function 25, and a history information recording function 26. The monitoring function unit 24 also includes an inattentive driving detection function 24a, a drowsiness detection function 24b, a poor posture detection function 24c, an action detection function 24d, a posture detection function 24e, a seat belt detection function 24f, and a steering wheel holding detection function 24g.
[0028] Feature detection function 21 detects various features of each part of the upper body and the hand area of driver 50 by image processing the image data acquired from first camera 10A. Feature detection function 22 detects various features of the face area of driver 50 by image processing the image data acquired from second camera 10B.
[0029] Based on the detection states of feature detection functions 21 and 22, timing control function 23 controls the timing at which first camera 10A and second camera 10B each take pictures (including control of illumination light), the timing at which image data taken by first camera 10A and second camera 10B each is sent to main control unit 20 (or the timing at which main control unit 20 requests images), and so on.
[0030] The monitoring function unit 24 detects the current state of each of the various monitoring items required in the vehicle. The inattentiveness detection function 24a has a function of detecting whether the driver 50 is driving while facing a direction unrelated to driving, based on face information of the driver 50.
[0031] The drowsiness detection function 24b has a function of detecting whether the driver 50 is driving with his / her eyes closed or in a drowsy state, based on information on the face of the driver 50. The posture error detection function 24c has a function of detecting whether the driver 50 is driving in an abnormal posture different from a normal driving posture, based on information on the upper body of the driver 50 and the like.
[0032] The behavior detection function 24d has a function of detecting whether the driver 50 is performing a dangerous driving behavior, such as operating a smartphone, based on information about the upper body of the driver 50 and the like.
[0033] The posture detection function 24e has a function of detecting information such as the posture of the entire body of the driver 50, the horizontal and front-to-back orientation of the face, the direction of the line of sight, and the inclination of the upper body. The seat belt detection function 24f has a function of detecting whether or not the driver 50 is wearing a seat belt.
[0034] The steering wheel holding detection function 24g has a function of detecting whether or not the driver 50 is holding the steering wheel in a state capable of driving. The warning function 25 has a function of supporting the driver in safe driving by notifying the driver of an abnormality, for example by outputting a voice or an alarm sound, when any function of the monitoring function unit 24 detects a situation requiring a warning regarding the driving condition of the driver 50.
[0035] The history information recording function 26 records and stores information detected by each function of the monitoring function unit 24 regarding the driving state of the driver 50 in a specified non-volatile recording medium as history information, for example, in association with the current date and time and the current location of the vehicle.
[0036] <Main control unit operation> An example of the operation of the main control unit 20 is shown in Fig. 4. That is, when the driver monitoring system (DMS: Driver Monitoring System) of the vehicle is started, the computer of the main control unit 20 starts the operation shown in Fig. 4. The operation shown in Fig. 4 will be described below.
[0037] As an initialization process for each of the first camera 10A and the second camera 10B, the main control unit 20 automatically adjusts the exposure of each camera and the gain of the image processing circuit to appropriate values, and records information indicating the state (S11). In the initial state, the lights of each camera are turned off.
[0038] In this embodiment, the main control unit 20 normally processes only the image captured by the second camera 10B to monitor the driver 50. Therefore, first, only the LED lighting built into or attached to the second camera 10B starts to be turned on (S12). In addition, the brightness of the image captured by the second camera 10B is detected, and the amount of brightness fluctuation that occurs over a certain period of time is recorded. The emission wavelength of the LED lighting is in the near infrared range so that the camera can capture images even at night.
[0039] The main control unit 20 periodically acquires image data captured by the second camera 10B and processes the image data using the feature detection function 22, thereby repeatedly and periodically executing the process of detecting the face position of the driver 50 (S13).
[0040] The main control unit 20 compares the length of time during which the face position of the driver 50 can be detected with a predetermined threshold time (S14). If the length of time during which the face position can be detected is equal to or longer than the threshold time, the process proceeds to S16. If the condition of S14 is not satisfied, the process proceeds to S17, and the main control unit 20 outputs information indicating that the face of the driver 50 cannot be detected.
[0041] The main control unit 20 compares both the face direction and line of sight direction of the driver 50 with a specific range determined in advance (S16). This specific range corresponds to the range of face direction and line of sight direction assumed when the driver 50 gazes at the screen of a mobile terminal such as a smartphone, when operating the mobile terminal with his / her hand, or when performing an action such as a call on the mobile terminal. If the condition of S16 is satisfied, proceed to S24.
[0042] If the condition of S16 is not satisfied, the main control unit 20 determines whether the vehicle is stopped in S18. If the vehicle is stopped, the process proceeds to S19, where it determines whether both the face direction and line of sight of the driver 50 continue to be stationary for a predetermined threshold time or more. If the vehicle is not stopped, the process proceeds to S20, where it determines whether both the face direction and line of sight of the driver 50 continue to be outside the predetermined range for a predetermined threshold time or more.
[0043] The main control unit 20 proceeds to S21 if the vehicle is stopped and the condition of S19 is satisfied, and proceeds to S13 if the condition of S19 is not satisfied. Also, if the vehicle is not stopped and the condition of S20 is satisfied, the main control unit 20 proceeds to S21, and if the condition of S19 is not satisfied, the main control unit 20 proceeds to S13.
[0044] The main control unit 20 outputs an inattentive driver warning in S21. Furthermore, the presence or absence of a predetermined warning cancellation signal is identified in S22, and if the warning cancellation signal is detected, the warning in S21 is cancelled in the next S23 before proceeding to S13. If the warning cancellation signal is not detected, the process proceeds to S18. For example, the inattentive warning cancellation signal can be generated by the driver operating a predetermined button, etc.
[0045] On the other hand, if the condition of S16 is satisfied, the main control unit 20 starts turning on the LED lighting built into or attached to the first camera 10A in S24 so that the first camera 10A can start shooting. In addition, the main control unit 20 reflects the brightness fluctuation amount of the second camera 10B detected in S12 in the exposure amount and gain setting values of the first camera 10A.
[0046] The main control unit 20 periodically acquires image data obtained by the first camera 10A and processes it using the feature detection function 21, and periodically repeats the detection process of a mobile device such as a smartphone that the driver 50 may be holding in his or her hand (S25).
[0047] For example, if the position of the driver's 50's hand is detected and tracked in the image to be processed, and a contour shape equivalent to that of a smartphone is detected within a rectangular area nearby, it can be deemed that a smartphone has been detected. Alternatively, if the driver 50 has a hand on the side of the driver's 50's face, such as when the driver 50 is talking on the phone, and a contour shape equivalent to that of a smartphone is detected within a rectangular area nearby, it can be deemed that a smartphone has been detected.
[0048] When a smartphone is detected based on the image of the first camera 10A, the main control unit 20 issues a warning about distracted driving regarding smartphone operation in S27. Furthermore, in S29, it is determined whether the vehicle is stopped, and if not, the process returns to S25, and if the vehicle is stopped, the warning about distracted driving is cancelled (S30).
[0049] On the other hand, if no smartphone is detected in S26, the main control unit 20 proceeds to S28. Then, it is determined whether or not both the face direction and the line of sight direction of the driver 50 are kept constant, and if they are constant, the process returns to S25, and if there is a change, the process proceeds to S18.
[0050] When main control unit 20 is executing the processes of S24 to S30 in FIG. 4, the image data captured by second camera 10B is not needed, so the content of the process may be changed so as to temporarily turn off the illumination of second camera 10B and stop the capturing operation of second camera 10B and the transmission of image data.
[0051] <Device operation timing> <Example-1> Fig. 5 shows an example 1 of the operation timing in the driver monitoring device 100 of Fig. 1. The operation shown in Fig. 5 will be described below.
[0052] In the operation shown in FIG. 5, the driver monitoring device 100 normally uses only the second camera 10B to capture images at a fixed cycle, i.e., at each time t1, t2, t3, t4, ... in FIG. 5, and processes the images each time to recognize the driving state of the driver 50.
[0053] 5, if the state of looking away from the vehicle in the designated range continues for a certain period of time or more (the range of times t6 to t9), the main control unit 20 assumes that there is a possibility of distracted driving, and starts processing to detect the smartphone. That is, the first camera 10A starts capturing images at time t10A, and captures images at fixed intervals of time, such as at times t11A, t12A, ...
[0054] Then, based on the images captured by the first camera 10A at each time t10A, t11A, t12A, ..., the main control unit 20 extracts a rectangular area near the hand, like that of the driver 50B in Figure 5, and detects whether or not a smartphone is present within this area (corresponding to S25 and S26 in Figure 4).
[0055] In the operation shown in FIG. 5, the times t10, t11, t12, ... when second camera 10B takes pictures and the times t11A, t12A, ... when first camera 10A take pictures are asynchronous and shifted so as not to overlap each other.
[0056] <Example-2> Fig. 6 shows an example 2 of the operation timing in the driver monitoring device 100 of Fig. 1. The operation shown in Fig. 6 will be described below.
[0057] In the operation shown in Figure 6, the driver monitoring device 100 normally uses only the second camera 10B to capture images at a fixed cycle, i.e., repeatedly at times t1, t2, t3, ... in Figure 6, but there are predetermined times (times t5, t10, ...) when the second camera 10B does not capture images.
[0058] The main control unit 20 processes the images captured by the second camera 10B at each time t1, t2, t3, . . . to recognize the driving state of the driver 50. Then, when the state where the driver is looking away in the specified range continues for a certain period of time or more (the range from time t6 to t9) like driver 50A shown in Fig. 6, main control unit 20 assumes that there is a possibility of distracted driving and performs processing to detect a smartphone. That is, first camera 10A takes an image at time t10, and based on the image taken by first camera 10A, main control unit 20 extracts a rectangular area near the hand like driver 50B in Fig. 6 and detects whether or not a smartphone is present within this area (corresponding to S25 and S26 in Fig. 4).
[0059] <Example-3> Fig. 7 shows an example 3 of operation timing in the driver monitoring device 100 of Fig. 1. The operation shown in Fig. 7 will be described below.
[0060] In the operation shown in Figure 7, the driver monitoring device 100 normally uses only the second camera 10B to capture images at a fixed cycle, i.e., repeatedly at times t1, t2, t3, ... in Figure 7, but there are predetermined times (times t5, t10, ...) when the second camera 10B does not capture images.
[0061] The main control unit 20 processes the images captured by the second camera 10B at each time t1, t2, t3, . . . to recognize the driving state of the driver 50. 7, if the state of looking away from the vehicle in the designated range continues for a certain period of time or more (the range of times t6 to t9), the main control unit 20 assumes that there is a possibility of distracted driving, and starts processing to detect the smartphone. That is, the first camera 10A starts taking pictures at time t10, and then repeats taking pictures at each of the predetermined times t10, t11, t12, ... at regular intervals.
[0062] In addition, in order to prevent the timing of shooting by first camera 10A and second camera 10B from overlapping, shooting by second camera 10B is stopped at timings (t11, t12, ...) after first camera 10A starts shooting.
[0063] Based on the images captured by the first camera 10A at each time t10, t11, t12, ..., the main control unit 20 extracts a rectangular area near the hand, as in the case of the driver 50B in Figure 7, and detects whether or not a smartphone is present within this area (corresponding to S25 and S26 in Figure 4).
[0064] As described above, in the driver monitoring device 100 of this embodiment, the main control unit 20 processes only the images of the second camera 10B at times t1 to t9 in Figs. 5 to 7 during normal operation, so that the processing load on the main control unit 20 can be reduced. In particular, since the second camera 10B has a narrow shooting range, the face direction and the line of sight direction can be sufficiently detected even in images with low resolution, which is effective in reducing the processing load. Furthermore, when the driver 50 is detected looking away from the road, the image captured over a wide range using the first camera 10A is processed, so that it becomes easy to detect a smartphone or the like held by the driver 50, and the situation of distracted driving can be correctly detected. Furthermore, as shown in Figs. 5 to 7, the shooting and processing of the second camera 10B are stopped at the timing of processing the images of the first camera 10A, so that the processing load on the main control unit 20 can be reduced.
[0065] The present invention is not limited to the above-described embodiment, and can be appropriately modified, improved, etc. In addition, the material, shape, size, number, arrangement location, etc. of each component in the above-described embodiment are arbitrary as long as the present invention can be achieved, and are not limited.
[0066] Here, the features of the driver monitoring device and the monitoring program according to the above-mentioned embodiment of the present invention will be briefly summarized and listed in the following [1] to [5]. [1] A first imaging unit (first camera 10A) capable of imaging a driver of a vehicle and its surroundings; A second imaging unit (second camera 10B) having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the driver's face; A control unit (main control unit 20, timing control function 23) that controls the imaging timing of the first imaging unit and the second imaging unit; A determination unit (main control unit 20, action detection function 24d) that determines whether or not the driver is performing an action unrelated to driving operation from images acquired from the first imaging unit and the second imaging unit; and when the determination unit detects that the driver is looking aside based on the image input from the second imaging unit while the input from the first imaging unit is stopped, the determination unit controls the control unit to start acquiring an image from the first imaging unit (S24, S25), and determines whether or not the image acquired from the first imaging unit includes a detection target or a driver's posture that is to be detected (S26). A driver monitoring device (100).
[0067] According to the driver monitoring device having the configuration of [1] above, it is possible to easily detect situations such as distracted driving by a driver who drives a vehicle. That is, by processing the image of the second imaging unit having a narrow imaging range, it is possible to easily detect the situation of inattentive driving. Also, by processing the image of the first imaging unit in the situation of inattentive driving, it is possible to detect the smartphone operated by the driver and the posture of the driver who is gazing at the smartphone screen. That is, it is possible to identify distracted driving such as operating a smartphone in the situation of inattentive driving. Moreover, it is possible to significantly reduce the load on the processing by appropriately adjusting the imaging timing and processing timing of the first imaging unit and the second imaging unit. This makes it possible to suppress the increase in heat generation and power consumption of the control device, and also suppress the resource consumption of the control device.
[0068] [2] The determination unit detects a mobile terminal present in an area near the driver based on the image captured by the first imaging unit (S24 to S26). A driver monitoring device as described in [1] above.
[0069] According to the driver monitoring device having the configuration [2] above, the mobile terminal is detected based on the image captured by the first imaging unit, which captures a relatively wide range, so that distracted driving can be easily detected even when the driver is holding and operating a mobile terminal that is far away from the driver's face.
[0070] [3] The determination unit detects at least one of a face direction and a line of sight direction of the driver based on the image captured by the second imaging unit (S12 to S16). A driver monitoring device according to the above [1] or [2].
[0071] According to the driver monitoring device having the configuration of [3] above, the direction of the driver's face or the direction of his / her gaze is detected, making it possible to detect situations in which the driver is looking away from the road or at the screen of a mobile device, etc.
[0072] [4] The determination unit detects at least one of the driver's face direction and line of sight direction based on the image captured by the second imaging unit (S12 to S16), and determines that the driver is looking away if the detected face direction or line of sight direction is within a range of the looking away condition for a certain period of time or more (S16, S24). A driver monitoring device as described in [1] above.
[0073] According to the driver monitoring device having the configuration of [4] above, the direction of the driver's face or gaze is detected, and the length of time that the same state continues is identified, making it possible to more accurately detect situations in which the driver is looking away from the road or staring at the screen of a mobile device, etc.
[0074] [5] A monitoring program executable by a computer (main control unit 20) of a monitoring device including a first imaging unit capable of imaging a driver of a vehicle and his / her surroundings, a second imaging unit having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the face of the driver, a control unit that controls imaging timing of the first imaging unit and the second imaging unit, and a determination unit that determines whether the driver is performing an action unrelated to driving operation from images acquired from the first imaging unit and the second imaging unit, A step (S12 to S16) of detecting whether the driver is looking aside based on the second image input from the second imaging unit; a step (S24 to S26) of starting acquisition of a first image from the first imaging unit when it is detected based on the second image that the driver is looking aside while input from the first imaging unit is stopped, and determining whether or not the first image includes a detection target or a driver's posture that is to be detected; A monitoring program having
[0075] By executing the monitoring program of the procedure [5] above on a specified computer, it is possible to easily detect situations such as distracted driving by a driver who drives a vehicle. That is, by processing the image of the second imaging unit with a narrow imaging range, it is possible to easily detect situations of inattentive driving. In addition, by processing the image of the first imaging unit in a situation of inattentive driving, it is possible to detect the smartphone operated by the driver and the posture of the driver gazing at the smartphone screen. In other words, it is possible to identify distracted driving such as operating a smartphone in a situation of inattentive driving. Moreover, it is possible to significantly reduce the load on processing by appropriately adjusting the imaging timing and processing timing of the first imaging unit and the second imaging unit. This makes it possible to suppress the increase in heat generation and power consumption of the control device, and also suppress the resource consumption of the control device. [Explanation of symbols]
[0076] 10A 1st Camera 10B Second Camera 20 Main control section 21,22 Feature detection function 23 Timing control function 24 Monitoring function section 24a Inattentive detection function 24b Drowsiness detection function 24c Posture correction detection function 24d Activity Detection Function 24e Attitude detection function 24f Seat belt detection function 24g Handle holding detection function 25 Alarm function 26 History information recording function 50, 50A, 50B Driver 51 Handle 52 Seatbelt 100 Driver monitoring device A1, A2 shooting range i1, i2 captured images
Claims
1. A first imaging unit capable of imaging a driver of a vehicle and its surroundings; a second imaging unit having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the face of the driver; A control unit that controls imaging timing of the first imaging unit and the second imaging unit; a determination unit that determines whether or not the driver is performing an action unrelated to driving operation based on images acquired from the first imaging unit and the second imaging unit; and when the determination unit detects that the driver is looking aside based on the image input from the second imaging unit in a state where the input from the first imaging unit is stopped, the determination unit controls the control unit to start acquiring an image from the first imaging unit and determines whether or not the image acquired from the first imaging unit includes a detection target or a driver's posture that is to be detected. Driver monitoring device.
2. The determination unit detects a mobile terminal present in an area near the driver based on the image captured by the first imaging unit. A driver monitoring device according to claim 1.
3. The determination unit detects at least one of a face direction and a line of sight direction of the driver based on the image captured by the second imaging unit. A driver monitoring device according to claim 1.
4. The determination unit detects at least one of a face direction and a line of sight direction of the driver based on the image captured by the second imaging unit, and determines that the driver is looking inattentively when the detected face direction or line of sight direction is within a range of a condition for looking inattentive for a certain period of time or more. A driver monitoring device according to claim 1.
5. A monitoring program executable by a computer of a monitoring device including a first imaging unit capable of imaging a driver of a vehicle and the surroundings thereof, a second imaging unit having a narrower imaging range than the first imaging unit and capable of imaging features appearing on the face of the driver, a control unit controlling imaging timings of the first imaging unit and the second imaging unit, and a determination unit determining whether or not the driver is performing an action unrelated to driving operation from images acquired from the first imaging unit and the second imaging unit, a step of detecting whether the driver is looking aside based on a second image input from the second imaging unit; a step of starting acquisition of a first image from the first imaging unit when it is detected based on the second image that the driver is looking aside while input from the first imaging unit is stopped, and determining whether or not the first image includes a detection target or a driver's posture that is to be detected; A monitoring program having
Citation Information
Patent Citations
Driving support device and driving support method
JP2014225099A
Drive shift control device and drive shift control method
JP2018103670A
Driver monitoring device
JP2021043637A
Mobile device usage monitoring for commercial vehicle fleet management
US20200327345A1
