Driver monitoring device, driver monitoring method, and program

Abstract

To prevent a driver from calling while holding, in a hand, a mobile phone located outside a shooting range of a driver monitor camera.SOLUTION: A driver monitoring device 20 comprises an acquisition unit 231 for acquiring a driver monitor camera image including a face of a driver of a vehicle 1 captured by a driver monitor camera 3 and an estimation unit 233 for estimating whether or not the driver is calling while holding, in a hand, a mobile phone located outside a shooting range of the driver monitor camera 3 based on the driver monitor camera image acquired by the acquisition unit 231. The estimation unit 233 estimates that the driver is calling while holding, in a hand, a mobile phone located outside the shooting range of the driver monitor camera 3 if a state where the mobile phone and the driver's hand holding the mobile phone are included in the driver monitor camera image is detected, and then a state where the mobile phone and the driver's hand holding the mobile phone are not included in the driver monitor camera image is detected and further distinctive behavior of the driver indicating the continuation of a call is detected.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a driver monitoring device, a driver monitoring method, and a program. [Background technology]

[0002] Patent Document 1 describes that a camera located directly in front of the driver's seat captures an image of the driver, that head posture information, eye state information, mouth state information, etc. are obtained from the driver's image, and that based on this, the driver's distracted state is detected and it is determined whether a distracted action (such as a telephone conversation) has occurred (including determining an action target frame that includes local areas of the face (mouth area, eye area, etc.), hand areas, and action-related objects (cell phone, etc.)). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2021-527900 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 cannot determine whether the driver is talking on a mobile phone that is outside the camera's range (i.e., a mobile phone that is not included in the driver's image), and therefore cannot prevent the driver from talking on a mobile phone that is outside the camera's range.

[0005] In view of the above, the present disclosure aims to provide a driver monitoring device, a driver monitoring method, and a program that can suppress the driver's behavior of holding a mobile phone that is located outside the shooting range of the driver monitor camera and making a call. [Means for solving the problem]

[0006] One aspect of the present disclosure is a driver monitoring device that includes an acquisition unit that acquires a driver monitor camera image including the face of the driver of the vehicle captured by a driver monitor camera, and an estimation unit that estimates whether the driver is talking on a mobile phone that is located outside the capture range of the driver monitor camera based on the driver monitor camera image acquired by the acquisition unit, and the estimation unit estimates that the driver is talking on a mobile phone that is located outside the capture range of the driver monitor camera when a state is detected in which the mobile phone and the driver's hand holding the mobile phone are included in the driver monitor camera image, and then a state is detected in which the mobile phone and the driver's hand holding the mobile phone are no longer included in the driver monitor camera image and a characteristic behavior of the driver indicating the continuation of the call is detected. Effect of the Invention

[0007] According to the present disclosure, it is possible to prevent a driver from making a call by holding a mobile phone that is outside the range of the driver monitor camera. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a vehicle 1 to which a driver monitoring device 20 of a first embodiment is applied. [Diagram 2] 2 is a diagram showing an example of a specific configuration of the driver monitoring device 20 shown in FIG. [Diagram 3] FIG. 2 is a diagram showing an example of the arrangement of the driver monitor camera 3 shown in FIG. [Figure 4] 2 is a diagram for conceptually explaining how to use skeleton detection in the driver monitoring device 20 of the first embodiment. FIG. [Diagram 5] 4 is a flowchart for explaining an example of processing executed by the driver monitoring device 20 of the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of a driver monitoring device, a driver monitoring method, and a program according to the present disclosure will be described with reference to the drawings.

[0010] First Embodiment Fig. 1 is a diagram showing an example of a vehicle 1 to which a driver monitoring device 20 of a first embodiment is applied. Fig. 2 is a diagram showing an example of a specific configuration of the driver monitoring device 20 shown in Fig. 1. Fig. 3 is a diagram showing an example of the arrangement of the driver monitor camera 3 shown in Fig. 1. In the example shown in Figs. 1 to 3, a vehicle 1 includes a vehicle speed sensor 2, a driver monitor camera 3, an HMI (Human Machine Interface) 4, a GPS (Global Positioning System) unit 5, a map information unit 6, a vehicle control device 10, a steering actuator 11, a braking actuator 12, a driving actuator 13, an in-vehicle network 14, and a driver monitoring device 20. As shown in Fig. 3, the driver monitor camera 3 is disposed on an upper portion of a steering column (not shown) of the vehicle 1 so as to be able to capture an image of the face and part of the upper body of the driver of the vehicle 1. In other examples, the driver monitor camera 3 may be disposed in a center cluster of the vehicle 1, or in a room mirror, a meter panel, a meter hood, or the like of the vehicle 1.

[0011] 1 to 3, the driver monitoring device 20 includes a communication interface (I / F) 21, a memory 22, a processor 23, and a signal line 24. The processor 23 has a function as an acquisition unit 231, a function as a detection unit 232, a function as an estimation unit 233, a function as a determination unit 234, and a function as a warning unit 235. The acquisition unit 231 acquires a driver monitor camera image including the face of the driver of the vehicle 1 captured by the driver monitor camera 3. The detection unit 232 has a function of detecting the position of the face of the driver of the vehicle 1 included in the driver monitor camera image by using a known face recognition technology such as face recognition AI (artificial intelligence). The detection unit 232 also has a function of detecting the hands of the driver of the vehicle 1 included in the driver monitor camera image by using a technology similar to a known face part detection technology. Furthermore, the detection unit 232 has a function of detecting an object (e.g., a mobile phone, etc.) included in the driver monitor camera image by using a known object detection (object detection) technology. That is, the detection unit 232 has a function to detect a state in which the mobile phone and the hand of the driver of the vehicle 1 holding the mobile phone are included in the driver monitor camera image (that is, a state in which the hand of the driver of the vehicle 1 holding the mobile phone is near the face of the driver of the vehicle 1). The detection unit 232 also has a function to detect a state in which the mobile phone and the hand of the driver of the vehicle 1 holding the mobile phone are no longer included in the driver monitor camera image (that is, a state in which the mobile phone or the hand of the driver of the vehicle 1 is no longer near the face of the driver of the vehicle 1).

[0012] For example, the People's Republic of China national standard GB / T 41797-2022 requires that a warning be issued to the driver if the driver holds a smartphone 5 cm from the face for more than three seconds. However, it is not expected that the driver will immediately stop talking even if the driver is warned, and it is possible that the driver will continue the call by putting the phone on speakerphone. For example, in order to perform detection in accordance with the People's Republic of China national standard GB / T 41797-2022, the driver's face, hands near the face, the smartphone itself, etc. will be detected by driver monitoring. However, if the smartphone is held near the horn pad while on speakerphone, the driver's hands and smartphone will be out of the field of view of the driver monitor camera (outside the range of the driver monitor camera), making it difficult to detect the driver's hands and smartphone. If the driver continues talking on the smartphone by exploiting the weaknesses of driver monitoring, this will further encourage unsafe behavior, so it is necessary to detect this state.

[0013] In view of the above, in the example shown in Figures 1 to 3, the detection unit 232 has a function of detecting a characteristic action of the driver of the vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the shooting range of the driver monitor camera 3 (the "characteristic action of the driver of the vehicle 1 indicating the continuation of a call" will be explained in detail later). Specifically, the detection unit 232 detects a characteristic behavior of the driver of the vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera 3 based on the driver monitor camera image acquired by the acquisition unit 231 and the result of the skeleton detection (for example, skeleton detection using a technology described on the website indicated by the URL below) (for example, the direction of the upper arm, forearm, etc. of the driver who is the subject of skeleton detection and is located outside the imaging range of the driver monitor camera), and a model obtained by learning using teacher data, which is a data set of labels that are included in the driver monitor camera image and indicate whether the driver who is the subject of skeleton detection has performed a characteristic behavior that indicates the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera. https: / / www.autoconnectedcar.com / 2019 / 03 / eyesight-tech-knows-whos-in-the-car /

[0014] The technology described on the website indicated by the above URL can detect the direction of the driver's upper arm (only a part of which is included in the image) in the image on the homepage of the website. In addition, this skeletal detection can detect the direction of the passenger's forearm (up to the elbow is included in the image) in the passenger's seat.

[0015] In another example, the detection unit 232 may detect a characteristic behavior of the driver of the vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera 3, based on the driver monitor camera image acquired by the acquisition unit 231 and the results of skeletal detection (for example, the position, angle, etc. of the upper arm, forearm, etc. of the driver of the vehicle 1 that is located outside the imaging range of the driver monitor camera) without using the above-mentioned trained model.

[0016] In the example shown in Figures 1 to 3, characteristic actions of the driver of vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the shooting range of the driver monitor camera 3 include mouth movements of the driver of vehicle 1 indicating the continuation of the call. That is, the detection unit 232 has a function of detecting the mouth movement of the driver of the vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera 3. Specifically, the detection unit 232 detects the mouth movement of the driver of the vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera 3, based on the driver monitor camera image acquired by the acquisition unit 231, by using a model obtained by performing learning using teacher data that is a data set of, for example, a driver monitor camera image and a label indicating whether or not the driver included in the driver monitor camera image has made a mouth movement indicating the continuation of a call using a mobile phone that has been moved outside the imaging range of the driver monitor camera.

[0017] In addition, in the example shown in Figures 1 to 3, characteristic actions of the driver of vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the shooting range of the driver monitor camera 3 include the action of the driver of vehicle 1 moving the hand of the driver holding the mobile phone to a position other than the position where the steering wheel ST is gripped (a position outside the shooting range of the driver monitor camera 3). That is, the detection unit 232 has a function of detecting the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped, which indicates the continuation of a call using the mobile phone that has been moved outside the shooting range of the driver monitor camera 3. Specifically, the detection unit 232 detects the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped, by using the above-mentioned skeleton detection.

[0018] Fig. 4 is a diagram for conceptually explaining how to use skeleton detection in the driver monitoring device 20 of the first embodiment. In detail, Fig. 4(A) shows a state in which the driver of the vehicle 1 grips the steering wheel ST with both hands before holding the mobile phone in his / her hand (before starting a call with the mobile phone in his / her hand). Fig. 4(B) shows a state in which the driver of the vehicle 1 holds the mobile phone in his / her hand to his / her ear while driving and talks on the phone. Fig. 4(C) shows a state in which the driver of the vehicle 1 moves the hand holding the mobile phone to a position other than the position where the hand is gripping the steering wheel ST and continues the call. In one example of the driver monitoring device 20 of the first embodiment, a driver monitor camera image of the driver of the vehicle 1 gripping the steering wheel ST with both hands before holding a mobile phone in his / her hand (before holding the mobile phone in his / her hand and starting a call) as shown in FIG. 4(A) is stored in advance in the memory 22. In addition, in one example of the driver monitoring device 20 of the first embodiment, when the driver of the vehicle 1 transitions from a state in which the driver holds the mobile phone to his / her ear while driving (state shown in FIG. 4(B)) to a state in which the driver of the vehicle 1 moves the hand holding the mobile phone to a position other than the position where the steering wheel ST is gripped and continues the call (state shown in FIG. 4(C)), the mobile phone held by the driver of the vehicle 1 is located outside the shooting range of the driver monitor camera 3. The detection unit 232 detects, for example, the direction of the upper arm and forearm of the driver of the vehicle 1 by using skeletal detection (i.e., detects that the position and angle of the upper arm and forearm of the driver of the vehicle 1 have changed, and detects that the driver of the vehicle 1 has moved the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped (characteristic action of the driver of the vehicle 1).

[0019] In the example shown in Figures 1 to 3, the estimation unit 233 estimates whether the driver of the vehicle 1 is talking (continuing a call) on a mobile phone that is located outside the shooting range of the driver monitor camera 3 by comparing, for example, the results of skeleton detection in the state shown in Figure 4(A) with the results of skeleton detection in the state shown in Figure 4(C) based on the driver monitor camera image acquired by the acquisition unit 231. 1 to 3, the estimation unit 233 not only uses skeleton detection technology, but also uses a model obtained by learning using teacher data, which is a data set of driver monitor camera images in the state shown in FIG. 4(A) and driver monitor camera images in the state shown in FIG. 4(C), and labels indicating whether the driver is holding a mobile phone located outside the shooting range of the driver monitor camera in his / her hand and continuing a call when the state transitions from the state shown in FIG. 4(A) to the state shown in FIG. 4(C), to estimate whether the driver of the vehicle 1 is holding a mobile phone located outside the shooting range of the driver monitor camera 3 in his / her hand (continuing a call) based on the driver monitor camera image acquired by the acquisition unit 231. In another example, the estimation unit 233 may estimate whether the driver of the vehicle 1 is talking (continuing a call) on a mobile phone that is located outside the shooting range of the driver monitor camera 3 by comparing, for example, the results of skeleton detection in the state shown in Figure 4(A) with the results of skeleton detection in the state shown in Figure 4(C) without using the learned model described above.

[0020] Furthermore, in the example shown in Figures 1 to 3, characteristic behaviors of the driver of vehicle 1 indicating the continuation of a call using a mobile phone that has been moved outside the shooting range of the driver monitor camera 3 include the driver of vehicle 1 glancing at the mobile phone he or she is holding in his or her hand. That is, the detection unit 232 has a function of detecting the behavior of the driver of the vehicle 1 glancing at the mobile phone held in his / her hand, which indicates the continuation of a call using the mobile phone that has been moved out of the shooting range of the driver monitor camera 3. Specifically, the detection unit 232 detects the behavior of the driver of the vehicle 1 glancing at the mobile phone held in his / her hand based on the driver monitor camera image acquired by the acquisition unit 231, by using a model obtained by performing learning using teacher data, which is a data set of, for example, the driver monitor camera image and a label indicating whether the line of sight and / or the face direction of the driver included in the driver monitor camera image alternately turns to the front of the vehicle and a specific position other than the front of the vehicle in a short period.

[0021] 1 to 3, the estimation unit 233 has a function of estimating whether or not the driver of the vehicle 1 is talking on the phone. In detail, the estimation unit 233 has a function of estimating whether or not the driver of the vehicle 1 is talking on the phone with the hand of the driver of the vehicle 1 holding the mobile phone being placed near the face of the driver of the vehicle 1. Specifically, when detection unit 232 detects that a mobile phone and the hand of the driver of vehicle 1 holding the mobile phone are included in the driver monitor camera image (i.e., the hand of the driver of vehicle 1 holding the mobile phone is near the face of the driver of vehicle 1), estimation unit 233 estimates that the driver of vehicle 1 is talking with the hand of the driver of vehicle 1 holding the mobile phone near the face of the driver of vehicle 1. For example, if the detection unit 232 detects that the hands of the driver of vehicle 1 are not included in the driver monitor camera image (for example, the hand of the driver of vehicle 1 holding a mobile phone is not near the face of the driver of vehicle 1), and a certain amount of time has not passed since the alarm unit 235 issued the first alarm described below, the estimation unit 233 estimates that there is a possibility that the driver of vehicle 1 is talking (possibly continuing a call) on a mobile phone that is located outside the shooting range of the driver monitor camera 3. For example, if the detection unit 232 detects that the hands of the driver of vehicle 1 are not included in the driver monitor camera image when counter t = 0 and counter n = 0 described below, it is estimated that the driver of vehicle 1 is not holding a mobile phone and not making a call.

[0022] Furthermore, the estimation unit 233 has a function of estimating whether the driver of the vehicle 1 is talking on a mobile phone that is located outside the shooting range of the driver monitor camera 3 and is held in the hand. Specifically, when detection unit 232 detects a state in which the mobile phone and the hand of the driver of vehicle 1 holding the mobile phone are included in the driver monitor camera image, and then detection unit 232 detects a state in which the mobile phone and the hand of the driver of vehicle 1 holding the mobile phone are no longer included in the driver monitor camera image, and detection unit 232 detects characteristic actions of the driver of vehicle 1 indicating the continuation of the call (in detail, all of the following actions indicating the continuation of the call: the mouth movement of the driver of vehicle 1, the action of the driver of vehicle 1 moving the hand of the driver of vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped, and the action of the driver of vehicle 1 glancing at the mobile phone held in the hand), estimation unit 233 estimates that the driver of vehicle 1 is talking on a mobile phone located outside the shooting range of the driver monitor camera 3 (continuing the call). Furthermore, the estimation unit 233 has a function of estimating whether or not the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position for gripping the steering wheel ST is detected by the detection unit 232 by using the driver monitor camera image and skeleton detection. When the detection unit 232 detects the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position for gripping the steering wheel ST, the estimation unit 233 estimates (determines) that the driver of the vehicle 1 has moved the hand holding the mobile phone to a position other than the position for gripping the steering wheel ST.

[0023] 1 to 3, the determination unit 234 executes various determinations. The warning unit 235 has a function of issuing a first warning and a function of issuing a second warning in a manner different from the first warning (for example, a stronger warning than the first warning). That is, the warning unit 235 issues a warning in a manner different from that when the detection unit 232 detects a state in which the mobile phone and the hand of the driver of the vehicle 1 holding the mobile phone are included in the driver monitor camera image, and when the detection unit 232 detects a state in which the mobile phone and the hand of the driver of the vehicle 1 holding the mobile phone are no longer included in the driver monitor camera image and a characteristic action of the driver of the vehicle 1 indicating the continuation of the call is detected.

[0024] Fig. 5 is a flowchart for explaining an example of a process executed by the driver monitoring device 20 of the first embodiment. The process shown in Fig. 5 is executed, for example, when the vehicle speed sensor 2 detects a traveling state of the vehicle 1 (for example, a vehicle speed of 10 km / h or more). In the example shown in FIG. 5, in step S10, the acquisition unit 231 acquires a driver monitor camera image. In step S11, the detection unit 232 detects the position of the face of the driver of the vehicle 1. In step S12, the detection unit 232 detects the hand of the driver of the vehicle 1 and the object (mobile phone). In step S13, the estimation unit 233 estimates whether the driver of the vehicle 1 is talking. If the driver of the vehicle 1 is talking (the hand of the driver of the vehicle 1 holding the mobile phone is near the face of the driver of the vehicle 1), the process proceeds to step S14. If the driver of the vehicle 1 is not talking on the mobile phone (for example, the hand of the driver of the vehicle 1 is not included in the driver monitor camera image when the counter n=0), the process proceeds to step S19. If the hand of the driver of the vehicle 1 is not included in the driver monitor camera image and a certain time has not passed since the alarm unit 235 issued the first alarm, the process proceeds to S17.

[0025] In step S14, for example, the warning unit 235 increments a counter t. In step S15, the determination unit 234 determines whether the counter t is greater than the threshold value Th1. If t>Th1, the process proceeds to step S16. If t≦Th1, the process returns to step S10. In step S16, the warning unit 235 determines that the driver of the vehicle 1 is on a call (the hand of the driver of the vehicle 1 holding the mobile phone is near the face of the driver of the vehicle 1), and issues a first warning, for example, for a few seconds (outputs the first warning to the HMI 4). Next, the process proceeds to step S17. In step S17, the estimation unit 233 estimates whether the driver of the vehicle 1 is talking (continuing a call) while holding a mobile phone located outside the shooting range of the driver monitor camera 3. If the estimation unit 233 estimates that the driver of the vehicle 1 is talking while holding a mobile phone located outside the shooting range of the driver monitor camera 3, the process proceeds to step S18. If the estimation unit 233 does not estimate that the driver of the vehicle 1 is talking while holding a mobile phone located outside the shooting range of the driver monitor camera 3, the process proceeds to step S20. In step S18, the warning unit 235 issues a second warning that is stronger than the first warning (outputs the second warning to the HMI 4), for example.

[0026] In step S19, for example, the alarm unit 235 resets a counter t. In step S20, for example, the alarm unit 235 increments a counter n. In step S21, the determination unit 234 determines whether the counter n is greater than a threshold value Th2. If n>Th2, the process proceeds to step S22. If n≦Th2, the process returns to step S10. In step S22, for example, the alarm unit 235 resets the counter t and the counter n.

[0027] As described above, in the driver monitoring device 20 of the first embodiment, after a first alarm is issued in response to the driver of vehicle 1 talking on a mobile phone held in his / her hand, the driver of vehicle 1 does not give up the call in response to the first alarm, but continues the call by camouflaging the call as if he / she has stopped, by issuing a second alarm.

[0028] <Second embodiment> The vehicle 1 to which the driver monitoring device 20 of the second embodiment is applied is configured similarly to the vehicle 1 to which the driver monitoring device 20 of the first embodiment described above is applied, except for the points described below. As described above, in the driver monitoring device 20 of the first embodiment, when the detection unit 232 detects all of the following characteristic actions of the driver of the vehicle 1 indicating the continuation of a call: the movement of the driver's mouth indicating the continuation of a call, the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped, and the action of the driver of the vehicle 1 glancing at the mobile phone held in the hand, it is estimated that the driver of the vehicle 1 is talking on a mobile phone located outside the shooting range of the driver monitor camera 3 (continuing a call). On the other hand, in the driver monitoring device 20 of the second embodiment, when the detection unit 232 detects any one or two of the following characteristic actions of the driver of the vehicle 1 indicating the continuation of a call: the movement of the driver's mouth indicating the continuation of a call, the action of the driver of the vehicle 1 moving the hand of the driver of the vehicle 1 holding the mobile phone to a position other than the position where the steering wheel ST is gripped, and the action of the driver of the vehicle 1 glancing at the mobile phone he is holding in his hand, it is presumed that the driver of the vehicle 1 is talking (continuing a call) while holding a mobile phone that is outside the shooting range of the driver monitor camera 3 in his hand.

[0029] As described above, the embodiments of the driver monitoring device, the driver monitoring method, and the program of the present disclosure have been described with reference to the drawings, but the aggressive driving sign estimating device, the vehicle, the aggressive driving sign estimating method, and the program of the present disclosure are not limited to the above-mentioned embodiments, and may be modified as appropriate without departing from the spirit of the present disclosure. The configurations of the examples of the above-mentioned embodiments may be combined as appropriate. [Explanation of symbols]

[0030] 1...vehicle, 2...vehicle speed sensor, 3...driver monitor camera, 20...driver monitoring device, 23...processor, 231...acquisition unit, 232...detection unit, 233...estimation unit, 234...determination unit, 235...alarm unit

Claims

1. An acquisition unit that acquires a driver monitor camera image including a face of a driver of the vehicle captured by a driver monitor camera; and an estimation unit that estimates whether or not a driver is talking on a mobile phone that is located outside the imaging range of the driver monitor camera based on the driver monitor camera image acquired by the acquisition unit, The estimation unit is When a state in which the mobile phone and the driver's hand holding the mobile phone are included in the driver monitor camera image is detected, and then a state in which the mobile phone and the driver's hand holding the mobile phone are no longer included in the driver monitor camera image is detected and a characteristic behavior of the driver indicating the continuation of the call is detected, The driver monitoring device estimates that the driver is talking on a mobile phone that is located outside the imaging range of the driver monitor camera.

2. The characteristic behavior of the driver indicating the continuation of the call includes at least one of the following: the driver's mouth movement indicating the continuation of the call, the driver's hand holding the mobile phone moving to a position other than the position where the steering wheel is gripped, and the driver's glance at the mobile phone held in the hand; 2. The driver monitoring device according to claim 1, wherein the estimation unit has a function of estimating whether or not an action of moving the driver's hand holding a mobile phone to a position other than a position where the driver's hand is gripping the steering wheel has been detected by using the driver monitor camera image and skeletal detection.

3. 3. The driver monitoring device according to claim 1 or 2, further comprising an alarm unit that issues an alarm in a different manner when a state in which a mobile phone and the driver's hand holding the mobile phone are detected to be included in the driver monitor camera image and when a state in which the mobile phone and the driver's hand holding the mobile phone are no longer included in the driver monitor camera image and a characteristic behavior of the driver indicating the continuation of the call is detected.

4. An acquisition step in which the driver monitoring device acquires a driver monitor camera image including a face of the driver of the vehicle captured by the driver monitor camera; and an estimation step of estimating whether or not the driver is talking on a mobile phone that is located outside the image capturing range of the driver monitor camera, based on the driver monitor camera image acquired in the acquisition step, In the estimation step, When a state in which the mobile phone and the driver's hand holding the mobile phone are included in the driver monitor camera image is detected, and then a state in which the mobile phone and the driver's hand holding the mobile phone are no longer included in the driver monitor camera image is detected and a characteristic behavior of the driver indicating the continuation of the call is detected, This driver monitoring method estimates that the driver is talking on a mobile phone that is located outside the imaging range of the driver monitor camera.

5. The processor: An acquisition step of acquiring a driver monitor camera image including a face of a driver of the vehicle captured by a driver monitor camera; and an estimation step of estimating whether or not the driver is holding a mobile phone located outside the imaging range of the driver monitor camera and talking on the mobile phone based on the driver monitor camera image acquired in the acquisition step, In the estimation step, When a state in which the mobile phone and the driver's hand holding the mobile phone are included in the driver monitor camera image is detected, and then a state in which the mobile phone and the driver's hand holding the mobile phone are no longer included in the driver monitor camera image is detected and a characteristic behavior of the driver indicating the continuation of the call is detected, The program presumes that the driver is talking on a mobile phone that is located outside the range of the driver monitor camera.

Images