Driver's face recognition device and face recognition program

Abstract

This facial authentication device (7) for a driver comprises: an image acquisition unit (21) that acquires a facial image of a driver; a first software execution unit (22) that executes, in a predetermined cycle, unsuitable driving state detection software for determining whether a driver is in a suitable driving state or an unsuitable driving state; a second software execution unit (23) that executes facial authentication software for determining substitution of the driver; a substitutability state determination unit (24) for determining whether it is possible to substitute the driver; and a software execution control unit (25) that, when it is determined by the determination result of the substitutability state determination unit that it is possible to substitute the driver, causes the facial authentication software to be executed.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2021-105740 filed on June 25, 2021, the contents of which are incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a driver's face recognition device and a face recognition program. [Background technology]

[0003] For example, there is provided an occupant condition monitoring system for monitoring the condition of an occupant in a vehicle such as an automobile. The driver condition monitoring system for monitoring the condition of the driver includes an imaging unit for imaging the periphery of the headrest of the driver's seat, and an illumination unit for irradiating the periphery of the headrest of the driver's seat with light. The driver condition monitoring system uses the imaging unit to image the periphery of the driver's face illuminated with light while the driver is seated in the driver's seat, and analyzes the facial image of the driver included in the captured image to monitor the driver's condition (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6372388 Summary of the Invention

[0005] The driver status monitoring system judges, for example, whether the driver is looking away, closing the eyes, drowsiness, etc. from the facial image, and judges whether the driver is in a suitable driving state or an unsuitable driving state based on the judgment result. When the driver status monitoring system determines that the driver is in an unsuitable driving state, it activates, for example, a driver abnormality response system (hereinafter referred to as EDSS (Emergency Driving Stop System)) or disables driving by automatic driving or driving by advanced driving assistance system.

[0006] The driver status monitoring system periodically (e.g., 30 times / second) executes software for detecting unsuitable driving conditions to determine whether the driver is in a suitable driving condition or an unsuitable driving condition. The driver status monitoring system is also expected to periodically execute software for face recognition to determine driver change. However, in a configuration in which the software for detecting unsuitable driving conditions and the software for face recognition are separately executed periodically, the frequency with which both pieces of software are executed increases, the amount of calculation increases steadily, and a microcomputer with higher specifications is required.

[0007] An object of the present disclosure is to appropriately determine whether a driver is in an appropriate driving state or an unsuitable driving state, while avoiding an increase in the steady amount of calculations.

[0008] According to one aspect of the present disclosure, the image acquisition unit acquires a facial image of the driver. The first software execution unit executes software for detecting an unsuitable driving state at a predetermined cycle to determine whether the driver is in a suitable driving state or an unsuitable driving state. The second software execution unit executes software for facial recognition to determine a driver change. The change possibility situation determination unit determines whether a driver change is possible. When it is determined that a driver change is possible based on the determination result of the change possibility situation determination unit, the software execution control unit executes the facial recognition software. The first software execution unit executes one execution of the software for detecting an unsuitable driving state for an execution time shorter than the time of the predetermined cycle. The software execution control unit executes the face recognition software while the software for detecting unsuitable driving conditions is not executing, and if a request to execute the software for detecting unsuitable driving conditions occurs while the face recognition software is executing, it continues the execution of the face recognition software and cancels the execution of the next software for detecting unsuitable driving conditions.

[0009] The face recognition software is not executed periodically, but is executed on the condition that it is determined that the driver can be changed. By executing the software for detecting unsuitable driving conditions at a predetermined interval, it is possible to appropriately determine whether the driver is in a suitable driving state or an unsuitable driving state, and by executing the software for detecting unsuitable driving conditions on the condition that the driver can be changed, it is possible to reduce the frequency of executing the software for detecting unsuitable driving conditions. This makes it possible to appropriately determine whether the driver is in a suitable driving state or an unsuitable driving state while avoiding an increase in the amount of constant calculation. In other words, by not executing the software for detecting unsuitable driving conditions when there is no possibility of changing the driver, it is possible to avoid an increase in the amount of constant calculation. [Brief description of the drawings]

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a functional block diagram of a driver state monitoring system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing the relationship between the behavior of the sensor and the behavior of the driver; [Diagram 3] FIG. 3 is a diagram showing the relationship between the behavior of the sensor and the behavior of the driver; [Figure 4] FIG. 4 is a timing chart. [Diagram 5] FIG. 5 is a timing chart. [Figure 6] FIG. 6 is a timing chart. [Figure 7] FIG. 7 is a timing chart; [Figure 8] FIG. 8 is a timing chart. [Figure 9] FIG. 9 is a flow chart; [Figure 10] FIG. 10 is a diagram showing a notification screen when a face image is stored. [Figure 11]FIG. 11 is a diagram showing a notification screen when a face image is transmitted. [Figure 12] FIG. 12 is a diagram showing a notification screen when a driver is changed. [Figure 13] FIG. 13 is a diagram showing a notification screen when a face image is saved after a driver change; [Figure 14] FIG. 14 is a diagram showing a notification screen when a face image is transmitted after a driver change. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment will be described with reference to the drawings. A vehicle is equipped with a driver status monitoring system including a Driver Status Monitor (registered trademark) (hereinafter referred to as DSM (Driver Status Monitor)) as an occupant status monitoring system that monitors the status of the occupant. The driver status monitoring system judges whether the driver is looking away, closing the eyes, drowsiness, etc. based on the degree of drooping of the eyelids, the degree of pupil opening, the direction of gaze, the speed of gaze movement, etc. from a facial image, and judges whether the driver is in a suitable driving state or an unsuitable driving state based on the judgment result. When the driver status monitoring system determines that the driver is in an unsuitable driving state, it activates, for example, an EDSS or disables driving by automatic driving or driving by an advanced driving assistance system.

[0012] 1, the driver state monitoring system 1 includes a control device 2, a driver state recognition device 3, a vehicle information recognition device 4, a driving environment recognition device 5, and an HMI (Human Machine Interface) 6. A driver face recognition device 7 is configured as a part of the driver state monitoring system 1, and includes the control device 2 and the driver state recognition device 3.

[0013] The driver state recognition device 3 includes a driver camera 31, an LED 32, a buckle sensor 33, a door opening / closing sensor 34, a seating sensor 35, an electronic key interior detection sensor 36, a steering wheel touch sensor 37, and a steering wheel torque sensor 38. The driver camera 31 is a camera having an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and is disposed on the upper surface of the instrument panel, the meter panel, the rearview mirror, the steering column, or the like. The driver camera 31 captures an image of the area around the headrest of the driver's seat, and outputs the captured image to the control device 2. There may be a plurality of driver cameras 31, and the plurality of driver cameras 31 may be configured to perform the image capturing operation in synchronization.

[0014] The LED 32 emits near-infrared light around the headrest of the driver's seat. That is, when the driver is seated in the driver's seat, the near-infrared light emitted from the LED 32 is irradiated around the driver's face, and the area around the driver's face irradiated with the near-infrared light is captured by the driver camera 31. The number of the LEDs 32 may be more than one, and a configuration in which the multiple LEDs 32 perform illumination operations in synchronization may also be used.

[0015] The buckle sensor 33 detects whether the convex portion on the seat belt is inserted into the concave portion on the buckle, and outputs the detection result to the control device 2. The door opening / closing sensor 34 detects whether the door is open or closed, and outputs the detection result to the control device 2. The seating sensor 35 detects the pressure distribution on the driver's seat, and outputs the detected pressure distribution to the control device 2. The electronic key interior detection sensor 36 detects whether an electronic key is present in the vehicle interior, and outputs the detection result to the control device 2. The steering wheel touch sensor 37 detects the gripping state of the steering wheel, and outputs the detection result to the control device 2. The steering wheel torque sensor 38 detects the steering force of the steering wheel, and outputs the detection result to the control device 2.

[0016] The vehicle information recognition device 4 includes a vehicle speed sensor 41, a steering angle sensor 42, an accelerator sensor 43, a brake sensor 44, and a shift sensor 45. The vehicle speed sensor 41 detects the vehicle speed and outputs the detected vehicle speed to the control device 2. The steering angle sensor 42 detects the steering angle of the steering wheel and outputs the detected steering angle to the control device 2. The accelerator sensor 43 detects the amount of operation of the accelerator pedal and outputs the detected amount of operation to the control device 2. The brake sensor 44 detects the amount of operation of the brake pedal and outputs the detected amount of operation to the control device 2. The shift sensor 45 detects the state of the shift position and outputs the detected state of the shift position to the control device 2.

[0017] The driving environment recognition device 5 includes a front camera 51, a rear camera 52, a front sensor 53, a rear sensor 54, a navigation device 55, and a G sensor 56. The front camera 51 captures an image in front of the vehicle, including markings painted on the road surface, and outputs the captured image to the control device 2. The rear camera 52 captures an image in the rear of the vehicle and outputs the captured image to the control device 2. The front sensor 53 is a sensor such as a millimeter wave sensor, radar, or LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), and detects objects such as a leading vehicle or pedestrian in front of the vehicle and outputs the detection result to the control device 2. The rear sensor 54 is a sensor such as a millimeter wave sensor, radar, or LiDAR, and detects objects such as a following vehicle or pedestrian behind the vehicle and outputs the detection result to the control device 2. The driving environment recognition device 5 calculates the relative speed with respect to the forward vehicle based on the distance to the forward vehicle detected by the forward sensor 53, and calculates the relative speed with respect to the rear vehicle based on the distance to the following vehicle detected by the rear sensor 54.

[0018] The navigation device 55 performs navigation processing such as determining the current vehicle position by measuring GPS position coordinates based on a GPS signal transmitted from a GPS (Global Positioning System) satellite, calculating a route from the determined current vehicle position to a destination, and guiding along the calculated route. Note that the satellite positioning system is not limited to GPS, and various GNSS (Global Navigation Satellite Systems) such as GLONASS, Galileo, BeiDou, and IRNSS can be adopted. The G sensor 56 detects three-dimensional acceleration in the forward / rearward, left / right, and up / down directions of the vehicle, and outputs the detection results to the control device 2. The G sensor 56 may be a sensor provided in the navigation device 55, or a sensor installed for other purposes.

[0019] The HMI 6 includes a display 61, a speaker 62, and a cancel switch 63. The display 61 has a flat display such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and a touch panel is formed on the front surface of the flat display. When the display 61 detects a touch operation on the touch panel by the driver or passenger, it performs screen control such as switching the screen or displaying an icon on the screen in response to the touch operation.

[0020] As one of the pieces of information to be displayed, the display 61 displays a status indicating the degree of the driver's posture deterioration in five stages. The lower the status number, the lower the degree of deterioration, the driver's posture is normal, and the driver is in an appropriate driving state suitable for driving, while the higher the status number, the higher the degree of deterioration, the driver's posture is abnormal, and the driver is in an unsuitable driving state suitable for driving. The control device 2 outputs a driver status signal indicating the driver's status to the display 61, thereby causing the status to be displayed on the display 61. The driver can grasp his or her own driving posture by viewing the status displayed on the display 61.

[0021] The speaker 62 is a speaker shared with the navigation device 55, an audio device, etc. The control device 2 outputs a driver state signal indicating the state of the driver to the speaker 62, thereby causing the speaker 62 to output the status as audio. The driver can grasp his / her driving posture by listening to the status being output as audio from the speaker 62.

[0022] The cancel switch 63 is a switch that temporarily stops detection of the driver's condition. When the cancel switch 63 is operated before a trip, detection of the driver's condition is stopped for the period of one trip immediately following the operation. In addition, when the cancel switch 63 is operated during a trip, detection of the driver's condition is stopped for the period during which the cancel switch 63 is operated or for a certain period of time (about several seconds) after the cancel switch 63 is operated. In other words, for example, by operating the cancel switch 63 in advance when the driver is about to pick up an object, the driver will not be erroneously detected as being in an unfit state for driving even if his / her posture becomes distorted.

[0023] The control device 2 is composed of a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory) and I / O (Input / Output). The microcomputer executes a computer program stored in a non-transient tangible storage medium, thereby performing processing corresponding to the computer program and controlling the overall operation of the driver state monitoring system 1. The microcomputer has the same meaning as a processor. In the driver state monitoring system 1, the non-transient tangible storage medium may share hardware with other computer resources.

[0024] The control device 2 is capable of data communication with the above-mentioned driver state recognition device 3, vehicle information recognition device 4, driving environment recognition device 5, and HMI 6 via a wired connection such as CAN (Controller Area Network) (registered trademark) or a wireless connection such as wireless LAN or Bluetooth (registered trademark).

[0025] The control device 2 includes an image acquisition unit 21, a first software execution unit 22, a second software execution unit 23, a replacement availability status determination unit 24, a software execution control unit 25, a face image storage unit 26, and a face image transmission unit 27. Each of these units 21 to 27 corresponds to a function executed by a face detection program. That is, the control device 2 performs the functions of each of the units 21 to 27 by executing the face detection program.

[0026] The image acquisition unit 21 inputs an image captured by the driver camera 31 from the driver camera 31 and acquires an image of the driver's face captured by the driver camera 31 .

[0027] The first software execution unit 22 executes software for detecting an unsuitable driving state at a predetermined cycle to determine whether the driver is in a suitable driving state or an unsuitable driving state. That is, the first software execution unit 22 executes the software for detecting an unsuitable driving state at a predetermined cycle, thereby enabling determination of inattentiveness, closed eyes, drowsiness, etc. based on the degree of drooping of the eyelids, the degree of pupil opening, the direction of gaze, the speed of gaze movement, etc. from the face image of the driver, thereby enabling determination of whether the driver is in a suitable driving state or an unsuitable driving state.

[0028] The second software execution unit 23 executes face recognition software for determining a driver change. The second software execution unit 23 registers the face image of the driver to be compared when the face recognition software is executed in the following patterns. As a first pattern, the second software execution unit 23 registers the face image of the current driver when the ignition is switched from off to on and the face recognition software for determining a driver change is executed for the first time. When the second software execution unit 23 executes the face recognition software after registering the face image of the current driver, it compares the registered face image of the driver with the face image of the driver when it is determined that the situation is such that a driver change is possible, and determines whether or not they are the same person. If it is determined that they are the same person, the second software execution unit 23 continues to use the registered face image of the driver as it is for the next determination, and if it is determined that they are not the same person, it discards the registered face image of the driver and newly registers the face image of the driver when it is determined that the situation is such that a driver change is possible.

[0029] As a second pattern, after the ignition is switched from off to on, when the second software execution unit 23 receives the face image of the driver transmitted from the server, the second software execution unit 23 registers the received face image of the driver. When the second software execution unit 23 executes the face authentication software after registering the face image of the driver received from the server, the second software execution unit 23 also compares the registered face image of the driver with the face image of the driver when it is determined that the situation allows driver change, and determines whether or not they are the same person. If it is determined that they are the same person, the second software execution unit 23 continues to use the registered face image of the driver as it is for the next determination, and if it is determined that they are not the same person, it discards the registered face image of the driver and newly registers the face image of the driver when it is determined that the situation allows driver change.

[0030] As a third pattern, if there is a facial image of a driver registered in a past ride and the elapsed time from the registration date and time to the current date and time is less than a predetermined period, the second software execution unit 23 continues to use the facial image of the registered driver. When the second software execution unit 23 executes the facial recognition software thereafter, it also compares the registered facial image of the driver with the facial image of the driver when it is determined that the situation allows for a driver change, and determines whether or not they are the same person. If it is determined that they are the same person, the second software execution unit 23 continues to use the registered facial image of the driver for the next determination, and if it is determined that they are not the same person, it discards the registered facial image of the driver and newly registers the facial image of the driver when it is determined that the situation allows for a driver change.

[0031] The change possibility situation determination unit 24 determines whether or not a driver change is possible. The change possibility situation determination unit 24 periodically acquires CAN signals from various sensors, identifies the behavior of the various sensors based on the acquired CAN signals, and determines the driver behavior that may cause a driver change from the behavior of the various sensors as shown in Figs. 2 and 3, and determines whether or not a driver change is possible. The various sensors for determining the driver behavior that may cause a driver change include a vehicle speed sensor that detects the vehicle speed, an accelerator sensor that detects the state of the accelerator pedal, a brake sensor that detects the state of the brake pedal, a buckle sensor that detects the state of the seat belt, a door opening / closing sensor that detects the open / closed state of the door, a seating sensor that detects the seating state of the driver, an electronic key interior detection sensor that detects the state in which the electronic key is present in the vehicle interior, a shift sensor that detects the state of the shift position, and the like, and determines the driver behavior that may cause a driver change based on one detection result or a combination of several detection results. In addition to these sensors, for example, a sensor that detects the operation of the parking brake, etc. may be used. For example, if the vehicle speed sensor is not referenced, the vehicle speed sensor may detect 0 km / h (indicating the vehicle is stopped) or 50 km / h (indicating the vehicle is moving). In other words, if the circumstantial evidence flag is established based on the behavior of the various sensors, the second software execution unit 23 executes the face recognition software regardless of whether the vehicle is moving or stopped.

[0032] When the change possibility situation determination unit 24 determines that the situation allows a driver change, it sets a circumstantial evidence flag, and when it determines that the situation does not allow a driver change, it does not set the circumstantial evidence flag. Specifically, as shown in Figures 2 and 3, the change possibility situation determination unit 24 determines that the situation allows a driver change and sets a circumstantial evidence flag in the following cases (a) to (f). Note that Figures 2 and 3 are examples of several cases in which driver behavior that may cause a driver change is determined from the behavior of various sensors, and the present invention is not limited to these.

[0033] (a) When a change from a "door closed state to a door open state," a "change from the seat sensor on state to the seat sensor off state," a "change from the seat sensor off state to the seat sensor on state," and a "change from the door closed state to the door open state" are detected in sequence. (b) When the door remains closed (constraint condition), a change from the seat sensor ON state to the seat sensor OFF state and a change from the seat sensor OFF state to the seat sensor ON state are detected in sequence.

[0034] (c) If the vehicle is not equipped with a seat sensor, and the door is kept closed and the buckle sensor is kept off (restraint condition), the vehicle sequentially detects a "change from shift position P to shift position D" and then a "no change from the shift position D after the above." (d) When the vehicle is not equipped with a seat sensor and detects the following in sequence: "a change from the buckle sensor on state to the buckle sensor off state," "a change from the door closed state to the door open state," "a change from the door open state to the door closed state," and "a change from the buckle sensor off state to the buckle sensor on state."

[0035] (e) When the vehicle is not equipped with a seat sensor and the door is kept closed (restraint condition), a "change from the buckle sensor on state to the buckle sensor off state" and a "change from the buckle sensor off state to the buckle sensor on state" are detected in sequence. (f) If the vehicle is not equipped with a seat sensor or a buckle sensor, and the following are detected in sequence: "a change from a door closed state to a door open state," "a change from a state in which the electronic key is inside the vehicle to a state in which it is outside the vehicle," "a change from a state in which the electronic key is outside the vehicle to a state in which it is inside the vehicle," and "a change from a state in which the door is open to a state in which the door is closed."

[0036] On the other hand, even if the change possibility situation determination unit 24 detects, for example, a "change from a door closed state to a door open state" and a "change from a door open state to a door closed state" in succession while the seat occupancy sensor is maintained in the on state, it determines that the situation is not suitable for a driver change and does not set the situational evidence flag.

[0037] The software execution control unit 25 causes the first software execution unit 22 to execute the software for detecting an inappropriate driving state at a predetermined cycle. Moreover, the software execution control unit 25 causes the second software execution unit 23 to execute the software for face recognition not at a predetermined cycle, but on the condition that the result of the determination by the change possibility status determination unit 24 indicates that a driver change is possible.

[0038] In this case, the software execution control unit 25 adjusts the execution of the software for detecting unsuitable driving conditions and the software for face recognition in the following pattern so that the software for detecting unsuitable driving conditions is executed while the software for detecting unsuitable driving conditions is not executed. As shown in Fig. 4, the software execution control unit 25 executes the software for detecting unsuitable driving conditions at a cycle of 33 [msec], and executes the software for detecting unsuitable driving conditions once every 28 [msec].

[0039] (1) If the circumstantial evidence flag is established while the software for detecting unsuitable driving conditions is not running As shown in FIG. 5, when the software execution control unit 25 determines that the circumstantial evidence flag is set while the software for detecting inappropriate driving conditions is not being executed, it starts the execution of the software for face recognition immediately after determining that the circumstantial evidence flag is set.

[0040] (2) If a circumstantial evidence flag is established while the software for detecting unsuitable driving conditions is running. As shown in Figure 6, when the software execution control unit 25 determines that the circumstantial evidence flag is established while the software for detecting inappropriate driving conditions is being executed, it starts the execution of the software for detecting face recognition after terminating the execution of the software for detecting inappropriate driving conditions.

[0041] (3) When a request to run software for detecting inappropriate driving conditions occurs while the facial recognition software is running (case 1) 7, when the software execution control unit 25 determines that a request to execute the software for detecting unsuitable driving conditions has occurred while the software for detecting unsuitable driving conditions is being executed, the software execution control unit 25 suspends the execution of the software for detecting unsuitable driving conditions and starts the execution of the software for detecting unsuitable driving conditions. After terminating the execution of the software for detecting unsuitable driving conditions, the software execution control unit 25 resumes the execution of the software for detecting unsuitable driving conditions.

[0042] (4) When a request to run software for detecting inappropriate driving occurs while the facial recognition software is running (Part 2) As shown in Figure 8, when the software execution control unit 25 determines that a request has been made to execute software for detecting inappropriate driving conditions while the face recognition software is being executed, it continues the execution of the face recognition software and cancels the execution of the next software for detecting inappropriate driving conditions.

[0043] The facial image storage unit 26 stores the facial image of the driver when the circumstantial evidence flag is established by the switching possibility situation determination unit 24. The facial image transmission unit 27 transmits the facial image of the driver to the server when the circumstantial evidence flag is established by the switching possibility situation determination unit 24.

[0044] Next, the operation of the above-mentioned configuration will be described with reference to FIGS. The control device 2 starts the driver's state monitoring process when a start condition for the driver's state monitoring process is satisfied, for example, when the ignition is switched from off to on. The start condition for the driver's state monitoring process is not limited to, for example, when the ignition is switched from off to on, but may also be when the driver performs a predetermined operation. When the control device 2 starts the driver's state monitoring process, it periodically acquires CAN signals from various sensors (S1), determines the driver's behavior from the behavior of the various sensors, and determines whether or not the situation allows for driver switching (S2, which corresponds to a switching possibility situation determination procedure).

[0045] When the control device 2 determines that the driver change is not possible (S2: NO), it does not set the circumstantial evidence flag and determines whether the end condition of the driver state monitoring process is satisfied (S3). When the control device 2 determines that the end condition of the driver state monitoring process is satisfied, for example, because the ignition is switched from on to off (S3: YES), it ends the driver state monitoring process. The end condition of the driver state monitoring process is not limited to, for example, the ignition being switched from on to off, but may also be the driver performing a predetermined operation. When the control device 2 determines that the end condition of the driver state monitoring process is not satisfied, for example, because the ignition is still on, it returns to the above-mentioned step S1 and repeats step S1 and subsequent steps.

[0046] On the other hand, when the control device 2 determines that the driver can be changed (S2: YES), it sets a circumstantial evidence flag (S4), saves the driver's face image (S5), and notifies the driver that the face image has been saved (S6). As shown in Fig. 10, the control device 2 displays a notification screen when the face image is saved on the display 61, and outputs a notification sound when the face image is saved from the speaker 62. The driver can know that the face image has been saved by the notification screen when the face image is saved being displayed on the display 61 and the notification sound when the face image is saved being output from the speaker 62.

[0047] The control device 2 causes the driver's facial image to be transmitted to the server (S7), and notifies the driver that the facial image has been transmitted to the server (S8). As shown in Fig. 11, the control device 2 causes a notification screen for when the facial image is transmitted to be displayed on the display 61, and causes a notification sound for when the facial image is transmitted to be output from the speaker 62. The driver can know that the facial image has been transmitted by the notification screen for when the facial image is transmitted being displayed on the display 61 and the notification sound for when the facial image is transmitted being output from the speaker 62.

[0048] The control device 2 judges whether or not the condition for starting the execution of the face recognition software is satisfied (S9), and when it judges that the condition for starting the execution of the face recognition software is satisfied (S9: YES), it starts the execution of the face recognition software (S10, which corresponds to the software execution control procedure). When it judges that the condition for starting the execution of the face recognition software is not satisfied (S9: NO), the control device 2 waits until the condition is satisfied. In this case, the control device 2 executes the face recognition software according to the above-mentioned situations: (1) when the circumstantial evidence flag is established while the software for detecting unsuitable driving conditions is not being executed, (2) when the circumstantial evidence flag is established while the software for detecting unsuitable driving conditions is being executed, (3) when a request for executing the software for detecting unsuitable driving conditions occurs while the software for detecting unsuitable driving conditions is being executed (case 1), and (4) when a request for executing the software for detecting unsuitable driving conditions occurs while the software for detecting unsuitable driving conditions is being executed (case 2).

[0049] When the control device 2 finishes the execution of the face recognition software, it judges the face recognition result, judges whether the current face recognition result and the previous face recognition result are the same person, and judges whether the driver has changed (S11). If the control device 2 judges that the current face recognition result and the previous face recognition result are the same person and that the driver has not changed (S11: NO), it judges whether the end condition of the driver state monitoring process is established (S3).

[0050] When the control device 2 determines that the current face authentication result and the previous face authentication result are not of the same person and that the driver has changed (S11: YES), it notifies the driver change (S12). As shown in Fig. 12, the control device 2 causes a notification screen for driver change to be displayed on the display 61 and a notification sound for driver change to be output from the speaker 62. The driver can know that the system has correctly recognized the driver change by the notification screen for driver change being displayed on the display 61 and the notification sound for driver change being output from the speaker 62.

[0051] The control device 2 saves the face image of the driver after the driver change (S13) and notifies the driver that the face image of the driver after the driver change has been saved (S14). The face image of the driver saved in this way is used as the face image to be registered in the first pattern described above. As shown in FIG. 13, the control device 2 displays a notification screen when the face image of the driver after the driver change is saved on the display 61 and outputs a notification sound when the face image of the driver after the driver change is saved from the speaker 62. The driver can know that the face image of the driver after the driver change has been saved by the notification screen when the face image of the driver after the driver change is saved being displayed on the display 61 and the notification sound when the face image of the driver after the driver change is saved being output from the speaker 62.

[0052] The control device 2 transmits the face image of the driver after the change to the server (S15) and notifies the server that the face image of the driver after the change has been transmitted (S16). The face image of the driver transmitted to the server in this manner is used as the face image to be registered in the above-mentioned second pattern. As shown in FIG. 14, the control device 2 displays a notification screen when the face image is transmitted after the driver change on the display 61 and outputs a notification sound when the face image is transmitted after the driver change from the speaker 62). The driver can know that the face image after the change has been transmitted by the notification screen when the face image is transmitted after the driver change being displayed on the display 61 and the notification sound when the face image is transmitted after the driver change being output from the speaker 62.

[0053] The above has been an example of displaying various screens on the display 61 and outputting various sounds from the speaker 62 as notifications to the driver, but it is also possible to only display various screens on the display 61 or only output various sounds from the speaker 62.

[0054] As described above, according to this embodiment, the following advantageous effects can be obtained. In the driver's face detection device 7, the face recognition software is not executed periodically, but is executed on the condition that it is determined that the driver can be changed. By executing the software for detecting an unsuitable driving state at a predetermined interval, it is possible to appropriately determine whether the driver is in a suitable driving state or an unsuitable driving state, and by executing the software for face recognition on the condition that the driver can be changed, it is possible to reduce the frequency of executing the software for face recognition. This makes it possible to appropriately determine whether the driver is in a suitable driving state or an unsuitable driving state while avoiding an increase in the amount of regular calculations. In other words, by not executing the software for face recognition when there is no possibility of changing the driver, it is possible to avoid an increase in the amount of regular calculations.

[0055] The face recognition software is executed while the software for detecting unsuitable driving conditions is not executed. By executing the face recognition software without affecting the execution of the software for detecting unsuitable driving conditions at a predetermined interval, it is possible to ensure a state in which it is possible to appropriately determine whether a driving condition is suitable or unsuitable.

[0056] Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and concept of the present disclosure.

[0057] The control unit and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be realized by one or more special-purpose computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. Also, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer.

Claims

1. An image acquisition unit (21) for acquiring a face image of a driver; a first software execution unit (22) that executes software for detecting an inappropriate driving state at a predetermined interval to determine whether a driver is in an appropriate driving state or an inappropriate driving state; a second software execution unit (23) that executes face recognition software for determining a driver change; A change possibility situation determination unit (24) that determines whether or not a driver change is possible; a software execution control unit (25) that executes the face authentication software when it is determined that a driver change is possible based on a result of the determination by the change possibility situation determination unit, The first software execution unit executes the software for detecting the inappropriate driving state once for an execution time shorter than the predetermined cycle time, The software execution control unit executes the face recognition software while the software for detecting unsuitable driving conditions is not executing, and when a request to execute the software for detecting unsuitable driving conditions occurs while the face recognition software is executing, the software execution control unit continues the execution of the face recognition software and cancels the execution of the next software for detecting unsuitable driving conditions.

2. The driver's face recognition device of claim 1, wherein the software execution control unit starts execution of the face recognition software when it is determined that a driver change is possible while the software for detecting an inappropriate driving state is not being executed.

3. The driver's face recognition device of claim 1, wherein the software execution control unit starts execution of the face recognition software after terminating execution of the software for detecting inappropriate driving conditions when it is determined that a driver change is possible during execution of the software for detecting inappropriate driving conditions.

4. The driver face recognition device of claim 1, wherein the change possibility situation determination unit determines whether or not a driver change is possible based on the detection results of at least one of the following: a buckle sensor that detects the state of a seat belt, a door opening / closing sensor that detects the open / closed state of a door, a seating sensor that detects the driver's seated state, an electronic key interior detection sensor that detects the presence of an electronic key in the vehicle interior, a steering wheel touch sensor that detects the state of hands gripping the steering wheel, a steering wheel torque sensor that detects the steering force of the steering wheel, a vehicle speed sensor that detects the vehicle speed, a steering angle sensor that detects the steering angle of the steering wheel, an accelerator sensor that detects the state of an accelerator pedal, a brake sensor that detects the state of a brake pedal, and a shift sensor that detects the state of a shift position.

5. A facial image storage unit (26) for storing a facial image of the driver, The driver face recognition device according to claim 1 , wherein the face image storage unit stores a face image of the driver when the change possibility situation determination unit determines that a driver change is possible.

6. A facial image transmission unit (27) for transmitting a facial image of the driver to a server, The driver face recognition device according to any one of claims 1 to 4, wherein the face image transmission unit transmits a face image of the driver to the server when the change possibility situation determination unit determines that a driver change is possible based on a result of the determination.

7. A driver's face recognition device (7) comprising: a first software execution unit (22) that executes software for detecting an unsuitable driving state at a predetermined cycle to determine whether the driver is in a suitable driving state or an unsuitable driving state; and a second software execution unit (23) that executes software for face recognition to determine whether the driver should be changed, wherein the first software execution unit executes the software for detecting an unsuitable driving state once for an execution time shorter than the time of the predetermined cycle, a change possibility determination step for determining whether a driver change is possible; a software execution control procedure for executing the face recognition software when it is determined that a driver change is possible based on a result of the change possibility situation determination procedure; The software execution control procedure executes the face recognition software while the software for detecting unsuitable driving conditions is not executing, and when a request to execute the software for detecting unsuitable driving conditions occurs while the face recognition software is executing, the driver's face recognition program continues execution of the face recognition software and cancels execution of the next software for detecting unsuitable driving conditions.

Images