Authentication device

Abstract

The authentication device includes: a detection unit that detects and acquires facial feature information based on an image showing the driver's face; a first determination unit that determines whether the acquired facial feature information detected based on the first image is consistent with registered facial feature information; a second determination unit that, if the acquired facial feature information is consistent with the registered facial feature information, determines, based on the first image and the second image, whether the driver is wearing an accessory on their face or has removed an accessory from their face; a third determination unit that, if the driver is wearing an accessory on their face or has removed an accessory from their face, determines whether the acquired facial feature information detected based on the second image is consistent with registered facial feature information; and a registration unit.

Description

Technical Field

[0001] This disclosure relates to authentication devices. Background Technology

[0002] An authentication device for authenticating drivers of vehicles has been proposed.

[0003] For example, in a vehicle, driving data for each driver is collected, and the collected driving data is recorded in association with the certified driver.

[0004] Driver authentication uses the driver's biometric information, such as facial feature information representing the features of the driver's face.

[0005] The authentication device obtains facial feature information based on an image representing the driver's face while driving the vehicle. If the obtained facial feature information matches the registered facial feature information, the driver is authenticated. Typically, the registered facial feature information is obtained based on an image showing a face without glasses, masks, or other coverings.

[0006] Drivers sometimes wear glasses or masks while driving. If the registered facial features are obtained based on the driver's face when not wearing such a device, it may be impossible to authenticate the driver when the device is worn on their face.

[0007] Therefore, a technical solution has been proposed to pre-register facial feature information when no clothing is worn on the face and facial feature information when clothing such as glasses or masks is worn on the face (for example, see Japanese Patent Application Publication No. 2014-71680). Summary of the Invention

[0008] However, in addition to pre-registering facial feature information when the wearer is not wearing the device, it is also necessary to pre-register facial feature information when the wearer is wearing glasses or a mask, which is a tedious task for drivers.

[0009] This disclosure provides an authentication device that can authenticate drivers wearing face coverings without prior registration of their facial features.

[0010] (1) According to one embodiment, an authentication device is provided.

[0011] The authentication device has the following features:

[0012] The memory stores the registered facial feature information of one or more drivers' faces;

[0013] The detection unit detects facial feature information of the driver's face based on an image representing (displaying) the face of the driver who is driving the vehicle.

[0014] The first determination unit determines whether the facial feature information detected by the detection unit based on the first image is consistent with any one of the one or more registered facial feature information stored in the memory;

[0015] The second determination unit, when determined by the first determination unit to obtain facial feature information that matches any one of one or more registered facial feature information, determines, based on the first image and the second image representing the face of the driver of the vehicle and obtained after the first image, whether the driver is wearing an accessory on his face or has removed an accessory from his face.

[0016] The third determination unit, when the second determination unit determines that the driver is wearing an accessory on their face or has removed an accessory from their face, determines whether the facial feature information detected by the detection unit based on the second image matches any one of the registered facial feature information stored in the memory; and

[0017] If the third determination unit determines that the facial feature information obtained by the detection unit based on the second image is inconsistent with any one of the registered facial feature information, the registration unit records the facial feature information obtained by the detection unit based on the second image as new registered facial feature information of the driver's face in the memory.

[0018] (2) In the authentication device of (1), it is preferable that if the second determination unit determines that the driver is wearing an accessory on his face when the accessory that is not shown in the first image is shown in the second image, and if the accessory shown in the first image is not shown in the second image, it is determined that the accessory has been removed from the driver's face.

[0019] (3) In the authentication device of (1) or (2), it is preferable to further include a fourth determination unit, which determines whether the driver whose face is shown in the second image is the same driver as the driver whose face is shown in the first image if the second determination unit determines that the driver is wearing an accessory on his face or has removed the accessory from his face. If the third determination unit determines that the obtained facial feature information is inconsistent with any one or more registered facial feature information, and the fourth determination unit determines that the driver whose face is shown in the second image is the same driver as the driver whose face is shown in the first image, the registration unit records the obtained facial feature information detected by the detection unit based on the second image as new registered facial feature information of the driver's face in the memory.

[0020] (4) In any of the authentication devices in (1) to (3), preferably, if the second determination unit determines that the driver is wearing a device on his face or has removed a device from his face before a predetermined reference time has elapsed from the time point when the first determination unit determines that the facial feature information is consistent with any one of the registered facial feature information, the fourth determination unit determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image.

[0021] (5) In any of the authentication devices in (1) to (4), preferably, if the vehicle door is not opened or closed during the period between the first time point when the first determination unit determines that the facial feature information obtained matches any one of the one or more registered facial feature information and the second time point when the second determination unit determines that the driver is wearing an accessory on his face or has removed an accessory from the driver's face, the fourth determination unit determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image.

[0022] The authentication device disclosed herein can authenticate drivers wearing such items on their faces without pre-registering their facial features. Attached Figure Description

[0023] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which the same reference numerals denote the same elements, and wherein:

[0024] Figure 1 This is a diagram illustrating the general operation of the authentication device according to this embodiment.

[0025] Figure 2 This is a hardware configuration diagram of a vehicle equipped with the authentication device of this embodiment.

[0026] Figure 3 This is an example of an action flowchart related to the determination process of the authentication device in this embodiment.

[0027] Figure 4A This is a diagram representing an example of a registration form;

[0028] Figure 4B This is an example diagram showing a registration form with added registration and authentication information.

[0029] Figure 5 This is an example of an action flowchart related to the authentication process of the authentication device in this embodiment.

[0030] Figure 6This is an example of an operation flowchart related to a variation of the authentication process of the authentication device in this embodiment. Detailed Implementation

[0031] Figure 1 This is a diagram illustrating the general operation of the authentication device according to this embodiment. (As shown...) Figure 1 As shown, vehicle 10 has a collection device 11 and an authentication device 12. Driver 40 is seated in driver's seat 31 inside vehicle compartment 30 in a manner that allows him to operate the steering wheel, accelerator pedal, brake pedal, etc. The collection device 11 collects driving data of driver 40 on vehicle 10.

[0032] The authentication device 12 obtains facial feature information of the driver 40 based on a facial image representing the driver 40 acquired using the surveillance camera 2, and determines whether the obtained facial feature information matches the registered facial feature information. The obtained facial feature information is one example of obtaining facial feature information. The registered facial feature information is one example of registering facial feature information.

[0033] If the obtained facial feature information matches the registered facial feature information, the authentication device 12 notifies the collection device 11 that the driver 40 has been authenticated. The collection device 11 records the collected driving data in association with the authenticated driver 40.

[0034] First, driver 40 boards vehicle 10 without wearing glasses or a mask. Authentication device 12 obtains facial feature information of driver 40 based on a facial image P1 representing driver 40's face acquired using surveillance camera 2, and determines that the obtained facial feature information matches the registered facial feature information. Authentication device 12 then notifies collection device 11 that driver 40 has been authenticated.

[0035] Furthermore, the driver 40 wore glasses while driving the vehicle 10. A facial image P2 showing the face of the driver 40 wearing glasses was obtained using the surveillance camera 2.

[0036] The authentication device 12 determines that the driver 40 is wearing glasses based on the facial image P1 taken when the driver 40 is being authenticated and the facial image P2 taken after the driver 40 is taken using the surveillance camera 2. Therefore, the authentication device 12 determines whether the facial feature information detected based on the facial image P2 is consistent with the registered facial feature information.

[0037] Since the facial feature information detected based on facial image P2 is inconsistent with the registered facial feature information, the authentication device 12 records the facial feature information detected based on facial image P2 as the new facial feature information of the driver 40. Therefore, in the future, the driver 40 wearing glasses can be authenticated by the authentication device 12.

[0038] That is, when a driver 40 wearing glasses is riding in vehicle 10, the authentication device 12 obtains the driver 40's facial feature information based on a facial image P2 representing the driver 40's face obtained using the surveillance camera 2. The authentication device 12 determines that the obtained facial feature information matches the registered facial feature information.

[0039] As explained above, the authentication device 12 of this embodiment can authenticate the driver 40 when wearing a device on his face without prior registration of the driver's facial feature information when wearing glasses or a mask.

[0040] Figure 2 This is a hardware configuration diagram of a vehicle 10 equipped with the authentication device 12 of this embodiment. The vehicle 10 includes a surveillance camera 2, an opening / closing sensor 3, a collection device 11, and the authentication device 12, etc.

[0041] The surveillance camera 2, the opening / closing sensor 3, the collection device 11, and the authentication device 12 can be communicatively connected via an in-vehicle network 13 that conforms to a standard controller area network.

[0042] The surveillance camera 2 is configured inside the vehicle compartment 30 in a manner capable of acquiring facial images, including the face of the driver 40 who is driving the vehicle 10. The surveillance camera 2 is, for example, configured on the dashboard. The face of the driver 40, seated in the driver's seat 31, is shown in the facial image acquired by the surveillance camera 2.

[0043] The surveillance camera 2 has a two-dimensional detector composed of an array of photoelectric conversion elements such as CCD or C-MOS that are sensitive to infrared light, and an imaging optical system that forms an image of the area to be acquired on the two-dimensional detector. The surveillance camera 2 outputs the facial image and the time of image acquisition to the authentication device 12 via the in-vehicle network 13.

[0044] The opening / closing sensor 3 generates an open signal indicating that "the driver's side door is open" and a closed signal indicating that "the driver's side door is closed", and outputs them to the authentication device 12 via the in-vehicle network 13.

[0045] The collecting device 11 collects driving data from the vehicle 10 and records it in association with the driver 40 of the vehicle 10. The driving data includes, for example, information such as the vehicle 10's speed, acceleration, and deceleration. Additionally, the driving data may also include information indicating the operation of the steering wheel, accelerator pedal, brake pedal, etc.

[0046] The authentication device 12 performs detection processing, judgment processing, and registration processing. Therefore, the authentication device 12 has a communication interface (IF) 21, a memory 22, and a processor 23. The communication interface 21, memory 22, and processor 23 are connected via signal lines 24. The communication interface 21 has interface circuitry for connecting the authentication device 12 to the in-vehicle network 13.

[0047] Memory 22 is an example of a storage unit, such as a volatile semiconductor memory or a non-volatile semiconductor memory. Furthermore, memory 22 stores computer programs used in information processing executed by processor 23, as well as various data.

[0048] The authentication device 12 may have all or part of its functions implemented, for example, by a computer program operating on the processor 23. The processor 23 has a detection unit 231, a determination unit 232, and a registration unit 233. The determination unit 232 is an example of a first determination unit, a second determination unit, a third determination unit, and a fourth determination unit. Alternatively, the functional modules of the processor 23 may be dedicated arithmetic circuits provided on the processor 23. The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may also have other arithmetic circuits such as logic operation units, numerical operation units, or graphics processing units.

[0049] The collection device 11 and the authentication device 12 are, for example, electronic control units (ECUs). Figure 2 In this document, the collection device 11 and the authentication device 12 are described as different devices (e.g., Electronic Control Unit: ECU), but these devices may also be configured as a single device.

[0050] Figure 3 This is an example of an operation flowchart related to the determination process of the authentication device 12 in this embodiment. The authentication device 12, at determination times with predetermined cycles, performs... Figure 3 The action flowchart shown is used to perform the decision-making process.

[0051] First, the detection unit 231 acquires the facial image output from the surveillance camera 2 (S101).

[0052] Next, the detection unit 231 detects facial feature points (facial feature information) of the driver 40's face based on the facial image (S102). The detection unit 231 has a recognizer that detects faces and facial feature points from an input image. As such a recognizer, the detection unit 231 can, for example, use a deep neural network (DNN) that has been pre-learned to detect faces and facial feature points shown in the input image. Facial feature points can include the eyes, nose, mouth, and facial contours. Then, the detection unit 231 detects the positions of the eyes, nose, and mouth, as well as the shape of the facial contours, as facial feature information. This facial feature information is one example; other facial feature information can also be used.

[0053] When glasses are worn on the face, the detection unit 231 can detect the positions of the eyes, nose, and mouth, as well as the shape of the facial contours, as facial feature information. The position of the eyes sometimes differs when glasses are worn on the face compared to when glasses are not worn. Furthermore, sometimes the eyes are not detected as facial feature points from the image when glasses are worn on the face. In this case, information other than the position of the eyes is detected as facial feature information.

[0054] When a mask is worn on the face, the detection unit 231 can detect the position of the eyes and the shape of the facial contours as facial feature information. Sometimes, the position of the nose is also detected as facial feature information when a mask is worn on the face.

[0055] When glasses and masks are worn on the face, the detection unit 231 can detect the position of the eyes and the shape of the facial contours as facial feature information. Sometimes, when glasses and masks are worn on the face, the position of the nose is detected as facial feature information.

[0056] Next, the determination unit 232 determines whether the facial feature information detected based on the facial image matches any one of the facial feature information registered in the registration table 100 stored in the memory 22.

[0057] Figure 4AThis is a diagram illustrating an example of registration form 100. In registration form 100, one or more facial feature information entries are registered for each driver. Registration form 100 includes a driver ID field 101 for registering driver identification information (ID) used to identify the driver and a first facial feature information field 102 for registering first facial feature information. Registration form 100 has a second facial feature information field 103 for registering second facial feature information, a third facial feature information field 104 for registering third facial feature information, and a fourth facial feature information field 105 for registering fourth facial feature information. The driver ID, first facial feature information, second facial feature information, third facial feature information, and fourth facial feature information are associated and registered in registration form 100. Furthermore, the number of facial feature entries associated with a single driver ID in registration form 100 is an example and is not limited to this.

[0058] Regarding the driver identified by driver ID001, first facial feature information F1-1 is registered. For example, first facial feature information F1-1 is on the face where no protective device is worn (e.g., refer to...). Figure 1 The facial feature information of the face image P1.

[0059] Regarding the driver identified by driver ID002, first facial feature information F2-1 is registered. For example, first facial feature information F2-1 is the face of someone wearing glasses (e.g., see reference). Figure 1 The facial feature information of the face image P2.

[0060] Regarding the driver identified by driver ID003, the following facial feature information is registered: first facial feature information F3-1, second facial feature information F3-2, third facial feature information F3-3, and fourth facial feature information F3-4. For example, first facial feature information F3-1 is the facial feature information of a face without any covering. For example, second facial feature information F3-2 is the facial feature information of a face with glasses. For example, third facial feature information F3-3 is the facial feature information of a face with a mask (for example, see reference). Figure 1 The facial feature information of the face image P3). For example, the fourth facial feature information F3-4 is a face wearing glasses and a mask (e.g., see reference). Figure 1 The facial feature information of the face image (P4).

[0061] If the facial feature information detected based on the facial image matches any of the facial feature information registered in the registration form 100, the determination unit 232 performs the following processing. Specifically, the determination unit 232 generates a first determination result (S103) indicating that "facial feature information detected based on the facial image is registered," and ends the series of processes. The first determination result includes the driver ID associated with the registered facial feature information that matches the facial feature information detected based on the facial image, the facial image, and the first determination time at which the first determination result was generated.

[0062] Furthermore, if the facial feature information detected based on the facial image is inconsistent with any of the facial feature information registered in the registration form 100, the determination unit 232 performs the following processing. That is, the determination unit 232 generates information indicating "no facial feature information detected based on the facial image is registered" as the first determination result (S103), and ends the series of processing.

[0063] Figure 5 This is an example of an operation flowchart related to the authentication process of the authentication device in this embodiment. The authentication device 12, at authentication times with a predetermined cycle, performs... Figure 5 The flowchart shown illustrates the action to perform the authentication process. The authentication process cycle is preferably longer than the determination process cycle described above.

[0064] First, the determination unit 232 obtains the first determination result of the above-described determination process (S201). If the first determination result indicates that facial feature information detected based on the facial image is registered, the determination result includes the driver ID, the facial image, and the first determination time.

[0065] Next, the determination unit 232 determines whether the driver 40 represented by the face image is authenticated (S202). If the first determination result indicates that facial feature information detected based on the face image is registered, the determination unit 232 determines that the driver 40 represented by the face image is authenticated. On the other hand, if the first determination result indicates that facial feature information detected based on the face image is not registered, the determination unit 232 determines that the driver 40 represented by the face image is not authenticated.

[0066] If the driver 40 represented by the facial image is determined to be authenticated (S202: Yes), the determination unit 232 obtains a new facial image output from the surveillance camera 2 (S203).

[0067] Next, the determination unit 232 determines, based on the facial image included in the first determination result and the new facial image, whether the driver 40 is wearing an accessory on their face or has removed an accessory from their face (S204). Hereinafter, the facial image included in the first determination result is also referred to as the first image. Hereinafter, the new facial image is also referred to as the second image. The second image is an image acquired by the surveillance camera 2 at a time after the moment the first image was acquired.

[0068] The detection unit 231 detects whether an object is being worn on the face represented by the first image and the second image, respectively. The detection unit 231 has a recognizer that detects the face and the object being worn by inputting an image. As such a recognizer, the detection unit 231 can, for example, use a deep neural network (DNN) that has been pre-learned to detect the face and the object being worn in the input image. Examples of objects being worn include glasses, masks, etc. However, the objects are not limited to these.

[0069] For each of the first and second images, the detection unit 231 notifies the determination unit 232 of whether a face is shown and, in the case of a detected garment, the type of garment. Here, if no "face is shown" information is obtained for the second image, the process can return to before S201 or S202. Regarding the first image, "face is shown" is detected.

[0070] The determination unit 232 determines whether there has been a change in the face covering of the driver 40 shown in both the first and second images (S205). For example, if no wearable object is detected in either the first or second image, the determination unit 232 determines that the face covering has not changed. Furthermore, if the same face covering is detected in both the first and second images, the determination unit 232 also determines that the face covering has not changed.

[0071] On the other hand, if an accessory not shown in the first image is shown in the second image, the determination unit 232 determines that the driver 40 is wearing an accessory on their face. Conversely, if an accessory shown in the first image is not shown in the second image, it is determined that the accessory has been removed from the driver 40's face. Whether the driver 40 is wearing an accessory on their face or has removed an accessory from their face, the determination unit 232 determines that there has been a change in the accessory worn on the face.

[0072] If two or more objects are worn on the face in the first image, and at least one wearing part changes in the second image, the determination unit 232 determines that there is a change in the objects worn on the face.

[0073] If the face covering changes (S205: Yes), the determination unit 232 performs a determination process (S206). The determination unit 232 determines whether the facial feature information detected by the detection unit 231 based on the second image matches any one or more facial feature information registered in the registration table 100 stored in the memory 22. Similar to S102 described above, the detection unit 231 detects the positions of the eyes, nose, and mouth, as well as the shape of the facial contour, as facial feature information based on the second image.

[0074] If the facial feature information detected based on the second image matches any of the facial feature information registered in the registration form 100, the determination unit 232 performs the following processing: The determination unit 232 generates information indicating "facial feature information detected based on the facial image is registered" as a second determination result. On the other hand, if the facial feature information detected based on the facial image does not match any of the facial feature information registered in the registration form 100, the determination unit 232 performs the following processing: The determination unit 232 generates information indicating "no facial feature information detected based on the second image is registered" as a second determination result. The second determination result includes the driver ID obtained in S201 along with the first image.

[0075] Next, the determination unit 232 determines whether the driver 40 represented by the second image has been authenticated (S207). If the second determination result indicates that "facial feature information detected based on the second image is registered," the determination unit 232 determines that the driver 40 represented by the second image has been authenticated. On the other hand, if the second determination result indicates that "facial feature information detected based on the second image is not registered," the determination unit 232 determines that the driver 40 represented by the second image has not been authenticated.

[0076] If the driver 40 shown in the second image is not certified (S207: No), the determination unit 232 determines whether the driver whose face is shown in the second image is the same driver whose face is shown in the first image (S208).

[0077] For example, before a predetermined reference time has elapsed since the first determination time, a second determination result is generated, determining that the driver is wearing an accessory on their face or has removed an accessory from their face. In this case, the determination unit 232 can also determine that the driver whose face is shown in the second image is the same driver whose face is shown in the first image. The reference time can be set to, for example, 1 to 10 seconds. If such a time is used, it is possible to presume that the driver driving the vehicle 10 is the same driver.

[0078] If, during the period between the first determination time and the time point at which the second determination result is generated (hereinafter also referred to as the second determination time), the vehicle 10 door is not opened or the vehicle 10 door is not closed, the determination unit 232 determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image. The second determination result is the determination that the driver is wearing a device on their face or has removed a device from their face.

[0079] If no open signal indicating that the driver's side door is open is input during the period between the first determination time and the second determination time, the door of vehicle 10 is not opened. Similarly, if no close signal indicating that the driver's side door is closed is input during the period between the first determination time and the second determination time, the door of vehicle 10 is not closed.

[0080] If, during the period between the first and second determination times, the driver's side door is opened or closed, the driver may have been changed. On the other hand, if, during the period between the first and second determination times, the driver's side door is neither opened nor closed, the driver is considered to be the same driver.

[0081] The determination unit 232 can also determine, if the ignition switch is turned on during the period between the first determination time and the second determination time, that the driver whose face is shown in the second image is not the same driver as the driver whose face is shown in the first image. The determination unit 232 can also determine, if it detects that the driver has newly taken up a seat using a seating sensor located in the driver's seat, that the driver whose face is shown in the second image is not the same driver as the driver whose face is shown in the first image.

[0082] If the driver whose face is shown in the second image is the same driver whose face is shown in the first image (S208: Yes), the registration unit 233 performs the following processing. That is, the registration unit 233 associates the facial feature information detected based on the second image with the driver ID included in the first determination result and registers it in the registration table 100, ending the series of processes. The registration table 100 containing the facial feature information detected based on the second image is recorded in the memory 22.

[0083] On the other hand, if the face covering remains unchanged (S205: No), the determination unit 232 notifies the collection device 11 that the driver 40 has been certified. Alternatively, if the driver 40 is certified as shown in the second image (S207: Yes), the determination unit 232 notifies the collection device 11 that the driver 40 has been certified. The collection device 11 records the collected driving data in association with the certified driver 40.

[0084] On the other hand, if the driver 40 shown in the first image is not certified (S202: No), the determination unit 232 notifies the collection device 11 that the driver has not been certified. Alternatively, if the driver whose face is shown in the second image is not the same driver whose face is shown in the first image (S208: No), the determination unit 232 notifies the collection device 11 that the driver has not been certified. Since the driver is not certified, the collection device 11 does not record the collected driving data.

[0085] Next, refer to the following Figure 1 , Figure 4A as well as Figure 4B A specific example of the authentication process of authentication device 12 will be explained.

[0086] (1) In the first image, as Figure 1 The facial image P1 shows the face of a driver who is not wearing glasses or a mask. The driver is identified by driver ID 001, and as the first facial feature information F1-1, the facial feature information of the driver who is not wearing any coverings is registered in registration form 100. The authentication device 12 authenticates the driver shown in the first image. Afterwards, the driver puts on glasses.

[0087] In the second image, as Figure 1 Image P2 shows the face of a driver wearing glasses. The type of face covering varies, and the driver shown in the second image is not identified, but the driver shown in the second image is the same driver as the driver shown in the first image. Therefore, as... Figure 4B As shown, the authentication device 12 registers the second facial feature information F1-2 detected based on the second image in the registration form 100, associating it with the driver ID001. Therefore, drivers wearing glasses will be authenticated by the authentication device 12 in the future. However, there are also cases where the driver is not authenticated because the facial feature information is different when the driver is wearing glasses that are different from those registered.

[0088] (2) In the first image, as Figure 1Image P2 shows the face of a driver wearing glasses. The driver is identified by driver ID 002, and the facial feature information of the driver wearing glasses is registered in registration form 100 as first facial feature information F2-1. The authentication device 12 authenticates the driver shown in the first image. Afterwards, the driver wears a mask on their face.

[0089] In the second image, as Figure 1 Image P4 shows the face of a driver wearing glasses and a mask. The facial adornments vary, and the driver shown in the second image is not identified, but the driver shown in the second image is the same driver shown in the first image. Therefore, as... Figure 4B As shown, the authentication device 12 associates the second facial feature information F2-2 detected based on the second image with the driver ID002 and registers it in the registration form 100. Therefore, in the future, drivers wearing glasses and masks on their faces will be authenticated by the authentication device 12.

[0090] (3) In the first image, as Figure 1 Image P3 shows the face of a driver wearing a mask. The driver is identified by driver ID 003. As the first facial feature information F3-1, the facial feature information of a driver not wearing any clothing is registered in registration form 100. Additionally, as the second facial feature information F3-2, the facial information of a driver wearing glasses is registered in registration form 100. Furthermore, as the third facial feature information F3-3, the facial feature information of a driver wearing a mask is registered in registration form 100. Additionally, as the fourth facial feature information F3-4, the facial feature information of a driver wearing both glasses and a mask is registered in registration form 100. The authentication device 12 authenticates the driver shown in the first image. Afterwards, the driver puts on glasses.

[0091] In the second image, as Figure 1 Image P4 shows the face of a driver wearing glasses and a mask. While the items worn on the driver's face vary, the facial features shown in Image 2 are associated with driver ID 003 and registered as facial feature information F3-4 in registration form 100, thus the driver is authenticated.

[0092] As detailed above, the authentication device according to this embodiment can authenticate drivers wearing objects on their faces without pre-registering their facial feature information.

[0093] Figure 6 This is an example of an operation flowchart related to a variation of the authentication process of the authentication device 12 in this embodiment. In this variation, "omitted" Figure 5 The difference lies in the handling of S208 in this respect.

[0094] Will Figure 6 The processing time shown is set to be sufficiently short compared to the time required for driver replacement, and will Figure 3 The determination process cycle is set to be shorter than the identification process cycle. Thus, the driver whose face is shown in the second image is considered to be the same driver as the driver whose face is shown in the first image, so this can be omitted. Figure 5 S208 in it. For example, it can also be... Figure 6 The processing cycle shown is set to 0.1 seconds to 1.0 seconds.

[0095] In this disclosure, the authentication device of the above-described embodiments can be appropriately modified as long as it does not depart from the spirit of this disclosure. Furthermore, the technical scope of this disclosure is not limited to these embodiments, but extends to the invention as described in the claims and its equivalents.

Claims

1. An authentication device, comprising: The memory stores the registered facial feature information of one or more drivers' faces; The detection unit detects facial feature information of the driver's face based on an image showing the driver's face while driving the vehicle. The first determination unit determines whether the obtained facial feature information detected by the detection unit based on the first image is consistent with any one of the registered facial feature information stored in the memory; The second determination unit, when determined by the first determination unit to be consistent with any one of the registered facial feature information, determines, based on the first image and a second image showing the face of the driver of the vehicle and obtained after the first image, whether the driver is wearing an accessory on his face or has removed an accessory from his face. The third determination unit, when the second determination unit determines that the driver is wearing a device on his face or has removed a device from the driver's face, determines whether the facial feature information detected by the detection unit based on the second image is consistent with any one of the registered facial feature information stored in the memory; as well as If the third determination unit determines that the acquired facial feature information detected by the detection unit based on the second image is inconsistent with any one or more of the registered facial feature information, the registration unit records the acquired facial feature information detected by the detection unit based on the second image as new registered facial feature information of the driver's face in the memory.

2. The authentication device according to claim 1, The second determination unit determines that the driver is wearing an adornment on his face if the adornment not shown in the first image is shown in the second image, and determines that the adornment has been removed from the driver's face if the adornment shown in the first image is not shown in the second image.

3. The authentication device according to claim 1 or 2, It also has a fourth determination section. If the second determination unit determines that the driver is wearing a face covering or has removed a face covering from the driver's face, the fourth determination unit determines whether the driver shown in the second image is the same driver as the driver shown in the first image. If the third determination unit determines that the acquired facial feature information is inconsistent with any one or more of the registered facial feature information, and the fourth determination unit determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image, the registration unit records the acquired facial feature information detected by the detection unit based on the second image as new registered facial feature information of the driver's face in the memory.

4. The authentication device according to claim 3, If, before a predetermined reference time has elapsed since the first determination unit determines that the obtained facial feature information matches any one of the registered facial feature information, and the second determination unit determines that the driver is wearing an accessory on their face or has removed an accessory from their face, the fourth determination unit determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image.

5. The authentication device according to claim 3, If, during the period between the first determination unit's determination that the obtained facial feature information matches any one of the registered facial feature information and the second determination unit's determination that the driver is wearing an accessory on their face or has removed an accessory from their face, and if the vehicle door is not opened or closed, the fourth determination unit determines that the driver whose face is shown in the second image is the same driver whose face is shown in the first image.

Images