Vehicle driver authentication device

Abstract

To provide a vehicle driver authentication device that can reliably prevent authentication from being bypassed through driver impersonation. [Solution] The ECU 150's processor 152 includes: a face image acquisition unit 152a that acquires a face image of the driver attempting to start the vehicle before the vehicle is started; a face image determination unit 152b that compares the face image with a pre-registered face image and determines whether the face image matches the registered face image; a brightness change calculation unit 152c that calculates the time change of the brightness value of the face image; a biological determination unit 152d that performs frequency analysis of the time change to calculate the intensity in a predetermined frequency range and determines that the driver represented in the face image is a living being if the intensity is above a predetermined threshold; and a start permission unit 152f that permits the driver to start the vehicle if it is determined that the face image matches a registered face image and that the driver represented in the face image is a living being.

Description

Technical Field

[0001] The present invention relates to a driver authentication device for a vehicle.

Background Art

[0002] Conventionally, in a vehicle anti-theft device using face recognition technology, a vehicle anti-theft device that uses a smart key system as a primary authentication and a face recognition system as a secondary authentication is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technology described in the above patent document, if the primary authentication and the secondary authentication are individually breached, there is a risk that the vehicle will be driven by a third party. Specifically, if the face authentication is breached by forgery using a face photo or a mannequin, and the authentication of the smart key system is also breached due to theft of the key or the like, there is a risk that the vehicle will be driven by a third party. Therefore, it is necessary to reliably suppress the breach of face authentication by forgery using a face photo or a mannequin.

[0005] Therefore, an object of the present invention is to provide a driver authentication device for a vehicle that can reliably suppress the breach of authentication by forgery of a driver.

Means for Solving the Problems

[0006] The gist of the present disclosure is as follows.

[0007] (1) A facial image acquisition unit that acquires a facial image of the driver who is about to start the vehicle before the vehicle is started, A face image determination unit compares the aforementioned face image with a pre-registered registered face image and determines whether the aforementioned face image matches the registered face image. A brightness change calculation unit that calculates the time change in the brightness value of the face image, A biological determination unit that performs frequency analysis on the aforementioned time change to calculate the intensity in a predetermined frequency range, and determines that the driver represented in the facial image is a living organism if the intensity is above a predetermined threshold, When it is determined that the facial image matches the registered facial image, and that the driver represented in the facial image is a living person, the starting permission unit allows the driver to start the vehicle. A vehicle driver authentication device equipped with [the following features].

[0008] (2) The vehicle driver authentication device described in (1) above, wherein the starting permission unit does not permit the driver to start the vehicle in at least one of the following cases: when the facial image is not determined to match the registered facial image, or when the driver represented in the facial image is not determined to be a living person.

[0009] (3) A driver authentication device for the vehicle described in (1) or (2) above, wherein the predetermined frequency range is 0.7 Hz or more and 2 Hz or less. [Effects of the Invention]

[0010] According to the present invention, a vehicle driver authentication device is provided that can reliably prevent authentication from being bypassed by driver impersonation. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a vehicle driver assistance system according to one embodiment. [Figure 2] This is a schematic diagram illustrating the authentication method according to this embodiment, which performs facial recognition and biometric authentication simultaneously. [Figure 3]This is a schematic diagram illustrating the authentication method according to this embodiment, which performs facial recognition and biometric authentication simultaneously. [Figure 4] This is a schematic diagram showing the functional blocks of the ECU's processor. [Figure 5] The average brightness value of the facial image is calculated, and the temporal change in the average brightness value is analyzed in the frequency domain. [Figure 6] This is a flowchart showing the processing performed by the ECU's processor. [Modes for carrying out the invention]

[0012] Several embodiments of the present invention will be described below with reference to the drawings. However, these descriptions are intended to be merely illustrative of preferred embodiments of the present invention and are not intended to limit the present invention to such specific embodiments.

[0013] Figure 1 is a schematic diagram of a vehicle driver assistance system 1000 according to one embodiment. The driver assistance system 1000 is mounted on a vehicle such as an automobile and includes a driver monitor camera 110, an ignition switch 120, and an electronic control unit (ECU: Electronic Control Unit, hereinafter referred to as ECU) 150. The driver monitor camera 110, the ignition switch 120, and the ECU 150 are each connected to communicate via an in-vehicle network compliant with standards such as a Controller Area Network (CAN).

[0014] The driver monitoring camera 110 includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The driver monitoring camera 110 is provided toward the assumed position of the vehicle driver near the dashboard, steering column, or windshield inside the vehicle, photographs the driver's face, and generates an image (face image) in which the driver's face is captured. The driver monitoring camera 110 performs photographing at a predetermined photographing cycle (for example, 1 / 30 second to 1 / 10 second). Each time the driver monitoring camera 110 generates an image, it outputs the generated image to the ECU 150 via the in-vehicle network.

[0015] The ignition switch 120 is a switch that the driver operates when starting (activating) the vehicle. When the ignition switch 120 is turned on by the driver's operation, power is supplied to the vehicle and the vehicle starts.

[0016] The ECU 150 is an aspect of the driver authentication device for the vehicle according to the present disclosure. The ECU 150 includes a processor 152, a memory 154, and a communication interface 156. The processor 152 includes one or more CPUs (Central Processing Units) and peripheral circuits thereof. The processor 152 may further include other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. The memory 154 has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores data related to the processing according to the present embodiment. The communication interface 156 has an interface circuit for connecting the ECU 150 to the in-vehicle network.

[0017] As described above, in face authentication technology, it is difficult to counter impersonation using a face photo, mannequin, etc. Even in multi-factor authentication that combines face authentication with authentication using a smart key or smartwatch, if each authentication is breached, impersonation by a third party other than the person cannot be prevented.

[0018] In this embodiment, by performing face authentication and biometric authentication simultaneously using the face image generated by the driver monitoring camera 110, authentication is performed with high accuracy, and breakthrough of authentication by forgery is suppressed.

[0019] FIGS. 2 and 3 are schematic diagrams for explaining an authentication method for simultaneously performing face authentication and biometric authentication according to this embodiment. FIG. 2 shows a case where a third party B attempts to impersonate the vehicle owner A and perform face authentication using the face photo 10 of the owner A to start the vehicle. In this case, since the face image 20 generated by the driver monitoring camera 110 shows the owner A, as a result of comparison with the face image of the owner A stored in advance, the two match. Thereby, it is determined that the owner A is shown in the face image 20, and the face authentication is successful. Therefore, authentication can be broken through only by face authentication.

[0020] However, in this embodiment, in addition to face authentication by image, biometric authentication based on biometric information obtained from the face image 20 is performed. In biometric authentication, a change in the brightness (luminance) of the face image 20 due to a change in blood flow corresponding to the heartbeat is detected, and the presence or absence of biometric information is determined. In the case of FIG. 2, since the face image 20 is an image of the face photo 10, a change in the brightness of the face image 20 is not detected, and biometric information is not detected. Therefore, the biometric authentication is unsuccessful, and the owner A shown in the face image 20 is determined to be a forgery using a face photo or a mannequin. Since the authentication is finally successful only when both face authentication and biometric authentication are successful, in the example of FIG. 2, the authentication is unsuccessful, and the authentication cannot be broken through by the impersonation of the third party B. Thereby, starting of the vehicle by the third party B is not permitted.

[0021] Figure 3 shows a scenario where vehicle owner A authenticates themselves and attempts to start the vehicle, rather than being impersonated. In this case, the face image 20 generated by the driver monitor camera 110 represents owner A, and when compared with a pre-stored face image of owner A, they match. As a result, it is determined that the face image 20 is of owner A, and face authentication is successful. In addition, the face image 20 generated by the driver monitor camera 110 detects temporal changes in brightness 30 caused by changes in blood flow corresponding to the heartbeat, so biometric authentication is also successful. Therefore, when vehicle owner A performs authentication, authentication is ultimately successful. As a result, owner A is permitted to start the vehicle.

[0022] As described above, according to the method of this embodiment, it is possible to determine whether the user is impersonating a third party or whether the user is a user who can properly drive the vehicle, based on information acquired from a single sensor (driver monitor camera 110). Since both facial recognition and biometric authentication are performed simultaneously based on a single facial image 20, individual authentications cannot be bypassed by impersonation, authentication can be performed with high accuracy, and it is possible to suppress authentication bypasses by impersonation.

[0023] Figure 4 is a schematic diagram showing the functional blocks of the processor 152 of the ECU 150. The processor 152 of the ECU 150 includes a face image acquisition unit 152a, a face image determination unit 152b, a brightness change calculation unit 152c, a biometric determination unit 152d, an authentication unit 152e, and a start permission unit 152f. Each of these parts of the processor 152 is a functional module realized, for example, by a computer program running on the processor 152. In other words, the functional blocks of the processor 152 consist of the processor 152 and a program (software) to make it function. The program may also be recorded in the memory 154 of the ECU 150 or on an externally connected recording medium. Alternatively, each of these parts of the processor 152 may be a dedicated arithmetic circuit provided on the processor 152.

[0024] The face image acquisition unit 152a acquires face images generated by the driver monitoring camera 110. More specifically, the face image acquisition unit 152a acquires a face image of the driver who is about to start the vehicle before the vehicle is started.

[0025] The face image acquisition unit 152a acquires the driver's face image from the image generated by the driver monitor camera 110, for example, by template matching between a template image and an image generated by the driver monitor camera 110, or by inputting the image generated by the driver monitor camera 110 into a classifier trained on machine learning for target detection.

[0026] For example, the face image acquisition unit 152a can use a segmentation classifier as such a classifier. This classifier takes an input image and, for each pixel of the image, outputs the likelihood that an object is represented in that pixel for each type of object that could potentially be represented in that pixel. The classifier can then identify the object with the highest likelihood as being represented. The face image acquisition unit 152a can use a deep neural network (DNN) with a segmentation-oriented convolutional neural network (CNN) architecture, such as a Fully Convolutional Network (FCN), as such a classifier.

[0027] The face image determination unit 152b compares the face image with a pre-registered face image and determines whether the face image matches a registered face image. The face image determination unit 152b may determine the degree of agreement between the face image and the registered face image by, for example, pattern matching, and determine that the face image matches a registered face image if the degree of agreement is greater than or equal to a predetermined value. The registered face image is a face image of a person authorized to drive the vehicle, for example, a face image of the vehicle owner. The registered face image may be registered by being stored in memory 154 in advance.

[0028] The luminance change calculation unit 152c calculates the time change in the luminance value of the face image. More specifically, the luminance change calculation unit 152c calculates the average luminance value of the face image generated by the driver monitor camera 110 and outputs the time change in the average luminance value. The luminance change calculation unit 152c may calculate the average luminance value in a predetermined region 20a of the face, for example, as shown in Figures 2 and 3. The predetermined region 20a may be the entire face or the lower half of the face. In particular, by setting the predetermined region 20a to the lower half of the face, it becomes possible to reliably detect the time change in luminance caused by changes in blood flow in response to heart rate.

[0029] The biological detection unit 152d performs frequency analysis on the time change in the brightness value of the face image to calculate the intensity (spectral intensity, amplitude) in a predetermined frequency range (for example, the heart rate frequency range of 0.7 Hz to 2 Hz). If the intensity is above a predetermined threshold, it determines that the driver represented in the face image is a living organism. In other words, the biological detection unit 152d performs frequency analysis on the time change in the brightness value and determines that the driver represented in the face image has a heartbeat if the intensity in a predetermined frequency range is above a predetermined threshold.

[0030] Figure 5 shows the process for calculating the average brightness value of a face image 20, performing frequency analysis on the temporal change in the average brightness value 30, and calculating the intensity in a predetermined frequency range. For example, if the temporal change in the average brightness value 30 is performed frequency analysis using the Fast Fourier Transform, and the maximum value of the intensity in a predetermined frequency range is greater than or equal to a predetermined threshold TH, it is determined that a heartbeat is present. Alternatively, if the integral value of the intensity in a predetermined frequency range is greater than or equal to a predetermined threshold, it may be determined that a heartbeat is present.

[0031] The authentication unit 152e performs user authentication based on the facial image determination result and the heart rate determination result. The authentication unit 152e determines that driver authentication is successful if it determines that the facial image matches a registered facial image and that the driver represented in the facial image is a biological entity. The authentication unit 152e determines that driver authentication is unsuccessful (authentication failed) if it determines that the facial image does not match a registered facial image, or that the driver represented in the facial image is not a biological entity.

[0032] The start permission unit 152f, upon determining that the facial image matches a registered facial image and that the driver represented in the facial image is a living person, authorizes the driver to start the vehicle. In other words, the start permission unit 152f authorizes the vehicle to start when the authentication unit 152e determines that authentication has been successful. As a result, when the driver operates the ignition switch 120, the vehicle starts.

[0033] The start authorization unit 152f will not allow the vehicle to start if the authentication unit 152e determines that user authentication has failed. In other words, the start authorization unit 152f will not allow the driver to start the vehicle if, at least one of the following occurs: the facial image is not determined to match the registered facial image, or the driver represented in the facial image is not determined to be a living person. In this case, even if the driver operates the ignition switch 120, the vehicle will not start. Therefore, authentication cannot be bypassed by a third party impersonating the user.

[0034] Figure 6 is a flowchart showing the processing performed by the processor 152 of the ECU 150 at predetermined control cycles. First, it is determined whether or not a face is shown in the image generated by the driver monitor camera 110 (step S10). If a face is shown in the image, the face image acquisition unit 152a acquires a face image from the image generated by the driver monitor camera 110 (step S12).

[0035] Next, the face image determination unit 152b compares the acquired face image with a pre-registered face image (step S14) and determines whether the face image and the registered face image match (step S16).

[0036] If the face image matches the registered face image in step S16 (if YES in step S16), the brightness change calculation unit 152c calculates the time change in the brightness value of the face image (step S18), and the biometric determination unit 152d performs frequency analysis on the time change in brightness value to calculate the intensity in the frequency domain of the heartbeat (step S20).

[0037] Then, if the intensity of the heart rate frequency range is above a threshold (if the answer is YES in step S22), the biometric detection unit 152d determines that the driver represented in the facial image is a living person (step S24), and the start permission unit 152f grants permission to start the vehicle (step S26).

[0038] As described above, according to this embodiment, since the driver monitor camera 110 generates a facial image, it is possible to perform biometric verification along with facial recognition of the driver, thereby reliably preventing the vehicle from being started by someone impersonating a person using a facial photograph or a mannequin. [Explanation of Symbols]

[0039] 150...Electronic control unit (ECU), 152...Processor, 152a...Face image acquisition unit, 152b...Face image determination unit, 152c...Brightness change calculation unit, 152d...Biometric determination unit, 152f...Start permission unit, 1000...Driving support system

Claims

1. A facial image acquisition unit acquires a facial image of the driver attempting to start the vehicle before the vehicle is started, A face image determination unit compares the aforementioned face image with a pre-registered registered face image and determines whether the aforementioned face image matches the registered face image. A brightness change calculation unit that calculates the time change in the brightness value of the face image, A biological determination unit that performs frequency analysis on the aforementioned time change to calculate the intensity in a predetermined frequency range, and determines that the driver represented in the facial image is a living organism if the intensity is above a predetermined threshold, When it is determined that the facial image matches the registered facial image, and that the driver represented in the facial image is a living person, the starting permission unit allows the driver to start the vehicle. A vehicle driver authentication device equipped with [the following features].

2. The vehicle driver authentication device according to claim 1, wherein the start permission unit does not permit the driver to start the vehicle in at least one of the following cases: when the face image is not determined to match the registered face image, or when the driver represented in the face image is not determined to be a living person.

3. The vehicle driver authentication device according to claim 1 or 2, wherein the predetermined frequency range is 0.7 Hz or more and 2 Hz or less.

Images