Equipment control devices, methods, and programs

Abstract

The present invention enables a user to operate specific equipment if two or more types of biometric authentication are successful, without reducing user convenience. An integrating means (13) determines whether or not a first user successfully authenticated in an authentication performed by a first authenticating means (11) in relation to the operation of first equipment (30) and a second user successfully authenticated in an authentication performed by a second authenticating means (12) in relation to the operation of second equipment (40) are the same person. If the users are determined to be the same person, the integrating means (13) determines that the second user is successfully authenticated in the first authenticating means (11).

Description

Technical Field

[0001] The present disclosure relates to an equipment control device, an authentication device, a method, and a computer-readable medium.

Background Art

[0002] In automobiles, technologies that enable the use of automobiles using biometric information instead of keys are known. As a related technology, Patent Document 1 discloses a vehicle authentication device. The vehicle authentication device described in Patent Document 1 can perform two or more types of biometric authentication. Specifically, the vehicle authentication device can perform face authentication as the first biometric authentication and vein authentication as the second biometric authentication. When the first biometric authentication is successful, the vehicle control ECU (Electric Control Unit) unlocks the vehicle door. When the second biometric authentication is successful, the engine ECU permits the engine to start.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Biometric authentication has the advantages of being difficult to steal or forge and having high convenience. Multimodal biometric authentication can improve security by combining multiple biometric authentications. For example, assume that the false acceptance rate of the first biometric authentication is 1 / 10,000 and the false acceptance rate of the second biometric authentication is 1 / 10,000. In that case, the probability that an unauthorized person passes both the first biometric authentication and the second biometric authentication simultaneously is 1 / 10,000 × 1 / 10,000 = 1 / 100,000,000.

[0005] <000D028>However, Patent Document 1 does not consider the possibility that the person who unlocks the doors and the person who starts the engine may be different. For example, when using a car, a person other than the driver may unlock the doors using facial recognition and get into the vehicle. In that case, a person other than the one who unlocked the doors may sit in the driver's seat and perform vein authentication. In this case, the driver has only performed vein authentication, and Patent Document 1 may allow the engine to start based on the result of only one type of biometric authentication. In other words, the driver's biometric authentication may not be multimodal.

[0006] In view of the above circumstances, this disclosure aims to provide a device control device, an authentication device, a method thereof, and a computer-readable medium that enable the operation of a specific device when two or more types of biometric authentication are successful. [Means for solving the problem]

[0007] To achieve the above objective, the Disclosure provides, in a first aspect, an equipment control device. The equipment control device includes control means for controlling a first device and a second device, a first authentication means for authenticating a user using first biometric information, a second authentication means for authenticating a user using second biometric information, and an integration means for integrating the authentication result of the first authentication means and the authentication result of the second authentication means. In the equipment control device, the control means permits operation of the first device if the user is successfully authenticated by the first authentication means, and permits operation of the second device if the user is successfully authenticated by both the first and second authentication means. The first authentication means performs user authentication in relation to the operation of the first device. The second authentication means performs user authentication in relation to the operation of the second device. The integration means determines whether the first user who was successfully authenticated in the authentication performed by the first authentication means in connection with the operation of the first device and the second user who was successfully authenticated in the authentication performed by the second authentication means in connection with the operation of the second device are the same person. If it determines that they are the same person, it determines that the second user was successfully authenticated by the first authentication means.

[0008] This disclosure provides, in a second aspect, an authentication device. The authentication device includes a first authentication means for authenticating a user using first biometric information in connection with the operation of a first device, a second authentication means for authenticating a user using second biometric information in connection with the operation of a second device, and an integration means for integrating the authentication result of the first authentication means and the authentication result of the second authentication means. The first device is a device whose operation is permitted when the first authentication means successfully authenticates. The second device is a device whose operation is permitted when the first authentication means successfully authenticates and the second authentication means successfully authenticates. In the authentication device, the integration means determines that the second user has been successfully authenticated by the first authentication means if the first user who has been successfully authenticated in the authentication performed by the first authentication means in connection with the operation of the first device and the second user who has been successfully authenticated in the authentication performed by the second authentication means in connection with the operation of the second device are the same person.

[0009] This disclosure provides, in a third aspect, a device control method. The device control method includes authenticating a user using first biometric information in connection with the operation of a first device, and if the authentication using the first biometric information is successful, permitting the operation of the first device; authenticating a user using second biometric information in connection with the operation of a second device, determining whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person, and if it is determined that the first user and the second user are the same person, permitting the operation of the second device.

[0010] This disclosure provides, in a fourth aspect, an authentication method. The authentication method includes authenticating a user using first biometric information in connection with the operation of a first device whose operation is permitted only if authentication using first biometric information is successful; authenticating a user using second biometric information in connection with the operation of a second device whose operation is permitted only if authentication using the first biometric information and authentication using second biometric information are successful; determining whether the first user who successfully performed authentication using the first biometric information in connection with the operation of the first device and the second user who successfully performed authentication using the second biometric information in connection with the operation of the second device are the same person; and determining that the first user and the second user are the same person, determining that the second user has succeeded in authentication using the first biometric information.

[0011] This disclosure provides, in a fifth aspect, a computer-readable medium. The computer-readable medium stores a program that causes a processor to execute a process including: authenticating a user using first biometric information in connection with the operation of a first device, and if the authentication using the first biometric information is successful, permitting the operation of the first device; authenticating a user using second biometric information in connection with the operation of a second device, determining whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person, and if it is determined that the first user and the second user are the same person, permitting the operation of the second device.

[0012] In a sixth aspect of this disclosure, a computer-readable medium is provided, the computer-readable medium stores a program for causing a processor to execute a process that includes: authenticating a user using first biometric information in connection with the operation of a first device whose operation is permitted upon successful authentication using first biometric information; authenticating a user using second biometric information in connection with the operation of a second device whose operation is permitted upon successful authentication using the first biometric information and authentication using second biometric information; determining whether a first user who has successfully undergone authentication using the first biometric information in connection with the operation of the first device and a second user who has successfully undergone authentication using the second biometric information in connection with the operation of the second device are the same person; and determining, if it is determined that the first user and the second user are the same person, determining that the second user has successfully undergone authentication using the first biometric information. [Effects of the Invention]

[0013] The device control device, authentication device, methods thereof, and computer-readable media related to this disclosure can allow a user to operate a specific device upon successful authentication of two or more types of biometric authentication. [Brief explanation of the drawing]

[0014] [Figure 1] A block diagram showing an example of the schematic configuration of the equipment control device related to this disclosure. [Figure 2] A block diagram showing an example of the configuration of a device control device according to one embodiment of this disclosure. [Figure 3] A flowchart showing the operation procedure of the device control unit when the door lock is released. [Figure 4] A flowchart showing the operating procedure of the equipment control system during engine startup. [Figure 5] A block diagram showing an example of the physical configuration of an equipment control device. [Modes for carrying out the invention]

[0015] Prior to describing embodiments of this disclosure, an overview of this disclosure will be provided. Figure 1 shows a schematic configuration of an equipment control device according to this disclosure. The equipment control device 10 includes a first authentication means 11, a second authentication means 12, an integration means 13, and a control means 14.

[0016] The first authentication means 11 authenticates the user using first biometric information. The second authentication means 12 authenticates the user using second biometric information. The integration means 13 integrates the authentication result of the first authentication means 11 and the authentication result of the second authentication means 12. The first authentication means 11, the second authentication means 12, and the integration means 13 constitute the authentication device 20.

[0017] The control means 14 controls the first device 30 and the second device 40. The control means 14 permits the operation of the first device 30 if the user successfully authenticates with the first authentication means 11. The control means 14 also permits the operation of the second device 40 if the user successfully authenticates with both the first authentication means 11 and the second authentication means 12.

[0018] The first authentication means 11 performs user authentication in relation to the operation of the first device 30. The second authentication means 12 performs user authentication when related to the operation of the second device 40. The integration means 13 determines whether the first user who succeeded in authentication in the authentication performed by the first authentication means 11 in relation to the operation of the first device 30 and the second user who succeeded in authentication in the authentication performed by the second authentication means 12 in relation to the operation of the second device 40 are the same person. When the integration means 13 determines that the first user and the second user are the same person, the integration means 13 determines that the second user has succeeded in authentication by the first authentication means 11.

[0019] In the present disclosure, when the first user authenticated during the operation of the first device 30 and the second user during the operation of the second device 40 are the same person, the integration means 13 determines that the second user is authenticated by the first authentication means 11. Even when the second user is not authenticated by the first authentication means 11 during the operation of the second device 40, the control means 14 permits the operation of the second device 40 because the second user has already been authenticated by the first authentication means 11 during the operation of the first device 30. When the first user and the second user are not the same person, the first authentication means 11 can perform authentication of the second user. When the second user is authenticated by the first authentication means 11, the control means 14 can permit the operation of the second device 40.

[0020] In the present disclosure, for example, when a user authenticated during the operation of the first device 30 operates the second device 40, authentication in the first authentication means 11 during the operation of the second device 40 can be omitted. Even in that case, since the user has successfully authenticated with the first authentication means 11 during the operation of the first device 30, the biometric authentication is multimodal. The device control apparatus 10 according to the present disclosure can allow the user to operate the second device 40 when two or more types of biometric authentication are successful. In the present disclosure, since a plurality of biometric authentications are used for the operation of the second device 40, the security can be improved. In the present disclosure, if the user has successfully authenticated in the first authentication means 11 during the operation of the first device 30, there is no need to perform the authentication in the first authentication means 11 again during the operation of the second device 40. Therefore, while being multimodal biometric authentication, the convenience of the user can be improved.

[0021] Hereinafter, embodiments of the present disclosure will be described in detail while referring to the drawings. Note that in the following description and drawings, for the sake of clarity of explanation, appropriate omissions and simplifications are made. Also, in each drawing, the same elements and similar elements are denoted by the same reference numerals, and duplicate explanations are omitted as necessary.

[0022] FIG. 2 shows a device control apparatus 100 according to an embodiment of the present disclosure. The device control apparatus 100 includes a face authentication unit 101, an iris authentication unit 102, an integration unit 103, a device control unit 104, and a tracking unit 105. In the device control apparatus 100, the face authentication unit 101, the iris authentication unit 102, the integration unit 103, and the tracking unit 105 constitute an authentication apparatus 120. The device control apparatus 100 is configured as a device having, for example, one or more processors and one or more memories. At least a part of the functions of each unit in the device control apparatus 100 can be realized by a processor executing processing according to a program read from a memory.

[0023] In this embodiment, the device control device 100 is, for example, mounted on a vehicle and controls the operation of multiple devices in the vehicle. The vehicle is a mobile vehicle such as a passenger car, taxi, or van. In this embodiment, the vehicle can be used by multiple users. The controlled devices controlled by the device control device 100 include, for example, devices installed in the vehicle or devices operated by the occupants when driving the vehicle. For example, the controlled devices include devices such as the engine (main switch), power windows, horn, door locks, and air conditioner. The controlled devices may also include in-vehicle equipment such as a display audio (DA) and a car navigation system.

[0024] In this embodiment, the devices operated by the device control unit 100 include a door lock control unit 130 and a vehicle control ECU 140. The door lock control unit 130 controls the locks provided on each door of the vehicle. The door lock control unit 130 may, for example, lock or unlock each door individually, or lock or unlock all doors at once. The vehicle control ECU 140 controls the devices related to the vehicle's operation. The vehicle control ECU 140 can start the vehicle's engine or activate the vehicle's driving system. The device control unit 100 corresponds to the device control unit 10 shown in Figure 1. The door lock control unit 130 and the vehicle control ECU 140 correspond to the first device 30 and the second device 40 shown in Figure 1, respectively.

[0025] In this embodiment, the device control unit 100 controls the operation of the controlled device using multiple biometric authentication methods. In this embodiment, facial recognition and iris recognition are used as authentication methods using biometric information. If the user successfully performs one type of biometric authentication, for example, facial recognition, the device control unit 100 permits the operation of devices in the vehicle that are not related to driving. Furthermore, if the user successfully performs two types of biometric authentication, for example, facial recognition and iris recognition, the device control unit 100 permits the operation of devices in the vehicle that are related to driving the vehicle. In the following, if facial recognition is successful, the device control unit 100 permits the operation of the door lock control unit 130, and if both facial recognition and iris recognition are successful, it permits the operation of the vehicle control ECU 140.

[0026] The biometric authentication used in the device control unit 100 is not limited to facial recognition and iris recognition. The device control unit 100 may use biometric authentication selected from fingerprints, palm prints, finger vein patterns, and ear acoustics to control the operation of the controlled device.

[0027] The face image acquisition unit 200 acquires a face image as the user's first biometric information. The face image acquisition unit 200 includes, for example, a camera. The face image acquisition unit 200 includes, for example, a camera mounted in a vehicle that photographs a user outside the vehicle from inside the vehicle. The face image acquisition unit 200 also includes a camera inside the vehicle that photographs, for example, a user sitting in the driver's seat. The face image acquisition unit 200 may also include a camera positioned near the vehicle's parking space that photographs the vehicle from the outside. The face recognition unit 101 acquires a face image from the face image acquisition unit 200.

[0028] The facial recognition unit 101 performs facial recognition on the user's facial image. The facial recognition unit 101 performs facial recognition in connection with the user's operation to unlock the door lock. Here, "in connection with the user's operation to unlock the door lock" includes, for example, the user performing the operation to unlock the door lock, the user attempting to unlock the door lock, and the user approaching the door to unlock it. The facial recognition unit 101 performs facial recognition, for example, when the user performs the operation to unlock the door lock. In other words, the facial recognition unit 101 performs facial recognition when the user operates the door lock control unit 130. The facial recognition unit 101 may perform facial recognition if a face is detected within the shooting range of the facial image acquisition unit 200. Alternatively, the facial recognition unit 101 may perform facial recognition if a face is detected at a position within a certain distance from the facial image acquisition unit 200.

[0029] For example, the facial recognition unit 101 has one or more registered users registered as users who can perform the door lock release operation. When the door lock release operation is performed, the facial recognition unit 101 compares the user's facial image acquired by the facial image acquisition unit 200 with the facial images of the registered users that have been registered in advance. The facial recognition unit 101 calculates a comparison score between the user's facial image and the registered user's facial image. The comparison score indicates the similarity between the user's facial image and the registered user's facial image. The facial recognition unit 101 determines that facial recognition was successful if the comparison score meets a predetermined criterion, for example, if the comparison score is equal to or greater than a predetermined judgment threshold. The facial recognition unit 101 also identifies the registered user with the highest comparison score as the user who performed the door lock release operation. The facial recognition unit 101 corresponds to the first authentication means 11 shown in Figure 1.

[0030] The iris image acquisition unit 210 acquires an iris image as the user's second biometric information. The iris image acquisition unit 210 includes, for example, a near-infrared light source and a near-infrared camera. The near-infrared light source and the near-infrared camera are arranged, for example, inside a vehicle, in front of the driver's seat. The iris image acquisition unit 210 acquires, for example, an iris image of a user sitting in the driver's seat.

[0031] The iris authentication unit 102 acquires an iris image from the iris image acquisition unit 210. The iris authentication unit 102 performs iris authentication on the user's iris image. The iris authentication unit 102 performs iris authentication in connection with the user's operation of starting the engine or starting the driving system. Here, "in connection with the user's operation of starting the engine or starting the driving system" includes, for example, the user performing the engine start operation, the user attempting to perform the engine start operation, and the user sitting in the driver's seat. The iris authentication unit 102 performs iris authentication when, for example, the user starts the engine or starts the driving system. In other words, the iris authentication unit 102 performs iris authentication when the user operates the vehicle control ECU 140. The iris authentication unit 102 may perform iris authentication if a face or eyes are detected within the shooting range of the iris image acquisition unit 210. Alternatively, the iris authentication unit 102 may perform iris authentication if a face or eyes are detected at a position within a certain distance from the face image acquisition unit 200.

[0032] For example, the iris authentication unit 102 has one or more registered users registered as users who can perform the engine start operation. When the engine start operation is performed, the iris authentication unit 102 compares the iris image of the user acquired by the iris image acquisition unit 210 with the iris images of the registered users that have been registered in advance. The iris authentication unit 102 calculates a comparison score between the user's iris image and the registered user's iris image. The comparison score indicates the similarity between the user's iris image and the registered user's iris image. If the comparison score meets a predetermined standard, for example, if the comparison score is equal to or greater than a predetermined judgment threshold, the iris authentication unit 102 determines that iris authentication has been successful. The iris authentication unit 102 also identifies the registered user with the highest comparison score as the user who performed the engine start operation. The iris authentication unit 102 corresponds to the second authentication means 12 shown in Figure 1.

[0033] The integration unit 103 integrates the authentication results of facial recognition and iris recognition. The integration unit 103 determines whether the first user who was successfully authenticated by the facial recognition unit 101 when the door lock release operation is performed and the second user who was successfully authenticated by the iris recognition unit 102 when the engine start operation is performed are the same person. For example, the integration unit 103 determines whether the first registered user who was successfully authenticated by the facial recognition unit 101 and whose matching score meets a predetermined standard and the second registered user who was successfully authenticated by the iris recognition unit 102 and whose matching score meets a predetermined standard represent the same user. If the first registered user and the second registered user represent the same user, the integration unit 103 determines that the first user and the second user are the same person. If the integration unit 103 determines that they are the same person, it determines that the second user was successfully authenticated by the facial recognition unit 101. If the integration unit 103 determines that the individuals are not the same person, it instructs the facial recognition unit 101 to perform facial recognition on a second user, i.e., a user who has performed the engine start operation and whose authentication has been successfully performed by the iris recognition unit 102.

[0034] The integration unit 103 may determine whether the first user and the second user are the same person if iris authentication is performed by the iris authentication unit 102 within a predetermined time after facial authentication is performed by the facial authentication unit 101. The integration unit 103 may have the facial authentication unit 101 perform user authentication without determining whether the first user and the second user are the same person if iris authentication is performed after a predetermined time has elapsed since facial authentication.

[0035] Furthermore, the integration unit 103 may determine whether the first user and the second user are the same person, depending on whether the user attempted to fraudulently bypass the facial recognition process at the facial recognition unit 101. Whether the user attempted to fraudulently bypass the facial recognition process can be determined, for example, from the user's posture and possessions. Specifically, if a user points a photograph of another person's face, i.e., a registered user's face, at the facial image acquisition unit 200, that user is considered a fraudulent user. In that case, the integration unit 103 may determine that the first user and the second user are not the same person. The integration unit 103 corresponds to the integration means 13 shown in Figure 1.

[0036] The in-vehicle camera 150 is a camera that captures users outside and inside the vehicle. The in-vehicle camera 150 is positioned to capture, for example, users outside the driver's side door and users sitting in the driver's seat. The tracking unit 105 acquires camera images from the in-vehicle camera 150 and tracks user actions in the camera images. For example, the tracking unit 105 tracks the movements of a user who has successfully performed facial recognition and unlocked the door. The tracking unit 105 corresponds to the tracking means.

[0037] The integration unit 103 may, instead of, or in addition to, the method for determining whether the first user and the second user are the same person as described above, determine whether the first user and the second user are the same person based on the user's actions tracked by the tracking unit 105. For example, the integration unit 103 may determine whether the user has moved to a location where the iris image acquisition unit 210 acquires an iris image, such as the driver's seat, after successfully performing facial recognition. The integration unit 103 may determine that the first user and the second user are the same person if the user who has successfully performed facial recognition has moved to the driver's seat and the iris authentication unit 102 performs iris authentication.

[0038] The device control unit 104 permits the door lock release operation if the user successfully performs facial recognition using the facial recognition unit 101. The device control unit 104 restricts the operation of the door lock control unit 130, for example, until the user successfully performs facial recognition. When the device control unit 104 successfully performs facial recognition, it receives a signal indicating that the door lock can be released to the door lock control unit 130. When the door lock control unit 130 receives the signal indicating that the door lock can be released, it releases the door lock, for example, if the user touches the door handle or presses a button located near the door handle.

[0039] Furthermore, the device control unit 104 permits the engine start operation if the user has successfully performed facial recognition in the facial recognition unit 101 and iris recognition in the iris recognition unit 102. The device control unit 104 restricts the operation of the vehicle control ECU 140 until the user has successfully performed both facial recognition and iris recognition. In this embodiment, the user must have successfully performed both iris recognition and facial recognition when starting the engine. The user may have successfully performed facial recognition when unlocking the doors, or may have successfully performed facial recognition when starting the engine. If the user has successfully performed both facial recognition and iris recognition, the device control unit 104 transmits a signal to the vehicle control ECU 140 indicating that the engine can be started. The vehicle control ECU 140 starts the engine when the user presses the engine start button located near the driver's seat. The device control unit 104 corresponds to the control means 14 shown in Figure 1.

[0040] Next, the operation procedure of the device control device 100, including the device control method and authentication method, will be described. Figure 3 shows the operation procedure of the device control device 100 in the case of unlocking the door. The face image acquisition unit 200 acquires the user's face image (step A1). The face image acquisition unit 200 acquires, for example, the face image of a person approaching the vehicle door and transmits the acquired face image to the face authentication unit 101. The face authentication unit 101 performs face authentication on the face image acquired in step A1 (step A2). In step A2, the face authentication unit 101 compares, for example, the face image acquired in step A1 with the registered person's face image and calculates a comparison score for each registered person's face image.

[0041] The facial recognition unit 101 determines whether or not facial recognition was successful (step A3). For example, if the matching score is above the judgment threshold, the facial recognition unit 101 determines that facial recognition was successful. If facial recognition is successful, the facial recognition unit 101 identifies which of the registered users the user is. If the matching score is below the judgment threshold, the facial recognition unit 101 determines that facial recognition failed. If it is determined in step A3 that facial recognition failed, the process returns to step A1 and the facial image is acquired.

[0042] If facial recognition is determined to be successful in step A3, the integration unit 103 transmits an authentication result to the device control unit 104 indicating that the user has successfully performed facial recognition. If the device control unit 104 confirms that the user has successfully performed facial recognition, it transmits a signal to the door lock control unit 130 indicating that the door lock can be released. When the door lock control unit 130 receives the signal indicating that the door lock can be released, it releases the door lock in response to the user's operation (step A4).

[0043] Figure 4 shows the operation procedure of the equipment control device 100 when the engine is started. The iris image acquisition unit 210 acquires the user's iris image (step B1). The iris image acquisition unit 210 acquires the iris image of, for example, a person sitting in the driver's seat and transmits the acquired iris image to the iris authentication unit 102. The iris authentication unit 102 performs iris authentication on the iris image acquired in step B1 (step B2). In step B2, the iris authentication unit 102 compares the iris image acquired in step B1 with the registered user's iris image and calculates a comparison score for each registered user's iris image.

[0044] The iris authentication unit 102 determines whether iris authentication was successful or not (step B3). For example, if the matching score is above the judgment threshold, the iris authentication unit 102 determines that iris authentication was successful. If iris authentication is successful, the iris authentication unit 102 identifies which of the registered users the user is. If the matching score is below the judgment threshold, the iris authentication unit 102 determines that iris authentication failed. If it is determined in step B3 that iris authentication failed, the process returns to step B1 and the iris image is acquired.

[0045] If iris authentication is determined to be successful in step B3, the integration unit 103 determines whether the user who successfully performed facial authentication to unlock the door and the user who successfully performed iris authentication in step B2 are the same person (step B4). If the integration unit 103 determines that they are the same person, it transmits an authentication result to the equipment control unit 104 indicating that the user has succeeded in both facial authentication and iris authentication. If the user has succeeded in both facial authentication and iris authentication, the equipment control unit 104 transmits a signal to the vehicle control ECU 140 indicating that the engine can be started (step B9). When the vehicle control ECU 140 receives the signal indicating that the engine can be started, it starts the engine in response to user operation.

[0046] If the person is not determined to be the same person in step B4, the integration unit 103 instructs the facial recognition unit 101 to perform facial recognition. In this case, the facial image acquisition unit 200 acquires the user's facial image (step B5). In step B5, the facial image acquisition unit 200 acquires the facial image of the person sitting in the driver's seat and transmits the acquired facial image to the facial recognition unit 101. The facial recognition unit 101 performs facial recognition on the facial image acquired in step B5 (step B6).

[0047] The facial recognition unit 101 determines whether or not facial recognition was successful (step B7). For example, if the matching score is above the judgment threshold, the facial recognition unit 101 determines that facial recognition was successful. If facial recognition is successful, the facial recognition unit 101 identifies which of the registered users the user is. If the matching score is below the judgment threshold, the facial recognition unit 101 determines that facial recognition failed. If it is determined in step A3 that facial recognition failed, the process returns to step B5 and the facial image is acquired.

[0048] If facial recognition is determined to be successful in step B7, the integration unit 103 determines whether the user who was successfully authenticated by iris recognition in step B2 and the user who was successfully authenticated by facial recognition in step B6 are the same person (step B8). If the integration unit 103 determines that they are the same person, it sends an authentication result to the equipment control unit 104 indicating that the user has succeeded in both facial recognition and iris recognition. The process then proceeds to step B9, where the equipment control unit 104 sends a signal to the vehicle control ECU 140 indicating that the engine can be started.

[0049] If the integration unit 103 determines in step B8 that the users are not the same person, it sends a signal indicating authentication failure to the equipment control unit 104 (step B10). In this case, the equipment control unit 104 does not send a signal to the vehicle control ECU 140 indicating that the engine can be started, and the vehicle control ECU 140 does not permit the user to start the engine.

[0050] In this embodiment, the facial recognition unit 101 authenticates the user using facial recognition when the user unlocks the door lock. The device control unit 104 transmits a signal to the door lock control unit 130 indicating that the door lock can be unlocked if the user's facial recognition by the facial recognition unit 101 is successful. The iris recognition unit 102 also authenticates the user using iris recognition when the user starts the engine. The integration unit 103 determines whether the user who successfully performed iris recognition is the same person as the user who successfully performed facial recognition to unlock the door lock. If the integration unit 103 determines that they are the same person, it determines that the user starting the engine has also successfully performed facial recognition. In this case, the device control unit 104 transmits a signal to the vehicle control ECU 140 indicating that the engine can be started.

[0051] If the results of facial recognition and iris recognition are not integrated, for example, a user other than the driver could unlock the doors using facial recognition, and then the driver could perform iris recognition to start the engine. However, in that case, the driver would only have performed iris recognition, and the biometric authentication would not be multimodal. On the other hand, if facial recognition is required when starting the engine, the driver who unlocked the doors using facial recognition would also need to perform facial recognition again, which would reduce user convenience.

[0052] In this embodiment, the integration unit 103 determines that if the user who successfully performs facial recognition when unlocking the door and the user who successfully performs iris recognition when starting the engine are the same person, that user has succeeded in both biometric authentications. In that case, facial recognition at engine start can be omitted. In this embodiment, facial recognition is performed at engine start only if the users are not the same person, thereby preventing duplicate facial recognition for a single person. In this embodiment, it is possible to guarantee that the user has succeeded in multiple types of biometric authentication at engine start, and while avoiding a decrease in user convenience, the user can be allowed to start the engine if two biometric authentications are successful.

[0053] In the above embodiment, an example was described in which the device control unit 100 performs all of facial recognition, iris recognition, and device control. However, this disclosure is not limited to this. The device control unit 100 does not necessarily have to be a single device. The device control unit 100 may be composed of multiple physically separated devices. Some of the functions of the device control unit 100 may be performed by other devices, such as a cloud server. For example, the functions of the facial recognition unit 101 and the iris recognition unit 102 may be located on a cloud server, and the integration unit 103 may obtain the authentication results from the cloud server.

[0054] In the above embodiment, an example was described in which the device control unit 104 controls door unlocking and engine starting using the authentication result. However, this disclosure is not limited thereto. The device control unit 104 may control multiple devices with different numbers of required authentications, and the controlled devices are not limited to specific devices. Furthermore, the controlled devices are not limited to vehicle devices.

[0055] Next, the hardware configuration of the device control unit 100 will be described. Figure 5 shows an example of the hardware configuration of the device control unit 100. The device control unit 100 has a processor (CPU: Central Processing Unit) 501, a ROM (read-only memory) 502, and a RAM (random access memory) 503. In the device control unit 100, the processor 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. The device control unit 100 may include other circuits such as peripheral circuits, communication circuits, and interface circuits, although these are not shown in the figure.

[0056] ROM 502 is a non-volatile memory device. A semiconductor memory device, such as a relatively small-capacity flash memory, is used for ROM 502. ROM 502 stores the program executed by the processor 501.

[0057] The above program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include RAM, ROM, flash memory, solid-state drive (SSD) or other memory technologies, Compact Disc (CD), digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include, a temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0058] RAM503 is a volatile memory device. Various semiconductor memory devices, such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory), can be used for RAM503. RAM503 can be used as an internal buffer for temporarily storing data.

[0059] The processor 501 loads the program stored in the ROM 502 into the RAM 503 and executes the program. By executing the program, the CPU 501 can realize the functions of each part within the device control unit 100.

[0060] Although embodiments of this disclosure have been described in detail above, this disclosure is not limited to the embodiments described above, and modifications and changes to the embodiments described above, without departing from the spirit of this disclosure, are also included in this disclosure.

[0061] For example, some or all of the above embodiments may also be described as follows, but are not limited to the following.

[0062] [Note 1] Control means for controlling the first device and the second device, A first authentication means that authenticates the user using first biometric information, A second authentication method that authenticates the user using second biometric information, The system includes an integration means for integrating the authentication result of the first authentication means and the authentication result of the second authentication means, The control means permits operation of the first device if the user is successfully authenticated by the first authentication means, and permits operation of the second device if the user is successfully authenticated by both the first and second authentication means. The first authentication means performs user authentication in connection with the operation of the first device, The second authentication means performs user authentication in connection with the operation of the second device, The integration means determines whether a first user who was successfully authenticated in an authentication performed by the first authentication means in connection with the operation of the first device and a second user who was successfully authenticated in an authentication performed by the second authentication means in connection with the operation of the second device are the same person, and if it is determined that they are the same person, the device control device determines that the second user has been successfully authenticated by the first authentication means.

[0063] [Note 2] The device control device according to Appendix 1, wherein the integration means causes the first authentication means to perform user authentication if the first user and the second user are not the same person.

[0064] [Note 3] The equipment control device described in Appendix 1 or 2, wherein the first equipment is equipment related to the operation of the vehicle other than driving, and the second equipment is equipment related to the operation of the vehicle.

[0065] [Note 4] The first authentication means is an equipment control device according to any one of the appendices 1 to 3, which authenticates the user using first biometric information acquired outside the vehicle.

[0066] [Note 5] The equipment control device described in any one of the appendices 1 to 4, wherein the second authentication means authenticates the user using second biometric information acquired inside the vehicle.

[0067] [Note 6] The first authentication means compares the first biometric information obtained from a user associated with the operation of the first device with the first biometric information of one or more first registered users who are registered as users capable of operating the first device, and determines that authentication has been successful if the comparison score indicating the similarity between the first biometric information obtained from the user and the first biometric information of the first registered users meets a predetermined standard. The second authentication means compares the second biometric information obtained from the user associated with the operation of the second device with the second biometric information of one or more second registered users who are registered as users capable of operating the second device, and determines that authentication has been successful if the comparison score indicating the similarity between the second biometric information obtained from the user and the second biometric information of the second registered users meets a predetermined standard. The integration means determines that the first user and the second user are the same person when the first registrant whose matching score in the first authentication means meets a predetermined standard and the second registrant whose matching score in the second authentication means meets a predetermined standard indicate the same user. This is the device control device according to any one of the appendices 1 to 5.

[0068] [Note 7] The integration means determines whether the first user and the second user are the same person if authentication is performed by the second authentication means within a predetermined time after authentication is performed by the first authentication means, as described in any one of the appendices 1 to 6.

[0069] [Note 8] The system further includes tracking means for tracking the actions of the first user, The integration means determines that the first user and the second user are the same person when the first user moves to a location where the second biometric information is acquired and the second authentication means authenticates the user, as described in any one of Appendix 1 to 7.

[0070] [Note 9] The integration means determines whether the first user and the second user are the same person, depending on whether the user attempted to fraudulently bypass authentication in the first authentication means, as described in any one of the appendices 1 to 8.

[0071] [Note 10] The device control device according to any one of the appendices 1 to 9, wherein the control means restricts the operation of the first device until the user is successfully authenticated by the first authentication means, and restricts the operation of the second device until the user is successfully authenticated by both the first authentication means and the second authentication means.

[0072] [Note 11] A first authentication means for authenticating a user using first biometric information in connection with the operation of the first device, In connection with the operation of the second device, a second authentication means for authenticating the user using second biometric information, The system includes an integration means for integrating the authentication result of the first authentication means and the authentication result of the second authentication means, The first device is a device whose operation is permitted when the first authentication means successfully authenticates, and the second device is a device whose operation is permitted when the first authentication means successfully authenticates and the second authentication means successfully authenticates, The integration means is an authentication device that determines that the second user has been successfully authenticated by the first authentication means if the first user who has been successfully authenticated in authentication performed by the first authentication means in connection with the operation of the first device and the second user who has been successfully authenticated in authentication performed by the second authentication means in connection with the operation of the second device are the same person.

[0073] [Note 12] The authentication device described in Appendix 11, wherein the integration means causes the first authentication means to perform user authentication if the first user and the second user are not the same person.

[0074] [Note 13] The first authentication means is an authentication device according to Appendix 11 or 12, which authenticates the user using first biometric information acquired outside the vehicle.

[0075] [Note 14] The second authentication means is an authentication device according to any one of the appendices 11 to 13, which authenticates the user using second biometric information acquired inside the vehicle.

[0076] [Note 15] In connection with the operation of the first device, the user is authenticated using the first biometric information, If authentication using the first biometric information is successful, the operation of the first device is permitted. In connection with the operation of the second device, the user is authenticated using the second biometric information. It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person. A device control method that, when it is determined that the first user and the second user are the same person, allows the operation of the second device.

[0077] [Note 16] In relation to the operation of a first device, which is authorized to be operated only if authentication using the first biometric information is successful, the user is authenticated using the first biometric information, In relation to the operation of a second device whose operation is permitted upon successful authentication using the first biometric information and authentication using the second biometric information, the user is authenticated using the second biometric information, It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person. An authentication method comprising determining that the first user and the second user are the same person, and that the second user has successfully authenticated using the first biometric information.

[0078] [Note 17] In connection with the operation of the first device, the user is authenticated using the first biometric information, If authentication using the first biometric information is successful, the operation of the first device is permitted. In connection with the operation of the second device, the user is authenticated using the second biometric information. It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person. A non-temporary computer-readable medium that stores a program causing a processor to perform a process including authorizing the operation of the second device, if it is determined that the first user and the second user are the same person.

[0079] [Note 18] In relation to the operation of a first device, which is authorized to be operated only if authentication using the first biometric information is successful, the user is authenticated using the first biometric information, In relation to the operation of a second device whose operation is permitted upon successful authentication using the first biometric information and authentication using the second biometric information, the user is authenticated using the second biometric information, It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device are the same person. A non-temporary computer-readable medium that stores a program causing a processor to perform a process including determining that the second user has succeeded in authentication using the first biometric information, if it is determined that the first user and the second user are the same person. [Explanation of symbols]

[0080] 10: Equipment control device 11: First Authentication Method 12: Second Authentication Method 13: Integration methods 14: Control means 20: Authentication device 30: First device 40: Second device 100: Equipment control device 101: Facial Recognition Department 102: Iris Recognition Department 103: Integration Department 104: Equipment Control Unit 105: Tracking Department 120: Authentication device 130: Door lock control unit 140: Vehicle control ECU 150: In-car camera 200: Face image acquisition unit 210: Iris image acquisition unit

Claims

1. Control means for controlling the first device and the second device, A first authentication means that authenticates the user using first biometric information, A second authentication method that authenticates the user using second biometric information, The system includes an integration means for integrating the authentication result of the first authentication means and the authentication result of the second authentication means, The control means permits operation of the first device if the user is successfully authenticated by the first authentication means, and permits operation of the second device if the user is successfully authenticated by both the first and second authentication means. The first authentication means performs user authentication in connection with the operation of the first device, The second authentication means performs user authentication in connection with the operation of the second device, The integration means determines whether a first user who was successfully authenticated in an authentication performed by the first authentication means in connection with the operation of the first device and a second user who was successfully authenticated in an authentication performed by the second authentication means in connection with the operation of the second device performed after the operation of the first device are the same person, and if it is determined that they are the same person, the device control device determines that the second user has been successfully authenticated by the first authentication means.

2. The device control device according to claim 1, wherein the integration means causes the first authentication means to perform user authentication if the first user and the second user are not the same person.

3. The device control device according to claim 1 or 2, wherein the first authentication means authenticates the user using first biometric information obtained from the user who is located outside the vehicle.

4. The device control device according to any one of claims 1 to 3, wherein the second authentication means authenticates the user using second biometric information obtained from the user who is inside the vehicle.

5. Computers In connection with the operation of the first device, the user is authenticated using the first biometric information, If authentication using the first biometric information is successful, the operation of the first device is permitted. In connection with the operation of the second device, the user is authenticated using the second biometric information. It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device performed after the operation of the first device are the same person. A device control method that, when it is determined that the first user and the second user are the same person, allows the operation of the second device.

6. In connection with the operation of the first device, the user is authenticated using the first biometric information, If authentication using the first biometric information is successful, the operation of the first device is permitted. In connection with the operation of the second device, the user is authenticated using the second biometric information. It is determined whether the first user who successfully authenticated using the first biometric information in connection with the operation of the first device and the second user who successfully authenticated using the second biometric information in connection with the operation of the second device performed after the operation of the first device are the same person. A program that causes a processor to execute a process that includes authorizing the operation of the second device when it is determined that the first user and the second user are the same person.

Images