Authentication apparatus, authentication method, imaging apparatus and control method thereof, computer program product, and storage medium

Abstract

The invention provides an authentication apparatus, an authentication method, an imaging apparatus and a control method thereof, a computer program product, and a storage medium. The authentication apparatus includes: a detection unit configured to detect a subject from an input image; a comparison unit configured to compare the subject detected by the detection unit with a pre-registered authentication object, and output a comparison score indicating a degree of similarity between the subject and the authentication object; an updating unit configured to update an authentication score, which is an evaluation value for indicating a degree to which the subject and the authentication target coincide, based on the contrast score; and an authentication unit configured to authenticate the subject based on the authentication score, in which the update unit changes an update method of the authentication score based on a magnitude relationship between the contrast score and the authentication score.

Description

Technical Field

[0001] The present invention relates to an authentication device for authenticating a person. Background Art

[0002] Conventionally, imaging device products such as digital cameras having a tracking autofocus (AF) mode have been put into practical use. The tracking AF mode is a mode in which a person's face or pupil is detected from an image continuously output from an imaging element, and the focus state and exposure state are continuously optimized for the detected person's face or pupil. In addition, Japanese Patent Application Laid-Open No. 2008-187591 describes a technique for registering a person in advance in order to select a desired tracking object from among a plurality of subjects and using face authentication to select the registered person.

[0003] Generally, in face authentication registration processing, feature information is extracted from a face image and stored in a non-volatile memory. At the time of imaging, a face is detected in an image continuously output from the imaging element, and feature information of the detected face is extracted. The similarity between the extracted feature information and the feature information stored in the non-volatile memory is compared to determine whether the person is registered.

[0004] In recent years, the use of deep learning as an algorithm for face authentication has become widespread, and while the performance has been improved, the processing load has also increased. In particular, in devices using embedded software with limited resources (such as digital cameras), it is difficult to perform high-processing-load authentication processing in real time using the limited resources.

[0005] Therefore, as a method for efficiently performing authentication processing using limited resources, Japanese Patent No. 5963525 proposes a method of inheriting an authentication state by using tracking without performing a comparison process after a subject has been successfully authenticated once.

[0006] However, in the method of inheriting an authentication state by using tracking, it is conceivable that another subject may be erroneously tracked due to subject interleaving or crowding, etc., and in such a case, the authentication state will ultimately be inherited by another person ("erroneous inheritance state"). For this reason, even if a subject has been successfully authenticated once, it is necessary to perform a comparison again as needed, and it is necessary to cancel the authentication state in the case of an "erroneous inheritance state". On the other hand, if a comparison is performed again, even if the subject should be kept in the authentication state, the comparison score may decrease depending on the state, and there is a possibility that the person himself / herself may be rejected. Summary of the Invention

[0007] The present invention has been made in view of the above problems, and aims to prevent the authentication status of others from being wrongly inherited and prevent the user himself / herself from being rejected due to re - authentication when authenticating a person.

[0008] According to a first aspect of the present invention, there is provided an authentication device including: a detection unit configured to detect a subject from an input image; a comparison unit configured to compare the subject detected by the detection unit with a pre - registered authentication target and output a comparison score indicating a similarity between the subject and the authentication target; an update unit configured to update an authentication score based on the comparison score, the authentication score being an evaluation value indicating the degree of coincidence between the subject and the authentication target; and an authentication unit configured to authenticate the subject based on the authentication score, wherein the update unit changes an update method of the authentication score based on a magnitude relationship between the comparison score and the authentication score.

[0009] According to a second aspect of the present invention, there is provided an authentication method including: detecting a subject from an input image; comparing the detected subject with a pre - registered authentication target and outputting a comparison score indicating a similarity between the subject and the authentication target; updating an authentication score based on the comparison score, the authentication score being an evaluation value indicating the degree of coincidence between the subject and the authentication target; and authenticating the subject based on the authentication score, wherein in the updating, an update method of the authentication score is changed based on a magnitude relationship between the comparison score and the authentication score.

[0010] According to a third aspect of the present invention, there is provided a photographing device including: a detection unit configured to detect a subject from a photographed image; a comparison unit configured to compare the subject detected by the detection unit with a pre - registered authentication target and output a comparison score indicating a similarity between the subject and the authentication target; an update unit configured to update an authentication score based on the comparison score, the authentication score being an evaluation value indicating the degree of coincidence between the subject and the authentication target; and a determination unit configured to determine a main subject based on the authentication score, wherein the update unit changes an update method of the authentication score based on a magnitude relationship between the comparison score and the authentication score.

[0011] According to a fourth aspect of the present invention, there is provided a control method for a photographing apparatus, including: detecting a subject from a captured image; comparing the detected subject with a pre-registered authentication object and outputting a comparison score indicating a similarity between the subject and the authentication object; updating an authentication score based on the comparison score, the authentication score being an evaluation value indicating a degree of coincidence between the subject and the authentication object; and determining a main subject based on the authentication score, wherein, in the updating, a method of updating the authentication score is changed based on a magnitude relationship between the comparison score and the authentication score.

[0012] According to a fifth aspect of the present invention, there is provided a computer program product including a program for causing a computer to execute the respective steps of the above method.

[0013] According to a sixth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing a program for causing a computer to execute the respective steps of the above method.

[0014] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 FIG. is a diagram illustrating the configuration of a digital single-lens camera as an embodiment of the photographing apparatus of the present invention.

[0016] Figure 2 FIG. is a block diagram of a control system of the camera.

[0017] Figure 3 FIG. is a flowchart for describing control from when an image is acquired until a main subject is determined.

[0018] Figure 4 FIG. is a flowchart for describing in detail comparison processing.

[0019] Figure 5 FIG. is a diagram illustrating a specific example of a scenario in which a person can be prevented from being rejected.

[0020] Figure 6 FIG. is a diagram illustrating a specific example of a scenario in which an authentication state of another person can be prevented from being erroneously inherited. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but the invention is not limited to an invention that requires all such features, and multiple such features can be appropriately combined. Further, in the drawings, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof are omitted.

[0022] Figure 1 FIG. 3 is a diagram illustrating the configuration of a digital single-lens camera (hereinafter also simply referred to as a camera) 100 as an embodiment of the imaging device of the present invention. Figure 2 FIG. 4 is a diagram illustrating a configuration related to the control of the digital single-lens camera 100.

[0023] In the camera 100 of the present embodiment, as Figure 1 illustrated in FIG. 5, a detachable interchangeable lens unit 120 is mounted on the front side (subject side) of the camera body 101. The lens unit 120 includes a focusing lens 121, an aperture 122, etc., and is electrically connected to the camera body 101 via a mounting contact unit 123. Through this electrical connection, the control unit 201 (refer to Figure 2 ) of the camera body 101 can control the lens unit 120 to adjust the amount of light entering the camera body 101 and the focus position. Note that the focusing lens 121 can be manually adjusted by the user.

[0024] The imaging element 104 for capturing a subject image is composed of a CCD, a CMOS sensor, etc., and includes an infrared cut-off filter, a low-pass filter, etc. The imaging element 104 performs photoelectric conversion on the subject image formed by the light passing through the imaging optical system of the lens unit 120 during imaging, and transmits signal information for generating a captured image to the computing device 102. The computing device 102 generates a captured image based on the received signal information, stores the captured image in an external storage device 107 (refer to Figure 2 ), and displays the captured image on a display unit 105 such as an LCD. The shutter 103 shields the imaging element 104 during non-imaging, and opens during imaging to expose the imaging element 104.

[0025] Next, the configuration related to the control of the camera 100 will be described with reference to Figure 2 FIG. 6.

[0026] The computing device 102 includes a multi-core CPU capable of parallel processing of multiple tasks, a RAM, a ROM, and a dedicated circuit for high-speed execution of specific computational processing, etc. The computing device 102 includes a control unit 201, a main subject calculation unit 202 for detecting a subject, a tracking calculation unit 203, a focusing calculation unit 204, and an exposure calculation unit 205, etc. The control unit 201 controls each unit of the camera body 101 and the lens unit 120.

[0027] The main subject calculation unit 202 includes a subject detector 211 for detecting a subject, a main subject determination unit 212 for performing authentication and main subject determination, a reference object selection unit 213 for selecting a reference object, an authentication score update unit 214 for updating the authentication score, and a comparison unit 215 for performing comparison.

[0028] The consecutive images acquired from the imaging element 104 are sequentially input to the subject detector 211, and a process for detecting one or more subject regions in each image is performed. In the present embodiment, the target subject is a human face. As the detection method, any known method such as AdaBoost or a method using a convolutional neural network can be used. In addition, the detection method can be implemented as a program running on a CPU, implemented as dedicated hardware, or implemented as a combination of a program and dedicated hardware. The subject detection result obtained from the subject detector 211 is sent to the main subject determination unit 212 and the reference object selection unit 213.

[0029] The reference object selection unit 213 selects a reference object from the detected subjects. The number of subjects to be compared at one time can be determined by the processing speed of the comparison unit 215 and the frame rate of the consecutive images, and one or more subjects can be compared at one time. In addition, there may be frames in which a subject is detected but no reference object is selected. The selected reference object is sent to the comparison unit 215 after being cropped and scaled to a predetermined size in the face region.

[0030] When receiving the face region image, the comparison unit 215 extracts feature information from the image and compares the similarity with the feature information of the subject registered in advance as an authentication object in the database 216 to perform a comparison process. The comparison result is used in the authentication to determine which registered subject the reference object is or to determine whether there is a corresponding registered subject. When a series of comparison processes are completed, the comparison result is sent to the authentication score update unit 214.

[0031] Multiple subjects can be registered in the database 216, and for each subject, subject information, feature information, and an image of the subject region are stored. In addition, the user can set a priority order for each registered subject and store the priority order in each subject information.

[0032] The authentication score update unit 214 uses the received comparison result to update the authentication status and authentication score of each subject. The main subject determination unit 212 determines the main subject based on the authentication status and the subject detection result.

[0033] The tracking calculation unit 203 calculates the AF area and the automatic exposure (AE) area in the live view (LV) image (i.e., in the imaging element 104) to track the main subject determined by the main subject determination unit 212.

[0034] The focus calculation unit 204 acquires focus information (contrast evaluation value of the LV image and defocus amount of the imaging optical system) in the AF area. The control unit 201 sends a focus instruction for controlling the position of the focus lens 121 to the lens unit 120 based on the focus information. The lens unit 120 drives the focus lens 121 in response to the focus instruction. As a result, tracking AF is performed as the focus control for the main subject.

[0035] The exposure calculation unit 205 acquires brightness information in the AE area. The control unit 201 sends an aperture instruction for controlling the aperture amount of the aperture 122 to the lens unit 120 based on the brightness information. The lens unit 120 drives the aperture 122 in response to the aperture instruction. As a result, tracking AE is performed as the exposure control for the main subject.

[0036] Next, the reference Figure 3 and Figure 4 will describe in detail the main subject determination process and the comparison process. Figure 3 is a flowchart for explaining the control from image acquisition to main subject determination, and Figure 4 is a flowchart for explaining the comparison process in more detail. The processes of these flowcharts are implemented by the control unit 201 loading the control program stored in the ROM of the computing device 102 into the RAM and executing the program.

[0037] In step S301, as a series of images to be processed, the control unit 201 acquires one frame of the image from the imaging element 104. The acquired image is converted into a resolution and file format suitable for subsequent detection processing and comparison processing.

[0038] In step S302, the control unit 201 inputs the image to be processed into the subject detector 211 and acquires the detection result from the subject detector 211. The detection result consists of the coordinates of the detected subject area and a score indicating the reliability of the subject in the detected subject area. The subject area is rectangular, and the coordinates of the subject area can be represented by a combination of the center coordinates, width, and height or the upper left coordinates and the lower right coordinates. Since there is a possibility that the detected subject is misdetected, filtering is performed with reference to the confidence level score to obtain only the detection results with a predetermined threshold or higher. Each detected subject is assigned a unique subject ID. Authentication information such as the authentication object, authentication score, and authentication status is managed for each subject ID.

[0039] In this step, multi-object tracking (MOT) is further performed by associating the objects detected in the previous frame with the objects detected in the current frame. That is, the detection results of these two frames are compared, and for the detection results with similar object regions, the objects are regarded as the same, and the object ID is inherited from the previous frame. As a method for judging the similarity between object regions, for example, a method of calculating the intersection over union (IoU) and judging that the objects are similar when the similarity is equal to or greater than a predetermined threshold can be used.

[0040] In step S303, the control unit 201 determines whether an object is detected by referring to the detection result. When there is no object, since there is no main object, the control unit 201 ends the process. When there is an object, the control unit 201 causes the process to proceed to step S304.

[0041] In step S304, the control unit 201 selects an object to be compared in the current frame from the detected objects. Since the comparison unit 215 in this embodiment can compare one object per frame, when there are multiple detected objects, it is necessary to select one of the multiple detected objects. The method of selecting the comparison object can be any method. For example, this can be a method of storing the frame in which the comparison object was last selected and selecting the object that has passed the longest time, or a method of preferentially selecting the object closest to the center of the image, etc.

[0042] In step S305, the control unit 201 checks whether a comparison object is selected in step S304. The control unit 201 causes the process to proceed to step S306 when a comparison object is selected, and causes the process to proceed to step S315 when no comparison object is selected.

[0043] In step S306, the control unit 201 performs a comparison process.

[0044] Here, reference will be made to Figure 4 Describe the comparison process of step S306.

[0045] In step S401, the control unit 201 reads the facial region image of the object selected in step S304. In the facial region image, the cropping region is determined with reference to the coordinates of the detection result, and the cropping region is cut out from the image to be processed. Preprocessing for subsequent processing, such as scaling to a predetermined resolution, is performed on the cut-out image.

[0046] In step S402, the control unit 201 uses the preprocessed facial region image to detect organs such as eyes, nose, or mouth. The organs are detected as points, and the organ point scores representing the coordinates of the organs and the confidence of the organs are output.

[0047] In step S403, the control unit 201 determines whether the detected organ is valid or invalid by referring to the organ point score. When the organ is valid, the control unit 201 causes the process to proceed to step S404, and when the organ is not valid, the control unit 201 causes the process to proceed to step S405.

[0048] In step S404, the control unit 201 normalizes the image using the coordinates of the obtained organ points. When normalizing, an affine transformation is performed so that each organ can be placed at a predetermined position.

[0049] In step S405, the control unit 201 calculates feature information from the normalized facial region image. Examples of methods for calculating feature information include rule-based algorithms for obtaining feature information from the coordinates of feature points of facial parts such as eyes, nose, or mouth, and algorithms for inputting an image into a neural network and obtaining the feature information as an output of the neural network. In the present embodiment, the facial region image is input into a learned deep neural network (DNN), and the feature information of the subject is extracted.

[0050] In step S406, the control unit 201 reads the feature information of the authentication object stored in the database 216. The authentication object is a subject pre-registered by the user; a facial region image is obtained in the same manner as the control object, and the feature information is obtained by inputting the facial region image into the pre-processed DNN. Multiple authentication objects can be registered, and in this step, all the feature information of the authentication objects is read.

[0051] In step S407, the control unit 201 compares the similarity of the feature information of all the control objects obtained from the database 216 with the feature information of the authentication object obtained in step S405. In the present embodiment, the cosine similarity is used to compare the similarity, and a real number from -1 to +1 is output as the similarity score. The comparison is made in a 1:N manner, and the similarity score is calculated for each authentication object among the N authentication objects for one control object.

[0052] In step S408, the control unit 201 searches for the authentication object with the highest similarity score among the N similarity scores, and sets the score of this authentication object as the control score. The control score, the authentication object corresponding to the control score, and the organ detection result obtained in step S402 are output, and a series of control processes end.

[0053] Return to Figure 3, in step S307, the control unit 201 determines whether the comparison object is appropriate. Here, the organ detection result obtained as the result of the authentication process in step S306 is used for the determination. If one or several of the detected organ points have low scores, it can be assumed that a part of the face is hidden by an obstruction or the subject is facing backward. For this reason, each organ point score is checked, and if there is an organ point below a predetermined threshold, it is determined that the comparison object is inappropriate, and the control unit 201 causes the process to proceed to step S308. The threshold can be determined for each organ point, or can be determined for the sum or average of all organ point scores. If the score of one eye is high, the threshold can be dynamically changed, for example, by relaxing the threshold for the other eye.

[0054] In step S309, the control unit 201 confirms the past comparison history based on the authentication information of the subject as the comparison object, and confirms whether the subject has been compared even once. If no comparison has been made even once and this is the first comparison, the control unit 201 causes the process to proceed to step S310, and if a comparison has been made, the control unit 201 causes the process to proceed to step S311.

[0055] In step S311, the control unit 201 determines whether the comparison score obtained in the current frame is lower than the authentication score (the magnitude relationship between the comparison score and the authentication score), where the authentication score is an evaluation value indicating the degree of consistency between the subject and the authentication object. The control unit 201 causes the process to proceed to step S312 when the comparison score is lower than the authentication score, and causes the process to proceed to step S313 when the comparison score is equal to or higher than the authentication score.

[0056] In step S312, the control unit 201 updates the authentication score by the first update method. In step S313, the control unit 201 updates the authentication score by the second update method. In step S308, the control unit 201 updates the authentication score by the third update method. In step S310, the control unit 201 updates the authentication score by the fourth update method. The details of the first to fourth authentication score update methods will be described later.

[0057] In step S314, the control unit 201 updates the authentication status of each subject based on the updated authentication score. The authentication status is a flag indicating whether the subject is a person registered in the database. If the authentication score of the subject as the comparison target is equal to or greater than a predetermined threshold, the authentication success status is set, and if the authentication score is less than the predetermined threshold, the authentication failure status is set. In addition, the authentication target is also updated. However, for the same authentication target, it is not allowed for multiple different subjects to enter the authentication success status. For this reason, when updating to the authentication success status, for the same authentication target, it is checked whether there is another subject in the authentication success status, and when there is such another subject, the subject with the higher authentication score is updated to the authentication success status and the subject with the lower authentication score is updated to the authentication failure status.

[0058] In step S315, the control unit 201 determines the main subject using the main subject determination unit 212 based on the updated authentication status. First, the number of subjects in the authentication success status among all the subjects is confirmed. When the number of subjects in the authentication success status is one, that subject is set as the main subject. When there are multiple subjects in the authentication success status, the subject with the highest priority order of the authentication target preset (designated) by the user is set as the main subject.

[0059] Next, the first authentication score update method to the fourth authentication score update method will be described. In the first update method, using the authentication score N(n - 1) in the previous frame of the subject and the comparison score S(n) obtained in the current frame, the authentication score is updated according to the exponential moving average.

[0060] Assuming that the smoothing coefficient is α, the authentication score N(n) of the current frame is obtained by N(n)=αS(n)+(1 - α)N(n - 1). Here, the smoothing coefficient α is a real number between 0 and 1. When S(n)<N(n - 1) or S(n)≤N(n - 1) is satisfied, the first update method is adopted. Thus, even when an extremely low comparison score is obtained instantaneously, the authentication score will gradually decrease, thereby being able to suppress the rejection of the person due to an incorrect authentication failure status.

[0061] In the second update method, similarly, the authentication score is updated according to the exponential moving average. However, the smoothing coefficient is β different from α, and the authentication score is updated according to N(n)=βS(n)+(1 - β)N(n - 1). In this case, α < β. As a result, when the authentication score decreases, the change is gradual, and when the authentication score increases, the change is rapid, thereby being able to quickly transition to the authentication success status.

[0062] As described in step S302, MOT is performed on the subject across frames. However, in the case where the subject is staggered or crowded, etc., incorrect tracking occurs, and there is a case where the authentication success state is inherited by the incorrect subject, resulting in an incorrect inheritance state. By setting the second update method, even in the case of an incorrect inheritance state, it is possible to quickly return to the correct authentication state. It is also possible to set β to 1, and in this case, use the comparison score as the authentication score as it is.

[0063] In the third update method, the authentication score is updated without using the comparison score, and the score is updated by subtraction or multiplication using a predetermined value γ. When subtraction is performed, any real number γ is used, and the score is updated according to N(n) = N(n - 1) - γ. When multiplication is performed, a real number γ between 0 and 1 is used, and the score is updated according to N(n) = γN(n - 1). In the following description, it is assumed that subtraction is performed.

[0064] When the comparison object is inappropriate, the third update method is selected. At this time, since the comparison score cannot be trusted, it is necessary to update the authentication score without using the comparison score. When the comparison score cannot be used, it is appropriate to lower the authentication score because the risk of incorrect tracking in MOT increases over time.

[0065] The fourth update method is a method in which the comparison score is set as the authentication score as it is and is represented by N(n) = S(n). When the comparison object is compared for the first time, the fourth update method is selected. In this case, N(n - 1) is an undefined value and thus cannot be used. A new subject that has entered the frame, etc., corresponds to this case. However, in this case, by immediately increasing the authentication score and entering the authentication success state, usability can be improved.

[0066] In view of the above, Figure 5 and Figure 6 The specific examples illustrated in will be used to describe the changes in the authentication score and the authentication state. Let the comparison score and the authentication score be integers between 0 and 1000, and let the threshold of the authentication score be 500. Assume that subjects A to C appear, and only subject A is registered in the database. The smoothing coefficient α of the first update method is 0.3, the smoothing coefficient β of the second update method is 0.9, and the specified value γ of the third update method is 10. In addition, a rectangular frame is given for the main subject in each frame.

[0067] Figure 5 is a diagram illustrating a specific example of the effect of this embodiment, which can suppress the scenario where the user is rejected. However, for the sake of clarity of explanation, it is assumed that the frames in which the comparison object is not subject A are omitted, and subject A is selected as the comparison object in all frames in which subject A exists.

[0068] In frame 1, there is no authentication object in the object image to be processed. In this case, since the main subject is determined based on the composition and the size of the subject, subject B is selected as the main subject.

[0069] In frame 2, in the object image to be processed, subject A enters the frame and is selected as the reference object. As a result of the authentication process, 800 is obtained as the reference score. Since subject A is being referenced for the first time, the fourth update method is selected, and the authentication score is 800, which is the same as the reference score. Since the authentication score of subject A exceeds the threshold of 500, subject A enters the authentication success state and becomes the main subject.

[0070] In frame 3, since a part of subject A is hidden, the reference score decreases to 400. Because the reference score is lower than the authentication score of 800 in the previous frame, the first update method is selected. According to the expression of the exponential moving average with a smoothing coefficient of 0.3, the authentication score drops to 680. Although the reference score is less than the threshold of 500, the authentication score is greater than the threshold, so the authentication success state can be maintained, and this subject continues to be the main subject.

[0071] In frame 4, since the hidden area of subject A is even larger, the reference score decreases to 20. At this time, since the left eye is completely hidden, the organ point score drops below the threshold. As a result, it is determined that the reference object is inappropriate, and the third update method is selected.

[0072] In this case, the authentication score is updated to 670 by subtracting a predetermined value of 10 from the authentication score of 680 in the previous frame. If the first update method is selected for this frame, the authentication score will become 482 and drop below the threshold, entering the authentication failure state, and the main subject will be changed to subject B. In other words, since the third update method is selected, subject A can be kept in the authentication success state.

[0073] In frame 5, subject A terminates the hidden state, and the reference score becomes 800 again. Because the reference score is higher than the authentication score of 670 in the previous frame, the second update method is selected. According to the expression of the exponential moving average with a smoothing coefficient of 0.9, the authentication score rises to 787. Even assuming that the authentication score was less than the threshold in frame 3 or frame 4, due to the large smoothing coefficient, it is possible to transition to the authentication success state in frame 5, and subject A can be made the main subject again.

[0074] Figure 6 It is a diagram illustrating a specific example of a scenario that can suppress error inheritance as an effect of this embodiment.

[0075] In Frame 1, assume that the subject A being tracked as ID:1 is selected as the reference object. In this frame, the authentication score of the subject A is updated by the fourth update method, and the subject A is in the authentication success state.

[0076] In Frame 2, the reference object is switched to the subject B being tracked as ID:2. Since the subject B is not registered in the database and is compared with the feature information of the subject A in the database, the reference score indicates a low value. The authentication score is updated by the fourth update method.

[0077] In Frame 3, the subject A and the subject B are interleaved with each other. In this case, since only one subject is detected, the subject with ID:1 or ID:2 disappears. Here, the primary subject in the previous frame is preferred, and ID:1 is retained. Therefore, the subject B is identified as ID:1 and is wrongly tracked. The reference object is the subject B, and a low reference score is indicated as in the previous frame. At this time, the authentication score is updated by the second update method. However, since wrong tracking has occurred, the authentication score in the previous frame inherits the score of the subject A. As a result of the calculation based on the exponential moving average, the authentication score is 710. Since the authentication score exceeds the threshold of 500, it enters the authentication success state and becomes the primary subject. Since the subject B is not actually the registered subject, it enters the error inheritance state.

[0078] In Frame 4, the subject A reappears and is assigned ID:3 as a new subject. In this frame, the subject A is selected as the reference object. The authentication score is updated by the fourth update method, and the authentication score becomes 750. Since this score exceeds the threshold of 500, it can transition to the authentication success state. However, since the subject B was also in the authentication success state in the previous frame, one of the subject A and the subject B needs to be placed in the authentication failure state. When comparing the authentication scores of the respective subjects, since the authentication score of the subject A is higher, the subject A is placed in the authentication success state and the subject B is placed in the authentication failure state. Since the authentication score of the subject in the error inheritance state was reduced by the second update method in Frame 3 and the reference score is accepted as the authentication score as it is by the fourth update method in Frame 4, the error inheritance state can be corrected immediately when the correct authentication object appears.

[0079] In this example, since only the subject A is registered in the database as the authentication target, only one subject is in the authentication success state. However, when multiple people are registered in the database, there can be subjects in the authentication success state for each registered authentication target. That is, when N people are registered in the database, at most N subjects can be in the authentication success state. In this case, the main subject is determined according to the priority order of the subjects registered in the database set by the user in advance.

[0080] As described above, in this embodiment, in the face authentication system, one of multiple authentication score update methods is selected according to the situation, and the control score and the authentication score in the previous frame are used to update the authentication score in the current frame. As a result, it is possible to stably and sensitively transition to the authentication success state in response to a temporary decrease in the control score caused by, for example, the expression change or occlusion of the subject, and it is possible to achieve both suppression of error inheritance and suppression of rejection of the user himself / herself.

[0081] Other embodiments

[0082] The embodiments of the present invention can also be implemented by the following method, that is, software (including a computer program product including computer programs / instructions) that executes the functions of the above embodiments is provided to a system or device through a network or various storage media, and the computer (central processing unit (CPU), microprocessing unit (MPU)) of the system or device reads and executes the computer programs / instructions.

[0083] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims will be given the broadest interpretation so as to cover all such modifications as well as equivalent structures and functions.

Claims

1. An authentication device, comprising: a detection unit configured to detect a subject from an input image; a comparison unit configured to compare the subject detected by the detection unit with a pre-registered authentication object and output a comparison score indicating the similarity between the subject and the authentication object; an update unit configured to update an authentication score based on the comparison score, the authentication score being an evaluation value indicating the degree of coincidence between the subject and the authentication object; and an authentication unit configured to authenticate the subject based on the authentication score, wherein the update unit changes the update method of the authentication score based on the magnitude relationship between the comparison score and the authentication score.

2. The authentication device according to claim 1, wherein, The detection unit detects a subject from each image in a series of input images.

3. The authentication device according to claim 1, further comprising a selection unit configured to select a subject to be compared with the authentication object from the subjects detected by the detection unit.

4. The authentication device according to claim 1, wherein, The update unit updates the authentication score using a first update method when the comparison score is less than the authentication score, and updates the authentication score using a second update method when the comparison score is greater than or equal to the authentication score.

5. The authentication device according to claim 4, wherein, The first update method and the second update method update the authentication score based on an exponential moving average with different smoothing coefficients.

6. The authentication device according to claim 5, wherein, The smoothing coefficient in the first update method is less than the smoothing coefficient in the second update method.

7. The authentication device according to claim 1, wherein The update unit uses a third update method to update the authentication score when a comparison score of a subject cannot be obtained.

8. The authentication device according to claim 7, wherein, In the third update method, the authentication score is updated by multiplying the authentication score by a predetermined coefficient.

9. The authentication device according to claim 7, wherein, In the third update method, the authentication score is updated by subtracting a predetermined value from the authentication score.

10. The authentication device according to claim 1, wherein, The update unit uses a fourth update method of setting the comparison score as the authentication score to update the authentication score for a subject that is first compared with the authentication object.

11. An authentication method, comprising: detecting a subject from an input image; comparing the detected subject with a pre-registered authentication object and outputting a comparison score indicating the similarity between the subject and the authentication object; updating an authentication score based on the comparison score, the authentication score being an evaluation value indicating the degree of coincidence between the subject and the authentication object; and authenticating the subject based on the authentication score, wherein, in the update, the update method of the authentication score is changed based on the magnitude relationship between the comparison score and the authentication score.

12. A imaging device, comprising: a detection unit configured to detect a subject from a captured image; a comparison unit configured to compare the subject detected by the detection unit with a pre-registered authentication object and output a comparison score indicating the similarity between the subject and the authentication object; An update unit configured to update an authentication score based on the comparison score, the authentication score being an evaluation value for indicating the degree of consistency between the subject and the authentication object; and A determination unit configured to determine a main subject based on the authentication score, wherein the update unit changes the update method of the authentication score based on the magnitude relationship between the comparison score and the authentication score.

13. The imaging device according to claim 12, wherein, The detection unit detects a subject from each of the consecutive images captured.

14. The imaging device according to claim 12, further comprising a selection unit configured to select a subject to be compared with the authentication object from the subjects detected by the detection unit.

15. The imaging device according to claim 12, wherein, The update unit updates the authentication score using a first update method when the comparison score is less than the authentication score, and updates the authentication score using a second update method when the comparison score is greater than or equal to the authentication score.

16. The imaging device according to claim 15, wherein, The first update method and the second update method update the authentication score based on exponential moving averages with different smoothing coefficients.

17. The imaging device according to claim 16, wherein, The smoothing coefficient in the first update method is less than the smoothing coefficient in the second update method.

18. The imaging device according to claim 12, wherein, The update unit uses a third update method to update the authentication score when the comparison score of the subject cannot be obtained.

19. The imaging device according to claim 18, wherein, In the third update method, the authentication score is updated by multiplying the authentication score by a predetermined coefficient.

20. The imaging device according to claim 18, wherein, In the third update method, the authentication score is updated by subtracting a predetermined value from the authentication score.

21. The imaging device according to any one of claims 12 to 20, wherein, The update unit uses a fourth update method of setting the comparison score as the authentication score to update the authentication score for the subject that is first compared with the authentication object.

22. The imaging device according to claim 12, wherein, The determination unit determines the subject with the highest authentication score as the main subject when multiple subjects are authenticated as the same authentication object.

23. The imaging device according to claim 12, wherein, The determination unit determines the subject authenticated as the authentication object with the highest priority specified by the user as the main subject when multiple subjects are authenticated as different authentication objects.

24. The imaging device according to claim 12, wherein, For the subject determined by the determination unit as the main subject, focus control or exposure control is preferentially performed.

25. A control method for an imaging device, comprising: Detecting a subject from a captured image; Comparing the detected subject with a pre-registered authentication object and outputting a comparison score for indicating the similarity between the subject and the authentication object; Updating an authentication score based on the comparison score, the authentication score being an evaluation value for indicating the degree of consistency between the subject and the authentication object; and Determining a main subject based on the authentication score, wherein, in the update, the update method of the authentication score is changed based on the magnitude relationship between the comparison score and the authentication score.

26. A computer program product comprising a program for causing a computer to execute the steps of the method according to claim 11 or 25.

27. A non-transitory computer-readable storage medium stores a program for causing a computer to execute the steps of the method according to claim 11 or 25.

Images