Authentication device, imaging device, control method thereof, program, and storage medium

Abstract

To achieve, in authentication of a person, both prevention of reception of another person caused by takeover of an authentication state and rejection of a person himself / herself caused by re-authentication.SOLUTION: An authentication apparatus comprises: a detection unit that detects a subject from an input image; a collation unit that collates the subject detected by the detection unit with an authentication target registered in advance and outputs a collation score indicating the similarity between the subject and the authentication target; an update unit that updates an authentication score being an evaluation value indicating a degree at which the subject matches the authentication target on the basis of the collation score; and an authentication unit that authenticates the subject on the basis of the authentication score. The update unit changes a method for updating the authentication score on the basis of the magnitude relationship between the collation score and the authentication score.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an authentication device for authenticating a person. [Background technology]

[0002] Conventionally, imaging devices such as digital cameras have been put to practical use, including products with a tracking AF (autofocus) mode. The tracking AF mode is a mode that detects a person's face and eyes from images continuously output from an imaging element, and continuously optimizes the focus and exposure conditions for the detected person's face or eyes. Patent Document 1 also describes a technology that registers people in advance and uses face recognition to select a desired tracking target from multiple subjects.

[0003] Generally, in the registration process for face authentication, feature information is extracted from a face image and stored in non-volatile memory. When taking a photograph, a face is detected from the images continuously output from the image sensor, and feature information of the detected face is extracted. The similarity between this feature information and the feature information stored in the non-volatile memory is compared to determine whether the person is a registered person.

[0004] In recent years, the use of deep learning as a facial recognition algorithm has become common, and while performance has improved, the processing load has also increased. In particular, in devices such as digital cameras that have limited resources and use embedded software, it is difficult to perform authentication processing in real time, which requires a high processing load within limited resources.

[0005] Therefore, Patent Document 2 proposes a method for efficiently performing authentication processing with limited resources, in which a subject that has been successfully authenticated once does not undergo further matching processing, but instead continues to be authenticated by tracking. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-187591 [Patent Document 2] Patent No. 5963525 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the method of inheriting the authentication status by tracking, there is a possibility that a different subject may be mistakenly tracked due to overlapping or crowding of subjects, and in such a case, the authentication status may be inherited by another person (false acceptance state). For this reason, even if a subject has been successfully authenticated once, it is necessary to perform re-matching as appropriate, and if the authentication status is in a false acceptance state, it is necessary to cancel the authentication state. On the other hand, if re-matching is performed, even if the subject should maintain its authentication status, the matching score may decrease depending on the state, and the person may be rejected (false rejection).

[0008] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to achieve both the prevention of false acceptance due to the handover of authentication status and the prevention of false rejection due to re-authentication in person authentication. [Means for solving the problem]

[0009] The authentication device according to the present invention motion image Each frame of a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; and an authentication means for authenticating the subject based on the authentication score, wherein the update means When calculating the authentication score for the current frame by taking an exponential moving average of the authentication score for the previous frame of the video and the matching score for the current frame, the value of a smoothing coefficient is changed based on the magnitude relationship between the matching score and the authentication score to calculate the authentication score for the current frame. It is characterized by: [Effects of the Invention]

[0010] According to the present invention, in person authentication, it is possible to simultaneously suppress the acceptance of a false person due to the takeover of the authentication state and the rejection of the true person due to re-authentication. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing the configuration of a digital single-lens camera as an embodiment of an imaging apparatus of the present invention. [Figure 2] Block diagram of the camera's control system. [Figure 3] 10 is a flowchart illustrating control from image acquisition to main subject determination. [Figure 4] 10 is a flowchart illustrating a collation process in detail. [Figure 5] 10A and 10B are diagrams showing specific examples of scenes in which false rejection can be suppressed. [Figure 6] 10A and 10B are diagrams showing specific examples of scenes in which false acceptance can be suppressed. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] Fig. 1 is a diagram showing the configuration of a digital single-lens camera (hereinafter also simply referred to as camera) 100, which is an embodiment of an imaging device of the present invention. Fig. 2 is a diagram showing the configuration related to control of the digital single-lens camera 100.

[0014] In camera 100 of this embodiment, as shown in Fig. 1, a detachable, interchangeable lens unit 120 is attached to the front side (subject side) of camera body 101. Lens unit 120 has a focus lens 121, an aperture 122, etc., and is electrically connected to camera body 101 via mount contacts 123. Via this electrical connection, control unit 201 (see Fig. 2) of camera body 101 controls lens unit 120, and is able to adjust the amount of light taken into camera body 101 and the focal position. Note that focus lens 121 can also be adjusted manually by the user.

[0015] The image sensor 104, which captures a subject image, is composed of a CCD or CMOS sensor or the like, and includes an infrared cut filter, a low-pass filter, etc. The image sensor 104 photoelectrically converts the subject image formed through the photographing optical system of the lens unit 120 during photography, and transmits signal information for generating a photographed image to the arithmetic unit 102. The arithmetic unit 102 generates a photographed image from the received signal information, stores it in an external storage device 107 (see FIG. 2), and displays it on a display unit 105 such as an LCD. The shutter 103 shields the image sensor 104 from light when not photographing, and opens it during photography to expose the image sensor 104 to light.

[0016] Next, the configuration relating to the control of the camera 100 will be described with reference to FIG.

[0017] The arithmetic unit 102 includes a multi-core CPU capable of parallel processing of multiple tasks, RAM, ROM, and a dedicated circuit for executing specific arithmetic processing at high speed. The arithmetic unit 102 includes a control unit 201, a main subject calculation unit 202 for detecting a subject, a tracking calculation unit 203, a focus calculation unit 204, an exposure calculation unit 205, etc. The control unit 201 controls each unit of the camera body 101 and the lens unit 120.

[0018] The main subject calculation unit 202 includes a subject detector 211 that detects the subject, a main subject determination unit 212 that performs authentication and main subject determination, a matching target selection unit 213 that selects a matching target, an authentication score update unit 214 that updates the authentication score, and a matching unit 215 that performs matching.

[0019] The subject detector 211 sequentially receives successive images acquired from the image sensor 104 and performs processing to detect one or more subject regions from each image. In this embodiment, the subject of interest is a human face. Any known method such as AdaBoost or a convolutional neural network can be used as the detection method. The implementation form may be a program running on a CPU, dedicated hardware, or a combination of these. The subject detection results obtained from the subject detector 211 are sent to the main subject determination unit 212 and the matching target selection unit 213.

[0020] The matching target selection unit 213 selects a matching target from the detected subjects. The number of subjects to be matched at one time can be determined based on the processing speed of the matching unit 215 and the frame rate of consecutive images, and may be one or more people. Also, there may be frames in which no matching target is selected even if a subject is detected. The selected matching target is sent to the matching unit 215 after the facial area is trimmed and enlarged or reduced to a predetermined size.

[0021] When the matching unit 215 receives the face area image, it extracts feature information from the image and performs matching processing by comparing the similarity with feature information of subjects registered in advance as authentication targets in the database 216. Using the matching result, it authenticates which registered subject the matching target is, or whether there is no corresponding registered subject. When the series of matching processes is completed, the matching result is sent to the authentication score update unit 214.

[0022] Multiple subjects can be registered in the database 216, and for each subject, subject information, feature information, and an image of the subject area are stored. In addition, the user can set priorities for the registered subjects, and the priorities are stored in each piece of subject information.

[0023] The authentication score update unit 214 updates the authentication state and authentication score of each subject using the received matching result. The main subject is determined by the main subject determination unit 212 based on the authentication state and subject detection result.

[0024] Tracking calculation section 203 calculates an AF area and an AE (auto exposure) area on an LV (live view) image (that is, image sensor 104) so ​​as to track the main subject determined by main subject determination section 212.

[0025] The focus calculation unit 204 acquires focus information in the AF area (the contrast evaluation value of the LV image and the defocus amount of the imaging optical system). The control unit 201 transmits a focus instruction to the lens unit 120 to control the position of the focus lens 121 based on the focus information. The lens unit 120 drives the focus lens 121 in accordance with the focus instruction. This performs tracking AF as focus control for the main subject.

[0026] Furthermore, the exposure calculation unit 205 acquires brightness information in the AE region. The control unit 201 transmits an aperture instruction to the lens unit 120 to control the opening amount of the aperture 122 based on the brightness information. The lens unit 120 drives the aperture 122 in accordance with the aperture instruction. This performs tracking AE as exposure control for the main subject.

[0027] Next, the main subject determination process and the matching process will be described in detail using Figures 3 and 4. Figure 3 is a flowchart explaining the control from image acquisition to main subject determination, and Figure 4 is a flowchart explaining the matching process in particular in more detail. The processes of these flowcharts are realized by the control unit 201 loading a control program stored in the ROM in the arithmetic device 102 into the RAM and executing it.

[0028] In step S301, the control unit 201 acquires one frame of an image as a series of images to be processed from the image sensor 104. The acquired image is converted into a resolution and file format suitable for the subsequent detection and matching processes.

[0029] In step S302, the control unit 201 inputs the image to be processed into the object detector 211 and obtains the detection results. The detection results consist of the coordinates of the detected object area and a score that represents the reliability of the object. The object area is rectangular, and its coordinates can be expressed as a combination of the center coordinate and width and height, or the top left coordinate and bottom right coordinate. Since the detected objects may include false detections, the reliability score is referenced and only detection results above a predetermined threshold are filtered. A unique object ID is assigned to each detected object. Authentication information such as the authentication target, authentication score, and authentication status is managed for each object ID.

[0030] In this step, MOT (Multi Object Tracking) is performed by associating the object detected in the previous frame with the object detected in the current frame. That is, the detection results in both frames are compared, and if the object area is similar, it is determined to be the same object and the object ID is inherited from the previous frame. One method for determining the similarity of the object areas is to calculate the IoU (Intersection over Union) and determine that the objects are similar if it is equal to or greater than a predetermined threshold.

[0031] In step S303, the control unit 201 determines whether or not a subject has been detected by referring to the detection result. If no subject is present, the main subject is also absent, and the control unit 201 ends the process. If a subject is present, the control unit 201 proceeds to step S304.

[0032] In step S304, the control unit 201 selects a subject to be matched in the current frame from the detected subjects. Since the matching unit 215 in this embodiment can match one subject per frame, if there are multiple detected subjects, it is necessary to select one from among them. Any method may be used to select the subject to be matched. For example, a method may be used in which the last frame selected as the subject to be matched is stored and the subject with the longest elapsed time is selected, or a method may be used in which the subject closest to the center of the image is preferentially selected.

[0033] In step S305, the control unit 201 checks whether a match target has been selected in step S304. If a match target has been selected, the control unit 201 proceeds to step S306, and if not, the control unit 201 proceeds to step S315.

[0034] In step S306, the control unit 201 performs a matching process.

[0035] The collation process in step S306 will now be described with reference to FIG.

[0036] In step S401, the control unit 201 reads the face region image of the subject selected in step S304. A trimming region is determined based on the coordinates of the detection result, and the face region image is cut out from the image to be processed. The cut-out image is subjected to preprocessing for subsequent processing, such as enlarging or reducing the image to a predetermined resolution.

[0037] In step S402, the control unit 201 detects organs such as the eyes, nose, and mouth using the face region image that has undergone preprocessing. The organs are detected as points, and organ point scores that represent their coordinates and reliability are output.

[0038] In step S403, the control unit 201 refers to the organ point score and determines whether the detected organ is valid or invalid. If the organ is valid, the control unit 201 proceeds to step S404, and if it is not valid, the control unit 201 proceeds to step S405.

[0039] In step 404, the control unit 201 normalizes the image using the coordinates of the acquired organ points. During normalization, an affine transformation is performed so that each organ can be placed at a predetermined position.

[0040] In step S405, the control unit 201 calculates feature information from the normalized face area image. Methods for calculating feature information include a rule-based algorithm that calculates feature information from the coordinates of feature points of facial parts such as the eyes, nose, and mouth, and an algorithm that inputs an image into a neural network and calculates feature information as its output. In this embodiment, the face area image is input into a trained DNN (deep neural network) to extract feature information of the subject.

[0041] In step S406, the control unit 201 reads the feature information of the authentication target stored in the database 216. The authentication target is a subject registered in advance by a user, and a face region image is acquired in the same way as the matching target, and after preprocessing, the image is input to the DNN to acquire the feature information. It is possible to register multiple people as the authentication target, and in this step, the feature information of all the authentication targets is read.

[0042] In step S407, the control unit 201 compares the similarity between the feature information of all authentication targets acquired from the database 216 and the feature information of the matching target acquired in step S405. In this embodiment, cosine similarity is used to compare the similarity, and a real number between -1 and +1 is output as the similarity score. Matching is performed on a 1:N basis, and a similarity score with each of the N authentication targets is calculated for one matching target.

[0043] In step S408, the control unit 201 searches for the authentication target with the highest similarity score from the N similarity scores and sets this as the matching score. The matching score, the authentication target corresponding to the matching score, and the organ detection result acquired in step 402 are output, and the series of matching processes is completed.

[0044] Returning to FIG. 3, in step S307, the control unit 201 determines whether the matching target is appropriate. Here, this determination is made using the organ detection results acquired as a result of the authentication process in step 306. If one or more of the detected organ point scores are low, it is possible that part of the face is hidden by an obstruction or that the subject is facing away from the camera. For this reason, the organ point scores are checked, and if there is an organ point whose score is below a predetermined threshold, the control unit 201 determines that the matching target is inappropriate, and proceeds to step S308. The threshold may be determined for each organ point, or may be determined based on the sum or average of all organ point scores. Furthermore, the threshold may be dynamically changed, for example, if one eye has a high score, the threshold for the other eye may be relaxed.

[0045] In step S309, the control unit 201 checks the past verification history from the authentication information of the subject to be verified, and checks whether verification has been performed at least once. If verification has never been performed and this is the first time verification has been performed, the control unit 201 proceeds to step S310, and if verification has been performed before, the control unit 201 proceeds to step S311.

[0046] In step S311, the control unit 201 determines whether the matching score acquired in the current frame from the authentication information of the subject to be matched is lower than the authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target (the magnitude relationship between the matching score and the authentication score). If the matching score is lower than the authentication score, the control unit 201 proceeds to step S312, and if it is equal to or higher than the authentication score, the control unit 201 proceeds to step S313.

[0047] In step S312, the control unit 201 updates the authentication score using the first update method. In step S313, the control unit 201 updates the authentication score using the second update method. In step S308, the control unit 201 updates the authentication score using the third update method. In step S310, the control unit 201 updates the authentication score using the fourth update method. Details of the first to fourth authentication score update methods will be described later.

[0048] In step S314, the control unit 201 updates the authentication status of each subject based on the updated authentication score. The authentication status is a flag that indicates whether the subject is a person registered in the database. If the authentication score of the subject to be matched is equal to or greater than a predetermined threshold, the authentication is successful, and if it is less than the predetermined threshold, the authentication is unsuccessful. At the same time, the authentication target is also updated. However, it is not permitted for multiple different subjects to be in the authentication success state for the same authentication target. For this reason, when updating to the authentication success state, it is checked whether there is another subject in the authentication success state for the same authentication target, and if there is, the one with the higher authentication score is updated to the authentication success state, and the one with the lower authentication score is updated to the authentication failure state.

[0049] In step S315, the control unit 201 determines the main subject using the main subject determination unit 212 based on the updated authentication status. First, the number of subjects who have been successfully authenticated is confirmed from all subjects. If there is one subject who has been successfully authenticated, this subject is determined to be the main subject. If there are multiple subjects who have been successfully authenticated, the subject with the highest priority for authentication, which has been set (specified) in advance by the user, is determined to be the main subject.

[0050] Next, the first to fourth authentication score update methods will be explained. In the first update method, the authentication score is updated by exponential moving average using the authentication score N(n-1) of the subject in the previous frame and the matching score S(n) acquired in the current frame.

[0051] If the smoothing coefficient is α, the authentication score N(n) of the current frame is obtained by N(n) = αS(n) + (1 - α)N(n - 1). Here, the smoothing coefficient α is a real number between 0 and 1. The first update method is adopted when S(n) < N(n - 1) or S(n) ≤ N(n - 1) is satisfied. Thereby, even when an extremely low matching score is obtained instantaneously, the decrease in the authentication score becomes gentle, so that it is possible to suppress the rejection of the user due to an accidental authentication failure state.

[0052] Also in the second update method, the authentication score is updated by exponential moving average in the same way. However, the smoothing coefficient is β which is different from α, and the authentication score is updated by N(n) = βS(n) + (1 - β)N(n - 1). In this case, α < β. Thereby, when the authentication score decreases, the change becomes gentle, while when the authentication score increases, the change becomes sensitive, so that it is possible to quickly transition to the authentication success state.

[0053] As described in step S302, the subject is performing MOT across the frame. However, in the case where the subjects are interleaved or crowded, etc., false tracking may occur, and in some cases, the authentication success state may be inherited by the wrong subject and the system may enter the unauthorized acceptance state. By providing the second update method, even when the unauthorized acceptance state occurs, it is possible to quickly return to the correct authentication state. β may be set to 1, and in this case, the matching score is directly adopted as the authentication score.

[0054] The third update method is a method of updating the authentication score without using the matching score, and the score is updated by subtraction or multiplication using a predetermined value γ. When subtraction is performed, an arbitrary real number γ is used, and it is updated by N(n) = N(n - 1) - γ. When multiplication is performed, a real number γ between 0 and 1 is used, and it is updated by N(n) = γN(n - 1). The following description assumes that subtraction is performed.

[0055] The third update method is selected when the matching target is inappropriate. In this case, the matching score is unreliable, so the authentication score must be updated without using the matching score. Since the risk of mistracking in MOT increases over time, it is appropriate to lower the authentication score if the matching score cannot be used.

[0056] The fourth update method uses the matching score as the authentication score as is, and is expressed as N(n) = S(n). The fourth update method is selected when the matching target is being matched for the first time. In this case, N(n-1) is an indefinite value and cannot be used. This applies to a new subject that enters the frame, and in this case, usability can be improved by immediately raising the authentication score and quickly achieving a successful authentication state.

[0057] Based on the above, changes in the authentication score and authentication status will be explained using the specific examples shown in Figures 5 and 6. However, the matching score and authentication score are integers between 0 and 1000, and the authentication score threshold is 500. Subjects A to C appear, and only subject A is registered in the database. The smoothing coefficient α for the first update method is 0.3, the smoothing coefficient β for the second update method is 0.9, and the predetermined value γ for the third update method is 10. In addition, a rectangular frame is drawn around the main subject in each frame.

[0058] 5 is a diagram showing a specific example of a scene in which false rejection can be suppressed as an effect of this embodiment. However, for ease of understanding, frames in which subjects other than subject A are the subject of matching are omitted, and subject A is selected as the subject of matching in all frames in which subject A is present.

[0059] In frame 1, there is no target to be authenticated in the image to be processed. In this case, the main subject is determined based on the composition and the size of the subject, so subject B is selected as the main subject.

[0060] In frame 2, subject A frames into the image to be processed and is selected as the subject to be matched. As a result of the authentication process, a matching score of 800 is obtained. Because this is the first time that subject A has been matched, the fourth update method is selected, and the authentication score is also set to 800, the same as the matching score. Because subject A's authentication score exceeds the threshold of 500, subject A is successfully authenticated and becomes the main subject.

[0061] In frame 3, subject A is partially occluded, so the matching score drops to 400. Because this is lower than the authentication score of 800 from the previous frame, the first update method is selected. The authentication score drops to 680 based on the exponential moving average formula with a smoothing coefficient of 0.3. Although the matching score is below the threshold of 500, the authentication score is above the threshold, so the authentication success state can be maintained and subject A continues to be the main subject.

[0062] In frame 4, the area of ​​object A that is hidden increases further, and the matching score drops to 20. At this point, the left eye is completely hidden, so the organ point score drops below the threshold. As a result, the matching target is determined to be inappropriate, and the third update method is selected.

[0063] In this case, the authentication score of the previous frame, 680, is subtracted by a predetermined value of 10, and the authentication score is updated to 670. If the first update method had been selected in this frame, the authentication score would have been 482, which is below the threshold, resulting in an authentication failure state and changing the main subject to subject B. In other words, because the third update method was selected, subject A can maintain the authentication success state.

[0064] In frame 5, subject A's occlusion state is resolved, and the matching score becomes 800 again. Because this exceeds the authentication score of 670 from the previous frame, the second update method is selected. The authentication score rises to 787 from the exponential moving average formula with a smoothing coefficient of 0.9. Even if the authentication score had fallen below the threshold in frame 3 or frame 4, because the smoothing coefficient is large, it is possible to transition to an authentication success state in frame 5, and subject A can once again be designated as the main subject.

[0065] FIG. 6 is a diagram showing a specific example of a scene in which the acceptance of other people can be suppressed as an effect of this embodiment.

[0066] In frame 1, subject A, who is being tracked as ID: 1, is selected as the target for matching. In this frame, the authentication score of subject A is updated using the fourth update method, and the authentication is successful.

[0067] In frame 2, the matching target switches to subject B, which is being tracked with ID:2. Subject B is not registered in the database, and its feature information is compared with that of subject A in the database, resulting in a low matching score. The authentication score is updated using the fourth update method.

[0068] In frame 3, subjects A and B cross paths. In this case, only one subject was detected, so either subject ID:1 or ID:2 disappears. Here, priority is given to the main subject in the previous frame, and ID:1 remains. As a result, subject B is recognized as ID:1 and mistracked. Subject B becomes the target for matching, and, like the previous frame, shows a low matching score. At this time, the authentication score is updated using the second update method. However, because mistracking has occurred, the authentication score in the previous frame continues to be that of subject A. As a result of calculation using the exponential moving average, the authentication score becomes 710. Because this authentication score exceeds the threshold of 500, authentication is successful and subject B becomes the main subject. Because subject B is not actually a registered subject, the state is false acceptance.

[0069] In frame 4, subject A appears again and is assigned ID: 3 as a new subject. In this frame, subject A is selected as the target for matching. The authentication score is updated using the fourth update method to 750. Because this score exceeds the threshold of 500, it is possible to transition to an authentication success state. However, because subject B was also in an authentication success state in the previous frame, one of them must be in an authentication failure state. When the authentication scores of each subject are compared, subject A's authentication score is higher, so subject A is placed in an authentication success state and subject B is placed in an authentication failure state. By lowering the authentication score of the subject in the false acceptance state using the second update method in frame 3 and accepting the matching score as the authentication score using the fourth update method in frame 4, it is possible to immediately correct the false acceptance state when the correct target for authentication appears.

[0070] In this example, only subject A was registered in the database as an authentication target, so only one subject achieved successful authentication. However, if multiple people are registered in the database, there will be subjects who achieve successful authentication for each of the registered authentication targets. In other words, if N people are registered in the database, a maximum of N subjects will achieve successful authentication. In this case, the main subject is determined according to the priority order of subjects registered in the database, which is set in advance by the user.

[0071] As described above, in this embodiment, the face authentication system selects one of a plurality of update methods for the authentication score depending on the situation, and updates the authentication score for the current frame using the matching score and the authentication score for the previous frame. This makes it possible to transition to a successful authentication state stably and sensitively even when the matching score temporarily drops due to a change in the subject's facial expression, occlusion, etc., and makes it possible to suppress both false acceptance and false rejection.

[0072] The disclosure of this specification includes the following authentication device, authentication method, imaging device and control method therefor, program, and storage medium.

[0073] (Item 1) a detection means for detecting a subject from an input image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication means for authenticating the subject based on the authentication score; Equipped with The authentication device is characterized in that the update means changes a method for updating the authentication score based on the magnitude relationship between the matching score and the authentication score.

[0074] (Item 2) 2. The authentication device according to item 1, wherein the detection means detects a subject from each of the input consecutive images.

[0075] (Item 3) 3. The authentication device according to item 1 or 2, further comprising a selection unit for selecting an object to be matched with the authentication target from the subjects detected by the detection unit.

[0076] (Item 4) The authentication device described in any one of items 1 to 3, characterized in that the update means updates the authentication score using a first update method when the matching score is smaller than the authentication score, and updates the authentication score using a second update method when the matching score is equal to or greater than the authentication score.

[0077] (Item 5) 5. The authentication device according to item 4, wherein the first update method and the second update method update the authentication score using exponential moving averages with different smoothing coefficients.

[0078] (Item 6) 6. The authentication device according to item 5, wherein the smoothing coefficient in the first update method is smaller than the smoothing coefficient in the second update method.

[0079] (Item 7) 7. The authentication device according to any one of items 1 to 6, wherein the update means updates the authentication score using a third update method when the matching score of the subject cannot be obtained.

[0080] (Item 8) 8. The authentication device according to item 7, wherein the third update method updates the authentication score by multiplying the authentication score by a predetermined coefficient.

[0081] (Item 9) 8. The authentication device according to item 7, wherein the third update method updates the authentication score by subtracting a predetermined value from the authentication score.

[0082] (Item 10) The authentication device described in any one of items 1 to 9, characterized in that, for a subject that is being matched with the authentication target for the first time, the update means updates the authentication score using a fourth update method in which the matching score is used as the authentication score.

[0083] (Item 11) a detection step of detecting a subject from an input image; a matching step of matching the subject detected in the detection step with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an updating step of updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication step of authenticating the subject based on the authentication score; and The authentication method according to claim 1, wherein the updating step changes a method for updating the authentication score based on the magnitude relationship between the matching score and the authentication score.

[0084] (Item 12) a detection means for detecting a subject from a captured image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with The imaging device, wherein the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score.

[0085] (Item 13) Item 13. The imaging device according to item 12, wherein the detection means detects a subject from each of the captured consecutive images.

[0086] (Item 14) 14. The imaging device according to item 12 or 13, further comprising a selection unit for selecting an object to be matched with the authentication object from the subjects detected by the detection unit.

[0087] (Item 15) The imaging device described in any one of items 12 to 14, characterized in that the update means updates the authentication score using a first update method when the matching score is smaller than the authentication score, and updates the authentication score using a second update method when the matching score is equal to or greater than the authentication score.

[0088] (Item 16) Item 16. The imaging device according to item 15, wherein the first update method and the second update method update the authentication score using exponential moving averages with different smoothing coefficients.

[0089] (Item 17) Item 17. The imaging device according to item 16, wherein the smoothing coefficient in the first update method is smaller than the smoothing coefficient in the second update method.

[0090] (Item 18) 18. The imaging device according to any one of items 12 to 17, wherein the update means updates the authentication score using a third update method when the matching score of the subject cannot be obtained.

[0091] (Item 19) Item 19. The imaging device according to item 18, wherein the third update method updates the authentication score by multiplying the authentication score by a predetermined coefficient.

[0092] (Item 20) Item 19. The imaging device according to item 18, wherein the third updating method updates the authentication score by subtracting a predetermined value from the authentication score.

[0093] (Item 21) The imaging device described in any one of items 12 to 20, characterized in that, for a subject that is being matched with the authentication target for the first time, the update means updates the authentication score by a fourth update method in which the matching score is used as the authentication score.

[0094] (Item 22) 22. The imaging device according to any one of items 12 to 21, wherein the determination means determines the subject with the higher authentication score to be the main subject when multiple subjects are authenticated as the same authentication target.

[0095] (Item 23) The imaging device described in any one of items 12 to 21, characterized in that when multiple subjects are authenticated as different authentication targets, the determination means determines the subject authenticated as the authentication target with the highest priority specified by the user to be the main subject.

[0096] (Item 24) 24. The imaging device according to any one of items 12 to 23, wherein focus control or exposure control is performed preferentially on the subject determined to be the main subject by the determination means.

[0097] (Item 25) a detection step of detecting a subject from the captured image; a matching step of matching the subject detected in the detection step with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an updating step of updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination step of determining a main subject based on the authentication score; and The control method for an imaging device, wherein the updating step changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score.

[0098] (Item 26) 26. A program for causing a computer to execute each step of the method according to item 11 or 25.

[0099] (Item 27) A computer-readable storage medium storing a program for causing a computer to execute each step of the method according to item 11 or 25.

[0100] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions.

[0101] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0102] 100: Digital single-lens camera, 102: Computing device, 104: Image sensor, 201: Control unit, 202: Main subject computing unit, 211: Subject detector, 212: Main subject determination unit, 213: Matching target selection unit, 214: Authentication score update unit, 215: Matching unit, 216: Database

Claims

1. a detection means for detecting a subject from each frame of an input moving image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target, and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication means for authenticating the subject based on the authentication score; Equipped with The authentication device is characterized in that the update means calculates the authentication score for the current frame by taking an exponential moving average of the authentication score for the previous frame of the moving image and the matching score for the current frame, and changes the value of a smoothing coefficient based on the magnitude relationship between the matching score and the authentication score to calculate the authentication score for the current frame.

2. 2. The authentication device according to claim 1, further comprising a selection unit for selecting an object to be matched with the authentication target from the subjects detected by the detection unit.

3. The authentication device according to claim 1, characterized in that the updating means updates the authentication score using a first updating method in which the smoothing coefficient is an exponential moving average of a first value when the matching score is smaller than the authentication score, and updates the authentication score using a second updating method in which the smoothing coefficient is an exponential moving average of a second value larger than the first value when the matching score is equal to or greater than the authentication score.

4. 2. The authentication device according to claim 1, wherein the updating means updates the authentication score using a third updating method that uses only the authentication score when the matching score of the subject cannot be obtained.

5. A detection means for detecting a subject from an input image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target, and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication means for authenticating the subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; when the matching score of the subject cannot be obtained, the updating means updates the authentication score using a third updating method using only the authentication score; In the third updating method, the authentication score is updated by multiplying the authentication score by a predetermined coefficient.

6. A detection means for detecting a subject from an input image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication means for authenticating the subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; when the matching score of the subject cannot be obtained, the updating means updates the authentication score using a third updating method using only the authentication score; In the third updating method, the authentication device updates the authentication score by subtracting a predetermined value from the authentication score.

7. The authentication device according to claim 1, wherein the updating means updates the authentication score for a subject that is being matched with the authentication target for the first time using a fourth updating method in which the matching score is used as the authentication score.

8. a detection step of detecting a subject from each frame of an input video; a matching step of matching the subject detected in the detection step with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an updating step of updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; an authentication step of authenticating the subject based on the authentication score; and In the updating step, the authentication score for the current frame is calculated by taking an exponential moving average of the authentication score for the previous frame of the video and the matching score for the current frame, and the authentication score for the current frame is calculated by changing the value of a smoothing coefficient based on the magnitude relationship between the matching score and the authentication score.

9. a detection means for detecting a subject from each frame of a captured moving image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with The imaging device is characterized in that the update means calculates the authentication score for the current frame by taking an exponential moving average of the authentication score for the previous frame of the moving image and the matching score for the current frame, and changes the value of a smoothing coefficient based on the magnitude relationship between the matching score and the authentication score to calculate the authentication score for the current frame.

10. 10. The imaging device according to claim 9, further comprising a selection unit for selecting an object to be matched with the authentication object from the subjects detected by the detection unit.

11. The imaging device described in claim 9, characterized in that the update means updates the authentication score using a first update method in which the smoothing coefficient is an exponential moving average of a first value when the matching score is smaller than the authentication score, and updates the authentication score using a second update method in which the smoothing coefficient is an exponential moving average of a second value larger than the first value when the matching score is greater than or equal to the authentication score.

12. 10. The imaging device according to claim 9, wherein the updating means updates the authentication score using a third updating method that uses only the authentication score when the matching score of the subject cannot be obtained.

13. A detection means for detecting a subject from a captured image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; when the matching score of the subject cannot be obtained, the updating means updates the authentication score using a third updating method using only the authentication score; In the third updating method, the authentication score is updated by multiplying the authentication score by a predetermined coefficient.

14. A detection means for detecting a subject from a captured image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; when the matching score of the subject cannot be obtained, the updating means updates the authentication score using a third updating method using only the authentication score; In the third updating method, the authentication score is updated by subtracting a predetermined value from the authentication score.

15. The imaging device according to any one of claims 9 to 14, characterized in that, for a subject that is being matched with the authentication target for the first time, the update means updates the authentication score using a fourth update method in which the matching score is used as the authentication score.

16. A detection means for detecting a subject from a captured image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; The imaging device is characterized in that, when a plurality of subjects are authenticated as the same authentication target, the determination means determines the subject with the higher authentication score to be the main subject.

17. A detection means for detecting a subject from a captured image; a matching means for matching the subject detected by the detection means with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an update means for updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination means for determining a main subject based on the authentication score; Equipped with the updating means changes a method for updating the authentication score based on a magnitude relationship between the matching score and the authentication score; The imaging device is characterized in that, when multiple subjects are authenticated as different authentication targets, the determination means determines the subject authenticated as the authentication target with the highest priority specified by the user to be the main subject.

18. 10. The imaging apparatus according to claim 9, wherein focus control or exposure control is performed preferentially on the subject determined to be the main subject by the determining means.

19. a detection step of detecting a subject from each frame of the captured video; a matching step of matching the subject detected in the detection step with a pre-registered authentication target and outputting a matching score indicating the degree of similarity between the subject and the authentication target; an updating step of updating an authentication score, which is an evaluation value indicating the degree to which the subject matches the authentication target, based on the matching score; a determination step of determining a main subject based on the authentication score; and In the updating process, the authentication score for the current frame is calculated by taking an exponential moving average of the authentication score for the previous frame of the moving image and the matching score for the current frame, and the authentication score for the current frame is calculated by changing the value of a smoothing coefficient based on the magnitude relationship between the matching score and the authentication score.

20. A program for causing a computer to execute each step of the method according to claim 8 or 19.

21. A computer-readable storage medium storing a program for causing a computer to execute each step of the method according to claim 8 or 19.

Images