Facial recognition device and its program

Abstract

To provide a face recognition device that can reduce the amount of calculation.SOLUTION: A face recognition device 1 comprises: a face detection processing unit 10 that detects a face area from a face image of a person to be recognized; a face database 20 that registers in advance a plurality of face feature quantities for every registrant; a face recognition processing unit 30 that calculates the similarity in the face feature quantity between the person to be recognized and the registrants; and a data processing unit 40 that removes an overlapping portion for the same registrant from the similarity by using a hash map table 60, and identifies the person to be recognized.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a face recognition device and a program therefor. [Background technology]

[0002] Facial recognition technology is known for identifying people by recognizing faces in images. For example, in the security field, facial recognition technology has become highly accurate due to recent advances in AI technology, and is now at the level of practical application. Furthermore, a method is known in which facial recognition processing is used for tagging video content.

[0003] These face recognition processes are based on a mechanism in which face images for learning are collected, facial features for recognizing differences between people are calculated using machine learning processing, and faces are recognized based on the similarity of the facial features (for example, Patent Document 1).

[0004] Here, if the facial state, such as the facial direction or expression, changes significantly, the accuracy of the face recognition process may decrease. Therefore, a method has been proposed to improve the accuracy of the face recognition process by calculating facial feature amounts from multiple facial images of the same person (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6669952 [Patent Document 2] Japanese Patent Publication No. 2021-033664 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method described in Patent Document 2 has a problem in that the amount of data for facial features increases because multiple facial images of the same person are used, resulting in a huge amount of calculation.

[0007] Therefore, an object of the present invention is to provide a face recognition device and a program therefor that can reduce the amount of calculation. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the face recognition device of the present invention is a face recognition device that identifies a person to be recognized from a facial image of the person to be recognized, and is configured to include a face detection processing unit, a facial feature calculation unit, a face database, a similarity calculation unit, a sorting processing unit, an overlapping portion removal unit, and a recognition target person identification unit.

[0009] According to this configuration, the face detection processing unit detects a face area from the face image of the person to be recognized. The face feature amount calculation unit calculates face feature amounts for each face area detected by the face detection processing unit. The face database pre-registers a plurality of facial feature amounts calculated from different facial images for each registrant. The similarity calculation unit calculates a similarity matrix that stores the similarity between the facial feature amount for each face region of the person to be recognized calculated by the facial feature amount calculation unit and each facial feature amount of the registrant registered in the face database.

[0010] The sorting unit sorts the elements of the similarity matrix calculated by the similarity calculation unit in order of similarity. The overlapping portion removal unit removes overlapping portions for the same registered person from the sorted similarity matrix using a hash-type map table that associates the identification information of the person to be recognized with each facial feature amount of the registered person. The recognition target person identification unit identifies the recognition target person based on the similarity matrix from which the overlapping portion has been removed by the overlapping portion removal unit. In this way, the face recognition device can reduce the enormous amount of calculation required when using multiple facial feature amounts by removing overlapping portions from the similarity matrix based on the hash-type map table.

[0011] The present invention can also be realized by a program for causing a computer to function as the face recognition device described above. [Effects of the Invention]

[0012] According to the present invention, the amount of calculation can be reduced. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing the configuration of a face recognition device according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a face recognition processing unit according to the first embodiment. [Figure 3] FIG. 2 is a block diagram showing the configuration of a data processing unit according to the first embodiment. [Figure 4] FIG. 2 is an explanatory diagram illustrating a hash-type map table in the first embodiment. [Figure 5] FIG. 2 is an explanatory diagram illustrating a specific example of face recognition processing in the first embodiment. [Figure 6] 10 is a flowchart showing the processing of a registration interface in the first embodiment. [Figure 7] 10 is a flowchart showing the processing of a face recognition device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Each embodiment will be described below with reference to the drawings. However, each embodiment described below is intended to embody the technical concept of the present invention, and unless otherwise specified, the present invention is not limited to the following. Furthermore, the same means will be given the same reference numerals, and their description may be omitted.

[0015] (First embodiment) [Outline of face recognition device] Referring to FIG. 1, an overview of a face recognition device 1 according to a first embodiment will be described. 1, the face recognition device 1 identifies a person to be recognized from a facial image of the person to be recognized, and as shown in Fig. 1, it includes a face detection processing unit 10, a face database 20, a face recognition processing unit 30, a data processing unit 40, and a registration interface (face database registration unit) 50. Note that a person to be recognized by the face recognition device 1, that is, a person photographed in a facial image input to the face recognition device 1, will be referred to as a person to be recognized.

[0016] The face recognition device 1 aims to improve recognition accuracy by registering multiple facial feature amounts for the same person in the face database 20. A person who registers facial feature amounts in the face database 20 is referred to as a registrant. The input to the face recognition device 1 is a facial image that includes the facial region of the person to be recognized (this facial image includes a moving image). In addition, a hash-type map table 60, which will be described later, is set in advance in the data processing unit 40.

[0017] First, the face detection processing unit 10 detects a face region in the face image that contains a face. Next, the face recognition processing unit 30 performs face recognition processing on each face region. At this time, the face recognition processing unit 30 calculates facial features from the face region and calculates the similarity with the facial features of pre-calculated registrants. The face database 20 stores the pre-calculated facial features of registrants. Finally, the data processing unit 40 identifies the person in each face region based on the similarity. At this time, the data processing unit 40 reduces the amount of calculation by using a hash-type map table 60 that associates the identification information of the person to be recognized with each facial feature of registrants registered in the face database 20.

[0018] The registration of facial features in the face database 20 is performed manually via the registration interface 50. At this time, the process of extracting facial regions that contribute greatly to improving recognition accuracy and excluding facial regions that result in a deterioration of recognition accuracy is automatically performed (second embodiment).

[0019] [Configuration of face recognition device] <Face detection processing section> The configuration of the face recognition device 1 will be described below. The face detection processing unit 10 detects a face area from a face image of a person to be recognized. Here, the face detection processing unit 10 detects a face area where a face is present from the face image, and outputs information about the detected area (for example, a bounding area surrounding the face) to the face recognition processing unit 30. For example, the face detection processing unit 10 can use a general face area detection method such as a method using deep learning processing (Japanese Patent Application Laid-Open No. 2020-017082).

[0020] <Face Database> The face database 20 registers in advance facial feature values ​​calculated from facial images of different facial states, such as facial orientation and facial expression, for each registrant. In this embodiment, the face database 20 registers a plurality of facial feature values ​​in a matrix format for each registrant, including the person to be recognized. For example, a user of the face recognition device 1 manually registers data in the face database 20 via a registration interface 50, which will be described later.

[0021] <Face recognition processing section> The face recognition processing unit 30 calculates face feature amounts from the face area detected by the face detection processing unit 10, and calculates the similarity with the face feature amounts of registrants registered in the face database 20. As shown in FIG. 2, the face recognition processing unit 30 includes a face feature amount calculation unit 31, a similarity calculation unit 32, and a sorting processing unit 33.

[0022] The facial feature amount calculation unit 31 calculates facial feature amounts for each face region detected by the face detection processing unit 10. For example, the facial feature amount calculation unit 31 calculates a one-dimensional facial feature vector representing the facial feature amounts using the deep learning process described in Japanese Patent Application Laid-Open No. 2017-33372.

[0023] The similarity calculation unit 32 calculates a similarity matrix that stores the similarity between the facial feature values ​​for each facial area of ​​the person to be recognized calculated by the facial feature calculation unit 31 and each facial feature value of the registered person registered in the face database 20.

[0024] Here, the number of face regions detected by the face detection processing unit 10 is defined as n_dets. Furthermore, the length of the face feature vector, i.e., the number of dimensions of the face feature, is defined as n_feats. Then, the face feature output by the face feature calculation unit 31 is a matrix F of size (n_dets, n_feats). Furthermore, the face database 20 stores a plurality of face features corresponding to each registrant in matrix format. This matrix is ​​referred to as R. If the total number of face features of a certain registrant is defined as n_regs, the size of matrix R is (n_regs, n_feats).

[0025] Furthermore, the number of people registered in the face database 20 is n_people, and the number of facial features of the i-th person registered is n_faces[i]. Then, the relationship of the following equation (1) is established.

[0026]

number

[0027] When the two matrices F and R are defined in this way, the similarity can be calculated by a matrix operation using the matrices F and R. In this embodiment, the similarity calculation unit 32 performs a cosine similarity operation. In this case, the cosine similarity is expressed by the following equation (2). Note that F t represents the transposition.

[0028]

number

[0029] As a result, the similarity calculation unit 32 outputs a similarity matrix Sim of size (n_regs, n_feats) to the sorting processing unit 33. This similarity matrix Sim is a matrix whose elements are the similarities between all face features stored in the face database 20 and face features calculated from the detected face area.

[0030] The sorting unit 33 sorts the elements of the similarity matrix Sim calculated by the similarity calculation unit 32 in order of similarity. That is, the sorting unit 33 sorts the similarity matrix Sim by row number in descending order of similarity for each face region of a registrant. Hereinafter, the sorted similarity matrix will be referred to as ArgSim. By sorting the similarity matrix Sim in this way, the data processing unit 40 (described later) can delete duplicate portions for the same registrant.

[0031] <Data processing unit> Returning to FIG. 1, the description of the face recognition device 1 will continue. The data processing unit 40 removes duplicated portions from the sorted similarity matrix ArgSim using a hash-type map table 60. As shown in FIG. 3, the data processing unit 40 includes an overlapping portion removal unit 41 and a recognition target person identification unit 42.

[0032] <<Hash type map table>> With reference to FIG. 4, the three hash-type map tables 60 (61, 62, 63) used by the data processing unit 40 will be described.

[0033] The multi_id2id table 61 is a hash-type map table that associates each facial feature (multi_id) of a registrant with identification information (id). Here, multi_id represents the identification information of each facial feature of a registrant, that is, the row number (e.g., 0 to 5) of the sorted similarity matrix ArgSim described later.

[0034] The id2person table 62 is a hash-type map table that associates the identification information (id) of a registrant with the registrant name (person). In this id2person table 62, the registrant name for id=1 is A, the registrant name for id=2 is B, and the registrant name for id=2 is C.

[0035] Here, when registering facial features in the face database 20, the user of the face recognition device 1 inputs the name of the registrant (person). At this time, identification information (id) is automatically assigned as a consecutive number to each registrant. Therefore, the id and person are stored in association with each other in the id2person table 62. Furthermore, a multi_id is automatically assigned as a consecutive number to each facial feature registered in the face database 20. Therefore, the multi_id and id are stored in association with each other in the multi_id2id table 61.

[0036] The part2multi_id table 63 is a hash-type map table for narrowing down registrants of a specific group from the sorted similarity matrix ArgSim. Here, part_id is identification information for the group to which the registrant belongs. This group can be defined arbitrarily (for example, address, occupation, place of employment, hobbies, gender). Note that the part2multi_id table 63 is not required if the sorted similarity matrix ArgSim is not narrowed down.

[0037] 4, for ease of explanation, the data stored in the hash-type map table 60 is shown as is. In reality, the hash-type map table 60 stores data in a hash format consisting of keys and values ​​(for example, a Python dict type). For example, the hash-type map table is described in Reference 1 below, so further explanation will be omitted.

[0038] Reference 1: “Hash Table”, [online], [Retrieved May 16, 2022], Internet <URL:https: / / ja.wikipedia.org / wiki / %E3%83%8F%E3%83%83%E3%82%B7%E3%83%A5%E3%83%86%E3%83%BC%E3%83%96%E3%83%AB>

[0039] Returning to FIG. 3, the description of the data processing unit 40 will continue. The overlap removal unit 41 removes overlapping portions for the same registrant from the sorted similarity matrix ArgSim using a hash-type map table (multi_id2id table 61) that associates each facial feature of a registrant with identification information. That is, the overlap removal unit 41 uses the multi_id2id table 61 to obtain a similarity matrix in which overlapping portions have been removed from the sorted similarity matrix ArgSim. Hereinafter, the removed similarity matrix will be referred to as ArgSimP. The size of this removed similarity matrix ArgSimP is (n_people, n_dets).

[0040] The recognition target person identification unit 42 identifies the recognition target person based on the similarity matrix ArgSimP from which the overlapping portions have been removed by the overlapping portion removal unit 41. In this embodiment, the recognition target person identification unit 42 performs conversion processing using the id2person table 62, and outputs the name of the recognition target person as the recognition result.

[0041] <<Example of face recognition processing>> A specific example of face recognition processing by the face recognition device 1 will be described with reference to FIG. In this specific example, there are three registrants, A, B, and C. Furthermore, three facial features AF1, AF2, and AF3 are registered for registrant A in the face database 20. Furthermore, two facial features BF1 and BF2 are registered for registrant B in the face database 20. Furthermore, one facial feature CF1 is registered for registrant C in the face database 20. Furthermore, the face image 90 includes one face region 91 (n_dets=1). Furthermore, the number of dimensions of the facial features is set to n_feats=512. Furthermore, the data shown in FIG. 4 is set in the multi_id2id table 61 and the id2person table 62.

[0042] In this case, the matrix F output by the facial feature amount calculation unit 31 has a size of (1,512). Also, the matrix R acquired from the face database 20 is a vertical stack of the feature vectors of the facial features AF1, AF2, AF3, BF1, BF2, and CF1, and has a size of (6,512).

[0043] Therefore, the similarity calculation unit 32 calculates a similarity matrix Sim having a size of (6, 1) using the above-mentioned formula (2). Here, it is assumed that the similarity matrix Sim expressed by the following formula (3) is calculated.

[0044]

number

[0045] In the similarity matrix Sim in equation (3), the similarity 0.75 is in row 0, the similarity 0.84 is in row 1, the similarity 0.63 is in row 2, the similarity 0.78 is in row 3, the similarity 0.43 is in row 4, and the similarity 0.66 is in row 5. For convenience of implementation, the row numbers of the similarity matrix Sim start from 0.

[0046] Further, the sorting unit 33 sorts the elements of the similarity matrix Sim of equation (3) in order of similarity. The similarity matrix Sim′ sorted in order of similarity is expressed by the following equation (4).

[0047]

number

[0048] The sorted similarity matrix ArgSim stores the row numbers of the similarity matrix Sim′ in equation (4), and is therefore expressed by equation (5) below.

[0049]

number

[0050] Here, the first similarity, 0.84, in the similarity matrix Sim' of equation (4) is the first row in the similarity matrix Sim of equation (3). Therefore, in the sorted similarity matrix ArgSim of equation (5), row number 1 of the similarity 0.84 is stored as the first element. Also, the second similarity, 0.78, in the similarity matrix Sim' of equation (4) is the third row in the similarity matrix Sim of equation (3). Therefore, in the sorted similarity matrix ArgSim of equation (5), row number 3 of the similarity 0.78 is stored as the second element.

[0051] The data processing unit 40 obtains the final recognition result for the sorted similarity matrix ArgSim using the multi_id2id table 61 and the id2person table 62.

[0052] First, the overlapping portion removal unit 41 converts each element (row number) of the sorted similarity matrix ArgSim into the identification information (id) of the registrant using the multi_id2id table 61, as shown in the following equation (6).

[0053]

number

[0054] As described above, the row numbers of the sorted similarity matrix ArgSim correspond to multi_id in the multi_id2id table 61. That is, the first element, row number 1, in the sorted similarity matrix ArgSim corresponds to multi_id=1 in the multi_id2id table 61, and therefore id=0. Therefore, the first row number 1 in the sorted similarity matrix ArgSim is converted to id=0, which is the identification information of the registrant. Similarly, the second element, row number 3 in the sorted similarity matrix ArgSim, corresponds to multi_id=3 in the multi_id2id table 61, and therefore id=1. Therefore, the second row number 3 in the sorted similarity matrix ArgSim is converted to id=1, which is the identification information of the registrant.

[0055] Next, the overlapping part removal unit 41 removes overlapping parts from the transformation result of the right side of equation (6) to obtain the removed similarity matrix ArgSimP. Here, since the third, fifth, and sixth elements, which overlap with id=0, 1, are removed from the transformation result of the right side of equation (6), the removed similarity matrix ArgSimP is expressed by the following equation (7).

[0056]

number

[0057] Next, the recognition target person specification unit 42 converts each element (registrant's id) of the removed similarity matrix ArgSimP into a registrant's name using the id2person table 62, as shown in the following equation (8).

[0058]

number

[0059] In the id2person table 62, the registrant name for id=0 is A. Therefore, the first element of the removed similarity matrix ArgSimP, id=0, is converted to the registrant name A. Also, in the id2person table 62, the registrant name for id=1 is B. Therefore, the second element of the removed similarity matrix ArgSimP, id=1, is converted to the registrant name B.

[0060] Here, in the removed similarity matrix ArgSimP, ids are stored in descending order of similarity. Therefore, the recognition target person specification unit 42 outputs the registered person name corresponding to the first element in the removed similarity matrix ArgSimP as the identification result. In this specific example, the registered person name A is the identification result.

[0061] The following describes how to narrow down the sorted similarity matrix ArgSim by group using the id2person table 62. Here, the search is narrowed down to the group to which registrants A and C belong. In other words, registrant B belongs to a different group from registrants A and C, so he is excluded from the search.

[0062] As shown in FIG. 4, in the id2person table 62, the ids of registrants A and C are 0 and 2. In the multi_id2id table 61, the multi_ids corresponding to id=0 are 0, 1, and 2, and the multi_id corresponding to id=2 is 5. Therefore, the part2multi_id table 63 should store multi_id=0, 1, 2, and 5 corresponding to registrants A and C. Note that part_id is a sequential number automatically assigned to each multi_id. In the example of FIG. 4, 0 to 3 are stored as part_id in the part2multi_id table 63.

[0063] Here, when the sorted similarity matrix ArgSim of equation (5) is narrowed down by the part2multi_id table 63 of Fig. 4, it is expressed by the following equation (9). In the sorted similarity matrix ArgSim of equation (9), multi_id=3,4 corresponding to registrant B is removed from the sorted similarity matrix ArgSim of equation (5).

[0064]

number

[0065] Thereafter, the overlapping portion removal unit 41 may perform the same processing as above on the sorted similarity matrix ArgSim of equation (9).

[0066] <Registration Interface> Returning to FIG. 1, the description of the face recognition device 1 will continue. The registration interface 50 is used to register facial features in the face database 20. As shown in Fig. 6, the registration interface 50 performs registration by combining a person's name and a face image.

[0067] First, in step S1, the face detection processing unit 10 detects a facial region from the facial image of the registrant, similar to the facial image of the recognition target. In step S2, the facial feature amount calculation unit 31 calculates a facial feature amount F for each facial region of the registrant, similar to the facial feature amount of the recognition target. In step S3, the similarity calculation unit 32 acquires the facial feature amount R[id] of the registrant from the face database 20. At this time, the similarity calculation unit 32 obtains the id corresponding to the registrant name (person) using the person2id table 64, which is a reverse map version of the id2person table 62. In step S4, the similarity calculation unit 32 calculates the similarity SimScore between the facial feature amount F of the registrant calculated by the facial feature amount calculation unit 31 and each facial feature amount R[id] of the registrant registered in the face database.

[0068] Next, in step S5, the registration interface 50 registers the facial feature amount F calculated by the facial feature amount calculation unit 31 in the face database by threshold determination of the similarity SimScore calculated in step S4. Specifically, the registration interface 50 determines whether the similarity SimScore is less than a preset threshold Thres_sim. If the similarity SimScore is less than the threshold Thres_sim (Y in step S5), the registration interface 50 cancels the registration of the facial feature amount F (step S6). On the other hand, if the similarity SimScore is equal to or greater than the threshold Thres_sim (N in step S5), the registration interface 50 registers the facial feature amount F in the face database 20 (step S7).

[0069] In this way, the registration interface 50 can suppress erroneous recognition because it does not register the facial feature amount F obtained from a face image with significantly different features in the face database 20. Furthermore, this registration process only requires selecting a combination of the registrant's name (person) and face image (image), so there is no need to input the area in which the registrant is visible, which is convenient for the user of the face recognition device 1.

[0070] [Actions and Effects] As described above, by employing the hash-type map table 60, the face recognition device 1 can suppress the enormous amount of calculation required when using a plurality of facial feature amounts. Furthermore, the face recognition device 1 uses multiple facial feature amounts, which can improve the accuracy of face recognition. In particular, the face recognition device 1 can accurately recognize important recognition targets by registering multiple facial feature amounts obtained in various facial states for the targets in the face database 20. Furthermore, the face recognition device 1 can selectively register facial feature amounts that contribute to improving the accuracy of face recognition using the registration interface 50. For example, the face recognition device 1 can efficiently perform processing to add the name of a person to be recognized as metadata to video content.

[0071] (Second embodiment) With reference to FIGS. 1 and 7, a face recognition device 1B according to the second embodiment will be described with respect to differences from the first embodiment. As shown in FIG. 7, the face recognition device 1B differs from the first embodiment in that a similarity calculation unit 32B and a sorting unit 33B perform two-stage processing.

[0072] Since multiple facial feature amounts are used, the amount of data increases, which may strain the resources of face recognition device 1B, such as memory. To reduce this effect, face recognition device 1B performs similarity calculation and sorting processing in stages.

[0073] In step S10, as the first stage of processing, one facial feature (matrix R_S) is prepared for each registrant. For example, the matrix R_S has an element that is the average vector of all facial features for each registrant. The size of this matrix R_S is (n_people, n_feats). The first stage of processing is performed using this matrix R_S.

[0074] The similarity calculation unit 32B calculates a similarity matrix that stores the similarity between the facial feature F for each facial region of the recognition target person and one facial feature (matrix R_S) previously assigned to the registrant. Then, the sorting unit 33B sorts the elements in the similarity matrix up to a predetermined ranking topS in order of similarity.

[0075] In this way, in the first-stage sorting process, instead of sorting all elements, the top S elements with the highest similarity are sorted, and it is possible to register the registrants with the highest similarity up to the top S.

[0076] In step S11, as a second-stage process, face recognition processing is performed using a plurality of facial feature amounts limited to the topS registrants. Specifically, the similarity calculation unit 32B calculates a similarity matrix storing the facial feature amount F for each face region of the recognition target person and the similarity between each facial feature amount of the registrants up to the topS in similarity ranking. Thereafter, the similarity matrix calculated in the second-stage process can be used as the similarity matrix Sim.

[0077] [Actions and Effects] As described above, the face recognition device 1B can suppress the enormous amount of calculation required when using a plurality of facial feature amounts, similar to the first embodiment. Furthermore, in the sorting process, face recognition device 1B can significantly reduce the amount of data compared to when multiple facial features are used for all registered individuals, thereby suppressing a decrease in processing speed and avoiding strain on face recognition device 1B's resources, such as memory.

[0078] Although the embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design modifications and the like within the scope of the present invention.

[0079] In the above-described embodiment, the face recognition device is described as an independent piece of hardware, but the present invention is not limited to this. For example, the present invention can also be realized by a program that causes hardware resources such as a CPU, memory, and hard disk of a computer to function as the face recognition device. This program may be distributed via a communication line or written to a recording medium such as a CD-ROM or flash memory. [Explanation of symbols]

[0080] 1,1B Facial Recognition Device 10 Face detection processing unit 20 Face Database 30 Face recognition processing unit 31 Facial feature calculation unit 32,32B Similarity calculation part 33,33B Sorting processing section 40 Data processing unit 41 Overlapping part removal part 42 Recognition target identification unit 50 Registration interface (face database registration section) 60 Hash Map Table

Claims

1. A face recognition device that identifies a person to be recognized from a facial image of the person to be recognized, a face detection processing unit that detects a face area from a face image of the person to be recognized; a face feature amount calculation unit that calculates a face feature amount for each face area detected by the face detection processing unit; a face database in which a plurality of facial feature amounts calculated from different facial images are registered in advance for each registrant; a similarity calculation unit that calculates a similarity matrix that stores the similarity between the facial feature amount for each face region of the person to be recognized calculated by the facial feature amount calculation unit and each facial feature amount of a registrant registered in the face database; a sorting processing unit that sorts each element of the similarity matrix calculated by the similarity calculation unit in order of the similarity; an overlapping portion removal unit that removes overlapping portions for the same registrant from the sorted similarity matrix using a hash-type map table that associates each facial feature amount of the registrant with identification information of the registrant; a target person identification unit that identifies the target person based on the similarity matrix from which the overlapping portion has been removed by the overlapping portion removal unit; A face recognition device comprising:

2. As the first stage of processing, the similarity calculation unit calculates a similarity matrix storing a similarity between a facial feature amount for each facial region of the recognition target person and one facial feature amount pre-assigned to the registrant; the sorting unit sorts elements in the similarity matrix up to a predetermined rank of the similarity; As the second stage of processing, 2. The face recognition device according to claim 1, wherein the similarity calculation unit calculates a similarity matrix that stores the similarities between the facial feature values ​​of each facial region of the person to be recognized and the facial feature values ​​of each registered person whose similarity is up to the rank.

3. the face detection processing unit detects a face area from a face image of the registrant; the facial feature amount calculation unit calculates a facial feature amount for each face area of ​​the registrant; the similarity calculation unit calculates a similarity between the facial feature amount calculated by the facial feature amount calculation unit and each facial feature amount of a registrant registered in the face database; 2. The face recognition device according to claim 1, further comprising a face database registration unit that registers the facial feature amount calculated by the facial feature amount calculation unit in the face database based on a threshold determination of the calculated similarity.

4. A program for causing a computer to function as the face recognition device according to any one of claims 1 to 3.

Images