Multi-scene adaptive model fusion method and face recognition system

Abstract

The present application provides a multi-scenario adaptive model fusion method and a face recognition system. The present application selects several face recognition models that meet the computing speed requirements according to different target platforms, combines them into several model combinations, and then evaluates the accuracy and threshold of each model combination in different scenarios. According to the accuracy and threshold of different model combinations, a model combination with high accuracy and strong threshold compatibility is selected to construct a fusion model. The fusion model thus obtained can achieve compatibility with multiple face recognition scenarios with a single threshold, can be quickly deployed in different recognition scenarios, and improve the recognition speed and recognition accuracy of the system for face recognition.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and more specifically to a multi-scene adaptive model fusion method and a face recognition system. Background Art

[0002] Face recognition technology analyzes and processes facial visual feature information to identify people. Compared with other biometric features, face features have advantages such as naturalness, convenience, and non-contact, making them have great application prospects in security monitoring, identity authentication, and human-computer interaction. Due to the wide application of face recognition technology, face recognition currently occupies an important position in the computer field.

[0003] Generally speaking, the face recognition process is divided into two processes: face feature extraction and face similarity score calculation. The face feature extraction process is to extract some key features of the face image to form a face feature vector, and the face similarity score calculation process is to calculate the similarity between two face feature vectors. The higher the similarity, the more likely it is that the two face images are from the same person. Conversely, the more likely it is that the two face images are from different people. In some cases, the face feature extraction part is more concerned.

[0004] Existing facial feature extraction methods include the LBP (local binary pattern) method and its variants. These local texture feature extraction methods form a histogram vector by dividing the entire face image into blocks, and cascade the histogram vectors of each block to finally form a facial feature vector. Since this method extracts local texture features from the entire face, the dimension of the feature vector formed is relatively large, which contains some redundant information and is easily affected by facial occlusions, light, and environmental background, resulting in recognition bias. Existing face recognition technology is not robust to changes in expression or posture in complex environments. Existing various types of face recognition models may have the problem of overfitting in a certain scene, that is, the recognition accuracy is high in a certain scene, but the accuracy is very low in other scenes. Summary of the invention

[0005] In view of the shortcomings of the prior art, this application provides a multi-scenario adaptive model fusion method and a face recognition system. This application achieves compatibility with multiple face recognition scenarios with a single threshold through a fusion model, can be quickly deployed in different recognition scenarios, and improves the recognition speed and recognition accuracy of the system for face recognition. This application specifically adopts the following technical solutions.

[0006] Firstly, to achieve the above purpose, a multi-scenario adaptive model fusion method is proposed, which includes the following steps: the first step is to exhaustively enumerate the combinations of face recognition models corresponding to different scenes in the model library; the second step is to screen out the combinations whose operation speed meets the requirements of the target platform in the first step, and record them as model combinations; the third step is to calculate the accuracy and threshold C{a1,t1}, C{a2,t2},...,C{an,tn} of each model combination obtained in the second step in each scene, and then normalize each accuracy and each threshold to obtain the normalized accuracy An and normalized threshold Tn of each model combination in each scene; wherein n represents the scene number, an represents the accuracy of the model in the nth scene, and tn represents the threshold of the model in the nth scene; the fourth step is to calculate the weighted sum of the normalized accuracy ACC=C{w1*A1+w2*A2+…+wn*An} of each model combination in each scene, and calculate the variance VAR=var(T1, T2, T3,… , Tn); wherein wn represents the weighted value of the normalized accuracy of the model combination corresponding to the nth scene; the fifth step, according to the weighted sum of the normalized accuracy ACC and the variance VAR of the normalized threshold, respectively calculate the evaluation value Eval=ACC+(1-VAR) of each model combination and the sum; the sixth step, screen out the model combination with the highest evaluation value Eval, and build a fusion model based on the model combination to splice and combine the feature vectors extracted by each face recognition model in the model combination and perform face recognition based on the feature vector group obtained after splicing and combination.

[0007] Optionally, in the third step of the multi-scenario adaptive model fusion method as described in any of the above, the accuracy an of the model combination in the nth scene is obtained by the following steps: calculating the ROC curve of the model combination in the nth scene according to the test set corresponding to the nth scene; searching the ROC curve for the recall rate or false detection rate that meets the false detection rate requirements, and calculating the accuracy an of the model combination in the nth scene.

[0008] Optionally, in the third step of the multi-scenario adaptive model fusion method as described in any of the above, the threshold tn of the model combination in the nth scene is obtained by the following steps: calculating the ROC curve of the model combination in the scene according to the test set corresponding to the nth scene; and searching the ROC curve for the threshold tn that meets the false detection rate requirements.

[0009] Optionally, a multi-scenario adaptive model fusion method as described in any of the above, wherein the step of calculating the ROC curve of the model combination in the scenario according to the test set corresponding to the nth scene includes: step r1, extracting the model feature vectors corresponding to each face image in the test set according to each face recognition model included in the model combination; step r2, concatenating the model feature vectors extracted by each face recognition model in step r1 into a multidimensional vector; step r3, comparing the inter-vector distance between the multidimensional vector obtained by the concatenated combination and the recognition vector corresponding to each face image, and obtaining the false detection rate and recall rate under the threshold according to different thresholds.

[0010] Optionally, a multi-scene adaptive model fusion method as described in any of the above, wherein the fusion model is used to perform recognition processing on the face image to be recognized according to the following steps: step S1, extracting the model feature vector corresponding to the face image to be recognized according to each face recognition model contained in the fusion model; step S2, concatenating the model feature vectors extracted by each face recognition model in step S1 into a multidimensional vector; step S3, comparing the inter-vector distance between the multidimensional vector obtained by the combination and the recognition vector corresponding to each recognition object, and when the Euclidean distance between the two vectors is less than a threshold value Tn, outputting the recognition result as the recognition object corresponding to the recognition vector.

[0011] Optionally, in a multi-scenario adaptive model fusion method as described above, the recognition vectors corresponding to each recognition object are pre-stored in a storage unit according to the following steps: first, the model feature vectors corresponding to the recognition object are extracted according to each face recognition model included in the fusion model; then, the model feature vectors extracted from each face recognition model are combined into a one-dimensional recognition vector; the recognition vector is stored in a storage unit and the correspondence between it and the recognition object is marked.

[0012] At the same time, in order to achieve the above-mentioned purpose, the present application also provides a face recognition system, which includes: an image acquisition module, used to acquire face images to be identified; a first storage unit, which stores a model library, and each face recognition model in the model library corresponds to different scenes; a second storage unit, which stores an executable program, and when the executable program is executed by the processor, the processor constructs a fusion model according to any of the method steps described above, so as to record the recognition vectors corresponding to each recognition object according to the constructed fusion model, and perform recognition processing on the face image to be identified according to the constructed fusion model.

[0013] Optionally, in a face recognition system as described in any of the above, the specific steps of performing recognition processing on the face image to be recognized according to the obtained fusion model include: step S1, extracting the model feature vector corresponding to the face image to be recognized according to each face recognition model contained in the fusion model; step S2, splicing and combining the model feature vectors extracted by each face recognition model in step S1 into a multi-dimensional vector; step S3, comparing the inter-vector distance between the multi-dimensional vector obtained by the splicing combination and the recognition vector corresponding to each recognition object, and when the Euclidean distance between the two vectors is less than a threshold value Tn, outputting the recognition result as the recognition object corresponding to the recognition vector, otherwise it is judged that the recognition has failed.

[0014] Optionally, a face recognition system as described in any of the above, further includes an interactive interface, which is used to receive the setting of the weighted value wn of the normalized accuracy corresponding to the model combination in the nth scenario in the fourth step.

[0015] Optionally, a face recognition system as described in any of the above, further includes a recognition object storage unit for storing recognition vectors corresponding to each recognition object; the recognition vectors are stored by the following steps: first, the model feature vectors corresponding to the recognition object are extracted respectively according to each face recognition model included in the fusion model; then, the model feature vectors extracted respectively by each face recognition model are concatenated and combined into a multi-dimensional recognition vector; the multi-dimensional recognition vector is stored in the recognition object storage unit and the correspondence between it and the recognition object is marked.

[0016] Beneficial Effects

[0017] This application selects several face recognition models that meet the computing speed requirements according to different target platforms, combines them into several model combinations, and then evaluates the accuracy and threshold of each model combination in different scenarios. According to the accuracy and threshold of different model combinations, the model combination with high accuracy and strong threshold compatibility is selected to build a fusion model. The fusion model obtained in this way can achieve compatibility with multiple face recognition scenarios with a single threshold, can be quickly deployed in different recognition scenarios, and improve the recognition speed and recognition accuracy of the system for face recognition.

[0018] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:

[0020] Figure 1It is a schematic diagram of the process steps of the multi-scenario adaptive model fusion method of the present application;

[0021] Figure 2 It is a schematic diagram of the principle of multi-scenario adaptive model fusion in this application. DETAILED DESCRIPTION

[0022] In order to make the purpose and technical solution of the embodiment of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0024] The meaning of "and / or" described in this application means that the situations where each exists alone or both exist at the same time are included.

[0025] The term “connection” as used in this application may mean a direct connection between components or an indirect connection between components via other components.

[0026] Figure 1 A multi-scenario adaptive model fusion method according to the present application can be Figure 2 The fusion model suitable for various application scenarios can be constructed in the manner shown. The fusion model can be installed in the face recognition system with the image acquisition module through the installation program. Figure 1 The steps are to realize effective recognition of face images in different working environments according to a unified threshold.

[0027] The above-mentioned face recognition system may be configured to include:

[0028] An image acquisition module, which may be implemented by a camera or an image sensor, and is used to acquire the face image to be identified;

[0029] A first storage unit, which can be arranged inside the face recognition system or can realize cloud interaction through a communication network, wherein a model library is stored in the first storage unit, and each face recognition model in the model library corresponds to a different scene;

[0030] The second storage unit can also be set locally in the image acquisition module or can provide recognition operations through cloud interaction. The second storage unit stores an executable program. When the executable program is executed by the local processor of the image acquisition module or the cloud processor, the corresponding processor can be set to build a fusion model according to the following method steps, so as to record the recognition vector corresponding to each recognition object according to the constructed fusion model, and perform recognition processing on the face image to be recognized according to the constructed fusion model:

[0031] The first step is to exhaustively list the combinations of face recognition models corresponding to different scenarios in the model library;

[0032] The second step is to select the combinations whose face recognition model operation speed meets the requirements of the target platform among the combinations in the first step, and record them as model combinations;

[0033] The third step is to calculate the accuracy and threshold of each model combination obtained in the second step in each scene. Taking one of the model combinations as an example, its accuracy and threshold in n scenes can be recorded as C{a1,t1}, C{a2,t2},..., C{an,tn}, respectively. Then, each accuracy and each threshold are normalized to obtain the normalized accuracy An and normalized threshold Tn of each model combination in each scene; where n represents the scene number, an represents the accuracy of the model in the nth scene, and tn represents the threshold of the model in the nth scene.

[0034] The fourth step is to unify the thresholds of the fusion model in multiple scenes, calculate the weighted sum of the normalized accuracy of each model combination in n scenes ACC=C{w1*A1+w2*A2+…+wn*An}, and calculate the variance of the normalized threshold of each model combination in n scenes VAR=var(T1, T2, T3,… , Tn). The smaller the variance of the threshold, the stronger the cross-scene compatibility of the model; where wn represents the weighted value of the normalized accuracy corresponding to the currently calculated model combination in the nth scene;

[0035] The fifth step is to calculate the evaluation value Eval=ACC+(1-VAR) of each model combination according to the weighted sum of the normalized accuracy ACC and the variance VAR of the normalized threshold, and use the comprehensive accuracy and threshold variance to evaluate the index. The higher the comprehensive value, the stronger the scene adaptability of the fusion model.

[0036] The sixth step is to select the model combination with the highest evaluation value Eval, and build a fusion model based on the model combination, so as to combine the feature vectors extracted by each face recognition model in the model combination through the fusion model, and compare the recognition vectors corresponding to each recognition object according to the feature vector group obtained after the combination, and perform face recognition based on whether the distance between the vectors calculated by the comparison reaches the threshold value Tn. For example, if a fusion model contains 3 face recognition models, and each face recognition model can output a 512-dimensional feature vector respectively, the feature vector after the combination will contain the vectors of the three models, which will be fused into a 3x512-dimensional feature vector, which will be compared with the recognition vectors corresponding to each recognition object, so as to determine whether the feature vector is the recognition object corresponding to the recognition vector.

[0037] In the above process, the accuracy an of any model combination in the nth scene is obtained by the following steps:

[0038] First, establish a test set for face recognition. Generally, each different face recognition scenario, such as adult scenario, child scenario, mask-wearing scenario, etc., needs to correspond to a different test set. In order to evaluate the accuracy of the face recognition model in different scenarios, it is necessary to calculate the ROC curve on each test set to obtain the recall rate under a certain false positive rate. At the same time, corresponding to different thresholds, the test set needs to contain the base database of the same person and the snapshots in the corresponding scenarios;

[0039] Then, each face recognition model included in the model combination is used to extract the model feature vectors corresponding to each face in the base database and each face in the captured photo in the test set, the model feature vectors extracted by each of the above face recognition models are spliced ​​and combined into a multi-dimensional vector, and the Euclidean distance is calculated between the face base database and the face captured features;

[0040] Taking the number of databases as n and the number of snapshots as m as an example, the above calculation can obtain mxn groups of Euclidean distances, so the comparison results of the features between the face database and the snapshots can be sorted from large to small, and the vector distance between the multidimensional vector obtained by the splicing combination and the recognition vector corresponding to each face image is obtained. According to the requirements for the false detection rate in the scene, the false detection rate and recall rate corresponding to different thresholds are calculated, thereby obtaining the ROC curve of the test set corresponding to each scene in the scene of the model combination;

[0041] By finding the recall rate or false detection rate that meets the false detection rate requirements in the ROC curve, the accuracy an of the model combination in the nth scenario can be calculated.

[0042] The ROC curve is used as an indicator to measure the model. The horizontal axis of the curve is the false positive rate, and the vertical axis is the recall rate. Therefore, each point on the curve can represent the pass rate under different false positive rates, corresponding to different thresholds. In order to control the false positive rate of face recognition, the recall rate under a certain false positive rate is generally selected as the indicator for evaluating the model.

[0043] Similarly, the threshold tn of any model combination in the nth scene is obtained by the following steps:

[0044] Calculate the ROC curve of the model combination in the scenario according to the test set corresponding to the scenario No. n;

[0045] Find the threshold tn that meets the false positive rate requirement in the ROC curve.

[0046] Figure 2 The fusion model obtained by this method can be used to identify the face image to be identified according to the following steps:

[0047] First, the recognition vectors corresponding to each recognition object are pre-stored in the storage unit according to the following steps ac as a reference for face recognition evaluation:

[0048] Step a, extracting the model feature vector corresponding to the recognition object according to each face recognition model included in the fusion model,

[0049] Step b: combining the model feature vectors extracted by each face recognition model into a one-dimensional recognition vector.

[0050] Step c, storing the recognition vector in a storage unit and marking the corresponding relationship between the recognition vector and the recognition object;

[0051] Then, perform face recognition on the identification object in the storage unit according to steps S1 to S3:

[0052] Step S1, extracting the model feature vector corresponding to the face image to be recognized according to each face recognition model included in the fusion model;

[0053] Step S2, combining the model feature vectors extracted by each face recognition model in step S1 into a multi-dimensional vector;

[0054] Step S3, compare the vector distance between the multidimensional vector obtained by the combination and the identification vector corresponding to each identification object. When the Euclidean distance between the two vectors is less than the threshold Tn, the identification result is output as the identification object corresponding to the identification vector. Otherwise, if the distance between the two vectors always exceeds the threshold, it can be judged that the identification has failed.

[0055] Taking into account different scenarios, different weighted values ​​need to be set according to the specific scenarios during the construction of the fusion model. Therefore, the present application may also preferably add an interactive interface in the above-mentioned face recognition system to receive the setting of the weighted value wn of the normalized accuracy corresponding to the model combination in the nth scenario in the fourth step.

[0056] Therefore, in order to solve the scene adaptability problem of face recognition models, this application proposes a cross-scene adaptive face recognition model fusion method. This method can fuse several face recognition models into a fusion model that adapts to multiple scenes, and realize face recognition in different scenes through a unified threshold. This application can solve the problem of overfitting of a single scene of a face recognition model through model fusion technology, and can use the same threshold to be compatible with multiple usage scenarios. This method can also limit the speed of the fusion model to obtain a model with the highest accuracy that meets the speed requirements.

[0057] The above is only an implementation method of the present application, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present application. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, which all belong to the protection scope of the present application.

Claims

1. A multi-scenario adaptive model fusion method, characterized in that the steps include: The first step is to exhaustively list the combinations of face recognition models corresponding to different scenarios in the model library; The second step is to select the combinations in the first step whose operation speed meets the requirements of the target platform and record them as model combinations; The third step is to calculate the accuracy and threshold C{a1,t1}, C{a2,t2},..., C{an,tn} of each model combination obtained in the second step in each scene, and then normalize each accuracy and threshold to obtain the normalized accuracy An and normalized threshold Tn of each model combination in each scene; where n represents the scene number, an represents the accuracy of the model in the nth scene, and tn represents the threshold of the model in the nth scene; The fourth step is to calculate the weighted sum of the normalized accuracy of each model combination in each scene ACC=C{w1*A1+w2*A2+…+wn*An}, and calculate the variance of the normalized threshold of each model combination in each scene VAR=var(T1, T2, T3,… ,Tn); where wn represents the weighted value of the normalized accuracy of the model combination corresponding to the nth scene; The fifth step is to calculate the evaluation value Eval=ACC+(1-VAR) of each model combination according to the weighted sum of the normalized accuracy ACC and the variance VAR of the normalized threshold; Step 6: Select the model combination with the highest evaluation value Eval, and build a fusion model based on the model combination to combine the feature vectors extracted by each face recognition model in the model combination and perform face recognition based on the feature vector group obtained after the combination. Among them, in the third step, the accuracy an of the model combination in the nth scene is obtained by the following steps: according to the test set corresponding to the nth scene, the ROC curve of the model combination in the scene is calculated; Find the recall rate or false detection rate that meets the false detection rate requirement in the ROC curve, and calculate the accuracy an of the model combination in the nth scenario; The step of calculating the ROC curve of the model combination in the scenario according to the test set corresponding to the scenario No. n includes: Step r1, extracting the model feature vector corresponding to each face image in the test set according to each face recognition model included in the model combination; Step r2, combining the model feature vectors extracted by each face recognition model in step r1 into a multi-dimensional vector; Step r3, comparing the inter-vector distances between the multi-dimensional vector obtained by the splicing combination and the recognition vectors corresponding to each face image, and obtaining the false positive rate and the recall rate under different thresholds.

2. The multi-scenario adaptive model fusion method according to claim 1, characterized in that: In the third step, the threshold tn of the model combination in the nth scene is obtained by the following steps: according to the test set corresponding to the nth scene, the ROC curve of the model combination in the scene is calculated; Find the threshold tn that meets the false positive rate requirement in the ROC curve.

3. The multi-scenario adaptive model fusion method according to claim 2, characterized in that: The fusion model is used to perform recognition processing on the face image to be recognized according to the following steps: Step S1, extracting the model feature vector corresponding to the face image to be recognized according to each face recognition model included in the fusion model; Step S2, combining the model feature vectors extracted by each face recognition model in step S1 into a multi-dimensional vector; Step S3, comparing the inter-vector distance between the multidimensional vector obtained by the combination and the identification vector corresponding to each identification object, when the Euclidean distance between the two vectors is less than a threshold value Tn, outputting the identification result as the identification object corresponding to the identification vector.

4. The multi-scenario adaptive model fusion method according to claim 3, characterized in that: The recognition vectors corresponding to each recognition object are pre-stored in the storage unit according to the following steps: first, according to each face recognition model included in the fusion model, the model feature vector corresponding to the recognition object is extracted respectively; Then, the model feature vectors extracted by each face recognition model are combined into a one-dimensional recognition vector; The recognition vector is stored in a storage unit and the corresponding relationship between the recognition vector and the recognition object is marked.

5. A face recognition system, characterized in that: include: An image acquisition module, used for acquiring facial images to be recognized; A first storage unit, which stores a model library, wherein each face recognition model in the model library corresponds to a different scene; A second storage unit stores an executable program therein. When the executable program is executed by the processor, the processor constructs a fusion model according to the method steps described in any one of claims 1 to 4, so as to record the recognition vector corresponding to each recognition object according to the constructed fusion model, and perform recognition processing on the face image to be recognized according to the constructed fusion model.

6. The face recognition system according to claim 5, characterized in that: The specific steps of performing recognition processing on the face image to be recognized according to the obtained fusion model include: step S1, extracting the model feature vector corresponding to the face image to be recognized according to each face recognition model included in the fusion model; Step S2, combining the model feature vectors extracted by each face recognition model in step S1 into a multi-dimensional vector; Step S3, compare the inter-vector distance between the multidimensional vector obtained by the splicing combination and the identification vector corresponding to each identification object. When the Euclidean distance between the two vectors is less than the threshold Tn, the identification result is output as the identification object corresponding to the identification vector, otherwise it is judged that the identification fails.

7. The face recognition system according to claim 6, characterized in that: It also includes an interactive interface, which is used to receive the setting of the weighted value wn of the normalized accuracy corresponding to the model combination in the nth scenario in the fourth step.

8. The face recognition system according to claim 7, characterized in that: It also includes an identification object storage unit for storing the identification vector corresponding to each identification object; The identification vector is stored by the following steps: Firstly, according to each face recognition model included in the fusion model, the model feature vector corresponding to the recognition object is extracted respectively; Then, the model feature vectors extracted by each face recognition model are concatenated and combined into a multi-dimensional recognition vector; The multi-dimensional recognition vector is stored in a recognition object storage unit and the corresponding relationship between the multi-dimensional recognition vector and the recognition object is marked.

Images