A self-learning method and system for face feature aggregation analysis based on multiple algorithms

Abstract

The present invention discloses a multi-algorithm-based face feature aggregation analysis self-learning method and system, the method steps are as follows: monitor the face photos and user basic information uploaded by the front-end face recognizer; decrypt the photos, verify the legitimacy of the user's basic information; detect the key point information; double check the user identity information; save the photos and feature values ​​into the user feature tracking library; calculate the similarity of the face feature values; save them to the user feature replacement library, and send the latest face photos and feature values ​​regularly according to the service strategy; the front-end recognition device obtains the user's face base library information and replaces the local recognition library; the system includes a background computing server, a face recognizer, a management system, an original photo acquisition module, a face detection module, a face recognition module, an aggregation analysis algorithm module, and a self-learning replacement module. The present invention provides a method for realizing dynamic adjustment of face feature information as the time dimension changes, thereby improving recognition accuracy and recognition speed.

Description

Technical Field

[0001] The present invention relates to image recognition technology, and in particular to a multi-algorithm based facial feature aggregation analysis self-learning method and system. Background Art

[0002] Face recognition technology in image recognition has been applied on a large scale in many fields such as security monitoring in key areas, office attendance and access control, and high-speed rail ticket inspection and verification. The establishment and update of the face feature database in the application of face recognition technology is the basis for ensuring the accuracy of face recognition. The current common practice is for users to collect data during initial registration, and then collect additional data when recognition anomalies occur. This method is cumbersome to operate and the replacement is not timely, which affects the user experience and may even cause recognition errors. Especially in application scenarios such as attendance and access control in corporate parks, because the accuracy requirements for recognition are relatively low, there will be a situation where errors are not handled for a long time. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a self-learning method and system for facial feature aggregation analysis based on multiple algorithms, so as to realize dynamic adjustment of facial feature information as the time dimension changes, thereby improving recognition accuracy and recognition speed.

[0004] Technical solution: The multi-algorithm-based face feature aggregation analysis self-learning method of the present invention comprises the following steps:

[0005] (1) Monitor the face photos and basic user information uploaded by the front-end face recognition device; the basic user information includes name, face ID, recognition result score, timestamp, and device number.

[0006] (1.1) Establish a monitoring webservice service through the background computing server to receive the user recognition result records transmitted by the front-end face recognition device, including user basic information and face photos.

[0007] (1.2) Face photos are encoded using base64 and encrypted using RSA.

[0008] (1.3) HTTPS+Access Token+signature method is used at the interface transmission layer to ensure the security of data transmission and prevent the leakage of user privacy data.

[0009] (2) Photo decryption and verification of the legitimacy of user basic information.

[0010] (2.1) After receiving the user identification result record, the server queries the database to see whether the device that uploaded the data is a legitimate terminal.

[0011] (2.2) If it is a legitimate terminal, the user's basic information will be verified to determine whether it is a registered user and whether the recognition result score is greater than the threshold set by the system (usually not less than 0.8, the full score is 1); if the conditions are met, the decryption key is used to decrypt the face photo to obtain the complete data of the record.

[0012] (2.3) If it is not a legitimate terminal, the data is discarded.

[0013] (3) Detect key point information through face detection algorithm.

[0014] (3.1) Detecting face photos: Using face detection algorithms to perform image recognition on face photos, we can obtain five pieces of information: photo clarity, face angle, face occlusion range, light brightness, and face size. At the same time, based on the flaws of the face in the photo (moiré, imaging deformity, etc.), we can determine whether the photo is a secondary photo and filter out faces that do not meet the detection standards.

[0015] (3.2) Determine the detection results separately: Determine the face detection results according to step (3.1). Considering the complex environment and low user cooperation of the front-end face recognition terminal in actual application scenarios, in order to improve the availability of photos and improve the quality of replaced photos to be significantly better than the existing database, select face photos that meet all five conditions.

[0016] (4) Perform secondary verification of user identity information using three or more face recognition algorithms.

[0017] (4.1) Perform secondary identity verification on photos that meet the face detection conditions.

[0018] (4.2) In this process, in order to improve the accuracy of user photos, a 1:N comparison of face search is performed based on more than 3 algorithm models to ensure the correctness of user photos.

[0019] (5) Save the photo and feature value into the user feature tracking library.

[0020] (6) The facial feature values ​​are used to calculate similarity through aggregation analysis algorithm.

[0021] (7) Save to the user feature replacement library, and send the latest face photo and feature value regularly according to the service strategy.

[0022] (8) The front-end recognition device actively obtains the user's facial database information and automatically replaces the local recognition database.

[0023] A multi-algorithm based facial feature aggregation analysis self-learning system, the system includes the following modules:

[0024] Backend computing server: The computing server is connected to the front-end face recognition device via Ethernet, and detects and identifies the collected original photos to form a user feature tracking library, and performs aggregation analysis, self-learning replacement and other processing;

[0025] Face recognizer: front-end acquisition device, used to collect original recognition photos and encrypt and transmit them to the back-end computing server;

[0026] Management system: including basic information management of users, storage and management of facial feature tracking records generated by the backend computing server, storage and query of original records, and privacy protection of user photo data;

[0027] Original photo acquisition module: uses Webservice to receive original photo records transmitted by the front end through HTTP protocol, and verifies the legitimacy of the front end data source;

[0028] Face detection module: Use the face detection algorithm to detect key points of the original photo, and obtain photos that meet the requirements of the algorithm model based on the face angle, lighting, occlusion, clarity and other conditions in the detection results;

[0029] Face recognition module: The original photo is compared with the user's current face database through a face recognition algorithm, and more than three face algorithms are used for mutual verification to ensure that the collected photo is the user's own;

[0030] Aggregation analysis algorithm module: uses the cosine distance method to judge the similarity based on the facial feature values ​​within a certain period of time, and calculates the optimal value through the clustering algorithm;

[0031] Self-learning replacement module: Through the update strategy, the latest user feature values ​​are selected from the user feature replacement library and sent to the front-end device at regular intervals, thereby completing the real-time replacement of the user face base library.

[0032] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned multi-algorithm-based facial feature aggregation analysis self-learning method.

[0033] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned multi-algorithm-based facial feature aggregation analysis self-learning method is implemented.

[0034] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0035] 1. Use face detection algorithms and multi-algorithm recognition technology to conduct preliminary screening and comparison of user facial features within a period of time, filter the facial data uploaded by the front end, and form a user feature tracking library as the basic data for analysis and calculation. Use a self-developed aggregation analysis algorithm to perform a 1:1 comparison of user facial feature values ​​within a period of time to form a comparison result. Based on the result, select face images and feature values ​​that meet the threshold setting as the user feature replacement library to solve the problem of reduced recognition accuracy and speed caused by the long-term lack of update of the user's face base library;

[0036] 2. Use face detection algorithms to detect the angle, illumination, and occlusion of the face. At the same time, multiple face recognition algorithms are used for mutual verification to ensure that the person entering the user feature tracking library is the user himself and the availability of the face photo, so as to avoid the problem of user face base library replacement errors due to misidentification or poor photo quality, which may lead to the problem of user unrecognition;

[0037] 3. Through the background self-learning timed replacement mechanism, the user's facial feature values ​​are updated regularly, reducing manual operations and effectively reducing the problem of reduced recognition speed and accuracy caused by the time dimension;

[0038] 4. In order to improve the high availability and reusability of the system of this method, a background service mode is adopted to perform analysis and calculation when the server is idle (generally between 1 am and 5 am), reducing the pressure of normal computing during the day and preventing the impact on normal business;

[0039] 5. Through the interaction of face recognition data between the front-end acquisition equipment and the management platform, real-time information such as personnel information and face recognition records are transmitted back. Combined with big data aggregation analysis, a user face feature tracking and replacement mechanism is established to better provide customers with high-quality, continuously self-optimizing face recognition services. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a step flow chart;

[0041] Figure 2 This is the system principle block diagram;

[0042] Figure 3 This is the flow chart of the secondary face comparison and verification. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0044] like Figure 1 As shown, a multi-algorithm based face feature aggregation analysis self-learning method includes the following steps:

[0045] (1) Monitor the face photos and basic user information uploaded by the front-end face recognition device; the basic user information includes name, face ID, recognition result score, timestamp, and device number.

[0046] (1.1) Establish a monitoring webservice service through the background computing server to receive the user recognition result records transmitted by the front-end face recognition device, including user basic information and face photos.

[0047] (1.2) Face photos are encoded using base64 and encrypted using RSA.

[0048] (1.3) HTTPS+Access Token+signature method is used at the interface transmission layer to ensure the security of data transmission and prevent the leakage of user privacy data.

[0049] (2) Photo decryption and verification of the legitimacy of user basic information.

[0050] (2.1) After receiving the user identification result record, the server queries the database to see whether the device that uploaded the data is a legitimate terminal.

[0051] (2.2) If it is a legitimate terminal, the user's basic information will be verified to determine whether it is a registered user and whether the recognition result score is greater than the threshold set by the system (usually not less than 0.8, the full score is 1); if the conditions are met, the decryption key is used to decrypt the face photo to obtain the complete data of the record.

[0052] (2.3) If it is not a legitimate terminal, the data is discarded.

[0053] (3) Detect key point information through face detection algorithm.

[0054] (3.1) Detecting face photos: Performing image recognition on face photos through face detection algorithm to obtain five pieces of information: photo clarity, face angle, face occlusion range, light brightness, and face size. At the same time, based on the flaws of the face in the photo (moiré, imaging deformity, etc.), determine whether the picture is a secondary photo and filter out faces that do not meet the detection standards; the judgment conditions are shown in Table 1.

[0055] Table 1 Face detection judgment table

[0056]

[0057] (3.2) Determine the detection results separately: Determine the face detection results according to step (3.1). Considering the complex environment and low user cooperation of the front-end face recognition terminal in actual application scenarios, in order to improve the availability of photos and improve the quality of replaced photos to be significantly better than the existing database, select face photos that meet all five conditions.

[0058] (4) Perform secondary verification of user identity information using three or more face recognition algorithms.

[0059] (4.1) Perform secondary identity verification on photos that meet the face detection conditions.

[0060] (4.2) In this process, in order to improve the accuracy of user photos, face search 1:N comparison is performed based on more than 3 algorithm models to ensure the correctness of user photos. The specific judgment process is as follows: Figure 3 As shown, firstly, face recognition comparison is performed according to different algorithm models to obtain the user's matching score. The recommended threshold is generally not less than 80 points. If the scores of the three comparison results are all greater than the threshold, the secondary verification is passed, and the facial feature values ​​in the photo are extracted and returned. If any of the returned results are less than the threshold, the secondary verification is judged to have failed, and this user record is not saved.

[0061] (5) Save the photo and feature value into the user feature tracking library.

[0062] At the same time, the user's photo and the corresponding feature value are saved in the user feature tracking library, and each user saves no less than 10 feature tracking records.

[0063] (6) The facial feature values ​​are used to calculate similarity through aggregation analysis algorithm.

[0064] The similarity of facial feature values ​​within a certain time range is calculated through the aggregation analysis algorithm, and the cosine distance is used to measure the similarity of facial features to determine the optimal feature value. The specific method is as follows:

[0065] Define the eigenvalues ​​of image 1 and image 2: vector1 and vector2, use the cosine distance, and use the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between the two individuals. Calculate cosθ, the formula is as follows:

[0066] Since the threshold of cosθ is -1 to 1, when the angle θ between two vectors approaches 0, the closer the two vectors are, the smaller the difference is. So the cosine value of the two eigenvalues ​​is obtained by multiplying them, V i =|V1|×|V2|×cosθ where V iThat is cosθ. At this time, cosθ=1, that is, the closer it is to 1, the more similar the faces are.

[0067] The sample implementation code is as follows:

[0068] import numpy as np

[0069] dot=np.sum(np.multiply(vector1,vector2),axis=1)

[0070] norm=np.linalg.norm(vector1,axis=1)*np.linalg.norm(vector2,axis=1)

[0071] dis=dot / norm

[0072] Thus, the comparison result score of eigenvalue 1 and eigenvalue 2 is obtained. Similarly, the approximate value matrix is ​​obtained by calculating the nearly 10 eigenvalues ​​of the user, as shown in Table 2. The calculation results of each eigenvalue are accumulated and the average is calculated to obtain the final calculation result. The eigenvalue closest to 1 is selected as the optimal value, and the best quality eigenvalue record information is returned.

[0073] Table 2 Approximation matrix

[0074]

[0075] (7) Save to the user feature replacement library, and send the latest face photo and feature value regularly according to the service strategy.

[0076] The user feature record returned by step (6) is saved in the user feature replacement library, and the display information of the background management system and the basic personnel information library are updated synchronously. At the same time, according to the server setting strategy, the latest face photo and feature value corresponding to the front-end face recognition device are updated regularly.

[0077] (8) The front-end recognition device actively obtains the user's facial database information and automatically replaces the local recognition database.

[0078] The front-end recognition device obtains information based on the heartbeat packet to determine whether the background service user face database has been updated. When it finds that the personnel database needs to be updated, it starts the automatic update mechanism to download the user's latest feature value and replace the local user face recognition database.

[0079] like Figure 2 As shown, a multi-algorithm based face feature aggregation analysis self-learning system includes the following modules:

[0080] Backend computing server: The computing server is connected to the front-end face recognition device via Ethernet, and detects and identifies the collected original photos to form a user feature tracking library, and performs aggregation analysis, self-learning replacement and other processing;

[0081] Face recognizer: front-end acquisition device, used to collect original recognition photos and encrypt and transmit them to the back-end computing server;

[0082] Management system: including basic information management of users, storage and management of facial feature tracking records generated by the backend computing server, storage and query of original records, and privacy protection of user photo data;

[0083] Original photo acquisition module: uses Webservice to receive original photo records transmitted by the front end through HTTP protocol, and verifies the legitimacy of the front end data source;

[0084] Face detection module: Use the face detection algorithm to detect key points of the original photo, and obtain photos that meet the requirements of the algorithm model based on the face angle, lighting, occlusion, clarity and other conditions in the detection results;

[0085] Face recognition module: The original photo is compared with the user's current face database through a face recognition algorithm, and more than three face algorithms are used for mutual verification to ensure that the collected photo is the user's own;

[0086] Aggregation analysis algorithm module: uses the cosine distance method to judge the similarity based on the facial feature values ​​within a certain period of time, and calculates the optimal value through the clustering algorithm;

[0087] Self-learning replacement module: Through the update strategy, the latest user feature values ​​are selected from the user feature replacement library and sent to the front-end device at regular intervals, thereby completing the real-time replacement of the user face base library.

[0088] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned multi-algorithm-based facial feature aggregation analysis self-learning method.

[0089] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned multi-algorithm-based facial feature aggregation analysis self-learning method is implemented.

Claims

1. A multi-algorithm based face feature aggregation analysis self-learning method, characterized in that: The following steps are involved: (1) Monitor the facial photos and basic user information uploaded by the front-end face recognition device; (2) Photo decryption and verification of the legitimacy of user basic information; (3) Detect key point information using a face detection algorithm; the key point information includes five pieces of information: photo clarity, face angle, facial occlusion range, light brightness, and face size; (4) Perform secondary verification of user identity information through three or more face recognition algorithms; (5) Save the face photo and face feature value into the user feature tracking library; (6) Calculate the similarity of the facial feature values ​​in the user feature tracking library through the aggregation analysis algorithm; (7) The calculation result of step (6) is saved in the user feature replacement library, and the latest face photo and face feature value are sent regularly according to the service strategy; (8) The front-end recognition device actively obtains the user's facial database information and automatically replaces the local recognition database; Face detection algorithms and multi-algorithm recognition technology are used to conduct preliminary screening and comparison of user facial features over a period of time, filter the facial data uploaded by the front end, and form a user feature tracking library.

2. According to the multi-algorithm based face feature aggregation analysis self-learning method of claim 1, it is characterized in that: The basic user information in step (1) includes name, face ID, recognition result score, timestamp, and device number.

3. The multi-algorithm based face feature aggregation analysis self-learning method according to claim 2, characterized in that: The step (1) is specifically as follows: (1.1) Establish a monitoring webservice through the backend computing server to receive the user recognition result records transmitted by the front-end face recognition device, including user basic information and face photos; (1.2) Face photos are transcoded using base64 and encrypted using RSA; (1.3) HTTPS + Access Token + sign signature method is used at the interface transmission layer to ensure the security of data transmission.

4. According to claim 3, a multi-algorithm based face feature aggregation analysis self-learning method is characterized in that: The step (2) is specifically as follows: (2.1) After receiving the user identification result record, the server queries the database to see whether the device that uploaded the data is a legitimate terminal; (2.2) If it is a legitimate terminal, the user's basic information will be verified to determine whether it is a registered user and whether the recognition result score is greater than the threshold set by the system; if the conditions are met, the decryption key will be used to decrypt the face photo to obtain the complete data of the record; (2.3) If it is not a legitimate terminal, the data will be discarded.

5. The multi-algorithm based face feature aggregation analysis self-learning method according to claim 1, characterized in that: The step (3) is specifically as follows: (3.1) Detecting face photos: Performing image recognition on face photos through face detection algorithms to obtain five pieces of information: photo clarity, face angle, face occlusion range, light brightness, and face size. At the same time, based on the flaws of the face in the photo, determine whether the photo is a secondary photo and filter out faces that do not meet the detection standards; the flaws of the face include moiré and imaging deformity; (3.2) Determine the detection results separately: Determine the face detection results according to step (3.1) and select face photos that meet all five conditions.

6. The multi-algorithm based face feature aggregation analysis self-learning method according to claim 5, characterized in that: The step (4) is specifically as follows: (4.1) Perform secondary identity verification on photos that meet the face detection conditions; (4.2) Perform 1:N face search comparison based on more than 3 algorithm models to ensure the accuracy of user photos.

7. A multi-algorithm based face feature aggregation analysis self-learning system, characterized in that: The system includes the following modules: Backend computing server: The computing server is connected to the front-end face recognition device via Ethernet, and detects and identifies the collected original photos to form a user feature tracking library, and performs aggregation analysis and self-learning replacement processing; Face recognizer: front-end acquisition device, used to collect original recognition photos and encrypt and transmit them to the back-end computing server; Management system: including basic information management of users, storage and management of facial feature tracking records generated by the backend computing server, storage and query of original records, and privacy protection of user photo data; Original photo acquisition module: uses Webservice to receive original photo records transmitted by the front end through HTTP protocol, and verifies the legitimacy of the front end data source; Face detection module: Use the face detection algorithm to detect key points of the original photo, and obtain photos that meet the requirements of the algorithm model based on the face angle, light brightness, facial occlusion range, photo clarity, and face size conditions in the detection results; Face recognition module: The original photo is compared with the user's current face database through a face recognition algorithm, and more than three face algorithms are used for mutual verification to ensure that the collected photo is the user's own; Aggregation analysis algorithm module: uses the cosine distance method to judge the similarity based on the facial feature values ​​within a certain period of time, and calculates the optimal value through the clustering algorithm; Self-learning replacement module: Through the update strategy, the latest user feature values ​​are selected from the user feature replacement library and sent to the front-end device at regular intervals, thereby completing the real-time replacement of the user face base library.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements a multi-algorithm based facial feature aggregation analysis self-learning method as described in any one of claims 1 to 6.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a multi-algorithm based facial feature aggregation analysis self-learning method as described in any one of claims 1-6.

Images