Method and system for constructing face image recognition feature library

Abstract

The invention provides a face image recognition feature library construction method and system, and the method comprises the steps: obtaining the information of a plurality of monitored object persons, extracting face images, and constructing a face image library; aligning the face images to enable the faces in the face images to be in a unified angle and size; adjusting the face image by using image augmentation, and adding the face image into a face image library; extracting face features, generating feature vectors and constructing a first face feature library; for each feature vector, noise features are randomly sampled and reversed, and random noise vector features corresponding to the noise signals are generated; the random noise vector features and the feature vectors are discriminated, and according to the discrimination result, the result is used as an increment to be supplemented to a first face feature library; and using the updated face feature library to finely adjust the face recognition model and update the face recognition model. According to the invention, a better face recognition model can be obtained, the model performance is improved, and efficient and high-accuracy recognition of people in a specific range is realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security technology, in particular to identity recognition technology, and specifically to a method and system for constructing a facial image recognition feature library. Background Art

[0002] With the continuous optimization of computer image vision algorithms and machine learning algorithms, the use of facial image recognition technology for identity recognition has been widely used, from traditional access control systems, security and monitoring, financial services, smart devices (smart collection, computers, etc.), retail and customer service, traffic management, social media, medical health, education and training, tourism and hotels, smart homes, etc., covering multiple aspects from security monitoring to personalized services, facial recognition technology has exerted its diversity and application potential.

[0003] In the security, monitoring and tracking system based on facial image recognition, special identification and tracking are carried out for specific personnel, sensitive personnel, etc. On the one hand, a 1:N method is used for identification, comparison and tracking. At this time, the identity information and facial information of the monitored object have been pre-entered into the database of the security monitoring system. After the system collects a photo of "me", it finds the image that matches the current user's facial data from the massive portrait database, and matches it to find out "who I am", realizing high-precision identity recognition and security management; on the other hand, the system can also judge suspicious persons according to specific algorithms and track and trace the identified suspicious persons to realize security warnings and alarms.

[0004] Face recognition technology acquires video images and extracts facial images from them, then uses face recognition algorithms to calculate and analyze features such as the position of facial features, face shape, and angles, and then compares them with the existing feature library in its own database to determine the user's true identity. Face recognition algorithms mainly extract facial features, compare them with known faces in inventory, and complete the final classification output. The accuracy and robustness of face recognition algorithms are the prerequisites for accurate recognition, which are obtained through training based on a specific algorithm model based on a massive collection and processing of facial feature libraries.

[0005] Building a face recognition database is a crucial step. It is the core part of the security and tracking system based on face image recognition, and it directly affects the accuracy and reliability of the system. The face recognition database contains a large amount of feature information of different individuals. Improving the performance and performance of the face recognition model through training can improve the system's accurate recognition rate of the target and reduce false positives and false negatives. For identified personnel, such as key security personnel, sensitive personnel, fugitives, etc., although the recognition algorithm trained using a general face feature database and algorithm model can achieve basic identity recognition, such as reaching a commercial recognition accuracy rate of more than 90%, there is still a certain false detection rate, which poses a hidden danger to the safety of certain areas with public attributes and crowds such as public places. Summary of the invention

[0006] In view of the problems existing in the prior art, according to the first aspect of the present invention, a method for constructing a facial image recognition feature library is proposed, comprising:

[0007] Acquire multiple monitored person information, each monitored person information including personal information and personal image;

[0008] Extracting facial images from personal images in the first database using a face detection model to construct a facial image library;

[0009] Align the extracted face images so that the faces in the face images are at a uniform angle and size;

[0010] Adjust the facial image using at least one of image rotation, flipping, scaling, and brightness, and add each adjusted facial image to a facial image library;

[0011] Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library;

[0012] For the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal;

[0013] Discriminate the random noise vector feature and the feature vector, and use the result as an increment according to the discrimination result, add it to the first face feature library and update the first face feature library;

[0014] The trained face recognition model is fine-tuned using the updated first face feature library to obtain an updated face recognition model.

[0015] A method for constructing a facial image recognition feature library, characterized by comprising:

[0016] Acquire multiple monitored person information, each monitored person information including personal information and personal image;

[0017] Extracting facial images from personal images in the first database using a face detection model to construct a facial image library;

[0018] Align the extracted face images so that the faces in the face images are at a uniform angle and size;

[0019] Adjust the facial image using at least one of image rotation, flipping, scaling, and brightness, and add each adjusted facial image to a facial image library;

[0020] Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library;

[0021] For the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal;

[0022] Discriminate the random noise vector feature and the feature vector, and use the result as an increment according to the discrimination result, add it to the first face feature library and update the first face feature library;

[0023] The trained face recognition model is fine-tuned using the updated first face feature library to obtain an updated face recognition model.

[0024] As an optional embodiment, for the feature vector corresponding to each face image in the first face feature library, randomly sampling one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, reversing the noise feature, and generating a random noise vector feature corresponding to the noise signal, includes:

[0025] For the feature vector corresponding to each face image in the first face feature library, Gaussian noise, salt and pepper noise, and Poisson noise functions are constructed, and one of the three types of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled to construct a random noise vector feature. Reverse engineering is performed through the diffusion model DM to generate a random noise vector feature corresponding to the noise signal.

[0026] As an optional embodiment, the diffusion model DM is implemented using neural network training so that it can effectively reverse the noise process in the reverse process. The root mean square error RMSE and MSE objective functions are used during the training process to make the model converge, so as to minimize the difference between the original feature vector and the reverse restored data.

[0027] As an optional embodiment, the step of discriminating the random noise vector feature from the feature vector and using the discriminating result as an increment to supplement the first face feature library and update the first face feature library includes:

[0028] An adversarial domain adaptation model is used for discrimination, with random noise vector features as the target domain and feature vectors as the source domain, both of which have a unified feature dimension; the adversarial domain adaptation model includes a domain classifier and a feature generator, wherein the feature generator attempts to generate features that cannot be distinguished by the domain classifier as being from the source domain or the target domain, thereby forcing the generated features to have domain invariance; adversarial training is used to encourage the feature generator to generate features that can deceive the domain classifier, and the random noise vector features discriminated as the source domain are used as increments to supplement the aforementioned first face feature library.

[0029] According to a second aspect of the present invention, a method for constructing a facial image recognition feature library is also proposed, comprising:

[0030] Acquire multiple monitored person information, each monitored person information including personal information and personal image;

[0031] Extracting facial images from personal images in the first database using a face detection model to construct a facial image library;

[0032] Align the extracted face images so that the faces in the face images are at a uniform angle and size;

[0033] Adjust the facial image using at least one of image rotation, flipping, scaling, and brightness, and add each adjusted facial image to a facial image library;

[0034] Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library;

[0035] For the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal;

[0036] The random noise vector feature is discriminated from the feature vector, and the result is used as an increment according to the discrimination result, and the first face feature library is added and updated until the samples in the first face feature library reach the expected number;

[0037] Obtain a public face dataset, wherein the public face dataset contains diverse face images, including face images of different ages, genders, races, expressions, and lighting conditions;

[0038] Adjusting the face images in the public face dataset using at least one of image rotation, flipping, scaling, and brightness, and adding each adjusted face image to the face image library;

[0039] Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image and adding the feature vector to the first facial feature library;

[0040] The first facial feature library is used to train a deep learning-based face recognition model for subsequent face recognition task execution.

[0041] As an optional method, for the feature vector corresponding to each face image in the first face feature library, randomly sampling one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, reversing the noise feature, and generating a random noise vector feature corresponding to the noise signal, includes:

[0042] For the feature vector corresponding to each face image in the first face feature library, Gaussian noise, salt and pepper noise, and Poisson noise functions are constructed, and one of the three types of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled to construct a random noise vector feature. Reverse engineering is performed through the diffusion model DM to generate a random noise vector feature corresponding to the noise signal.

[0043] The diffusion model DM is implemented by neural network training, so that it can effectively reverse the noise process in the reverse process. The root mean square error RMSE and MSE objective functions are used in the training process to make the model converge, so as to minimize the difference between the original feature vector and the reverse restored data.

[0044] As an optional method, the step of discriminating the random noise vector feature from the feature vector and using the discriminating result as an increment to supplement the first face feature library and update the first face feature library includes:

[0045] An adversarial domain adaptation model is used for discrimination, with random noise vector features as the target domain and feature vectors as the source domain, both of which have a unified feature dimension; the adversarial domain adaptation model includes a domain classifier and a feature generator, wherein the feature generator attempts to generate features that cannot be distinguished by the domain classifier as being from the source domain or the target domain, thereby forcing the generated features to have domain invariance; adversarial training is used to encourage the feature generator to generate features that can deceive the domain classifier, and the random noise vector features discriminated as the source domain are used as increments to supplement the aforementioned first face feature library.

[0046] According to a third aspect of the present invention, a computer system is also provided, comprising:

[0047] one or more processors;

[0048] Memory, storing instructions that can be operated;

[0049] Among them, when the instruction is executed by one or more processors, the aforementioned one or more processors perform operations, and the operations include the process of executing the aforementioned method for constructing a facial image recognition feature library.

[0050] In combination with the method and system for constructing a facial image recognition feature library proposed in the above embodiments, it is possible to improve the quality and data diversity of the feature library for identified security monitoring personnel, such as key security personnel, sensitive personnel, etc., and build a facial recognition model or fine-tune an existing facial recognition model based on this. It is particularly suitable for security and monitoring systems in key industries, key areas, and public management areas, improves the performance and recognition accuracy of the system's facial recognition model, achieves accurate and efficient recognition of specific personnel, and reduces security risks and hidden dangers.

[0051] It should be understood that all combinations of the aforementioned concepts and the additional concepts described in more detail below can be considered as part of the inventive subject matter of the present disclosure as long as such concepts are not mutually inconsistent. In addition, all combinations of the claimed subject matter are considered as part of the inventive subject matter of the present disclosure.

[0052] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are not intended to be drawn to scale. In the accompanying drawings, each identical or nearly identical component shown in each figure may be represented by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings.

[0054] Figure 1 It is a schematic diagram of a method for constructing a facial image recognition feature library according to the first embodiment of the present invention.

[0055] Figure 2 It is a schematic diagram of a method for constructing a facial image recognition feature library according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0057] Various aspects of the present invention are described in this disclosure with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the present disclosure are not necessarily intended to include all aspects of the present invention. It should be understood that the various concepts and embodiments introduced above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed by the present invention are not limited to any implementation. In addition, some aspects disclosed by the present invention can be used alone or in any appropriate combination with other aspects disclosed by the present invention.

[0058] {Example 1}

[0059] Combination Figure 1 The method for constructing the face image recognition feature library of the example shown includes the following steps:

[0060] S101, obtaining information of multiple monitored persons, where each monitored person information includes personal information and personal image;

[0061] S102, extracting face images from personal images in the first database using a face detection model to construct a face image library;

[0062] S103, aligning the extracted face images so that the faces in the face images are at a uniform angle and size;

[0063] S104, adjusting the facial image using at least one of image rotation, flipping, scaling, and brightness, and adding each adjusted facial image to a facial image library;

[0064] S105, extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library;

[0065] S106, for each feature vector corresponding to the face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse processing on the noise feature, and generate a random noise vector feature corresponding to the noise signal;

[0066] S107, discriminating the random noise vector feature and the feature vector, and taking the result as an increment according to the discrimination result, adding the result to the first face feature library, and updating the first face feature library; and

[0067] S108: Use the updated first facial feature library to fine-tune the trained face recognition model to obtain an updated face recognition model.

[0068] It should be understood that in the embodiments of the present invention, the monitored personnel refer in particular to the personnel who are the focus of attention and monitoring, including but not limited to the supervised personnel clearly defined by the financial system, enterprise public management system, tax management system, judicial execution management system, credit management system, social security management system, etc., such as fugitives, major suspects, persons under execution, persons restricted from high consumption, persons who evaded payment, persons who lost their credibility, etc. The information of these personnel is collected and aggregated into a unified security key monitoring database to achieve key deployment, especially in the video monitoring systems of airports, railway stations, subways, banks, financial institutions and other places and venues involving public security and public management.

[0069] The aforementioned personal information includes personal name, and at least one of the following information: gender information, identity number information, contact address information, and contact number.

[0070] The aforementioned personal image includes the personal image on the identity document or the personal image acquired by other image acquisition devices, and the personal image includes the image of the facial area.

[0071] As an optional embodiment, in the face detection step of the aforementioned step S102, one of the Haar cascade classification detection algorithm, the LBP feature classifier detection algorithm, the FisherFace classification algorithm based on linear discriminant analysis, and the MTCNN multi-task convolutional neural network detection algorithm is used to perform face region detection to extract the face image in the image. Of course, other detection algorithms based on deep learning network models, such as the Yolo deep learning detection method, can also be used to extract the face region from the image to obtain the face image.

[0072] Furthermore, on the basis of the extracted facial image, in order to reduce the influence of posture changes and angles, especially to reduce the influence of the facial image extracted from a personal image that is not an identification document, in step S103 of an embodiment of the present invention, the extracted facial image is aligned so that the faces in the facial image are at a uniform angle and size, such as a frontal image with a size of 128*128, 64*64, etc. The present invention adopts a size of 64*64.

[0073] As an optional method, the aforementioned face images are aligned by using a radial transformation method, which includes:

[0074] The detected face area is further processed to extract the key points of the face, including the eyes, nose, and mouth corners;

[0075] An affine transformation matrix is ​​calculated according to the positions of the key points to align the face to a standard position. In the implementation of the present invention, two eyes and a nose are selected as reference points for alignment;

[0076] The face image is rotated and scaled so that the horizontal positions of the two eyes are aligned, the positions of the nose and mouth are consistent, and the posture, angle and scale of the face are standardized;

[0077] Finally, the aligned face area is cropped to a fixed size (such as 64*64 as mentioned above), and the pixel values ​​are normalized to eliminate brightness differences to ensure that the image size, angle and brightness input to the subsequent model are consistent.

[0078] As an optional embodiment, in step S104 of the aforementioned method, the facial image is adjusted using at least one of image rotation, flipping, scaling, and brightness, and each adjusted facial image is added to a facial image library. Through data augmentation processing such as rotation, flipping, scaling, and brightness adjustment, the model can meet the requirements of sample diversity and improve the robustness of subsequent training models.

[0079] As an optional embodiment, in the aforementioned step S105, facial features are extracted from each facial image in the facial image library, a feature vector of a fixed length is generated for each facial image, and a first facial feature library is constructed, including:

[0080] Pre-trained models (such as FaceNet and VGGFace) in deep learning frameworks (such as TensorFlow and PyTorch) are used to extract facial features and obtain feature vectors of fixed length, for example, 128 dimensions are selected in the embodiment of the present invention for subsequent training, matching and recognition.

[0081] The length of the fixed-length feature vector is preconfigured and is greater than or equal to 108.

[0082] It should be understood that in a specific implementation, for the constructed first facial feature library, a suitable database management system (such as MySQL, MongoDB) can be selected to store feature vectors and related information, such as name, ID and other personal information data, to constitute sample data.

[0083] Combination Figure 1 In the example shown, in the aforementioned step 106, based on the facial feature library, for the feature vector corresponding to each facial image in the first facial feature library, one of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled as a noise feature, and the noise feature is reversed to generate a random noise vector feature corresponding to the noise signal.

[0084] Specifically, for the feature vector corresponding to each face image in the first face feature library, Gaussian noise, salt and pepper noise, and Poisson noise functions are constructed, and one of the three types of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled to construct a random noise vector feature. Reverse engineering is performed through the diffusion model DM to generate a random noise vector feature corresponding to the noise signal.

[0085] In the embodiment of the present invention, the diffusion model DM includes two functions: on the one hand, in the forward stage, starting from the feature vector, noise (salt and pepper noise, Poisson noise or Gaussian noise) is continuously introduced on its basis, so that the data is transformed into pure noise through multiple time steps. This process is achieved through a series of conditional probability small steps, and each step adds more and more noise to the data until the data becomes pure noise; on the other hand, in the reverse generation stage, it is the reverse process of forward diffusion, and the purpose is to gradually reconstruct the original data from the pure noise state. In this stage, the model removes noise at each step by learning and gradually restores the original structure of the data. This process can be parameterized by neural network to remove noise.

[0086] As an optional implementation, the aforementioned diffusion model DM is implemented using neural network training so that it can effectively reverse the noise process in the reverse process. During the training process, the root mean square error RMSE and MSE objective functions are used to make the model converge, so as to minimize the difference between the original feature vector and the inverse restored data.

[0087] It should be understood that Gaussian noise, salt and pepper noise, and Poisson noise are typical noises in images. In an embodiment of the present invention, by randomly sampling Gaussian noise, salt and pepper noise, and Poisson noise, the diversity of facial features of the determined range samples (key monitored personnel) is improved, and the scale and quality of the samples are increased by the introduced noise, thereby improving the accuracy of the later model training. In particular, after simulating the addition of Poisson noise to the image, after expanding the data sample, and thus training the enhanced model, it is particularly beneficial to process image enhancement and face recognition effects in indoor environments, low light conditions, or poor lighting conditions (not in outdoor daylight natural environments), and improve the accuracy of recognition.

[0088] It should be understood that in the embodiments of the present invention, after simulating the addition of salt and pepper noise to the facial image features, the diversity of the training data is increased, and the robustness of the model in processing poorly collected image data input is improved.

[0089] As an optional implementation, in the aforementioned step S107, the random noise vector feature is discriminated with the feature vector, and the result is used as an increment according to the discrimination result to supplement the first face feature library and update the first face feature library, including:

[0090] An adversarial domain adaptation model is used for discrimination, with random noise vector features as the target domain and feature vectors as the source domain, both of which have a unified feature dimension; the adversarial domain adaptation model includes a domain classifier and a feature generator, wherein the feature generator attempts to generate features that cannot be distinguished by the domain classifier as being from the source domain or the target domain, thereby forcing the generated features to have domain invariance.

[0091] In an embodiment of the present invention, adversarial training is used to encourage the feature generator to generate features that can deceive the domain classifier, and the random noise vector features identified as the source domain are used as increments to supplement the aforementioned first face feature library.

[0092] As an example, the aforementioned adversarial domain adaptation model adopts the DANN (Domain-Adversarial Neural Network) deep adversarial neural network model.

[0093] Therefore, by randomly sampling noise features and continuously performing inverse and domain discrimination, the high-quality face feature library is expanded and supplemented, the signal-to-noise ratio of the samples is improved, and by judging whether the number of samples in the first face feature library has reached the expected number, it is decided whether to continue to randomly sample noise features. If the expected number is reached, the sampling of noise features is stopped, otherwise, the random sampling of noise signals and reverse generation of new incremental data are continued to be expanded to the first face feature library.

[0094] As an example of the present invention, the number of samples in the first facial feature library is set to be determined according to the number m of key monitored persons determined, and is configured to be 10 to 50 times the number m.

[0095] In the embodiment of the present invention, in the aforementioned step S108, the trained face recognition model is fine-tuned using the updated first face feature library to obtain an updated face recognition model.

[0096] It should be understood that the aforementioned trained face recognition model refers to a pre-trained face recognition model, especially a recognition model based on deep learning, including but not limited to a face detection model based on OpenCV, a face detection model based on Dlib, a face recognition model based on CNNs (such as a face recognition model based on VGG-16, a face recognition model based on FaceNet developed by Google, a face recognition model based on DeepFace developed by Facebook, etc.), a face recognition model based on RNNs (such as a face recognition model developed based on RNN, LSTM, GRU, etc.), a face recognition model based on integrated learning, etc.

[0097] On the one hand, these pre-trained face recognition models based on public face feature libraries are universal and robust. On the other hand, their performance and accuracy in identifying people within a specific range need to be improved. On this basis, the present invention fine-tunes (trains) the first face feature library updated in the method of the above embodiment to obtain a face recognition model with better performance, thereby achieving efficient and high-accuracy detection and identification of people within a specific range.

[0098] As an optional example, we illustrate below the method and process of randomly sampling Gaussian noise, salt and pepper noise, and Poisson noise.

[0099] Gaussian noise, also known as normal noise, is a random noise whose statistical characteristics conform to the normal distribution. Its characteristic is that the noise value added to each pixel in the image is randomly drawn from the same normal distribution. The mathematical expression of Gaussian noise is N(mu, sigma^2), where mu is the mean (usually configured as 0) and sigma is the standard deviation, which is used to control the intensity of the noise. By adding Gaussian noise to the training data, the diversity of samples is increased, making it robust to small differences and interference in subsequent training and learning, and improving model generalization.

[0100] The steps of randomly sampling Gaussian noise include:

[0101] Determine the noise parameters: select the mean and standard deviation. The mean can be set to 0 and the standard deviation can be adjusted according to the required noise intensity.

[0102] Generate a noise matrix: For each pixel of the image, extract a value from the Gaussian distribution N(0, sigma^2) to generate a noise matrix of the same size as the image;

[0103] Apply Noise to Image: Add the noise matrix to the original image features.

[0104] Salt and pepper noise, also known as impulse noise, is manifested by randomly setting the values ​​of some pixels in the image to the highest or lowest values, simulating data errors that may occur during digital image transmission. This noise usually appears as randomly distributed black and white dots, hence the name "salt and pepper".

[0105] When constructing salt and pepper noise, first determine the noise ratio and set the proportion of pixels to be modified as noise points. Usually there are two parameters: p- is used to control the proportion of pixels that become white (maximum value), and q- is used to control the proportion of pixels that become black (minimum value). Then determine the noise generation pattern, that is, randomly decide whether to add noise at each pixel position in the image, and the type of noise (black or white). Finally, modify the pixel value of the original image according to the generated noise pattern.

[0106] Poisson noise is a common type of noise in image and signal processing, especially in imaging systems under low light conditions, especially in foggy and cloudy days, where the randomness of photons reaching the image sensor causes noise errors. The characteristic of Poisson noise is that its intensity depends on the signal intensity, that is, the variance of the noise is proportional to the mean value of the signal.

[0107] When random sampling is performed, the following steps are specifically included:

[0108] Determine signal strength: The value of each pixel represents the signal strength at that location;

[0109] Generate noise: For each pixel in the image, use its pixel value as the parameter lambda of the Poisson distribution to generate noise;

[0110] Applied Noise: The generated noise directly represents the new value of that pixel, since Poisson noise reflects random fluctuations in the actual photon counts.

[0111] {Example 2}

[0112] Combination Figure 2 As shown, according to the present invention, a method for constructing a facial image recognition feature library is also proposed, comprising:

[0113] Step S201, obtaining information of multiple monitored persons, where each monitored person information includes personal information and personal image;

[0114] Step S202: extracting face images from personal images in the first database using a face detection model to construct a face image library;

[0115] Step S203, aligning the extracted facial images so that the faces in the facial images are at a uniform angle and size;

[0116] Step S204, adjusting the facial image using at least one of image rotation, flipping, scaling, and brightness, and adding each adjusted facial image to a facial image library;

[0117] Step S205, extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library;

[0118] Step S206: for the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal;

[0119] Step S207, distinguishing the random noise vector feature from the feature vector, and taking the result as an increment according to the distinguishing result, adding it to the first face feature library and updating the first face feature library, until the samples in the first face feature library reach the expected number;

[0120] Step S208, obtaining a public face dataset, wherein the public face dataset contains diverse face images, including face images of different ages, genders, races, expressions, and lighting conditions;

[0121] Step S209, adjusting the face images in the public face dataset using at least one of image rotation, flipping, scaling, and brightness, and adding each adjusted face image to the face image library;

[0122] Step S210, extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image and adding the feature vector to the first facial feature library;

[0123] Step S211: Use the first facial feature library to train a deep learning-based face recognition model for subsequent face recognition task execution.

[0124] It should be understood that the method for constructing a facial image recognition feature library proposed in Example 2 of the present invention, based on Example 1, does not adopt the method of fine-tuning training. Instead, on the basis of expanding the facial feature library of a determined personnel range, the facial feature library is further combined with a public face database to jointly construct the facial feature library. On this basis, a face recognition model based on deep learning is trained to achieve the purpose of model training.

[0125] It should be understood that the processing of the aforementioned steps S201 to S207 is the same as the corresponding processing method in the aforementioned embodiment 1.

[0126] In step S208, the aforementioned public face dataset refers to public face image datasets such as LFW, VGGFace, CASIA, OpenCV, etc., which are available for download and use.

[0127] In step S209, at least one of the image rotation, flipping, scaling, and brightness adjustment methods used can be implemented using a classical image data augmentation operation corresponding to the method in Example 1.

[0128] It should be understood that in step S210, facial features may be extracted in the same manner as step S205, and the lengths of the feature vectors obtained are the same.

[0129] As an optional embodiment, in step S211, the face recognition model based on deep learning includes but is not limited to a face detection model based on OpenCV, a face detection model based on Dlib, a face recognition model based on CNNs (such as a face recognition model based on VGG-16, a face recognition model based on FaceNet developed by Google, a face recognition model based on DeepFace developed by Facebook, etc.), a face recognition model based on RNNs (such as a face recognition model developed based on RNN, LSTM, GRU, etc.), a face recognition model based on ensemble learning, etc.

[0130] It should be understood that, whether in the aforementioned embodiment 1 or embodiment 2, the security key monitoring database that uniformly stores the key concerned and monitored persons supports periodic updating of the database content, that is, dynamically adding the information of the monitored persons.

[0131] On the basis of the dynamically updated key security monitoring database, for the added monitoring personnel information, the corresponding facial feature database can be dynamically updated according to the incremental data, and the model can be further fine-tuned based on the dynamic incremental part to achieve dynamic update.

[0132] It should be understood that for security systems that are not deployed in different ways, the face recognition model can be updated through online upgrades or offline upgrades.

[0133] {Example 3}

[0134] In combination with the method for constructing a facial image recognition feature library in the above embodiment, a computer system is also proposed according to the present invention, comprising: one or more processors; and a memory storing operable instructions.

[0135] Among them, when the aforementioned instructions are executed by one or more processors, one or more processors perform operations, and the operations include the process of the aforementioned method for constructing a facial image recognition feature library.

[0136] {Example 4}

[0137] In combination with the method for constructing a facial image recognition feature library in the above embodiments, a computer-readable medium for storing software is also proposed according to the present invention. The aforementioned software includes instructions that can be executed by one or more computers. When these instructions are executed by one or more computers, they perform the process of the method for constructing a facial image recognition feature library as in the aforementioned Embodiment 1, Embodiment 2, and embodiments thereof.

[0138] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A method for constructing a facial image recognition feature library, characterized in that: include: Acquire multiple monitored person information, each monitored person information including personal information and personal image; Extracting facial images from personal images in the first database using a face detection model to construct a facial image library; Align the extracted face images so that the faces in the face images are at a uniform angle and size; Adjust the facial image using at least one of image rotation, flipping, scaling, and brightness, and add each adjusted facial image to a facial image library; Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library; For the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal; Discriminate the random noise vector feature and the feature vector, and use the result as an increment according to the discrimination result, add it to the first face feature library and update the first face feature library; The trained face recognition model is fine-tuned using the updated first face feature library to obtain an updated face recognition model.

2. The method for constructing a facial image recognition feature library according to claim 1, characterized in that: The length of the fixed-length feature vector is preconfigured and is greater than or equal to 108.

3. The method for constructing a facial image recognition feature library according to claim 1, characterized in that: The step of randomly sampling one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature for the feature vector corresponding to each face image in the first face feature library, and reversing the noise feature to generate a random noise vector feature corresponding to the noise signal includes: For the feature vector corresponding to each face image in the first face feature library, Gaussian noise, salt and pepper noise, and Poisson noise functions are constructed, and one of the three types of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled to construct a random noise vector feature. Reverse engineering is performed through the diffusion model DM to generate a random noise vector feature corresponding to the noise signal.

4. The method for constructing a facial image recognition feature library according to claim 3, characterized in that: The diffusion model DM is implemented by neural network training so that it can effectively reverse the noise process in the reverse process. The root mean square error RMSE and MSE objective functions are used in the training process to make the model converge and minimize the difference between the original feature vector and the reverse restored data.

5. The method for constructing a facial image recognition feature library according to claim 1, characterized in that: The step of discriminating the random noise vector feature from the feature vector and using the discriminating result as an increment to supplement the first face feature library and update the first face feature library includes: An adversarial domain adaptation model is used for discrimination, with random noise vector features as the target domain and feature vectors as the source domain, both of which have a unified feature dimension; the adversarial domain adaptation model includes a domain classifier and a feature generator, wherein the feature generator attempts to generate features that cannot be distinguished by the domain classifier as being from the source domain or the target domain, thereby forcing the generated features to have domain invariance; adversarial training is used to encourage the feature generator to generate features that can deceive the domain classifier, and the random noise vector features discriminated as the source domain are used as increments to supplement the aforementioned first face feature library.

6. The method for constructing a facial image recognition feature library according to claim 5, characterized in that: The adversarial domain adaptation model adopts a DANN deep adversarial neural network model.

7. The method for constructing a facial image recognition feature library according to claim 1, characterized in that: In the method, whether to continue randomly sampling noise features is determined by judging whether the number of samples in the first facial feature library has reached the expected number. If the expected number is reached, sampling noise features is stopped; otherwise, random sampling of noise signals is continued and new incremental data is generated in reverse to expand the first facial feature library.

8. A method for constructing a facial image recognition feature library, characterized in that: include: Acquire multiple monitored person information, each monitored person information including personal information and personal image; Extracting facial images from personal images in the first database using a face detection model to construct a facial image library; Align the extracted face images so that the faces in the face images are at a uniform angle and size; Adjust the facial image using at least one of image rotation, flipping, scaling, and brightness, and add each adjusted facial image to a facial image library; Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image, and constructing a first facial feature library; For the feature vector corresponding to each face image in the first face feature library, randomly sample one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature, perform inverse engineering on the noise feature, and generate a random noise vector feature corresponding to the noise signal; The random noise vector feature is discriminated from the feature vector, and the result is used as an increment according to the discrimination result, and the first face feature library is added and updated until the samples in the first face feature library reach the expected number; Obtain a public face dataset, wherein the public face dataset contains diverse face images, including face images of different ages, genders, races, expressions, and lighting conditions; Adjusting the face images in the public face dataset using at least one of image rotation, flipping, scaling, and brightness, and adding each adjusted face image to the face image library; Extracting facial features from each facial image in the facial image library, generating a feature vector of a fixed length for each facial image and adding the feature vector to the first facial feature library; The first facial feature library is used to train a deep learning-based face recognition model for subsequent face recognition task execution.

9. The method for constructing a facial image recognition feature library according to claim 8, characterized in that: The step of randomly sampling one of Gaussian noise, salt and pepper noise, and Poisson noise as a noise feature for the feature vector corresponding to each face image in the first face feature library, and reversing the noise feature to generate a random noise vector feature corresponding to the noise signal includes: For the feature vector corresponding to each face image in the first face feature library, Gaussian noise, salt and pepper noise, and Poisson noise functions are constructed, and one of the three types of Gaussian noise, salt and pepper noise, and Poisson noise is randomly sampled to construct a random noise vector feature. Reverse engineering is performed through the diffusion model DM to generate a random noise vector feature corresponding to the noise signal.

10. The method for constructing a facial image recognition feature library according to claim 9, characterized in that: The diffusion model DM is implemented by neural network training so that it can effectively reverse the noise process in the reverse process. The root mean square error RMSE and MSE objective functions are used in the training process to make the model converge and minimize the difference between the original feature vector and the reverse restored data.

11. The method for constructing a facial image recognition feature library according to claim 8, characterized in that: The step of discriminating the random noise vector feature from the feature vector and using the discriminating result as an increment to supplement the first face feature library and update the first face feature library includes: An adversarial domain adaptation model is used for discrimination, with random noise vector features as the target domain and feature vectors as the source domain, both of which have a unified feature dimension; the adversarial domain adaptation model includes a domain classifier and a feature generator, wherein the feature generator attempts to generate features that cannot be distinguished by the domain classifier as being from the source domain or the target domain, thereby forcing the generated features to have domain invariance; adversarial training is used to encourage the feature generator to generate features that can deceive the domain classifier, and the random noise vector features discriminated as the source domain are used as increments to supplement the aforementioned first face feature library.

12. A computer system, characterized in that: include: one or more processors; Memory, storing instructions that can be operated; Wherein, when the instruction is executed by one or more processors, the aforementioned one or more processors perform an operation, and the operation includes the process of executing the method for constructing a facial image recognition feature library as described in any one of claims 1-11.

Images