A face recognition method and system for a dense crowd

Abstract

The present invention discloses a face recognition method and system for a dense crowd. The present invention relates to the technical field of biometric recognition, and includes the following steps: establishing a true-color face data template library; collecting thermal infrared images in a densely populated area, preprocessing the collected thermal infrared images, extracting face targets from the preprocessed thermal infrared images, cropping the thermal infrared images after labeling the face targets to obtain thermal infrared images of face targets to be recognized, performing colorized face conversion to obtain true-color face images to be recognized; calculating the similarity between the true-color face images to be recognized and each user in the true-color face data template library; calculating the average similarity and the highest similarity between m true-color face feature templates of each user and the true-color face images to be recognized, and generating a decision similarity by combining the highest similarity with the average similarity; comparing with a preset decision similarity threshold, and judging the user information to be recognized according to different comparison results.

Description

Technical Field

[0001] The present invention relates to the field of biometric technologies, and particularly to a face recognition method and system for a dense crowd. Background Art

[0002] In the past few decades, with the booming development of the mobile Internet, biometric recognition technologies have also received increasing attention, and the accuracy requirements for authentication and recognition systems are getting higher and higher. The inherent differences in human biometric characteristics make various methods based on biometric recognition robust and repeatable, so they have become important means for authentication and recognition. Biometric recognition based on biometrics includes face recognition. Face recognition has characteristics such as non-contact and scalability, and is easy for users to accept without psychological barriers. Face recognition technology has wide application value in various environments and fields, especially in high-security protection, access control, and computer security fields. Face recognition is a technology that can verify identity by using a computer to analyze facial images and extract effective feature information from them.

[0003] Whether the recognition system can adapt to the constantly changing application environments in the real world is a key issue in face recognition. Researchers have developed many complex algorithms in the field of visible light face recognition to handle these changing situations, but it is still not enough. For example, in constantly changing lighting conditions, different facial expressions, and the case of wearing makeup, these all make visible light face recognition have the defect of being unable to recognize.

[0004] In the prior art, the publication number CN105868695B discloses a face recognition method and system, which includes: constructing a face feature template library, where the face feature template library stores face feature templates and user basic information of N users, and each user corresponds to at least one face feature template; extracting face features from the face image of the user to be recognized to obtain the face feature template to be recognized; calculating the similarity between the face feature template to be recognized and each user's face feature template in the face feature template library, first taking the maximum value among the maximum similarities between the face feature template to be recognized and each user, and determining whether the user is the user corresponding to this maximum value. If not, then taking the maximum value among the average similarities between the face feature template to be recognized and each user, and determining whether the user is the user corresponding to this maximum value; this method can effectively improve the accuracy of face recognition, but the premise is that the face feature template to be recognized has high accuracy. When the face feature template to be recognized is in a crowded area or under constantly changing lighting conditions, resulting in reduced accuracy, there may be a defect of being unable to recognize.

[0005] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure, so it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The object of the present invention is to provide a face recognition method and system for a dense crowd to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A face recognition method for a dense crowd, the specific steps including:

[0009] Establish a true-color face data template library, where the true-color face data template library stores true-color face feature templates and user name information of n users, and each user corresponds to m true-color face feature templates, where m is a positive integer and m≥3;

[0010] Collect thermal infrared images in the dense area of the crowd to be recognized; preprocess the collected thermal infrared images, and the preprocessing includes thermal infrared image denoising and enhancement processing to obtain preprocessed thermal infrared images;

[0011] Extract face targets from the preprocessed thermal infrared images, and label all face targets in the images after face target extraction;

[0012] Crop the thermal infrared images with labeled face targets to obtain thermal infrared images of face targets to be recognized, and perform color face conversion on the thermal infrared images of face targets to be recognized to obtain true-color face images to be recognized;

[0013] Perform user recognition by comparing the obtained true-color face images to be recognized with the true-color face data template library, and calculate the similarity between the true-color face images to be recognized and each face feature template of each user in the true-color face data template library;

[0014] Compare the similarity between the obtained true-color face images to be recognized and the face feature templates of each user in the true-color face data template library with a similarity threshold, and extract and record the user information in the true-color face data template library corresponding to the similarity requirements to generate a first user information set;

[0015] Calculate the average similarity and the highest similarity between the m true-color face feature templates of each user in the first user information set and the true-color face images to be recognized, and generate a decision similarity by combining the highest similarity with the average similarity; compare the generated decision similarity with a preset decision similarity threshold, and judge the user information to be recognized according to different comparison results.

[0016] Further, a true-color face data template library is established. The true-color face data template library stores the true-color face feature data and user name information of n users. In the established true-color face data template library, each user corresponds to m true-color face feature templates, where m is a positive integer and m≥3.

[0017] Further, thermal infrared images in the area where the crowd to be identified is dense are collected. The method for collecting thermal infrared images in the dense crowd area is as follows: Use a thermal infrared camera to conduct video monitoring on the crowd area to be collected, and extract frame images from the collected video as thermal infrared images.

[0018] Further, the collected thermal infrared images are preprocessed. The preprocessing includes denoising and enhancement processing of the thermal infrared images;

[0019] Among them, the denoising method of wavelet transform is used to denoise the thermal infrared images. The specific steps of the wavelet transform denoising method include: decomposing the thermal infrared images through wavelet transform to obtain wavelet coefficients of the images at different scales and directions; performing threshold processing on the wavelet coefficients, setting the wavelet coefficients with low amplitudes to zero, and retaining the wavelet coefficients with high amplitudes; performing inverse transform on the wavelet coefficients after threshold processing, and reconstructing the processed coefficients into images to achieve the denoising processing of the thermal infrared images:

[0020] The logic for the enhancement processing of thermal infrared images is as follows: Use the method of linear stretching to expand the gray-level dynamic range of the thermal infrared images to a specified range according to the linear mapping relationship formula. The formula for linear stretching is:

[0021]

[0022] In the formula, f LMj (V) represents the new gray value obtained after the gray value of the j-th pixel point in the thermal infrared image is processed by linear stretching. (V) represents the gray value as the independent variable of the function, and V j is the gray value of the j-th pixel point in the original image, V min represents the minimum gray value in the original image, V max represents the maximum gray value in the original image, D represents the dynamic range after the image gray level is compressed, and D1 represents the minimum gray value of the compressed image.

[0023] Further, face targets are extracted from the preprocessed images. The logic for labeling all face targets in the images after face target extraction is as follows: Use the YOLOv5s network to perform thermal infrared face detection on the preprocessed images, identify the face information in the images, and after organizing the data containing thermal infrared faces, use the labeling tool LabelImg to label the thermal infrared face data set.

[0024] Furthermore, the thermal infrared image with the face target marked is cropped to obtain a thermal infrared face target image, and the thermal infrared face target image is subjected to colorized face conversion to obtain a true color face image to be recognized. Among them, the CycleGAN model is used to convert the thermal infrared face target image into a true color face. By training a generator G, the function of the generator G is to input a thermal infrared face image and output a true color face image. Through the iterative learning process, a better quality colorized face image is generated.

[0025] Furthermore, the similarity between the obtained true color face image to be recognized and each user's face feature template in the true color face data template library is compared with the similarity threshold, and the user information in the true color face data template library corresponding to the similarity requirement is extracted and recorded to generate a first user information set;

[0026] Among them, the logic for determining the data in the first user information set is:

[0027] The similarity threshold is calibrated as yz, and the similarity between the true color face image to be recognized and each user's face feature template is S i , where i = 1, 2,..., n*m;

[0028] When 0 ≤ S i < yz, it is determined that the similarity between the user's face feature template and the image to be recognized does not meet the requirements, and the user information is not saved to the first user information set;

[0029] When S i ≥ yz, it is determined that the similarity between the user's face feature template and the image to be recognized meets the requirements, and the corresponding user information is extracted into the first user information set;

[0030] Among them, as long as there is one face feature template image of each user whose similarity with the image to be recognized meets the requirements, the user information is extracted into the first user information set, and at the same time, each user's information is only extracted once.

[0031] Furthermore, calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be recognized, and generate a decision similarity by combining the highest similarity and the average similarity;

[0032] Among them, the formula for calculating the average similarity between the true color face image to be recognized and each user is:

[0033]

[0034] In the formula, S avgi is the average similarity of the i-th user in the first user information set, j = 1, 2,..., m, S jIndicates the similarity between the j-th true-color face feature template of the user and the true-color face image to be recognized;

[0035] The formula based on the highest similarity is:

[0036] S maxi = max(S1, S2,..., S j )

[0037] In the formula, S maxi Indicates the highest similarity between the m true-color face feature templates of the i-th user in the first user information set and the true-color face image to be recognized;

[0038] The formula for generating the decision similarity by combining the highest similarity and the average similarity is:

[0039] S zdi = a * S avgi + S maxi * b

[0040] In the formula, S zdi Is the decision similarity of the i-th user in the first user information set, and a and b respectively represent the preset weight coefficients of the average similarity S avgi And the highest similarity S maxi , where a and b are greater than 0 and a < b.

[0041] Furthermore, comparing the generated decision similarity with a preset decision similarity threshold, according to different comparison results, the logic for judging the user information to be recognized is:

[0042] Calibrate the preset decision similarity threshold as yz jd ;

[0043] When S zdi < yz jd , it is judged that the user to be recognized is not this user;

[0044] When S zdi ≥ yz jd , it is judged that the user to be recognized is the current user.

[0045] The present invention also provides a face recognition system for a dense crowd. The face recognition system for a dense crowd is used to execute the above-mentioned face recognition method for a dense crowd, including:

[0046] A data template library establishment module, which is used to establish a true-color face data template library. The true-color face data template library stores the true-color face feature templates and user name information of n users. Each user corresponds to m true-color face feature templates, where m is a positive integer and m ≥ 3;

[0047] A thermal infrared image acquisition module, which is used to acquire thermal infrared images within the densely populated area to be recognized; preprocess the acquired thermal infrared images, and the preprocessing includes denoising and enhancement processing of the thermal infrared images to obtain preprocessed thermal infrared images;

[0048] A target extraction module, which is used to extract face targets from the preprocessed thermal infrared images and label all face targets in the images after face target extraction;

[0049] An image colorization module, which crops the thermal infrared image with labeled face targets to obtain a thermal infrared image of the face target to be recognized, and performs color face conversion on the thermal infrared image of the face target to be recognized to obtain a true color face image to be recognized;

[0050] A similarity calculation module, which is used to perform user recognition by comparing the obtained true color face image to be recognized with the true color face data template library, and calculate the similarity between each face feature template of each user in the true color face data template library and the true color face image to be recognized;

[0051] A user information extraction module, which is used to compare the similarity between each user's face feature template in the obtained true color face data template library and the true color face image to be recognized with a similarity threshold, extract and record the user information in the true color face data template library that meets the similarity requirements, and generate a first user information set;

[0052] A face recognition comparison module, which is used to calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be recognized, and generate a decision similarity by combining the highest similarity and the average similarity; compare the generated decision similarity with a preset decision similarity threshold, and judge the user information to be recognized according to different comparison results.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows: A true-color face data template library is established, which stores true-color face feature templates and user basic information of n users, and each user corresponds to at least 3 true-color face feature templates; Thermal infrared images in crowded areas are collected, preprocessed, and then face targets are extracted from the preprocessed thermal infrared images. All face targets in the images after face target extraction are labeled; The thermal infrared images with labeled face targets are cropped to obtain thermal infrared images of face targets to be recognized, and the thermal infrared images of face targets to be recognized are subjected to colorized face conversion to obtain true-color face images to be recognized; User recognition is performed by comparing the obtained true-color face images to be recognized with the true-color face data template library, and the similarity between the true-color face images to be recognized and each face feature template of each user in the true-color face data template library is calculated; The similarity between the obtained true-color face images to be recognized and each user in the true-color face data template library is compared with the similarity threshold, and the user information in the true-color face data template library corresponding to the similarity meeting the requirements is extracted and recorded to generate a first user information set; The average similarity and the highest similarity between the m true-color face feature templates of each user in the first user information set and the true-color face images to be recognized are calculated, and the highest similarity is combined with the average similarity to generate a decision similarity; The generated decision similarity is compared with a preset decision similarity threshold, and according to different comparison results, the user information to be recognized is judged; The thermal infrared face images are basically not affected by environmental illumination, have strong anti-counterfeiting performance, good concealment, and do not require compensation. They have better effects in solving face recognition problems and facial disguise problems under changing lighting conditions, etc., and have good applications in crowded areas. At the same time, by calculating the highest similarity and the average similarity of user images in the template library to judge the image to be recognized, the accuracy of face recognition is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0055] Figure 2 It is a schematic diagram of the overall system flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to specific embodiments.

[0057] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. The terms such as "comprising" or "including" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0058] Embodiment:

[0059] Please refer to Figure 1 , the present invention provides a technical solution:

[0060] A face recognition method for a dense crowd, the specific steps include:

[0061] Step 1: Establish a true-color face data template library, which stores true-color face feature templates and user name information of n users. Each user corresponds to m true-color face feature templates, where m is a positive integer and m≥3;

[0062] Among them, in the established true-color face data template library, each user corresponds to m true-color face feature templates, where m is a positive integer and m≥3; the number of face feature templates of different users may be different.

[0063] Step 2: Collect thermal infrared images in the area where the crowd to be recognized is dense; preprocess the collected thermal infrared images, and the preprocessing includes thermal infrared image denoising and enhancement processing to obtain the preprocessed thermal infrared images;

[0064] Collect thermal infrared images in the area where the crowd is dense. The area where the crowd is dense includes stations, scenic spots and streets. The method for collecting thermal infrared images in the area where the crowd is dense is: use a thermal infrared camera to perform video monitoring on the area where the crowd to be collected is located, and extract frame images from the collected video as thermal infrared images;

[0065] The thermal infrared camera is composed of an infrared detector, an optical lens and a circuit board. The photosensitive element on the infrared detector receives the energy distribution radiated by the target object, and the optomechanical scanning is placed between the infrared detector and the optical lens for infrared thermal imaging of the object to be detected;

[0066] The advantages of infrared thermal imaging technology are mainly reflected in the following aspects: First, it has a high safety factor and high efficiency. Moreover, the operation is simple and it can effectively observe the temperature distribution. Second, it has strong target tracking ability and strong anti-interference ability. When tracking a target, it is not affected by the electromagnetic materials around the target, and at the same time, it also has a good detection effect on distant targets. Then, it can monitor targets all-weather and can play a good detection effect in complex and harsh weather or environments. And, it has high detection accuracy and long detection distance, and has a good use effect in complex environments with dense crowds.

[0067] Preprocess the collected thermal infrared images, and the preprocessing includes denoising and enhancement processing of the thermal infrared images.

[0068] Among them, the denoising method using wavelet transform is used to denoise the thermal infrared images, and wavelet transform is often used when processing thermal infrared images. The specific steps of the wavelet transform denoising method include: reading the thermal infrared image and converting it into a grayscale image, normalizing the image so that the pixel values are between 0 and 1; selecting a suitable wavelet basis function and determining the wavelet function (such as the Daubechies wavelet db4); selecting the number of layers of wavelet decomposition, generally choosing 3 to 5 layers; performing threshold processing on the wavelet coefficients, selecting the threshold method, and the threshold methods include hard threshold or soft threshold, and calculating the threshold; performing inverse wavelet transform on the wavelet coefficients after threshold processing to reconstruct the denoised image; among them, the VisuShrink method is used to calculate the threshold, and the specific calculation formula is:

[0069]

[0070] In the formula, lam is the threshold, sig is the standard deviation of the noise, and p is the total number of pixels in the image.

[0071] The logic for the enhancement processing of thermal infrared images is as follows: Use the method of linear stretching. Linear stretching (LinearMap) is a simple and commonly used image enhancement technique, mainly used to adjust the contrast of the image, map the dynamic range of the gray level of the original image (that is, the range of pixel values) to a specified range, and expand the dynamic range of the gray level of the thermal infrared image to the specified range according to the linear mapping relationship. The infrared camera converts the captured analog radiation signal into a digital signal through an analog-to-digital converter (ADC). The converted digital signal represents the gray value of each pixel point. The 14-bit A / D original image data is processed and converted into 8-bit image data through the method of linear stretching.

[0072] The formula on which its linear stretching is based is:

[0073]

[0074] In the formula, f LMj(V) represents the new gray value obtained after the gray value of the j-th pixel in the thermal infrared image is linearly stretched. (V) represents the gray value as the independent variable of the function, and V j is the gray value of the j-th pixel in the original image. The first pixel in the upper left corner of the image to be preprocessed is the first pixel, and the labels are arranged in row-major order in sequence, V min represents the minimum gray value in the original image, V max represents the maximum gray value in the original image. D represents the dynamic range after the image gray level is compressed, and D1 represents the minimum gray value of the compressed image.

[0075] Step 3: Extract the face targets from the preprocessed thermal infrared image, and label all the face targets in the image after the face targets are extracted;

[0076] The logic of extracting the face targets from the preprocessed image and labeling all the face targets in the image is as follows: Use the YOLOv5s network to perform thermal infrared face detection on the preprocessed image, identify the face information in the image, and after organizing the data containing the thermal infrared faces, use the labeling tool LabelImg to label the thermal infrared face dataset;

[0077] Among them, the main function of the YOLOv5s network is to perform real-time object detection. Object detection is an important task in the field of computer vision, and its goal is to detect various different types of objects from images or videos and accurately mark their positions. YOLOv5s is a model in the YOLO (You Only Look Once) series, which achieves a balance between real-time performance and accuracy and is suitable for scenarios with high requirements for speed and lightweight models;

[0078] Use the thermal infrared image with labeled face information as the input to train the YOLOv5s network. Use the bounding box corresponding to each face image as the label to train the YOLOv5s network to obtain the YOLOv5s model for thermal infrared face recognition; Input the preprocessed thermal infrared image into the trained YOLOv5s model to extract the face targets from the thermal infrared image;

[0079] Label all the face targets in the image after the face targets are extracted. Use the LabelImg tool to make a VOC format dataset, and then convert the VOC format dataset into a yolo format dataset suitable for the YOLOv5s network. The category is face, labeled as face.

[0080] Step 4: Crop the thermal infrared image after labeling the face targets to obtain the thermal infrared image of the face target to be recognized, and perform colorized face conversion on the thermal infrared image of the face target to be recognized to obtain the true color face image to be recognized;

[0081] The method for cropping the thermal infrared image after annotating the face target is as follows: cropping based on the minimum bounding rectangle, and the specific steps include: using the face detection algorithm YOLOv5s network and the LabelImg tool to annotate the bounding box of the face; obtaining the coordinates of the minimum bounding rectangle of the annotated face, including the coordinates of the upper left corner boundary point and the lower right corner boundary point of the minimum bounding rectangle; cropping the thermal infrared image of the face target to be recognized containing the complete face image according to the coordinates of the minimum bounding rectangle of the annotated face;

[0082] Among them, using the CycleGAN model to convert the thermal infrared image of the face target into a true color face, the main idea of the CycleGAN model is to learn the mapping relationship between two domains through adversarial training. It consists of two generator networks and two discriminator networks. The specific steps for using the CycleGAN model to convert the thermal infrared image of the face target into a true color face are as follows:

[0083] Train two generators, prepare two image domains, one image domain stores thermal infrared face images, and the other image domain stores true color face images. One generator maps the thermal infrared face images in the infrared face image domain to the other true color face image domain, and the other generator maps the images of the latter back to the former. The two discriminator networks are respectively used to identify whether the generated images are real and provide feedback to the generator network to improve the quality of the generated images. Through the iterative learning process, higher quality colorized face images are generated.

[0084] Finally, use the trained generator G, which inputs the thermal infrared image of the face target and outputs the true color face image to be recognized.

[0085] Step 5: Perform user recognition by comparing the obtained true color face image to be recognized with the true color face data template library, and calculate the similarity of each face feature template of each user in the true color face data template library and the true color face image to be recognized:

[0086] Among them, the calculation method of similarity is as follows: use a deep learning model (such as a face recognition model based on a convolutional neural network) to extract features and calculate similarity for face images. By comparing the feature representations of two face images, the similarity score between them can be calculated. The Siamese network is a neural network structure used to measure the similarity between two inputs.

[0087] Use several face images as training data for the Siamese network; select the Contrastive Loss function as the loss function of the model, and use the prepared several face images and the defined loss function to train the Siamese network. During the training process, the network learns how to extract face features and compares the similarity of two face images through distance metrics.

[0088] By inputting the true-color face image to be recognized and each face feature template of each user in the true-color face data template library into the trained Siamese network, the network will output a score or probability representing their similarity.

[0089] Step 6: Compare the similarity between the obtained true-color face image to be recognized and each user in the true-color face data template library with the similarity threshold, and extract and record the user information in the true-color face data template library corresponding to the meeting of the similarity requirements to generate the first user information set;

[0090] The logic for determining the data in the first user information set is as follows:

[0091] Calibrate the similarity threshold as yz, and the similarity between the true-color face image to be recognized and each user's face feature template is S i , where i = 1, 2,..., n*m;

[0092] When 0 ≤ S i <yz, it is determined that the similarity between the user's face feature template and the image to be recognized does not meet the requirements, and the user information is not saved to the first user information set;

[0093] When S i ≥yz, it is determined that the similarity between the user's face feature template and the image to be recognized meets the requirements, and the corresponding user information is extracted into the first user information set;

[0094] According to the similarity between the true-color face image to be recognized and each face feature template of each user in the true-color face data template library calculated by the Siamese network, there are n users, and each user has m true-color face feature models. A total of n*m similarities need to be calculated. When the similarity S between the true-color face image to be recognized and each user's face feature template i exceeds the similarity threshold yz, it indicates that the user may be the user to be recognized, and their information is extracted and recorded for the next recognition and confirmation.

[0095] Among them, as long as one face feature template image of each user has a similarity that meets the requirements with the image to be recognized, the user information will be extracted into the first user information set, and at the same time, each user's information is only extracted once.

[0096] Step 7: Calculate the average similarity and the highest similarity between the m true-color face feature templates of each user in the first user information set and the true-color face image to be recognized, and generate a decision similarity by combining the highest similarity with the average similarity; Compare the generated decision similarity with a pre-set decision similarity threshold, and judge the user information to be recognized according to different comparison results.

[0097] The formula for calculating the average similarity between the true-color face image to be recognized and each user is as follows:

[0098]

[0099] In the formula, S avgi is the average similarity of the i-th user in the first user information set, j = 1, 2,..., m, and S j represents the similarity between the j-th true-color face feature template of this user and the true-color face image to be recognized.

[0100] The formula for the highest similarity is as follows:

[0101] S maxi = max(S1, S2,..., S j )

[0102] In the formula, S maxi represents the highest similarity between the m true-color face feature templates of the i-th user in the first user information set and the true-color face image to be recognized.

[0103] The formula for generating the decision similarity by combining the highest similarity with the average similarity is as follows:

[0104] S zdi = a * S avgi + S maxi * b

[0105] In the formula, S zdi is the decision similarity of the i-th user in the first user information set, and a and b respectively represent the preset weight coefficients of the average similarity S avgi and the highest similarity S maxi , where a and b are greater than 0 and a < b; The larger the values of the average similarity S avgi and the highest similarity S maxi , the larger the decision similarity S zdi , showing a positive correlation; The decision similarity S avgi is generated through the average similarity S maxi and the highest similarity S zdi , and the users in the first user information set are further recognized according to the decision similarity S zdi .

[0106] According to different comparison results, the logic for judging the user information to be recognized is as follows:

[0107] The determined similarity threshold set for calibration is yz jd ;

[0108] When S zdi <yz jd it is determined that the user to be recognized is not this user;

[0109] When S zdi ≥yz jd it is determined that the user to be recognized is the current user.

[0110] The users in the first user information set calculate the average similarity S avgi and the highest similarity S maxi between the true-color face feature template and the true-color face image to be recognized, and generate the determined similarity S zdi for further recognition and judgment. When the determined similarity S zdi of a certain user exceeds the preset determined similarity threshold yz jd it can be determined that this user is the user to be recognized; otherwise, it is determined that the user to be recognized is not this user.

[0111] Please refer to Figure 2 The present invention also provides a face recognition system for a dense crowd. The face recognition system for a dense crowd is used to execute the above-mentioned face recognition method for a dense crowd, and includes:

[0112] A data template library establishment module for establishing a true-color face data template library. The true-color face data template library stores the true-color face feature templates and user name information of n users. Each user corresponds to m true-color face feature templates, where m is a positive integer and m≥3;

[0113] A thermal infrared image acquisition module for acquiring thermal infrared images in the area where the crowd to be recognized is dense; preprocessing the acquired thermal infrared images, and the preprocessing includes thermal infrared image denoising and enhancement processing to obtain preprocessed thermal infrared images;

[0114] A target extraction module for extracting face targets from the preprocessed thermal infrared images and labeling all face targets in the images after the face targets are extracted;

[0115] An image colorization module for cropping the thermal infrared images with labeled face targets to obtain thermal infrared images of face targets to be recognized, and performing color face conversion on the thermal infrared images of face targets to be recognized to obtain true-color face images to be recognized;

[0116] A similarity calculation module, which is used to perform user identification by comparing the obtained true-color face image to be recognized with the true-color face data template library, and calculate the similarity between the true-color face image to be recognized and each face feature template of each user in the true-color face data template library;

[0117] A user information extraction module, which is used to compare the similarity between the obtained true-color face image to be recognized and each user's face feature template in the true-color face data template library with the similarity threshold, extract and record the user information in the true-color face data template library that meets the similarity requirements, and generate a first user information set;

[0118] A face recognition comparison module, which is used to calculate the average similarity and the highest similarity between the m true-color face feature templates of each user in the first user information set and the true-color face image to be recognized, generate a decision similarity by combining the highest similarity and the average similarity; compare the generated decision similarity with a preset decision similarity threshold, and judge the user information to be recognized according to different comparison results.

[0119] All the above formulas are dimensionless and take their numerical values for calculation. The formula is obtained by collecting a large amount of data for software simulation to get a formula that is closest to the actual situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation.

[0120] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. They may be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application.

Claims

1. A method for face recognition in a dense crowd, characterized in that: The specific steps include: Establishing a true color face data template library, wherein the true color face data template library stores true color face feature templates and user name information of n users, and each user corresponds to m true color face feature templates, where m is a positive integer and m≥3; Collect thermal infrared images in a densely populated area to be identified, and preprocess the collected thermal infrared images, wherein the preprocessing includes thermal infrared image denoising and enhancement processing to obtain a preprocessed thermal infrared image; Extracting facial targets from the preprocessed thermal infrared image, and marking all facial targets in the image after facial targets are extracted; The thermal infrared image after the facial target is marked is cropped to obtain the thermal infrared image of the facial target to be identified, and the thermal infrared image of the facial target to be identified is subjected to colorization face conversion to obtain the true color face image to be identified; wherein the CycleGAN model is used to convert the thermal infrared image of the facial target into a true color face; Perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and the face feature template corresponding to each user in the true color face data template library; The similarity calculation method is as follows: use a deep learning model to extract features and calculate similarity of face images. By comparing the feature representations of two face images, the similarity score between them can be calculated. Use several face images as training data for the Siamese network. Select the contrastive loss function as the loss function of the model. Use several prepared face images and the defined loss function to train the Siamese network. During the training process, the network learns how to extract face features and compares the similarity of two face images through distance measurement. Compare the obtained true color face image to be identified with the facial feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information records in the true color face data template library corresponding to the similarity requirement, and generate a first user information set; The logic for determining the data in the first user information set is: The calibration similarity threshold is yz, and the similarity between the true color face image to be identified and each user's face feature template is S i , where i = 1, 2, ..., n*m; When 0 ≤ S i <yz, it is determined that the similarity between the user's face feature template and the image to be recognized does not meet the requirements, and the user information is not saved to the first user information set; When S i ≥yz, it is determined that the similarity between the user's facial feature template and the image to be identified meets the requirements, and the corresponding user information is extracted into the first user information set; For each user, only one facial feature template image is required to meet the similarity requirement with the image to be identified, and the user information is extracted into the first user information set. Meanwhile, the information of each user is extracted only once. Calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and combine the highest similarity with the average similarity to generate a decision similarity; compare the generated decision similarity with a preset decision similarity threshold, and determine the user information to be identified according to different comparison results; The calculation formula for the average similarity between the true color face image to be identified and each user is: In the formula, S avgi is the average similarity of the i-th user in the first user information set, j = 1, 2, ..., m, S j Indicates the similarity between the jth true color face feature template of the user and the true color face image to be identified; The formula for the highest similarity is: S maxi =max(S1,S2,…,S j ) In the formula, S maxi Indicates the highest similarity between the m true color face feature templates of the i-th user in the first user information set and the true color face image to be identified; The formula for determining similarity by combining the highest similarity with the average similarity is: S zdi =a*S avgi +S maxi *b In the formula, S zdi is the decision similarity of the i-th user in the first user information set, a and b represent the average similarity S avgi and the highest similarity S maxi The preset weight coefficient, where a, b are greater than 0 and a <b; The logic for determining the user information to be identified is: The decision similarity threshold set by the calibration is yz jd ; When S zdi <yz jd , it is determined that the user to be identified is not the user; When S zdi ≥yz jd , it is determined that the user to be identified is the current user.

2. The method for face recognition in a dense crowd according to claim 1, characterized in that: Collect thermal infrared images in the densely populated area to be identified, wherein the method for collecting thermal infrared images in the densely populated area is: use a thermal infrared camera to perform video monitoring of the densely populated area to be collected, and extract frame images from the collected video as thermal infrared images.

3. The method for face recognition in a dense crowd according to claim 2, characterized in that: Preprocessing the collected thermal infrared images, wherein the preprocessing includes denoising and enhancing the thermal infrared images; The wavelet transform denoising method is used to denoise the thermal infrared image. The specific steps of the wavelet transform denoising method include: decomposing the thermal infrared image through wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and realizing the denoising of the thermal infrared image; The logic of thermal infrared image enhancement is: using the linear stretching method, the grayscale dynamic range of the thermal infrared image is expanded to the specified range according to the linear mapping relationship. The linear stretching is based on the formula: In the formula, f LMj (V) represents the new grayscale value obtained after the grayscale value of the j-th pixel in the thermal infrared image is linearly stretched, (V) represents the grayscale value as the function independent variable, V is the grayscale value of the j-th pixel in the original image, V min Represents the minimum gray value in the original image, V max It represents the maximum gray value in the original image, D represents the dynamic range of the image after gray is compressed, and D1 represents the minimum gray value of the compressed image.

4. The method for face recognition in a dense crowd according to claim 3, characterized in that: The logic of extracting facial targets from the preprocessed image and labeling all facial targets in the image after facial targets are extracted is as follows: use the YOLOv5s network to perform thermal infrared face detection on the preprocessed image, identify the facial information in the image, organize the data containing thermal infrared faces, and then use the labeling tool LabelImg to label the thermal infrared face dataset.

5. A face recognition system for dense crowds, characterized by: The face recognition system for dense crowds is used to execute the face recognition method for dense crowds according to any one of claims 1 to 4, comprising: A data template library establishment module is used to establish a true color face data template library, wherein the true color face data template library stores true color face feature templates and user name information of n users, and each user corresponds to m true color face feature templates, where m is a positive integer and m≥3; The thermal infrared image acquisition module is used to acquire thermal infrared images in the densely populated area to be identified; preprocess the acquired thermal infrared images, the preprocessing including thermal infrared image denoising and enhancement processing, to obtain the preprocessed thermal infrared images; The target extraction module is used to extract the face target from the preprocessed thermal infrared image and mark all the face targets in the image after the face targets are extracted; The image colorization module crops the thermal infrared image after the facial target is annotated to obtain the thermal infrared image of the facial target to be identified, and performs colorization face conversion on the thermal infrared image of the facial target to be identified to obtain the true color facial image to be identified; A similarity calculation module is used to perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library; A user information extraction module is used to compare the obtained true color face image to be identified with the facial feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information records in the true color face data template library corresponding to the similarity requirement, and generate a first user information set; The face recognition comparison module is used to calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and to generate a decision similarity by combining the highest similarity with the average similarity; the generated decision similarity is compared with a pre-set decision similarity threshold, and the user information to be identified is judged according to different comparison results.

Images