Face recognition method and system for dense crowd
By establishing a true color face data template library and thermal infrared image processing technology, combined with YOLOv5s and CycleGAN models, the problem of low facial recognition accuracy under dense crowds and changing lighting is solved, and higher recognition accuracy and anti-camouflage ability.
Patent Information
- Application Number
- CN202411405944.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-10-10
AI Technical Summary
The existing facial recognition technology has reduced the recognition accuracy under dense crowds and changing lighting conditions, making it difficult to effectively deal with expressions and makeup.
Establish a true color face data template library, collect thermal infrared images for preprocessing and facial target extraction, face detection and colorization are performed through YOLOv5s and CycleGAN models, and combined with wavelet transform to denoise, calculate the similarity of face feature templates to Implement user identification.
It improves the accuracy of facial recognition, enhances resistance to ambient light changes and facial disguise, and is suitable for crowded areas.
Smart Images

Figure CN118918628A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of biometric identification technology, and in particular to a method and system for face recognition in dense crowds. Background Art
[0002] In the past few decades, with the booming development of mobile Internet, biometric recognition technology has received more and more attention, and the accuracy of identity authentication and recognition systems has become increasingly demanded. The inherent differences in human biometric features make various biometric-based methods robust and repeatable, and therefore become an important means of identity authentication and recognition. Biometric-based feature recognition includes face recognition, which has the characteristics of non-contact and scalability, and is easy for users to accept and has no psychological barriers. Face recognition technology has wide application value in various environments and fields, especially in high security, access control, and computer security. Face recognition is a technology that can verify identity by using computers to analyze facial images and extract effective feature information from them.
[0003] Whether the recognition system can adapt to the ever-changing application environment in the real world is a key issue in face recognition. Researchers have developed many complex algorithms in the field of visible light face recognition to handle these changing situations, but it is still not enough. For example, in the ever-changing lighting conditions, different facial expressions and makeup, these all make visible light face recognition have defects that cannot be recognized.
[0004] In the prior art, a method and system for face recognition is disclosed in publication number CN105868695B, which comprises: constructing a face feature template library, storing face feature templates and basic user information of N users in the face feature template library, and each user corresponds to at least one face feature template; extracting face features in the face image of the user to be identified, and obtaining the face feature template to be identified; calculating the similarity between the face feature template to be identified and the face feature template of each user in the face feature template library, first taking the maximum value of the maximum similarity between the face feature template to be identified and each user, and judging whether the user is the user corresponding to the maximum value, if not, taking the maximum value of the average similarity between the face feature template to be identified and each user, and judging whether the user is the user corresponding to the maximum value; this method can effectively improve the accuracy of face recognition, but the premise is that the face feature template to be identified has a high accuracy. When the face feature template to be identified is in a crowded area or under changing lighting conditions, the accuracy is reduced, and a defect of being unable to be recognized may occur.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention
[0006] The purpose of the present invention is to provide a method and system for face recognition in dense crowds to solve the problems raised in the above background technology.
[0007] To achieve the above object, the present invention provides the following technical solutions:
[0008] A method for face recognition in a dense crowd, the specific steps comprising:
[0009] Establish a true color face data template library, the true color face data template library stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates;
[0010] Collect thermal infrared images in the densely populated area to be identified; preprocess the collected thermal infrared images, wherein the preprocessing includes thermal infrared image denoising and enhancement processing to obtain a preprocessed thermal infrared image;
[0011] Extracting face targets from the preprocessed thermal infrared image, and marking all face targets in the image after face target extraction;
[0012] The thermal infrared image after the facial target is marked is cropped to obtain a thermal infrared image of the facial target to be identified, and the thermal infrared image of the facial target to be identified is subjected to colorization face conversion to obtain a true color facial image to be identified;
[0013] Perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library;
[0014] Compare the obtained true color face image to be identified with the face feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information in the true color face data template library corresponding to the similarity requirement, and generate a first user information set;
[0015] Calculate the average similarity and the highest similarity between the m true color facial feature templates of each user in the first user information set and the true color facial image to be identified, and generate a decision similarity by combining the highest similarity with the average similarity; compare the generated decision similarity with a pre-set decision similarity threshold, and judge the user information to be identified according to different comparison results.
[0016] Furthermore, a true color face data template library is established, wherein the true color face data template library stores The true color face feature data and user name information of each user are stored in the established true color face data template library. Each user has m true color face feature templates, where m is a positive integer and .
[0017] Furthermore, thermal infrared images are collected in the densely populated area to be identified, wherein the method for collecting thermal infrared images in the densely populated area is: use a thermal infrared camera to perform video monitoring of the densely populated area to be collected, and extract frame images from the collected video as thermal infrared images.
[0018] Further, preprocessing is performed on the collected thermal infrared image, and the preprocessing includes thermal infrared image denoising and enhancement processing;
[0019] The wavelet transform denoising method is used to denoise the thermal infrared image. The specific steps of the wavelet transform denoising method include: decomposing the thermal infrared image through wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and realizing the denoising of the thermal infrared image;
[0020] The logic of thermal infrared image enhancement is: using the linear stretching method, the grayscale dynamic range of the thermal infrared image is expanded to the specified range according to the linear mapping relationship. The linear stretching is based on the formula:
[0021]
[0022] In the formula, The original image The gray value of a pixel, Represents the minimum gray value in the original image. Represents the maximum gray value in the original image. Indicates the dynamic range of the image after grayscale compression. Indicates the minimum grayscale value of the compressed image.
[0023] Furthermore, the logic of extracting facial targets from the preprocessed image and labeling all facial targets in the image after facial targets are extracted is as follows: use the YOLOv5s network to perform thermal infrared face detection on the preprocessed image, identify the facial information in the image, organize the data containing thermal infrared faces, and then use the labeling tool LabelImg to label the thermal infrared face dataset.
[0024] Furthermore, the thermal infrared image of the labeled face target is cropped to obtain a thermal infrared image of the face target, and the thermal infrared image of the face target is converted into a color face to obtain a true color face image to be identified, wherein the CycleGAN model is used to convert the thermal infrared image of the face target into a true color face, and a generator G is trained. The function of the generator G is to input a thermal infrared face image and output a true color face image, and a colorized face image with better quality is generated through an iterative learning process.
[0025] Further, the obtained true color face image to be identified is compared with the similarity and similarity threshold of each user face feature template in the true color face data template library, and the user information in the true color face data template library corresponding to the similarity requirement is extracted and recorded to generate a first user information set;
[0026] The logic for determining the data in the first user information set is:
[0027] The calibration similarity threshold is , the similarity between the true color face image to be identified and each user's face feature template is ,in ;
[0028] when , it is determined that the similarity between the user's facial feature template and the image to be identified does not meet the requirement, and the user information is not saved in the first user information set;
[0029] when , determining that the similarity between the user's facial feature template and the image to be identified meets the requirements, and extracting the corresponding user information into the first user information set;
[0030] For each user, only one facial feature template image needs to be presented and the similarity with the image to be identified meets the requirement, and the user information will be extracted into the first user information set. Meanwhile, the information of each user is extracted only once.
[0031] Further, the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified are calculated, and the highest similarity is combined with the average similarity to generate a decision similarity;
[0032] The calculation formula for the average similarity between the true color face image to be identified and each user is:
[0033]
[0034] In the formula, The first user information set The average similarity of users, , Indicates that the user The similarity between a true color face feature template and the true color face image to be identified;
[0035] The formula for the highest similarity is:
[0036]
[0037] In the formula, Indicates the first user information set The highest similarity between the m true color face feature templates of the user and the true color face image to be identified;
[0038] The formula for determining similarity by combining the highest similarity with the average similarity is:
[0039]
[0040] In the formula, The first user information set The decision similarity of users, Represents the average similarity and the highest similarity The preset weight coefficient of Greater than 0 and .
[0041] Furthermore, the generated decision similarity is compared with a preset decision similarity threshold, and according to different comparison results, the logic for determining the user information to be identified is as follows:
[0042] The decision similarity threshold set by the calibration is ;
[0043] when , it is determined that the user to be identified is not the user;
[0044] when , it is determined that the user to be identified is the current user.
[0045] The present invention also provides a face recognition system for dense crowds, wherein the face recognition system for dense crowds is used to execute the above-mentioned face recognition method for dense crowds, and comprises:
[0046] A data template library establishment module is used to establish a true color face data template library, wherein the true color face data template library stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates;
[0047] The thermal infrared image acquisition module is used to acquire thermal infrared images in the densely populated area to be identified; preprocess the acquired thermal infrared images, the preprocessing including thermal infrared image denoising and enhancement processing, to obtain the preprocessed thermal infrared images;
[0048] The target extraction module is used to extract the face target from the preprocessed thermal infrared image and mark all the face targets in the image after the face targets are extracted;
[0049] The image colorization module crops the thermal infrared image after the facial target is annotated to obtain the thermal infrared image of the facial target to be identified, and performs colorization face conversion on the thermal infrared image of the facial target to be identified to obtain the true color facial image to be identified;
[0050] A similarity calculation module is used to perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library;
[0051] A user information extraction module is used to compare the obtained true color face image to be identified with the facial feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information records in the true color face data template library corresponding to the similarity requirement, and generate a first user information set;
[0052] The face recognition comparison module is used to calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and to generate a decision similarity by combining the highest similarity with the average similarity; the generated decision similarity is compared with a pre-set decision similarity threshold, and the user information to be identified is judged according to different comparison results.
[0053] Compared with the prior art, the present invention has the following beneficial effects: a true color face data template library is established, wherein the true color face data template library stores The method comprises the following steps: collecting the true color facial feature templates and basic information of each user, each user corresponding to at least 3 true color facial feature templates; collecting thermal infrared images in crowded areas, preprocessing the collected thermal infrared images, extracting facial targets from the preprocessed thermal infrared images, and marking all facial targets in the images after facial targets are extracted; cropping the thermal infrared images after facial targets are marked to obtain thermal infrared images of facial targets to be identified, performing colorized face conversion on the thermal infrared images of facial targets to be identified, and obtaining true color facial images to be identified; performing user identification based on the comparison between the obtained true color facial images to be identified and the true color facial data template library, and calculating the similarity between the true color facial images to be identified and each facial feature template of each user in the true color facial data template library; comparing the obtained true color facial images to be identified with the corresponding facial features of each user in the true color facial data template library. The similarity is compared with the similarity threshold, and the user information in the true color face data template library corresponding to the similarity requirement is extracted to generate a first user information set; the average similarity and the highest similarity of the m true color face feature templates of each user in the first user information set and the true color face image to be identified are calculated, and the highest similarity is combined with the average similarity to generate a decision similarity; the generated decision similarity is compared with the pre-set decision similarity threshold, and the user information to be identified is judged according to different comparison results; the thermal infrared face image is basically not affected by the ambient light, has strong anti-counterfeiting and good concealment, and does not require compensation. It is better in solving face recognition problems and facial camouflage problems under conditions of changing light, and has good application in crowded areas. At the same time, the image to be identified is judged by calculating the highest similarity and average similarity of the user image in the template library to improve the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a schematic diagram of the overall method flow of the present invention;
[0055] Figure 2 It is a schematic diagram of the overall system flow of the present invention. DETAILED DESCRIPTION
[0056] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.
[0057] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0058] Example:
[0059] See also Figure 1 , the present invention provides a technical solution:
[0060] A method for face recognition in a dense crowd, the specific steps comprising:
[0061] Step 1: Establish a true color face data template library, which stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates;
[0062] In the established true color face data template library, each user corresponds to m true color face feature templates, where m is a positive integer and ; The number of facial feature templates for different users may be different.
[0063] Step 2: Collect thermal infrared images in the crowd-dense area to be identified; preprocess the collected thermal infrared images, including thermal infrared image denoising and enhancement processing, to obtain preprocessed thermal infrared images;
[0064] Collecting thermal infrared images in crowded areas, wherein the crowded areas include stations, scenic spots and streets, wherein the method for collecting thermal infrared images in crowded areas is: using a thermal infrared camera to perform video monitoring on the crowded areas to be collected, and extracting frame images from the collected videos as thermal infrared images;
[0065] The thermal infrared camera is composed of an infrared detector, an optical lens and a circuit board. The photosensitive element on the infrared detector receives the energy distribution radiated by the target object. The optical scanning is placed between the infrared detector and the optical lens to obtain infrared thermal imaging of the object to be inspected.
[0066] The advantages of infrared thermal imaging technology are mainly reflected in the following aspects: First, it has a high safety factor and high efficiency. It is easy to operate and can effectively observe temperature distribution. Secondly, it has strong target tracking ability and strong anti-interference ability. When tracking the target, it is not affected by the electromagnetic materials around the target, and it also has a good detection effect on distant targets. Then, it can monitor the target around the clock and can play a good detection effect in complex and severe weather or environment. In addition, it has high detection accuracy and long distance. It has a good use effect in complex environments with dense crowds.
[0067] Preprocessing the collected thermal infrared images, wherein the preprocessing includes denoising and enhancing the thermal infrared images;
[0068] Among them, the wavelet transform denoising method is used to denoise the thermal infrared image. Wavelet transform is often used when processing thermal infrared images. The specific steps of the wavelet transform denoising method include: reading the thermal infrared image and converting it into a grayscale image, normalizing the image so that the pixel value is between 0 and 1; selecting a suitable wavelet basis function and determining the wavelet function (such as Daubechies wavelet db4); selecting the number of layers of wavelet decomposition, generally 3 to 5 layers; thresholding the wavelet coefficients, selecting the threshold method, the threshold method includes hard threshold or soft threshold, and calculating the threshold; using the wavelet coefficients after thresholding to perform inverse wavelet transform and reconstruct the denoised image; the threshold is calculated using the VisuShrink method, and the specific calculation formula is:
[0069]
[0070] In the formula, is the threshold value, is the standard deviation of the noise, is the total number of pixels in the image;
[0071] The logic of thermal infrared image enhancement is as follows: linear stretching is used. Linear Map is a simple and commonly used image enhancement technology, which is mainly used to adjust the contrast of the image, map the grayscale dynamic range (i.e., the range of pixel values) of the original image to a specified range, and expand the grayscale dynamic range of the thermal infrared image to a specified range according to the linear mapping relationship. The infrared camera converts the captured analog radiation signal into a digital signal through an analog-to-digital converter (ADC). The converted digital signal represents the grayscale value of each pixel. The 14-bit A / D original image data is processed and converted into 8-bit image data through the linear stretching method.
[0072] The formula on which its linear stretch is based is:
[0073]
[0074] In the formula, The original image The grayscale value of each pixel, the first pixel in the upper left corner of the image to be preprocessed is the first pixel, and the numbers are arranged in order of row priority. Represents the minimum gray value in the original image. Represents the maximum gray value in the original image. Indicates the dynamic range of the image after grayscale compression. Indicates the minimum grayscale value of the compressed image.
[0075] Step 3: extract the face target from the preprocessed thermal infrared image, and mark all the face targets in the image after the face target is extracted;
[0076] The logic of extracting face targets from the preprocessed images and labeling all face targets in the images is as follows: Use the YOLOv5s network to perform thermal infrared face detection on the preprocessed images, identify the face information in the images, organize the data containing thermal infrared faces, and then use the labeling tool LabelImg to label the thermal infrared face dataset;
[0077] Among them, the main function of the YOLOv5s network is to perform real-time target detection. Target detection is an important task in the field of computer vision. Its goal is to detect various objects of different categories from images or videos and accurately mark their locations. YOLOv5s is a model in the YOLO (You Only Look Once) series. It strikes a balance between real-time and accuracy and is suitable for scenarios that require high speed and lightweight models.
[0078] Use the thermal infrared images with labeled facial information as input to train the YOLOv5s network, and use the bounding box corresponding to each facial image as a label to train the YOLOv5s network to obtain a YOLOv5s model for thermal infrared face recognition; input the preprocessed thermal infrared images into the trained YOLOv5s model to extract facial targets from the thermal infrared images;
[0079] After extracting the face targets, all the face targets in the image are labeled, and the VOC format dataset is created through the LabelImg tool, and then the VOC format dataset is converted into a YOLO format dataset suitable for the YOLOv5s network. The category is face, and it is labeled as face.
[0080] Step 4: crop the thermal infrared image after the facial target is annotated to obtain the thermal infrared image of the facial target to be identified, and perform color face conversion on the thermal infrared image of the facial target to be identified to obtain the true color facial image to be identified;
[0081] The method for cropping the thermal infrared image after marking the face target is: cropping based on the minimum bounding rectangle, and the specific steps include: using the face detection algorithm YOLOv5s network and the LabelImg tool to mark the bounding box of the face; obtaining the coordinates of the minimum bounding rectangle of the marked face, including the coordinates of the upper left corner boundary point and the coordinates of the lower right corner boundary point of the minimum bounding rectangle; cropping the thermal infrared image of the face target to be identified containing the complete face image according to the coordinates of the minimum bounding rectangle of the marked face;
[0082] The CycleGAN model is used to convert the target thermal infrared image of the face into a true color face. The main idea of the CycleGAN model is to learn the mapping relationship between the two domains through adversarial training. It consists of two generator networks and two discriminator networks. The specific steps of using the CycleGAN model to convert the target thermal infrared image of the face into a true color face are as follows:
[0083] Two generators are trained and two image domains are prepared. One image domain stores thermal infrared face images and the other image domain stores true color face images. One generator maps the thermal infrared face images in the infrared face image domain to another true color face image domain, while the other generator maps the latter images back to the former. The two discriminator networks are used to identify whether the generated images are real and provide feedback to the generator network to improve the quality of the generated images. Through the iterative learning process, higher quality colorized face images are generated.
[0084] Finally, the trained generator G is used, which inputs the thermal infrared image of the face target and outputs the true color face image to be identified.
[0085] Step 5: Perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library;
[0086] The similarity is calculated by using a deep learning model (such as a face recognition model based on a convolutional neural network) to extract features and calculate similarity of face images. By comparing the feature representations of two face images, the similarity score between them can be calculated. The Siamese network is a neural network structure used to measure the similarity between two inputs.
[0087] Use several face images as training data for the Siamese network; select the Contrastive Loss function as the loss function of the model, and use the prepared face images and the defined loss function to train the Siamese network. During the training process, the network learns how to extract facial features and compare the similarity of two face images through distance measurement.
[0088] By inputting the true color face image to be identified and each facial feature template of each user in the true color face data template library into the trained Siamese network, the network will output a score or probability representing their similarity.
[0089] Step 6: Compare the obtained true color face image to be identified with the similarity and similarity threshold of each user in the true color face data template library, extract the user information in the true color face data template library corresponding to the similarity requirement, and generate a first user information set;
[0090] The logic for determining the data in the first user information set is:
[0091] The calibration similarity threshold is , the similarity between the true color face image to be identified and each user's face feature template is ,in ;
[0092] when , it is determined that the similarity between the user's facial feature template and the image to be identified does not meet the requirement, and the user information is not saved in the first user information set;
[0093] when , determining that the similarity between the user's facial feature template and the image to be identified meets the requirements, and extracting the corresponding user information into the first user information set;
[0094] According to the Siamese network, the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library is calculated. users, each with True color face feature models need to be calculated Similarity, when the calculated similarity between the true color face image to be identified and each user's face feature template Exceeding the similarity threshold , it indicates that the user may be a user to be identified, and his information is extracted and recorded for the next step of identification and confirmation.
[0095] For each user, only one facial feature template image needs to be presented and the similarity with the image to be identified meets the requirement, and the user information will be extracted into the first user information set. Meanwhile, the information of each user is extracted only once.
[0096] Step 7: Calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and combine the highest similarity with the average similarity to generate a decision similarity; compare the generated decision similarity with a preset decision similarity threshold, and determine the user information to be identified based on different comparison results;
[0097] The calculation formula for the average similarity between the true color face image to be identified and each user is:
[0098]
[0099] In the formula, The first user information set The average similarity of users, , Indicates that the user The similarity between a true color face feature template and the true color face image to be identified;
[0100] The formula for the highest similarity is:
[0101]
[0102] In the formula, Indicates the first user information set The highest similarity between the m true color face feature templates of the user and the true color face image to be identified;
[0103] The formula for determining similarity by combining the highest similarity with the average similarity is:
[0104]
[0105] In the formula, The first user information set The decision similarity of users, Represents the average similarity and the highest similarity The preset weight coefficient of Greater than 0 and ; Average similarity and the highest similarity The larger the value, the more similar it is. The larger the value, the positive correlation is; through the average similarity and the highest similarity Generate decision similarity , according to the similarity Further identify users in the first user information set.
[0106] According to different comparison results, the logic for determining the user information to be identified is:
[0107] The decision similarity threshold set by the calibration is ;
[0108] when , it is determined that the user to be identified is not the user;
[0109] when , it is determined that the user to be identified is the current user.
[0110] The users in the first user information set calculate the average similarity between the true color face feature template and the true color face image to be identified. and the highest similarity , generating the decision similarity , to make further identification and judgment, when a user's decision similarity Exceeds the preset decision similarity threshold When the user is identified, it can be determined that the user is the user to be identified, otherwise it can be determined that the user to be identified is not the user.
[0111] See also Figure 2 The present invention also provides a face recognition system for dense crowds, wherein the face recognition system for dense crowds is used to execute the above-mentioned face recognition method for dense crowds, and comprises:
[0112] A data template library establishment module is used to establish a true color face data template library, wherein the true color face data template library stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates;
[0113] The thermal infrared image acquisition module is used to acquire thermal infrared images in the densely populated area to be identified; preprocess the acquired thermal infrared images, the preprocessing including thermal infrared image denoising and enhancement processing, to obtain the preprocessed thermal infrared images;
[0114] The target extraction module is used to extract the face target from the preprocessed thermal infrared image and mark all the face targets in the image after the face targets are extracted;
[0115] The image colorization module crops the thermal infrared image after the facial target is annotated to obtain the thermal infrared image of the facial target to be identified, and performs colorization face conversion on the thermal infrared image of the facial target to be identified to obtain the true color facial image to be identified;
[0116] A similarity calculation module is used to perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library;
[0117] A user information extraction module is used to compare the obtained true color face image to be identified with the facial feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information records in the true color face data template library corresponding to the similarity requirement, and generate a first user information set;
[0118] The face recognition comparison module is used to calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and to generate a decision similarity by combining the highest similarity with the average similarity; the generated decision similarity is compared with a pre-set decision similarity threshold, and the user information to be identified is judged according to different comparison results.
[0119] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0120] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.
[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0122] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. A method for face recognition in a dense crowd, characterized in that: The specific steps include: Establish a true color face data template library, the true color face data template library stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates; Collect thermal infrared images in a densely populated area to be identified, and preprocess the collected thermal infrared images, wherein the preprocessing includes thermal infrared image denoising and enhancement processing to obtain a preprocessed thermal infrared image; Extracting face targets from the preprocessed thermal infrared image, and marking all face targets in the image after face targets are extracted; The thermal infrared image after the facial target is marked is cropped to obtain a thermal infrared image of the facial target to be identified, and the thermal infrared image of the facial target to be identified is subjected to colorization face conversion to obtain a true color facial image to be identified; Perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and the face feature template corresponding to each user in the true color face data template library; Compare the obtained true color face image to be identified with the face feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information in the true color face data template library corresponding to the similarity requirement, and generate a first user information set; Calculate the average similarity and the highest similarity between the m true color facial feature templates of each user in the first user information set and the true color facial image to be identified, and generate a decision similarity by combining the highest similarity with the average similarity; compare the generated decision similarity with a pre-set decision similarity threshold, and judge the user information to be identified according to different comparison results.
2. The method for face recognition in a dense crowd according to claim 1, characterized in that: Establish a true color face data template library, the true color face data template library stores The true color face feature template and user name information of each user. In the established true color face data template library, each user corresponds to m true color face feature templates, where m is a positive integer and .
3. The method for face recognition in a dense crowd according to claim 1, characterized in that: Collect thermal infrared images in the densely populated area to be identified, wherein the method for collecting thermal infrared images in the densely populated area is: use a thermal infrared camera to perform video monitoring of the densely populated area to be collected, and extract frame images from the collected video as thermal infrared images.
4. The method for face recognition in a dense crowd according to claim 3, characterized in that: Preprocessing the collected thermal infrared images, wherein the preprocessing includes denoising and enhancing the thermal infrared images; The wavelet transform denoising method is used to denoise the thermal infrared image. The specific steps of the wavelet transform denoising method include: decomposing the thermal infrared image through wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and realizing the denoising of the thermal infrared image; The logic of thermal infrared image enhancement is: using the linear stretching method, the grayscale dynamic range of the thermal infrared image is expanded to the specified range according to the linear mapping relationship. The linear stretching is based on the formula: In the formula, The original image The gray value of a pixel, Represents the minimum gray value in the original image. Represents the maximum gray value in the original image. Indicates the dynamic range of the image after grayscale compression. Indicates the minimum grayscale value of the compressed image.
5. The method for face recognition in a dense crowd according to claim 4, characterized in that: The logic of extracting facial targets from the preprocessed image and labeling all facial targets in the image after facial targets are extracted is as follows: use the YOLOv5s network to perform thermal infrared face detection on the preprocessed image, identify the facial information in the image, organize the data containing thermal infrared faces, and then use the labeling tool LabelImg to label the thermal infrared face dataset.
6. The method for face recognition in a dense crowd according to claim 1, characterized in that: The thermal infrared image of the marked face target is cropped to obtain the thermal infrared image of the face target to be identified, and the thermal infrared image of the face target to be identified is colorized to obtain the true color face image to be identified, wherein the CycleGAN model is used to convert the thermal infrared image of the face target into a true color face.
7. The method for face recognition in a dense crowd according to claim 1, characterized in that: Compare the obtained true color face image to be identified with the similarity and similarity threshold of each user face feature template in the true color face data template library, extract the user information in the true color face data template library corresponding to the similarity requirement, and generate a first user information set; The logic for determining the data in the first user information set is: The calibration similarity threshold is , the similarity between the true color face image to be identified and each user's face feature template is ,in ; when , it is determined that the similarity between the user's facial feature template and the image to be identified does not meet the requirement, and the user information is not saved in the first user information set; when , determining that the similarity between the user's facial feature template and the image to be identified meets the requirements, and extracting the corresponding user information into the first user information set; For each user, only one facial feature template image needs to be presented and the similarity with the image to be identified meets the requirement, and the user information will be extracted into the first user information set. Meanwhile, the information of each user is extracted only once.
8. The method for face recognition in a dense crowd according to claim 1, characterized in that: Calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and combine the highest similarity with the average similarity to generate a decision similarity; The calculation formula for the average similarity between the true color face image to be identified and each user is: In the formula, The first user information set The average similarity of users, , Indicates that the user The similarity between a true color face feature template and the true color face image to be identified; The formula for the highest similarity is: In the formula, Indicates the first user information set The highest similarity between the m true color face feature templates of the user and the true color face image to be identified; The formula for determining similarity by combining the highest similarity with the average similarity is: In the formula, The first user information set The decision similarity of users, Represents the average similarity and the highest similarity The preset weight coefficient of Greater than 0 and .
9. The method for face recognition in a dense crowd according to claim 8, characterized in that: By comparing the generated decision similarity with the pre-set decision similarity threshold, the logic for determining the user information to be identified is as follows based on different comparison results: The decision similarity threshold set by the calibration is ; when , it is determined that the user to be identified is not the user; when , it is determined that the user to be identified is the current user.
10. A face recognition system for dense crowds, characterized by: The face recognition system for dense crowds is used to execute the face recognition method for dense crowds according to any one of claims 1 to 9, comprising: A data template library establishment module is used to establish a true color face data template library, wherein the true color face data template library stores True color face feature templates and user name information of each user, each user corresponds to at least 3 true color face feature templates; The thermal infrared image acquisition module is used to acquire thermal infrared images in the densely populated area to be identified; preprocess the acquired thermal infrared images, the preprocessing including thermal infrared image denoising and enhancement processing, to obtain the preprocessed thermal infrared images; The target extraction module is used to extract the face target from the preprocessed thermal infrared image and mark all the face targets in the image after the face targets are extracted; The image colorization module crops the thermal infrared image after the facial target is annotated to obtain the thermal infrared image of the facial target to be identified, and performs colorization face conversion on the thermal infrared image of the facial target to be identified to obtain the true color facial image to be identified; A similarity calculation module is used to perform user identification based on the comparison between the obtained true color face image to be identified and the true color face data template library, and calculate the similarity between the true color face image to be identified and each face feature template of each user in the true color face data template library; A user information extraction module is used to compare the obtained true color face image to be identified with the facial feature template similarity and similarity threshold of each user in the true color face data template library, extract the user information records in the true color face data template library corresponding to the similarity requirement, and generate a first user information set; The face recognition comparison module is used to calculate the average similarity and the highest similarity between the m true color face feature templates of each user in the first user information set and the true color face image to be identified, and to generate a decision similarity by combining the highest similarity with the average similarity; the generated decision similarity is compared with a pre-set decision similarity threshold, and the user information to be identified is judged according to different comparison results.
Citation Information
Patent Citations
A facial recognition method and system
CN105868695B
Human face recognition method and system
CN105868695A
Thermal infrared face recognition identity authentication method and device and storage medium
CN116453189A
Facial recognition method and apparatus, terminal, and computer readable storage medium
WO2020037937A1
Face recognition method and system
WO2023000864A1
Cited By
False face detection method based on agent-guided hybrid expert network and related equipment
CN121600581A
Method for detecting a fake face based on an agent-guided hybrid expert network and related device
CN121600581B
