A face tracking and recognition method, system and storage medium

Abstract

The present invention relates to a face tracking and recognition method, system, and storage medium. The method comprises the following steps: receiving and capturing a video sequence of a person to be identified; generating a first human image from the video sequence and performing image reconstruction; extracting human feature values ​​from the reconstructed human image to generate a human feature map; detecting a human head from the first human image and performing 3D facial image reconstruction; extracting facial feature values ​​from the reconstructed second human head image to generate a facial feature map; splicing the facial feature map and human feature values ​​to form a face-body feature map; and performing comparative identification of the person to be identified based on the face-body feature map. The present invention effectively addresses the issues affected by factors such as large scene size, low-pixel imaging equipment, long distance, and environmental factors, achieving face tracking and recognition in complex and large scenes, and the resulting facial images are clear and high-resolution.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a face tracking and recognition method, system and storage medium. Background Art

[0002] The monitoring system is one of the most widely used systems in security systems. The more suitable construction site monitoring system on the market is handheld video communication equipment, and video monitoring is now the mainstream.

[0003] At present, the existing face recognition and tracking system is affected by many factors such as large scenes, low pixels of imaging devices, long distances, and the environment. The human images collected by ordinary surveillance cameras at high altitudes will cause the facial feature data collected by the collection equipment to be inaccurate and unreliable, which in turn leads to errors in pedestrian recognition, inaccurate recognition effects, and very low recognition rates, so that the results of recognition and tracking using only facial features are poor. Summary of the Invention

[0004] In order to solve the above-mentioned problem of recognition rate, the present invention provides a face tracking and recognition method, system and storage medium.

[0005] In a first aspect, the present invention provides a face tracking and recognition method, which adopts the following technical solution:

[0006] A face tracking and recognition method, comprising the following steps:

[0007] Receiving a video sequence containing an identified person from surveillance acquisition;

[0008] Detecting individual image samples according to the video sequence, locating a human body in the individual image samples using a human body detection algorithm, and generating a first human body image;

[0009] performing image reconstruction on the first human body image to generate a second human body image;

[0010] Extracting human body feature values ​​from the generated second human body image to generate a human body feature map;

[0011] Performing human head detection on the first human head image to obtain a first human head image, and performing 3D facial image reconstruction on the first human head image to generate a second human head image;

[0012] Extracting facial feature values ​​based on the second human head image to generate a facial feature map;

[0013] The face feature map and the body feature map are feature-joined to form a face-body feature map, and the person to be identified is compared and identified based on the face-body feature map.

[0014] By adopting the above technical solution and utilizing the advantages of convolutional neural networks, a video sequence in a complex scene is first obtained, a first human image and a first human head image are detected from the video sequence, and the first human image sample is reconstructed to obtain an individual image with higher clarity, which is more conducive to obtaining human features. 3D face reconstruction is performed on the obtained first human head image to extract facial features and enhance the recognition rate of the face. Finally, the human features and facial features are spliced ​​together through feature splicing, and the person to be tracked is identified and tracked based on the spliced ​​features, resulting in more accurate recognition results. Compared with the prior art, the present invention extracts facial and human features through network models with different architectures, performs feature splicing, and identifies pedestrians based on the spliced ​​features, thereby improving recognition efficiency, saving resources, and ensuring the correspondence between the face and the body, solving the problem of poor recognition and tracking results, improving practicality, and effectively solving the influence of many factors such as large scenes, low pixels of imaging devices, long distances, and environments, thereby achieving face tracking and recognition in complex and large scenes, and obtaining clear human images with higher resolution.

[0015] Preferably, the step of performing image reconstruction on the first human body image specifically includes:

[0016] The first human body image is reconstructed using a pre-generated super-resolution reconstruction network model to obtain a second human body image.

[0017] Preferably, the method for generating the super-resolution reconstruction network model includes:

[0018] Set training parameters and content loss function to build the initial model;

[0019] Collecting a training sample set, wherein the training sample set includes individual image samples and corresponding low-resolution images;

[0020] A low-resolution image is input into the initial model, and the resolution of the low-resolution image is enhanced by the initial model to obtain an image SR. The image SR is compared with the individual image sample to obtain a difference value. The training parameters are adjusted using the difference value until the content loss function is reduced to a preset minimum threshold, thereby obtaining a super-resolution reconstruction network model.

[0021] Preferably, the human detection algorithm is a deep learning target detection algorithm.

[0022] Preferably, the method for generating the first human head image comprises the following steps:

[0023] Sending the first human body image to a pre-generated human head detection model;

[0024] Obtaining human head frame information based on the pre-generated human head detection model;

[0025] The first human body image is cut according to the human head frame information to obtain a first human head image.

[0026] Preferably, the method for generating the human head detection model includes:

[0027] Select the human head frame information to construct the original training set;

[0028] Randomly magnify the area within the human head frame to construct an augmented training set;

[0029] Input the augmented training set into the original human head detection model to obtain the predicted human head frame information;

[0030] The predicted human head frame information is combined with the human head frame information in the original training set to calculate the total loss function, train the total loss function, and obtain the human head detection model.

[0031] By adopting the above technical solution, a human head detection model is obtained by training a deep convolutional neural network, which avoids the block preprocessing operation of the face image, speeds up the acquisition of human head image annotation, supports the input of color images of any size, completes face editing operations based on human head image annotation, retains details that the human eye pays more attention to, such as eyes, hair, and background, enhances the realism of the edited image, and broadens the ideas of human head editing.

[0032] Preferably, the method for reconstructing a 3D face image includes:

[0033] Acquire a first human head image from the first human image;

[0034] The first human head image is reconstructed using a pre-generated 3D human face model to obtain a second human head image.

[0035] Preferably, the 3D face model includes: an identity consistency learning part and a scene optimization part;

[0036] The identity consistency learning part adopts an unsupervised deep learning method to decompose the input face image sample into a standard depth map and a standard original color map, and then constructs the reconstruction loss through the rendering result of the renderer;

[0037] The scene optimization part uses an encoder to extract real texture and depth information from the target image, and then integrates it into the "standard face" for 3D face reconstruction.

[0038] In a second aspect, the present invention provides a face tracking and recognition system, which adopts the following technical solutions:

[0039] A face tracking and recognition system in complex scenes includes an image acquisition module for acquiring a video sequence of a person to be identified in the complex scene, thereby obtaining individual image samples;

[0040] A human body positioning module is used to perform preliminary human body positioning processing on individual image samples to obtain a first human body image of a candidate human body region;

[0041] An image reconstruction module, configured to reconstruct the first human body image to obtain a second human body image with higher definition;

[0042] A human feature extraction module is used to extract human feature values ​​from the generated human image and generate a human feature map;

[0043] A human head positioning generation module is used to detect a human head from a first human image, select human head frame information, and cut the human image according to the human head frame information to obtain a first human head image;

[0044] A 3D face image reconstruction module is used to perform 3D face image reconstruction on the first human head image to obtain a second human head image;

[0045] A facial feature extraction module is used to extract facial feature values ​​from the second human head image and generate a facial feature map;

[0046] A feature splicing module is used to perform feature splicing on the face feature map and the body feature map to form a face-body feature map;

[0047] The comparison, recognition and tracking module compares the generated face-body feature map with the features of the person to be tracked to determine whether they are the same person, thereby achieving tracking and recognition.

[0048] By adopting the above technical solution, a video sequence in a complex scene is obtained through an image acquisition module, a first human image and a first human head image are detected from the video sequence, and then an image reconstruction module is used to perform image reconstruction to obtain a second human image with higher clarity, which is more conducive to obtaining human features. 3D face reconstruction is performed on the obtained first human head image, facial features are extracted, and the face recognition rate is enhanced. Finally, the face and body features are spliced ​​through a feature splicing module, and pedestrians are identified based on the spliced ​​features, thereby improving recognition efficiency, saving resources, and ensuring the correspondence between faces and bodies, solving the problem of poor recognition and tracking results, improving practicality, and effectively solving the influence of many factors such as large scenes, low pixels of imaging devices, long distances, and environments, thereby realizing face tracking and recognition in complex and large scenes, and the obtained facial images are clear and high in resolution.

[0049] In a third aspect, the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0050] A computer-readable storage medium stores a computer program capable of being loaded and executed by a processor according to the first aspect.

[0051] By adopting the above technical solution and taking advantage of the convolutional neural network, a video sequence in a complex scene is first obtained, a first human image is detected from the video sequence, and then the image reconstruction module is used to reconstruct the image to obtain a second human image with higher clarity, which is more conducive to obtaining human features. The obtained human head image samples are subjected to 3D face reconstruction, facial features are extracted, and the face recognition rate is enhanced. Finally, the facial features and human features are spliced ​​together through the feature splicing module, and the spliced ​​face-body feature map is compared to achieve tracking and recognition. The comparison result is more accurate, and the problem of being affected by many factors such as large scenes, low pixels of imaging devices, long distances, and the environment is effectively solved. Face tracking and recognition in complex and large scenes is achieved, and the obtained face images are clear and high in resolution.

[0052] In summary, the present invention has the following beneficial technical effects:

[0053] The face tracking and recognition method, system and storage medium described in the present invention, compared with the existing technology, extracts face and body features through network models with different architectures, that is, utilizes the advantages of convolutional neural networks, first obtains a video sequence in a complex scene, uses a human body detection algorithm to locate the human body in the image, and then reconstructs the image to obtain a second human body image with higher clarity, extracts human body features, performs 3D face reconstruction on the obtained first human head image, extracts facial features, enhances the recognition rate of the face, and performs feature splicing on the extracted facial features and human body features. Pedestrians are identified based on the spliced ​​features, thereby improving recognition efficiency, saving resources, and ensuring the correspondence between faces and bodies, solving the problem of poor recognition and tracking results, improving practicality, and effectively solving the influence of many factors such as large scenes, low pixels of imaging devices, long distances, and environment, thereby realizing face tracking and recognition in complex and large scenes, and the obtained face images are clear and high in resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of a face tracking and recognition method shown in an embodiment of the present invention.

[0055] Figure 2 This is a flow chart of a face tracking and recognition system shown in an embodiment of the present invention.

[0056] Explanation of the accompanying symbols: 1. Image acquisition module; 2. Human body positioning module; 3. Image reconstruction module; 4. Human body feature extraction module; 5. Human head positioning generation module; 6. 3D face image reconstruction module; 7. Face feature extraction module; 8. Feature stitching module; 9. Comparison, recognition and tracking module. DETAILED DESCRIPTION

[0057] The following combination Figure 1-Figure 2 The present invention is described in further detail.

[0058] The embodiment of the present invention discloses a face tracking and recognition method. Figure 1 , a face tracking and recognition method, the method comprising the following steps:

[0059] 100, receiving a video sequence containing a person to be identified from surveillance collection, detecting individual image samples based on the video sequence, locating the human body in the individual image sample using a human body detection algorithm, and generating a first human body image

[0060] 200, performing image reconstruction on the first human body image to generate a second human body image; 300, performing human body feature value extraction on the generated second human body image to generate a human body feature map;

[0061] 400, performing human head detection on the first human image to obtain a first human head image, and performing 3D facial image reconstruction on the first human head image to generate a second human head image;

[0062] 500, extracting facial feature values ​​based on the generated second human head image to generate a facial feature map;

[0063] 600 , performing feature splicing on the face feature map and the body feature map to form a face-body feature map, and performing comparative identification on the person to be identified based on the face-body feature map.

[0064] It should be noted that the deep learning target detection algorithm is used in step 100.

[0065] Human body images can be obtained by inputting image data into a data-trained deep learning target detection algorithm (such as Faster RCNN, YoloV3, CornerNet). For example, in an embodiment of the present invention, the reconstructed image is selected as input, and a human body detection frame can be obtained through the FasterRCNN algorithm, and a human body image can be obtained based on the human body detection frame.

[0066] In addition, the face image and the body image are two different features, but they are correlated. In order to better learn the distinguishing features, the human head image and the body image are respectively input into neural networks of different architectures for feature extraction. In this embodiment, the Googlenet neural network is used to extract features from the face image to obtain a face feature map; the residual network is used to extract features from the body image to obtain a body feature map, and the face feature map and the body feature map are feature spliced ​​to obtain a face-body feature image.

[0067] It should be understood that neural networks with different architectures are not limited to the above two networks, and can also be SqueezeNet networks or Mobilent networks, both of which can perform feature extraction on facial and / or body feature images, and there is no special limitation in the embodiments of the present invention.

[0068] Specifically, the step of reconstructing the first human body image in step 100 specifically includes:

[0069] The first human body image is reconstructed using a pre-generated super-resolution reconstruction network model to obtain a second human body image.

[0070] Specifically, the method for generating the super-resolution reconstruction network model includes:

[0071] Set training parameters and content loss function to build the initial model;

[0072] Collect a training sample set consisting of individual image samples and their corresponding low-resolution images. Downsample the individual image samples to obtain low-resolution images. The downsampling method involves downsampling each image by a factor of 3, reducing its length and width to 1 / 3 of its original size. If the original image is H and the downsampled image is L, then L and H constitute a valid image pair for subsequent model training.

[0073] A low-resolution image is input into the initial model, and the resolution of the low-resolution image is enhanced by the initial model to obtain an image SR. The image SR is compared with the individual image sample to obtain a difference value. The training parameters are adjusted using the difference value until the content loss function is reduced to a preset minimum threshold, thereby obtaining a super-resolution reconstruction network model.

[0074] Specifically, the method for generating the first human head image in step 400 includes the following steps:

[0075] Sending the first human body image to a pre-generated human head detection model;

[0076] Obtaining human head frame information based on the pre-generated human head detection model;

[0077] The first human body image is cut according to the human head frame to obtain a first human head image.

[0078] The method for generating the human head detection model includes:

[0079] Select the human head frame information to construct the original training set;

[0080] Randomly magnify the area within the human head frame to construct an augmented training set;

[0081] Input the augmented training set into the original human head detection model to obtain the predicted human head frame information;

[0082] The predicted human head frame information is combined with the human head frame information in the original training set to calculate the total loss function, train the weights, and obtain the human head detection model.

[0083] Specifically, the method for reconstructing a 3D face image in step 400 includes:

[0084] Acquire a first human head image from the first human image;

[0085] The first human head image is reconstructed using a pre-generated 3D human face model to obtain a second human head image.

[0086] The 3D face model includes: identity consistency learning part and scene optimization part;

[0087] The identity consistency learning component uses an unsupervised deep learning method to decompose the input head image into a standard depth map and a standard color map. The renderer then uses the resulting image to construct a reconstruction loss, ensuring that the reconstructed image output by the generator is similar to the input image, ensuring pixel-by-pixel reconstruction accuracy. The identity consistency learning component simultaneously feeds multiple photos of the same person into the 3D face model, allowing the 3D face model to calculate a depth map with reflectivity that matches the person's characteristics. Furthermore, the 3D face model learns weights to capture facial regions that best reflect individual features.

[0088] The goal of scene optimization is to restore the true expression in the target scene. An encoder is used to extract realistic texture and depth information from the target image, which is then integrated into the "standard face" for 3D face reconstruction. Because not all areas of the face require significant modification during scene optimization, such as the forehead, which may only require slight color adjustments based on lighting, the attention mechanism allows the model to better preserve features that do not require significant modification.

[0089] The embodiment of the present invention also discloses a face tracking and recognition system. Figure 2, a face tracking and recognition system includes:

[0090] Image acquisition module 1, which uses a large number of real high-speed surveillance cameras to acquire video data streams in complex scenes, thereby obtaining individual image samples;

[0091] The human body positioning module 2 is used to perform preliminary human body positioning processing on the individual image sample to obtain a first human body image of the candidate human body area;

[0092] An image reconstruction module 3 is used to reconstruct the first human body image to obtain a second human body image with higher definition;

[0093] The human body feature extraction module 4 is used to extract human body feature values ​​from the generated human body image and generate a human body feature map;

[0094] A human head positioning generation module 5 is used to perform human head detection on the first human image, select human head frame information, and cut the human image according to the human head frame information to obtain the first human head image;

[0095] 3D face image reconstruction module 6, used for performing 3D face image reconstruction on the first human head image to generate a face feature map;

[0096] A facial feature extraction module 7 is used to extract facial feature values ​​from the second human head image and generate a facial feature map;

[0097] A feature splicing module 8 is used to perform feature splicing on the face feature map and the body feature map to form a face-body feature map;

[0098] The comparison, recognition and tracking module 9 compares the generated face-body feature map with the features of the person to be tracked to determine whether they are the same person, thereby achieving tracking and recognition.

[0099] By adopting the above technical solution, a video sequence in a complex scene is obtained through the image acquisition module 1, a first human image and a first human head image are detected from the video sequence, and then the image reconstruction module 3 is used to reconstruct the image to obtain a second human image with higher clarity, which is more conducive to obtaining human features. 3D face reconstruction is performed on the obtained first human head image, facial features are extracted, and the face recognition rate is enhanced. Finally, the face and body features are spliced ​​through the feature splicing module 8, and pedestrians are identified based on the spliced ​​features, thereby improving recognition efficiency, saving resources, and ensuring the correspondence between the face and the body, solving the problem of poor recognition and tracking results, improving practicality, and effectively solving the influence of many factors such as large scenes, low pixels of imaging equipment, long distance, and environment, thereby realizing face tracking and recognition in complex and large scenes, and the obtained facial image is clear and has high resolution.

[0100] An embodiment of the present invention also discloses a computer-readable storage medium storing a computer program that can be loaded by a processor and execute a face tracking and recognition method as described above. The computer-readable storage medium includes, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0101] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A face tracking and recognition method, characterized by: The method comprises the following steps: Receiving a video sequence containing an identified person from surveillance acquisition; Detecting individual image samples according to the video sequence, locating a human body in the individual image samples using a human body detection algorithm, and generating a first human body image; Reconstructing the first human body image using a pre-generated super-resolution reconstruction network model to obtain a second human body image; Extracting human body feature values ​​from the generated second human body image to generate a human body feature map; Performing human head detection on the first human head image to obtain a first human head image, and performing 3D facial image reconstruction on the first human head image to generate a second human head image; extracting facial feature values ​​from the second human head image to generate a facial feature map; Performing feature splicing on the face feature map and the body feature map to form a face-body feature map, and performing comparative identification on the person to be identified based on the face-body feature map; The method for generating the super-resolution reconstruction network model includes: Set training parameters and content loss function to build the initial model; Collecting a training sample set, wherein the training sample set includes individual image samples and corresponding low-resolution images; Inputting a low-resolution image into the initial model, increasing the resolution of the low-resolution image through the initial model to obtain an image SR, comparing the image SR with the individual image sample to obtain a difference value, and using the difference value to adjust the training parameters until the content loss function is reduced to a preset minimum threshold, thereby obtaining a super-resolution reconstruction network model; The method for generating the first human head image comprises the following steps: Sending the first human body image to a pre-generated human head detection model; Obtaining human head frame information based on the pre-generated human head detection model; Cutting the first human body image according to the human head frame information to obtain a first human head image; The method for generating the human head detection model comprises: Select the human head frame information to construct the original training set; Randomly magnify the area within the human head frame to construct an augmented training set; Input the augmented training set into the original human head detection model to obtain the predicted human head frame information; Combine the predicted human head frame information with the human head frame information in the original training set to calculate the total loss function, train the total loss function, and obtain the human head detection model; The method for reconstructing a 3D face image comprises: Acquire a first human head image from the first human image; Reconstructing the first human head image using a pre-generated 3D face model to obtain a second human head image; The 3D face model includes: an identity consistency learning part and a scene optimization part; The identity consistency learning part adopts an unsupervised deep learning method to decompose the input first human head image into a standard depth map and a standard original color map, and then constructs the reconstruction loss through the rendering result of the renderer; The scene optimization part uses an encoder to extract real texture and depth information from the target image, and then integrates it into the standard face for 3D face reconstruction.

2. The face tracking and recognition method according to claim 1, wherein: The human detection algorithm is a deep learning target detection algorithm.

3. A face tracking and recognition system, executing the face tracking and recognition method according to claim 1, characterized in that: include: An image acquisition module (1) is used to acquire a video sequence of a person to be identified in a complex scene, thereby obtaining individual image samples; A human body positioning module (2) is used to perform preliminary human body positioning processing on individual image samples to obtain a first human body image of a candidate human body region; An image reconstruction module (3) is used to reconstruct the first human body image to obtain a second human body image with higher definition; A human feature extraction module (4) is used to extract human feature values ​​from the generated human image and generate a human feature map; A human head positioning generation module (5) is used to detect a human head from a first human image, select human head frame information, and cut the human image according to the human head frame information to obtain a first human head image; A 3D face image reconstruction module (6) is used to perform 3D face image reconstruction on the first human head image to obtain a second human head image; A facial feature extraction module (7) is used to extract facial feature values ​​from the second human head image and generate a facial feature map; A feature splicing module (8) is used to perform feature splicing on the face feature map and the body feature map to form a face-body feature map; The comparison, recognition and tracking module (9) compares the generated face-body feature map with the features of the person to be tracked to determine whether they are the same person, thereby achieving tracking and recognition.

4. A computer-readable storage medium, characterized in that: The computer program is stored and can be loaded by a processor to execute the method according to any one of claims 1 to 2.

Images