Dynamic face recognition method and system based on deep learning
By establishing a three-dimensional face database and deformation model based on deep learning, reconstructing three-dimensional faces and defuzzing, the fuzzing problem in dynamic face recognition is solved, and the fast and accurate recognition effect is achieved, meeting the real-time requirements.
Patent Information
- Application Number
- CN202411035135.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the dynamic face recognition, the camera is not focused or the human body is moving in multiple angles, making it difficult to realize real-time recognition. Although the deblurgation effect of DeblurGAN_V2 is good, the speed is insufficient, and it cannot meet the real-time requirements of face recognition. .
Using deep learning-based methods, a three-dimensional face database and deformation model are established, and the shape and texture coefficients are predicted through cascade residual networks, and the three-dimensional face is reconstructed, and the face detection and de-ambiguity model evaluation is used to use the OTSU algorithm to perform face detection and de-ambiguity model evaluation. Add defuzzing to identify blurred dynamic faces.
It realizes fast and accurate dynamic face recognition, meets the real-time requirements of face recognition, accelerates the recognition speed and training speed, and improves the accuracy and efficiency of recognition.
Smart Images

Figure CN118982854A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of face recognition, and in particular to a dynamic face recognition method and system based on deep learning. Background Art
[0002] With the rapid development of artificial intelligence, face recognition, as an important research direction, has been widely used in many scenarios. In some scenarios, due to the problem of face recognition angle, the comparison face image corresponding to the input target face image cannot be obtained in the database, and thus recognition is impossible. At present, a single face image can establish three-dimensional face details based on the depth of the picture, but the face depth has strict requirements on lighting conditions;
[0003] In dynamic face recognition, the captured face is often blurred due to inaccurate camera focus or multi-angle movement of the human body. Pedestrians are often in motion and all people are at different heights and distances. In the existing technology, DeblurGAN_V2 has a good deblurring effect on the overall image, but it is difficult to find the Nash equilibrium point, resulting in its speed still not meeting the real-time requirements of face recognition. The key to face recognition lies in whether the key facial feature points can be detected. Therefore, it is necessary to find a new method to intercept the generative adversarial network. At this time, although the generator has not reached the optimal state, the deblurring algorithm has met the requirements of face recognition, and face recognition can be performed. Therefore, it is necessary to selectively add deblurring processing operations. Summary of the invention
[0004] The purpose of the present invention is to provide a dynamic face recognition method and system based on deep learning to solve the problems raised in the above background technology.
[0005] To achieve the above object, the present invention provides the following technical solution: a dynamic face recognition method based on deep learning, comprising the following steps:
[0006] S1. Establishing a 3D face database: A large number of 3D face data sets are stored in the 3D face database. A face picture is input, and face analysis is performed on the picture. Then, several faces similar to the face are searched in the 3D face database by exhaustive method, and weights of shape coefficient and texture coefficient are assigned to each face. The sum of the weights of shape coefficient and texture coefficient is 1 respectively.
[0007] S2. Establish a deformation model: input the 3D face database into the deformation model, predict the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalize the 3D face database in the deformation model, and then uniformly represent the 3D face in vector form, and output the shape vector and texture vector;
[0008] S3, reconstruction of 3D face: extract facial feature points in the target face image, obtain facial deflection angles based on the facial feature points, reconstruct the frontal facial feature points, and construct a 3D face shape based on the frontal facial feature points. After that, a face gray model is constructed by the shape coefficient, and then the face gray model is colored and textured by the texture coefficient to finally complete the 3D face reconstruction;
[0009] S4, self-feedback of deformation model: calculate the face similarity, and perform self-feedback on the deformation model based on the face similarity comparison result;
[0010] S5. Face detection: perform binary segmentation on the face images in the 3D face database to obtain a face grayscale image. The value of each pixel in the face grayscale image is 0 or 1. The OTSU algorithm is used to distinguish skin color areas from non-skin color areas.
[0011] S6. Identify blurred dynamic faces: Input an unprocessed blurred face image, then use the generator of the DeblurGAN_V2 network with a changed training cutoff condition to deblur the image, use the face blur model to evaluate the rate of change of face blur in the output image, and then selectively add deblurring operations to identify blurred dynamic faces.
[0012] The normalization formula of the 3D face database is:
[0013]
[0014]
[0015] Among them, C i represents the i-th decentered face, S i represents the shape vector of the i-th face in the 3D face database, represents the average face shape in the 3D face database, C represents the normalized 3D face in the 3D face database, Indicates the number of faces in the 3D face database;
[0016] Preferably, step S3 comprises the following steps:
[0017] S301, taking the input face picture as the target face picture, extracting face feature points from the target face picture, and distinguishing three different scenes according to the number of detected face feature points;
[0018] S302, using a face deflection angle algorithm to calculate the face deflection angle, and then combining it with an ASM algorithm to reconstruct the front face feature points, and inputting a shape vector based on the front face feature points to construct a three-dimensional face shape;
[0019] S303, then input the texture vector to perform texture coloring, reconstruct a whole three-dimensional face, and then rotate the three-dimensional face according to the face deflection angle obtained previously;
[0020] S304: perform face similarity comparison between the three-dimensional face and the target face image initially input, so as to realize multi-angle face recognition.
[0021] Preferably, in step S301, the three different scenarios include:
[0022] Scenario 1: When the face deflection angle is small and the number of feature points does not change significantly, the face deflection angle is calculated using the coordinates of the center of the two eyes and the center of the mouth.
[0023] Scenario 2: When the face deflection angle is large and the number of feature points fluctuates greatly, the face deflection angle is calculated using the translation vector;
[0024] Scenario 3: The face has the largest deflection angle and the least number of feature points, making it difficult to identify. The posture of the face is estimated based on the 3D deflection angle, and then the deflection angle of the face is calculated.
[0025] Preferably, step S4 comprises the following steps:
[0026] S401, when the face similarity is greater than or equal to 85%, it indicates that the deep learning prediction result is accurate and there is no need to perform self-feedback on the deformation model;
[0027] S402, when the face similarity is less than or equal to 60%, it means that the face 3D reconstruction result is seriously inconsistent with the input image, and the number of face images in the 3D face database is expanded;
[0028] S403: When the face similarity is between 60% and 85%, it indicates that the deep learning prediction result is inaccurate and the deformation model needs to be adjusted by self-feedback.
[0029] Preferably, in step S5, the OTSU algorithm implementation method includes the following steps:
[0030] S501, using skin color detection to screen out skin color areas, and then identifying face areas in the skin color areas;
[0031] S502, the image grayscale is divided into 256 levels, and a picture is segmented by binarization to divide the picture into two parts, one part is the pixel points with pixels greater than a threshold, and the other part is the pixel points with pixels less than or equal to the threshold;
[0032] S503, respectively calculating the probabilities of the two parts of the pixel points of an image in step S502, and then respectively calculating the average grayscale values of the two parts and the overall average grayscale value of the image, so as to maximize the inter-class variance and determine the threshold.
[0033] Preferably, step S6 comprises the following steps:
[0034] S601, the face blur change rate is represented as CJ, |·| represents the modulus, when 0≤|CJ|<2%, it means that the face blur change rate is too low, and the image generated by the generator is too stable at this time, and directly enters the face recognition;
[0035] S602, if |CJ|>30%, it means that the face blur change rate is too high, the generator training fails, and face recognition is performed directly;
[0036] S603. When 2%≤|CJ|≤30%, the generator state is stable and the result is controlled in accordance with the balance point range, post-processing operations are added, and then face recognition is performed.
[0037] Preferably, in step S6, the face blur model inputs a blurred face image, then calculates the face blur quality value, calculates the face blur quality value through gradient, extracts edges using the image gradient method, and then processes the face image in combination with spatial complexity masking and brightness masking, and then uses wavelet transform for secondary edge detection to extract the perceived edge map, and records the serial number for each face image, and finally calculates the perceived blur quality factor and face blur, and evaluates the face blur change rate of the output image.
[0038] A dynamic face recognition system based on deep learning, comprising a database building unit, a deformation model building unit, a face reconstruction unit, a deformation model self-feedback unit, a face detection unit and a fuzzy face recognition model unit;
[0039] The database establishment unit first establishes a three-dimensional face database, which stores a large number of three-dimensional face data sets, inputs a face picture, performs face analysis on the picture, and then searches for several faces similar to the face in the three-dimensional face database by exhaustive method, and assigns weights of shape coefficient and texture coefficient to each face, and the sum of the weights of shape coefficient and texture coefficient is 1 respectively;
[0040] The deformation model building unit inputs the three-dimensional face database into the deformation model, predicts the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalizes the three-dimensional face database in the deformation model, then uniformly represents the three-dimensional face in vector form, and outputs the shape vector and texture vector;
[0041] The face reconstruction unit is used to extract the face feature points in the target face image, obtain the face deflection angle according to the face feature points, reconstruct the front face feature points, and construct a three-dimensional face shape based on the front face feature points. After that, a face gray model is constructed by the shape coefficient, and then the face gray model is colored and textured by the texture coefficient, and finally the three-dimensional face reconstruction is completed;
[0042] The deformation model self-feedback unit is used to calculate the face similarity and perform self-feedback on the deformation model according to the face similarity comparison result;
[0043] The face detection unit performs binary segmentation on the face image in the three-dimensional face database to obtain a face grayscale image, in which the value of each pixel point is 0 or 1, and uses the OTSU algorithm to distinguish skin color areas from non-skin color areas;
[0044] The fuzzy face recognition model unit is used to input an unprocessed fuzzy face image, and then use the generator of the DeblurGAN_V2 network with a changed training cutoff condition to deblur the image, use the face fuzziness model to evaluate the face fuzziness change rate of the output image, and then selectively add a deblurring processing operation to identify fuzzy dynamic faces.
[0045] Compared with the prior art, the present invention has the following beneficial effects:
[0046] 1. The present invention extracts facial feature points based on the input target face picture and reconstructs the frontal face feature points in combination with the ASM algorithm, uses a cascaded residual network to predict the shape coefficient and texture coefficient in the deformation model, establishes a face gray model after predicting the shape coefficient, and adjusts it with the reconstructed face feature points as the main reference standard to construct a complete three-dimensional face model, rotates the three-dimensional face according to the face deflection angle, and compares the similarity with the input target face picture, and then performs self-feedback on the deformation model according to the result calculated by the face similarity calculation formula, which is also a verification of the accuracy of the predicted shape coefficient and texture coefficient;
[0047] 2. The present invention uses the OTSU algorithm to distinguish skin color areas and non-skin color areas, which can achieve the purpose of accelerating face detection. The face blur model is used to evaluate the face blur change rate of the output image, and then a deblurring operation is selectively added to identify fuzzy dynamic faces, so that the recognition speed meets the real-time requirements of face recognition, and the training speed and running speed of dynamic face recognition are accelerated. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A flow chart of the overall method provided by an embodiment of the present invention;
[0049] Figure 2A flow chart of a method for reconstructing a three-dimensional face provided by an embodiment of the present invention;
[0050] Figure 3 A flow chart of a method for self-feedback of a deformation model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] See also Figure 1-3 The present invention provides a technical solution: a dynamic face recognition method based on deep learning, comprising the following steps:
[0053] S1. Establishing a 3D face database: A large number of 3D face data sets are stored in the 3D face database. A face picture is input, and face analysis is performed on the picture. Then, several faces similar to the face are searched in the 3D face database by exhaustive method, and weights of shape coefficient and texture coefficient are assigned to each face. The sum of the weights of shape coefficient and texture coefficient is 1 respectively.
[0054] S2. Establish a deformation model: input the 3D face database into the deformation model, predict the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalize the 3D face database in the deformation model, and then uniformly represent the 3D face in vector form, and output the shape vector and texture vector;
[0055] The cascaded residual network is evolved from the residual network. The cascaded residual block is used to connect the global convolution kernel and the local convolution kernel. Through cascading, the features learned by the low-level convolution kernel can be quickly propagated to the high-level convolution kernel. The cascaded residual block performs local cascading, and the corresponding cascaded residual network performs global cascading.
[0056] S3, reconstruction of 3D face: extract facial feature points in the target face image, obtain facial deflection angles based on the facial feature points, reconstruct the frontal facial feature points, and construct a 3D face shape based on the frontal facial feature points. After that, a face gray model is constructed by the shape coefficient, and then the face gray model is colored and textured by the texture coefficient to finally complete the 3D face reconstruction;
[0057] S4, self-feedback of deformation model: calculate the face similarity, and perform self-feedback on the deformation model based on the face similarity comparison result;
[0058] S5. Face detection: perform binary segmentation on the face images in the 3D face database to obtain a face grayscale image. The value of each pixel in the face grayscale image is 0 or 1. The OTSU algorithm is used to distinguish skin color areas from non-skin color areas.
[0059] S6. Identify blurred dynamic faces: Input an unprocessed blurred face image, then use the generator of the DeblurGAN_V2 network with a changed training cutoff condition to deblur the image, use the face blur model to evaluate the rate of change of face blur in the output image, and then selectively add deblurring operations to identify blurred dynamic faces.
[0060] The normalization formula of the 3D face database is:
[0061]
[0062]
[0063] Among them, C i represents the i-th decentered face, S i represents the shape vector of the i-th face in the 3D face database, represents the average face shape in the 3D face database, C represents the normalized 3D face in the 3D face database, Indicates the number of faces in the 3D face database;
[0064] Step S3 includes the following steps:
[0065] S301, taking the input face picture as the target face picture, extracting face feature points from the target face picture, and distinguishing three different scenes according to the number of detected face feature points;
[0066] S302, using a face deflection angle algorithm to calculate the face deflection angle, and then combining it with an ASM algorithm to reconstruct the front face feature points, and inputting a shape vector based on the front face feature points to construct a three-dimensional face shape;
[0067] The ASM algorithm is a technology widely used in the field of face recognition and image processing. The basic principle of the ASM algorithm is to establish a shape model through a training set. This model can describe the basic shape and features of the face. In the application stage, the algorithm will look for shapes similar to those in the training set on the input face image to locate the key feature points of the face.
[0068] S303, then input the texture vector to perform texture coloring, reconstruct a whole three-dimensional face, and then rotate the three-dimensional face according to the face deflection angle obtained previously;
[0069] S304, performing face similarity comparison between the three-dimensional face and the target face image initially input, so as to realize multi-angle face recognition;
[0070] The formula for 3D face reconstruction is:
[0071]
[0072]
[0073] Among them, S m Represents the gray model of the face composed of shape vectors, T m Represents the facial details composed of texture vectors, a i and b i Respectively represent the shape coefficient and texture coefficient of the i-th face, represents the number of faces in the 3D face database, S P represents the i-th shape vector principal component. The face shape vector principal component is T P Represents the i-th texture vector principal component. The principal component of the face texture vector is represents the average face shape in the 3D face database, Represents the average face texture in a 3D face database;
[0074] In step S301, three different scenarios include:
[0075] Scenario 1: When the face deflection angle is small and the number of feature points does not change significantly, the face deflection angle is calculated using the coordinates of the center of the two eyes and the center of the mouth. The calculation formula is as follows:
[0076]
[0077] in, represents the face deflection angle, D represents the distance between the two eyes after rotation, D1 represents the distance from the mouth to the center of the two eyes after rotation, and arccos(·) represents the inverse cosine function;
[0078] Scenario 2: When the face deflection angle is large and the number of feature points fluctuates greatly, the face deflection angle is calculated using the translation vector. The calculation formula is as follows:
[0079]
[0080]
[0081] Where R represents the rotation matrix, represents the face deflection angle, A represents a constant, T x 、T y 、T zRespectively represent the translation vectors along the X-axis, Y-axis, and Z-axis, tan(·) represents the tangent function, and R 11 , R 12 , R 13 , R 21 , R 22 , R 23 , R 31 , R 32 , R 33 Respectively represent the elements of the rotation matrix;
[0082] Scenario 3: The face has the largest deflection angle and the least number of feature points, making it difficult to identify. The posture of the face is estimated based on the 3D deflection angle, and then the face deflection angle is calculated. The calculation steps are as follows:
[0083] Step 1: Mark the feature points of the face image and calculate the coordinate value of each feature point;
[0084] Step 2: Calculate the identity matrix;
[0085]
[0086]
[0087] Where H represents the unit matrix, (X, Y, Z) represents the vertex coordinates of the three-dimensional face model, (u, v) represents the coordinates of the feature points of the two-dimensional image, and t represents a constant;
[0088] Step 3: Estimate camera parameters;
[0089]
[0090] Among them, fu represents the camera parameters;
[0091] Step 4: Estimate face pose;
[0092]
[0093]
[0094] in, represents the face deflection angle, α, β, γ represent the rotation angles along the X-axis, Y-axis, and Z-axis respectively, fu represents the camera parameter, θ1, θ2, and θ3 represent numerical values respectively, sin(·) represents the sine function, cos(·) represents the cosine function, and tan(·) represents the tangent function;
[0095] In order to verify whether the shape coefficients and texture coefficients predicted by the cascaded residual network are accurate, the deformation model performs self-feedback, which can improve the accuracy of deep learning prediction results;
[0096] Step S4 includes the following steps:
[0097] S401, when the face similarity is greater than or equal to 85%, it indicates that the deep learning prediction result is accurate and there is no need to perform self-feedback on the deformation model;
[0098] S402, when the face similarity is less than or equal to 60%, it means that the face 3D reconstruction result is seriously inconsistent with the input image, and the number of face images in the 3D face database is expanded;
[0099] S403, when the face similarity is between 60% and 85%, it indicates that the deep learning prediction result is inaccurate and the deformation model needs to be adjusted by self-feedback;
[0100] The formula for calculating face similarity is:
[0101]
[0102] Where SIM represents face similarity, a represents brightness similarity, b represents contrast similarity, c represents structure similarity, max{·} represents maximum value, and min{·} represents minimum value;
[0103] In step S5, the OTSU algorithm implementation method includes the following steps:
[0104] S501, using skin color detection to screen out skin color areas, and then identifying face areas in the skin color areas;
[0105] S502, the image grayscale is divided into 256 levels, and a picture is segmented by binarization to divide the picture into two parts, one part is the pixel points with pixels greater than a threshold, and the other part is the pixel points with pixels less than or equal to the threshold;
[0106] S503, respectively calculating the probabilities of the two parts of the pixel points of a picture in step S502, and then respectively calculating the average grayscale values of the two parts and the overall average grayscale value of the image, so as to maximize the inter-class variance and determine the threshold;
[0107] The OTSU algorithm is an algorithm for determining the threshold value of image binary segmentation, also known as the maximum inter-class variance method. This method mainly uses the threshold value to segment the foreground and background. The method for determining the optimal threshold value is to maximize the inter-class variance. It divides the image into background and foreground according to the grayscale characteristics of the image. The segmentation with the maximum inter-class variance means the minimum probability of misclassification.
[0108] Step S6 includes the following steps:
[0109] S601, the face blur change rate is represented as CJ, |·| represents the modulus, when 0≤|CJ|<2%, it means that the face blur change rate is too low, and the image generated by the generator is too stable at this time, and directly enters the face recognition;
[0110] S602, if |CJ|>30%, it means that the face blur change rate is too high, the generator training fails, and face recognition is performed directly;
[0111] S603, when 2%≤|CJ|≤30%, the generator state is stable and the result control is in line with the balance point range, post-processing operations are added, and then face recognition is performed;
[0112] In step S6, the face blur model inputs a blurred face image, then calculates the face blur quality value, calculates the face blur quality value through gradient, extracts the edge by image gradient method, processes the face image in combination with spatial complexity masking and brightness masking, and then uses wavelet transform to perform secondary edge detection, extracts the perceived edge map, and records the serial number for each face image, finally calculates the perceived blur quality factor and face blur, and evaluates the face blur change rate of the output image;
[0113] The face fuzziness model formula is:
[0114]
[0115]
[0116]
[0117] Among them, PQ j represents the blur quality factor at resolution level j, L, W represent the length and width of the image respectively, (x, y) represents the pixel, PNB j Indicates the blur state of the pixel (x, y) at resolution j, PNE j Indicates the clarity of the pixel (x, y) at resolution j, J represents the upper limit of the resolution level, j represents the resolution, CI i represents the blurriness of the face with image number i, CJ represents the rate of change of face blurriness, CI u Indicates the blurriness of the face in the image with sequence number u, CI u+1 Indicates the blurriness of the face with image number u+1. represents the end image number of the face blur change rate calculation, λ represents the initial image number of the face blur change rate calculation, ξ represents the perceptible edge threshold constant, and its value is 0.63;
[0118] A dynamic face recognition system based on deep learning, comprising a database building unit 1, a deformation model building unit 2, a face reconstruction unit 3, a deformation model self-feedback unit 4, a face detection unit 5 and a fuzzy face recognition model unit 6;
[0119] The database establishment unit 1 first establishes a three-dimensional face database, which stores a large number of three-dimensional face data sets, inputs a face picture, performs face analysis on the picture, and then searches for several faces similar to the face in the three-dimensional face database by exhaustive method, and assigns weights of shape coefficient and texture coefficient to each face, and the sum of the weights of shape coefficient and texture coefficient is 1 respectively;
[0120] The deformation model building unit 2 inputs the three-dimensional face database into the deformation model, predicts the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalizes the three-dimensional face database in the deformation model, and then uniformly represents the three-dimensional face in vector form, and outputs the shape vector and texture vector;
[0121] The face reconstruction unit 3 is used to extract the face feature points in the target face image, obtain the face deflection angle according to the face feature points, reconstruct the front face feature points, and construct a three-dimensional face shape based on the front face feature points. After that, the face gray model is constructed by the shape coefficient, and the face gray model is colored and fine-tuned by the texture coefficient, and finally the three-dimensional face reconstruction is completed;
[0122] The deformation model self-feedback unit 4 is used to calculate the face similarity and perform self-feedback on the deformation model according to the face similarity comparison result;
[0123] The face detection unit 5 performs binary segmentation on the face image in the three-dimensional face database to obtain a face grayscale image. The value of each pixel in the face grayscale image is 0 or 1, and the skin color area and the non-skin color area are distinguished by using the OTSU algorithm;
[0124] The fuzzy face recognition model unit 6 is used to input an unprocessed fuzzy face image, and then use the generator of the DeblurGAN_V2 network with a changed training cutoff condition to deblur the image, use the face fuzziness model to evaluate the face fuzziness change rate of the output image, and then selectively add a deblurring processing operation to identify fuzzy dynamic faces.
[0125] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0126] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A dynamic face recognition method based on deep learning, characterized in that The following steps are involved: S1. Establishing a 3D face database: A large number of 3D face data sets are stored in the 3D face database. A face picture is input, and face analysis is performed on the picture. Then, multiple faces similar to the face are searched in the 3D face database by exhaustive method, and weights of shape coefficient and texture coefficient are assigned to each face. The sum of the weights of shape coefficient and texture coefficient is 1 respectively. S2. Establish a deformation model: input the 3D face database into the deformation model, predict the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalize the 3D face database in the deformation model, and then uniformly represent the 3D face in vector form, and output the shape vector and texture vector; S3, reconstruction of 3D face: extract facial feature points in the target face image, obtain facial deflection angles based on the facial feature points, reconstruct the frontal facial feature points, and construct a 3D face shape based on the frontal facial feature points. After that, a face gray model is constructed by the shape coefficient, and then the face gray model is colored and textured by the texture coefficient to finally complete the 3D face reconstruction; S4, self-feedback of deformation model: calculate the face similarity, and perform self-feedback on the deformation model based on the face similarity comparison result; S5. Face detection: Binarize the face images in the 3D face database to obtain the face grayscale image, and use the OTSU algorithm to distinguish the skin color area from the non-skin color area; S6. Identify blurred dynamic faces: Input an unprocessed blurred face image, then use the generator of the DeblurGAN_V2 network with a changed training cutoff condition to deblur the image, use the face blur model to evaluate the rate of change of face blur in the output image, and then selectively add deblurring operations to identify blurred dynamic faces.
2. The method for dynamic face recognition based on deep learning according to claim 1, characterized in that: The step S3 comprises the following steps: S301, taking the input face picture as the target face picture, extracting face feature points from the target face picture, and distinguishing three different scenes according to the number of detected face feature points; S302, using a face deflection angle algorithm to calculate the face deflection angle, and then combining it with the ASM algorithm to reconstruct the front face feature points, and inputting a shape vector based on the front face feature points, thereby constructing a three-dimensional face shape; S303, then input the texture vector to perform texture coloring, reconstruct a whole three-dimensional face, and then rotate the three-dimensional face according to the face deflection angle obtained previously; S304: perform face similarity comparison between the three-dimensional face and the target face image initially input, so as to realize multi-angle face recognition.
3. The method for dynamic face recognition based on deep learning according to claim 2, characterized in that: In step S301, the three different scenarios include: Scenario 1: When the face deflection angle is small and the number of feature points does not change significantly, the face deflection angle is calculated using the coordinates of the center of the two eyes and the center of the mouth. Scenario 2: When the face deflection angle is large and the number of feature points fluctuates greatly, the face deflection angle is calculated using the translation vector; Scenario 3: The face has the largest deflection angle and the least number of feature points, making it difficult to identify. The posture of the face is estimated based on the 3D deflection angle, and then the deflection angle of the face is calculated.
4. The method for dynamic face recognition based on deep learning according to claim 1, characterized in that: The step S4 comprises the following steps: S401, when the face similarity is greater than or equal to 85%, it indicates that the deep learning prediction result is accurate and there is no need to perform self-feedback on the deformation model; S402, when the face similarity is less than or equal to 60%, it means that the face 3D reconstruction result is seriously inconsistent with the input image, and the number of face images in the 3D face database is expanded; S403: When the face similarity is between 60% and 85%, it indicates that the deep learning prediction result is inaccurate and the deformation model needs to be adjusted by self-feedback.
5. The method for dynamic face recognition based on deep learning according to claim 1, characterized in that: In step S5, the OTSU algorithm implementation method includes the following steps: S501, using skin color detection to screen out skin color areas, and then identifying face areas in the skin color areas; S502, the image grayscale is divided into 256 levels, and a picture is segmented by binarization to divide the picture into two parts, one part is the pixel points with pixels greater than a threshold, and the other part is the pixel points with pixels less than or equal to the threshold; S503, respectively calculating the probabilities of the two parts of the pixel points of an image in step S502, and then respectively calculating the average grayscale values of the two parts and the overall average grayscale value of the image, so as to maximize the inter-class variance and determine the threshold.
6. The method for dynamic face recognition based on deep learning according to claim 1, characterized in that: The step S6 comprises the following steps: S601, the face blur change rate is represented as CJ, |·| represents the modulus, when 0≤|CJ|<2%, it means that the face blur change rate is too low, and the image generated by the generator is too stable at this time, and directly enters the face recognition; S602, if |CJ|>30%, it means that the face blur change rate is too high, the generator training fails, and face recognition is performed directly; S603. When 2%≤|CJ|≤30%, the generator state is stable and the result is controlled in accordance with the balance point range, a deblurring operation is added, and then face recognition is performed.
7. The method for dynamic face recognition based on deep learning according to claim 1, characterized in that: In step S6, the face blur model inputs a blurred face image, then calculates the face blur quality value, calculates the face blur quality value through gradient, extracts the edge using the image gradient method, and then processes the face image in combination with spatial complexity masking and brightness masking, and then uses wavelet transform for secondary edge detection, extracts the perceived edge map, and records the serial number for each face image, and finally calculates the perceived blur quality factor and face blur, and evaluates the face blur change rate of the output image.
8. A dynamic face recognition system based on deep learning, characterized in that: The deep learning-based dynamic face recognition system has the deep learning-based dynamic face recognition method described in any one of 1 to 7, including a database establishment unit (1), a deformation model establishment unit (2), a face reconstruction unit (3), a deformation model self-feedback unit (4), a face detection unit (5) and a fuzzy face recognition model unit (6); The database establishment unit (1) first establishes a three-dimensional face database, which stores a large number of three-dimensional face data sets, inputs a face picture, performs face analysis on the picture, and then searches for several faces similar to the face in the three-dimensional face database by exhaustive method, and assigns weights of shape coefficient and texture coefficient to each face, and the sum of the weights of shape coefficient and texture coefficient is 1 respectively; The deformation model building unit (2) inputs the three-dimensional face database into the deformation model, predicts the shape coefficient and texture coefficient in the deformation model through the cascade residual network, normalizes the three-dimensional face database in the deformation model, then uniformly represents the three-dimensional face in vector form, and outputs the shape vector and texture vector; The face reconstruction unit (3) is used to extract the face feature points in the target face image, obtain the face deflection angle according to the face feature points, reconstruct the front face feature points, and construct a three-dimensional face shape based on the front face feature points. After that, a face gray model is constructed by the shape coefficient, and then the face gray model is colored and textured by the texture coefficient, and finally the three-dimensional face reconstruction is completed; The deformation model self-feedback unit (4) is used to calculate the face similarity and to perform self-feedback on the deformation model according to the face similarity comparison result; The face detection unit (5) performs binary segmentation on the face image in the three-dimensional face database to obtain a face grayscale image, wherein the value of each pixel in the face grayscale image is 0 or 1, and the skin color area and the non-skin color area are distinguished by using the OTSU algorithm; The fuzzy face recognition model unit (6) is used to input an unprocessed fuzzy face image, then use the generator of the DeblurGAN_V2 network with a modified training cut-off condition to perform a deblurring operation on the image, use the face fuzziness model to evaluate the face fuzziness change rate of the output image, and then selectively add a deblurring operation to recognize fuzzy dynamic faces.
Citation Information
Patent Citations
Dynamic face recognition method based on novel adversarial learning deblurring theory
CN114783020A
Multi-angle face recognition method based on three-dimensional intelligent reconstruction
CN114882545A
Cited By
Method for identifying quick-moving target based on inverse color feature track
CN119904614A
