Face recognition mathematical model construction method and system, electronic equipment and storage medium

Abstract

The invention discloses a face recognition mathematical model construction method and system, an electronic device and a storage medium, and relates to the technical field of vision and pattern recognition, and the method comprises the steps: carrying out the preprocessing of a dynamic video sequence, extracting spatial-temporal features segment by segment through a three-dimensional convolution network, capturing the interaction change of the motion track of an occlusion object and the face features, and obtaining a face recognition model; generating a spatio-temporal feature tensor; inputting the spatial-temporal feature tensor into a joint modeling architecture to initialize a face recognition mathematical model, generating a primary face recognition mathematical model, analyzing an inter-frame dependency relationship through a multi-head attention mechanism, and generating a risk prediction result; and dynamically adjusting the initial face recognition mathematical model based on a risk prediction result, and generating a face recognition mathematical model through an adversarial training strategy. According to the method, the shielded face image with dynamic deformation is generated by fusing the key points of the five sense organs and the material parameters, high-fidelity data synthesis is achieved, the method is used for improving the generalization ability of the model, and the problem that shielding fitting is not natural is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual and pattern recognition, in particular to a face recognition mathematical model construction method and system, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, with the rapid development of deep learning and computer vision technology, face recognition technology has been widely applied in security monitoring, identity authentication, intelligent terminal interaction and other fields. Traditional face recognition methods mainly rely on the extraction and matching of complete face features in static images, and are usually based on principal component analysis, linear discriminant analysis or local binary pattern (LBP) for modeling. The integration of three-dimensional face reconstruction and physical simulation technology provides a new technical path for face recognition, especially in dynamic lighting, pose change and non-rigid deformation modeling.

[0003] Most occlusion perception models rely on pre-defined occlusion templates or simple image superposition methods to generate training data, and fail to fully consider the physical interaction between the occlusion and the face. In the process of modeling spatio-temporal features, existing methods often treat the motion trajectory of the occlusion as an independent variable, lack in-depth analysis of the coupling relationship between the motion of the face key points, and are difficult to capture the nonlinear deformation and light changes of the occlusion with the face action, resulting in limited identity discrimination ability of the model under dynamic occlusion. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a face recognition mathematical model construction method to solve the problem of how to construct a mathematical model with physical realism and adversarial robustness.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a face recognition mathematical model construction method, which comprises collecting face feature data and face occlusion data, and marking dynamic deformation parameters after obtaining the occlusion area range to generate a face image data set; performing real-time dynamic rendering on the face image data set, and simulating the elastic deformation and light reflection effect of the occlusion according to the motion trajectory of the face occlusion to generate a dynamic video sequence; preprocessing the dynamic video sequence, extracting spatio-temporal features by a three-dimensional convolution network, and capturing the interactive changes of the motion trajectory of the occlusion and the face features to generate a spatio-temporal feature tensor; inputting the spatio-temporal feature tensor into a joint modeling architecture to initialize a face recognition mathematical model, generating a preliminary version of the face recognition mathematical model, and analyzing the inter-frame dependency relationship through a multi-head attention mechanism to generate a risk prediction result; dynamically adjusting the preliminary version of the face recognition mathematical model based on the risk prediction result, and generating the face recognition mathematical model through an adversarial training strategy.

[0007] As a preferred scheme of the face recognition mathematical model construction method, after the face feature data and the face cover data are collected and the range of the covered area is obtained, a dynamic deformation parameter is marked to generate a face image data set, and the specific steps are as follows: the face feature data includes face detection frame coordinates and five facial feature point coordinates; the face cover data includes material characteristics and thickness data of the cover; after the range of the covered area is obtained based on the face feature data and the face cover data, the dynamic deformation parameter is marked to generate the face image data set.

[0008] As a preferred scheme of the face recognition mathematical model construction method, the face image data set is dynamically rendered in real time, and the specific steps are as follows: the motion trajectory of the face cover is collected, a rigid transformation mapping between a pixel coordinate system and a space coordinate system of the face image data set is established based on the face image data set to generate a three-dimensional scene entity; based on the three-dimensional scene entity, the face cover data and the motion trajectory of the face cover are combined to calculate the grid deformation displacement of each frame by vertex force analysis to generate a cover grid sequence.

[0009] As a preferred scheme of the face recognition mathematical model construction method, the elastic deformation and light reflection effect of the cover are simulated to generate a dynamic video sequence, and the specific steps are as follows: based on the cover grid sequence, the surface optical distribution is simulated by photon path tracking based on the material characteristics of the cover and real-time environmental light parameters to generate a light rendering frame sequence; after the light rendering frame sequence is sorted and the motion vector compensation is applied to eliminate jitter, the dynamic video sequence is obtained by time resampling.

[0010] As a preferred scheme of the face recognition mathematical model construction method, the dynamic video sequence is preprocessed, and the spatiotemporal features are extracted by a three-dimensional convolution network, and the specific steps are as follows: the dynamic video sequence is cut into video segments according to a fixed time window, and standardized processing is performed to generate a standardized video segment set; the three-dimensional convolution network architecture is used to extract features from the standardized video segment set to generate a primary spatiotemporal feature cube.

[0011] As a preferred scheme of the face recognition mathematical model construction method, the interaction between the motion trajectory of the cover and the face feature is captured to generate a spatiotemporal feature tensor, and the specific steps are as follows: based on the primary spatiotemporal feature cube, the motion trajectory feature and the face feature are processed by a double-branch attention mechanism to generate an enhanced spatiotemporal feature matrix; based on the enhanced spatiotemporal feature matrix, the spatiotemporal feature tensor is generated by feature fusion in the time dimension and the space dimension.

[0012] As a preferred scheme of the face recognition mathematical model construction method, the space-time feature tensor is input into the joint modeling architecture to initialize the face recognition mathematical model, generate a preliminary version of the face recognition mathematical model, and generate a risk prediction result by analyzing the inter-frame dependency through a multi-head attention mechanism. The specific steps are as follows: based on the space-time feature tensor, a double-branch joint modeling architecture is constructed, and a face recognition mathematical model framework is obtained through a feature fusion layer; according to the face recognition mathematical model framework, the space-time feature tensor is segmented according to the time step, and dynamic weight fusion is performed to generate a preliminary version of the face recognition mathematical model; a feature change index is calculated to obtain a motion anomaly score representing the degree of motion anomaly, and the feature change intensity of the facial feature region is calculated as a regional risk value, which is linearly combined according to the weight ratio to generate a risk prediction result.

[0013] As a preferred scheme of the face recognition mathematical model construction method, the space-time feature tensor is input into the joint modeling architecture to initialize the face recognition mathematical model, generate a preliminary version of the face recognition mathematical model, and generate a risk prediction result by analyzing the inter-frame dependency through a multi-head attention mechanism. The specific steps are as follows: based on the space-time feature tensor, a double-branch joint modeling architecture is constructed, and a face recognition mathematical model framework is obtained through a feature fusion layer; according to the face recognition mathematical model framework, the space-time feature tensor is segmented according to the time step, and dynamic weight fusion is performed to generate a preliminary version of the face recognition mathematical model; a feature change index is calculated to obtain a motion anomaly score representing the degree of motion anomaly, and the feature change intensity of the facial feature region is calculated as a regional risk value, which is linearly combined according to the weight ratio to generate a risk prediction result.

[0014] In a second aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the face recognition mathematical model construction method according to the first aspect of the present application is implemented.

[0015] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, any step of the face recognition mathematical model construction method according to the first aspect of the present application is implemented.

[0016] The present application has the following advantages: by fusing the facial feature key points and material parameters to generate a dynamic deformation occluded face image, high-fidelity data synthesis is achieved, which is used to improve the model generalization ability and solve the problem of unnatural occlusion fitting; by calculating the grid deformation frame by frame through vertex stress analysis, physically realistic dynamic occlusion simulation is achieved, which is used to construct a real training video and enhance the robustness of the model to complex deformation. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Fig. 1 The flow chart of the face recognition mathematical model construction method.

[0019] Fig. 2 The flow chart of the face feature data and occlusion data collection.

[0020] Fig. 3 The flow chart of the dynamic video sequence generation.

[0021] Fig. 4 The flow chart of the multi-modal feature optimization and model evaluation. DETAILED DESCRIPTION

[0022] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0023] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details, other than those described herein, and it is understood that the present application is not limited to the embodiments described herein and can be practiced with or without other means, structures, components, materials, acts, or functions other than those described herein. Therefore, the description herein is intended to be illustrative, but not to be limiting, of the scope of the present application.

[0024] Secondly, the "one embodiment" or "embodiment" referred to herein can include specific features, structures or characteristics contained in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0025] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides a face recognition mathematical model construction method, including the following steps: S1: Collect face feature data and face occlusion data, and after obtaining the range of the occlusion area, mark the dynamic deformation parameters to generate a face image data set; S1.1: The face feature data includes face detection frame coordinates and five facial feature key point coordinates. It should be noted that the face feature data obtains the face detection frame coordinates through face detection processing, which is specifically represented by four values of the rectangular frame upper left corner coordinates, width and height, (for example: (100, 150, 200, 250) represents that the upper left corner is located at the image pixel coordinates (100, 150), the width is 200 pixels, and the height is 250 pixels of the rectangular area); At the same time, the five feature point coordinates are obtained through the face key point positioning algorithm, including the left eye pupil coordinates, the right eye pupil coordinates, the nose tip coordinates, the left mouth corner coordinates and the right mouth corner coordinates five key point position data, (for example: left eye pupil coordinates (120, 180), right eye pupil coordinates (180, 180), nose tip coordinates (150, 220), left mouth corner coordinates (130, 250), right mouth corner coordinates (170, 250)), the final output face feature data contains two groups of structured data of face detection frame coordinates and five feature point coordinates.

[0026] S1.2: The face cover data includes the material characteristics and thickness data of the cover; It should be noted that the face cover data obtains the material characteristics of the cover through physical measuring instruments, which is specifically represented by three physical parameters of elastic modulus, friction coefficient and light transmittance, (for example: the elastic modulus of cotton material is 0.8, the friction coefficient is 0.3, and the light transmittance is 40%-60%); At the same time, the thickness data is obtained by laser thickness gauge, which is represented by millimeter level accurate value, (for example: the thickness of medical mask is 1.2 millimeter, and the thickness of blanket is 5.0 millimeter), the final output face cover data contains two groups of standardized parameters of material characteristics and thickness data of the cover.

[0027] S1.3: Based on the face feature data and the face cover data, the range of the covered area is obtained, and the dynamic deformation parameter is marked to generate the face image data set.

[0028] Further, based on the face detection frame coordinates and the five feature point coordinates in the face feature data, combined with the material characteristics and thickness data of the face cover data, the range of the covered area is obtained through geometric mapping method, which is specifically represented by the rectangular range coordinates of the mouth and nose area; According to the elastic modulus and thickness of the face cover data, the dynamic deformation parameters are marked, including deformation coefficient and recovery rate; Finally, the face image data set is generated, which contains the face image sequence labeled with covered area and dynamic deformation parameter.

[0029] S2: Real-time dynamic rendering is performed on the face image data set, and according to the motion trajectory of the face cover, the elastic deformation and light reflection effect of the cover are simulated to generate a dynamic video sequence; S2.1: The motion trajectory of the face cover is collected, the rigid transformation mapping between the face image data set pixel coordinate system and the space coordinate system is established combined with the face image data set, and the three-dimensional scene entity is generated; Furthermore, the collected motion trajectories of facial occluders include displacement direction data, velocity curve data, and acceleration data. Combining the coordinates of the face detection box and the coordinates of the key facial features in the face image dataset, a rigid transformation mapping between the pixel coordinate system and the spatial coordinate system of the face image dataset is established through the feature point correspondence method. This generates a three-dimensional scene entity, which includes a mesh sequence of occluders with motion trajectory annotations and a registered three-dimensional face model, where each vertex is associated with spatial coordinate system coordinates.

[0030] It should be noted that when establishing a rigid transformation mapping using the feature point correspondence method, firstly, fixed facial feature key points in the face image dataset are selected as the reference. Three-dimensional coordinates are assigned to the facial feature key points in the spatial coordinate system. The optimal rotation matrix and translation vector are obtained through the least squares method, so that the distance error between the key points in the pixel coordinate system and the corresponding points in the spatial coordinate system after transformation is minimized. Finally, a rigid transformation relationship between pixel coordinates and spatial coordinates is established.

[0031] S2.2: Based on the 3D scene entities, combined with the facial occlusion data and the facial occlusion motion trajectory, the mesh deformation displacement is calculated frame by frame through vertex force analysis to generate the occlusion mesh sequence; The formula for calculating the mesh deformation displacement is: ; in, This represents the amount of mesh deformation displacement. Indicates the first The elastic modulus of the material, Represents vertices initial coordinates, Key points representing the trajectory coordinates Indicates the material thickness parameter. This represents the material density parameter. Represents the time-dependent decay function. Indicate key points Radius of influence This represents the coefficient of inertia (range: 0.1-0.7). Indicates the forces exerted by the external environment. This indicates the total number of material types involved in the calculation.

[0032] It should be noted that, based on the vertex coordinate data in the 3D scene entity, combined with the material properties and thickness data of the face occlusion data, as well as the velocity curve data and acceleration data of the face occlusion motion trajectory, the force state of each vertex during the motion process is analyzed through physical simulation methods, and the mesh deformation displacement is calculated frame by frame; finally, an occlusion mesh sequence is generated, which includes a deformation mesh dataset with timestamps, and each mesh vertex records a 3D displacement vector.

[0033] The force state of each vertex during the motion process is analyzed as follows: first, the vertex mass is obtained according to the material characteristics and thickness data of the face cover data; the inertia force is obtained in combination with the acceleration data of the motion trajectory; the elastic restoring force is obtained according to the displacement and elastic modulus of the vertex; the viscous damping force is obtained according to the velocity data of the motion trajectory and the friction coefficient; the resultant force is obtained by superimposing the three forces; the vertex instantaneous acceleration is solved by Newton's second law; the displacement is updated by using the kinematic formula to complete the closed-loop analysis of the force state to the deformation displacement.

[0034] S2.3: Based on the sequence of the cover grid, combined with the material characteristics of the cover and the real-time environmental light parameters, the surface optical distribution is simulated by photon path tracking to generate a sequence of light rendering frames; Further, based on the deformation grid data in the sequence of the cover grid, combined with the material characteristic parameters of the cover and the real-time environmental light parameters, the interaction process of photons and grid surfaces is tracked by a Monte Carlo ray propagation simulation method to obtain the light energy distribution of each pixel point; and finally a sequence of light rendering frames is generated.

[0035] It should be noted that the Monte Carlo ray propagation simulation method realizes ray tracing by randomly sampling photon paths, emits photon beams from the light source, and obtains the interaction behavior of photons and grid surface vertices based on material characteristic parameters, including diffuse reflection, specular reflection and transmission; after each interaction, the propagation direction of the photon is adjusted according to the probability density function, and the position and energy of the photon reaching the imaging plane are recorded cumulatively; finally, the distribution of a large number of photons is generated by statistics to generate a surface optical distribution map; the photon energy data in the surface optical distribution map is mapped to the standard RGB color space by a colorimetric conversion method; first, the photon energy value is normalized to the interval (example value: 0-1); then, the XYZ tristimulus value is converted according to the color matching function; finally, the RGB pixel value (example value: 8 bits) is calculated through the sRGB conversion matrix to generate the final RGB value of each pixel in the sequence of light rendering frames.

[0036] S2.4: Frame sorting is performed on the sequence of light rendering frames, and after motion vector compensation is applied to eliminate jitter, a dynamic video sequence is obtained by time resampling.

[0037] Further, the light rendering frame sequence is arranged in chronological order to form a coherent frame group, the corresponding relationship between the inter-frame motion trajectories is established by analyzing the changes in the positions of the feature points, the motion vectors are smoothed by using a weighted average strategy, new frames are inserted between the existing frames according to the time proportion to complete resampling, and a dynamic video sequence is generated.

[0038] It should be noted that when establishing the inter-frame motion trajectory correspondence relationship through feature point position change analysis, first, the same five facial key points are detected in the adjacent two frames, the displacement vectors of each key point are obtained, the displacement vectors of all matched key points are median filtered, and the correspondence relationship matrix of the inter-frame motion trajectory is established. When inserting a new frame between the existing frames, first, the time ratio of the insertion position is determined, and the RGB values of the corresponding pixels of the previous and next two frames are linearly mixed according to the time ratio (for example: frame 4 pixel (100, 200) RGB (180, 200, 150) and frame 5 pixel (100, 200) RGB (190, 210, 160) are mixed according to 60% and 40% to obtain new frame pixel (100, 200) RGB (184, 204, 154)), and the same interpolation operation is performed on all pixel points to generate a complete new frame, and finally the frame rate of the dynamic video sequence is output.

[0039] S3: Preprocessing the dynamic video sequence, extracting spatio-temporal features by three-dimensional convolution network, and capturing the interactive changes of the occlusion motion trajectory and the face features, generating a spatio-temporal feature tensor; S3.1: Cutting the dynamic video sequence into video segment nodes according to a fixed time window, and performing standardization processing to generate a standardized video segment set; Specifically, the dynamic video sequence is segmented into video segment nodes according to a fixed length (example value: 16 frames), each video segment node is adjusted to a set value (example value: 224x224 pixels) and converted to YCbCr color space, the value range of the luminance component is expanded to a set interval (example value: 0-255 interval) through histogram equalization method, and a standardized video segment set is generated.

[0040] It should be noted that the histogram equalization operation first counts the pixel value distribution histogram of the luminance component, obtains the cumulative distribution function and establishes a mapping relationship table, and replaces the value of each pixel of the original image according to the mapping relationship table.

[0041] S3.2: Extracting features from the standardized video segment set through a three-dimensional convolution network architecture to generate a primary spatio-temporal feature cube; Specifically, the standardized video segment set is input into the three-dimensional convolution network architecture for processing, first, a cubic convolution kernel is used to scan the video segment for spatio-temporal features, a three-dimensional maximum pooling operation is used to compress the feature size, and a three-dimensional dilated convolution is used to capture long-range dependencies to generate a primary spatio-temporal feature cube.

[0042] It should be noted that the three-dimensional maximum pooling operation is realized by sliding a fixed-size cubic window on the input feature cube, and the maximum value in each window is extracted as the output, which preserves the spatio-temporal features while reducing the dimension of the features.

[0043] Three-dimensional dilated convolution expands the receptive field by inserting holes between the elements of the standard convolution kernel. In practice, the convolution kernel is sampled at intervals according to the hole rate (example value: 2), so that the calculation of each output position can cover a wider spatiotemporal region in the input feature map while keeping the number of parameters unchanged. This structure is particularly suitable for capturing long-range spatiotemporal dependencies across multiple frames in a video sequence.

[0044] The receptive field refers to the spatiotemporal region corresponding to each output feature point in the input data during convolution operation, which determines the size of the network's field of view for capturing features.

[0045] S3.3: Based on the primary spatiotemporal feature cube, a dual-branch attention mechanism is used to process motion trajectory features and face features respectively, generating an enhanced spatiotemporal feature matrix. Specifically, the primary spatiotemporal feature cube is input into the dual-branch attention mechanism for processing. The motion trajectory branch analyzes the displacement correlation between consecutive frames through a temporal self-attention node, and the face feature branch focuses on the key regions of the five features through a spatial attention node. The outputs of the two branches are interacted through a cross-attention layer to generate an enhanced spatiotemporal feature matrix.

[0046] It should be noted that the temporal self-attention node first divides the primary spatiotemporal feature cube into multiple segments along the time dimension, obtains the similarity scores between each pair of segments, normalizes the attention weights through the softmax function, and finally weights and sums the features of each segment according to the attention weights to realize the fusion of motion trajectory features across frames. The spatial attention node generates a two-dimensional Gaussian distribution weight map centered on each facial key point based on the facial key point coordinates, performs pixel-by-pixel multiplication operation between the two-dimensional Gaussian distribution weight map and the primary spatiotemporal feature cube, and reweights the importance of each channel feature through a (example value: 1x1) convolution kernel, finally outputting an enhanced spatial attention feature map.

[0047] S3.4: Based on the enhanced spatiotemporal feature matrix, a spatiotemporal feature tensor is generated through feature fusion in the time and space dimensions.

[0048] Specifically, based on the enhanced spatiotemporal feature matrix, a gated recurrent node is first applied along the time dimension to capture long-time dependencies, and a three-dimensional convolution is used in the spatial dimension to reorganize cross-channel features. Finally, the time features and spatial features are weighted and fused through a spatiotemporal cross-connection layer to generate a spatiotemporal feature tensor.

[0049] It should be noted that when capturing long-time dependencies by applying the gated recurrent node along the time dimension, the spatiotemporal feature matrix is input into the gated recurrent node in the order of time steps, the proportion of history memory reserved by updating the gate control is calculated, the degree of influence of the current input is reset by the gate control, the hidden state is fused with the history information and the current input, and the time feature vector output integrates the dependency relationship of all time steps.

[0050] The spatiotemporal cross-connection layer first performs an outer product operation on the time feature vector and the space feature vector to generate an interaction matrix, scores the importance of the interaction matrix through trainable weight parameters, selects the highest scoring feature combination, and finally splices the filtered spatiotemporal cross-feature and the original feature.

[0051] S4: input the spatiotemporal feature tensor into the joint modeling architecture to initialize the face recognition mathematical model, generate the initial version of the face recognition mathematical model, and analyze the inter-frame dependency relationship through the multi-head attention mechanism to generate the risk prediction result; S4.1: based on the spatiotemporal feature tensor, construct a double-branch joint modeling architecture, and obtain the face recognition mathematical model framework through the feature fusion layer; Further, based on the spatiotemporal feature tensor, a double-branch joint modeling architecture is constructed, a three-dimensional residual network is used in the spatiotemporal feature encoding branch to extract multi-level spatiotemporal features, a face key point detection node is loaded in the key point auxiliary branch to generate an attention mask, and the outputs of the two branches are multiplied element by element and convolved to generate a face recognition mathematical model framework.

[0052] It should be noted that the face key point detection node first locates the coordinates of the key points of the five organs (for example, left eye pupil (120, 180), right mouth corner (170, 250)), generates a two-dimensional Gaussian distribution heat map centered on the key points, performs a channel-by-channel multiplication operation on the heat map and the input feature map, adjusts the channel dimension through a (example value: 1x1) convolution, and finally outputs the key point attention mask.

[0053] The three-dimensional residual network extracts multi-level spatiotemporal features by stacking multiple residual blocks, each residual block includes two (example value: 3x3x3) convolution layers and a skip connection structure, the initial spatiotemporal feature data provided by the skip connection spatiotemporal feature tensor is added to the convolution output, the features are processed through batch normalization and ReLU activation function, and multi-level spatiotemporal features are generated.

[0054] S4.2: according to the face recognition mathematical model framework, the spatiotemporal feature tensor is segmented according to the time step, and dynamic weight fusion is performed to generate the initial version of the face recognition mathematical model; Furthermore, based on the mathematical model framework for face recognition, the spatiotemporal feature tensor is divided into several segments with a fixed time step. Each time slice is converted into a feature vector through an independent fully connected layer. A gating attention mechanism is used to obtain the dynamic weights of each time slice. The feature vectors of all time slices are weighted and summed to generate the initial version of the face recognition mathematical model.

[0055] It should be noted that the gated attention mechanism first inputs the feature vector of each time slice into the fully connected layer to generate a query vector, a key vector, and a value vector. It then obtains the dot product similarity score between the query vector and the key vectors of all time slices, normalizes it into attention weights using the softmax function, and finally sums the value vectors of each time slice according to the attention weights to output the fused feature vector after dynamic weight allocation.

[0056] S4.3: Calculate the feature change index to obtain the motion abnormality score that represents the degree of motion abnormality, calculate the feature change intensity of the five sensory regions as the regional risk value, and linearly combine them according to the weight ratio to generate the risk prediction result.

[0057] The formula for calculating the characteristic change index is: ; in, express The index of dynamic change of characteristics at any given time. express The spatiotemporal feature tensor at time t, express The characteristic stability coefficient at time (range: 0.1-1.2). Represents a numerical stability constant (example value: 1e-6). Indicates the first Temporal attention weights of frames, Indicates the length of the analysis time window. It represents the relative displacement over time.

[0058] The formula for calculating the intensity of feature changes in the facial region is: ; in, Indicates the intensity of characteristic changes in the facial features. Indicates the first The key points of the five facial features are in the first The feature vector of a frame, Indicates the first Mean temporal features of key points Indicates the length of the analysis time window. Indicates the index of key points of the facial features. Indicates the timing number of the video frame.

[0059] It should be noted that the feature change index is calculated based on the space-time feature tensor, and the motion abnormality degree is obtained by the norm of the feature difference between adjacent frames within the time window; the regional feature change intensity is calculated for the feature vector of the five key points, and the deviation degree of each key point feature from the time sequence average is obtained; the motion abnormality score and the regional risk value are combined in proportion to the weight, and the final risk prediction result is generated (example: R = 0.6D t +0.4Y).

[0060] S5: Dynamically adjusting the initial version of the face recognition mathematical model based on the risk prediction result, and generating the face recognition mathematical model through the adversarial training strategy.

[0061] S5.1: Based on the risk prediction result, configure the decay coefficient to dynamically adjust the learning rate, and inversely adjust the batch size to generate a set of regulation parameters; Further, based on the risk prediction result, the three-level risk grade is divided (example: R < 0.3 is low risk, 0.3 ≤ R ≤ 0.7 is medium risk, R > 0.7 is high risk), and the decay coefficient is configured (example: low risk α = 0.9, medium risk α = 0.5, high risk α = 0.1); According to the decay coefficient, the learning rate is dynamically adjusted, and the batch size is adjusted in inverse proportion to the risk prediction result to generate a set of regulation parameters.

[0062] S5.2: According to the set of regulation parameters, generate adversarial samples through the projected gradient descent method, and mix the face image dataset with the adversarial samples to form a training dataset; Further, based on the learning rate and batch size in the regulation parameter set, the projected gradient descent method is used to generate adversarial perturbations on the original face image, and the generated adversarial samples are mixed with the original face image in proportion to obtain mixed samples, and a training dataset is generated.

[0063] It should be noted that the projected gradient descent method first obtains the loss function gradient of the original face image under the current face recognition mathematical model, applies perturbation in the gradient direction, and constrains the perturbation in the allowed range through the projection operation (example: limit the pixel change to not more than ±5), iteratively execute gradient calculation and perturbation application until the maximum iteration number is reached, and the generated adversarial samples retain the visual structure of the original image but contain targeted perturbations.

[0064] S5.3: Using the training dataset, combining the regulation parameter set to update the initial version of the face recognition mathematical model parameters through the back propagation algorithm, and generating the updated face recognition mathematical model; Further, using the mixed samples in the training data set, based on the learning rate and batch size set by the set of regulation parameters, the gradient of the loss function to the model parameters is calculated by the back propagation algorithm, and the weight parameters of the face recognition mathematical model are updated according to the learning rate proportion. After multiple rounds of iterative optimization, the updated face recognition mathematical model is generated.

[0065] S5.4: Multi-modal feature optimization is performed on the updated face recognition mathematical model, and the recognition accuracy of the multi-modal feature optimized face recognition mathematical model in the occlusion scene is evaluated to generate the face recognition mathematical model.

[0066] Further, the updated face recognition mathematical model is processed by the multi-modal feature optimization method, the outputs of the space-time feature branch and the texture feature branch are fused by the attention mechanism, the recognition accuracy of the multi-modal feature optimized face recognition mathematical model in the occlusion scene is evaluated on the occlusion test data set, and finally the face recognition mathematical model is generated.

[0067] It should be noted that the attention mechanism fusion first converts the space-time feature vector and the texture feature vector into attention scores through independent fully connected layers, obtains the dot product similarity of the two groups of attention scores, and normalizes them into fusion weights through softmax, and then sums the original feature vectors according to the weights. The recognition accuracy of the multi-modal feature optimized face recognition mathematical model is realized by executing forward inference on an independent test data set, which contains occlusion samples. The face feature vector output by the multi-modal feature optimized face recognition mathematical model is compared with the labeled identity label for similarity, and the proportion of correctly matched samples is taken as the accuracy.

[0068] The embodiment also provides a computer device suitable for the face recognition mathematical model construction method, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the face recognition mathematical model construction method proposed in the above embodiment.

[0069] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0070] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the method for constructing a face recognition mathematical model according to the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0071] To sum up, the present application achieves high-fidelity data synthesis by fusing the key points of five features and material parameters to generate occluded face images with dynamic deformation, thereby improving the model generalization capability and solving the problem of unnatural occlusion fitting. The physically realistic dynamic occlusion simulation is achieved by calculating the grid deformation frame by frame through vertex force analysis, thereby constructing a real training video and enhancing the robustness of the model to complex deformation.

[0072] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for constructing a mathematical model for face recognition, characterized in that: The application relates to a face recognition method and device. The face feature data and the face cover data are collected, the dynamic deformation parameters are marked after the range of the cover area is obtained, the face image data set is generated, and the face image data set is generated. The face image data set is dynamically rendered in real time, the elastic deformation and light reflection effect of the cover are simulated according to the motion track of the face cover, and a dynamic video sequence is generated. The dynamic video sequence is preprocessed, the spatiotemporal features are extracted by a three-dimensional convolution network, and the interactive changes of the motion track of the cover and the face features are captured to generate a spatiotemporal feature tensor. The spatiotemporal feature tensor is input into a joint modeling architecture to initialize a face recognition mathematical model, generate a preliminary face recognition mathematical model, analyze the interframe dependency relationship through a multi-head attention mechanism, and generate a risk prediction result. The preliminary face recognition mathematical model is dynamically adjusted based on the risk prediction result, and the face recognition mathematical model is generated through an adversarial training strategy.

2. The method of claim 1, wherein: The face feature data includes face detection frame coordinates and five facial feature point coordinates. The face cover data includes material characteristics and thickness data of the cover. Based on the face feature data and the face cover data, the range of the cover area is obtained, and the dynamic deformation parameters are marked to generate the face image data set. The face image data set is dynamically rendered in real time, and the motion track of the face cover is as follows.

3. The method of claim 2, wherein: The motion track of the face cover is collected, the rigid transformation mapping of the face image data set pixel coordinate system and the space coordinate system is established by combining the face image data set, and a three-dimensional scene entity is generated. According to the three-dimensional scene entity, the face cover data and the motion track of the face cover are combined, the grid deformation displacement is calculated frame by frame through vertex force analysis, and a cover grid sequence is generated. The elastic deformation and light reflection effect of the cover are simulated to generate a dynamic video sequence, and the specific steps are as follows.

4. The method of claim 3, wherein: Based on the cover grid sequence, the surface optical distribution is simulated through photon path tracking based on the material characteristics of the cover and the real-time environmental light parameters, and a light rendering frame sequence is generated. The light rendering frame sequence is sorted, and the dynamic video sequence is obtained through time resampling after applying motion vector compensation to eliminate jitter. The dynamic video sequence is preprocessed, and the spatiotemporal features are extracted by a three-dimensional convolution network, and the specific steps are as follows.

5. The method of claim 4, wherein the method further comprises: The dynamic video sequence is cut into video segments according to a fixed time window, and the standardized video segment set is generated after standardization processing. The three-dimensional convolution network architecture is used to extract features from the standardized video segment set to generate a primary spatiotemporal feature cube. The interactive changes of the motion track of the cover and the face features are captured to generate a spatiotemporal feature tensor, and the specific steps are as follows.

6. The method of claim 5, wherein: According to the primary spatiotemporal feature cube, the motion track features and the face features are processed through a double-branch attention mechanism to generate an enhanced spatiotemporal feature matrix. Based on the enhanced spatiotemporal feature matrix, the spatiotemporal feature tensor is generated through feature fusion in the time dimension and the space dimension. ​ 7. The method of claim 6, wherein the method further comprises: The spatiotemporal feature tensor is input into the joint modeling architecture to initialize the face recognition mathematical model, generate a preliminary version of the face recognition mathematical model, and analyze the inter-frame dependency relationship through the multi-head attention mechanism to generate a risk prediction result, the specific steps being as follows, Based on the spatiotemporal feature tensor, a double-branch joint modeling architecture is constructed, and a face recognition mathematical model framework is obtained through a feature fusion layer; According to the face recognition mathematical model framework, the spatiotemporal feature tensor is segmented according to the time step and dynamically weighted to generate a preliminary version of the face recognition mathematical model; The feature change index is calculated to obtain a motion anomaly score representing the degree of motion anomaly, the feature change intensity of the facial feature region is calculated as a regional risk value, and the risk prediction result is generated by linear combination according to the weight proportion.

8. The method of claim 7, wherein the method further comprises: The preliminary version of the face recognition mathematical model is dynamically adjusted based on the risk prediction result, and the face recognition mathematical model is generated through an adversarial training strategy, the specific steps being as follows, Based on the risk prediction result, a decay coefficient is configured to dynamically adjust the learning rate, and the batch size is inversely adjusted to generate a set of regulation parameters; According to the set of regulation parameters, an adversarial sample is generated through a projection gradient descent method, the face image dataset and the adversarial sample are mixed to form a training dataset; Using the training dataset, the preliminary version of the face recognition mathematical model is updated through a back propagation algorithm combined with the set of regulation parameters to generate an updated face recognition mathematical model; The updated face recognition mathematical model is subjected to multi-modal feature optimization, and the recognition accuracy of the face recognition mathematical model after multi-modal feature optimization in a shielding scene is evaluated to generate the face recognition mathematical model. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the face recognition mathematical model construction method of any one of claims 1-8.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the face recognition mathematical model construction method of any one of claims 1-8.

Images