A fast moving face recognition method and system based on multi-frame image enhancement

Abstract

The application discloses a kind of fast moving face recognition method and system based on multi-frame image enhancement, comprising the following steps: through imaging device acquisition multi-frame image sequence, obtain multi-frame image after pre-processing;The multi-frame image after pre-processing is input into the improved MonST3R network, carries out spatial alignment and structure analysis, obtains the image after alignment and three-dimensional structure information;The above result is input into CRNet network, separates and enhances multi-scale feature, and obtains enhanced feature;Enhanced feature is input into space-time attention fusion module, and multi-frame feature representation is obtained by weighted fusion;Fusion feature is input into identification network, and dimension reduction is carried out, and face embedding feature vector is obtained;Similarity comparison is carried out between embedding feature and database template, and recognition result and confidence prompt are output.The application can realize efficient, accurate and resource-friendly face recognition under dynamic complex environment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and image processing technology, and in particular to a fast moving face recognition method and system based on multi-frame image enhancement. Background Technology

[0002] With the development of artificial intelligence and computer vision technologies, facial recognition has been widely applied in security monitoring, financial payments, and mobile terminals. Traditional facial recognition methods are mostly based on single-frame static images, relying on two-dimensional feature extraction and comparison, which makes it difficult to maintain high accuracy and stability in dynamic scenarios such as fast movement, complex lighting, changes in facial expressions, and partial occlusion. Especially in practical applications such as mobile devices and public places, the target face often causes image blurring and drastic changes in posture due to rapid movement, resulting in the loss of single-frame information or insufficient discriminative power.

[0003] In recent years, face recognition methods based on multi-frame sequences have gradually attracted attention. By integrating multi-frame temporal features, spatial alignment, and dynamic analysis, they can help improve adaptability to dynamic changes. However, existing technologies generally have three limitations: First, the depth of multi-frame feature fusion is insufficient, failing to fully explore cross-frame spatial structure and temporal dynamic information; second, there is a lack of efficient 3D structural modeling and alignment mechanisms, making it difficult to take into account both the global structure of the face and local expression details; and third, in resource-constrained environments, complex models can easily lead to computational redundancy and recognition delays, which is not conducive to practical deployment.

[0004] Therefore, how to provide a fast moving face recognition method and system based on multi-frame image enhancement is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a fast moving face recognition method and system based on multi-frame image enhancement. This invention achieves high-precision face recognition in dynamic environments such as rapid movement, pose changes, and facial expression changes by combining multi-frame image enhancement, spatial alignment, and 3D information modeling with high- and low-frequency multi-scale feature separation, adaptive enhancement, spatiotemporal attention fusion, and efficient feature compression. This method not only effectively improves the accuracy and robustness of face recognition in dynamic, blurred, or partially occluded scenes, but also significantly reduces the system's computational and storage resource consumption. It is suitable for mobile terminals and large-scale practical deployments, and has good application prospects and promotional value.

[0006] A fast moving face recognition method based on multi-frame image enhancement according to an embodiment of the present invention includes the following steps:

[0007] A multi-frame image sequence is acquired by an imaging device, and the multi-frame image sequence is preprocessed to obtain preprocessed multi-frame images.

[0008] The preprocessed multi-frame images are input into the 3D information modeling module based on the improved MonST3R network for spatial alignment and dynamic structure analysis to obtain the aligned multi-frame images and 3D structure information.

[0009] The aligned multi-frame images and 3D structural information are input into a feature enhancement network based on CRNet, which separates low-frequency and high-frequency multi-scale features and performs adaptive enhancement to obtain enhanced multi-scale features.

[0010] The enhanced multi-scale features are input into the spatiotemporal attention fusion module. Through spatial attention mechanism and temporal dynamic modeling, the multi-frame features are weighted and fused to obtain a fused multi-frame feature representation.

[0011] The face embedding feature vector is obtained by inputting multi-frame feature representations into a recognition network that includes a feature compression mechanism, performing structural adaptive transformation and feature dimensionality reduction.

[0012] The facial embedding feature vector is compared with the template features in the database for similarity. Based on the set threshold strategy, the recognition result and confidence level are output.

[0013] Furthermore, the preprocessing operation includes denoising and normalizing the acquired multi-frame image sequence.

[0014] Furthermore, the improved MonST3R network includes:

[0015] For each input image frame, the overall structural features are extracted through a global feature backbone network, and high-resolution detail features of key facial regions are extracted through multi-scale dilated convolution and local attention mechanisms. The global and local features are then fused to obtain multi-scale spatial features, mainly including:

[0016] For each frame of input image, a custom adjustment of the MobileNetV2 backbone network is used to apply it to the full-resolution face ROI region to extract global spatial features that reflect the overall pose and main structure of the face, including contours, facial feature layout, and light and dark partitions.

[0017] Multi-scale dilated convolution modules are set up for global spatial features, which are parallel three-way or four-way convolutions. After each dilated convolution, an ROI mask attention mechanism is embedded. The ROI region is assigned the first weight, and the non-ROI region is assigned the second weight. The feature map is then weighted element by element to obtain local convolution features.

[0018] Global spatial features and local convolutional features are concatenated along the channel dimension, and a weighted summation method is used to obtain multi-scale spatial features.

[0019] Based on multi-scale spatial features, the point cloud regression module is used to predict the three-dimensional point cloud of each frame of the face. The three-dimensional coordinates are regressed according to each spatial location to generate a preliminary point cloud. The facial key region mask is generated by the detection of facial key points. The sampling method with a sampling interval of less than or equal to 2 pixels is used in the area covered by the mask, and the sampling interval with a sampling interval of greater than 2 pixels is used in other areas. The three-dimensional point cloud with a sampling interval of less than or equal to 2 pixels is output for each frame.

[0020] For each frame of 3D point cloud with a sampling interval of less than or equal to 2 pixels, rigid transformation parameter estimation and non-rigid deformation parameter estimation are performed respectively. The outputs of the two are weighted and fused to obtain the aligned multi-frame images and 3D structural information.

[0021] Furthermore, the estimation of rigid transformation parameters and non-rigid deformation parameters includes:

[0022] For a 3D point cloud with a sampling interval of less than or equal to 2 pixels per frame, a rigid transformation estimation subnetwork is input. The subnetwork is based on a least squares fitting strategy to predict the rotation matrix and translation vector in 3D space for the entire face point cloud.

[0023] Compare the spatial distribution of the point cloud in the current frame with that of the global standard template, minimize the point-to-point Euclidean distance loss between them, and perform end-to-end backpropagation optimization on the rotation and translation parameters to obtain the globally registered point cloud.

[0024] For each frame of rigidly registered global registration point cloud, a non-rigid deformation estimation network is input. The non-rigid deformation estimation network is based on a multilayer perceptron structure, which models the local neighborhood relationship and feature differences between each point cloud point, and regresses the local elastic deformation of each point to obtain the local elastic correction point cloud.

[0025] Non-rigid deformation parameters are set with adaptive weights for key dynamic areas of the face, and targeted fine-grained corrections are made for local deformations caused by facial expression changes and muscle movements.

[0026] The globally registered point cloud obtained by rigid transformation and the locally elastically corrected point cloud output by the non-rigid deformation network are fused by a linear weighted fusion method. The fusion weights are adaptively adjusted according to the spatial location of the point cloud, feature similarity, or regional importance.

[0027] By maintaining rigid alignment as the dominant force in the global facial region and increasing the proportion of non-rigid correction in key detail areas, the final output is a multi-frame image and 3D structural information that is globally consistent and restores local details.

[0028] Furthermore, the CRNet network includes the following processing steps:

[0029] The aligned multi-frame images and 3D structural information are respectively feature-encoded, and the encoded features are input into the high- and low-frequency feature separation module.

[0030] In the high- and low-frequency feature separation module, a parallel structure is adopted:

[0031] The input features are processed through a max pooling branch, and the max pooling operation is used to extract low-frequency features, resulting in low-frequency feature components that represent the overall contour and main structure of the face.

[0032] The input features are passed through a Laplacian filter branch and then convolved with a Laplacian convolution kernel to extract high-frequency feature components that highlight edges and details.

[0033] For low-frequency feature components, an adaptive contrast enhancement operation is further adopted to adjust the feature amplitude based on the local contrast factor;

[0034] For high-frequency feature components, residual convolution enhancement is used to activate facial details and edge features through convolution and ReLU.

[0035] The enhanced high-frequency and low-frequency features are concatenated along the channel dimension, and then subjected to BatchNorm normalization and ReLU nonlinear activation processing to be concatenated and integrated into enhanced multi-scale features.

[0036] Furthermore, the spatiotemporal attention fusion module includes the following processing steps:

[0037] By using a spatial attention mechanism, global average pooling is performed on multi-scale features to obtain the response intensity of features at each spatial location. Attention weights are then assigned to different spatial locations in the feature map based on the response intensity of features at each spatial location, resulting in spatially weighted features.

[0038] The temporal modeling unit assembles multi-scale features into a sequence in chronological order, uses a recurrent neural network to model the dynamic relationship between features between frames over time, calculates the correlation between each frame and the overall sequence, assigns temporal attention weights to the features of each frame, and the temporal attention weights are positively correlated with the correlation between each frame and the overall sequence, thus obtaining time-weighted features.

[0039] Spatial weighted features and temporal weighted features are linearly weighted and fused according to preset weighting coefficients to obtain a fused multi-frame feature representation.

[0040] Furthermore, the step of obtaining the face embedding feature vector includes:

[0041] The identification network is the MobileFaceNetz network;

[0042] At the front end of the MobileFaceNetz network, a multi-frame spatiotemporal feature input interface is set up to receive the fused multi-frame feature representation. The feature distribution of each channel or frame is normalized by mean and variance. Based on the matching relationship between the actual number of channels of the fused multi-frame feature representation and the standard input channel number of the MobileFaceNetz backbone network, a channel adaptation layer is set up. The channel adaptation layer adopts a pointwise convolution method to adjust the number of channels of the multi-frame fused features to the number of input channels required by the backbone network.

[0043] A three-dimensional structural information embedding mechanism is introduced in the middle layer of the backbone network. The three-dimensional structural information and the fused multi-frame feature representation are aligned in the channel dimension. The three-dimensional structural information is linearly interpolated to make the spatial resolution consistent with the fused multi-frame feature representation. The three-dimensional structural information and the fused multi-frame feature representation are merged along the channel dimension by channel stitching. The three-dimensional structural information and the fused multi-frame feature representation are adaptively captured and modeled to obtain the final fused feature tensor.

[0044] In the deep layers of the backbone network, a feature compression mechanism is set up to reduce the dimensionality of the three-dimensional feature vector in the final fused feature tensor and output the face embedding feature vector.

[0045] Furthermore, the feature compression mechanism specifically includes:

[0046] The three-dimensional feature vectors in the final fused feature tensor are input into one or more sets of depthwise separable convolutional layers. Spatial convolution is performed independently on each channel, and then feature recombination and dimensionality reduction are performed in the channel dimension through pointwise convolution.

[0047] The feature tensor output by depthwise separable convolution is flattened into a one-dimensional vector and input into a fully connected dimensionality reduction layer to further compress the feature dimension to a predetermined two-dimensional feature.

[0048] A feature distillation unit is set up, and by introducing a distillation loss function, the two-dimensional features of the main network are aligned with the one-dimensional features of the teacher network;

[0049] The entire feature compression mechanism is jointly optimized through the main loss function and distillation loss during end-to-end training, and finally outputs the face embedding feature vector.

[0050] Furthermore, the similarity comparison includes the following steps:

[0051] The face embedding feature vector is compared with the template features pre-stored in the database. The similarity comparison is performed using cosine similarity.

[0052] When the similarity is greater than the first threshold, output the corresponding identification and confidence level prompts;

[0053] When the similarity is between the first threshold and the second threshold, output "Uncertain" and the similarity value;

[0054] When the similarity is below the second threshold, output a "no match" message.

[0055] A fast-moving face recognition system based on multi-frame image enhancement according to an embodiment of the present invention includes the following modules:

[0056] The multi-frame image acquisition and preprocessing module is used to acquire a multi-frame image sequence through an imaging device, and to perform noise reduction and normalization processing on the acquired multi-frame image sequence, and output the preprocessed multi-frame image.

[0057] The 3D information modeling module is used to perform spatial alignment and dynamic structural analysis on preprocessed multi-frame images, thereby obtaining aligned multi-frame images and 3D structural information.

[0058] The feature enhancement module is used to encode the features of the aligned multi-frame images and 3D structural information, separate high- and low-frequency multi-scale features, perform adaptive enhancement, and output the enhanced multi-scale features.

[0059] The spatiotemporal attention fusion module is used to perform weighted fusion of enhanced multi-scale features through spatial attention mechanism and temporal dynamic modeling, and output fused multi-frame feature representation;

[0060] The recognition network module is used to receive and fuse multi-frame feature representations. Through front-end normalization and channel adaptation, combined with the 3D structural information embedding mechanism in the backbone network and deep feature compression processing, it outputs face embedding feature vectors.

[0061] The similarity comparison and recognition output module is used to compare the similarity between the face embedded feature vector and the template features in the database, and output the recognition result and confidence level prompt according to the preset threshold.

[0062] The beneficial effects of this invention are:

[0063] This invention achieves accurate extraction and comparison of facial features in complex scenarios such as rapid movement, posture changes, and facial expression changes by using multi-frame image spatial alignment, dynamic structure analysis, and 3D information modeling. It overcomes the performance degradation caused by information loss and blurring in dynamic environments in single-frame recognition, and significantly improves the accuracy and stability of recognition.

[0064] By employing spatial alignment based on 3D point clouds, ROI attention mechanism, multi-scale feature enhancement, and spatiotemporal attention fusion method, it is possible to simultaneously capture the overall structural information and local detail changes of the face, improve the modeling ability of subtle features such as expressions and micro-movements, and make the recognition system more adaptable and discriminative to complex facial dynamics.

[0065] Through feature compression mechanisms, depthwise separable convolutions, and end-to-end integration optimization, the computational load and resource consumption of the system are effectively reduced, enabling real-time recognition in resource-constrained environments such as mobile terminals and embedded devices. At the same time, the functions can be flexibly expanded according to actual needs to meet the deployment requirements of large-scale practical applications. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a flowchart of a fast moving face recognition method based on multi-frame image enhancement proposed in this invention;

[0068] Figure 2 This is a flowchart of the multi-scale feature enhancement and spatiotemporal fusion of a fast moving face recognition method based on multi-frame image enhancement proposed in this invention.

[0069] Figure 3 This is a schematic diagram illustrating the feature extraction of a three-dimensional face structure in a fast moving face recognition method based on multi-frame image enhancement proposed in this invention.

[0070] Figure 4 This is a structural diagram of a fast moving face recognition system based on multi-frame image enhancement proposed in this invention. Detailed Implementation

[0071] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0072] refer to Figure 1-3 A fast moving face recognition method based on multi-frame image enhancement includes the following steps:

[0073] A multi-frame image sequence is acquired by an imaging device, and the multi-frame image sequence is preprocessed to obtain preprocessed multi-frame images.

[0074] The preprocessed multi-frame images are input into the 3D information modeling module based on the improved MonST3R network for spatial alignment and dynamic structure analysis to obtain the aligned multi-frame images and 3D structure information.

[0075] The aligned multi-frame images and 3D structural information are input into a feature enhancement network based on CRNet, which separates low-frequency and high-frequency multi-scale features and performs adaptive enhancement to obtain enhanced multi-scale features.

[0076] The enhanced multi-scale features are input into the spatiotemporal attention fusion module. Through spatial attention mechanism and temporal dynamic modeling, the multi-frame features are weighted and fused to obtain a fused multi-frame feature representation.

[0077] The face embedding feature vector is obtained by inputting multi-frame feature representations into a recognition network that includes a feature compression mechanism, performing structural adaptive transformation and feature dimensionality reduction.

[0078] The facial embedding feature vector is compared with the template features in the database for similarity. Based on the set threshold strategy, the recognition result and confidence level are output.

[0079] This invention significantly improves the accuracy and robustness of face recognition in fast-moving, expression-changing, and complex dynamic scenes through multi-frame image sequence enhancement, 3D spatial alignment, multi-scale feature adaptive enhancement, and spatiotemporal attention fusion. Employing feature compression and adaptive structural design, it not only ensures recognition accuracy but also effectively reduces system resource consumption, making it suitable for mobile terminals and large-scale practical deployments, demonstrating outstanding engineering application value and promising prospects for widespread adoption.

[0080] Furthermore, the preprocessing operation includes denoising and normalization of the acquired multi-frame image sequence. Specifically, after the multi-frame image sequence is acquired, Gaussian filtering and median filtering algorithms are first used to denoise each frame, removing noise, background interference, and abrupt pixel changes introduced during image acquisition, thereby improving image quality and signal-to-noise ratio. Subsequently, all image frames are normalized, mapping pixel values ​​to a unified standard range ([0,1]), eliminating differences in brightness, contrast, etc., caused by changes in imaging conditions, making the subsequent feature extraction process more stable and consistent. Through the above denoising and normalization operations, the interference of noise on recognition features can be effectively suppressed, improving the overall quality and feature expression consistency of the multi-frame image sequence, laying a solid foundation for subsequent 3D modeling and deep feature extraction, and improving the recognition accuracy and robustness of the entire system.

[0081] Furthermore, the improved MonST3R network includes:

[0082] For each input image frame, the overall structural features are extracted using a global feature backbone network, and the detailed features of key facial regions are extracted using multi-scale dilated convolution and local attention mechanisms. The global and local features are then fused to obtain multi-scale spatial features, which mainly include:

[0083] For each frame of input image, the MobileNetV2 backbone network is customized and applied to the full-resolution face ROI region to extract global spatial features that reflect the overall pose and main structure of the face, including contours, facial feature layout, and light and dark partitions. This can enhance the overall perception and anti-interference ability of the face under different angles and complex lighting conditions.

[0084] A multi-scale dilated convolution module is set up for global spatial features, which is a parallel three-way or four-way convolution. After each dilated convolution, an ROI mask attention mechanism is embedded. The ROI region is given the first weight, and the non-ROI region is given the second weight. The feature map is weighted element by element, which can highlight the contribution of key detail regions such as eyes and mouth to 3D modeling and dynamic analysis, while suppressing noise interference from background and irrelevant regions.

[0085] Global spatial features and local convolutional features are concatenated in the channel dimension and a weighted summation fusion method is used to obtain multi-scale spatial features, ensuring that the feature representation contains both global structural information and enhances local dynamic details.

[0086] Based on multi-scale spatial features, the point cloud regression module is used to predict the 3D point cloud of each frame of the face. The 3D coordinates are regressed according to each spatial location to generate a preliminary point cloud. Facial key region mask is generated by facial key point detection. A sampling method with a sampling interval of less than or equal to 2 pixels is used in the area covered by the mask, and a sampling interval of greater than 2 pixels is used in other areas. The 3D point cloud with a sampling interval of less than or equal to 2 pixels is output for each frame, so as to balance the fine reconstruction of facial details and the efficiency of the overall space in 3D modeling.

[0087] For each frame of 3D point cloud with a sampling interval of less than or equal to 2 pixels, rigid transformation parameter estimation and non-rigid deformation parameter estimation are performed respectively. The outputs of the two are weighted and fused to achieve alignment of multiple frames of images in global space. At the same time, detailed corrections are made for local dynamic changes such as expressions and actions. Finally, the aligned multi-frame images and 3D structural information are obtained, providing a 3D dynamic foundation for subsequent face recognition.

[0088] Furthermore, the estimation of rigid transformation parameters and non-rigid deformation parameters includes:

[0089] For a 3D point cloud with a sampling interval of less than or equal to 2 pixels per frame, a rigid transformation estimation subnetwork is input. This subnetwork, based on a least-squares fitting strategy, predicts the rotation matrix and translation vector in 3D space for the entire face point cloud. ;

[0090] in, Indicates the first point in the current frame point cloud The three-dimensional coordinates of the points This represents the three-dimensional coordinates of the corresponding point in the global standard template. It is a three-dimensional rotation matrix. It is a three-dimensional translation vector. This represents the total number of points in the point cloud. Indicates Euclidean distance;

[0091] It can quickly achieve overall registration of point clouds in the global coordinate system, reducing alignment errors caused by large movements or attitude changes;

[0092] By comparing the spatial distribution of the current frame point cloud with that of the global standard template, minimizing the point-to-point Euclidean distance loss between them, and optimizing the rotation and translation parameters through end-to-end backpropagation, a global registration point cloud is obtained. This ensures that the 3D alignment process is differentiable and the optimal solution converges, thereby improving the registration accuracy and adaptability.

[0093] For each frame of rigidly registered global point cloud, a non-rigid deformation estimation network is input. This network, based on a multilayer perceptron structure, models the local neighborhood relationships and inter-point feature differences for each point in the point cloud, regressing the local elastic deformation of each point to obtain a locally elastically corrected point cloud.

[0094] ;

[0095] in, Indicates the first Local elastic deformation at a point, Indicates the first point in the current frame point cloud The three-dimensional coordinates of the points Indicating the first point in the local elastic correction point cloud The three-dimensional coordinates of each point;

[0096] Non-rigid deformation parameters are set with adaptive weights for key facial regions, and targeted fine-grained corrections are made for local deformations caused by facial expression changes and muscle movements. This can enhance the dynamic modeling capability of key regions while maintaining the consistency of the global structure.

[0097] The globally registered point cloud obtained by rigid transformation and the locally elastically corrected point cloud output by the non-rigid deformation network are fused by a linear weighted fusion method. The fusion weights are adaptively adjusted according to the spatial location of the point cloud, feature similarity, or regional importance, so that the fused point cloud retains the overall spatial consistency while taking into account local dynamic flexibility and discriminative ability.

[0098] By maintaining rigid alignment as the dominant force in the global facial region and increasing the proportion of non-rigid correction in key detail areas, the final output is a multi-frame image and 3D structural information that is globally consistent and restores local details, providing a solid 3D foundation for subsequent face recognition and dynamic analysis.

[0099] Furthermore, the CRNet network includes the following processing steps:

[0100] The aligned multi-frame images and 3D structural information are respectively feature-encoded, and the encoded features are input into the high- and low-frequency feature separation module to improve the feature representation ability of the original input information, which helps the subsequent branch modules to extract more discriminative features.

[0101] In the high- and low-frequency feature separation module, a parallel structure is adopted to process the input features separately, ensuring that high-frequency details and low-frequency structural information are extracted simultaneously, thereby enhancing the system's ability to capture multi-scale features.

[0102] The input features are passed through the max pooling branch, and the max pooling operation (such as a pooling window size of 3×3 and a step size of 1) is used to extract low-frequency features, which are then used to obtain low-frequency feature components that represent the overall contour and main structure of the face. This helps the model to stably perceive the global shape, pose and brightness distribution of the face, and improves its robustness to global perturbations such as illumination and occlusion.

[0103] The input features are passed through a Laplacian filter branch and convolved with a (e.g., a standard 3×3) Laplacian convolution kernel to extract high-frequency feature components that highlight edges and details.

[0104] For low-frequency feature components, an adaptive contrast enhancement operation is further employed, adjusting the feature amplitude based on the local contrast factor, specifically as follows:

[0105] ;

[0106] Where L represents the low-frequency characteristics, C(L) is the local contrast, and γ is the adaptive enhancement coefficient. This refers to the enhanced low-frequency characteristics;

[0107] For high-frequency feature components, residual convolution enhancement is applied to activate facial details and edge features through convolution and ReLU, specifically:

[0108] ;

[0109] in, For the enhanced high-frequency features, For residual weights, This represents a 3×3 convolution. For activation function, High-frequency characteristics;

[0110] The enhanced high-frequency and low-frequency features are concatenated along the channel dimension, and then subjected to BatchNorm normalization and ReLU nonlinear activation processing. The concatenated features are integrated into enhanced multi-scale features, which are used in subsequent spatiotemporal attention fusion and recognition networks.

[0111] This step, through multi-branch separation, enhancement, and fusion, effectively improves the feature representation capabilities of the global structure and local details of the face, providing a solid foundation for subsequent recognition.

[0112] Furthermore, the spatiotemporal attention fusion module includes the following processing steps:

[0113] By using a spatial attention mechanism, global average pooling is performed on multi-scale features to obtain the response intensity of features at each spatial location. Attention weights are then assigned to different spatial locations in the feature map based on the response intensity of features at each spatial location, resulting in spatially weighted features.

[0114] The temporal modeling unit assembles multi-scale features into a sequence in chronological order, uses a recurrent neural network to model the dynamic relationship between features between frames over time, calculates the correlation between each frame and the overall sequence, assigns temporal attention weights to the features of each frame, and the temporal attention weights are positively correlated with the correlation between each frame and the overall sequence, thus obtaining time-weighted features.

[0115] Spatial weighted features and temporal weighted features are linearly weighted and fused according to preset weighting coefficients to obtain a fused multi-frame feature representation.

[0116] Furthermore, the step of obtaining the face embedding feature vector includes:

[0117] The identification network is the MobileFaceNetz network;

[0118] At the front end of the MobileFaceNetz network, a multi-frame spatiotemporal feature input interface is set up to receive the fused multi-frame feature representations. The mean and variance of the feature distribution of each channel or frame are normalized. The specific processing is as follows:

[0119] ;

[0120] in, For the input features in the th Each channel and spatial location The value, and The first The mean and standard deviation of each channel;

[0121] Based on the matching relationship between the actual number of channels in the multi-frame feature representation and the standard input channel number of the MobileFaceNetz backbone network, a channel adaptation layer is set up. The channel adaptation layer adopts a pointwise convolution method to adjust the number of channels in the multi-frame fused features to the input channel number required by the backbone network, so as to ensure that features from different sources can be efficiently integrated.

[0122] A 3D structural information embedding mechanism is introduced in the middle layer of the backbone network. The 3D structural information and the fused multi-frame feature representation are aligned in the channel dimension. The 3D structural information is linearly interpolated to adjust the spatial resolution to keep it consistent with the fused multi-frame feature representation. The 3D structural information and the fused multi-frame feature representation are merged along the channel dimension by channel splicing. This adaptively captures and models the 3D structure and dynamic changes of the face to obtain the final fused feature tensor. This approach can adaptively capture and model the 3D structure and dynamic changes of the face, improving the network's ability to adapt to complex poses and facial expressions.

[0123] In the deep layers of the backbone network, a feature compression mechanism is set up to reduce the dimensionality of the three-dimensional feature vector in the final fused feature tensor and output the face embedding feature vector. The specific process includes first performing channel dimensionality reduction through 1×1 convolution, and then further compressing the feature dimension through a fully connected dimensionality reduction layer, and finally outputting the face embedding feature vector.

[0124] Furthermore, the feature compression mechanism specifically includes:

[0125] The three-dimensional feature vector in the final fused feature tensor is input into one or more sets of depthwise separable convolutional layers. Spatial convolution (i.e., Depthwise convolution) is performed independently on each channel. Then, feature recombination and dimensionality reduction are performed in the channel dimension through pointwise convolution to reduce redundant channels and improve feature expression efficiency.

[0126] The feature tensor output by depthwise separable convolution is flattened into a one-dimensional vector and input into a fully connected dimensionality reduction layer to further compress the feature dimension to a predetermined low dimension (such as 128 or 256 dimensions), ensuring that the feature representation is compact and suitable for subsequent similarity comparison and practical engineering deployment.

[0127] A feature distillation unit is set up, and by introducing a distillation loss function, the two-dimensional features of the main network are aligned with the one-dimensional features of the teacher network;

[0128] The entire feature compression mechanism is jointly optimized by the main loss function (classification loss or triplet loss) and distillation loss during end-to-end training to ensure that the dimensionality-reduced features are not only compact but also highly discriminative, and finally output face embedding feature vectors to improve the overall recognition performance and deployment efficiency of the system.

[0129] Furthermore, the similarity comparison includes the following steps:

[0130] The face embedding feature vector is compared with the template features pre-stored in the database. The similarity comparison is performed using cosine similarity.

[0131] When the similarity is greater than the first threshold (e.g., 0.75), the corresponding identification and confidence level prompts are output.

[0132] When the similarity is between the first and second thresholds (e.g., 0.6 to 0.75), the system outputs "Uncertain" and the similarity value, prompting the user to confirm or assist in the verification process.

[0133] When the similarity is below the second threshold (e.g., 0.6), an "No match" message is output, indicating that the current input has no valid match with the database template.

[0134] refer to Figure 4 A fast moving face recognition system based on multi-frame image enhancement includes the following modules: a multi-frame image acquisition and preprocessing module, used to continuously acquire multi-frame image sequences through an imaging device (such as an RGB camera or video capture card), and to perform denoising and normalization processing on the acquired multi-frame image sequences. After processing with Gaussian filtering and median filtering algorithms, noise and artifacts are removed, and all frames are uniformly normalized to a preset pixel value range ([0,1]) to improve the stability and robustness of subsequent feature extraction and output the preprocessed multi-frame images;

[0135] The 3D information modeling module is used to perform spatial alignment and dynamic structure analysis on preprocessed multi-frame images. It adopts a structure based on an improved MonST3R network. For each frame image, global spatial structure features are first extracted through the backbone network (such as MobileNetV2). Combined with multi-scale dilated convolution and ROI mask attention mechanism, high-resolution details of key facial regions are highlighted. After spatial alignment, the features of each frame are input into the point cloud regression submodule to regress the 3D point cloud of each frame. By using deformable high-density sampling for key facial regions (such as eyes and mouth), the accuracy of 3D structure description is significantly improved. The module outputs aligned multi-frame images and high-density 3D structure information, which effectively improves the robustness of subsequent recognition under pose changes and facial expression dynamics.

[0136] The feature enhancement module, based on the CRNet network, is used to encode the features of the aligned multi-frame images and 3D structural information, separate high- and low-frequency multi-scale features, and extract the overall contour and detail edges of the face through parallel max pooling and Laplacian filtering branches. It also performs adaptive contrast enhancement on low-frequency features and residual convolution to enhance high-frequency features, outputting enhanced multi-scale features.

[0137] The spatiotemporal attention fusion module is used to perform weighted fusion of enhanced multi-scale features through spatial attention mechanism and temporal dynamic modeling. The spatial attention mechanism assigns attention weights according to the response intensity of features at each spatial location, highlighting key facial region information. The temporal dynamic modeling unit uses recurrent neural networks or self-attention methods to capture temporal changes and dynamic relationships between multiple frames. Spatial weighted features and temporal weighted features are integrated through weighted fusion to output a fused multi-frame feature representation.

[0138] The recognition network module adopts the structure-optimized MobileFaceNetz network to receive and fuse multi-frame feature representations. Through front-end normalization and channel adaptation, combined with the 3D structural information embedding mechanism in the backbone network and deep feature compression processing, it outputs face embedding feature vectors. This module improves the recognition accuracy of complex expressions, poses and occluded scenes, and reduces the system's computing resource consumption.

[0139] The similarity comparison and recognition output module is used to compare the similarity of the face embedded feature vector with the template features in the database, and output the recognition result and confidence level according to the preset threshold. When the similarity is higher than the first threshold, the corresponding identity and confidence level are directly output; when it is between the two thresholds, "uncertain" and similarity are output; when it is lower than the second threshold, "not matched" result is output. This module can effectively prevent false recognition and improve the practicality and security of the system.

[0140] Example 1:

[0141] To verify the feasibility of this invention in practice, it was applied to a campus facial recognition system. Ten volunteers were selected, and each volunteer performed actions such as rapidly moving their heads left and right and changing their facial expressions under standard indoor lighting conditions. A high-definition camera captured video sequences at 40 frames per second, with each video lasting 3 seconds, for a total of 300 frames. Ten consecutive frames were selected as a multi-frame recognition window.

[0142] The acquired multi-frame image sequence was subjected to Gaussian filtering for noise reduction. The filter kernel size was [value], and the standard deviation was 0.5. All images were normalized to the range [0,1].

[0143] Using an improved MonST3R network, global structural features are extracted for each frame image using a custom MobileNetV2 backbone, and key region details are extracted by adding ROI mask attention (weight ratio of 2:1) through three-way dilated convolution (dilation rates of 1, 2, and 3 respectively).

[0144] Using the key point at the tip of the nose as the registration anchor point, a high-density point cloud (approximately 5000 points, with the sampling of areas such as the eyes and mouth being 3 times that of the background) is generated for each frame using the point cloud regression module.

[0145] For point clouds, the rigid registration loss converges to 0.002, and the average amplitude of the non-rigid elastic adjustment is 1.3 pixels.

[0146] For the aligned multi-frame images and point cloud encoded features, the max pooling window is... Extracting low-frequency features using Laplacian convolution kernels Extract high-frequency features.

[0147] Low-frequency feature contrast enhancement coefficient set to The high-frequency feature residual convolution weight is 0.2, and the output is concatenated by channels and standardized by BatchNorm.

[0148] After global average pooling, Convolution and Sigmoid are used to obtain a spatial attention map, with spatial attention weights ranging from 0.32 to 0.88.

[0149] The multi-frame features are input into a two-layer LSTM (128 hidden units), and the temporal weight of each frame after Softmax normalization ranges from 0.07 to 0.21.

[0150] Spatial and temporal weighted features in proportion The fusion process outputs a multi-frame fused feature tensor.

[0151] After multi-frame fusion feature normalization, the number of channels was reduced from 256. The convolution is adjusted to 64 input channels in the MobileFaceNetz backbone.

[0152] The intermediate 3D structural information is aligned with the spatial resolution and stitched together through linear interpolation. Deep compression uses depthwise separable convolution (channel number 64→16), and fully connected layers reduce the dimensionality to 128 dimensions.

[0153] The distillation loss and ArcFace loss were trained together, and the average norm of the final face embedding feature output was approximately 8.2.

[0154] Establish a template library, and collect 5 sets of static standard photos for each person to embed features as templates.

[0155] Cosine similarity was used for comparison, with thresholds set to 0.75 (match) and 0.6 (uncertain).

[0156] Output the identified identity and confidence level, or a "uncertain" / "no match" result.

[0157] Table 1. Statistical Table of Experimental Data and Results

[0158]

[0159] As shown in Table 1, the embodiments of the present invention achieve an average recognition accuracy of 97.88% in fast-moving dynamic face recognition scenarios, significantly outperforming traditional single-frame 2D methods (which achieve only about 93% under the same conditions). The mean confidence level for each subject remained above 0.8, and the uncertainty and non-match rates were both below 2%, indicating that the system can effectively distinguish between clear recognition and blurred boundary scenes, possessing strong adaptive discrimination capabilities. The average processing latency is approximately 50 milliseconds, fully meeting the requirements of real-time applications.

[0160] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fast moving face recognition method based on multi-frame image enhancement, characterized in that, The method comprises the following steps: A plurality of image sequences are collected by an imaging device, and preprocessing operations are performed on the plurality of image sequences to obtain preprocessed plurality of images; The preprocessed plurality of images are input into a three-dimensional information modeling module based on an improved MonST3R network to perform spatial alignment and dynamic structure analysis, and aligned plurality of images and three-dimensional structure information are obtained; The improved MonST3R network comprises: For each input image, global structure features are extracted by a global feature backbone network, detail features of facial key regions are extracted by multi-scale hollow convolution and local attention mechanism, and global and local features are fused to obtain multi-scale spatial features, mainly including: For each input image, a self-defined adjusted MobileNetV2 backbone network is used to act on a full-resolution face ROI region to extract global spatial features reflecting the overall posture and main structure of the face, including contour, facial feature layout, and light-dark partition; A multi-scale hollow convolution module is set for the global spatial features, and parallel three or four convolution is performed, and after each hollow convolution, an ROI mask attention mechanism is embedded, the ROI region is given a first weight, and the non-ROI region is given a second weight, and the feature map is weighted element by element to obtain local convolution features; The global spatial features and the local convolution features are spliced in the channel dimension, and a weighted summation fusion method is used to obtain multi-scale spatial features; Based on the multi-scale spatial features, a point cloud regression module is used to predict the three-dimensional point cloud of each frame of face, and three-dimensional coordinates are regressed for each spatial position to generate a preliminary point cloud, a face key region mask is generated by face key point detection, a sampling interval less than or equal to 2 pixels is used in the region covered by the mask, and a sampling interval greater than 2 pixels is used in other regions, and a three-dimensional point cloud with a sampling interval less than or equal to 2 pixels is output for each frame; For the three-dimensional point cloud with a sampling interval less than or equal to 2 pixels of each frame, rigid transformation parameter estimation and non-rigid deformation parameter estimation are performed, and the outputs are weighted and fused to obtain aligned plurality of images and three-dimensional structure information; The aligned plurality of images and three-dimensional structure information are input into a feature enhancement network based on a CRNet network to separate low-frequency and high-frequency multi-scale features and perform adaptive enhancement to obtain enhanced multi-scale features; The enhanced multi-scale features are input into a space-time attention fusion module, the multi-frame features are weighted and fused by a spatial attention mechanism and a time dynamic modeling to obtain a fused multi-frame feature representation; The fused multi-frame feature representation is input into a recognition network comprising a feature compression mechanism to perform structure adaptive transformation and feature dimension reduction to obtain a face embedding feature vector; The face embedding feature vector is compared with template features in a database in terms of similarity, and a recognition result and a confidence prompt are output according to a set threshold strategy.

2. The method of claim 1, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The preprocessing operations comprise denoising and normalization of the collected plurality of image sequences.

3. The method of claim 2, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The rigid transformation parameter estimation and the non-rigid deformation parameter estimation comprise: For each frame of three-dimensional point cloud with a sampling interval less than or equal to 2 pixels, a rigid transformation estimator network is input, and the sub-network predicts a rotation matrix and a translation vector in three-dimensional space for the whole face point cloud based on a least square fitting strategy; The point cloud spatial distribution of the current frame point cloud and the global standard template is compared, the Euclidean distance loss between the two is minimized, and the rotation and translation parameters are optimized through end-to-end back propagation to obtain a globally registered point cloud; For each frame of globally registered point cloud after rigid registration, a non-rigid deformation estimation network is input, the non-rigid deformation estimation network is based on a multi-layer perception structure, models the local neighborhood relationship and feature difference between point clouds, and outputs the local elastic deformation variable of each point to obtain a local elastic correction point cloud; The non-rigid deformation parameters set adaptive weights for the key dynamic regions of the face, and the local deformation caused by expression changes and muscle movements is corrected in a targeted and fine-grained manner; The globally registered point cloud obtained by the rigid transformation and the local elastic correction point cloud output by the non-rigid deformation network are fused through linear weighting fusion, and the fusion weight is adaptively adjusted according to the spatial position, feature similarity or regional importance of the point cloud; The rigid alignment is maintained in the global region of the face, and the non-rigid correction proportion is increased in the key detail section, and finally the multi-frame images and three-dimensional structure information with global consistency and local detail restoration are output.

4. The method of claim 1, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The CRNet network includes the following processing steps: The aligned multi-frame images and three-dimensional structure information are respectively encoded, and the encoded features are input into a high-low frequency feature separation module; In the high-low frequency feature separation module, a parallel structure is adopted: The input features are separated through a maximum pooling branch, and low-frequency features are extracted through a maximum pooling operation to obtain low-frequency feature components representing the overall contour and main structure of the face; The input features are separated through a Laplace filter branch, and high-frequency feature components highlighting edge and detail information are extracted through convolution of the input features with a Laplace convolution kernel; For the low-frequency feature components, an adaptive contrast enhancement operation is further adopted to adjust the feature amplitude based on a local contrast factor; For the high-frequency feature components, a residual convolution enhancement operation is adopted to enhance the facial detail and edge features through convolution and ReLU activation; The enhanced high-frequency features and low-frequency features are spliced in the channel dimension, and are subjected to BatchNorm normalization and ReLU nonlinear activation processing to obtain enhanced multi-scale features.

5. The method of claim 1, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The spatio-temporal attention fusion module includes the following processing steps: Through a spatial attention mechanism, global average pooling is performed on the multi-scale features to obtain the response intensity of each spatial position feature, and attention weights are assigned to different spatial positions in the feature map according to the response intensity of each spatial position feature to obtain spatial weighted features; A time modeling unit models the dynamic relationship of inter-frame features over time by arranging the multi-scale features in a sequence, and calculates the correlation of each frame with the overall sequence to assign a time attention weight to each frame, which is positively correlated with the correlation of each frame with the overall sequence, to obtain time weighted features; The spatial weighted feature and the temporal weighted feature are linearly weighted and fused according to a preset weighting coefficient to obtain a fused multi-frame feature representation.

6. The method of claim 1, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The obtaining of the face embedding feature vector comprises: The recognition network is a MobileFaceNetz network. At the front end of the MobileFaceNetz network, a multi-frame space-time feature input interface is arranged to receive the fused multi-frame feature representation, the feature distribution of each channel or frame is normalized by mean and variance, a channel adaptation layer is arranged according to the matching relationship between the actual channel number of the fused multi-frame feature representation and the standard input channel number of the MobileFaceNetz backbone network, and the channel adaptation layer adjusts the channel number of the multi-frame fused feature to the input channel number required by the backbone network by using a point-by-point convolution method. A three-dimensional structure information embedding mechanism is introduced at the middle layer of the backbone network, the three-dimensional structure information is size-aligned with the fused multi-frame feature representation in the channel dimension, the three-dimensional structure information is linearly interpolated and adjusted to keep the spatial resolution consistent with the fused multi-frame feature representation, the three-dimensional structure information and the fused multi-frame feature representation are merged along the channel dimension by channel concatenation, the face three-dimensional structure and dynamic changes are adaptively captured and modeled, and a final fused feature tensor is obtained. At the deep layer of the backbone network, a feature compression mechanism is arranged to reduce the dimension of the three-dimensional feature vector in the final fused feature tensor, and a face embedding feature vector is output.

7. The method of claim 6, wherein the method is a fast moving face recognition method based on multi-frame image enhancement. The feature compression mechanism specifically comprises: The three-dimensional feature vector in the final fused feature tensor is input into one or more groups of depth separable convolution layers, spatial convolution is independently performed on each channel, and then feature reorganization and dimension reduction are performed on the channel dimension by point-by-point convolution; The feature tensor output by the depth separable convolution is flattened into a one-dimensional vector, which is input into a fully connected dimension reduction layer to further compress the feature dimension to a predetermined two-dimensional feature. A feature distillation unit is arranged, the main network two-dimensional feature is aligned with the one-dimensional feature of the teacher network by introducing a distillation loss function; The entire feature compression mechanism is jointly optimized by a main loss function and a distillation loss in an end-to-end training process, and finally outputs a face embedding feature vector.

8. The method of claim 1, wherein the method is a multi-frame image enhancement based fast moving face recognition method. The similarity comparison comprises the following steps: The face embedding feature vector is compared with a template feature pre-stored in a database in terms of similarity, and the similarity comparison adopts cosine similarity comparison; When the similarity is greater than a first threshold, the corresponding recognition identity and confidence prompt are output; when the similarity is between the first threshold and a second threshold, "uncertain" and the similarity value are output; and when the similarity is lower than the second threshold, an "unmatched" prompt is output.

9. A fast moving face recognition system based on multi-frame image enhancement, performing the fast moving face recognition method based on multi-frame image enhancement of any one of claims 1 to 8, characterized in that, The system comprises the following modules: A multi-frame image acquisition and preprocessing module is configured to acquire a multi-frame image sequence by using an imaging device, and to perform denoising and normalization on the acquired multi-frame image sequence to output preprocessed multi-frame images; A three-dimensional information modeling module is configured to perform spatial alignment and dynamic structure analysis on the preprocessed multi-frame images to obtain aligned multi-frame images and three-dimensional structure information. The feature enhancement module is configured to encode the aligned multi-frame images and the three-dimensional structure information, separate high and low frequency multi-scale features, and perform adaptive enhancement to output enhanced multi-scale features. The spatio-temporal attention fusion module is configured to weight and fuse the enhanced multi-scale features through a spatial attention mechanism and temporal dynamic modeling to output fused multi-frame feature representations. The recognition network module is configured to receive the fused multi-frame feature representations, perform normalization and channel adaptation at the front end, combine a three-dimensional structure information embedding mechanism in the backbone network, and perform deep feature compression processing to output a face embedding feature vector. The similarity comparison and recognition output module is configured to compare the face embedding feature vector with template features in a database in terms of similarity, and output a recognition result and a confidence prompt according to a preset threshold.

Images