Low-resolution traffic video face super-resolution reconstruction and recognition method

Abstract

The invention discloses a low-resolution traffic video face super-resolution reconstruction and recognition method, and relates to the technical field of face recognition. Performing space-time semantic fusion preprocessing on the low-resolution video frame containing the human face and structured data such as a license plate and a vehicle; extracting local, global and context features of the face by adopting a dynamic attention mechanism; carrying out super-resolution reconstruction by utilizing a generative adversarial network in combination with various constraints; the reconstructed face features are fused with the structured data, and a graph convolutional network and a twin neural network are used for recognition; and the recognition threshold is dynamically adjusted according to the scene complexity, and adaptive decision is realized. According to the method, multi-modal data are fused, dynamic attention and multi-constraint reconstruction are utilized, and image convolution recognition and self-adaptive decision are combined, so that the quality and recognition accuracy of a low-resolution traffic video face image are improved, the system robustness is enhanced, and efficient and reliable traffic scene face analysis is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos. Background Art

[0002] With the rapid development of intelligent transportation systems, traffic video surveillance has become a vital component of urban management. In traffic scenarios, facial images often suffer from poor quality due to factors such as limited resolution of monitoring equipment, long shooting distances, poor lighting conditions, and motion blur, posing significant challenges to subsequent recognition and analysis. Traditional facial super-resolution reconstruction methods primarily focus on improving the quality of single images, but fail to fully consider the specific characteristics of traffic scenarios, such as the correlation of multimodal data, high real-time requirements, and complex environmental interference. This results in limited reconstruction results and makes it difficult to meet practical application requirements.

[0003] Existing technologies have many shortcomings when processing faces in low-resolution traffic videos. On the one hand, traditional methods do not effectively utilize structured data in traffic scenes, such as license plate information, vehicle characteristics, GPS positioning, etc. This data is potentially associated with facial information and can provide important supplementary clues for face reconstruction and recognition. On the other hand, existing super-resolution models lack robustness in complex traffic environments and have poor handling capabilities for problems such as illumination changes, motion blur, and occlusion. This results in loss of details in the reconstructed facial image and inaccurate feature extraction, ultimately affecting recognition accuracy. In addition, current methods are difficult to meet the requirements of large-scale traffic monitoring systems in terms of computational efficiency and real-time performance. The high model complexity and high deployment costs limit the practical application of the technology.

[0004] In recent years, deep learning technology has made significant progress in image super-resolution. However, further optimization is still needed to address the unique characteristics of faces in traffic videos. Facial images in traffic scenes are characterized by low resolution, multi-angle, dynamic changes, and are also affected by factors such as lighting, weather, and occlusion. Traditional super-resolution methods struggle to effectively address these challenges. Therefore, developing efficient and robust super-resolution reconstruction and recognition methods for faces in low-resolution traffic videos, incorporating prior knowledge of traffic scenes, has become a research hotspot in the field of intelligent transportation. Summary of the Invention

[0005] The present invention proposes a method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos to solve the problems mentioned in the above-mentioned prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos, comprising:

[0007] Spatiotemporal semantic fusion preprocessing step: constructing multimodal tensors in is the tensor product operation, It is the feature splicing operation;

[0008] Dynamic attention pyramid feature extraction step: Design spatiotemporal attention mechanism, through formula A st =α s A space +α t A time Calculate the spatiotemporal attention weights, where α s +α t =1;

[0009] Constrained resolution reconstruction step: build a generative adversarial network and design a hybrid loss function L total =λ1L pixel +λ2L perceptual +λ3L adversarial +λ4L traffic ;

[0010] Feature fusion recognition step: Construct a heterogeneous feature graph G = (V, E) and update node features through a graph convolutional network

[0011] Adaptive threshold decision step: Dynamically adjust the recognition threshold τ according to scene requirements adaptive =τ base +Δτ.

[0012] Furthermore, it also includes:

[0013] Motion blur compensation steps: estimate motion vector through spatiotemporal continuity analysis and design blur kernel estimation network K est =Net blur (F t , F t-1 , F t+1 ), deblurring is performed through Wiener filtering.

[0014] Lighting robustness enhancement steps: extract the local binary pattern LBP(F) of the face area and calculate the lighting robustness score; design an adaptive histogram equalization function H adjust =AHE(F,α·R illum +β), where α and β are light adjustment parameters.

[0015] Furthermore, in the motion blur compensation step, the formula Estimate motion vectors, where weights

[0016] Furthermore, in the feature fusion recognition step, a dual-branch attention fusion network is used to obtain the feature fusion information through the formula F fusion=W1·(F face ⊙A face )+W2·(F context ⊙A context ) to calculate the fusion features.

[0017] Furthermore, in the illumination robustness enhancement step, the formula Compute the lighting robustness score.

[0018] Furthermore, in the adaptive threshold decision step, a confidence calibration mechanism is introduced, through the formula S calib =S·exp(-λ·σ(S)) to calibrate the similarity score, where σ(S) is the standard deviation of the score distribution.

[0019] Furthermore, it also includes:

[0020] Multi-scale feature pyramid network: building a deep separation convolutional pyramid and designing a feature fusion mechanism where α i is the adaptive weight.

[0021] Federated learning deployment steps: Deploy the client model M on the edge node edge , the central server deploys the global model M center ; Design federated learning aggregation strategy

[0022] Real-time performance optimization steps: Design an adaptive model compression strategy by calculating the neuron importance score Prune redundant connections; use quantization-aware training to quantize 32-bit floating-point weights to 8-bit integers.

[0023] Compared with the existing technology, the beneficial effects of the present invention are:

[0024] Through a spatiotemporal semantic fusion preprocessing step, facial features are deeply integrated with multimodal data such as license plates, vehicles, and GPS, fully leveraging the contextual information in traffic scenes to effectively improve the accuracy of face reconstruction and the reliability of recognition. This method can automatically associate multi-source data in the same spatiotemporal space, providing richer information support for facial analysis and enhancing the system's robustness, especially in low-quality image conditions.

[0025] The introduction of a dynamic attention pyramid feature extraction mechanism enables the model to adaptively focus on key facial regions and temporal feature changes. By dynamically adjusting spatiotemporal attention weights, it highlights facial features critical for recognition while suppressing background noise. This mechanism is particularly well-suited for traffic monitoring scenarios with varying facial poses and complex lighting conditions, improving the effectiveness and relevance of feature extraction.

[0026] The multi-constrained super-resolution reconstruction step combines pixel-level, perceptual-level, and adversarial losses, and innovatively introduces traffic scene constraints. This makes the reconstructed facial images not only more visually clear and natural, but also retains more detailed features that aid recognition. This method excels in handling motion blur and uneven lighting, significantly improving the quality of low-resolution facial images and providing more reliable input for face recognition.

[0027] The multimodal feature fusion recognition step constructs a heterogeneous feature map, organically combining facial features with traffic scene information. This fusion approach not only improves recognition accuracy but also empowers the system with stronger generalization capabilities. In practical applications, it can effectively reduce false positives and missed recognitions, demonstrating significant advantages in complex traffic environments.

[0028] An adaptive threshold decision mechanism dynamically adjusts the recognition threshold based on scene complexity. Combined with confidence calibration and risk assessment, this mechanism further improves the reliability of recognition results. This mechanism automatically adapts to changing traffic scenarios, effectively reducing false alarm rates while ensuring recognition accuracy, providing more reliable technical support for traffic management and safety monitoring.

[0029] The method in this application also boasts excellent scalability and deployment flexibility. Through a federated learning framework, it enables distributed model training and updates while protecting data privacy, making it suitable for large-scale traffic monitoring networks. Furthermore, the application of model compression and quantization techniques reduces computing resource requirements and improves the system's real-time processing capabilities, enabling efficient operation on edge devices and promising broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic block diagram of the low-resolution traffic video face super-resolution reconstruction and recognition method proposed by the present invention;

[0031] Figure 2 This is a schematic diagram showing the comparison of face reconstruction quality of the low-resolution traffic video face super-resolution reconstruction and recognition method proposed in the present invention;

[0032] Figure 3 This is a schematic diagram comparing the error rates of the low-resolution traffic video face super-resolution reconstruction and recognition methods proposed in the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0035] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below with reference to the accompanying drawings.

[0036] Reference Figure 1 and Figure 3 :A specific implementation method of a low-resolution traffic video face super-resolution reconstruction and recognition method.

[0037] Taking a city traffic monitoring system upgrade project as an example, the specific implementation process of this patented method is explained in detail.

[0038] Spatiotemporal semantic fusion preprocessing: Hikvision DS-2CD8647FWDV2-ZS surveillance cameras with a resolution of 2560×1440 and a frame rate of 25fps were deployed along major urban roads and intersections. The video stream parsing module extracted three frames per second of low-resolution 64×64 pixel images containing faces. Simultaneously, these cameras acquired license plate information (in GB2312 character encoding format) from the license plate recognition system, vehicle feature vectors (including 12-dimensional features such as vehicle model and color), and GPS positioning data (with sub-meter accuracy) from the Beidou positioning module.

[0039] Convert low-resolution face image into feature tensor F t , the vehicle feature vector is quantized as V t , the license plate information is encoded as the feature vector L t , GPS positioning data is converted to G t . Constructing a multimodal tensor The tensor product operation In Python's PyTorch library, feature splicing operations are implemented through the torch.tensordot() function. This is done via the torch.cat() function.

[0040] Design traffic scenario association matrix A scene =σ(W s ·T fusion +b s ), where W s is a learnable weight matrix of dimension (128, 256), b s Is a bias vector of dimension \((128,)\), and the activation function σ adopts the ReLU function. By formula Calculate the semantic association between the face and the vehicle, where the semantic distance function D sem The cosine distance is used and is calculated using the cosine_similarity() function in the Scikit-learn library.

[0041] Dynamic attention pyramid feature extraction: On an edge computing server (configured with an Intel Xeon Gold 6248R processor, an NVIDIA A100 GPU, and 128GB of memory), a three-level pyramid feature extraction network was built based on the PyTorch framework. The first layer uses a convolutional layer with a kernel size of \(3×3\) and a stride of 2 to extract local facial features F. local ; The second layer extracts global features F by combining global average pooling with 1×1 convolution global ; The third layer uses hole convolution to obtain contextual features F context .

[0042] Design the spatiotemporal attention mechanism, through formula A st =α s A space +α t A time Calculate spatiotemporal attention weights. Spatial Attention Where W space is a learnable weight matrix of dimension (64, 64); temporal attention W time Is a learnable weight matrix of dimension (32, 32). s +α t =1, adaptively adjust α through the meta-learning algorithm (using the MAML framework) s and α t The value of , the meta-learning rate is set to 0.001, and the training period is 50 epochs.

[0043] Multi-constrained super-resolution reconstruction: A generative adversarial network (GAN) was constructed on a data center server cluster. The generator uses a U-Net architecture, consisting of seven downsampling layers and seven upsampling layers, with the number of channels per layer being 64, 128, 256, 512, 1024, 1024, and 1024, respectively. The discriminator adopts a PatchGAN structure, consisting of five convolutional layers with an output dimension of \(16×16×1\).

[0044] Design the hybrid loss function L total =λ1L pixel +λ2L perceptual +λ3L adversarial +λ4L traffic +λ5L prior . The pixel loss L pixel Using mean square error MSE, the perceptual loss L is calculated in PyTorch through nn.MSELoss() perceptual Calculate MSE after extracting features based on the pre-trained VGG16 network; adversarial loss L adversarial Adopt the loss form of WGAN-GP; traffic scene constraint term L traffic By formula Calculate, where D is the Euclidean distance, It is a priori feature extracted from the historical high-resolution traffic face image library; the prior constraint In Python, the Fourier transform and inverse transform are implemented by the fftpack.fft2() and fftpack.ifft2() functions of the SciPy library. high is a high-frequency enhancement mask generated by threshold segmentation.

[0045] The Adam optimizer is used for model training. The generator learning rate is set to 0.0002, the discriminator learning rate is set to 0.0004, the training cycle is 200 epochs, and the batch size is 16.

[0046] Multimodal feature fusion recognition: Fuse the reconstructed facial features with the structured data from the preprocessing stage. Construct a joint feature vector F fusion =W f ·[F SR ; V t ;L t ]+b f , where W f is a fusion matrix of dimension \((256, 1024)\), b f is a bias vector of dimension (256,).

[0047] Construct a heterogeneous feature graph G = (V, E), where the node set V = {v face , v vehicle , v plate , v location}, the edge set E represents the relationship between nodes. Update node features through graph convolutional network The graph convolution layer adopts the GCN2 architecture and is implemented in the DGL library.

[0048] The twin neural network is used to calculate the feature similarity, through the formula Calculate the similarity score, where γ is set to 0.3 and the spatiotemporal consistency score C st It is calculated by comparing the Euclidean distance between the face position and pose of the previous and next frames.

[0049] Adaptive threshold decision: Dynamically adjust the recognition threshold τ according to the complexity of the scene adaptive =τ base +Δτ, where the base threshold τ base =0.6, Δτ=α·σ(F local )+β·ρ(V flow ), α = 0.2, β = 0.3, α = 0.2, β = 0.3, σ is the feature variance, and ρ is the optical flow field density (calculated by the Farneback optical flow algorithm).

[0050] Introduce the confidence calibration mechanism, through the formula S calib =S·exp(-λ·σ(S)) to calibrate the similarity score, where λ=0.5. Construct a risk assessment function When R risk >τ risk = 0.1, triggering the secondary verification process, which uses a more complex feature matching algorithm.

[0051] Implementation effect data characterization

[0052]

[0053] The above data fully demonstrates the advantages of the method of this application. In terms of face reconstruction quality, the significant improvement in PSNR and SSIM indicators intuitively shows that the reconstructed face image far exceeds the traditional method in clarity and structural similarity, and can better restore the details of the real face. The recognition accuracy rate increased by 32.5% and the false recognition rate decreased by 82.9%, indicating that the multimodal feature fusion and adaptive decision-making mechanism effectively enhanced the system's ability to recognize faces in complex traffic scenarios and reduced misjudgments. The average processing delay was reduced by 78.6%, thanks to the application of model optimization and edge computing technology, enabling the system to meet the real-time requirements of traffic monitoring. In practical applications, the method of this application effectively improves the level of urban traffic security, helps traffic management departments to identify target persons more accurately, while reducing labor costs and the risk of misjudgment, and has important social and economic value.

[0054] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos, characterized by: include: Spatiotemporal semantic fusion preprocessing step: constructing multimodal tensors in is the tensor product operation, It is the feature splicing operation; Dynamic attention pyramid feature extraction step: Design spatiotemporal attention mechanism, through formula A st =α s A spoce +α t A time Calculate the spatiotemporal attention weights, where α s +α t =1; Constrained resolution reconstruction step: build a generative adversarial network and design a hybrid loss function L total =λ1L pixel +λ2L perceptual +λ3L adversarial +λ4L traffic ; Feature fusion recognition step: Construct a heterogeneous feature graph G = (V, E) and update node features through a graph convolutional network Adaptive threshold decision step: Dynamically adjust the recognition threshold τ according to scene requirements adaptive =τ base +Δτ.

2. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1 is characterized in that: Also includes: Motion blur compensation steps: estimate motion vector through spatiotemporal continuity analysis and design blur kernel estimation network K est =Net blur (F t ,F t-1 ,F t+1 ), deblurring is performed through Wiener filtering.

3. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1 is characterized in that: Also includes: Lighting robustness enhancement step: extract the local binary pattern LBP(F) of the face area and calculate the lighting robustness score; Design adaptive histogram equalization function H adjust =AHE(F,α·R illum +β), where α and β are light adjustment parameters.

4. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 2 is characterized in that: In the motion blur compensation step, the formula Estimate motion vectors, where weights 5. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1 is characterized in that: In the feature fusion recognition step, a dual-branch attention fusion network is used, and the formula F fusion =W1·(F face ☉A face )+W2·(F context ☉A context ) to calculate the fusion features.

6. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic video according to claim 3 is characterized in that: In the illumination robustness enhancement step, the formula Compute the lighting robustness score.

7. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic video according to claim 1, characterized in that: In the adaptive threshold decision step, a confidence calibration mechanism is introduced, and the confidence calibration mechanism is calculated by formula S calib =S·exp(-λ·σ(S)) to calibrate the similarity score, where σ(S) is the standard deviation of the score distribution.

8. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1, characterized in that: Also includes: Multi-scale feature pyramid network: building a deep separation convolutional pyramid and designing a feature fusion mechanism where α i is the adaptive weight.

9. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1, characterized in that: Also includes: Federated learning deployment steps: Deploy the client model M on the edge node edge , the central server deploys the global model M center ; Design federated learning aggregation strategy 10. The method for super-resolution reconstruction and recognition of faces in low-resolution traffic videos according to claim 1, characterized in that: Also includes: Real-time performance optimization steps: Design an adaptive model compression strategy by calculating the neuron importance score Prune redundant connections; Quantization-aware training is used to quantize 32-bit floating-point weights into 8-bit integers.

Images