A face recognition method and system suitable for smart city

Abstract

The application discloses a face recognition method and system suitable for a smart city, and relates to the technical field of face recognition. The method comprises the following steps: collecting a face image, performing data labeling on the face image, and constructing a face data set; performing preprocessing on the face data set to obtain a preprocessed face data set, performing illumination enhancement on the preprocessed face data set by using a multi-scale retinal cortex enhancement algorithm, and obtaining an illumination-enhanced face data set; constructing a face recognition model based on a lightweight local occlusion face recognition algorithm, training the face recognition model by using the illumination-enhanced face data set, and obtaining a trained face recognition model; obtaining a real-time face image, performing preprocessing and illumination enhancement on the real-time face image, obtaining an illumination-enhanced real-time face image, inputting the illumination-enhanced real-time face image into the face recognition model, and obtaining a face recognition result. The face recognition result is generated by deep learning, and the face recognition is realized.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, specifically to a facial recognition method and system suitable for smart cities. Background Technology

[0002] With the continuous advancement of smart city construction, facial recognition technology has been widely applied in core scenarios such as access control, public security, and traffic checkpoints, becoming a key support for identity verification and intelligent management. In these scenarios, facial images often face partial occlusion due to masks, sunglasses, scarves, etc., and need to be adapted to the lightweight deployment requirements of edge devices (such as embedded cameras and portable access control terminals). Therefore, dual requirements are placed on the occlusion robustness and computational efficiency of the recognition model. It must accurately extract effective identity features in complex occlusion scenarios while controlling the number of model parameters and inference latency to meet the demands of real-time applications.

[0003] Traditional facial recognition methods are primarily based on deep learning with fixed convolutional kernels. This method constructs a convolutional neural network, uses fixed convolutional kernels to extract features from the input facial image, and then performs identity verification. The core idea is to learn general facial features through stacking deep networks.

[0004] However, this deep learning method based on fixed convolution kernels only relies on the general feature learning ability of fixed convolution kernels and cannot dynamically adjust the feature extraction weights for local occlusion scenarios. As a result, invalid features in the occluded area excessively interfere with valid identity information, which cannot meet the recognition accuracy requirements in local occlusion scenarios in smart cities. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a facial recognition method and system suitable for smart cities, thereby resolving the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a facial recognition method and system suitable for smart cities, comprising the following steps: Step S1: Acquire facial images, annotate the facial images, and construct a facial dataset; Step S2: Preprocess the facial dataset to obtain a preprocessed facial dataset, and use a multi-scale retinal cortex enhancement algorithm to enhance the illumination of the preprocessed facial dataset to obtain an illuminated facial dataset. Step S3: Construct a facial recognition model based on a lightweight partial occlusion face recognition algorithm, and train the facial recognition model using the illumination-enhanced facial dataset to obtain a trained facial recognition model; Step S4: Acquire a real-time facial image and perform preprocessing and illumination enhancement to obtain an illumination-enhanced real-time facial image. Input the illumination-enhanced real-time facial image into the trained facial recognition model to obtain the facial recognition result.

[0007] Preferably, acquiring facial images and annotating the facial images includes: Facial images are collected, and the sources include two parts: images obtained from multiple public facial databases and image fragments extracted from real-world smart city scenarios. After data collection, data annotation is performed, including identity annotation and attribute annotation. Identity annotation is completed by annotators on a dedicated annotation platform, and each facial image ultimately corresponds to an identity tag. ; After completing the identity labeling, implement the attribute labeling process; first, label whether there are any occlusions, i.e., occlusion label vectors. Secondly, mark the head posture. ; The face region is divided into N sub-regions based on facial landmarks, and weights are assigned accordingly. For the v-th person's face region in the facial image, determine the proportion of pixels that are occluded. To calculate the facial occlusion rate The calculation formula is as follows: ; Where O is the face occlusion rate; It represents the total number of sub-regions that the face is divided into; It is a facial region index; It is the weight of the v-th sub-region; It is the occlusion ratio of the vth sub-region.

[0008] Preferably, constructing a facial dataset includes: The final constructed face dataset D can be formalized as a collection of... The sample consists of three triplet pairs: ; Where n is the sample index; This is the nth facial image; For the identity label of the nth facial image, the occlusion label vector is... Head posture and facial occlusion rate Together they form the attribute annotation set of the nth facial image.

[0009] Preferably, the facial dataset is preprocessed to obtain a preprocessed facial dataset, which includes: For all facial images in the facial dataset D Preprocessing is performed, which includes noise suppression, face alignment, and background cropping. Noise suppression employs a nonlocal mean filtering algorithm to obtain a denoised facial image; Locate facial feature points in the denoised facial image, align the facial feature points to a standard facial template, and output the aligned facial image; Background cropping is based on the aligned face image, resulting in a preprocessed face image. ; Ultimately based on preprocessed facial images Output preprocessed face dataset .

[0010] Preferably, the preprocessed facial dataset is illuminated using a multi-scale retinal cortex enhancement algorithm to obtain an illuminated facial dataset comprising: The preprocessed face dataset was processed using Gaussian filters of three different scales. Preprocessed facial images Convolution is performed to separate the illumination components containing different spatial frequency information. Based on illumination components Calculate the reflection component in the logarithmic domain ; Calculate each preprocessed facial image global average brightness Based on global brightness average For the reflection component Adaptive enhancement is performed to obtain the enhanced reflection component. The formula is as follows: ; in, It is a dynamic threshold; It is the shadow enhancement factor; It is the offset of the dark area; It is the highlight protection factor; It is the offset of the bright areas; Enhanced reflection components at three scales Weighted fusion is performed based on their scale characteristics to obtain the weighted reflection components. For weighted reflection components By applying range constraints, the final reflection vector is obtained. ; the final reflected component By combining the illumination component and transforming it back to the linear domain, an illumination-enhanced facial image is obtained. Ultimately, facial images are enhanced based on illumination. Output illumination-enhanced face dataset .

[0011] Preferably, the facial recognition model constructed based on a lightweight partial occlusion face recognition algorithm includes: A facial recognition model is constructed by integrating the lightweight partial occlusion face detection algorithm LMFD and the lightweight partial occlusion face recognition algorithm LMFR. Enhanced facial images with lighting Perform feature extraction and output shallow features. and deep features Based on shallow features with deep features Calculate the feature difference Based on feature difference Channel attention weights are calculated using global average pooling and fully connected layers. Spatial attention weights are obtained through 7×7 convolution. Based on feature difference Channel attention weight Spatial attention weights Calculate the enhanced detail features and the original shallow features Residual connections are used to obtain the final output feature map of the lightweight partial occlusion face detection submodule. ; Use the final output feature map Predict the center point heatmap, width, height, and offset, based on the predicted center point heatmap. Calculate the center point loss Based on predicted width and height and offset Calculate width and height loss and offset loss ; Loss at the center point Width and height loss and offset loss Weighted summation yields the detection loss ; The detection head outputs the face bounding box B; from the final output feature map Extract the face bounding box feature map corresponding to face bounding box B. Extracted face bounding box feature map As input to the lightweight partial occlusion face recognition submodule; facial bounding box feature map Input the lightweight partial occlusion face recognition submodule and output the final feature vector. .

[0012] Preferably, the lightweight partial occlusion face recognition submodule includes: The Lightweight Partial Occlusion Face Recognition Submodule (LMFR) performs identity determination based on detection, taking the face bounding box feature map output by the Lightweight Partial Occlusion Face Detection Submodule as input. ; Global average pooling is performed to obtain global pooling features. Then global pooling features Attention weights are generated after a fully connected layer and a Softmax layer. Based on attention weights Generate dynamic convolution kernels and dynamic convolution bias Based on dynamic convolution kernel and dynamic convolution bias Obtain convolution output ; After the dynamic convolution operation, the convolution output is... The high-level feature map is obtained by applying a non-linear activation function. ; for high-level feature maps Perform a 1×1 convolution to obtain the final feature map. ; for the final feature map For each channel u, calculate the average of the activation values ​​at all spatial locations (h, w). The formula is as follows: ; H and W are the final feature maps. The space's height and width; It is the final feature map coordinate index, It refers to the position in the vertical direction. It refers to the position in the width direction. It is the channel number; After applying this operation to all d channels, the d average values ​​will be... The features are concatenated to obtain the final feature vector. ; The recognition loss is calculated based on the final feature vector f. .

[0013] Preferably, the face recognition model is trained using the illumination-enhanced face dataset to obtain a trained face recognition model, including: The training process uses an illumination-enhanced facial dataset. Perform end-to-end training; Simultaneous optimization of detection loss during training and identification loss Total loss The weighted sum for detecting and identifying losses is formulated as follows: ; in, It is the detection loss weight; It is to identify the loss weight; The face recognition model's ability to generalize to unknown occlusions is improved by using data augmentation strategies. The training cycle is about 200 rounds until the loss converges; the final output is the trained face recognition model.

[0014] Preferably, the real-time illuminated facial image is input into the trained facial recognition model to obtain facial recognition results, including: Acquire real-time facial images, perform preprocessing and illumination enhancement to obtain illuminated real-time facial images; The real-time illuminated facial image is input into the trained facial recognition model to obtain the real-time final feature vector. ; The real-time final feature vector Compared with all J template feature vectors pre-stored in the database Perform comparisons and calculate cosine similarity sequentially. The calculation formula is: ; in, It is the j-th cosine similarity; It is the real-time final feature vector extracted from real-time images; It is the j-th template feature vector; Represents the L2 norm; Select the highest similarity from the J cosine similarities. And record its corresponding identity index. ; The highest similarity With preset threshold Compare; if ≥ Then the identity is determined by the corresponding template feature vector. Associated identity Otherwise, mark it as "unknown"; Finally, output the facial recognition result; if the highest similarity is... ≥ Then it is determined to be the corresponding identity. It outputs the identity label and the highest similarity score; otherwise, it is judged as an "unknown" identity.

[0015] A facial recognition system suitable for smart cities includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0016] Beneficial effects This invention provides a facial recognition method suitable for smart cities, involving machine learning and deep learning technologies, which has the following beneficial effects: (1) By constructing a facial recognition model based on the Lightweight Partial Occlusion Face Recognition (LMFR) algorithm, the model's parameter count and inference latency can be minimized while ensuring occlusion recognition accuracy. The LMFR algorithm, through its lightweight design of depthwise separable convolution and dynamic convolution, is fully adaptable to the computing power limitations of smart city edge devices and can achieve real-time response without relying on high-performance hardware. At the same time, the algorithm has been specifically optimized for partial occlusion scenarios such as masks and sunglasses, which can effectively deal with occlusion problems in high-frequency scenarios such as smart city access control, monitoring, and attendance. It not only meets the recognition reliability in complex environments but also reduces the equipment deployment and maintenance costs, demonstrating strong engineering practicality.

[0017] (2) By introducing a lightweight partial occlusion face detection and recognition architecture into the facial recognition model, and integrating an occlusion perception detection submodule and a dynamic convolutional recognition submodule, a closed-loop optimization of the entire process is achieved. The occlusion perception detection submodule is based on the anchorless box idea and strengthens the feature extraction of key unoccluded areas through a feature attention mechanism, which can accurately filter out invalid interference from occluded areas; the dynamic convolutional recognition submodule embeds an attention mechanism at the convolutional kernel channel level, which can dynamically adjust the weight allocation according to the occlusion type and focus on effective identity features. The two modules work together to solve the problem of excessive reliance on occluded area features in traditional models, and avoid the blindness of feature extraction, thereby improving the recognition accuracy in complex occlusion scenarios and significantly enhancing the robustness of the facial recognition model to diverse occlusion scenarios in smart cities.

[0018] (3) By using the multi-scale Retinex enhancement algorithm to enhance the lighting of the preprocessed facial dataset, the problem of loss of facial image details caused by different lighting scenarios in smart cities can be effectively solved. The illumination component and reflection component of the image are separated by three-scale Gaussian filtering, and the enhancement intensity of dark and bright parts is dynamically adjusted by adaptive piecewise logarithmic transformation. This can restore facial texture details in low-light scenarios, avoid detail saturation in overexposed areas, and accommodate the local enhancement needs of occluded scenarios. The image processed by this algorithm has uniform brightness and clear features, reduces the recognition error caused by uneven lighting, provides high-quality input data for subsequent feature extraction, and ensures that the facial recognition model can maintain high accuracy in the complex lighting environment of smart cities. Attached Figure Description

[0019] Figure 1 This is a flowchart of a facial recognition method suitable for smart cities proposed in this invention.

[0020] Figure 2This invention provides a hierarchical map of an illumination-enhanced facial dataset for a facial recognition method suitable for smart cities.

[0021] Figure 3 This invention presents a hierarchical diagram of facial recognition results obtained using a facial recognition method suitable for smart cities. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figures 1-3 This invention provides a technical solution: a facial recognition method suitable for smart cities. Specifically, the following facial recognition method suitable for smart cities is provided; please refer to [link / reference]. Figure 1 The method includes the following steps: Step S1: Collect facial images, annotate the facial images, and construct a facial dataset.

[0024] The main goal of this step is to build a facial dataset that can support robust facial recognition models in complex urban environments.

[0025] First, facial images are collected. The data sources consist of two parts: images obtained from multiple publicly available facial databases, and image fragments extracted from real-world smart city scenarios (such as privacy-de-identified surveillance footage of public places). To ensure data diversity, a target total number of images is set for the collection. And ensure that these facial images cover M different lighting conditions (such as sunlight, shadow, backlight, and low light at night) and K different acquisition environments (such as subway turnstiles, community access control, and road checkpoints).

[0026] After data collection, data annotation is performed, including identity annotation and attribute annotation. Identity annotation is completed by trained annotators on a dedicated annotation platform. Each facial image is assigned to at least two annotators for independent annotation, determining the corresponding real identity based on a clear dominant face in the image. A unique identifier is assigned to each individual identity. (Where j = 1, 2, ..., J, and J is the total number of identities). The annotation results are cross-checked, and in case of discrepancies, senior annotators arbitrate to ensure that each facial image corresponds to an accurate and consistent identity label. .

[0027] Building upon the identity labeling, the attribute labeling process is implemented in parallel, which is crucial for supporting the subsequent invention of "occlusion detection and feature reconstruction." The labelers need to perform fine-grained attribute judgment and labeling on the faces in each facial image. First, the presence of specific occlusions is identified by defining a set of binary label vectors, i.e., occlusion label vectors. If there is a corresponding obstruction, the corresponding obstruction label is 1; otherwise, it is 0. For example, if a mask is present, the mask label... The value is 1. Next, the head pose is labeled. They are categorized into discrete categories such as "looking straight ahead", "turning left", "turning right", "looking up", and "looking down".

[0028] To quantify the degree of occlusion, the aligned face region is divided into N sub-regions of different importance (such as left eye, right eye, nose, mouth, etc.) based on facial landmarks, and each region is assigned a predefined weight. (satisfy The weight values ​​are determined through prior statistics based on the contribution of the region to identity recognition. For the v-th person's face region in the facial image, the proportion of occluded pixels is determined through pixel-level segmentation or estimation by the annotator. The facial occlusion rate of the entire facial image It is calculated using the following formula: ; Where O is the face occlusion rate, and the value range is [0,1]. 0 represents no occlusion and 1 represents complete occlusion. It represents the total number of sub-regions that the face is divided into; It is a facial region index; It is the weight of the v-th sub-region; This represents the occlusion percentage of the v-th sub-region. This metric provides a continuous and perceptually important supervision signal for occlusion intensity during subsequent model training.

[0029] The final constructed face dataset D can be formally defined as consisting of... The sample consists of three triplet pairs: ; Where n is the sample index; This is the nth facial image; For the identity label of the nth facial image, the occlusion label vector is... Head posture and facial occlusion rate Together they form the attribute annotation set of the nth facial image.

[0030] This facial dataset D comprehensively covers different lighting, occlusion, and pose scenarios, providing crucial and diverse sample support for training facial recognition models that can adapt to the complex real-world application scenarios of smart cities.

[0031] Step S2: Preprocess the facial dataset to obtain a preprocessed facial dataset, and use a multi-scale retinal cortex enhancement algorithm to enhance the illumination of the preprocessed facial dataset to obtain an illuminated facial dataset.

[0032] Based on the face dataset D constructed in step S1, this step sequentially performs preprocessing and illumination enhancement to eliminate image noise, standardize faces, and compensate for illumination variations.

[0033] First, analyze all facial images in the facial dataset D. Preprocessing is performed, including noise suppression, face alignment, and background cropping. Noise suppression employs a nonlocal mean filtering algorithm to reduce random noise in the image while preserving edge details. The nonlocal mean filtering algorithm achieves noise reduction by calculating similarity weights between pixel neighborhoods; the specific formula is as follows: ; Where w(p,q) represents the similarity weight between pixels p and q, which is used to measure their similarity; and These represent 5×5 pixel neighborhood blocks centered at pixels p and q, respectively, containing local pixel value information; is the square of the L2 norm, used to calculate the difference between two neighboring blocks; h is the filter strength parameter (usually set to 12 for a better balance between preserving edge details and suppressing noise). The algorithm sets the search window size to 15×15 pixels (optimized for 128×128 face images) to ensure comparison of similar blocks over a wide range, effectively reducing the standard deviation of Gaussian noise by about 40%, thus generating a denoised face image.

[0034] Next, face detection and alignment are performed. This involves real-time detection of facial landmarks and standardization of the denoised facial image to a unified template, eliminating pose and scale variations. First, a dlib68 point detector is used to locate facial landmarks (such as the corners of the eyes, nose, and mouth) in the denoised facial image. The detector predicts the coordinates of these landmarks based on a trained regression model. After detection, a geometric transformation matrix is ​​computed to align the detected facial landmarks to a predefined standard face template (such as the average face template). The transformation is implemented using an affine transformation, and its matrix form is as follows: ; Where (x, y) are the coordinates of the denoised facial image; , ) represents the transformed coordinates; A is a 2×3 affine transformation matrix, controlling rotation, scaling, and shearing parameters; b is a 2×1 translation vector. The matrix parameters are obtained by fitting the correspondence between the detection points and the standard points using the least squares method, ensuring consistent positions of the aligned facial feature points. The alignment process also references the head pose annotation in step S1. (e.g., "turn left" or "tilt head"), dynamically adjust transformation parameters (e.g., increase rotation angle) for images with large pose deviations to improve the standardization effect. This step outputs an aligned face image, laying the foundation for subsequent cropping.

[0035] Background cropping determines the cropping region based on the aligned face image. First, the minimum bounding rectangle of all facial feature points is calculated, which tightly encloses the face region. Then, the rectangle's boundary is expanded to preserve contextual information (such as hairstyle). The expansion ratio is dynamically adjusted based on the occlusion label vector T(n) in step S1, with the following specific expansion rules: In the absence of occlusion, each side of the rectangle expands by 15%; with hat occlusion, the top expands by 25%, and other sides by 15%; with scarf occlusion, the bottom expands by 25%, and other sides by 15%; for multiple occlusions, the maximum expansion ratio in each direction is used. The cropped image is then the preprocessed facial image. Meanwhile, the image size is standardized to a fixed size (e.g., 128×128 pixels) to ensure data consistency. The cropping process directly removes non-face backgrounds, reducing computational redundancy, and retains the attribute information (such as occlusion rate) marked in step S1 for subsequent processing.

[0036] The preprocessing stage ultimately outputs a preprocessed face dataset. Each sample contains a preprocessed image, identity labels, and all attribute annotations inherited from step S1, formally defined as: ; in, Indicates facial image The preprocessed facial image obtained after noise suppression, face alignment and background cropping.

[0037] Next, the preprocessed facial dataset Illumination enhancement is performed. A multi-scale retinal cortex enhancement algorithm is employed to compensate for uneven illumination in the preprocessed facial image. First, three Gaussian filters of different scales (scale parameters are respectively) are applied. =20, =60, =120) for preprocessed face datasets Preprocessed facial images Convolution is performed to separate the illumination components containing different spatial frequency information. Illumination components The calculation formula is: ; in, It is the illumination component at the c-th scale; It is a preprocessed facial image The pixel intensity value at coordinates (x, y); This represents the convolution operation; The standard deviation is The two-dimensional Gaussian kernel function.

[0038] Subsequently, based on the illumination component Calculate the reflection component in the logarithmic domain The formula is as follows: ; in, It is the preprocessed facial image at the c-th scale. The reflection component at coordinates (x, y); It is the light component.

[0039] The reflection component is considered to contain essential information about an object's properties, but it often suffers from blurred details under low-light conditions. To enhance details in low-light environments, adaptive enhancement of the reflection component is necessary. First, the reflection component is calculated for each preprocessed facial image. global average brightness As a quantitative indicator of its illumination intensity, the preprocessed facial image is used. After converting to grayscale, the arithmetic mean of the intensities of all pixels is calculated to obtain the global average brightness. .

[0040] Based on global brightness average For the reflection component Adaptive enhancement is performed to obtain the enhanced reflection component. The enhancement process uses a global brightness average. and the preprocessed historical facial images Occlusion rate The piecewise logarithmic transformation function driven by the common factors is as follows: ; in, It is a dynamic threshold. , The initial threshold is set based on experience (for example, 30% of the maximum pixel value of 255, i.e.) =76.5), k is an adjustment coefficient greater than 0 (for example, set k=1.0). This is the shadow enhancement factor, with an initial value of 2.0, which increases with the global brightness average. The increments are linearly decreasing in steps of 0.015. It is the shadow offset. =150- ; This is the brightness protection factor, with an initial value of 0.8, which varies with the global brightness average. The increments linearly in increments of 0.005. It is the offset of the highlights. =30+0.5 Meanwhile, regarding occlusion rate In areas with higher occlusion (such as the mouth and nose covered by the mask), the intensity will be locally adjusted to better restore details, as follows: for occlusion rate Preprocessed facial images with a value >0.3, in the corresponding occlusion regions (determined by the occlusion label vector). (Location) Perform local enhancement adjustments: In areas obscured by the mask (such as obscuring label vectors) (The area marked with the nose and mouth) will have its shadow enhancement factor increased. An additional 0.2 is added; in the area obscured by the glasses, the shadow enhancement factor is increased. Increase by 0.1 to enhance the restoration of eye details.

[0041] Finally, the enhanced reflection components at the three scales Weighted fusion is performed based on their scale characteristics to obtain the weighted reflection components. The specific formula is as follows: ; in, , and These are multi-scale fusion weights, corresponding to three different scale parameters. =0.2 ( =20), =0.5 ( =60), =0.3 ( =120). To ensure that the enhanced pixel values ​​are within the effective range, the weighted reflection component is adjusted. By applying range constraints, the final reflection vector is obtained. .

[0042] Then the final reflection component By combining the illumination component and transforming it back to the linear domain, an illumination-enhanced facial image is obtained. .

[0043] The final output is an illumination-enhanced face dataset. Its definition is as follows: ; in, Preprocessed facial images Illumination-enhanced facial images obtained after applying a multi-scale retinal cortex enhancement algorithm This lighting-enhanced face dataset It effectively eliminates uneven lighting and low-light interference while preserving rich facial details, providing high-quality input for the subsequent training of facial recognition models.

[0044] Step S3: Construct a facial recognition model based on a lightweight partial occlusion face recognition algorithm, and train the facial recognition model using the illumination-enhanced facial dataset to obtain a trained facial recognition model.

[0045] The core objective of this step is to build an end-to-end face recognition model that integrates the lightweight partial occlusion face detection algorithm LMFD and the lightweight partial occlusion face recognition algorithm LMFR. By collaboratively optimizing the detection and recognition process, robustness in complex occlusion scenarios is improved. This step directly utilizes the illumination-enhanced face dataset output from step S2. As input, the lightweight partial occlusion face detection submodule first achieves accurate localization and feature extraction of occluded faces, and then the lightweight partial occlusion face recognition submodule completes identity verification.

[0046] The lightweight partial occlusion face detection submodule LMFD is responsible for enhancing facial images based on illumination. This lightweight, locally occluded face detection submodule is based on the CenterNet anchor-free bounding box concept, avoiding the complex post-processing of traditional anchor-bound methods and improving detection efficiency. Its process begins with feature extraction and ends with the detection output.

[0047] First, enhance the facial image with light. Feature extraction was performed using a hybrid architecture of the lightweight backbone network MobileNetV3-Small and ResNet to enhance facial images from illumination. Multi-scale feature maps are extracted. The first three layers employ the inverted residual structure of MobileNetV3 (few parameters, fast inference), while the last two layers use ResNet residual blocks (strong feature representation ability, preserving deep semantic information). The total number of parameters is controlled within 2.5M to meet the deployment requirements of edge devices. The backbone network reduces the number of parameters by stacking depthwise separable convolutional blocks while preserving hierarchical features. Illumination-enhanced facial images are then used. With dimensions of H×W×3, after convolution and downsampling, the output feature map scales are respectively... × ×C (shallow features) )and × ×4C (Deep Features) ), where C is the baseline number of channels (e.g., 64). These multi-scale feature maps capture information from shallow details (e.g., edge texture) to deep semantics (e.g., overall structure), laying the foundation for multi-scale detection. The feature extraction formula can be simplified to: ; Backbone For lightweight convolutional networks; It is a multi-scale feature map. The output is a shallow feature map. Deep features As input to subsequent modules, ensure that the features originate from the illumination enhancement data of step S2, and maintain the quality after illumination compensation.

[0048] Shallow features were extracted through the backbone network. and deep features Subsequently, to enhance the robustness of the facial recognition model in occluded scenarios, a feature difference attention module was introduced. This module aims to strengthen the detailed features of unoccluded key regions (such as the eyes and eyebrows) to compensate for feature loss caused by occlusion. This is achieved by calculating shallow features. with deep features Feature difference To highlight details, among which Upsampled aligned size. Feature difference. Enhance important regions using the CBAM attention mechanism (combining channel and spatial attention). Channel attention weights. Calculated using global average pooling and fully connected layers: ; in, and It is a learnable parameter matrix of two fully connected layers. , The reduction ratio r=16 balances computational efficiency and expressive power; It is global average pooling; It is the feature difference.

[0049] Spatial attention weights Generated using 7×7 convolution and Sigmoid: ; in, It is a 7×7 convolution operation used to capture a large range of spatial context.

[0050] Ultimately, the enhanced details are: and the original shallow features Residual connections are used to obtain the final output feature map of the lightweight partial occlusion face detection submodule. The final output feature map Scale is × ×C (and shallow features) (Spatial dimensions are consistent). This feature attention module extracts details using interpolation and focuses on unoccluded areas through attention weights, ultimately outputting a feature map. It can be used directly in the detection head to improve positioning accuracy under obstruction.

[0051] The detection head and loss calculation are based on an anchorless design, using the final output feature map. Predict the center point heatmap, width, height, and offset. Let... This is a real heatmap (generated using a Gaussian distribution). To predict the heatmap, the center point loss is calculated. Focal Loss was used, with hyperparameters set to α=2.0 (for hard samples) and β=4.0 (for easy samples). The ratio of positive to negative samples was automatically adjusted using the Gaussian radius to avoid class imbalance. Centroid loss was also employed. The calculation formula is: ; in, It is the value of the actual heatmap at location (x, y); This is the predicted value of the heatmap at location (x, y); α is the hard sample weight adjustment factor, α = 2.0; β is the easy sample suppression strength adjustment factor, β = 4.0; N is the number of centroids. This centroid loss... Punishing difficult samples improves detection robustness.

[0052] Predicted width and height and offset Using L1 loss and width / height loss and offset loss The specific formula is as follows: ; ; in, These are the actual width and height values; This is the actual offset value.

[0053] Detection loss For weighted sum: ; in, and It is loss weight. =0.1、 =1, balancing all terms to ensure gradient stability. Detection loss. The output guides network optimization, and its gradient is backpropagated to the feature extraction layer, forming an end-to-end training loop.

[0054] The detection head outputs a face bounding box B=(x,y,w,h), where (x,y) are the center point coordinates, and w and h are the width and height. The RoI Align operation is then used to extract the final output feature map. Extract the face bounding box feature map of the corresponding bounding box region ,Right now ,in It outputs the feature map size, which is the extracted face bounding box feature map. As input to the lightweight partial occlusion face recognition submodule, it ensures spatial alignment and scale uniformity.

[0055] The Lightweight Partial Occlusion Face Recognition Submodule (LMFR) performs identity determination based on detection, and its input is the face bounding box feature map output by the Lightweight Partial Occlusion Face Detection Submodule. (Corresponding to the cropped and aligned facial region features), the core is the combination of dynamic convolution and ArcFace loss function, focusing on the discriminative features of the unoccluded region.

[0056] First, the dynamic convolutional structure embeds an attention mechanism at the kernel channel level, dynamically adjusting weights for different occlusion types. Let's consider the face bounding box feature map. (i.e., the output features of LMFD), dynamic convolution aggregation =4 static convolutional kernels These static convolutional kernels are randomly generated using methods such as Kemin initialization during the initialization of the face recognition model, and are optimized along with other network parameters through gradient descent during end-to-end training to cover common occlusion patterns (no occlusion, mask, glasses, hat). Attention weights The computational network consists of two fully connected layers. Global average pooling is used first to obtain globally pooled features. Then it is generated through a fully connected layer and Softmax: ; in, These are the attention weights of the k-th convolutional kernel; and These are learnable parameters. , ; It is a global pooling feature. The dynamic convolution kernel is... ,in It is the k-th static convolution kernel; the dynamic convolution bias is... , It is the bias of the k-th convolution kernel; the convolution output is .

[0057] After the dynamic convolution operation, the convolution output is... It needs to be processed by a non-linear activation function (such as ReLU) to introduce non-linear expressive power, thereby obtaining a truly high-level feature map. ,Right now ,in ∈ .

[0058] After extracting high-level features through dynamic convolutional layers, the final feature map The number of channels is expanded from the baseline channel count C=64 to d=512 through 1×1 convolution to enhance the expressive power of the features. This operation can be represented as: ; in, It is a high-level feature map; It is a 1×1 convolutional layer, which expands the number of channels from C to d, to obtain ∈ .

[0059] Then the final feature map Perform global average pooling, and calculate the average of the activation values ​​at all spatial locations (h, w) for each channel u. The formula is as follows: ; Where H and W are the final feature maps. The space's height and width; It is the final feature map coordinate index, It refers to the position in the vertical direction. It refers to the position in the width direction. This refers to the channel number. Applying this operation to all d channels yields a final d-dimensional feature vector. The final feature vector f is the compact feature representation used for identity recognition. It will then be input into the fully connected layer and participate in the calculation of the recognition loss to optimize intra-class compactness and inter-class separability.

[0060] The recognition loss function adds an angular margin *m* to the Softmax function to increase the inter-class distance. Let... Let j be the weight vector of class j. The final feature vector f and weight vector The included angle, to identify the loss for: ; Where s is the scaling factor, s=64; m is the angular margin, m=0.5; The angle between the final feature vector and the weight vector is denoted by V; V is the batch size. This is the recognition loss. The feature distribution is optimized by cosine similarity, and its output gradient is backpropagated to the dynamic convolutional layer to ensure that the feature learning is consistent with the recognition target.

[0061] The training process uses the illumination-enhanced face dataset output from step S2. End-to-end training is performed. The Adam optimizer is used during training, with the following parameter settings: the initial learning rate is... Cosine annealing scheduling is used; weight decay is... (To prevent overfitting); Batch size is 64 (based on GPU memory optimization); Training cycle is 200 rounds, with an early stopping strategy (terminating when the validation set loss does not decrease for 10 consecutive rounds).

[0062] Simultaneous optimization of detection loss during training and identification loss Total loss Weighted sum for detecting and identifying losses: ; in, It is the detection loss weight. =1; It is to identify loss weights. =1, gradient normalization ensures that the gradient magnitudes of the two tasks are consistent. In the early stages of training, the focus is on detection loss ( =1.5, =0.5), with a later focus on identifying the loss ( =0.5, =1.5), to realize course learning.

[0063] Data augmentation strategies include random occlusion generation, geometric transformation, and color perturbation, where random occlusion generation is based on the occlusion label vector from step S1. Simulated occlusions were added to the corresponding areas. When the face was covered by a mask, a rectangular mask was added to the lower half of the face; when covered by glasses, an elliptical mask was added to the eye area. The occlusion probability was 0.3, and the occlusion area ratio was 0.2-0.5. The geometric transformations were random rotation ±15°, scaling [0.9, 1.1], and translation ±0.053. The color perturbations were brightness ±0.2, contrast ±0.2, and saturation ±0.2. Data augmentation strategies were used to improve the facial recognition model's generalization ability to unknown occlusions. The training cycle was approximately 200 epochs until the loss converged.

[0064] Finally, step S3 outputs the trained facial recognition model, which achieves high-precision detection and recognition in occluded scenarios, providing reliable support for smart city applications.

[0065] The inference process of a trained facial recognition model in actual deployment: Input a real-time image I, and detect the bounding boxes of all faces through the LMFD submodule. For each detected face, features are extracted using RoI Align. The LMFR submodule extracts a d-dimensional feature vector (d=512), which is the real-time final feature vector. Calculate cosine similarity with the template feature vectors in the registration library; set a threshold. =0.7, similarity greater than The identity is identified as a match; the recognition result and confidence score are output.

[0066] Step S4: Acquire a real-time facial image and perform preprocessing and illumination enhancement to obtain an illumination-enhanced real-time facial image. Input the illumination-enhanced real-time facial image into the trained facial recognition model to obtain the facial recognition result.

[0067] This step is the real-time application stage of the facial recognition model, which aims to perform fast and accurate identity recognition on the input real-time facial images.

[0068] First, acquire real-time facial images (such as streaming data captured from a camera), which may be affected by changes in ambient lighting, noise, or partial occlusion.

[0069] Next, preprocessing and illumination enhancement operations consistent with those described in step S2 are performed: including noise suppression, face detection and alignment, background region cropping, and illumination enhancement based on a multi-scale retinal cortex enhancement algorithm. These operations output an illuminated real-time facial image with quality consistent with the output of step S2, ensuring the standardization of the input data and illumination robustness.

[0070] Subsequently, the illuminated real-time facial image is input into the facial recognition model trained in step S3 to obtain the real-time final feature vector. (Where d=512). Identity determination requires the real-time final feature vector. Compared with all J template feature vectors pre-stored in the database (Each template feature vector) Corresponding to a registered identity The face recognition model trained in step S3 extracts and stores the registered images, then compares them. Specifically, the process involves sequentially calculating the real-time final feature vector. With each template feature vector Cosine similarity between Thus, J cosine similarities are obtained, calculated using the following formula: ; in, It is the j-th cosine similarity, with a value in the range [-1, 1]. The closer the value is to 1, the more similar the similarity is. It is the real-time final feature vector extracted from real-time images; is the j-th template feature vector in the database (extracted and stored by the model trained in step S3); · represents the dot product operation; This represents the L2 norm.

[0071] Then, the highest similarity is selected from the J cosine similarities. And record its corresponding identity index. The highest similarity With preset threshold ( Compare with (=0.7). If ≥ Then the identity is determined by the corresponding template feature vector. Associated identity Otherwise, it is marked as "unknown". This threshold is determined based on the ROC curve on the validation set in step S3 to balance the false recognition rate and the false recognition rate.

[0072] Finally, the facial recognition results are output. If the highest similarity score is... ≥ Then it is determined to be the corresponding identity. It outputs the identity label and the highest similarity score; otherwise, it is judged as an "unknown" identity.

[0073] Thanks to the lightweight model design in step S3 (parameter count < 2.5M) and the preprocessing optimization in step S2, the entire recognition process can be processed in real time on mainstream embedded devices (such as devices equipped with ARM Cortex-A series CPUs), with a frame rate of no less than 20 fps, meeting the smoothness requirements of real-time monitoring in smart cities.

[0074] This technical solution constructs a historical facial dataset with multi-attribute annotations including identity, occlusion, and pose. After preprocessing and multi-scale Retinex illumination enhancement to optimize data quality, a facial recognition model is built based on a lightweight LMFD+LMFR architecture. The facial recognition model integrates an occlusion perception and detection submodule and a dynamic convolutional recognition submodule. The former accurately locates key unoccluded regions, while the latter dynamically adjusts convolutional kernel weights to focus on effective features, enhanced by the ArcFace loss function. The solution balances lightweight design (<2.5M parameters) with robustness, achieving a recognition rate exceeding 90% with a 60% occlusion rate and an inference frame rate ≥20fps. It is suitable for deployment on smart city edge devices, effectively addressing pain points in low-light and partial occlusion scenarios, improving recognition accuracy and real-time performance in access control, monitoring, and other scenarios, and possesses significant engineering application value.

[0075] The present invention also protects a facial recognition system suitable for smart cities, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0076] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the phrase "comprising an element defined as..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their likenesses.

Claims

1. A facial recognition method suitable for smart cities, characterized in that, Includes the following steps: Step S1: Acquire facial images, annotate the facial images, and construct a facial dataset; Step S2: Preprocess the facial dataset to obtain a preprocessed facial dataset, and use a multi-scale retinal cortex enhancement algorithm to enhance the illumination of the preprocessed facial dataset to obtain an illuminated facial dataset. Step S3: Construct a facial recognition model based on a lightweight partial occlusion face recognition algorithm, and train the facial recognition model using the illumination-enhanced facial dataset to obtain a trained facial recognition model; Step S4: Acquire a real-time facial image and perform preprocessing and illumination enhancement to obtain an illumination-enhanced real-time facial image. Input the illumination-enhanced real-time facial image into the facial recognition model to obtain the facial recognition result.

2. The facial recognition method suitable for smart cities according to claim 1, characterized in that, Acquire facial images and perform data annotation on the facial images, including: Facial images are collected, and the sources include two parts: images obtained from multiple public facial databases and image fragments extracted from real-world smart city scenarios. After data collection, data annotation is performed, including identity annotation and attribute annotation. Identity annotation is completed by annotators on a dedicated annotation platform, and each facial image ultimately corresponds to an identity tag. ; After completing the identity labeling, implement the attribute labeling process; first, label whether there are any occlusions, i.e., occlusion label vectors. Secondly, mark the head posture. ; The face region is divided into N sub-regions based on facial landmarks, and weights are assigned accordingly. For the v-th person's face region in the facial image, determine the proportion of pixels that are occluded. To calculate the facial occlusion rate The calculation formula is as follows: ； Where O is the face occlusion rate; It represents the total number of sub-regions that the face is divided into; It is a facial region index; It is the weight of the v-th sub-region; It is the occlusion ratio of the vth sub-region.

3. The facial recognition method suitable for smart cities according to claim 2, characterized in that, Construct a facial dataset, including: The final constructed face dataset D can be formalized as a collection of... The sample consists of three triplet pairs: ； Where n is the sample index; This is the nth facial image; For the identity label of the nth facial image, the occlusion label vector is... Head posture and facial occlusion rate Together they form the attribute annotation set of the nth facial image.

4. A facial recognition method suitable for smart cities according to claim 3, characterized in that, The facial dataset is preprocessed to obtain a preprocessed facial dataset, including: For all facial images in the facial dataset D Preprocessing is performed, which includes noise suppression, face alignment, and background cropping. Noise suppression employs a nonlocal mean filtering algorithm to obtain a denoised facial image; Locate facial feature points in the denoised facial image, align the facial feature points to a standard facial template, and output the aligned facial image; Background cropping is based on the aligned face image, resulting in a preprocessed face image. ; Ultimately based on preprocessed facial images Output preprocessed face dataset .

5. A facial recognition method suitable for smart cities according to claim 4, characterized in that, The preprocessed facial dataset was illuminated using a multi-scale retinal cortex enhancement algorithm to obtain an illuminated facial dataset, including: The preprocessed face dataset was processed using Gaussian filters of three different scales. Preprocessed facial images Convolution is performed to separate the illumination components containing different spatial frequency information. Based on illumination components Calculate the reflection component in the logarithmic domain ; Calculate each preprocessed facial image global average brightness Based on global brightness average For the reflection component Adaptive enhancement is performed to obtain the enhanced reflection component. The formula is as follows: ； in, It is a dynamic threshold; It is the shadow enhancement factor; It is the offset of the dark area; It is the highlight protection factor; It is the offset of the bright areas; Enhanced reflection components at three scales Weighted fusion is performed based on their scale characteristics to obtain the weighted reflection components. For weighted reflection components By applying range constraints, the final reflection vector is obtained. ; the final reflected component By combining the illumination component and transforming it back to the linear domain, an illumination-enhanced facial image is obtained. Ultimately, facial images are enhanced based on illumination. Output illumination-enhanced face dataset .

6. A facial recognition method suitable for smart cities according to claim 5, characterized in that, A facial recognition model is constructed based on a lightweight partial occlusion face recognition algorithm, including: A facial recognition model is constructed by integrating the lightweight partial occlusion face detection algorithm LMFD and the lightweight partial occlusion face recognition algorithm LMFR. Enhanced facial images with lighting Perform feature extraction and output shallow features. and deep features Based on shallow features with deep features Calculate the feature difference Based on feature difference Channel attention weights are calculated using global average pooling and fully connected layers. Spatial attention weights are obtained through 7×7 convolution. Based on feature difference Channel attention weight Spatial attention weights Calculate the enhanced detail features and the original shallow features Residual connections are used to obtain the final output feature map of the lightweight partial occlusion face detection submodule. ; Use the final output feature map Predict the center point heatmap, width, height, and offset, based on the predicted center point heatmap. Calculate the center point loss Based on predicted width and height and offset Calculate width and height loss and offset loss ; Loss at the center point Width and height loss and offset loss Weighted summation yields the detection loss ; The detection head outputs the face bounding box B; from the final output feature map Extract the face bounding box feature map corresponding to face bounding box B. Extracted face bounding box feature map As input to the lightweight partial occlusion face recognition submodule; facial bounding box feature map Input the lightweight partial occlusion face recognition submodule and output the final feature vector. .

7. A facial recognition method suitable for smart cities according to claim 6, characterized in that, The lightweight partial occlusion face recognition submodule includes: The Lightweight Partial Occlusion Face Recognition Submodule (LMFR) performs identity determination based on detection, taking the face bounding box feature map output by the Lightweight Partial Occlusion Face Detection Submodule as input. ; Face bounding box feature map Global average pooling is performed to obtain global pooling features. Then global pooling features Attention weights are generated after a fully connected layer and a Softmax layer. Based on attention weights Generate dynamic convolution kernels and dynamic convolution bias Based on dynamic convolution kernel and dynamic convolution bias Obtain convolution output ; After the dynamic convolution operation, the convolution output is... The high-level feature map is obtained by applying a non-linear activation function. ; for high-level feature maps Perform a 1×1 convolution to obtain the final feature map. ; for the final feature map For each channel u, calculate the average of the activation values ​​at all spatial locations (h, w). The formula is as follows: ； H and W are the final feature maps. The space's height and width; It is the final feature map coordinate index, It refers to the position in the vertical direction. It refers to the position in the width direction. It is the channel number; After applying this operation to all d channels, the d average values ​​will be... The features are concatenated to obtain the final feature vector. ; The recognition loss is calculated based on the final feature vector f. .

8. A facial recognition method suitable for smart cities according to claim 7, characterized in that, The face recognition model is trained using the aforementioned illumination-enhanced face dataset to obtain a trained face recognition model, including: The training process uses an illumination-enhanced facial dataset. Perform end-to-end training; Simultaneous optimization of detection loss during training and identification loss Total loss The weighted sum for detecting and identifying losses is formulated as follows: ； in, It is the detection loss weight; It is to identify the loss weight; The face recognition model's ability to generalize to unknown occlusions is improved by using data augmentation strategies. The training cycle is 100-300 rounds until the loss converges; the final output is the trained face recognition model.

9. A facial recognition method suitable for smart cities according to claim 8, characterized in that, The enhanced real-time facial image is input into a trained facial recognition model to obtain facial recognition results, including: Acquire real-time facial images, perform preprocessing and illumination enhancement to obtain illuminated real-time facial images; The real-time illuminated facial image is input into the trained facial recognition model to obtain the real-time final feature vector. ; The real-time final feature vector Compared with all J template feature vectors pre-stored in the database Perform comparisons and calculate cosine similarity sequentially. The calculation formula is: ； in, It is the j-th cosine similarity; It is the real-time final feature vector extracted from real-time images; It is the j-th template feature vector; Represents the L2 norm; Select the highest similarity from the J cosine similarities. And record its corresponding identity index. ; The highest similarity With preset threshold Compare; if ≥ Then the identity is determined by the corresponding template feature vector. Associated identity Otherwise, mark it as "unknown"; Finally, output the facial recognition result; if the highest similarity is... ≥ Then it is determined to be the corresponding identity. It outputs the identity label and the highest similarity score; otherwise, it is judged as an "unknown" identity.

10. A facial recognition system suitable for smart cities, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-9.

Images