Multi-channel supervised face recognition method based on entropy control

Abstract

The present invention discloses a multi-channel supervised face recognition method based on entropy regulation, comprising: constructing a face recognition model: applying a first type of data enhancement and a second type of data enhancement to training images respectively to generate two types of enhanced image sets; wherein the first type of enhancement retains key facial structural features, and the second type of enhancement introduces a higher degree of deformation or occlusion; dividing the two types of enhanced images into image blocks of fixed sizes, respectively, and inputting them into a Transformer encoder after feature mapping and position encoding to obtain corresponding first prediction results and second prediction results; using the first prediction result to supervise the second prediction result, and jointly optimizing model parameters through a classification loss function and a consistency constraint loss function; calculating the information entropy of each image block in the second type of enhanced image, generating adaptive weights based on entropy value aggregation and weighting the classification loss function to guide the model to focus on information sparse areas; inputting the face image to be recognized into the trained model, and outputting the face recognition result.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a multi-channel supervised face recognition method based on entropy regulation. Background Art

[0002] Facial recognition technology, a core branch of biometrics, has gradually evolved from laboratory research to large-scale application since the 1990s. Early methods (such as LBP based on geometric features and PCA based on statistical learning) were limited by their ability to express features, enabling only basic recognition under ideal lighting and unobstructed conditions. With the rise of deep learning, convolutional neural networks (CNNs), leveraging their local perception and weight-sharing inductive bias, have achieved breakthroughs in recognition accuracy in complex scenarios, becoming a mainstream solution in fields such as mobile payments and public security.

[0003] However, the local receptive field of CNNs naturally limits their ability to capture global structural relationships in faces, such as cross-regional expressions and age variations. Since its introduction to computer vision in 2020, the Transformer model (such as the Vision Transformer, or ViT) has demonstrated enhanced representational capabilities in facial attribute analysis (such as age and gender) and cross-pose recognition, gradually becoming a key advancement in facial recognition technology. ViT uses a self-attention mechanism to model global dependencies. This has led to a surge in the popularity of ViT, demonstrating enhanced representation capabilities in facial attribute analysis (such as age and gender) and cross-pose recognition.

[0004] While the Transformer significantly improves recognition robustness in complex scenarios, its data-hungry nature has become a key bottleneck for widespread application. Unlike CNNs, which rely on local inductive biases, the Transformer lacks prior assumptions about the spatial structure of images and requires large-scale annotated data (typically millions or more) to learn its underlying patterns. Otherwise, it is prone to overfitting (memorizing only the local noise of the training samples and failing to generalize to new data). This strong reliance on large datasets raises multiple challenges in practical applications. On the one hand, the acquisition and annotation of high-quality facial data is expensive, and the data involves sensitive personal information (such as facial details and usage scenarios), significantly increasing the privacy risks of data storage and transmission. On the other hand, emerging application scenarios (such as insider identification in specific industries and small-scale community security) often only provide small-scale data in the thousands or tens of thousands, making it difficult to support effective Transformer training and limiting the application of the technology in vertical fields.

[0005] Furthermore, the Transformer's self-attention mechanism exhibits a tendency to "localize"—the model tends to concentrate computational resources on highly discriminative areas of the face while ignoring less salient areas containing key clues. This localized focus can further degrade model performance when data augmentation (such as occlusion and color perturbations) disrupts these highly discriminative areas. Therefore, training a Transformer face recognition model that balances global perception and robustness against disturbances on small datasets has become a core area of ​​current technical research.

[0006] Reducing Transformer's reliance on large-scale data not only directly reduces companies' costs for data acquisition, labeling, and storage, but also accelerates model deployment and iteration in small-sample scenarios (such as new business cold starts and privacy-sensitive scenarios), helping companies gain a first-mover advantage in technology application. Furthermore, reducing data size also means reducing the scope of personal information collected, providing more flexibility for the compliant promotion of technology. Summary of the Invention

[0007] In response to the defects of the existing Transformer-based face recognition model in the prior art, such as its reliance on large-scale data sets and prone to overfitting, and the local focus of the self-attention mechanism leading to the neglect of key facial areas, the present invention provides a multi-channel supervised face recognition method based on entropy regulation. Through the collaborative design of multi-channel supervision and entropy regulation technology, this method effectively reduces the model's dependence on large-scale data and improves the recognition robustness under small data sets: on the one hand, a differentiated data enhancement strategy is adopted to generate a first-class enhanced image that retains key structures and a second-class enhanced image that destroys the structure. The prediction results of the preserved structure are used to supervise the prediction results of the destroyed structure, alleviating the overfitting problem of small data sets; on the other hand, by calculating the Gaussian upper bound information entropy of each image block in the second-class enhanced image, combining the entropy-aware weight mechanism to generate adaptive weights that are negatively correlated with the information entropy, and weighted classification loss to guide the model to focus on information-sparse areas such as the nose and lips that are easily overlooked, thereby enhancing global perception capabilities. While reducing data annotation costs and protecting user privacy, the present invention significantly improves the recognition accuracy and stability of the Transformer face recognition model in small sample scenarios.

[0008] The present invention specifically adopts the following technical solutions:

[0009] A multi-channel supervised face recognition method based on entropy regulation, comprising:

[0010] Face recognition model construction:

[0011] Apply the first and second types of data augmentation to the training images to generate two sets of enhanced images. The first type of augmentation retains key facial structural features, while the second type of augmentation introduces a higher degree of deformation or occlusion.

[0012] The two types of enhanced images are divided into fixed-size image blocks, which are then input into the Transformer encoder after feature mapping and position encoding to obtain the corresponding first and second prediction results;

[0013] The first prediction result is used to supervise the second prediction result, and the model parameters are optimized jointly by the classification loss function and the consistency constraint loss function;

[0014] Calculating the information entropy of each image block in the second type of enhanced image, generating adaptive weights based on entropy aggregation, and weighting the classification loss function to guide the model to focus on information sparse areas;

[0015] Face recognition applications:

[0016] Input the face image to be recognized into the trained model and output the face recognition result.

[0017] Furthermore, the first type of data enhancement includes horizontal flipping, vertical flipping, rotation or scaling; the second type of data enhancement includes color jittering, CutMix or Cutout.

[0018] Furthermore, the information-sparse area corresponds to easily overlooked features of the face, such as the nose, lips or chin.

[0019] Furthermore, the adaptive weight is generated as follows:

[0020] Calculate the Gaussian upper bound information entropy of each image block;

[0021] The information entropy of each image block is aggregated using the entropy-aware weight mechanism.

[0022] Furthermore, the consistency constraint loss function is a mean square error loss function.

[0023] Furthermore, the classification loss function is an ArcFace loss function.

[0024] Furthermore, the number of image block divisions is determined by the image height, width and block size;

[0025] The feature map is to convert the image block into a feature vector through linear transformation, and the position encoding is used to retain the spatial position information of the image block.

[0026] Furthermore, the weights are minimized during the training process to facilitate the model to mine image block features with less information.

[0027] And, a computer device, characterized in that it includes a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, it implements the multi-channel supervised face recognition method based on entropy regulation as described above.

[0028] A non-transitory computer-readable storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processor, it implements the multi-channel supervised face recognition method based on entropy regulation as described above.

[0029] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0030] The "multi-channel supervision mechanism" effectively reduces the Transformer model's dependence on large-scale datasets. Through differentiated data augmentation strategies (a first-type augmentation that preserves key structure and a second-type augmentation that destroys structure), the structure-preserving predictions are used to supervise the structure-destroying ones. This not only alleviates the overfitting problem of the model in small datasets, but also reduces the excessive damage to key facial structures caused by strong data augmentation. This significantly reduces the costs of data annotation, storage, and privacy protection, providing a feasible path for model deployment in small sample scenarios (such as the cold start of new services and privacy-sensitive areas).

[0031] The model's global perception capabilities are enhanced through the "entropy control module." By calculating the Gaussian upper bound of information entropy for each image block in the second-category enhanced image, and combining it with an entropy-aware weighting mechanism to generate adaptive weights that are negatively correlated with information entropy, the model is then weighted with a classification loss to guide the model's focus on information-sparse, easily overlooked areas such as the nose, lips, and chin. This addresses the "local focus" flaw of the Transformer self-attention mechanism and improves the model's recognition robustness in complex scenarios such as occlusion and color perturbations.

[0032] Model training efficiency was optimized through feature processing and loss function design. Fixed-size image block segmentation, linear projection, and learnable position embedding compress feature dimensions while preserving spatial information, reducing computational complexity. ArcFace loss enhances inter-class discrimination, while consistency loss constrains prediction consistency, further ensuring the model's training stability and feature expression capabilities in small datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0034] Figure 1 Schematic diagram of the principle of an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Hereinafter, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. Based on these detailed descriptions, those skilled in the art will be able to clearly understand the present application and implement the present application. Without violating the principles of the present application, the features of different embodiments may be combined to obtain new implementations, or certain features of certain embodiments may be substituted to obtain other preferred implementations.

[0036] To make the features and advantages of the present invention more clearly understood, the following embodiments are specifically described in detail with reference to the accompanying drawings.

[0037] The present invention proposes a multi-channel supervised face recognition method based on entropy control. Based on the Transformer, a multi-channel strong-weak supervised learning mechanism and an entropy control module are designed. The multi-channel strong-weak learning mechanism utilizes the prediction results of weakly augmented images to supervise and guide the model prediction of strongly augmented images. This effectively addresses the overfitting problem of the Transformer on small datasets while reducing the destruction of key facial structural information, significantly improving the overall performance and robustness of the model. Furthermore, considering that the Transformer tends to focus on specific facial areas such as the eyes and forehead while ignoring other locations containing facial cues, the entropy control module calculates a Gaussian upper bound on the information entropy for a series of fixed-size image blocks fed into the Transformer. It aggregates the information entropy using an entropy-aware weighting mechanism and adaptively assigns importance weights to each image block. This fully exploits the features of easily overlooked areas of each face, such as the nose, lips, and chin. Even when strong data augmentation may destroy key structural information, it can still use the remaining facial cues to make stable inferences.

[0038] In this embodiment, a small-scale annotated dataset is used to train a Transformer-based face recognition model, and information entropy control and multi-channel supervision technology are combined for optimization. Figure 1 As shown in Figure 2, the construction of the model specifically includes the following steps:

[0039] Step S1 Dataset Processing: Perform strong and weak data enhancement on the face recognition image to generate the corresponding strong enhanced image , and weakly enhanced images .

[0040] Step S2 Multi-channel supervision: and The image is divided into a series of fixed-size image blocks (patches) to complete the image block expansion; linear projection is performed, that is, each "patch" block is mapped to a feature vector through a linear layer, ensuring that all "patch" blocks have the same representation dimension, effectively extracting and compressing image features while reducing computational complexity; in order to retain the position information of the "patch" block in the model, the position embedding is added to each feature vector and then input into the Transformer model to obtain a strongly enhanced prediction result. and weak enhancement prediction results ;use To supervise and guide , and use ArcFace loss as the classification loss function as , and impose consistency constraints on the two prediction results. The consistency loss function is recorded as .

[0041] Step S3 Entropy Control: Calculate the Gaussian upper bound information entropy of each image "patch" block of the strongly enhanced dataset ; The information entropy of each “patch” block is aggregated using the entropy-aware weight mechanism to adaptively assign an importance weight to each image, denoted as ; Use this weight as the classification loss function By minimizing the classification loss, the model is encouraged to focus on difficult samples with less information during the optimization process.

[0042] Step S4: Model training and parameter update: Based on the calculated supervision loss , and consistency loss , train the model and update the parameters.

[0043] As a preferred solution of this embodiment, the strong data enhancement contents in step S1 include the following: color jitter, CutMix, and Cutout; the weak data enhancement contents include: horizontal flip, vertical flip, rotation, and scaling.

[0044] As a preferred solution of this embodiment, the image block expansion in step S2 is to divide the image into "patch" blocks with equal height and width. W C, H and W are the height and width of the image, C is the number of channels, and the "patch" size is P , the image will be segmented into A "patch".

[0045] As a preferred solution of this embodiment, the linear projection in step S2 uses a linear layer to map each "patch" block to a feature vector of length D, as shown in the following formula:

[0046]

[0047] in is the i-th flattened “patch”, W and b are the weight matrix and bias vector of the linear layer respectively.

[0048] As a preferred solution of this embodiment, the position embedding in step S2 is a learnable parameter, which represents the position of each "patch" and is expressed as follows:

[0049]

[0050] The position embedding matrix Size is N D, N is the number of "patches" and D is the dimension of the feature vector; represents the embedding vector representing the i-th position in the position embedding matrix.

[0051] As a preferred solution of this embodiment, the formula of ArcFace loss in step S2 is as follows:

[0052]

[0053] Where y is the label, s is the scaling factor, and m is the angle interval. It is the angle between the feature vector and the target class weight vector.

[0054] As a preferred solution of this embodiment, the formula of the consistency loss function in step S2 is as follows:

[0055]

[0056] In this embodiment, the larger the upper bound information entropy of each "patch" block obtained in step S3, the more information it contains, and vice versa, the less information it contains, so as to evaluate the degree of attention the model pays to each "patch" block.

[0057] As a preferred solution of this embodiment, the formula for the Gaussian upper bound information entropy in step S3 is as follows:

[0058]

[0059] in is the variance of the image information entropy, and the information entropy formula of image x is as follows:

[0060]

[0061] in is the probability of the gray level of the i-th pixel.

[0062] As a preferred solution of this embodiment, the formula of the entropy-aware weight mechanism in step S3 is as follows:

[0063]

[0064] in is the i-th “patch” in image X.

[0065] As a preferred solution of this embodiment, when minimizing the classification loss in step S3, the model is encouraged to focus on difficult samples with less information during the optimization process, that is, to minimize Minimize the weight when . This is equivalent to maximizing the Gaussian upper bound information entropy , maximizing the overall information of each "patch" block, which will promote the model to fully explore the facial clues contained in each "patch" block.

[0066] In this embodiment, in step S4, the importance weight is minimized while minimizing the loss function. , which encourages the model to fully exploit details in the face that were originally considered unimportant (such as the nose, lips, and chin), thereby significantly improving the robustness of the model.

[0067] As a preferred solution of this embodiment, the sum of the loss functions in step S4 is for:

[0068]

[0069] Preferably, this example includes an entropy control module. After obtaining the Gaussian upper bound entropy for each image "patch," it aggregates the entropy of each "patch" using an entropy-aware weighting mechanism to form the sample entropy. Based on this entropy, each sample is assigned an importance weight. By minimizing the importance weight, the system forces the network to pay more attention to commonly overlooked details (such as the nose, lips, and chin), significantly expanding the model's field of view, effectively improving the model's ability to capture features, and enhancing the model's overall recognition accuracy and stability.

[0070] This embodiment provides a multi-channel supervised face recognition strategy based on entropy regulation, which is divided into three steps. The first step is to perform strong data enhancement and weak data enhancement on the dataset respectively. Strong data enhancement includes color jittering, CutMix, and Cutout operations, while weak data enhancement retains more original features; the second step is to input the two datasets processed with strong data enhancement and weak data enhancement into the model respectively. The model output of the weak data enhancement dataset is used to supervise and guide the model output of the strong data enhancement dataset, and ensure the consistency of the feature extraction between the two; the third step is to calculate the Gaussian upper bound information entropy of each "patch" block, and use the entropy-aware weight mechanism to obtain the importance weight of each sample. These weights are combined with the classification loss function to improve the overall performance and robustness of the model.

[0071] Preferably, in this embodiment, a multi-channel supervised face recognition method based on entropy regulation is implemented, and a small-scale data set is used to train a system with an accuracy comparable to that of a large-scale data set training model, effectively alleviating the Transformer's requirements for data set size in the field of face recognition.

[0072] In view of the cost, performance and privacy issues brought about by Transformer's reliance on large-scale datasets in the field of face recognition, the present invention aims to use small-scale datasets to train a system with accuracy comparable to that of models trained on large-scale datasets, thereby bringing significant cost savings, performance optimization and data privacy protection to enterprises. To this end, this method proposes a multi-channel strong and weak supervision learning framework, which effectively alleviates the overfitting problem of the model on small-scale datasets. At the same time, this method adopts an entropy control mechanism to guide the model to pay attention to more detailed information in difficult areas, enhance its global perception ability, and improve the robustness of the model. The present invention can reduce the dependence of Transformer-based models on large-scale datasets, enhance the global perception ability of the model, and effectively promote the development of face recognition technology.

[0073] This embodiment first performs strong data enhancement and weak data enhancement on the data set, and inputs the enhanced data sets into the model respectively. Then, the model output of the weak data enhancement data set is used to supervise and guide the model output of the strong data enhancement data set to ensure the consistency of the two in feature extraction. Afterwards, the Gaussian upper bound information entropy of each image "patch" block divided by the Transformer is calculated, and the entropy-aware weight mechanism is used to aggregate the information entropy of each "patch" block to form the overall information entropy of the sample, and then the importance weight of each sample is obtained as a weight parameter to guide the training process of the model. This embodiment effectively reduces the cost of data acquisition and storage, improves model performance, and reduces privacy leakage. In short, this method not only has many technical innovations, but also brings the dual advantages of cost-effectiveness and performance improvement to enterprises. It has broad application prospects and huge benefits.

[0074] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0075] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, performs the above-described method. The storage medium may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0076] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0077] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present disclosure. Various changes and improvements may be made to the present disclosure without departing from the spirit and scope of the present disclosure, and such changes and improvements shall fall within the scope of the present disclosure.

[0078] The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive various other forms of multi-channel supervised face recognition methods based on entropy regulation under the inspiration of the present invention. All equal changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A multi-channel supervised face recognition method based on entropy control, characterized in that: include: Face recognition model construction: Apply the first type of data augmentation and the second type of data augmentation to the training images respectively to generate two types of enhanced image sets; The first type of enhancement preserves key facial structural features, while the second type of enhancement introduces a higher degree of deformation or occlusion; The two types of enhanced images are divided into fixed-size image blocks, which are then input into the Transformer encoder after feature mapping and position encoding to obtain the corresponding first and second prediction results; The first prediction result is used to supervise the second prediction result, and the model parameters are optimized jointly by the classification loss function and the consistency constraint loss function; Calculating the information entropy of each image block in the second type of enhanced image, generating adaptive weights based on entropy aggregation, and weighting the classification loss function to guide the model to focus on information sparse areas; The adaptive weight is generated as follows: Calculate the Gaussian upper bound information entropy of each image block; Aggregate the information entropy of each image block using the entropy-aware weight mechanism; Face recognition applications: Input the face image to be recognized into the trained model and output the face recognition result.

2. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The first type of data enhancement includes horizontal flipping, vertical flipping, rotation or scaling; the second type of data enhancement includes color jittering, CutMix or Cutout.

3. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The information-sparse area corresponds to the nose, lips or chin features in the human face.

4. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The consistency constraint loss function is a mean square error loss function.

5. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The classification loss function is the ArcFace loss function.

6. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The number of image block divisions is determined by the image height, width and block size; The feature map is to convert the image block into a feature vector through linear transformation, and the position encoding is used to retain the spatial position information of the image block.

7. The multi-channel supervised face recognition method based on entropy control according to claim 1, characterized in that: The weights are minimized during training to encourage the model to mine image patch features with less information.

8. A computer device, characterized in that: The invention comprises a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, the multi-channel supervised face recognition method based on entropy regulation according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the multi-channel supervised face recognition method based on entropy regulation according to any one of claims 1 to 7 is implemented.

Images