Face wearing object recognition method and system based on feature fusion

Abstract

The invention discloses a face wearing object recognition method and system based on feature fusion, relates to the technical field of computer vision, and solves the technical problems of serious feature cross interference, poor posture adaptability and low recognition precision in existing face wearing object recognition. Comprising the following steps: firstly, extracting initial features through a lightweight identity encoder and a wearing object encoder with a double-attention module; constructing a single generator-single discriminator adversarial learning network and combining with a double collaborative judgment mechanism; extracting local region features of the wearing object, relation features of the face parts and global features of the face, and constructing a multi-dimensional feature set; feature weights are dynamically adjusted through a posture perception attention mechanism, and fusion is completed; and finally, outputting the type of the wearing article by adopting a double-branch classification structure and combining a secondary identification mechanism. According to the invention, efficient and accurate recognition of various face wearing objects is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a face-wearing object recognition method and system based on feature fusion. BACKGROUND

[0002] With the wide application of computer vision technology in the fields of security, attendance and intelligent terminals, the demand for accurate recognition of face-wearing objects (such as glasses, masks, hats, earphones, etc.) is increasingly urgent.

[0003] However, the existing face-wearing object recognition technology still has the following key bottlenecks: in the traditional method, the face identity features and the wearing object attribute features are often coupled with each other, so that the wearing object feature extraction is interfered by the identity information, especially when the wearing object and the face are highly fitted (such as glasses, masks), the feature confusion is serious, and the recognition accuracy is greatly reduced; the face posture change (turning sideways, tilting) will cause the wearing object to be blocked and deformed in the image, the existing technology mostly adopts a fixed feature extraction method, which cannot dynamically adjust the feature weight according to the posture, and the recognition accuracy is greatly reduced under the non-frontal posture; most technologies only focus on the local features of the wearing object or the global features of the face, ignoring the spatial relationship between the face components such as eyes, nose and the wearing object, resulting in insufficient feature representation ability and difficulty in distinguishing similar wearing objects (sunglasses and ordinary glasses, duckbill cap and baseball cap), and the existing classification model mostly adopts a single branch structure, which has insufficient coverage of rare category wearing object samples, and lacks effective confidence checking mechanism, which is easy to cause misclassification, especially in complex background or light change scene, the stability is difficult to guarantee.

[0004] At the same time, the core goal of the existing technology is to eliminate the interference of the wearing object on the face identity recognition, rather than accurately recognizing the wearing object itself, the technical route does not match the actual wearing object recognition demand, and cannot meet the recognition requirements in diversified scenes.

[0005] Therefore, a face-wearing object recognition method and system based on feature fusion are in urgent need. SUMMARY

[0006] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present application proposes a face-wearing object recognition method and system based on feature fusion, which is used to solve the following technical problems: In the traditional method, the facial identity features and the wearing object attribute features are often coupled with each other, so that the wearing object feature extraction is interfered by the identity information, especially when the wearing object and the face are highly attached (such as glasses, masks), the feature confusion is serious, and the recognition accuracy is greatly reduced; the face posture change (side turning, pitching) will cause the wearing object to be blocked and deformed in the image, the existing technology mostly adopts a fixed feature extraction method, which cannot dynamically adjust the feature weight according to the posture, and the recognition accuracy is greatly reduced under the non-frontal posture; most technologies only focus on the local features of the wearing object or the global features of the face, ignore the spatial relationship between the face components such as eyes, nose and the wearing object, and the feature representation ability is insufficient, which is difficult to distinguish similar wearing objects (sunglasses and ordinary glasses, duckbill cap and baseball cap), and the existing classification model mostly adopts a single branch structure, which has insufficient coverage for the wearing object samples of minority categories, and lacks an effective confidence checking mechanism, which is prone to misclassification, especially in complex background or light change scenes, the stability is difficult to guarantee.

[0007] To solve the above problems, the first aspect of the present application provides a face wearing object recognition method based on feature fusion, comprising the following steps: S1: acquiring face image data containing wearing objects, constructing an identity encoder and a wearing object encoder, taking the face image data as input, and extracting facial identity features and wearing object attribute features respectively; S2: constructing an adversarial learning network, inputting the facial identity features and the wearing object attribute features into the adversarial learning network, and decoupling the two types of features through the internal preset adversarial mechanism of the adversarial learning network to obtain decoupled facial identity features and wearing object attribute features; S3: processing the decoupled wearing object attribute features, extracting wearing object local region features, and detecting face components of the face image data to determine the position information of the face key components, so as to construct a component relationship model; After extracting the component relationship features from the component relationship model, global feature extraction is performed on the decoupled facial identity features to obtain facial global features; S4: constructing a posture-aware attention mechanism model, inputting the facial global features, the wearing object local region features and the component relationship features into the attention mechanism model, detecting the posture of the face image data to obtain the face posture information, dynamically adjusting the weight coefficients of the facial global features, the wearing object local region features and the component relationship features, and dynamically weighting and fusing the three types of features to obtain fusion features; S5: inputting the fusion features into a preset classification recognition model for analysis and processing, and finally outputting the category information of the wearing object.

[0008] Preferably, in the step S1, the identity encoder and the wearing object encoder are constructed, comprising the following steps: The identity encoder adopts a lightweight convolutional neural network structure, including: 1 input layer, 4 feature extraction units and 1 fully connected layer. Each feature extraction unit is a concatenated structure of a convolutional layer, a batch normalization layer, a ReLU activation function and a max pooling layer. The convolutional layers uniformly use a 3*3 pixel kernel with a fixed stride of 1. The 1st and 3rd feature extraction units contain only 1 convolutional layer, while the 2nd and 4th feature extraction units contain 2 convolutional layers. The fully connected layer flattens the feature map output by the 4th feature extraction unit into a one-dimensional vector and outputs the final face identity feature vector. The wearable encoder is based on the ResNet-18 framework. After each residual block in its third and fourth stages, a channel-spatial dual attention module is embedded. The module calculates the weight coefficients of each channel in the feature map output by the residual block through the channel attention branch, and then inputs the channel-weighted feature map into the spatial attention branch to locate the spatial region of the wearable on the feature map. The pixels in the region are assigned high weights and the background region is assigned low weights. Finally, the wearable attribute feature vector is output.

[0009] Preferably, in step S2, constructing the adversarial learning network includes the following steps: The adversarial learning network adopts a single generator-single discriminator architecture. The generator is a parallel structure of the identity encoder and the wearable encoder, which synchronously outputs the face identity feature F_ID and the wearable attribute feature F_A. The discriminator adopts a two-layer fully connected layer combined with a sigmoid activation structure to determine whether the input features are pure feature pairs (F_ID, F_A) without cross-interference.

[0010] Preferably, determining whether the input features are pure feature pairs without cross-interference includes the following steps: The discriminator employs a dual collaborative judgment mechanism embedded within a two-layer fully connected structure combined with a Sigmoid activation structure. This mechanism includes a first judgment stage and a second judgment stage, specifically: In the first judgment step, the dimensional compliance of the input F_ID and F_A is checked. The first 128-dimensional features of F_ID and the full 128-dimensional features of F_A are extracted through the preset dimensional mask matrix, and the cosine similarity between the two is calculated. If the cosine similarity is less than or equal to the preset similarity threshold, the second judgment is performed; otherwise, it is directly defined as a feature pair containing cross-interference. In the second judgment step, for F_ID and F_A that have passed the first judgment step, the variance value within the feature is calculated respectively, and the interference evaluation value I is calculated as I = σF_ID - σF_AmaxσF_ID, σF_A, where σF_ID is the variance value of F_ID and σF_A is the variance value of F_A. If the interference evaluation value I ≤ the preset interference threshold, then F_ID and F_A are input to the fully connected layer; For feature pairs that pass through the dual collaborative judgment mechanism, after processing by the first fully connected layer and the second fully connected layer, the probability value is output by activating the Sigmoid function. If the probability value is greater than 0.85, it is determined to be a pure feature pair without cross interference.

[0011] Preferably, in step S2, optimizing the parameters of the two encoders through adversarial training includes the following steps: Construct a composite loss function, specifically as follows: Ltotal=ωL1*L1+ωL2*L2+ωL3*L3 L1=max0,F_IDa-F_IDpos22-FIDa-FIDne22+α L2=-1Ni=1Nyi∙logpi+1-yi∙log1-pi×ui L3=EFID, FA~PdatalogDFID, FA+EFID', FA'~Pnoiselog1-DFID', FA' Where Ltotal is the total loss value of the composite loss function, ωL1, ωL2 and ωL3 are the weights of the corresponding loss functions, L1 is the identity feature fidelity loss, F_IDa is the face identity feature of the anchor sample, F_IDpos is the face identity feature of the positive sample, FIDne is the face identity feature of the negative sample, α is the interval parameter, L2 is the wear attribute classification loss, N is the number of samples, i is the sample index, yi is the true label of the wear attribute, pi is the prediction probability, ui is the class balance weight, which is set according to the inverse ratio of the number of each category of wear in the sample set, L3 is the adversarial loss, E∙~P∙ are the expectation functions, D∙ is the discriminator output probability, Pdata is the true feature distribution, Pnoise is the interference feature distribution with added Gaussian noise, and FID' and FA' are the interference face identity feature and interference wear attribute feature after adding Gaussian noise, respectively. The parameters of the two encoders are optimized by iterating through an adaptive estimator until the composite loss function converges.

[0012] Preferably, in step S3, the decoupled wearable attribute features are processed, including the following steps: A multi-scale sliding window mechanism is adopted, with three window sizes set, each with a step size of 8 pixels. The decoupled wearable attribute feature map is then sampled to obtain multi-scale local feature blocks. An adaptive feature selection unit is introduced to calculate the information entropy of each feature unit. Feature units with information entropy greater than a preset information entropy threshold are selected and then stitched together through a 1*1 convolutional layer to obtain the local region features of the wearer. The preset information entropy threshold is obtained by calculating the average information entropy of all feature units. An improved cascaded convolutional neural network is used. In the first stage, a lightweight convolutional layer is used to locate the entire face region. In the second stage, a bounding box regression branch is introduced to correct the entire face region. In the third stage, six component detection heads are set up to detect the eyes, nose, mouth, eyebrows, cheeks and forehead respectively, and output the bounding box coordinates of each component.

[0013] Preferably, in step S3, constructing the component relationship model includes the following steps: The component relationship model is constructed by using the center coordinates of the bounding box of each face component as nodes to build an undirected graph model G=V,E, where V=v1, v2, ..., v6 represent 6 face component nodes and E is the edge between nodes. Define the edge weight eij = 11 + dij × cosθij, where dij is the Euclidean distance between nodes vi and vj, and θij is the angle between the line connecting the two nodes and the horizontal direction. Feature learning is performed on an undirected graph model using a graph convolutional network to output 64-dimensional component relationship features; The decoupled facial identity feature map is subjected to a dual pooling operation combining global average pooling and global max pooling to obtain two 128-dimensional feature vectors. These vectors are then fused by attention weighting to obtain 256-dimensional global facial features.

[0014] Preferably, step S4 includes the following steps: A 3D pose regression network is used to take face image data as input, output face Euler angles, and classify five pose levels based on the Euler angles. A pose-attention mapping table is set up, and an initial attention weight matrix is ​​preset for each pose level. A dynamic weight adjustment module is introduced. With face pose information as input, the weight correction coefficient is output through a multilayer perceptron to correct the initial attention weight matrix. After obtaining the attention weights for pose adaptation, the global face features, local features of the worn object, and component relationship features are multiplied by the corresponding attention weights respectively. The feature importance score S for the three types of enhanced features is calculated as follows: S=13×VarFglobalMaxVar,VarFlocalMaxVar,VarFrelationMaxVar Where Var∙ is the feature variance, MaxVar is the maximum value of the three types of feature variance, and Fglobal, Flocal and Frelation are the global features of the face, the local region features of the worn object and the feature of the component relationship, respectively. Based on the feature importance score S, the three types of features are dynamically weighted and fused to obtain the fused feature.

[0015] Preferably, in step S5, the preset classification and recognition model includes the following steps: A feature adaptation layer combined with a dual-branch classification structure is adopted. The feature adaptation layer is passed through two fully connected layers. The dual-branch classification structure includes a coarse classification branch and a fine classification branch. The coarse classification branch is used to identify the major category of the wearable item, and the fine classification branch is used to identify the sub-category of the wearable item. The fused features are input into the classification and recognition model. After dimensional mapping and feature optimization through the feature adaptation layer, they are input into the dual-branch classification layer. The coarse classification branch outputs the predicted probability of the major category of the wearable item, and selects the category with the highest probability as the coarse classification result. The fine classification branch calls the classifier of the corresponding subclass based on the coarse classification result, outputs the predicted probability of the wearable item subclass, and selects the category with the highest probability as the fine classification result. If the maximum probability of the coarse classification result is less than 0.6, a secondary recognition mechanism is activated to re-extract the local features of the wearer and splice them with the fused features. The results are then input into the classification recognition model again for recognition, and finally the major category and sub-category information of the wearer are output.

[0016] A second aspect of the present invention provides a face-wearing recognition system based on feature fusion, comprising the following modules: Feature extraction module: Acquires facial image data including the worn object, constructs an identity encoder and an object encoder, and extracts facial identity features and object attribute features respectively by taking the facial image data as input; Feature decoupling optimization module: Construct an adversarial learning network, input facial identity features and accessory attribute features into the adversarial learning network, and decouple the two types of features through its internal preset adversarial mechanism to obtain decoupled facial identity features and accessory attribute features; Multi-dimensional feature refinement and extraction module: Processes the decoupled attribute features of the wearable object, extracts local region features of the wearable object, and performs facial component detection on the facial image data to determine the location information of key facial components in order to build a component relationship model; After extracting component relationship features from the component relationship model, global feature extraction is performed on the decoupled face identity features to obtain global face features; Pose-aware feature fusion module: Constructs a pose-aware attention mechanism model, inputs global facial features, local features of the worn object, and component relationship features into the attention mechanism model, performs pose detection on the facial image data, obtains facial pose information, dynamically adjusts the weight coefficients of global facial features, local features of the worn object, and component relationship features, and dynamically weights and fuses the three types of features to obtain fused features; Wearable item classification and recognition module: The fused features are input into a preset classification and recognition model for analysis and processing, and finally output the category information of the wearable item.

[0017] The beneficial effects of this invention are: This invention designs a single generator-single discriminator adversarial learning network. The generator is connected in parallel with an identity encoder and an object encoder, and the discriminator is embedded with cosine similarity. This initially filters feature pairs with no obvious coupling. Then, the interference evaluation value is calculated by feature variance to accurately identify latent cross interference. Finally, a composite loss function containing identity fidelity loss, attribute classification loss and adversarial loss is constructed. The encoder parameters are iteratively optimized by an adaptive optimizer to ensure the purity of the decoupled features and greatly reduce the feature cross interference rate. This invention outputs Euler angles of the face and classifies them into 5 pose levels through a 3D pose regression network, achieving accurate classification of different poses. Then, a pose-attention mapping table is constructed with preset initial weights. The weight matrix is ​​dynamically adjusted by outputting weight correction coefficients through a multilayer perceptron. This enables the dynamic weight correction coefficients to adjust the weights of each feature in real time according to pose changes. Finally, the feature importance score is calculated by combining feature variance. The global features of the face, the local features of the worn object, and the features related to the parts are dynamically weighted and fused to further optimize the rationality of weight allocation, thereby improving the recognition accuracy under non-frontal poses and breaking through the limitations of fixed feature extraction. This invention employs a multi-scale sliding window combined with adaptive information entropy filtering to extract multi-scale local features of the wearable object, retaining key feature units with information entropy higher than the mean. It detects six facial components using an improved cascaded convolutional neural network, constructs an undirected graph model with the component center coordinates as nodes, and extracts 64-dimensional component relationship features using a graph convolutional network. The decoupled identity features are then subjected to a dual pooling operation combining global average pooling and global max pooling, combined with attention-weighted fusion to obtain 256-dimensional global facial features. These three types of features work together to achieve a full-dimensional representation of details, associations, and contours, significantly improving the ability to distinguish similar wearable objects. This invention designs a feature adaptation layer combined with a dual-branch classification structure. The coarse classification branch identifies the major categories of wearable items, while the fine classification branch calls the corresponding subclass classifier based on the coarse classification result. When the maximum probability of the coarse classification is less than 0.6, a secondary recognition mechanism is activated to re-extract local features of the wearable item and concatenate them with the fused features for re-identification. The category balance weight is set inversely according to the number of samples to solve the classification bias caused by insufficient samples of niche categories. The secondary recognition mechanism enhances the feature representation ability of low-confidence samples by supplementing local features, thereby greatly improving the overall classification accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the module flow of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 As shown, this invention is a face-wearing recognition method based on feature fusion, comprising the following steps: S1: Obtain facial image data containing the worn items, construct an identity encoder and an item encoder, and extract facial identity features and item attribute features respectively by taking the facial image data as input. S2: Construct an adversarial learning network, input facial identity features and accessory attribute features into the adversarial learning network, and decouple the two types of features through its internal preset adversarial mechanism to obtain decoupled facial identity features and accessory attribute features; S3: Process the decoupled wearable attribute features, extract local area features of the wearable, and perform facial component detection on the facial image data to determine the location information of key facial components in order to build a component relationship model; After extracting component relationship features from the component relationship model, global feature extraction is performed on the decoupled face identity features to obtain global face features; S4: Construct a pose-aware attention mechanism model. Input the global features of the face, the local features of the wearing object, and the component relationship features into the attention mechanism model. Perform pose detection on the face image data to obtain the face pose information. Then, dynamically adjust the weight coefficients of the global features of the face, the local features of the wearing object, and the component relationship features. Perform dynamic weighted fusion of the three types of features to obtain the fused features. S5: Input the fused features into a preset classification and recognition model for analysis and processing, and finally output the category information of the wearer.

[0021] In one embodiment of the present invention, step S1, constructing the identity encoder and the wearable encoder, includes the following steps: The identity encoder adopts a lightweight convolutional neural network structure, including: 1 input layer, 4 feature extraction units and 1 fully connected layer. Each feature extraction unit is a concatenated structure of a convolutional layer, a batch normalization layer, a ReLU activation function and a max pooling layer. The convolutional layers uniformly use a 3*3 pixel kernel with a fixed stride of 1. The 1st and 3rd feature extraction units contain only 1 convolutional layer, while the 2nd and 4th feature extraction units contain 2 convolutional layers. The fully connected layer flattens the feature map output by the 4th feature extraction unit into a one-dimensional vector and outputs the final face identity feature vector. The wearable encoder is based on the ResNet-18 framework. After each residual block in its third and fourth stages, a channel-spatial dual attention module is embedded. The module calculates the weight coefficients of each channel in the feature map output by the residual block through the channel attention branch, and then inputs the channel-weighted feature map into the spatial attention branch to locate the spatial region of the wearable on the feature map. The pixels in the region are assigned high weights and the background region is assigned low weights. Finally, the wearable attribute feature vector is output.

[0022] Specifically, the four feature extraction units include: Feature Extraction Unit 1: Contains one 3×3 convolutional layer (32 kernels, stride = 1, padding = 1 to ensure the feature map size is not reduced) → Batch Normalization Layer (momentum = 0.9, epsilon = 1e-5) → ReLU activation function → 2×2 max pooling layer (stride = 2, padding = 0, compressing the feature map size from 224×224 to 112×112); Feature Extraction Unit 2: Contains one 3×3 convolutional layer (128 kernels, stride = 1, padding = 1) → Batch Normalization Layer → ReLU activation function → 2×2 max pooling layer (stride = 2, compressing the feature map size to 28×28); Feature Extraction Unit 3: Contains one 3×3 convolutional layer (128 kernels, stride = 1, padding = 1) → Batch Normalization Layer → ReLU activation function → 2×2 max pooling layer (stride = 2, compressing the feature map size to 28×28); 128 (stride=1, padding=1) → Batch normalization layer → ReLU activation function → 2×2 max pooling layer (stride=2, feature map size compressed to 28×28); 4th feature extraction unit: containing 2 layers of 3×3 convolutional layers (first layer convolutional kernel number=256, second layer convolutional kernel number=256, stride=1, padding=1) → Batch normalization layer → ReLU activation function → 2×2 max pooling layer (stride=2, feature map size compressed to 14×14); The channel attention branch operation involves: The feature map output from the residual block (denoted as X, dimension = H × W × C, where H is height, W is width, and C is the number of channels); global average pooling (H × W → 1 × 1) is performed on X to obtain the channel feature vector (dimensional = 1 × 1 × C); two fully connected layers (the first layer output dimension = C / 4, activation function is ReLU; the second layer output dimension = C, activation function is Sigmoid) are used to generate the channel weight coefficient vector (dimensional = 1 × 1 × C, denoted as W_c); the original feature map X is multiplied by the channel weight coefficients W_c for each channel to obtain the channel-weighted feature map X_c = X × W_c. The spatial attention branch operation involves: inputting a channel-weighted feature map X_c, performing max pooling and average pooling (both with 1×1 pooling kernels) along the channel dimension to obtain two single-channel feature maps (both with dimensions = H×W×1), and concatenating them into a feature map of H×W×2; compressing the feature map dimension to H×W×1 through a 3×3 convolutional layer (1 kernel, 1 padding), and then generating a spatial weight coefficient matrix (dimension = H×W×1, denoted as W_s) using a sigmoid activation function; multiplying the channel-weighted feature map X_c and the spatial weight coefficient W_s according to their spatial positions to obtain the final attention-enhanced feature map X_sc = X_c × W_s, where the pixel weights in the wearable area are ≥0.7 and the pixel weights in the background area are ≤0.3.

[0023] In one embodiment of the present invention, step S2, constructing an adversarial learning network, includes the following steps: The adversarial learning network adopts a single generator-single discriminator architecture. The generator is a parallel structure of the identity encoder and the wearable encoder, which synchronously outputs the face identity feature F_ID and the wearable attribute feature F_A. The discriminator adopts a two-layer fully connected layer combined with a sigmoid activation structure to determine whether the input features are pure feature pairs (F_ID, F_A) without cross-interference.

[0024] Specifically, the parallel structure is constructed as follows: a pre-processed face image (size 224×224×3, same as the encoder input standard) is received, and the same image is synchronously transmitted to the identity encoder and the wearable encoder through an image splitter. The identity encoder outputs 128-dimensional face identity features, and the wearable encoder synchronously outputs 128-dimensional wearable attribute features. The generator concatenates the two feature vectors in the order of F_ID first and F_A last to form a 256-dimensional feature pair (F_ID, F_A), which serves as the input data for the discriminator. The initial values ​​of the parameters of the identity encoder and the wearable encoder in the generator are based on their optimal parameters after independent training. During the training process, the parameters are updated collaboratively with the discriminator parameters to ensure that the feature extraction capability does not degrade. The discriminator is used to determine whether the input 256-dimensional feature pairs (F_ID, F_A) are pure feature pairs without cross-interference, i.e., F_ID does not contain the attribute information of the wearer and F_A does not contain the facial identity information. The two fully connected layers include the first fully connected layer: the input dimension is 256, the output dimension is set to 128, the activation function is LeakyReLU (negative slope = 0.2) to avoid gradient vanishing, and a Dropout layer (dropout probability = 0.5) is introduced to improve the generalization ability of the discriminator; the second fully connected layer: the input dimension is 128, the output dimension is set to 1, the activation function is Sigmoid, and the output value is mapped to the interval [0, 1]. The discriminator loss function adopts binary cross-entropy loss, the generator loss function adopts adversarial loss, the generator optimizer adopts AdamW (initial learning rate = 5e-5, weight decay = 1e-5) and connects with the encoder optimizer parameters to avoid parameter oscillation, and the discriminator optimizer adopts Adam (initial learning rate = 1e-4, β1 = 0.5, β2 = 0.999) to adapt to the gradient update requirements of binary classification tasks.

[0025] In one embodiment of the present invention, determining whether the input features are pure feature pairs without cross-interference includes the following steps: The discriminator employs a dual collaborative judgment mechanism embedded within a two-layer fully connected structure combined with a Sigmoid activation structure. This mechanism includes a first judgment stage and a second judgment stage, specifically: In the first judgment step, the dimensional compliance of the input F_ID and F_A is checked. The first 128-dimensional features of F_ID and the full 128-dimensional features of F_A are extracted through the preset dimensional mask matrix, and the cosine similarity between the two is calculated. If the cosine similarity is less than or equal to the preset similarity threshold, the second judgment is performed; otherwise, it is directly defined as a feature pair containing cross-interference. In the second judgment step, for F_ID and F_A that have passed the first judgment step, the variance value within the feature is calculated respectively, and the interference evaluation value I is calculated as I = σF_ID - σF_AmaxσF_ID, σF_A, where σF_ID is the variance value of F_ID and σF_A is the variance value of F_A. If the interference evaluation value I ≤ the preset interference threshold, then F_ID and F_A are input to the fully connected layer; For feature pairs that pass through the dual collaborative judgment mechanism, after processing by the first fully connected layer and the second fully connected layer, the probability value is output by activating the Sigmoid function. If the probability value is greater than 0.85, it is determined to be a pure feature pair without cross interference.

[0026] Specifically, the preset similarity threshold is determined by experimentally analyzing the cosine similarity distribution of pure feature pairs in the CelebA-Wearing dataset (generally concentrated in the range of 0.1-0.3), and the preset similarity threshold is set to 0.4 (the similarity of 95% of pure feature pairs is ≤0.4). Preset interference threshold setting: Through experimental verification, the interference evaluation value of pure feature pairs is concentrated in the range of 0.1-0.2. The preset interference threshold is set to 0.3 (the preset interference threshold for 90% pure feature pairs is ≤0.3). If the output probability value is greater than 0.85, it is determined to be a pure feature pair without cross-interference; otherwise, it is determined to be a non-pure feature pair with cross-interference.

[0027] In one embodiment of the present invention, step S2, optimizing the parameters of the two encoders through adversarial training, includes the following steps: Construct a composite loss function, specifically as follows: Ltotal=ωL1*L1+ωL2*L2+ωL3*L3 L1=max0,F_IDa-F_IDpos22-FIDa-FIDne22+α L2=-1Ni=1Nyi∙logpi+1-yi∙log1-pi×ui L3=EFID, FA~PdatalogDFID, FA+EFID', FA'~Pnoiselog1-DFID', FA' Where Ltotal is the total loss value of the composite loss function, ωL1, ωL2 and ωL3 are the weights of the corresponding loss functions, L1 is the identity feature fidelity loss, F_IDa is the face identity feature of the anchor sample, F_IDpos is the face identity feature of the positive sample, FIDne is the face identity feature of the negative sample, α is the interval parameter, L2 is the wear attribute classification loss, N is the number of samples, i is the sample index, yi is the true label of the wear attribute, pi is the prediction probability, ui is the class balance weight, which is set according to the inverse ratio of the number of each category of wear in the sample set, L3 is the adversarial loss, E∙~P∙ are the expectation functions, D∙ is the discriminator output probability, Pdata is the true feature distribution, Pnoise is the interference feature distribution with added Gaussian noise, and FID' and FA' are the interference face identity feature and interference wear attribute feature after adding Gaussian noise, respectively. The parameters of the two encoders are optimized by iterating through an adaptive estimator until the composite loss function converges.

[0028] Specifically, the parameter optimization process uses the Adaptive Moment Estimator (Adam) optimizer, with an initial learning rate of 1e-4. Every 50 iterations, the learning rate is reduced to 0.5 of the previous iteration. The process continues until the composite loss function converges (the fluctuation of the composite loss function is less than 1e-5 for 10 consecutive iterations, thus completing the parameter optimization of the two encoders). Wherein, ωL1, ωL2 and ωL3 are the weights of the corresponding loss functions. Before training, 1000 sets of samples were selected for forward calculation of the single loss function to statistically analyze the magnitude range of the three types of loss. Based on the principle of magnitude balance, the values ​​of ωL1, ωL2 and ωL3 were set to 0.3, 0.4 and 0.3, respectively.

[0029] In one embodiment of the present invention, step S3 involves processing the decoupled wearable attribute features, including the following steps: A multi-scale sliding window mechanism is adopted, with three window sizes set, each with a step size of 8 pixels. The decoupled wearable attribute feature map is then sampled to obtain multi-scale local feature blocks. An adaptive feature selection unit is introduced to calculate the information entropy of each feature unit. Feature units with information entropy greater than a preset information entropy threshold are selected and then stitched together through a 1*1 convolutional layer to obtain the local region features of the wearer. The preset information entropy threshold is obtained by calculating the average information entropy of all feature units. An improved cascaded convolutional neural network is used. In the first stage, a lightweight convolutional layer is used to locate the entire face region. In the second stage, a bounding box regression branch is introduced to correct the entire face region. In the third stage, six component detection heads are set up to detect the eyes, nose, mouth, eyebrows, cheeks and forehead respectively, and output the bounding box coordinates of each component.

[0030] Specifically, for local feature extraction of the wearable device: a multi-scale sliding window mechanism is adopted, with three window sizes: small window (2*2 pixels), medium window (3*3 pixels), and large window (4*4 pixels). During sampling, the feature map is fully scanned from left to right and from top to bottom. If the window edge exceeds the feature map boundary, the boundary pixels are filled with zero padding (the padding value is 0, which does not affect the effective features). The improved cascaded convolutional neural network used in face component detection has the following configuration: First, the network structure is configured as follows: The first stage uses a lightweight convolutional neural network (based on MobileNetV2), containing 3 convolutional layers. The second stage regression branch configuration adds a bounding box regression branch at the output of the first stage network, containing 2 fully connected layers. The first fully connected layer has an input dimension equal to the flattened dimension of the probability map from the first stage, and an output dimension of 128. The second fully connected layer has an output dimension of 4. Bounding box correction is calculated by predicting the offset (Δx1, Δy1, Δx2, Δy2) of the initial bounding box B1 through the regression branch. Here, (x1, y1) represents the pixel coordinates of the top-left vertex of bounding box B1, and (x2, y2) represents the offset of the top-left vertex of bounding box B1. The pixel coordinates of the bottom right corner vertex are given. The corrected bounding box B2 coordinates are (x1+Δx1, y1+Δy1, x2+Δx2, y2+Δy2), where (x1+Δx1, y1+Δy1) are the pixel coordinates of the top left corner vertex of the corrected bounding box B2, and (x2+Δx2, y2+Δy2) are the pixel coordinates of the top right corner vertex of the corrected bounding box B2. The offset values ​​range from [-0.1×(x2-x1), 0.1×(x2-x1)] (to avoid over-correction). If the corrected bounding box B2... If the intersection-union ratio (IoU) of the face region is ≥0.85 (compared with the manually labeled face region), then proceed to the third stage; otherwise, re-execute the first stage of localization. Third stage: Detector configuration: On the face region corrected in the second stage (based on the image region cropped from B2, with a uniform size of 224×224), build 6 independent component detectors, corresponding to the eyes, nose, mouth, eyebrows, cheeks, and forehead, respectively. Input the corrected face region image into the 6 detectors, and each detector outputs the bounding box coordinates of the corresponding component. Use the non-maximum suppression (NMS) algorithm to remove overlapping bounding boxes of the same component (IoU threshold = 0.3), retain the bounding box with the highest confidence, and output the final bounding box coordinates of the 6 key face components.

[0031] In one embodiment of the present invention, step S3, constructing the component relationship model, includes the following steps: The component relationship model is constructed by using the center coordinates of the bounding box of each face component as nodes to build an undirected graph model G=V,E, where V=v1, v2, ..., v6 represent 6 face component nodes and E is the edge between nodes. Define the edge weight eij = 11 + dij × cosθij, where dij is the Euclidean distance between nodes vi and vj, and θij is the angle between the line connecting the two nodes and the horizontal direction. Feature learning is performed on an undirected graph model using a graph convolutional network to output 64-dimensional component relationship features; The decoupled facial identity feature map is subjected to a dual pooling operation combining global average pooling and global max pooling to obtain two 128-dimensional feature vectors. These vectors are then fused by attention weighting to obtain 256-dimensional global facial features.

[0032] Specifically, the goal of decoupling is to ensure that facial identity features do not contain any information about what the person is wearing, and that the attribute features of the person wearing do not contain any information that can be used to identify a specific person. Based on the obtained pure features after decoupling, in order to further improve the robustness to delicate wearable objects and different postures, this invention further refines the feature extraction from three dimensions: local, correlation and global.

[0033] In one embodiment of the present invention, step S4 includes the following steps: A 3D pose regression network is used to take face image data as input, output face Euler angles, and classify five pose levels based on the Euler angles. A pose-attention mapping table is set up, and an initial attention weight matrix is ​​preset for each pose level. A dynamic weight adjustment module is introduced. With face pose information as input, the weight correction coefficient is output through a multilayer perceptron to correct the initial attention weight matrix. After obtaining the attention weights for pose adaptation, the global face features, local features of the worn object, and component relationship features are multiplied by the corresponding attention weights respectively. The feature importance score S for the three types of enhanced features is calculated as follows: S=13×VarFglobalMaxVar,VarFlocalMaxVar,VarFrelationMaxVar Where Var∙ is the feature variance, MaxVar is the maximum value of the three types of feature variance, and Fglobal, Flocal and Frelation are the global features of the face, the local region features of the worn object and the feature of the component relationship, respectively. Based on the feature importance score S, the three types of features are dynamically weighted and fused to obtain the fused feature.

[0034] Specifically, a 3D pose regression network is used. Face image data is input, and Euler angles (pitch angle β1, yaw angle β2, roll angle β3) are output. Five pose levels are defined based on the Euler angle values: frontal (β1≤15° and β2≤15°); slight leftward deviation (15°<β2≤30°); slight rightward deviation (-30°≤β2<-15°); slight upward deviation (15°<β1≤≤30°); slight downward deviation (-30°≤β1<-15°); corresponding to levels 1 to 5 respectively. Based on the feature importance score S, the three types of features are dynamically weighted and fused to obtain the fused feature, specifically: Ffusion=S1×Fglobal+S2×Flocal+S3×Frelation Wherein, Ffusion is the fusion feature, Fglobal, Flocal and Frelation are the global face feature, the local area feature of the worn object and the part relationship feature, respectively, and S1, S2 and S3 are the weight coefficients corresponding to the three features. In this embodiment, the values ​​of S1, S2 and S3 are 0.6, 1.0 and 0.4, respectively. The weight matrix dimensions are defined as follows: the attention weight matrices for global facial features, local features of the wearable device, and features related to the device are all column vectors of feature dimension × 1, denoted as W_g, W_l, and W_r, respectively. The initial weight preset rules are as follows: based on the effectiveness of features under different poses (e.g., when the face is turned to the side, the features related to the device are occluded and their weights need to be reduced; when the head is tilted up, the local features of the wearable device are more obvious and their weights need to be increased), the initial weights of each level are preset. The pose-attention mapping table is shown in Table 1:

[0035] In one embodiment of the present invention, step S5, the preset classification and recognition model includes the following steps: A feature adaptation layer combined with a dual-branch classification structure is adopted. The feature adaptation layer is passed through two fully connected layers. The dual-branch classification structure includes a coarse classification branch and a fine classification branch. The coarse classification branch is used to identify the major category of the wearable item, and the fine classification branch is used to identify the sub-category of the wearable item. The fused features are input into the classification and recognition model. After dimensional mapping and feature optimization through the feature adaptation layer, they are input into the dual-branch classification layer. The coarse classification branch outputs the predicted probability of the major category of the wearable item, and selects the category with the highest probability as the coarse classification result. The fine classification branch calls the classifier of the corresponding subclass based on the coarse classification result, outputs the predicted probability of the wearable item subclass, and selects the category with the highest probability as the fine classification result. If the maximum probability of the coarse classification result is less than 0.6, then the secondary recognition mechanism is activated. The local features of the worn object are re-extracted and spliced ​​with the fused features, then input into the classification and recognition model again for recognition, and finally the major category and sub-category information of the worn object are output.

[0036] Specifically, a structure combining a feature adaptation layer and a two-branch classification layer is adopted. The feature adaptation layer maps the fused features to 256-dimensional adaptation features through two fully connected layers (with 512 and 256 hidden neurons respectively). The two-branch classification layer includes a coarse classification branch and a fine classification branch. The coarse classification branch is used to identify the major categories of worn items (such as glasses, masks, hats, and accessories), using the Softmax activation function and cross-entropy loss. The fine classification branch is used to identify the subcategories of worn items (such as myopia glasses, sunglasses, medical masks, and KN95 masks), using a weighted Softmax activation function (the subclass weights are dynamically adjusted according to the proportion of subclass samples within the major category). The specific process of the secondary recognition mechanism is as follows: if the maximum probability of coarse classification is less than the threshold of 0.6, the system backtracks to the feature extraction step of the local area of ​​the wearer, uses more refined parameters (smaller sliding window step size) to re-extract features, and then concatenates the new local features with the original fused features and inputs them again into the dual-branch classifier for decision-making.

[0037] Please see Figure 2 As shown, this invention is a face recognition system based on feature fusion, comprising the following modules: Feature extraction module: Acquires facial image data including the worn object, constructs an identity encoder and an object encoder, and extracts facial identity features and object attribute features respectively by taking the facial image data as input; Feature decoupling optimization module: Construct an adversarial learning network, input facial identity features and accessory attribute features into the adversarial learning network, and decouple the two types of features through its internal preset adversarial mechanism to obtain decoupled facial identity features and accessory attribute features; Multi-dimensional feature refinement and extraction module: Processes the decoupled attribute features of the wearable object, extracts local region features of the wearable object, and performs facial component detection on the facial image data to determine the location information of key facial components in order to build a component relationship model; After extracting component relationship features from the component relationship model, global feature extraction is performed on the decoupled face identity features to obtain global face features; Pose-aware feature fusion module: Constructs a pose-aware attention mechanism model, inputs global facial features, local features of the worn object, and component relationship features into the attention mechanism model, performs pose detection on the facial image data, obtains facial pose information, dynamically adjusts the weight coefficients of global facial features, local features of the worn object, and component relationship features, and dynamically weights and fuses the three types of features to obtain fused features; Wearable item classification and recognition module: The fused features are input into a preset classification and recognition model for analysis and processing, and finally output the category information of the wearable item.

[0038] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A face-wear recognition method based on feature fusion, characterized in that, The method comprises the following steps: S1: obtaining face image data of a person wearing an object, constructing an identity encoder and an object encoder, inputting the face image data, and extracting face identity features and object attribute features respectively; S2: constructing an adversarial learning network, inputting the face identity features and the object attribute features into the adversarial learning network, and decoupling the two types of features through a preset adversarial mechanism in the network to obtain decoupled face identity features and object attribute features; S3: processing the decoupled object attribute features, extracting object local region features, and detecting face components of the face image data to determine the position information of the face key components, so as to construct a component relationship model; extracting component relationship features from the component relationship model, and extracting global features from the decoupled face identity features to obtain face global features; S4: constructing a posture-aware attention mechanism model, inputting the face global features, the object local region features, and the component relationship features into the attention mechanism model, detecting the posture of the face image data, and dynamically adjusting the weight coefficients of the face global features, the object local region features, and the component relationship features to dynamically weight and fuse the three types of features to obtain fused features; S5: inputting the fused features into a preset classification recognition model for analysis and processing, and finally outputting the category information of the object. 2.The feature fusion based face wearable recognition method of claim 1, wherein, In step S1, the identity encoder and the object encoder are constructed, comprising the following steps: The identity encoder adopts a lightweight convolutional neural network structure, including 1 input layer, 4 feature extraction units, and 1 fully connected layer. Each unit in the feature extraction unit is a series structure of a convolution layer, a batch normalization layer, a ReLU activation function, and a max-pooling layer. The convolution layer uses a 3*3 pixel size convolution kernel with a fixed step of 1. The first and third feature extraction units only contain one convolution layer, and the second and fourth feature extraction units contain two convolution layers. The fully connected layer flattens the feature map output by the fourth feature extraction unit into a one-dimensional vector and outputs the final face identity feature vector; The object encoder is based on a ResNet-18 framework, and a channel-spatial dual attention module is embedded after each residual block in the third and fourth stages. The module calculates the weight coefficients of each channel of the feature map output by the residual block through the channel attention branch, inputs the channel-weighted feature map into the spatial attention branch, locates the spatial region of the object on the feature map, assigns high weight to the region pixels and low weight to the background region, and finally outputs the object attribute feature vector. 3.The feature fusion based face wearable recognition method of claim 1, wherein, In step S2, the adversarial learning network is constructed, comprising the following steps: The adversarial learning network adopts a single generator-single discriminator architecture, the generator is a parallel structure of the identity encoder and the wearing encoder, synchronously outputs the face identity feature F_ID and the wearing attribute feature F_A, the discriminator adopts a 2-layer fully connected layer combined with a Sigmoid activation structure, and is used for judging whether the input feature is a pure feature pair (F_ID, F_A) without cross interference. 4.The feature fusion based face wearable recognition method of claim 3, wherein, The step of judging whether the input feature is a pure feature pair without cross interference includes the following steps: In the 2-layer fully connected layer combined with the Sigmoid activation structure of the discriminator, a double collaborative judgment mechanism is embedded, including a first judgment link and a second judgment link, and specifically: The first judgment link detects the dimension compliance of the input F_ID and F_A, extracts the first 128-dimensional features of F_ID and the full 128-dimensional features of F_A through a preset dimension mask matrix, calculates the cosine similarity of the two, and if the cosine similarity is less than or equal to a preset similarity threshold, it enters the second judgment; otherwise, it is directly defined as a cross-interference feature pair; The second judgment link calculates the variance value of the features of F_ID and F_A respectively through the first judgment link, and calculates the interference degree evaluation value I = σF_ID-σF_AmaxσF_ID, σF_A, wherein σF_ID is the variance value of F_ID, and σF_A is the variance value of F_A; If the interference degree evaluation value I is less than or equal to a preset interference degree threshold, F_ID and F_A are input to the fully connected layer; After the feature pair passing through the double collaborative judgment mechanism is processed by the first fully connected layer and the second fully connected layer, the probability value is output through the Sigmoid activation, and if the probability value is greater than 0.85, it is determined as a pure feature pair without cross interference. 5.The feature fusion based face wearable recognition method of claim 1, wherein, In step S2, the parameters of the two encoders are optimized through adversarial training, including the following steps: A composite loss function is constructed, specifically: Ltotal=ωL1∗L1+ωL2∗L2+ωL3∗L3 L1=max0,F_IDa-F_IDpos22-FIDa-FIDne22+α L2=-1Ni=1Nyi∙logpi+1-yi∙log1-pi×ui L3=EFID,FA~PdatalogDFID,FA+EFID',FA'~Pnoiselog1-DFID',FA' Wherein, Ltotal is the total loss value of the composite loss function, ωL1, ωL2 and ωL3 are the weights of the corresponding loss function, L1 is the identity feature fidelity loss, F_IDa is the face identity feature of the anchor sample, F_IDpos is the face identity feature of the positive sample, F_IDne is the face identity feature of the negative sample, α is the interval parameter, L2 is the wearing attribute classification loss, N is the number of samples, i is the sample index, yi is the wearing attribute true label, pi is the prediction probability, ui is the class balance weight, which is inversely proportional to the number of each category wearing in the sample set, L3 is the adversarial loss, E*P is the expected function, D is the discriminator output probability, Pdata is the real feature distribution, Pnoise is the interference feature distribution added with Gaussian noise, FID' and FA' are the interference face identity feature and interference wearing attribute feature added with Gaussian noise respectively; Through the adaptive estimation optimizer, iteration is performed until the composite loss function converges, so that the parameter optimization of the two encoders is completed. 6.The feature fusion based face wearable recognition method of claim 1, wherein, In the step S3, the decoupled wearing attribute feature is processed, including the following steps: A multi-scale sliding window mechanism is adopted, three window sizes are set, and the step length is 8 pixels. The decoupled wearing attribute feature map is slidingly sampled to obtain multi-scale local feature blocks. An adaptive feature screening unit is introduced to calculate the information entropy of each feature unit, and the feature units with information entropy greater than a preset information entropy threshold are screened. After the dimension is unified through a 1*1 convolution layer, the wearing local area feature is obtained by splicing. The preset information entropy threshold is obtained by calculating the average of the information entropy of all feature units. An improved cascaded convolutional neural network is adopted. In the first stage, a lightweight convolutional layer is used to locate the overall face region. In the second stage, a bounding box regression branch is introduced to correct the overall face region. In the third stage, six component detection heads are set to detect eyes, nose, mouth, eyebrows, cheeks and forehead respectively, and the bounding box coordinates of each component are output. 7.The feature fusion based face wearable recognition method of claim 1, wherein, In the step S3, a component relationship model is constructed, including the following steps: The component relationship model takes the center coordinates of the bounding boxes of each face component as nodes to construct an undirected graph model G=V,E, where V=v1, v2, …, v6 represents six face component nodes, and E is the edge between nodes. The edge weight eij is defined as 11+dij×cosθij, where dij is the Euclidean distance between nodes vi and vj, and θij is the angle between the two nodes and the horizontal direction. The feature learning of the undirected graph model is performed through the graph convolution network to output a 64-dimensional component relationship feature. For the decoupled face identity feature map, a double-pooling operation combining global average pooling and global maximum pooling is adopted to obtain two 128-dimensional feature vectors, and a 256-dimensional face global feature is obtained through attention weighted fusion. 8.The feature fusion based face wearable recognition method of claim 1, wherein, In the step S4, the following steps are included: A 3D pose regression network is adopted to input face image data and output face Euler angles, and five pose levels are divided according to the Euler angles. A posture-attention mapping table is set, an initial attention weight matrix is preset for each posture level, a dynamic weight adjustment module is introduced, the face posture information is taken as input, the weight correction coefficient is output through a multi-layer perception, the initial attention weight matrix is corrected, the posture-adapted attention weight is obtained, and the face global feature, the wearing object local area feature, and the component relationship feature are multiplied by the corresponding attention weight respectively; The feature importance score S of the three types of enhanced features is calculated, and the calculation is specifically as follows: S=13×VarFglobalMaxVar,VarFlocalMaxVar,VarFrelationMaxVar Wherein, Var is the feature variance, MaxVar is the maximum value of the three types of feature variances, Fglobal, Flocal and Frelation are the face global feature, the wearing object local area feature and the component relationship feature respectively; According to the feature importance score S, the three types of features are dynamically weighted and fused to obtain the fused feature. 9.The feature fusion based face wearable recognition method of claim 1, wherein, In the step S5, the preset classification recognition model comprises the following steps: A feature adaptation layer combined with a double-branch classification structure is adopted, the feature adaptation layer passes through 2 layers of full connection layer, the double-branch classification structure comprises a coarse classification branch and a fine classification branch, the coarse classification branch is used for identifying the wearing object category, and the fine classification branch is used for identifying the wearing object subcategory; The fused feature is input into the classification recognition model, after dimension mapping and feature optimization through the feature adaptation layer, the double-branch classification layer is input, the coarse classification branch outputs the wearing object category prediction probability, and the category with the maximum probability is selected as the coarse classification result; the fine classification branch calls the corresponding subcategory classifier according to the coarse classification result, outputs the wearing object subcategory prediction probability, and selects the category with the maximum probability as the fine classification result; Wherein, if the maximum probability of the coarse classification result is less than 0.6, a secondary recognition mechanism is started, the wearing object local area feature is re-extracted and spliced with the fused feature, and the classification recognition model is input again for recognition, and finally the wearing object category and subcategory information are output.

10. A face-wear recognition system based on feature fusion, for implementing the face-wear recognition method based on feature fusion according to any one of claims 1-9, characterized in that, The following modules are included: The feature extraction module: obtaining the face image data containing the wearing object, constructing the identity encoder and the wearing object encoder, taking the face image data as input, and extracting the face identity feature and the wearing object attribute feature respectively; The feature decoupling optimization module: constructing an adversarial learning network, inputting the face identity feature and the wearing object attribute feature into the adversarial learning network, decoupling processing the two types of features through the preset adversarial mechanism in the network, and obtaining the decoupled face identity feature and wearing object attribute feature; The multi-dimensional feature refinement extraction module: processing the decoupled wearing object attribute feature, extracting the wearing object local area feature, and detecting the face components of the face image data to determine the position information of the face key components, so as to construct a component relationship model; After extracting the component relationship feature from the component relationship model, the global feature of the decoupled face identity feature is extracted to obtain the face global feature; The posture perception feature fusion module: a posture perception attention mechanism model is constructed, the global facial feature, the local region feature of the wearing object and the component relationship feature are input into the attention mechanism model, the global facial feature, the local region feature of the wearing object and the component relationship feature are dynamically adjusted after the posture detection of the facial image data is performed and the facial posture information is obtained, the three types of features are dynamically weighted and fused to obtain the fused features; The wearing object classification and identification module: the fused features are input into a preset classification and identification model, analyzed and processed, and finally the category information of the wearing object is output.

Images