A power grid operation site face recognition method based on deep learning

Abstract

The application discloses the technical field of power grid operation, and particularly relates to a power grid operation site face recognition method based on deep learning, which comprises the following specific steps: S1: collecting face image pairs of power grid operation personnel and performing pretreatment to obtain a power grid operation personnel multispectral image pair dataset D1; S2: performing processing on any second face image pair P2 = <I''1, I'2> E D1 to obtain a power grid operation personnel multispectral image pair dataset D2; S3: constructing a face recognition model dynamic adversarial multi-modal network DAM-Net of the power grid site under the occlusion condition of the operation personnel, which is used for identity recognition of the operation personnel of the power grid operation site under the face occlusion condition; and S4: training and updating parameters of each layer of the DAM-Net network constructed in S3 to obtain a trained DAM-Net network. The DAM-Net of the application introduces a deformable convolution fusion submodule and a space-time attention mechanism submodule through a DODM, and has the advantages of being capable of significantly improving the accuracy and robustness of occlusion detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power grid operation, in particular to a power grid operation site face recognition method based on deep learning. BACKGROUND

[0002] In recent years, face recognition technology has been widely used in identity verification, security monitoring and other fields. However, in practical applications, power grid operators, security personnel and other industry practitioners often need to wear masks, safety helmets, goggles, scarves and other protective equipment, making it difficult for traditional methods based on complete face feature extraction to maintain high recognition accuracy.

[0003] Existing occluded face recognition algorithms mainly rely on generating restored face information for the occluded part to make the face features complete for recognition. However, the existing methods have the following problems:

[0004] 1. The traditional method uses predetermined geometric occlusions (such as rectangles and ellipses), which cannot adapt to the contour features of different target objects, resulting in excessive occlusion of the effective area or missing irregular occlusions.

[0005] 2. The traditional method lacks dynamic adaptability, and the existing mask parameters cannot be adjusted in real time according to the input features, resulting in significant increase in mask positioning error in scenes such as light changes and posture shifts.

[0006] 3. The discrimination mechanism is single, and the physiological features are missing.

[0007] Based on the above, the present application proposes a power grid operation site face recognition method based on deep learning. SUMMARY

[0008] To solve the above technical problems, according to one aspect of the present application, the present application provides the following technical scheme:

[0009] A power grid operation site face recognition method based on deep learning, comprising the following specific steps:

[0010] S1: Collecting face image pairs of power grid operators and pre-processing to obtain power grid operator multispectral image pair dataset D1;

[0011] S2: Processing any second face image pair P2 = <I'1, I'2> e D1 to obtain power grid operator multispectral image pair dataset D2;

[0012] S3: Constructing a face recognition model dynamic adversarial multi-modal network DAM-Net for power grid site operators under occlusion, for identity recognition of power grid site operators under face occlusion;

[0013] S4: training and updating the parameters of each layer of the DAM-Net network constructed in S3 to obtain a trained DAM-Net network;

[0014] S5: applying the trained DAM-Net network to identify the face-occluded personnel in the power grid operation scene, and outputting the identity label and confidence of the target personnel.

[0015] As a preferred scheme of the power grid operation site face recognition method based on deep learning, the specific steps of S1 are as follows:

[0016] S11: using a multispectral imaging device to collect face images of operation personnel at various power grid operation sites, and obtaining a first face image pair P1=<I1, I2> of the operation personnel, wherein I1 is a visible light image of the face of the operation personnel, and I2 is an infrared image of the face of the operation personnel;

[0017] S12: performing spatial alignment and image normalization processing on I1 and I2 to obtain a second face image pair P2=<I'1, I'2>.

[0018] S13: constructing a multispectral image pair dataset D1 of power grid operation personnel from all the second face image pairs P2=<I'1, I'2>.

[0019] As a preferred scheme of the power grid operation site face recognition method based on deep learning, the specific steps of S12 are as follows:

[0020] S121: performing spatial alignment on I1 to I2 through an affine transformation matrix to obtain a spatially aligned visible light image I'1 of the face, so that I'1 and I2 are matched in space, and the calculation formula of I'1 is as follows:

[0021] I'1 = warpAffine(I1, M)

[0022] wherein warpAffine represents affine transformation, M is an affine transformation matrix, and is a 2*3 matrix, and has the form of wherein a, b, c, and d control image rotation, scaling, and shearing; and c and f control translation.

[0023] S122: performing channel normalization on I'1 to obtain a normalized visible light image I"1 of the face, and performing linear normalization on I2 to obtain a normalized infrared image I'2 of the face, thereby improving the accuracy of subsequent processing and analysis.

[0024] As a preferred scheme of the power grid operation site face recognition method based on deep learning, wherein the specific steps of S2 are as follows:

[0025] S21: Geometric occlusion generation is performed on I"1 to obtain a visible light simulation occluded face image I m1 ; semantic occlusion generation is performed on I"1 through 3D projection technology and GAN texture generation network to obtain a visible light simulation occluded face image I m2 ; and fast gradient sign method-based adversarial occlusion generation is performed on I"1 to obtain a visible light simulation occluded face image I m3 ;

[0026] S22: The name of the personnel is taken as a label, and the face region of the visible light simulation occluded face image I m1 is framed and labeled to obtain an annotated visible light simulation occluded face image I' m1 ; the face region of the visible light simulation occluded face image I m2 is framed and labeled to obtain an annotated visible light simulation occluded face image I' m2 ; the face region of the visible light simulation occluded face image I m3 is framed and labeled to obtain an annotated visible light simulation occluded face image I' m3 ; the name of the personnel is taken as a label, and the face region of I'2 is framed and labeled to obtain an annotated infrared face image y;

[0027] S23: A third sub-operation personnel face image pair S1=<I' m1 , y> is constructed from I' m1 and y; a fourth sub-operation personnel face image pair S2=<I' m2 , y> is constructed from I' m2 and y; a fifth sub-operation personnel face image pair S3=<I' m3 , y> is constructed from I' m3 and y; and all S1, S2 and S3 are collected to obtain a power grid operation personnel multispectral image pair dataset D2.

[0028] As a preferred scheme of the power grid operation site face recognition method based on deep learning, wherein the specific steps of S3 are as follows:

[0029] S31: A dynamic occlusion detection module DODM is designed, P3=<x, y> is input into the DODM, and a face occlusion prediction heat map M _occ is obtained.

[0030] S32: An adversarial feature generator AFG is constructed, M _occand x are input into the AFG to generate a face repair feature map F face ;

[0031] S33: A multi-modal discriminant network MDN is constructed, x, M _occ and F face are input into the MDN to obtain a result matrix result, which includes a final identity label and a corresponding confidence.

[0032] As a preferred scheme of the power grid operation site face recognition method based on deep learning, the specific steps of S31 are as follows:

[0033] S311: x is input into the first three residual block groups of the ResNet18 encoder to obtain a simulated occluded face feature map F1; y is input into the ViT encoder to obtain an infrared face feature map F2;

[0034] S312: A deformable convolution fusion sub-module DCFS is designed, F1 and F2 are input into the DCFS to obtain a fusion feature map F4 with a dimension of after feature fusion;

[0035] S313: A space-time attention mechanism sub-module SAMS is designed, F4 is input into the SAMS to obtain a face occlusion prediction heat map M _occ with a dimension of 1xHxW;

[0036] The specific steps of S312 are as follows:

[0037] S3121: F1 and F2 are spliced through a dual-modal splicing method to obtain a spliced feature map F3 with a dimension of ;

[0038] S3122: F3 is input into an offset prediction layer, convolution operation is performed with a convolution kernel size of 3x3, a step of 1, and zero padding of 1, then input into a ReLU layer and a convolution layer with a convolution kernel size of 3x3, a step of 1, and zero padding of 1, nonlinear activation operation and convolution operation are performed to obtain an offset feature map △p with a dimension of ;

[0039] S3123: F1, F2 and △p are input into a deformable convolution layer and an upsampling layer to perform deformable convolution operation and upsampling operation to obtain a fusion feature map F4 with a dimension of ;

[0040] F4 = Upsample(DCNv1(F1, F2, △p))

[0041] Wherein, DCNv1 represents a deformable convolution layer operation; Upsample represents an up-sampling operation;

[0042] The specific steps of S313 are as follows:

[0043] S3131: design a channel attention network, input F4 into the channel attention network to obtain a channel-by-channel multiplication feature map F c ;

[0044] S3132: design a spatial attention network, input F c into the spatial attention network to obtain M _occ ;

[0045] The specific steps of S3131 are as follows:

[0046] S31311: input F4 into a global average pooling layer to perform an average pooling operation to obtain an average pooling feature map F5 with a dimension of ; then input F5 into a shared multi-layer perceptron to obtain an average weighted feature map F7 with a dimension of ;

[0047] S31312: input F4 into a global maximum pooling layer to perform a maximum pooling operation to obtain a maximum pooling feature map F6 with a dimension of ; then input F6 into a shared multi-layer perceptron to obtain a maximum weighted feature map F8;

[0048] S31313: perform an element-by-element addition operation on F7 and F8 to obtain a pooling concatenation feature map F9 with a dimension of ; then input F9 into Sigmoid to perform an activation operation to obtain a pooling concatenation activation feature map F with a dimension of 10 ;

[0049] F9=add(F7,F8)

[0050] F 10 =Sigmoid(F9)

[0051] Wherein, add(F7,F8) represents an add operation, i.e., element-by-element addition of F7 and F8, and Sigmoid represents a nonlinear activation operation;

[0052] S31314: perform a channel-by-channel multiplication operation on F4 and F 10 to obtain a channel-by-channel multiplication feature map F c with a dimension of ;

[0053]

[0054] wherein represents a channel-wise multiplication operation;

[0055] The specific steps of S3132 are as follows:

[0056] S31321: input F c to a channel average pooling layer to perform a channel average pooling operation to obtain an average pooling feature map F 11 with a dimension of 1xHxW;

[0057] S31322: input F C to a channel maximum pooling layer to perform a channel maximum pooling operation to obtain a channel pooling feature map F 12 with a dimension of 1xHxW;

[0058] S31323: perform a Concat splicing operation on F 11 and F 12 to obtain a pooling splicing feature map F 13 with a dimension of 2xHxW; then input F 13 to a convolution layer with a convolution kernel size of 7x7, a step of 1, and zero padding of 1 to perform a convolution operation to obtain a spatial attention feature map F 14 with a dimension of 1xHxW; subsequently input F 14 to a Sigmoid to perform an activation operation to obtain M _occ with a dimension of 1xHxW;

[0059] F 13 = Concat(F 11 , F 12 )

[0060] F 14 = Conv 7×7 (F 13 )

[0061] M _occ = Sigmoid(F 14 )

[0062] wherein Concat(F 11 , F 12 ) represents a Concat splicing operation on F 11 and F 12 .

[0063] As a preferred scheme of the power grid operation site face recognition method based on deep learning, the specific steps of S32 are as follows:

[0064] S321: input x and M_occ Performing a concat concatenation operation yields a face fusion feature map F with dimensions (C+1)×H×W. 15 ;

[0065] F 15 =Concat(x,M) _occ )

[0066] Where Concat(x,M) _occ ) represents the pair of x and M _occ Perform a Concat concatenation operation;

[0067] S322: Construct a structural restoration channel submodule SRC to connect F 15 The input is fed into the SRC to obtain the structural recovery feature map F. struct ;

[0068] S323: Construct a texture restoration channel submodule TRC, which will store M _occ The randomly generated noise vector z is input into the texture restoration channel to obtain a texture restoration feature map F with dimensions C×H×W. texture ;

[0069] S324: Construct a dynamic region fusion submodule DRFM, which will integrate F struct and F texture The input is fed into the Dynamic Region Fusion (DRFM) submodule to obtain F. face ;

[0070] The specific steps of S322 are as follows:

[0071] S3221: F 15 Inputting into a ReLU layer and a convolutional layer with a kernel size of 7×7, a stride of 2, and zero padding of 3, yields a dimension of... Face fusion enhancement feature map F 16 ;

[0072] F 16 =Conv 7×7 (ReLU(F 15 ))

[0073] S3222: F 16 The data is sequentially input into four cascaded void residual blocks to obtain the fused structure recovery feature map F. 17 The first void residual block, the second void residual block, the third void residual block, and the fourth void residual block have the same structure.

[0074] S3223: F 17The input is input to a convolution layer with a convolution kernel size of 3x3, a step of 1, and zero padding of 1, a ReLU layer, and an up-sampling layer, to perform convolution operation, activation operation, and up-sampling operation with an up-sampling factor of 2, to obtain F struct ;

[0075] F struct =Upsample(ReLU(Conv 3×3 (F 17 )))

[0076] where Upsample represents performing up-sampling operation;

[0077] The specific steps of the S323 are as follows:

[0078] S3231: randomly generating a noise vector z, inputting the z into a multi-layer perception to obtain a noise feature vector f1;

[0079] S3232: inputting the M _occ into a global average pooling layer to perform average pooling operation to obtain a face pooling scalar g; then inputting the g into a multi-layer perception to obtain a face feature vector f2;

[0080] S3233: performing Concat splicing operation on f1 and f2 to obtain a noise face splicing feature vector f3; inputting the f3 into a multi-layer perception MLP to obtain a style feature vector w;

[0081] S3234: performing AdaIN modulation operation on w and M _occ to obtain F texture ;

[0082] The specific steps of the S324 are as follows:

[0083] S3241: inputting F struct into a global average pooling layer and a convolution layer with a convolution kernel size of 1x1, a step of 1, and zero padding of 0, to perform global average pooling operation and convolution operation to obtain a structure weight vector a with a dimension of Cx1x1;

[0084] a=Conv 1×1 (GAP(F struct ))

[0085] S3242: inputting F texture into a convolution layer with a convolution kernel size of 3x3, a step of 1, and zero padding of 1, to perform convolution operation to obtain a texture weight feature map β with a dimension of 1xHxW;

[0086] β=Conv 3×3 (F texture )

[0087] S3243: F struct , F texture , alpha, beta are weighted fusion operations, and F face with a dimension of CxHxW is obtained

[0088] F face = alpha * F struct + (1-alpha) * (beta o F texture )

[0089] Where o represents the product operation; represents dot product; + represents vector addition.

[0090] As a preferred scheme of the power grid operation site face recognition method based on deep learning, wherein: the specific steps of S33 are as follows:

[0091] S331: An identity authentication sub-module is constructed, and x and F face are input into the identity authentication sub-module, and an identity similarity score vector S id is output.

[0092] S332: A repair authentication sub-module is constructed, and F face and M _occ are input into the repair authentication sub-module, and a real score vector D real is output.

[0093] S333: S id and D real are decision fusion operations, and then max operation is performed to obtain a result matrix result, wherein result is a two-dimensional matrix, including the final identification identity label and its corresponding confidence;

[0094] result = max (W id ·S id +W real ·D real )

[0095] Where W id represents the weight of the identity authentication branch; W real represents the weight of the generation authentication branch, and max represents the maximum value and its index.

[0096] The specific steps of S331 are as follows:

[0097] S3311: F face is subjected to feature extraction operation based on ArcFace algorithm to obtain repair face feature vector f gen ;

[0098] S3312: Perform feature extraction operation on x based on ArcFace algorithm to obtain occluded face feature vector f real ;

[0099] S3313: Perform cosine similarity calculation on f gen and f real to obtain identity similarity score vector S id ;

[0100] S id =1-cos({f gen |f real})

[0101] wherein cos is a cosine similarity calculation function.

[0102] The specific steps of S332 are as follows:

[0103] S3321: Input F face to the PatchGAN encoder to obtain the repaired face image discrimination feature map F PGAN with a dimension of ;

[0104] S3322: Input M _occ to the spatial attention layer to obtain the spatial attention weighted feature map F space with a dimension of ;

[0105] S3323: Perform Concat splicing operation on F PGAN and F space to obtain the multi-scale feature map F mul with a dimension of ; then input F mul to the convolution layer with a convolution kernel size of 1x1, a step of 1, and zero padding of 1, and perform convolution operation to obtain the authenticity score D real ;

[0106] F mul = Concat(F PGAN , F space )

[0107] D real = conv 1×1 (F mul )

[0108] wherein Concat(F PGAN , F space ) represents performing Concat splicing operation.

[0109] As a preferred scheme of the power grid operation site face recognition method based on deep learning, wherein the specific steps of S4 are as follows:

[0110] S41: initialize the hyperparameters required for training DAM-Net;

[0111] S42: divide the power grid operation personnel multispectral image pair data set constructed in S2 into a training set and a test set according to a certain proportion, and ensure that the two sets do not overlap; then divide the training set into multiple batches, input a batch of training set into the network each time for training, and calculate the loss value of the batch;

[0112] S43: after each round of traversing all batches of the training set, the performance of the model is evaluated periodically using the test set data; the test set is sent into the model according to the same batch size as the training stage, the prediction error of each batch is calculated respectively, and the test set loss value of the current round is obtained by summarizing; the test set loss value can be used to dynamically track the performance change of the network on non-training data; by continuously observing the change trend of the test set loss value, it is judged whether the model has overfitting signs; when the test set loss value no longer decreases or appears a rising trend in continuous multiple training periods, it can be judged that the model generalization ability decreases, and then the preset strategy adjustment mechanism is triggered; when the test set loss value tends to be stable with the training process, and reaches the performance expectation standard, it can be considered that the network training is completed, and the final network parameters will be used for subsequent face recognition of power grid operation personnel in the face covering situation.

[0113] As a preferred scheme of the power grid operation site face recognition method based on deep learning, wherein the specific steps of S5 are as follows:

[0114] S51: after the DAM-Net network training is completed, it is applied to the face recognition task of personnel in the actual power grid operation scene in the face covering situation;

[0115] S52: input the collected power grid operation personnel identity image into the DAM-Net network which has been trained, and the model identifies each power grid operation personnel in the image one by one, and outputs the corresponding identity label and confidence of each personnel in the image.

[0116] Compared with the prior art:

[0117] The DAM-Net of the application introduces deformable convolution fusion sub-modules and space-time attention mechanism sub-modules through DODM, has the advantages that the accuracy and robustness of occlusion detection can be significantly improved, in addition, through AFG, structure recovery channel sub-modules and texture recovery channel sub-modules are adopted and combined with a dynamic region fusion sub-module, has the advantages that the integrity and authenticity of the repaired features can be ensured, at the same time, the identity identification sub-module and the repair identification sub-module of the MDN are combined, and decision fusion is adopted, the reliability and accuracy of the occlusion feature recognition are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0118] Figure 1 It is a flowchart of the application;

[0119] Figure 2 It is a face recognition network DAM-Net structure diagram under the occlusion condition of the application;

[0120] Figure 3 It is a DODM module structure diagram of the application;

[0121] Figure 4 It is a deformable convolution fusion sub-module DCFS structure diagram of the application;

[0122] Figure 5 It is an offset prediction layer structure diagram of the application;

[0123] Figure 6 It is a space-time attention mechanism sub-module SAMS structure diagram of the application;

[0124] Figure 7 It is a channel attention network structure diagram of the application;

[0125] Figure 8 It is a spatial attention network structure diagram of the application;

[0126] Figure 9 It is an AFG structure diagram of the application;

[0127] Figure 10 It is a structure recovery channel sub-module SRC structure diagram of the application;

[0128] Figure 11 It is a texture recovery channel sub-module TRC structure diagram of the application;

[0129] Figure 12 It is a dynamic region fusion sub-module DRFM structure diagram of the application;

[0130] Figure 13 It is a MDN module structure diagram of the application;

[0131] Figure 14 It is an identity identification sub-module structure diagram of the application;

[0132] Figure 15 The structure diagram of the repair identification sub-module of the application. DETAILED DESCRIPTION

[0133] In order to make the purpose, technical scheme and advantages of the application more clear, the embodiments of the application will be further described in detail below with reference to the drawings.

[0134] The application provides a power grid operation site face recognition method based on deep learning, please refer to Figures 1-15 , including the specific steps as follows:

[0135] S1: collecting face image pairs of power grid operation personnel and performing preprocessing to obtain a power grid operation personnel multispectral image pair dataset D1;

[0136] The specific steps of S1 are as follows:

[0137] S11: using a multispectral imaging device to collect face images of operation personnel at various power grid operation sites, for any one operation personnel, obtaining a first face image pair P1=<I1, I2> of the operation personnel, wherein I1 is a face visible light image of the operation personnel, and I2 is a face infrared image of the operation personnel;

[0138] S12: performing spatial alignment and image normalization processing on I1 and I2 to obtain a second face image pair P2=<I'1, I'2>;

[0139] The specific steps of S12 are as follows:

[0140] S121: P1 performs spatial alignment on I1 to I2 through an affine transformation matrix to obtain a face visible light image I'1 after spatial alignment, so that I'1 and I2 are matched in space, and the calculation formula of I'1 is as follows:

[0141] I'1 = warpAffine(I1, M)

[0142] Wherein warpAffine represents affine transformation; M is an affine transformation matrix, which is a 2x3 matrix, and the form is as follows: Wherein a, b, c, d control image rotation, scaling, and shearing; c, f control translation;

[0143] The so-called matching refers to that the face visible light image and the face infrared image are aligned in a unified spatial coordinate system, and it is ensured that the position, shape and proportion of the same scene or object in the two images are consistent;

[0144] Wherein, the affine transformation matrix is a matrix used to represent linear transformation and displacement of an image or a spatial object in two-dimensional or three-dimensional space;

[0145] S122: Channel normalization is performed on I'1 to obtain a normalized visible light image I"1 of the face; linear normalization is performed on I2 to obtain a normalized infrared image I'2 of the face, thereby improving the accuracy of subsequent processing and analysis;

[0146] Channel normalization refers to standardizing each channel, which helps improve training stability and accelerate convergence;

[0147]

[0148] μ = Apool(I'1)

[0149]

[0150] where X' i represents the result of standardization of the i-th pixel value; X i represents the i-th pixel value; μ is the result of global average pooling of the image, i.e., the average of all pixel values, Apool represents global average pooling; σ is the standard deviation of the image, i.e., the standard deviation of all pixel values; N represents the total number of pixels of the image;

[0151] Linear normalization refers to a common data normalization method that scales data to a fixed range (usually [0, 1]);

[0152]

[0153] where X' I2 represents the result of linear normalization of X I2 , i.e., any pixel value of I'2; X I2 represents any pixel value in I2; represents the minimum pixel value in I2, represents the maximum pixel value in I2;

[0154] S13: All second face image pairs P2 = <I"1, I'2> form a power grid worker multispectral image pair dataset D1;

[0155] S2: Process any second face image pair P2 = <I"1, I'2> ∈ D1 to obtain a power grid worker multispectral image pair dataset D2;

[0156] The specific steps of S2 are as follows:

[0157] S21: Perform geometric occlusion generation on I"1 to obtain a visible light simulation occluded face image I m1; semantic occlusion generation on I"1 by 3D projection technology and GAN texture generation network, to obtain visible light simulation occluded face image I m2 ; adversarial occlusion generation on I"1 based on Fast Gradient-Sign Method (FGSM), to obtain visible light simulation occluded face image I m3 ;

[0158] Wherein: geometric occlusion generation generally refers to generating occlusion on images using simple geometric shapes (such as rectangles, circles, polygons, etc.); it is mainly used to simulate the visual occlusion caused by opaque objects, and is usually used to enhance the diversity of data;

[0159] 3D projection technology: using 3D modeling tools (such as Blender) to project actual objects (such as masks, glasses, etc.) onto face images to form semantic occlusion; this method can accurately simulate the influence of occlusion on face images according to the geometry and texture of the object;

[0160] GAN texture generation network: using CycleGAN or similar generative adversarial network to fuse the texture of the occluded area with the background image to ensure that the light and texture of the occlusion in the image are consistent with the actual situation;

[0161] FGSM adversarial attack algorithm: by calculating the gradient information of the image, a small perturbation is generated and added to the original image; specifically, the FGSM algorithm calculates the gradient of each pixel point of the image according to the loss function to obtain the sensitivity (gradient) of the image to the loss function; then, by symbol processing these gradient information, a perturbation is generated, and the perturbation is added to the original image to generate an adversarial sample; the perturbation has a disturbance effect, which can make the deep learning model incorrectly recognize the occluded area, but at the same time has little effect on the overall visual effect of the image;

[0162] S22: frame and label the face region of the visible light simulation occluded face image I m1 , to obtain the labeled visible light simulation occluded face image I' m1 ; frame and label the face region of the visible light simulation occluded face image I m2 , to obtain the labeled visible light simulation occluded face image I' m2 ; frame and label the face region of the visible light simulation occluded face image I m3 , to obtain the labeled visible light simulation occluded face image I' m3 ; frame and label the face region of I'2 with the name of the person as the label, to obtain the labeled infrared face image y;

[0163] S23: From I' m1 and y, construct the third sub-operator face image pair S1 = <I' m1 , y>; From I' m2 and y, construct the fourth sub-operator face image pair S2 = <I' m2 , y>; From I' m3 and y, construct the fifth sub-operator face image pair S3 = <I' m3 , y>; Collect all S1, S2, S3 to obtain a dataset D2 of power grid operator multispectral image pairs;

[0164] S3: Construct a dynamic adversarial multimodal network DAM-Net (Dynamic Adversarial Multimodal Net) for face recognition in the case of operator occlusion in the power grid site, which is used for identity recognition of operators in the power grid operation site under face occlusion;

[0165] The structure of the dynamic adversarial multimodal network DAM-Net constructed by the present invention is as Figure 2 shown;

[0166] For any operator face image pair P3 = <x, y> ∈ D2, where the visible light simulated occluded face image x refers to I' m1 , I' m2 or I' m3 ; Input P3 = <x, y> into the dynamic occlusion detection module DODM (Dynamic Occlusion Detection Module) to obtain the face occlusion prediction heat map M _occ ; Subsequently, input x and M _occ into the adversarial feature generator AFG (Adversarial Feature Generator) to obtain the face repair feature map F face ; Then input x, M _occ and F face into the multimodal discrimination network MDN (Multimodal Detection Network) to obtain the result matrix result, and result includes the final identity label and its corresponding confidence;

[0167] The specific steps of S3 are as follows:

[0168] S31: Design a dynamic occlusion detection module DODM (Dynamic Occlusion Detection Module), input P3 = <x, y> into DODM to obtain the face occlusion prediction heat map M _occ ;

[0169] The overall structure of the dynamic occlusion detection module DODM is shown in Figure 1, and the execution process is as follows: Figure 3

[0170] The x with the dimension of CxHxW is input into the first three residual block groups of the ResNet18 encoder to obtain the simulation occluded face feature map F1 with the dimension of The labeled infrared face image y with the dimension of 1xHxW is input into the ViT encoder to obtain the infrared face feature map F2 with the dimension of Subsequently, F1 and F2 are input into the deformable convolution fusion submodule DCFS to obtain the fusion feature map F4 with the dimension of Then, F4 is input into the spatio-temporal attention mechanism submodule SAMS to obtain the face occlusion prediction heat map M _occ with the dimension of 1xHxW.

[0171] The ResNet18 encoder processes images through multiple convolution layers and residual block groups; the image input into the ResNet18 encoder is first subjected to convolution layers to extract low-level features, then subjected to a series of residual block groups for deep feature extraction, and finally subjected to a fully connected layer to generate a high-level feature representation of the image; in the present application, the first three residual block groups are mainly used to extract shallow and middle semantic features of the image in layers, and at the same time, gradually compress the spatial scale and increase the number of feature channels through layer-by-layer down-sampling, thereby providing a basis for subsequent high-level semantic modeling.

[0172] The ViT (Vision Transformer) encoder does not rely on convolution operations, but uses a self-attention mechanism (Self-Attention) to process images; compared with a convolutional neural network (CNN), it can capture more rich global information.

[0173] The infrared image has only one channel (single channel), i.e., each pixel contains only one value (temperature or infrared radiation intensity).

[0174] The specific steps of the S31 are as follows:

[0175] S311: input x into the first three residual block groups of the ResNet18 encoder to obtain the simulation occluded face feature map F1; input y into the ViT encoder to obtain the infrared face feature map F2.

[0176] S312: design a deformable convolution fusion submodule DCFS (Deformable convolution fusion submodule) to input F1 and F2 into the DCFS to obtain a fusion feature with the dimension of ​a fusion feature map F4; the network structure of the deformable convolution fusion submodule DCFS designed by the application is as shown in Figure 4

[0177] The specific steps of S312 are as follows:

[0178] S3121: obtaining a spliced feature map F3 with a dimension of by splicing F1 and F2 through a double-modal splicing operation;

[0179] The double-modal splicing operation is to fuse data of two different modalities (for example, visible light and infrared images) together; the splicing operation is to connect feature maps of two modalities along a certain dimension (usually the channel dimension) so as to enable the model to simultaneously learn the features of the two modalities in a higher-dimensional space.

[0180] S3122: inputting F3 into an offset prediction layer to perform a convolution operation with a convolution kernel size of 3x3, a step of 1 and zero padding of 1, and then inputting into a ReLU layer and a convolution layer with a convolution kernel size of 3x3, a step of 1 and zero padding of 1 to perform a nonlinear activation operation and a convolution operation, to obtain an offset feature map Δp with a dimension of The structure diagram of the offset prediction layer is as shown in Figure 5

[0181] S3123: inputting F1, F2 and Δp into a deformable convolution layer and an up-sampling layer to perform a deformable convolution operation and an up-sampling operation, to obtain a fusion feature map F4 with a dimension of

[0182] F4 = Upsample(DCNv1(F1, F2, Δp))

[0183] Wherein, DCNv1 represents a deformable convolution layer operation; Upsample represents an up-sampling operation.

[0184] Wherein, deformable convolution (Deformable Convolution, DConv) is a technology aiming to enhance the spatial adaptability of a convolutional neural network.

[0185] S313: designing a spatiotemporal attention mechanism submodule SAMS, inputting F4 into the SAMS to obtain a face occlusion prediction heat map M _occ with a dimension of 1xHxW; the network structure of the spatiotemporal attention mechanism submodule SAMS designed by the application is as shown in Figure 6

[0186] ​​​​The specific steps of S313 are as follows:

[0187] S3131: A channel attention network is designed, F4 is input into the channel attention network, and a channel-by-channel multiplication feature map F is obtained c ;

[0188] The specific steps of S3131 are as follows:

[0189] S31311: F4 is input into a global average pooling layer to perform an average pooling operation to obtain an average pooling feature map F5 with a dimension of ; then F5 is input into a shared multi-layer perception to obtain an average weighted feature map F7 with a dimension of ;

[0190] Wherein, the shared multi-layer perception refers to applying the same MLP parameters to each element when processing multiple input elements (such as each point in the point cloud); this parameter sharing mechanism ensures that the model is not sensitive to the arrangement order of the input elements;

[0191] S31312: F4 is input into a global maximum pooling layer to perform a maximum pooling operation to obtain a maximum pooling feature map F6 with a dimension of ; then F6 is input into a shared multi-layer perception to obtain a maximum weighted feature map F8;

[0192] S31313: F7 and F8 are subjected to an element-by-element addition operation (i.e., an add operation in Figure 7 ), to obtain a pooling splicing feature map F9 with a dimension of ; then F9 is input into Sigmoid to perform an activation operation to obtain a pooling splicing activation feature map F 10 ;

[0193] F9 = add(F7, F8)

[0194] F 10 = Sigmoid(F9)

[0195] Wherein, add(F7, F8) represents an add operation, i.e., element-by-element addition of F7 and F8, and Sigmoid represents a nonlinear activation operation;

[0196] S31314: F4 and F 10 are subjected to a channel-by-channel multiplication operation to obtain a channel-by-channel multiplication feature map F c ;

[0197]

[0198] Wherein ​​representing a per-channel multiplication operation;

[0199] Embodiments:

[0200] Suppose that the size of the fusion feature map input to the channel attention network is CxHxW, wherein C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map;

[0201] The size of the fusion feature map F4 is 256x224x224, which is input to a global average pooling layer to obtain an average pooling feature map F5 with a size of 256x1x1; F5 is input to a shared MLP to obtain an average weighted feature map F7 with a size of 256x1x1;

[0202] F4 is input to a global maximum pooling layer to perform a maximum pooling operation to obtain a maximum pooling feature map F6 with a size of 256x1x1; then F6 is input to the shared MLP to obtain a maximum weighted feature map F8 with a size of 256x1x1;

[0203] F7 and F8 are subjected to an add operation to obtain a pooling splicing feature map F9 with a size of 256x1x1; then F9 is input to a Sigmoid to perform an activation operation to obtain a pooling splicing activation feature map F 10 ;

[0204] F4 and F 10 are subjected to a per-channel multiplication operation to obtain a per-channel multiplication feature map F c ;

[0205] S3132: a spatial attention network is designed, and F c is input to the spatial attention network to obtain M _occ ; the overall structure of the spatial attention network designed by the application is shown in Figure 8 ;

[0206] The specific steps of S3132 are as follows:

[0207] S31321: F c is input to a channel average pooling layer to perform a channel average pooling operation to obtain an average pooling feature map F 11 ;

[0208] S31322: F C is input to a channel maximum pooling layer to perform a channel maximum pooling operation to obtain a channel pooling feature map F 12 ;

[0209] S31323: F 11and F 12 Performing a concat concatenation operation yields a pooled concatenated feature map F with dimensions 2×H×W. 13 Then F 13 The input is fed into a convolutional layer with a kernel size of 7×7, a stride of 1, and zero padding of 1, and a convolution operation is performed to obtain a spatial attention feature map F with dimensions of 1×H×W. 14 ; then F 14 The input is fed into a sigmoid function, and an activation operation is performed to obtain an M with dimensions 1×H×W. _occ ;

[0210] F 13 =Concat(F 11 ,F 12 )

[0211] F 14 =Conv 7×7 (F 13 )

[0212] M _occ =Sigmoid(F 14 )

[0213] Where Concat(F) 11 ,F 12 ) represents F 11 and F 12 Perform a Concat concatenation operation;

[0214] Example:

[0215] Channel multiplication feature map F of input spatial attention network c The size is 256×224×224. When input to the channel average pooling layer, an average pooling feature map F with a size of 1×224×224 is obtained. 11 ; F c The input is fed into the channel max pooling layer to obtain an average pooling feature map F with a size of 1×224×224. 12 ;

[0216] F 11 and F 12 Performing a concat concatenation operation yields a pooled concatenated feature map F with a size of 2×224×224. 13 Then F 13 The input is fed into a convolutional layer with a kernel size of 7×7, a stride of 1, and zero padding of 1, and a convolution operation is performed to obtain a spatial attention feature map F with a size of 1×224×224. 14 ; then F 14The input is fed into a Sigmoid function and activated to obtain a face occlusion prediction heatmap M with a size of 1×224×224. _occ ;

[0217] S32: Construct an Adversarial Feature Generator (AFG) to generate M... _occ Inputting x into AFG generates a face restoration feature map F. face ;

[0218] The specific steps of S32 are as follows:

[0219] S321: Combine x and M _occ Performing a concat concatenation operation yields a face fusion feature map F with dimensions (C+1)×H×W. 15 ;

[0220] F 15 =Concat(x,M) _occ )

[0221] Where Concat(x,M) _occ ) represents the pair of x and M _occ Perform a Concat concatenation operation;

[0222] Among them, the Concat stitching operation is a way to increase the number of channels, which means that the number of channels of the image itself increases, but the information under each feature does not increase. It mainly achieves the superposition in horizontal or vertical space.

[0223] S322: Construct a Structural Recovery Channel (SRC) submodule to connect F 15 The input is fed into the SRC to obtain the structural recovery feature map F. struct This invention designs a Structural Recovery Channel (SRC) submodule, the structure of which is as follows: Figure 10 As shown;

[0224] The specific steps of S322 are as follows:

[0225] S3221: F 15 Inputting into a ReLU layer and a convolutional layer with a kernel size of 7×7, a stride of 2, and zero padding of 3, yields a dimension of... Face fusion enhancement feature map F 16 ;

[0226] F 16 =Conv 7×7 (ReLU(F15 ))

[0227] S3222: F 16 is input into four serial dilated residual blocks to obtain the fused structure recovery feature map F 17 , wherein the first dilated residual block, the second dilated residual block, the third dilated residual block and the fourth dilated residual block have the same structure;

[0228] The dilated residual block (DRB) is a deep learning module combining dilated convolution and residual connection, which is a publicly known technology.

[0229] S3223: F 17 is input into a convolution layer with a convolution kernel size of 3x3, a step of 1 and zero padding of 1, a ReLU layer and an up-sampling layer to perform convolution operation, activation operation and up-sampling operation with an up-sampling factor of 2, thereby obtaining F struct with a dimension of CxHxW.

[0230] F struct = Upsample(ReLU(Conv 3×3 (F 17 )))

[0231] Upsample represents up-sampling operation.

[0232] Embodiment:

[0233] Suppose the size of the face fusion feature map input into the structure recovery channel sub-module SRC is CxHxW, wherein C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map.

[0234] The face fusion feature map F 15 has a size of 4x224x224, which is input into a ReLU layer and a convolution layer with a convolution kernel size of 7x7, a step of 2 and zero padding of 3, thereby obtaining a face fusion enhanced feature map F 16 with a size of 192x112x112.

[0235] F 16 is input into four dilated residual blocks in sequence, thereby obtaining a fused structure recovery feature map F 17 with a size of 192x112x112.

[0236] The feature map F 17The input is input to a convolution layer with a kernel size of 3*3, a step of 1, and zero padding of 1, a ReLU layer and an up-sampling layer, and a convolution operation, an activation operation and an up-sampling operation with an up-sampling factor of 2 are performed, to obtain a structure recovery feature map F with a size of 3*224*224 struct ;

[0237] S323: Construct a texture restoration channel submodule TRC (Texture Restoration Channel), input M _occ and a randomly generated noise vector z into the texture restoration channel to obtain a texture restoration feature map F with a dimension of C*H*W texture ; the present application designs a texture restoration channel submodule TRC, and the structure is as shown in Figure 11 ;

[0238] The specific steps of S323 are as follows:

[0239] S3231: Randomly generate a noise vector z, and input z into a multilayer perceptron to obtain a noise feature vector f1;

[0240] The multilayer perceptron (MLP for short) is a classical feedforward artificial neural network model, and is widely used in classification, regression, feature extraction and other tasks;

[0241] S3232: input M _occ to a global average pooling layer to perform average pooling operation to obtain a face pooling scalar g; then input g to a multilayer perceptron to obtain a face feature vector f2;

[0242] S3233: perform Concat splicing operation on f1 and f2 to obtain a noise face splicing feature vector f3; input f3 to a multilayer perceptron MLP to obtain a style feature vector w;

[0243] S3234: perform AdaIN modulation operation on w and M _occ to obtain F texture ;

[0244] The AdaIN modulation (Adaptive Instance Normalization) is a feature normalization operation for style transfer and generative adversarial network;

[0245] Embodiment: assuming that the input to the texture restoration channel submodule TRC is M _occ with a size of C*H*W, wherein C is the number of channels, H is the height, and W is the width;

[0246] A 512-dimensional noise vector z is randomly generated; z is input into a multi-layer perception to obtain a 256-dimensional noise feature vector f1;

[0247] M _occ is input into a global average pooling layer to perform an average pooling operation to obtain a face pooling scalar g; then g is input into a multi-layer perception to obtain a 256-dimensional face feature vector f2;

[0248] f1 and f2 are subjected to a Concat splicing operation to obtain a 512-dimensional noise face splicing feature vector f3; f3 is input into a multi-layer perception MLP to obtain a 512-dimensional style feature vector w;

[0249] w and M _occ are subjected to an AdaIN modulation operation to obtain F texture with a size of 3x224x224;

[0250] S324: A dynamic region fusion sub-module DRFM is constructed, F struct and F texture are input into the dynamic region fusion sub-module DRFM to obtain F face ; a dynamic region fusion sub-module DRFM is designed in the application; the DRFM is used for repairing the face occlusion region to obtain F face ; wherein the structure of the DRFM is as shown in Figure 12

[0251] F struct is input into a global average pooling layer and a 1x1 convolution layer to perform global average pooling and convolution operations to obtain a structure weight vector alpha; F texture is input into a 3x3 convolution layer to perform convolution operations to obtain a texture weight feature map beta; and F struct , F texture , alpha and beta are input into a weighted fusion to obtain F face ;

[0252] The specific steps of S324 are as follows:

[0253] S3241: F struct is input into a global average pooling layer and a convolution layer with a convolution kernel size of 1x1, a step of 1, and zero padding of 0 to perform global average pooling operations and convolution operations to obtain a structure weight vector alpha with a dimension of Cx1x1;

[0254] alpha = Conv 1×1 (GAP(F struct ))

[0255] S3242: F texture ​Input into a convolution layer with a convolution kernel size of 3*3, a step of 1 and zero padding of 1, and perform convolution operation to obtain a texture weight feature map beta with a dimension of 1*H*W;

[0256] Beta = Conv 3×3 (F texture )

[0257] S3243: F struct , F texture , alpha, beta are weighted and fused to obtain F face with a dimension of C*H*W;

[0258] F face = alpha * F struct + (1-alpha) * (beta o F texture )

[0259] Wherein o represents multiplication operation; * represents dot multiplication; + represents vector addition;

[0260] S33: a multimodal detection network MDN is constructed, x, M _occ and F face are input into the MDN to obtain a result matrix result, and the result matrix result includes a final identity label and a corresponding confidence;

[0261] The overall structure of the multimodal detection network MDN constructed by the application is shown in Figure 13 ; and the operation process is as follows:

[0262] x and F face are input into an identity identification submodule to obtain an identity similarity score vector S id ; F face is input into a repair identification submodule to obtain a real score vector D real ; S id and D real are input into decision fusion to obtain a result matrix result, and the result matrix result includes a final identity label and a corresponding confidence;

[0263] The specific steps of the S33 are as follows:

[0264] S331: an identity identification submodule is constructed, x and F face are input into the identity identification submodule, and S id is output; an identity identification submodule is designed in the application, and the structure is shown in Figure 14 ;

[0265] The specific steps of the S331 are as follows:

[0266] S3311: Perform feature extraction operation on F face based on ArcFace algorithm to obtain the repaired face feature vector f gen ;

[0267] wherein ArcFace is a deep learning-based face recognition algorithm mainly used for face feature extraction and matching; it extracts the feature vector of the face through an optimized deep convolutional neural network (CNN), and then uses Euclidean distance or cosine similarity for matching;

[0268] S3312: Perform feature extraction operation on x based on ArcFace algorithm to obtain the occluded face feature vector f real ;

[0269] S3313: Perform cosine similarity calculation on f gen and f real to obtain the identity similarity score vector S id ;

[0270] S id =1-cos({f gen |f real})

[0271] wherein cos is a cosine similarity calculation function;

[0272] S332: Construct a repair discrimination sub-module, input F face and M _occ into the repair discrimination sub-module, and output the authenticity score vector D real ; the present application designs a repair discrimination sub-module, the structure of which is shown in Figure 15 ;

[0273] The specific steps of S332 are as follows:

[0274] S3321: Input F face into the PatchGAN encoder to obtain the repaired face image discrimination feature map F PGAN with a dimension of ;

[0275] wherein the PatchGAN encoder is a publicly available neural network structure for image processing tasks, used to encode images to extract features;

[0276] S3322: Input M _occ into the spatial attention layer to obtain the spatial attention weighted feature map F space with a dimension of ;

[0277] Among them, Spatial Attention (SA) enhances the feature representation of key regions and suppresses irrelevant information by calculating the spatial dimension weights of the feature map;

[0278] S3323: F PGAN and F space Performing the Concat concatenation operation yields the dimension as follows: Multiscale feature map F mul Then F mul The input is fed into a convolutional layer with a kernel size of 1×1, a stride of 1, and zero padding of 1, and a convolution operation is performed to obtain the authenticity score D. real ;

[0279] F mul =Concat(F PGAN ,F space )

[0280] D real =conv 1×1 (F mul )

[0281] Where Concat(F) PGAN ,F space () represents performing a Concat concatenation operation;

[0282] S333: S id and D real Perform a decision fusion operation, and then perform a max operation (e.g.) Figure 13 As shown in the figure, the result matrix result is obtained, where result is a two-dimensional matrix containing the final identified identity label and its corresponding confidence score;

[0283] result = max(W id ·S id +W real ·D real )

[0284] Among them W id W represents the weight of the identity verification branch. real The weights represent the weights for generating the identification branches, and max represents the maximum value and its index.

[0285] S4: Train and update the parameters of each layer of the DAM-Net network built in S3 to obtain a trained DAM-Net network;

[0286] The specific steps of S4 are as follows:

[0287] S41: initialize the hyperparameters required for training DAM-Net, such as batch size during training, initial learning rate, weight decay coefficient, and training rounds;

[0288] S42: divide the power grid worker multispectral image pair dataset constructed in S2 into a training set and a test set according to a certain proportion, ensuring that the two sets do not overlap; then divide the training set into multiple batches, and input each batch of the training set into the network for training each time to calculate the loss value of the batch;

[0289] Embodiment: In the experimental scheme of the present aspect, the proportion of the training set and the test set is 7:3;

[0290] S43: after each round of traversing all batches of the training set, the performance of the model is evaluated periodically using the test set data; the test set is sent into the model according to the same batch size as in the training stage, the prediction error of each batch is calculated respectively, and the test set loss value of the current round is obtained by summarizing; the test set loss value can be used to dynamically track the performance change of the network on non-training data; by continuously observing the change trend of the test set loss value, it is determined whether the model shows signs of overfitting; when the test set loss value no longer decreases or shows a rising trend in continuous multiple training periods, it can be determined that the generalization ability of the model decreases, and then a pre-set strategy adjustment mechanism is triggered; for example, stopping the training process or adaptively reducing the learning rate to optimize the model convergence path; when the test set loss value tends to be stable with the training process, and reaches the expected performance standard, it is considered that the network training is completed, and the final network parameters will be used for subsequent face recognition of workers in the power grid operation field under the condition of face occlusion;

[0291] S5: apply the trained DAM-Net network to recognize the face occlusion personnel in the power grid operation scene, and output the identity label and confidence of the target personnel;

[0292] The specific steps of S5 are as follows:

[0293] S51: after the DAM-Net network is trained, it is applied to the face recognition task of personnel in the actual power grid operation scene under the condition of personnel occlusion;

[0294] S52: input the collected power grid worker identity image into the DAM-Net network that has been trained, and the model identifies each power grid worker in the image one by one, and outputs the corresponding identity label and confidence of each worker in the image.

[0295] Although the present application has been described with reference to the embodiments above, various changes and modifications can be suggested to one skilled in the art, and it is intended that the present application encompass such changes and modifications as fall within the scope of the appended claims. Particularly, each feature disclosed in the description and / or the claims can be used in the combination with each of the features disclosed in the description and / or the claims, unless specifically stated otherwise. Therefore, the present application is not intended to be limited to the particular embodiments disclosed in the description and / or the claims.

Claims

1. A power grid operation site face recognition method based on deep learning, characterized in that, The specific steps of S1 are as follows: S1: Collect the face image pair of the power grid worker and pre-process to obtain the multispectral image pair dataset of the power grid worker ; S2: processing any second pair of face images processing to obtain a power grid operator multispectral image pair dataset ; The specific steps of S3 are as follows: S11: using a multispectral imaging device, collecting a face image of a worker at various power grid operation sites, for any one worker, obtaining a first face image pair of the worker wherein is a visible light image of a face of the worker, is an infrared image of the face of the worker; S12: performing spatial alignment and image normalization processing on the first face image and the second face image to obtain a second face image pair and performing spatial alignment and image normalization processing on the first face image and the second face image to obtain a second face image pair wherein is a normalized visible light image of the face, is a normalized infrared image of the face; S13: pair all second face images constitute a power grid worker multispectral image pair dataset ; S3: Constructing a face recognition model dynamic adversarial multi-modal network DAM-Net for workers in the power grid operation site under the condition of face occlusion, for identity recognition of workers in the power grid operation site under the condition of face occlusion; The specific steps of S3 are as follows: S31: design a dynamic occlusion detection module DODM, input into DODM, get face occlusion prediction heat map Wherein Refers to the power grid operator multispectral image pair data set Any one face image pair in , Or , its , , Visible light simulation occlusion face image, y refers to the labeled infrared face image;​ S32: constructing an adversarial feature generator AFG, inputting the face image and the face mask into the AFG, and generating a face repairing feature map and into the AFG, generating a face repairing feature map ; S33: Construct a multi-modal discriminant network MDN, input , and into MDN, get result matrix , including final identity label and its corresponding confidence. S4: Training and updating the parameters of each layer of the DAM-Net network constructed in S3 to obtain a trained DAM-Net network; S5: Applying the trained DAM-Net network to identify workers in the power grid operation site under the condition of face occlusion, and outputting the identity label and confidence of the target worker.

2. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S12 are as follows: S121: By the affine transformation matrix, the visible light image of the face after spatial alignment is obtained to spatial alignment, to obtain the visible light image of the face after spatial alignment , so that and spatially match, The calculation formula is as follows: wherein represents performing an affine transformation; M is an affine transformation matrix, which is a 2x3 matrix, in the form of where a, b, c, d control image rotation, scaling, shearing; c, f control translation; S122: performing channel normalization operation on the visible light image of the face to obtain a normalized visible light image of the face , thereby improving the accuracy of subsequent processing and analysis.​​​ 3. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S2 are as follows: S21: to perform geometric occlusion generation to obtain a visible light simulation occluded face image ; to perform semantic occlusion generation by 3D projection technology and GAN texture generation network to obtain a visible light simulation occluded face image ; to perform adversarial occlusion generation based on a fast gradient sign method to obtain a visible light simulation occluded face image ; S22: frame and label the face region of the visible light simulation blocked face image to obtain the labeled visible light simulation blocked face image S22: frame and label the face region of the visible light simulation blocked face image to obtain the labeled visible light simulation blocked face image ; Visible light simulation of occluded face images The face region is selected and labeled to obtain a labeled visible light simulated occlusion face image. ; Frame and label the face region of the visible light simulation occluded face image to obtain the labeled visible light simulation occluded face image Frame and label the face region of the visible light simulation occluded face image to obtain the labeled visible light simulation occluded face image Frame and label the face region of the visible light simulation occluded face image to obtain the labeled visible light simulation occluded face image Frame and label the face region of the visible light simulation occluded face image to obtain the labeled visible light simulation occluded face image ; S23: By and Constructing the facial image pair of the third sub-operator ;Depend on and Constructing the fourth sub-operator's facial image pair ;Depend on and Constructing the face image pair of the fifth sub-operator Collect all , , A dataset of multispectral images of power grid workers was obtained. .

4. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S31 are as follows: S311: input the first 3 residual block groups of the ResNet18 encoder to obtain the simulated occluded face feature map ; input the first 3 residual block groups of the ResNet18 encoder to obtain the simulated occluded face feature map ; input the first 3 residual block groups of the ResNet18 encoder to obtain the simulated occluded face feature map ; input the first 3 residual block groups of the ResNet18 encoder to obtain the simulated occluded face feature map ; input the first 3 residual block groups of the ResNet18 encoder to obtain the simulated occluded face feature map S312: design a deformable convolution fusion sub-module DCFS, input the feature maps of the first and second branches to the DCFS, and obtain a fused feature map with a dimension of and wherein is a channel number of the feature map, is a height of the feature map, is a width of the feature map.​​ S313: design a spatio-temporal attention mechanism submodule SAMS, which is input into SAMS to obtain a face occlusion prediction heat map with dimensions of ;​​ The specific steps of S312 are as follows: S3121: obtaining a feature map of the first image and a feature map of the second image by using the first image and the second image as input respectively and obtaining a spliced feature map with a dimension of by dual-mode splicing ; S3122: Will The input is fed into the offset prediction layer, where it undergoes a convolution operation with a kernel size of 3×3, a stride of 1, and zero padding of 1. Then, it is fed into a ReLU layer and a convolutional layer with a kernel size of 3×3, a stride of 1, and zero padding of 1, where non-linear activation and convolution operations are performed, resulting in a dimension of... Offset feature map ; S3123: input the , and to the deformable convolution layer and the up-sampling layer, perform the deformable convolution operation and the up-sampling operation, and obtain a fusion feature map with a dimension of ;​ wherein, denotes a deformable convolutional layer operation; represents an up-sampling operation; The specific steps of S313 are as follows: S3131: design a channel attention network, which is input into the channel attention network to obtain a channel-by-channel multiplication feature map ;​ S3132: design a spatial attention network, which is input into the spatial attention network to obtain ; and ; The specific steps of S3131 are as follows: S31311: input to the global average pooling layer, average pooling operation is performed to obtain the dimension of Input to the global average pooling layer, average pooling operation is performed to obtain the dimension of Average pooling feature map ; then input to the shared multi-layer perception to obtain the average weighted feature map with the dimension of ;​​ S31312: input to the global max-pooling layer, perform max-pooling operation to obtain a max-pooling feature map with a dimension of ; ; ; then input to the shared multi-layer perception to obtain a max-weighted feature map ; S31313: Will and Performing element-wise addition yields a dimension of Pooling splicing feature map Then The input is fed into the Sigmoid function, and an activation operation is performed to obtain the dimension. Pooling splicing activation feature maps ; wherein represents an add operation, i.e. adding and element-wise, Sigmoid represents performing a non-linear activation operation; S31314: a channel-wise multiplication operation is performed on and to obtain a channel-wise multiplication feature map with a dimension of ;​ wherein represents a per-channel multiplication operation; The specific steps of S3132 are as follows: S31321: input to the channel average pooling layer, perform channel average pooling operation, obtain the average pooling feature map with the dimension of ;​​ S31322: input to the channel max pooling layer, perform channel max pooling operation, obtain a channel-pooled feature map with dimensions of ;​​ S31323: Will and Performing the Concat concatenation operation yields the dimension as follows: Pooling splicing feature map Then The input is fed into a convolutional layer with a kernel size of 7×7, a stride of 1, and zero padding of 1, and a convolution operation is performed to obtain a dimension of Spatial attention feature map ; then The input is fed into the Sigmoid function, and an activation operation is performed to obtain the dimension. of ; wherein represents a concatenation operation. and performing a Concat concatenation operation.

5. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S32 are as follows: S321: Concatenate the and to obtain a face fusion feature map with dimensions of ;​ wherein representing a pair of and performing a Concat concatenation operation; S322: construct a structure recovery channel sub-module SRC, input into SRC, obtain a structure recovery feature map ;​ S323: Construct a texture restoration channel submodule TRC, and randomly generated noise vector Inputting it into the texture restoration channel yields a dimension of Texture recovery feature map ; S324: construct a dynamic region fusion sub-module DRFM, input the first image and the second image into the dynamic region fusion sub-module DRFM, and obtain a first image and a second image and input into the dynamic region fusion sub-module DRFM, to obtain a first image and a second image ; The specific steps of S322 are as follows: S3221: the input is to a ReLU layer and a convolutional layer with a kernel size of 7x7, a stride of 2, and zero padding of 3, to obtain a facial fusion enhanced feature map with a dimension of ;​ S3222: inputting the first fusion structure recovery feature map and the second fusion structure recovery feature map into the first empty residual block in sequence to obtain a first fusion structure recovery feature map inputting the first fusion structure recovery feature map and the second fusion structure recovery feature map into the second empty residual block in sequence to obtain a second fusion structure recovery feature map , wherein the first empty residual block, the second empty residual block, the third empty residual block and the fourth empty residual block are the same in structure S3223: input to a convolution layer with a kernel size of 3x3, a stride of 1, and zero padding of 1, a ReLU layer, and an up-sampling layer, and perform convolution operation, activation operation, and up-sampling operation with an up-sampling factor of 2 to obtain a feature map with a dimension of ;​​ wherein represents performing an up-sampling operation; The specific steps of S323 are as follows: S3231: randomly generate a noise vector , which is input into the multi-layer perception to obtain a noise feature vector ;​ S3232: Will The input is fed into a global average pooling layer for average pooling to obtain a face pooling scalar. Then The input is fed into a multilayer perceptron to obtain a facial feature vector. ; S3233: Concatenate and to obtain the noise face concatenation feature vector ; input to the multi-layer perception (MLP) to obtain the style feature vector ; S3234: apply AdaIN modulation operation to obtain and ; and ; The specific steps of S324 are as follows: S3241: Will The input is fed into a global average pooling layer and a convolutional layer with a kernel size of 1×1, a stride of 1, and zero padding of 0. Global average pooling and convolution operations are then performed to obtain a dimension of... Structural weight vector ; S3242: input to the convolution kernel size is 3x3, step is 1, zero padding is 1, convolution layer, convolution operation, get the dimension of Texture weight feature map ;​ S3243: perform a weighted fusion operation to obtain a feature vector with a dimension of , , , ;​​ wherein represents a multiplication operation; represents a dot product; represents a vector addition.

6. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S33 are as follows: S331: constructing an identity authentication submodule, inputting the identity similarity score vector and to the identity authentication submodule, and outputting an identity similarity score vector ; S332: Construct a repair identification submodule, input the feature vector and to the repair identification submodule, and output a authenticity score vector ; S333: perform a decision fusion operation on the result matrices and perform a max operation to obtain a result matrix wherein is a two-dimensional matrix containing the final recognition identity label and its corresponding confidence level; wherein represent the weights of the identity authentication branch; represent the weights of the generation authentication branch, and max represents the maximum value and its index; The specific steps of S331 are as follows: S3311: obtaining the feature vector of the repaired face by performing a feature extraction operation based on an ArcFace algorithm performing a feature extraction operation based on an ArcFace algorithm to obtain a feature vector of the repaired face ; S3312: obtaining the feature vector of the occluded face by performing the feature extraction operation based on the ArcFace algorithm performing a feature extraction operation based on an ArcFace algorithm to obtain an occluded face feature vector ; S3313: cosine similarity calculation is performed on the identity feature vectors of the two images to obtain an identity similarity score vector and ;​ which is a cosine similarity computation function; The specific steps of S332 are as follows: S3321: inputting the face image and the face image patch into the PatchGAN encoder to obtain a repaired face image discrimination feature map with a dimension of ;​​ S3322: input the spatial attention layer to obtain a spatial attention weighted feature map with a dimension of ; and S3323: input the spatial attention weighted feature map into the channel attention layer to obtain a channel attention weighted feature map with a dimension of ; and S3324: input the channel attention weighted feature map into the fusion layer to obtain a final feature S3323: Will and Performing the Concat concatenation operation yields the dimension as follows: Multiscale feature maps Then The input is fed into a convolutional layer with a kernel size of 1×1, a stride of 1, and zero padding of 1, and a convolution operation is performed to obtain a realism score. ; wherein represents a Concat concatenation operation.

7. The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S4 are as follows: S41: Initializing the hyperparameters required for training DAM-Net; S42: Dividing the multispectral image pair data set of workers in the power grid operation site constructed in S2 into a training set and a test set according to a certain proportion, ensuring that the two sets do not overlap; then dividing the training set into multiple batches, and inputting each batch of the training set into the network for training each time to calculate the loss value of the batch; S43: After each round of traversing all batches of the training set, the performance of the model is evaluated periodically using the test set data; the test set is sequentially input into the model according to the same batch size as the training stage, the prediction error of each batch is calculated respectively, and the test set loss value of the current round is obtained by summarizing; the test set loss value can be used to dynamically track the performance change of the network on non-training data; By continuously observing the change trend of the test set loss value, it is determined whether the model shows signs of overfitting; when the test set loss value no longer decreases or shows a rising trend in continuous multiple training periods, it is determined that the model generalization ability decreases, and then a pre-set strategy adjustment mechanism is triggered; when the test set loss value tends to be stable with the training process, and reaches the expected performance standard, it is considered that the network training is completed, and the final network parameters will be used for subsequent identity recognition of workers in the power grid operation site under the condition of face occlusion. 8.The power grid operation site face recognition method based on deep learning according to claim 1, characterized in that, The specific steps of S5 are as follows: S51: After the DAM-Net network is trained, it is applied to the face recognition task of workers in the actual power grid operation site under the condition of face occlusion; S52: Input the collected identity images of power grid workers into the trained DAM-Net network, and the model identifies each power grid worker in the image one by one, and outputs the corresponding identity label and confidence of each worker in the image.

Images