Face image privacy protection recognition method based on optical coding and reverse feature enhancement

Abstract

The application provides a face image privacy protection recognition method based on optical coding and reverse feature enhancement, and relates to the technical field of computer vision recognition, and the method comprises the following steps: S1, acquiring a face data set, performing data enhancement, and constructing a face enhanced data set; S2, constructing a binary neural network convolution coding layer based on a straight-through estimator quantization, and performing optical coding on the face enhanced data set; S3, constructing a CNN and Transformer dual-channel multiplexing reverse feature enhancement neural network model, and performing feature dimension enhancement; S4, using a hybrid constraint, and jointly optimizing the neural network coding layer and the dual-channel multiplexing reverse feature enhancement neural network model; and S5, constructing a face embedding database, comparing a high-dimensional coding value of a single-pixel detection value with features in the face embedding database, and realizing face recognition. The application jointly optimizes the neural network binary convolution coding layer for spatial light modulator optical coding and the dual-channel multiplexing reverse feature enhancement model, so that the recognition precision is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision recognition technology, specifically to a face image privacy protection recognition method based on optical coding and reverse feature enhancement. Background Technology

[0002] Target recognition technology plays a crucial role in fields such as security monitoring, autonomous driving, industrial inspection, and military reconnaissance. Traditional target recognition methods primarily rely on acquiring high-resolution images, processing the rich pixel information within the image to achieve target identification and detection. These methods typically depend on complex imaging equipment, such as high-resolution CMOS or CCD sensors. However, array-based image recognition has several limitations. On one hand, high-performance imaging hardware is usually bulky and energy-intensive, making it difficult to adapt to scenarios with strict size and power consumption requirements, such as drone reconnaissance and wearable security devices. On the other hand, the high transmission load of image data also challenges communication bandwidth, limiting its application in areas with limited communication. Furthermore, in the field of facial recognition, this image-first, perception-later detection mode requires acquiring and storing relatively complete images, which easily exposes sensitive information such as personal privacy, making it difficult to meet the application requirements with high privacy protection standards. This limits its feasibility and widespread application in privacy-sensitive environments, thus requiring further strengthening of privacy and security protection to prevent the leakage of facial information.

[0003] The core commonality of privacy coding lies in the fact that regardless of the technology used, the following principles must be met: Irreversibility: Stored / transmitted data cannot reconstruct the original face; Minimization: Only features necessary for identification are retained; Dynamic protection: Support for template updates or reversals; Compliance: Adaptation to regulations such as GDPR and BIPA. These methods can be used individually or in combination, and can be flexibly selected according to the needs of the scenario (real-time performance, cost, security level).

[0004] In recent years, the emergence of image-free sensing technology based on optically assisted physical compression coding using spatial light modulators and single-pixel detectors has provided a new approach to solving the aforementioned problems. This technology uses a single-pixel detector in conjunction with light field modulation and coding mechanisms to map high-dimensional scene information into a one-dimensional signal, and then uses algorithms to achieve image-free target recognition directly from one-dimensional measurements to target identification information. This method eliminates the need for complete image reconstruction, allowing direct extraction of target features from limited measurement data for target recognition. This approach significantly reduces dependence on imaging hardware, data transmission, and computing resources, and since the process from one-dimensional detection signals to target identification information does not store any image information, it demonstrates potential for widespread application in resource-constrained and privacy-sensitive environments.

[0005] However, current single-pixel imageless sensing technology acquires one-dimensional measurements of targets using a single-pixel detector. Compared to the two-dimensional image data acquired by traditional image sensors, the information dimensionality is significantly reduced, allowing for good performance only in simple scenes or datasets. When faced with complex datasets, the limited information is insufficient to fully describe the target's features, leading to a decrease in recognition accuracy. Transforming one-dimensional single-pixel measurement data into a feature space suitable for target recognition is a challenging process. In complex datasets, the feature distributions of different targets may be even more complex. How to accurately map low-dimensional data to a high-dimensional feature space to achieve effective target differentiation is one of the challenges facing single-pixel imageless sensing technology. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to provide a privacy-preserving face image recognition method based on optical coding and inverse feature enhancement. The method employs a pass-through estimator, a binary neural network convolutional coding layer, and an equivalent spatial light modulator to perform optical coding on the target object. The proposed dual-stream global-local feature fusion of a CNN and a Transformer effectively enhances the features that differentiate between different categories in complex datasets. By leveraging the advantages of both in local and global feature extraction, the method improves the understanding and representation of complex data, resulting in output features with good discriminative power and significantly enhanced recognition accuracy.

[0007] Specifically, the present invention provides a face image privacy protection recognition method based on optical coding and inverse feature enhancement, which includes the following steps:

[0008] S1: Obtain the face dataset, perform data augmentation on it, construct the face augmentation dataset, and divide it into training set O. train Validation set O val With test set O test ;

[0009] S2: Construct a binary neural network convolutional coding layer based on the pass-through estimator quantization to perform convolutional coding on the face enhancement dataset in step S1; set up the neural network convolutional coding layer, and use binarization operations and the pass-through estimator to update the parameters of the convolutional coding layer; map the binary neural network convolutional coding layer to the spatial light modulator, and obtain the encoded value from the light intensity value collected by the single-pixel detector; perform positive and negative modulation optical coding on the negative terms in the convolutional coding layer, and perform difference operations on the obtained single-pixel detector values ​​to simulate the original convolutional coding layer encoding the face, and obtain the low-dimensional encoded actual single-pixel detection value S;

[0010] S3: Construct a dual-path multiplexing reverse feature enhancement neural network model based on convolutional neural network (CNN) and deep learning model based on self-attention mechanism (Transformer). Perform feature dimensionality enhancement on the low-dimensional encoded actual single-pixel detection value S obtained in step S2 to obtain the high-dimensional encoded value H of the single-pixel detection value.

[0011] S4: Using a hybrid constraint of triplet loss, center loss and cross-entropy loss, construct a hybrid constraint loss function for jointly optimizing the neural network model parameter set θ in steps S2 and S3. Based on the face enhancement dataset, jointly optimize the neural network coding layer and the dual-path multiplexed inverse feature enhancement neural network model.

[0012] S5: Optimize the reverse feature enhancement neural network model in step S3 using step S4, construct a face embedding database, and compare the high-dimensional encoded value H of the single-pixel detection value output by the reverse feature enhancement neural network model in step S3 with the Euclidean distance of the features in the face embedding database to achieve face recognition.

[0013] Preferably, step S2 specifically includes:

[0014] S21: Set the optical coding matrix, construct the neural network convolutional coding layer for optically assisted target scene coding; use binarization operations and a pass-through estimator to set up the forward and backward propagation processes of the neural network convolutional coding layer;

[0015] S22: Train the face enhancement data set O train As input to the binary convolutional encoding matrix Perform convolution to obtain the single-pixel detection value S1 of the optically assisted face image encoding information;

[0016] S23: When testing the trained neural network model, the binary neural network convolutional coding layer is mapped to the spatial light modulator. The single-pixel detector values ​​obtained by positive and negative modulation optical coding are differentially processed to simulate the original convolutional coding layer encoding the face, resulting in the low-dimensional encoded actual single-pixel detector value S.

[0017] Preferably, step S3 specifically includes:

[0018] S31: Perform layer normalization processing, amplify features through pixel recombination upsampling layer, enhance model features through depth-separable convolutional layer, and assign weights by combining spatial attention mechanism and channel attention mechanism to obtain enhanced features;

[0019] S32: Construct a CNN-based local feature branch and a Transformer-based global feature branch. The CNN-based local feature branch consists of an adaptive convolutional layer, a two-dimensional average pooling layer, and a fully connected layer; the Transformer-based global feature branch consists of a visual SwinTransformer module.

[0020] S33: The enhanced features obtained in step S31 are processed by the local and global features obtained in step S32, and the two are concatenated to obtain a high-dimensional encoding value, thereby realizing the reverse enhancement from low-dimensional features to high-dimensional features.

[0021] Preferably, the hybrid constraint loss function in step S4 is as follows:

[0022]

[0023] Where J(θ) is the hybrid constraint loss function; ω is the weight parameter of the triplet loss; λ is the weight parameter of the center loss; and μ is the weight parameter of the cross-entropy loss. θ is the gradient of the hybrid constraint loss function with respect to the parameters; l is the learning rate of the neural network model; Loss1 is the triple constraint loss equation; Loss2 is the center constraint loss equation; Loss3 is the cross-entropy constraint loss equation; θ is the set of parameters of the neural network model.

[0024] Preferably, step S5 specifically includes:

[0025] S51: Train the face enhancement data set O train The feature space obtained from the joint optimization steps S2 and S3 is used as a face embedding database for face recognition.

[0026] S52: Expand the face enhancement data to the verification set O val With face enhancement data test set O test The input is fed into the optimized reverse feature enhancement neural network model in step S4 to obtain high-dimensional verification output features and high-dimensional test output features;

[0027] S53: Calculate the Euclidean distance between the output features of step S52 and all features stored in the face embedding database in step S51, and determine the feature category with the smallest Euclidean distance as the category of the verification face or test face.

[0028] Preferably, in step S21, when calculating the gradient during backpropagation, a pass-through estimator is used, and the gradient is calculated and updated based on the face enhancement dataset to avoid the influence of binarization operations; the specific formula for updating the convolutional coding layer parameters is as follows:

[0029]

[0030] Where C' represents the parameters of the convolutional coding layer updated after backpropagation; l represents the backpropagation learning rate; J represents the hybrid constraint loss function; C encode These are the parameters for the convolutional coding layer.

[0031] Preferably, in step S23, the single-pixel detector values ​​obtained by positive and negative modulation optical coding are subjected to differential operation to simulate the encoding of the face by the original convolutional coding layer; the actual single-pixel detector values ​​obtained by low-dimensional coding are:

[0032] S = S + -S - =P + O test -P - O test ;

[0033] Where S is the actual single-pixel detection value encoded in low dimension; S + S represents the detected light intensity value corresponding to the positively modulated light field. - P represents the intensity detection value corresponding to the negatively modulated optical field. + It is a positive modulation optical code; P - For negative modulation optical coding; O test This is a test set of face augmentation data.

[0034] Preferably, the input to step S31 is the actual single-pixel detection value S encoded in low dimension, and the specific process is as follows:

[0035]

[0036] h1 = PixelShuffle(h0)

[0037] b1 = GELU(BN(Conv) 1×1 (h1)))

[0038] b2=GELU(BN(Conv 1×1 (GELU(BN(DepthwiseConv 3×3 (h1))))));

[0039] b3 = GELU(BN(Conv) 1×1 (GELU(BN(DepthwiseConv 5×5 (h1))))))

[0040] b4 = GELU(BN(Conv) 1×1 (MaxPool(h1)))

[0041]

[0042] Wherein, Reshape is the size modification function; PixelShuffle is the pixel recombining upsampling layer; b1 is the first branch in the Inception layer; b2 is the second branch in the Inception layer; b3 is the third branch in the Inception layer; b4 is the fourth branch in the Inception layer; CBAM is the spatial attention mechanism and channel attention mechanism. The space is a concatenation of two vectors in dimension 1; GELU is a non-linear activation function layer; batchsize is the number of input samples in each forward and backward propagation; h0 is the output of the size modification function; h1 is the output of the pixel reassembly upsampling layer; h2 is the output of the spatial attention mechanism and channel attention mechanism; BN is a deep learning optimization layer; Conv is a convolutional layer; Depthwise is a depthwise convolutional layer; MaxPool is a max pooling layer; M is the number of compressed samples.

[0043] Preferably, the specific details of the CNN-based local feature branch in step S32 are as follows:

[0044]

[0045] Where GlobalAvgPool is the global average pooling layer; LayerNorm is the normalization layer; W1 is the first fully connected layer; W2 is the second fully connected layer; h3 is the output of the non-linear activation function layer; h4 is the output of the average pooling layer; H CNN This is the output of local features based on CNN branches.

[0046] Preferably, the global feature branch based on Transformer in step S32 is specifically as follows:

[0047]

[0048] Where h5 is the output of the block and linear projection image transformation function; h6 is the output of the multi-head self-attention module; PatchEmbed is the block and linear projection image transformation function; MSA is the multi-head self-attention module; LN is the layer normalization module; MLP is the multilayer perceptron layer; h7 is the multilayer perceptron layer output; H Swim-T The output is the global feature based on the Transformer branch; AvgPool is the average pooling layer.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] (1) The present invention proposes a direct-pass estimator binary neural network convolutional coding layer equivalent spatial light modulator to perform optical coding on the target object and fit the binarization requirements of the physical device; at the same time, it does not affect backpropagation during network joint training, and eliminates the prediction error caused by the mismatch between simulated optical coding and actual optical coding.

[0051] (2) The dual-stream global-local feature fusion CNN and Transformer proposed in this invention can effectively enhance the features that differentiate between different categories in complex datasets. By using dual-stream multiplexing of CNN and Transformer, the model can simultaneously utilize the advantages of both in local and global feature extraction, evaluate local details and global structure, thereby improving the understanding and expression of complex data. Constrained by the hybrid feature differential loss, the output features have good discriminativeness, providing more details and more representative features for subsequent feature comparison. Attached Figure Description

[0052] Figure 1 This is a flowchart of the face image privacy protection recognition method based on optical coding and reverse feature enhancement of the present invention;

[0053] Figure 2 The images are 40 classes of face data from the ORL face dataset used in this invention.

[0054] Figure 3 This is a partial image of the enhanced face dataset from the present invention.

[0055] Figure 4 This is a schematic diagram of the forward and backward propagation of the binary optical coding layer of the present invention;

[0056] Figure 5 This is a schematic diagram of the reverse feature enhancement neural network model based on dual-path multiplexing of CNN and Transformer constructed in this invention;

[0057] Figure 6 This is a diagram showing the combined optimized hybrid constraint loss curve of the present invention;

[0058] Figure 7 This is a schematic diagram of the average distance for feature comparison in this invention;

[0059] Figure 8 This is a confusion matrix diagram of the recognition results of the test set at a sampling rate of 0.012 according to the present invention;

[0060] Figure 9 This is a line graph showing the recognition results of the present invention at multiple sampling rates. Detailed Implementation

[0061] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0062] This invention proposes a face image privacy protection recognition method based on optical coding and inverse feature enhancement, such as... Figure 1 As shown, the process involves acquiring a face dataset, performing data augmentation, and constructing a face augmentation dataset. A binary neural network convolutional coding layer based on direct-pass estimator quantization is constructed to optically encode the face augmentation dataset. A neural network model based on dual-path multiplexing of CNN and Transformer is constructed to augment features. Hybrid constraints are used to jointly optimize the neural network coding layer and the dual-path multiplexing inverse feature augmentation neural network model. A face embedding database is constructed, and the high-dimensional encoded values ​​of single-pixel detection values ​​are compared with the features in the face embedding database to achieve face recognition. The specific steps include:

[0063] Step S1: Obtain the face dataset, perform data augmentation on it, construct the face augmentation dataset, and divide it into training set, validation set and test set.

[0064] This invention obtains a face dataset from a publicly available dataset, containing 40 directories corresponding to 40 categories. Each directory contains 10 basic face data images, totaling 400 basic face data images. These images were collected under different times, lighting conditions, facial expressions (open / closed eyes, smiling / not smiling), and facial detail (wearing glasses / not wearing glasses). Figure 2 The image shown represents 40 categories of face images from the basic face dataset used in this invention. The initial resolution of the basic face data images is 92 pixels * 112 pixels. Data augmentation is performed based on this initial resolution, specifically including: left-right translation and up-down translation, with the number of translation pixels limited to ±20 pixels; rotation, with the rotation angle limited to ±25 degrees; scaling, with the scaling ratio limited to between 0.8 and 1.2; flipping, with the flip direction limited to left-right; and brightness adjustment, with the brightness adjustment factor limited to between 0.5 and 1.3 to achieve accurate recognition in dark environments. One or more of these five data augmentation operations are randomly selected to enhance the basic face data, resulting in the face augmentation dataset. Figure 3 The image shown is a composite image of the enhanced face dataset of this invention, containing partial face images processed by different data augmentation operations. Pixels that extend beyond the original image boundaries during the enhancement process are filled with completely black pixels. In this embodiment, a total of 16,000 face enhancement data images were obtained, which were divided into a training set of 12,800 images in an 8:1:1 ratio. train 1600 face enhancement data verification sets val With a test set of 1600 face enhancement data images O test .

[0065] Step S2: Construct a binary neural network convolutional coding layer based on pass-through estimator quantization to perform convolutional coding on the face enhancement dataset in Step S1; in actual use, load the convolutional coding layer into the spatial light modulator to perform optical coding on the actual face.

[0066] Step S21: Set the optical coding matrix resolution to N×N, set the compressed sampling number to M, and the compressed sampling rate to SR = M / N. 2 A neural network convolutional coding layer with a size of 1×M×N×N is constructed for optically assisted target scene encoding. In this embodiment, the optical coding matrix resolution is 64 pixels × 64 pixels, the number of compressed samples M is set to 49, and the compression sampling rate is calculated to be 0.012. The neural network convolutional coding layer is initialized, and the pass-through estimator of the neural network convolutional coding layer during forward and backward propagation is set; for the convolutional kernel parameter θ encode During the forward propagation process, a binarization operation is performed, specifically as follows:

[0067]

[0068] in, C is a binary convolutional coding matrix; sign is the signal function; C encode θ represents the parameters of the convolutional coding layer. encode These are the convolution kernel parameters.

[0069] To avoid the impact of binarization on gradient propagation during backpropagation, a pass-through estimator is used. This estimator calculates and updates the gradient based on the face augmentation dataset, thus avoiding the influence of binarization. The specific formula for updating the convolutional coding layer parameters is as follows:

[0070]

[0071] Where C' represents the parameters of the convolutional coding layer updated after backpropagation; l represents the backpropagation learning rate; and J represents the hybrid constraint loss function.

[0072] The above formula essentially calculates the gradient of the loss function with respect to each parameter, and then adjusts the parameters according to the gradient direction. Specifically, the binarized neural network convolutional coding layer quantized by the pass-through estimator is binarized to -1 or 1 during forward propagation, but its true parameter values ​​C are stored simultaneously. encode After the neural network completes its forward propagation, the loss function calculates the gradient layer by layer from the output layer until the gradient with respect to the true parameter values ​​of the current convolutional coding layer is calculated. Finally, the parameters are updated using gradient descent with a learning rate l, and the new convolutional coding layer parameters C' are obtained by subtracting the update amount from the original parameters.

[0073] Traditional regularized convolutional coding layer parameter settings force the neural network's convolutional coding layer parameters to approach -1 or 1. However, due to the floating-point nature of neural network parameters, the final parameters only approach -1 or 1, not exactly equal to them. When this convolutional coding layer is later equivalently loaded into the spatial light modulator, forcibly setting it to -1 or 1 introduces significant errors, causing discrepancies between optical coding and convolutional coding. By setting a pass-through estimator, the neural network's convolutional coding layer parameters are forced to be binarized during forward propagation. When this convolutional coding layer is equivalently loaded into the spatial light modulator during backward propagation, no errors occur, making the actual optical coding completely equivalent to convolutional coding. Figure 4 This diagram illustrates the binarized convolutional coding layer and forward and backward propagation of the present invention. During forward propagation, all convolutional kernel parameters are set to -1 or 1, while during backward propagation, the original convolutional kernel parameters from the original convolutional coding layer are used to update the convolutional kernels.

[0074] Step S22: Train the face enhancement data set O generated in step S1. train As input to the neural network model, it is combined with the binarized convolutional encoding matrix in step S21. Convolution yields M-dimensional encoded values, which are equivalent to single-pixel detection values ​​used to obtain optically assisted face image encoding information through a single-pixel detector. Specifically:

[0075]

[0076] Where S is the single-pixel detection value of the optically assisted face image coding information; O train Training set for face augmentation data; This refers to the convolution operation in a neural network.

[0077] In this embodiment of the invention, during simulation training, a training set of 12,800 face enhancement data images is used. train The data is loaded into a data loader with a batch size of 64 (BatchSize is set to 64) for each forward and backward propagation. This data, with dimensions 64×1×64×64, is used cyclically as input to the neural network model, and is combined with the binarized convolutional encoding matrix from step S21. Performing convolution operations yields 64×49 dimensional encoded values, and this process is repeated across the entire face enhancement data training set O. train An epoch is one iteration cycle. Multiple epochs are repeated until the neural network model converges and training is complete.

[0078] Step S23: When testing the neural network model trained in step S22, it needs to be tested in an equivalent actual physical scene. The convolutional coding layer of the binary neural network is mapped to the spatial light modulator, and the encoded value is obtained from the light intensity value collected by the single-pixel detector. Specifically, the parameters of the convolutional coding layer of the simulated binary neural network are loaded into the spatial light modulator, and the face enhancement data test set O is tested. test The facial image scene is subjected to spatial light modulation physical compression encoding, and finally the total light intensity of the scene is collected using a single-pixel detector. When loading into the spatial light modulator, the binarized convolutional encoding matrix needs to be... The light field is magnified proportionally into a micromirror and used as a spatial physics compressed encoding light field.

[0079] Since the negative terms in the convolutional coding layer learned by the neural network model cannot be directly converted on the spatial light modulator, positive and negative modulation optical coding is required. The single-pixel detector values ​​obtained from positive and negative modulation optical coding are then differentially processed to simulate the original convolutional coding layer encoding the face. Positive modulation optical coding retains the corresponding positive terms in the binarized convolutional coding layer parameters, while setting the negative terms to 0, resulting in a modulation light field. Negative modulation optical coding takes the absolute value of the corresponding negative terms in the binarized optical coding layer parameters, while setting the positive terms to 0, resulting in a modulation light field. The actual single-pixel detector values ​​obtained from the low-dimensional encoding are:

[0080] S = S + -S - =P + O test -P - O test ;

[0081] Where S is the actual single-pixel detection value of the low-dimensional encoding, which is equivalent to the low-dimensional encoded value after optical-assisted physical compression encoding; S + S represents the detected light intensity value corresponding to the positively modulated light field. - P represents the intensity detection value corresponding to the negatively modulated optical field. + This is positive modulation optical coding, where positive terms corresponding to optical coding parameters are retained and negative terms are set to 0; P - This is negative modulation optical coding, where the negative terms corresponding to the optical coding parameters are set to 1 and the positive terms are set to 0; O test As a test set for face enhancement data, it interacts with spatial physical light fields in real-world scenarios and performs optical physical compression encoding to obtain single-pixel detection values.

[0082] Specifically, in practical scenarios, when encoding a face using a spatial light modulator, a lens is needed to focus the face image onto the micromirror array of the spatial light modulator to achieve optical encoding. Since the spatial light modulator can only display black and white images, the corresponding parameters are either 0 or 1. The learned optical encoding parameters (binarized convolutional coding layer parameters) are -1 and 1. Therefore, a binary convolutional coding layer parameter is split into two optical codes, which are used to optically encode the face image to obtain two single-pixel detection values ​​S. + With S - The difference between the two is S, which is equivalent to directly encoding the face image using the binarized convolutional coding layer parameters (-1 and 1).

[0083] In this embodiment of the invention, the spatial light modulator has a pixel size of 1024 pixels × 768 pixels, so the magnification ratio is 12 times, and a 64-pixel × 64-pixel binarized convolutional coding matrix is ​​generated. The image is enlarged to 768 pixels × 768 pixels, and the remaining pixels in the horizontal direction are set to 0 to form a two-dimensional structured light field with the same pixel size as the spatial light modulator. The binarized optical coding layer parameters are saved as a 1-bit .bmp file and loaded into the spatial light modulator to complete the optical physical compression coding of the face enhancement data test set in space. The encoded light intensity value is collected using a PDA100A2 single-pixel detector, which is the actual single-pixel detection value S of the low-dimensional coding.

[0084] Step S3: Construct a dual-path multiplexed inverse feature enhancement neural network model based on a convolutional neural network (CNN) and a deep learning model based on a self-attention mechanism (Transformer) to perform feature augmentation on the low-dimensional encoded actual single-pixel detection value S obtained in step S2, such as... Figure 5 The diagram shows the inverse feature enhancement neural network model based on dual-path multiplexing of CNN and Transformer constructed in this invention, including model input, model structure, and model output. In step S2, the actual single-pixel detection value S is obtained as a low-dimensional encoded value, which is input into the dual-path multiplexing neural network model of CNN and Transformer to output the enhanced high-dimensional feature. The mapping model from the low-dimensional encoded value to the output high-dimensional encoded value is as follows:

[0085]

[0086] Where H is the high-dimensional encoded value corresponding to a single pixel detection value; This is a process mapping from low-dimensional encoded values ​​to high-dimensional encoded output values, i.e., a reverse feature enhancement neural network model.

[0087] The process of mapping from low-dimensional encoded values ​​to output high-dimensional encoded values Specifically, it includes the following sub-steps:

[0088] Step S31: The low-dimensional encoded actual single-pixel detection value S obtained in step S2 is used as the input to the dual-path multiplexing inverse feature enhancement neural network model. In this embodiment, the low-dimensional encoded actual single-pixel detection value S obtained in step S2 is first normalized to meet generalization requirements. Then, the size is modified to 7×7. The feature map of the encoded value is effectively enlarged by pixel shuffle layers with (1,8) and (8,16) channels. A deep learning optimization layer BN and a non-linear activation function layer GELU are attached in the middle of the pixel shuffle convolutional layer. Then, the Inception layer based on depthwise separable convolution is passed to improve the computational efficiency of the model and reduce the number of parameters, while further enhancing the features. Finally, the enhanced features are weighted by combining spatial attention mechanism and channel attention mechanism to maximize the extraction of effective features.

[0089] The Inception layer based on depthwise separable convolution in this embodiment includes four 1x1 convolutional branches with 16 output channels each, a depthwise separable 3x3 convolutional branch, a depthwise separable 5x5 convolutional branch, and a pooling branch. Each branch contains a convolutional layer, a batch normalization layer, and an activation function layer, which are then concatenated to form a total output of 64 channels. The data output from the Inception layer based on depthwise separable convolution is processed by two sub-modules: ChannelAttention and SpatialAttention, which are used to process the feature map with attention mechanisms in both the channel and spatial dimensions. The ChannelAttention sub-module uses adaptive average pooling and adaptive max pooling to obtain global information, performs feature transformation through a multilayer perceptron, and finally generates channel attention weights using a sigmoid function. The SpatialAttention submodule performs average pooling and max pooling operations on the input feature map along the channel dimension, concatenates the results, and then passes them through a convolutional layer and the Sigmoid activation function to generate spatial attention weights, ensuring that the number of output channels remains unchanged at 64. Specifically, it is expressed as follows:

[0090]

[0091] h1 = PixelShuffle(h0)

[0092] b1 = GELU(BN(Conv) 1×1 (h1)))

[0093] b2=GELU(BN(Conv 1×1(GELU(BN(DepthwiseConv 3×3 (h1))))));

[0094] b3 = GELU(BN(Conv) 1×1 (GELU(BN(DepthwiseConv 5×5 (h1))))))

[0095] b4 = GELU(BN(Conv) 1×1 (MaxPool(h1)))

[0096]

[0097] Wherein, Reshape is the size modification function; PixelShuffle is the pixel recombining upsampling layer; b1 is the first branch in the Inception layer; b2 is the second branch in the Inception layer; b3 is the third branch in the Inception layer; b4 is the fourth branch in the Inception layer; CBAM is the spatial attention mechanism and channel attention mechanism. The space is concatenated with two vectors in dimension 1; GELU is a non-linear activation function layer; batchsize is the number of input samples for each forward and backward propagation; h0 is the output of the size modification function; h1 is the output of the pixel reassembly upsampling layer; h2 is the output of the spatial attention mechanism and channel attention mechanism; BN is a deep learning optimization layer; Conv is a convolutional layer; Depthwise is a depthwise convolutional layer; MaxPool is a max pooling layer.

[0098] Specifically, S, as the input to the dual-path multiplexing inverse feature enhancement neural network model, is first resized to a size acceptable to the neural network. In this embodiment, the number of compressed samples M is 49, and the batch size is 64. That is, the reshaped size is [64, 1, 7, 7], resulting in h0. Then, it passes through an upsampling layer (Pixelshuffle) to obtain h1. h1 is then input into a depthwise separable convolution-improved Inception layer (with four branches: a 1x1 convolution branch with 16 channels each, a depthwise separable 3x3 convolution branch, a depthwise separable 5x5 convolution branch, and a pooling branch) to obtain b1, b2, b3, and b4. Finally, the outputs of these four branches are weighted and concatenated using CBAM to obtain h2, with a size of [64, 64, 28, 28]. This pixel recombining upsampling scheme employs a progressive strategy, gradually amplifying low-dimensional features and expanding the receptive field without loss. The synergistic design of the depthwise separable Inception layer and the attention mechanism enables multi-scale feature fusion while reducing computational cost, enhancing key features in low-dimensional features.

[0099] Step S32: Construct a CNN-based local feature branch and a Transformer-based global feature branch. The CNN-based local feature branch uses three adaptive convolutional layers with (64, 128), (128, 256), and (256, 512) channels, respectively, and a two-dimensional average pooling layer to reduce dimensionality, expanding the receptive field while preventing overfitting. Finally, fully connected layers with sizes of (512, 256) and (256, 128) output the features to the final 128 dimensions. The specific representation of the CNN-based local feature branch is as follows:

[0100]

[0101] Where GlobalAvgPool is the global average pooling layer; LayerNorm is the normalization layer; W1 is the first fully connected layer; W2 is the second fully connected layer; h3 is the output of the non-linear activation function layer; h4 is the output of the average pooling layer; H CNN This is the output of local features based on CNN branches.

[0102] In the specific calculation process, the output h2 of step S31 is used as the input to the CNN branch and the Transformer branch. The CNN-based branch performs three convolutions on h2 (with channel numbers of (64, 128), (128, 256), and (256, 512) respectively) to obtain h3. Then, global average pooling is performed on h3 to obtain h4. Finally, the data dimension is output to 128 dimensions through two fully connected layers with parameters W1 and W2, and normalization is performed to obtain the final output H of the CNN branch. CNN Its size is [64, 128]. This branch extracts neighborhood features through the local receptive field of the convolution kernel, capturing local patterns such as details, edges, and neighborhoods of the encoded signal.

[0103] The global feature branch based on Transformer consists of a visual SwinTransformer module, whose self-attention mechanism can model long-range dependencies and analyze global features. Window partitioning reduces computational cost, and its hierarchical window merging gradually covers the global context. In this embodiment, a patch embedding layer, implemented by a convolutional layer, is first set up. An improved visual SwinTransformerBlock module list is constructed, specifying two layers and two blocks per layer, and is initialized. Finally, a normalization layer, an adaptive average pooling layer, and a classification head are set. The improved visual SwinTransformerBlock module requires a normalization layer, a window attention layer, and a random deactivation path layer. The window attention layer consists of a window self-attention module (WindowAttention), setting a query, key, and value linear transformation layer, calculating a relative position bias table, and registering relative position indices. A scaling factor and a projection layer are set during the attention calculation process. Finally, a multilayer perceptron (MLP) module, containing two fully connected layers and an activation function layer, is used to perform nonlinear transformations on the input features. Setting the embedding dimension to 256 ensures that the output dimension matches the embedding dimension, resulting in a final 256-dimensional output feature. The visual SwimTransformer branch is as follows:

[0104]

[0105] Where h5 is the output of the block and linear projection image transformation function; h6 is the output of the multi-head self-attention module; PatchEmbed is the block and linear projection image transformation function; MSA is the multi-head self-attention module; LN is the layer normalization module; MLP is the multilayer perceptron layer; h7 is the multilayer perceptron layer output; H Swim-T This is the global feature output based on the Transformer branch.

[0106] Specifically, PatchEmbed's core consists of two steps: block segmentation and linear projection. In this embodiment, nn.Conv2d is used to directly merge the block segmentation and projection steps. The block size is defined as 4, dividing the output h2 from step S31 into 49 blocks. Each pixel block is then linearly mapped to 256 dimensions. The output h5 has dimensions [64, 256, 7, 7], completing the block segmentation and linear projection transformation operation. The MSA multi-head self-attention mechanism is the core operation of the Transformer branch. First, the input h5 is layer normalized, and three different linear transformations are applied to obtain three values: Q (query), K (key), and V (value). Q, K, and V are then divided into multiple heads. In this embodiment, QKV is divided into 8 heads. Next, the attention mechanism score is calculated for each block. The specific calculation formula is as follows:

[0107]

[0108] Finally, the outputs of all the heads are concatenated and then connected to the original input h5 to complete the residual connection operation, resulting in h6. Subsequently, two layers of normalization and a fully connected multilayer perceptron (MLP) layer are used to obtain the final output h7. Since the embedding dimension is set to 256, only global pooling is needed to obtain the output H with dimensions [64, 256]. Swim-T .

[0109] Step S33: The enhanced features obtained in step S31 are processed by the local and global features obtained in step S32, and then concatenated to obtain a high-dimensional encoded value, achieving reverse enhancement from low-dimensional features to high-dimensional features. In this embodiment of the invention, the local features output 128-dimensional features, and the global features output 256-dimensional features. The two dimensional vectors are concatenated using the tensor concatenation tool torch.cat() in PyTorch, an open-source framework for machine learning and deep learning, ultimately forming a 384-dimensional high-dimensional feature, specifically represented as follows:

[0110]

[0111] Where H is the high-dimensional encoded value of a single pixel detection value.

[0112] The convolutional neural network (CNN) and the global feature branch are complemented by a visual Swing Transformer dual-path multiplexing architecture, which can complement multimodal features. The interaction optimization of the two branches can effectively alleviate the problem of local detail loss in pure Transformer. At the same time, compared with the pure Transformer structure, the global receptive field coverage is increased. Under the condition of limited input features, it focuses on local accuracy and global consistency to effectively enhance feature information.

[0113] Step S4: Using a hybrid constraint of triplet loss, center loss and cross-entropy loss, jointly optimize the neural network coding layer and the dual-path multiplexed inverse feature enhancement neural network model based on the face enhancement dataset.

[0114] A hybrid constraint loss function is constructed for jointly optimizing the neural network model parameter set θ in steps S2 and S3. The hybrid constraint equations include: triplet constraint loss equation, center constraint loss equation, and cross-entropy constraint loss equation, aiming to increase the feature similarity of similar face data and the feature dissimilarity of dissimilar face data. The triplet constraint loss equation is Loss1, the center constraint loss equation is Loss2, and the cross-entropy constraint loss equation is Loss3. The Adam optimizer and gradient descent algorithm are used to optimize the neural network parameter set θ, specifically:

[0115]

[0116] Where J(θ) is the hybrid constraint loss function; ω is the weight parameter of the triplet loss; λ is the weight parameter of the center loss; and μ is the weight parameter of the cross-entropy loss. θ is the gradient of the hybrid constraint loss function with respect to the parameters; l is the learning rate of the neural network model; Loss1 is the triple constraint loss equation; Loss2 is the center constraint loss equation; Loss3 is the cross-entropy constraint loss equation; θ is the set of neural network model parameters, including ω, λ, μ and l.

[0117] The specific calculation process during optimization is as follows: three different constraints are forward propagated through the neural network to obtain three different loss values, which are multiplied by different weight coefficients to obtain the total loss value J(θ). This loss value is related to the neural network parameter θ. By calculating the gradient of J(θ) with respect to θ and updating the parameters with a step size of l, the most suitable θ is found to minimize J(θ) and approach 0, thus completing the optimization process.

[0118] By setting ω, λ, μ, and l in the parameter set θ of the neural network model, the inverse enhancement features output by the neural network model can be clearly distinguished between different categories.

[0119] In the triplet-constrained loss equation Loss1, the face enhancement data in each batch of samples is reorganized into anchor samples, with positive and negative samples forming a triplet combination; let H be the feature vector output by the anchor sample. a The feature vector output by the positive sample is H. p The feature vector output by the negative sample is H. n The formula for Loss 1, the triplet constraint loss equation, is as follows:

[0120] loss1 = max{d(H a H p )-d(H a H n )+margin,0};

[0121] Where max is the maximization function; d(·,·) is the Euclidean distance function between two feature vectors; margin is the positive and negative sample distance interval parameter, a value greater than zero, used to control the distance interval between positive and negative samples; H a H is the feature vector output for the anchor sample. p H is the feature vector output for positive samples; n The feature vector output for negative samples.

[0122] In the central constraint loss equation Loss2, let H be the feature vector of the i-th sample. i Its category is y i The central feature vector of this category is The formula for the central constraint loss equation Loss2 is:

[0123]

[0124] Where ||·||2 is the Euclidean L2 norm; and the category center H is set as a variable that will be continuously updated during the training process. i Let y be the feature vector of the i-th sample; i Let i be the category to which the i-th sample belongs; Let be the central feature vector of the category to which the i-th sample belongs; i is the sample number of the face augmentation data.

[0125] In the cross-entropy constrained loss equation Loss3, let the true label of the sample be y, and the model predicted label be y; the formula for the cross-entropy constrained loss equation Loss3 is:

[0126]

[0127] Among them, y i Let i be the true label of the i-th sample; Let be the model's predicted label for the i-th sample; log is the logarithmic function.

[0128] Since cross-entropy loss requires class prediction, but the neural network model output is high-dimensional features without class prediction, an additional fully connected layer needs to be added to the output class before calculating the cross-entropy loss and jointly optimizing the neural network parameters to achieve class probability prediction. Figure 6 The figure shows the convergence curve of the hybrid constraint loss function in the joint optimization process of this invention. As can be seen from the figure, the loss function decreases continuously with iterative iterations, approaching 0, indicating that the neural network converges rapidly and effectively learns the mapping relationship between the training set and the categories.

[0129] Step S5: Optimize the reverse feature enhancement neural network model from Step S3 using Step S4, and train it based on the face enhancement data set O. train A face embedding database is constructed, and face recognition is achieved by comparing the high-dimensional encoded value H of the single-pixel detection value output by the reverse feature enhancement neural network model in step S3 with the Euclidean distance of the features in the face embedding database.

[0130] Step S51: Train the face enhancement data set O obtained in step S1. trainThe actual single-pixel detection value S of the low-dimensional encoding obtained by convolutional coding in the optical coding layer of the jointly optimized step S2 is input into the optical coding layer to simulate optical-assisted coding. The actual single-pixel detection value S of the low-dimensional encoding is then input into the dual-path multiplexed inverse feature enhancement neural network model of the jointly optimized step S3 to perform inverse feature enhancement and obtain the feature space for face recognition as a face embedding database. The specific process is as follows:

[0131]

[0132] Among them, H train This embeds all features stored in the database into the face. For the high-dimensional features corresponding to the face image data of the i-th sample in the training set; This refers to the mapping relationship between the model parameters after training and optimization. This refers to the mapping relationship between the model parameters after training and optimization; y train The training set consists of 12,800 face image labels.

[0133] Step S52: Verify the face enhancement data obtained in step S1 using the verification set O. val With face enhancement data test set O test The input is fed into the optimized reverse feature enhancement neural network model from step S4 to obtain high-dimensional validation output features and high-dimensional test output features, specifically:

[0134]

[0135] Among them, H val The high-dimensional validation output features obtained from the optimized network model on the validation set; H test The high-dimensional test output features obtained from the optimized network model of the test set; To verify the high-dimensional features corresponding to the face image of the i-th sample in the set; This refers to the high-dimensional features corresponding to the face image of the i-th sample in the test set.

[0136] Step S53: Calculate the high-dimensional verification output feature H using the method described in step S52. val and high-dimensional test output features H test Compared with all features H stored in the face embedding database in step S51 train The Euclidean distance is used to determine the feature class with the minimum Euclidean distance as the class of the verification or test face, thus completing the recognition task. Specifically:

[0137]

[0138] Where H is the high-dimensional encoded value of the model output, which is H when recognizing the face enhancement data validation set. val When recognizing the face enhancement data test set, it is H test ;d i The output feature is the Euclidean distance between the feature vector and the feature vector corresponding to the face image of the i-th sample in the face embedding database; k is the d i The index corresponding to the minimum Euclidean distance in the middle; y k To train a set of face image labels y train The face image label with index k is y; the face image prediction label is y, i.e., the recognition result; argmin is the set of independent variable values ​​that achieve the minimum value.

[0139] In this embodiment of the invention, face enhancement data set O is first used for training. train After joint optimization training in steps S2, S3, and S4, the neural network model yields a high-dimensional feature set of the face enhancement data training set, which serves as the face embedding database, i.e., the sample library. During testing, the detection values ​​obtained by the single-pixel detector are processed by the same trained neural network model to obtain the enhanced features corresponding to the compressed measurement values. When using Euclidean distance to compare the enhanced features of the test with the face embedding database for feature recognition, the closest distance (TOP1) is selected as the final recognition result.

[0140] This invention embodiment can also use the KNN nearest neighbor algorithm to select the top 5 nearest neighbors (where K=5 in the KNN algorithm) as the final recognition result. Specifically, it finds the index corresponding to the minimum distance in the Euclidean distance feature comparison vector, which points to the feature vector in the training set that is most similar to the current test feature. The corresponding face image label is obtained from the training label list; this face image label is the TOP1 prediction result for the current test sample. Alternatively, it obtains the indices corresponding to the 5 smallest distances in the distance vector, and based on the obtained 5 nearest indices, obtains the corresponding 5 labels from the training label list. Simultaneously, it obtains these 5 nearest distances from the distance vector, counts the occurrence frequency of the 5 labels, and finds the label with the highest occurrence frequency. If there is only one label with the highest occurrence frequency, it is directly selected as the prediction label. If multiple labels have the same occurrence frequency, the average distances corresponding to these labels are calculated, and the label with the smallest average distance is selected as the face image prediction label, which is the final face recognition result.

[0141] like Figure 7 The diagram illustrates the average distance between feature comparisons of the minimum and maximum values ​​in this invention, using category 12 as an example. As can be seen from the diagram, the average distance between feature comparisons within the same category is significantly smaller than the average distance between feature comparisons between different categories; therefore, this model can effectively distinguish between different face image categories.

[0142] In this embodiment of the invention, the jointly optimized optical coding layer, the dual-path multiplexed inverse feature extraction neural network model, and the central category features are all optimized using the Adam optimizer. The hybrid constraint weight parameters ω, λ, and μ are 1, 0.01, and 1, respectively. The initial learning rate of the network is set to 0.001, and the number of iterations (epochs) is set to 300. The result with the best recognition accuracy on the validation set out of 300 iterations is selected as the final output. Validation is then performed on the test set. Figure 8 The diagram shows the confusion matrix of the test set recognition results at a sampling rate of 0.012, with a recognition accuracy of 95.6%. To verify the performance of the proposed method, three training iterations were performed at multiple sampling rates (0.004, 0.006, 0.008, 0.012, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, and 0.10). The model with the best recognition accuracy on the validation set was selected as the final model for test set recognition. Figure 9 The diagram shows line graphs illustrating the recognition results of this invention at various sampling rates, with amplification applied to extremely low sampling rates (0.004, 0.006, 0.008, 0.012). At extremely low sampling rates, the recognition accuracy of this method reaches 90% (0.004), and the accuracy gradually increases with increasing sampling rates. To verify the actual contributions of the binarized neural network convolutional coding layer based on pass-through estimator quantization and the reverse feature enhancement neural network model based on CNN and Transformer dual-path multiplexing in this invention, ablation experiments were conducted on each module in this embodiment. In the compression encoding stage, the binarized neural network convolutional coding layer based on pass-through estimator quantization proposed in this invention was compared with the traditional regularization-based convolutional coding layer. In the feature inverse enhancement stage, the single-path CNN, single-path Transformer, and the proposed dual-path multiplexing model were compared, and the results are shown in Table 1. Ablation experiments were conducted at a compression sampling rate of 0.012. As can be seen from the table, the binarized neural network convolutional coding layer based on the pass-through estimator quantization improves the recognition accuracy by 2.8% compared to the traditional regularization method. The dual-path multiplexing feature inverse enhancement method increases the recognition accuracy by 1.3% compared to the single-path CNN model.

[0143] Table 1. Accuracy Comparison of Different Model Combinations

[0144]

[0145] The beneficial effects of this invention are as follows: This invention implements optical encoding of the target object through a direct-pass estimator binary neural network convolutional coding layer equivalent to a spatial light modulator, fitting the binarization requirements of the physical device. Simultaneously, during joint network training, backpropagation is not affected, eliminating prediction errors caused by the mismatch between the convolutional coding layer and the actual optical encoding. The proposed dual-stream global-local feature fusion processor combining CNN and Transformer can effectively enhance features that differentiate between different categories in complex datasets. By dual-streaming CNN and Transformer, the model can simultaneously utilize the advantages of both in local and global feature extraction; evaluating local details and global structure improves the understanding and representation of complex data. Constrained by the mixed feature differential loss, the output features have good discriminative power, providing more detailed and representative features for subsequent feature comparison. Validated on the ORL dataset, the recognition accuracy reaches 95.6% at a compression sampling rate of 0.012.

[0146] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A face image privacy protection recognition method based on optical coding and inverse feature enhancement, characterized in that, It includes: S1: Obtain the face dataset, perform data augmentation on it, construct the face augmentation dataset, and divide it into training set O. train Validation set O val With test set O test ; S2: Construct a binary neural network convolutional coding layer based on the pass-through estimator quantization to perform convolutional coding on the face enhancement dataset in step S1; set up the neural network convolutional coding layer, and use binarization operations and the pass-through estimator to update the parameters of the convolutional coding layer; map the binary neural network convolutional coding layer to the spatial light modulator, and obtain the encoded value from the light intensity value collected by the single-pixel detector; perform positive and negative modulation optical coding on the negative terms in the convolutional coding layer, and perform difference operations on the obtained single-pixel detector values ​​to simulate the original convolutional coding layer encoding the face, and obtain the low-dimensional encoded actual single-pixel detection value S; S3: Construct a dual-path multiplexing reverse feature enhancement neural network model based on convolutional neural network (CNN) and deep learning model based on self-attention mechanism (Transformer). Perform feature dimensionality enhancement on the low-dimensional encoded actual single-pixel detection value S obtained in step S2 to obtain the high-dimensional encoded value H of the single-pixel detection value. S4: Using a hybrid constraint of triplet loss, center loss and cross-entropy loss, construct a hybrid constraint loss function for jointly optimizing the neural network model parameter set θ in steps S2 and S3. Based on the face enhancement dataset, jointly optimize the neural network coding layer and the dual-path multiplexed inverse feature enhancement neural network model. S5: Optimize the reverse feature enhancement neural network model in step S3 using step S4, construct a face embedding database, and compare the high-dimensional encoded value H of the single-pixel detection value output by the reverse feature enhancement neural network model in step S3 with the Euclidean distance of the features in the face embedding database to achieve face recognition.

2. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: Step S2 is as follows: S21: Set the optical coding matrix, construct the neural network convolutional coding layer for optically assisted target scene coding; use binarization operations and a pass-through estimator to set up the forward and backward propagation processes of the neural network convolutional coding layer; S22: Train the face enhancement data set O train As input to the binary convolutional encoding matrix Perform convolution to obtain the single-pixel detection value S1 of the optically assisted face image encoding information; S23: When testing the trained neural network model, the binary neural network convolutional coding layer is mapped to the spatial light modulator. The single-pixel detector values ​​obtained by positive and negative modulation optical coding are differentially processed to simulate the original convolutional coding layer encoding the face, resulting in the low-dimensional encoded actual single-pixel detector value S.

3. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: Step S3 is as follows: S31: Perform layer normalization processing, amplify features through pixel recombination upsampling layer, enhance model features through depth-separable convolutional layer, and assign weights by combining spatial attention mechanism and channel attention mechanism to obtain enhanced features; S32: Construct a CNN-based local feature branch and a Transformer-based global feature branch. The CNN-based local feature branch consists of an adaptive convolutional layer, a two-dimensional average pooling layer, and a fully connected layer; the Transformer-based global feature branch consists of a visual SwinTransformer module. S33: The enhanced features obtained in step S31 are processed by the local features and global features obtained in step S32, and the two are concatenated to obtain a high-dimensional encoding value, thereby realizing the reverse enhancement from low-dimensional features to high-dimensional features.

4. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: The hybrid constraint loss function in step S4 is as follows: Where J(θ) is the hybrid constraint loss function; ω is the weight parameter of the triplet loss; λ is the weight parameter of the center loss; and μ is the weight parameter of the cross-entropy loss. θ is the gradient of the hybrid constraint loss function with respect to the parameters; l is the learning rate of the neural network model; Loss1 is the triple constraint loss equation; Loss2 is the center constraint loss equation; Loss3 is the cross-entropy constraint loss equation; θ is the set of parameters of the neural network model.

5. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: Step S5 is as follows: S51: Train the face enhancement data set O train The feature space obtained from the joint optimization steps S2 and S3 is used as a face embedding database for face recognition. S52: Expand the face enhancement data to the verification set O val With face enhancement data test set O test The input is fed into the optimized reverse feature enhancement neural network model in step S4 to obtain high-dimensional verification output features and high-dimensional test output features; S53: Calculate the Euclidean distance between the output features of step S52 and all features stored in the face embedding database in step S51, and determine the feature category with the smallest Euclidean distance as the category of the verification face or test face.

6. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: In step S21, during the backpropagation process, a pass-through estimator is used to calculate the gradient based on the face enhancement dataset for updating, avoiding the influence of binarization operations; the specific formula for updating the convolutional coding layer parameters is as follows: Where C' represents the parameters of the convolutional coding layer updated after backpropagation; l represents the backpropagation learning rate; J represents the hybrid constraint loss function; C encode These are the parameters for the convolutional coding layer.

7. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: In step S23, the single-pixel detector values ​​obtained from positive and negative modulation optical coding are subjected to differential operations to simulate the encoding of the face by the original convolutional coding layer; the resulting low-dimensional encoded actual single-pixel detector values ​​are: S=S + -S - =P + O test -P - O test Where S is the actual single-pixel detection value encoded in low dimension; S + S represents the detected light intensity value corresponding to the positively modulated light field. - P represents the intensity detection value corresponding to the negatively modulated optical field. + It is a positive modulation optical code; P - For negative modulation optical coding; O test This is a test set of face augmentation data.

8. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: The input to step S31 is the low-dimensional encoded actual single-pixel detection value S. The specific process is as follows: h1 = PixelShuffle(h0) b1=YELLOW(BN(Conv 1×1 (h1))) b2=GELU(BN(Conv 1×1 (GELU(BN(DepthwiseConv 3×3 (h1)))))) b3=GELU(BN(Conv 1×1 (GELU(BN(DepthwiseConv 5×5 (h1)))))) b4=GELU(BN(Conv 1×1 (MaxPool(h1))) Wherein, Reshape is the size modification function; PixelShuffle is the pixel recombining upsampling layer; b1 is the first branch in the Inception layer; b2 is the second branch in the Inception layer; b3 is the third branch in the Inception layer; b4 is the fourth branch in the Inception layer; CBAM is the spatial attention mechanism and channel attention mechanism. The space is a concatenation of two vectors in dimension 1; GELU is a non-linear activation function layer; batchsize is the number of input samples in each forward and backward propagation; h0 is the output of the size modification function; h1 is the output of the pixel reassembly upsampling layer; h2 is the output of the spatial attention mechanism and channel attention mechanism; BN is a deep learning optimization layer; Conv is a convolutional layer; Depthwise is a depthwise convolutional layer; MaxPool is a max pooling layer; M is the number of compressed samples.

9. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: The specific details of the CNN-based local feature branch in step S32 are as follows: h3=GELU(BN(Conv 3×3 (h2)))×3 h4 = GlobalAvgPool(h3) H CNN =LayerNorm(W2GELU(W1h4)) Where GlobalAvgPool is the global average pooling layer; LayerNorm is the normalization layer; W1 is the first fully connected layer; W2 is the second fully connected layer; h3 is the output of the non-linear activation function layer; h4 is the output of the average pooling layer; H CNN This is the output of local features based on CNN branches.

10. The face image privacy protection recognition method based on optical coding and inverse feature enhancement according to claim 1, characterized in that: The global feature branch based on Transformer in step S32 is as follows: h5 = PatchEmbed(h2) H Swim-T =GlobalAvgPool(LN(h7)) Where h5 is the output of the block and linear projection image transformation function; h6 is the output of the multi-head self-attention module; PatchEmbed is the block and linear projection image transformation function; MSA is the multi-head self-attention module; LN is the layer normalization module; MLP is the multilayer perceptron layer; h7 is the multilayer perceptron layer output; H Swim-T The output is the global feature based on the Transformer branch; AvgPool is the average pooling layer.

Images