A hardware-accelerated embedded system design method for face recognition

Abstract

The present invention provides a hardware accelerated embedded system design method for face recognition. Based on a DETR target detection model, a target feature map is dimensionally reduced and activated through an inverse residual module of a MobileNetV2 feature extraction network, an output result of a backbone layer is input into an encoder in which self-attention is replaced by HiLo-Attetion, images of different sizes are feature extracted through an improved DETR target detection model, and pictures obtained are input into a Neck layer, a custom model CGVV is used in a BN layer to perform a pruning operation to reduce the complexity of the model, and finally the trained final model parameters are stored in a storage medium, data is interacted with an FPGA, and the model is tested and verified. Through the above operations, the present invention aims to realize efficient face recognition applications, realize high-speed data interaction, and provide a new solution for the application of face recognition technology in an embedded environment.

Description

Technical Field

[0001] The invention relates to the field of embedded image processing, and in particular to a hardware accelerated embedded system design method for face recognition. Background Art

[0002] Face recognition systems usually consist of four basic steps: image acquisition, preprocessing, feature extraction, and matching recognition. In the image acquisition stage, a camera or sensor is used to capture a face image. Then, the image may be preprocessed, such as noise removal, alignment, and normalization. Next, features are extracted from the preprocessed image, which can be key points, lines, textures, etc. of the face. Finally, these features are used to identify or verify the face.

[0003] Mainstream face recognition algorithms include traditional feature-based methods and modern deep learning-based methods, which are widely used in the field of face detection. The disadvantages of feature-based methods are that they may have low accuracy for complex scenes and images with large changes, and their ability to represent details and texture information is relatively weak and may produce redundant features. They are insufficient in recognition scenes with large flow of people and complex backgrounds. Therefore, deep learning-based face recognition methods such as CNN are more suitable for today's social environment. FPGA (field programmable gate array) plays an important role in face recognition. Its parallel processing capability and hardware-level programmability can achieve fast face recognition while ensuring high performance. How to combine deep learning networks with hardware embedded processing is worth studying in terms of improving the accuracy and upper limit of convergence speed of image processing and face recognition technology. Summary of the invention

[0004] Purpose of the invention: In order to solve the above technical problems, the present invention proposes a design method for a face recognition hardware acceleration embedded system. Based on the improved DETR model, the CASIS-FaceV5 dataset is created, MobileNetV2 is used as a lightweight feature extraction network, and HiLo-Attention and variability cross-scale feature fusion modules are introduced to improve the efficiency of feature extraction and fusion. An accelerator HLS module is designed. This module optimizes the convolution algorithm and uses parallel computing, providing a new solution for the application of face recognition technology in embedded environments.

[0005] Technical solution:

[0006] The present invention provides a method for designing a face recognition hardware acceleration embedded system, the method comprising the following steps:

[0007] S1 builds a face dataset;

[0008] S2 builds and improves the DETR target detection model;

[0009] S2.1 builds a lightweight feature extraction network MobileNetV2 as the backbone network, and uses the inverse residual module to make the feature map undergo convolution dimension increase and final convolution dimension reduction;

[0010] S2.2 uses HiLo-Attetion to replace the self-attention in the original encoder layer, uses non-overlapping window division in Hi-Fi, concatenates the output of high-frequency attention and the output of low-frequency attention on the channel, and outputs the result to the subsequent cross-scale feature fusion module CCFM with integrated variability.

[0011] S3 is used to train the improved DETR target detection model with the dataset;

[0012] S4 applies dual-branch pruning technology to the final model BN layer for optimization and introduces a custom model CGVV model. One branch uses dual GSConv layers to optimize model performance; the other branch directly outputs the original feature map through Ghost bottleneck, uses shortcut connection and outputs the result to VoVGSCSP;

[0013] S5 designs the HLS accelerator module and generates an instruction set to be loaded onto the FPGA hardware;

[0014] S6 stores the trained final model parameters into DDR3, and FPGA connects to DDR3 through the AXI bus controller and performs data exchange;

[0015] S7 uses pictures for result testing.

[0016] Furthermore, the face dataset in S1 is constructed based on the CASIS-FaceV5 dataset, with an image size of (640, 480), and the validation set uses LFW.

[0017] Furthermore, the detailed steps of S2.1 are:

[0018] First, use 1x1 convolution to increase the dimension of the input feature map by a multiple of t=6, then activate it through BN+RELU6 and perform deep convolution processing, activate it again through BN+RELU6, and finally reduce the dimension through 1x1 convolution. A linear transformation activation function is attached at the end of the process.

[0019] Furthermore, the HiLo-Attetion described in S2.2 is detailed as follows: two effective attentions are used to decouple the high / low frequencies in the feature map, the Head is assigned to the high-frequency attention, and the fine-grained high frequency is captured through Local Window Self-Attention; average pooling is applied to each window to obtain the low-frequency signal, and the remaining Head is assigned to Lo-Fi, and the relationship between each query position in the input feature map and the average pooled low-frequency key of each window is modeled to achieve low-frequency attention.

[0020] Furthermore, the high-frequency attention HIFI operation steps are as follows:

[0021] S1 sets the window size to ws, that is, a window has ws x ws patches, that is, each row and column:

[0022] H group =H / ws

[0023] W group =W / ws

[0024] total_groups = H group *W group

[0025] Each column has H group Group window, each line has W group Group windows, that is, there are total_groups group windows in the whole picture;

[0026] S2 transforms the input X∈R through the reshape operation H×W×C Reshape to: Through three linear layers, we obtain Q, K, and V respectively:

[0027]

[0028] Q∈R total_group*ws*ws*h*d = reshape(Q)

[0029] Q∈R total_group*h*ws*ws*d = reshape(Q)

[0030] Among them, W q ∈R C×C C = h X d, where h is the number of attention heads and d is the number of channels of each attention head. K and V are processed in the same way as Q, so they have the same shape.

[0031] S3 performs attention operation between Q, K, V:

[0032]

[0033] S4 S represents the attention matrix between all patch pairs within each window, which is then weighted summed with V to update the patch representation within each window:

[0034] Y∈R total_group*ws*ws*h*d =S*V

[0035]

[0036] It is the output of high-frequency attention;

[0037] The low-frequency attention loFI operation steps are as follows:

[0038] S1 Lo-Fi first applies average pooling to each window to obtain the input X∈R H*W*C Low frequency signal in:

[0039] X∈R W*H*C = reshape(X)

[0040]

[0041] X_∈R HW / ws*ws*C = reshape(X_)

[0042] HW / (ws)2 represents the number of patches after pooling, HW=14x14=196 patches, the pooling window ws×ws=2×2=4 patches, after pooling, the remaining HW / (ws)2=196 / 4=49 patches;

[0043] The S2 average pooling feature map is projected to keys: K∈R HW / ws*ws*C and values:V∈R HW / ws*ws*C And reshape it using the reshape operation:

[0044] K∈R HW / (ws*ws)*C =X_W K

[0045] V∈R HW / (ws*ws)*C =X_W v

[0046] K∈R h*HW / ws*ws*d = reshape(K)

[0047] V∈R h*HW / (ws*ws)*d = reshape(V)

[0048] Where Wk,Wv∈R C*C C = h × d, where h is the number of attention heads and d is the number of channels of each attention head;

[0049] Q in S3 Lo-Fi still comes from the original feature map X∈R W*H*C

[0050] Q∈R W*H*C =XW q

[0051] Q∈R h*HW*d = reshape(Q)

[0052] S4 Then, a standard attention mechanism is applied to capture the rich low-frequency information in the feature map. Note that the complexity of Lo-Fi is significantly reduced due to the reduction of the spatial dimensions of K and V:

[0053]

[0054] U represents the attention matrix between the patch and the window, N = HW is the total number of patches, and HW / (ws)2 is the number of patches after pooling;

[0055] Perform weighted summation with V and update each patch representation:

[0056] Y L ∈R h*HW*d =UV

[0057] Y L ∈R H*W*C = reshape(Y L )

[0058] Y L ∈R H*W*C is the output of low-frequency attention.

[0059] Furthermore, the output of high-frequency attention and the output of low-frequency attention in S2.2 are concatenated on the channel as follows:

[0060] The output of the high-frequency attention and the output of the low-frequency attention are concatenated on the channel and restored to the same shape as the input:

[0061] Y∈R H*W*2C =Concat(Y h , Y L )

[0062] Y∈R N*2C) = reshape(Y).

[0063] Furthermore, the detailed steps of S4 are:

[0064] A custom CGVV model is introduced in the BN layer for pruning operations. The feature map is processed by a dual-branch method. One branch uses two GSConv convolutional layers, and the other uses Ghost bottlenec to output the original feature map. Ghostbottlenec consists of two stacked Ghost modules: the first Ghost module is used as an expansion layer to increase the number of channels;

[0065] The second Ghost module reduces the number of channels to match the shortcut path, and uses the shortcut to connect the input and output of the two Ghost modules. ReLU is not used after the second Ghost module. Batch normalization (BN) and ReLU nonlinear activation are applied to other layers after each layer. The outputs of the two modules are connected and output to VoVGSCSP. The fused feature map is input to the GSConv convolution layer again for output, eliminating the calculation of the BN layer and saving the obtained weight parameters and data set.

[0066] Furthermore, the accelerator HLS module in S5 designs multiple IP cores to accelerate different operations, including:

[0067] PWConv: accelerated point-wise convolution;

[0068] DWConv: accelerated depth-wise convolution;

[0069] conv: speeds up the convolution operation of the first layer of the network;

[0070] fc: accelerates the global average pooling layer and the fully connected layer;

[0071] The DW convolution operation is divided into three stages: loading data, calculating convolution, and writing back the results. The ping-pong operation masks the data transmission time.

[0072] PW convolution operations include increasing the interface bit width, increasing the burst transmission length, and increasing the number of interfaces;

[0073] Finally, block design is performed in Vivado to generate an instruction set that is loaded onto the FPGA hardware.

[0074] Furthermore, the S7 test steps include: sending the result obtained by the convolution calculation to the result buffer and then passing it to the result processing module, performing residual addition, pooling operation, ReLu and quantization operations on it in turn, and finally sending it to the output buffer, and transmitting it to the computer end through the AXI bus for verification.

[0075] Beneficial effects:

[0076] 1. This invention replaces the self-attention in the encoder with HiLo-Attetion, combining high-resolution fine features with low-resolution global features, helping the model to maintain the recognition of the overall face structure while also noticing small changes in detail features such as eyes, nose, and mouth. The design of this attention mechanism enables the model to capture features at different scales and improve recognition results.

[0077] 2. The present invention designs MobileNetV2 as the backbone network, which has a lower computing cost. Deploying the MobileNetV2 model on FPGA can make full use of the FPGA's parallel computing capabilities to achieve a fast and efficient reasoning process, with a faster recognition convergence speed at a lower cost.

[0078] 3. The present invention uses a custom model CGVV and applies dual-branch pruning technology to reduce the complexity of the model, improve the reasoning speed, and further improve the accuracy of the model as much as possible.

[0079] 4. The present invention further reduces the convergence delay through the hardware implementation of FPGA, responds to input data more quickly and generates recognition results. At the same time, it has lower power consumption than other processors. While increasing the convergence speed, it also improves the endurance and robustness in mobile or embedded device scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a process flow chart of the present invention;

[0081] Figure 2 This is a schematic diagram of the MobileNetV2 structure;

[0082] Figure 3 This is a schematic diagram of the HiLoAttention structure;

[0083] Figure 4 This is a schematic diagram of the custom model CGVV model;

[0084] Figure 5 Schematic diagram of the improved DETR model;

[0085] Figure 6 Schematic diagram of CCFM model. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0087] Step 1: The present invention uses the CASIS-FaceV5 dataset, which is a large-scale face dataset containing a large number of face image samples. These images come from different people and different scenes, covering a wide range of facial expressions, postures, and lighting conditions. The face images in the dataset are diverse, including changes in different age groups, races, genders, and other aspects. This diversity makes the CASIA-FaceV5 dataset suitable for many different face recognition and face verification tasks.

[0088] CASIS-FaceV5 contains a face dataset of 500 Asians, each of whom has 5 pictures with a size of (640, 480).

[0089] The validation set uses LFW. This is the most commonly used test set. Most of the face images in this dataset are from natural life scenes, and the recognition difficulty is greater than that of other datasets. Before the validation, the image is preprocessed, including converting the input image into a grayscale image, flipping the image, and randomly cropping the image to obtain a (128, 128) image.

[0090] Step 2: Build a lightweight feature extraction network MobileNetV2, which filters features through lightweight depth convolution and linear convolution while removing nonlinearities in narrow layers to maintain representation capabilities. MobileNetV2 is designed to reduce model size and computational complexity, making it suitable for mobile devices and resource-constrained environments. Although MobileNetV2 is a lightweight model, it performs well in recognizing and classifying images. It improves performance by designing network structure and utilizing advanced modules (such as inverted residual modules) while maintaining the lightweight nature of the model. Due to the lightweight design of MobileNetV2, it can run efficiently on a variety of mobile devices and embedded systems, including smartphones, tablets, IoT devices, etc. The architecture of MobileNetV2 is relatively flexible, and hyperparameters and network structures can be adjusted to meet the needs of specific applications. This makes it a general model suitable for a variety of image-related tasks.

[0091] The inverse residual module is the main component structure of MobileNetV2, such as Figure 2 As shown, 1x1 convolution is first used to increase the dimension of the input feature map, generally by a multiple of t=6, and then activated by a BN+RELU6, and then a deep convolution is used, followed by a BN+RELU6, and finally a 1x1 convolution is used to reduce the dimension, followed by a linear transformation activation function. Compared with the standard residual module, the standard residual module first reduces the dimension and then increases the dimension, and the tensors with more channels are short-circuited; the inverse residual module first increases the dimension and then reduces the dimension, and the tensors with fewer channels are short-circuited. The last activation layer is called linear bottleneck. ReLU6 is used because it can limit the weights to a relatively small range during the training phase, so that it can run stably on low-precision edge devices (robustness).

[0092] Step 3: Input the output of the backbone layer to the encoder layer. Given input X∈R N×C , N is the number of patches, C is the number of channels in each patch. Usually the size of an image is 224x224, that is, H'=W', so after dividing an image into patches, the distribution should be uniform, that is, the number of patches in each row is equal to the number of patches in each column, then: (Here, H and W are used to define the number of patches per row and column.) Reshape X∈R N×C is: X∈R H×W×C .

[0093] In this example, the self-attention in the encoder layer is replaced by HiLo-Attetion: HiLo uses two effective attentions to decouple the high / low frequencies in the feature map. Figure 3 As shown, in the upper path, several heads are assigned to high-frequency attention (Hi-Fi) to capture fine-grained high frequencies through Local Window Self-Attention (e.g., 2×2 windows), which is more effective than standard MSA. In the lower path, low-frequency attention (Lo-Fi) is implemented, and average pooling is first applied to each window to obtain low-frequency signals. Then, the remaining heads are assigned to Lo-Fi to model the relationship between each query position in the input feature map and the average pooled low-frequency key of each window. Benefiting from the reduction in key and value lengths, the complexity of Lo-Fi is significantly reduced.

[0094] Finally, the refined high / low frequency features are concatenated and the resulting output is forwarded to subsequent layers. Since neither Hi-Fi nor Lo-Fi is equipped with time-consuming operations such as dilation windows and recursion, the overall framework of HiLo is fast on GPU. Comprehensive benchmarks show that HiLo outperforms existing attention mechanisms in terms of performance, FLOPs, throughput, and memory consumption.

[0095] The detailed steps are as follows:

[0096] High-frequency attention (Hi-Fi)

[0097] 1) Since high frequencies encode local details of objects, applying global attention on feature maps may be redundant and computationally expensive. Hi-Fi is designed to capture fine-grained high-frequency features (e.g., through 2×2 local window self-attention), which greatly saves computational complexity. In addition, this embodiment adopts simple non-overlapping window partitioning in Hi-Fi, which is more hardware-friendly compared to time-consuming operations such as window shifting or multi-scale window partitioning.

[0098] 2) Set the window size to ws, that is, a window has ws x ws patches (for example, if the window size ws = 2, then a window has 4 patches). For each row or column:

[0099] H group =H / ws

[0100] W group =W / ws

[0101] total_groups = H group *W group

[0102] Each column has H group Group window, each line has W group Group windows, that is, the whole picture has a total of total_groups group windows L [for example: a picture has N = 14 × 14- = 196 patches, each window has 2 × 2 = 4 patches, then there are a total of: total_groups = (14 / 2) x (14 / 2) = 49 group windows].

[0103] 3) Then, the input X∈R is reshaped by the reshape operation H×W×C Reshape to: Through three linear layers, we obtain Q, K, and V respectively:

[0104]

[0105] Q∈R total_group*ws*ws*h*d = reshape(Q)

[0106] Q∈R total_group*h*ws*ws*d = reshape(Q)

[0107] Among them, W q ∈R C×C C = h X d, where h is the number of attention heads and d is the number of channels of each attention head. K and V are processed in the same way as Q, so the shapes are also the same.

[0108] 4) Perform attention operation between Q, K, V:

[0109]

[0110] 5) S represents the attention matrix between all patch pairs in each window, which is then weighted summed with V to update the patch representation in each window:

[0111] Y∈R total _ group*ws*ws*h*d =S*V

[0112]

[0113] is the output of high-frequency attention (usually the output needs to go through an additional linear layer, which is ignored here)

[0114] Low-frequency attention (Lo-Fi)

[0115] 1) The global attention of multi-head self-attention (MSA) helps capture low-frequency features. However, applying MSA directly to high-resolution feature maps requires huge computational cost. Since averaging is a low-pass filter, Lo-Fi first applies average pooling to each window [window size is ws×ws] to obtain the input X∈R H*W*C Low frequency signal in:

[0116] X∈R W*H*C = reshape(X)

[0117]

[0118] X_∈R HW / ws*ws*C = reshape(X_)

[0119] HW / (ws)2 represents the number of patches after pooling [same as the example of high-frequency attention, HW=14x14=196 patches, pooling window ws×ws=2×2=4 patches, after pooling, there are HW / (ws)2=196 / 4=49 patches left, which is equivalent to the number of windows].

[0120] 2) Next, the average pooled feature map is projected to keys: K∈R HW / ws*ws*C and values:V∈R HW / ws*ws*C And reshape it using the reshape operation:

[0121] K∈R HW / (ws*ws)*C =X_W K

[0122] V∈R HW / (ws*ws)*C =X_W v

[0123] K∈R h*HW / ws*ws*d = reshape(K)

[0124] V∈R h*HW / (ws*ws)*d = reshape(V)

[0125] Where Wk,Wv∈R C*C C = h × d, where h is the number of attention heads and d is the number of channels of each attention head.

[0126] 3) Q in Lo-Fi still comes from the original feature map X∈R W*H*C

[0127] Q∈R W*H*C =XW q

[0128] Q∈R h*HW*d = reshape(Q)

[0129] 4) Then, a standard attention mechanism is applied to capture the rich low-frequency information in the feature map. Note that due to the reduction in the spatial dimensions of K and V, the complexity of Lo-Fi is significantly reduced:

[0130]

[0131] U represents the attention matrix between the patch and the window, N = HW is the total number of patches, and HW / (ws)2 is the number of patches after pooling, or the number of windows. Then the weighted sum is taken with V to update each patch representation:

[0132] Y L ∈R h*HW*d =UV

[0133] Y L ∈R H*W*C = reshape(Y L )

[0134] Y L ∈R H*W*C is the output of low-frequency attention.

[0135] 5) The output of the high-frequency attention and the output of the low-frequency attention are concatenated on the channel and restored to the same shape as the input:

[0136] Y∈R H*W*2C =Concat(Y h , Y L )

[0137] Y∈R N*2C) = reshape(Y).

[0138] Step 4: The improved DETR lightweight feature extraction network extracts features from images of different sizes and inputs the obtained images into the Neck layer. The CCFM proposed in this embodiment at the Neck layer adjusts the output of the encoder to two dimensions in order to complete the subsequent cross-scale feature fusion. CCFM uses the feature fusion of the FPN+PAN feature pyramid. It extracts features at different levels in the deep network and constructs a multi-scale feature pyramid through upsampling and downsampling operations.

[0139] A deformable cross-scale feature fusion module (deformable CCFM) is proposed in the Neck layer. It extracts information from the input feature map through a series of lightweight convolutional layers, and then performs feature fusion through a lightweight convolution operation. This operation uses a 1x1 convolution kernel to achieve horizontal fusion of feature maps of different scales. The fused feature map is passed through a series of 1x1 convolutional layers to further enhance the feature representation capability. Finally, the processed feature map is passed to the subsequent detection head for prediction. The role of this module is to integrate features of different scales through fusion operations to enhance the model's adaptability to scale changes and its ability to detect small-scale objects. CCFM can effectively integrate detailed features and contextual information, thereby improving the overall performance of the model.

[0140] Step 5: Hyperparameter setting: Select the training batch size as 256. Select the AreFaceLoss loss function in the training, input the data set to train the model, and obtain the final model and the model parameters after training.

[0141] Step 6: After training is completed, in order to speed up the reasoning, a custom model CGVV is used in the BN layer for pruning to reduce the complexity of the model, improve the reasoning speed, and maintain the accuracy of the model as much as possible.

[0142] Processing feature maps through a dual-branch approach One branch uses two GSConv convolutional layers, which can reduce the complexity of the model and better balance the accuracy and speed of the model. The other uses Ghost bottlenec to output the original feature map. Ghost bottlenec mainly consists of two stacked Ghost modules. The first Ghost module is used as an expansion layer to increase the number of channels. Here, the ratio of the number of output channels to the number of input channels is called the expansion ratio. The second Ghost module reduces the number of channels to match the shortcut path. Then, the shortcut is used to connect the input and output of the two Ghost modules. Here, MobileNetV2 is borrowed. ReLU is not used after the second Ghost module, and batch normalization (BN) and ReLU nonlinear activation are applied to other layers after each layer. Then the two outputs are connected and output to VoVGSCSP, which is a cross-stage partial network module designed using a one-time aggregation method for effective information fusion between feature maps at different stages. The fused feature map is input to the GSConv convolutional layer again for output. In this way, the calculation of the BN layer can be omitted during inference. Save the obtained weight parameters and data set.

[0143] Step 7: Save the parameters of each part of the trained model into the corresponding binary file and store it in DDR3. FPGA exchanges data with DDR3 through AXI bus. The data in DDR3 first enters the input buffer area including feature map buffer area and weight buffer area. Then perform convolution calculation.

[0144] The storage module design is as follows:

[0145] The data set and weights are stored in DDR3 and data is exchanged through the AXI bus. By integrating the AXI bus controller in the FPGA design, high-speed data transmission between the FPGA and the DDR3 memory can be achieved. In this case, the DDR3 controller will be connected to the AXI bus controller, enabling the FPGA to read and write to the DDR3 memory through the AXI bus. Using the AXI bus for data exchange has many advantages, including high performance, low latency, and flexibility. At the same time, the AXI bus also provides a standardized interface that can be easily integrated with other AXI-compatible IP cores, thereby simplifying the system design and development process.

[0146] Step 8: Design multiple IPs for accelerated operations, mainly PWConv: accelerating point-wise convolution, DWConv: accelerating depth-wise convolution, conv: accelerating the convolution operation of the first layer of the network, and fc: accelerating the global average pooling layer and the fully connected layer. The DW convolution operation is divided into three stages: loading data, calculating convolution, and writing back the results. In order to cover the data transmission time, a ping-pong (double buffering) operation is performed. The PW convolution operation includes increasing the interface bit width, increasing the burst transmission length, and increasing the number of interfaces. Finally, block design is performed in vivado. Comprehensive implementation generates bitstream.

[0147] Step 9: Send the result of convolution calculation to the result buffer and then to the result processing module, perform residual addition, pooling, ReLu and quantization operations on it in sequence. Finally, send it to the output buffer. It is transmitted to the computer through the AXI bus for verification.

[0148] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A hardware accelerated embedded system design method for face recognition, characterized in that: The method comprises the following steps: S1 builds a face dataset; S2 builds and improves the DETR target detection model; S2.1 builds a lightweight feature extraction network MobileNetV2 as the backbone network, and uses the inverse residual module to make the feature map undergo convolution dimension increase and final convolution dimension reduction. First, use 1x1 convolution to increase the dimension of the input feature map by a multiple of t=6, then activate it with BN+RELU6 and perform deep convolution processing, activate it again with BN+RELU6, and finally reduce the dimension through 1x1 convolution. A linear transformation activation function is added at the end of the process. S2.2 uses HiLo-Attention to replace the self-attention in the original encoder layer, uses non-overlapping window division in Hi-Fi, and splices the output of high-frequency attention and the output of low-frequency attention on the channel, and outputs the result to the subsequent cross-scale feature fusion module CCFM with integrated variability; The HiLo-Attention is described in detail as follows: two effective attentions are used to decouple the high / low frequencies in the feature map, the Head is assigned to the high-frequency attention, and the fine-grained high frequency is captured through Local Window Self-Attention; average pooling is applied to each window to obtain the low-frequency signal, and the remaining Head is assigned to Lo-Fi, and the relationship between each query position in the input feature map and the average pooled low-frequency key of each window is modeled to achieve low-frequency attention; The operation steps of high-frequency attention HIFI are as follows: S2.2.1 Set the window size to ws, that is, a window has ws x ws patches, that is, each row and column: H group =H / ws W group =W / ws total_groups=H group *W group Each column has H group Group window, each line has W group Group windows, that is, there are total_groups group windows in the whole picture; S2.2.2 Reshape the input X∈R H×W×C Reshape to: Through three linear layers, we obtain Q, K, and V respectively: Q∈R total_group*ws*ws*h*d =reshape(Q) Q∈R total_group*h*ws*ws*d =reshape(Q) Among them, W q ∈R C×C C = h X d, where h is the number of attention heads and d is the number of channels of each attention head. K and V are processed in the same way as Q, so they have the same shape. S2.2.3 Perform attention operation between Q, K, V: S2.2.4S represents the attention matrix between all patch pairs in each window, which is then weighted summed with V to update the patch representation in each window: Y∈R total_group*ws*ws*h*d =S*V It is the output of high-frequency attention; The low-frequency attention loFI operation steps are as follows: S2.2.5 Lo-Fi first applies average pooling to each window to obtain the input X∈R H*W*C Low frequency signal in: X∈R W*H*C =reshape(X) X_∈R HW / ws*ws*C =reshape(X_) HW / (ws) 2 Represents the number of patches after pooling, HW = 14x14 = 196 patches, pooling window ws×ws = 2×2 = 4 patches, after pooling, the remaining HW / (ws) 2 =196 / 4=49 patches; S2.2.6 The average pooled feature map is projected to keys: K∈R HW / ws*ws*C and values:V∈R HW / ws*ws*C And reshape it using the reshape operation: K∈R HW / (ws*ws)*C =X_W K V∈R HW / (ws*ws)*C =X_W v K∈R h*HW / ws*ws*d =reshape(K) V∈R h*HW / (ws*ws)*d =reshape(V) Where Wk,Wv∈R C*C C = h × d, where h is the number of attention heads and d is the number of channels of each attention head; S2.2.7 Q in Lo-Fi still comes from the original feature map X∈R W*H*C Q∈R W*H*C =XW q Q∈R h*HW*d =reshape(Q) S2.2.8 Then, a standard attention mechanism is applied to capture the rich low-frequency information in the feature map. Note that the complexity of Lo-Fi is significantly reduced due to the reduction in the spatial dimensions of K and V: U represents the attention matrix between the patch and the window, N = HW is the total number of patches, and HW / (ws)2 is the number of patches after pooling; Perform weighted summation with V and update each patch representation: Y L ∈R h*HW*d =UV AND L ∈R H*W*C =reshape(Y L ) Y L ∈R H*W*C is the output of low-frequency attention; S3 is used to train the improved DETR target detection model with the dataset; S4 applies dual-branch pruning technology to the final model BN layer for optimization and introduces a custom model CGVV model. One branch uses dual GSConv layers to optimize model performance; the other branch directly outputs the original feature map through the Ghost bottleneck, uses shortcut connection and outputs the result to VoVGSCSP. The detailed steps are as follows: A custom CGVV model is introduced in the BN layer for pruning operations. The feature map is processed by a dual-branch method. One branch uses two GSConv convolutional layers, and the other uses Ghost bottlenec to output the original feature map. Ghostbottlenec consists of two stacked Ghost modules: the first Ghost module is used as an expansion layer to increase the number of channels; The second Ghost module reduces the number of channels to match the shortcut path. The shortcut is used to connect the input and output of the two Ghost modules. ReLU is not used after the second Ghost module. Batch normalization (BN) and ReLU nonlinear activation are applied to other layers after each layer. The outputs of the two modules are connected and output to VoVGSCSP. The fused feature map is input to the GSConv convolution layer again for output, eliminating the calculation of the BN layer and saving the obtained weight parameters and data set. S5 designs the HLS accelerator module to generate an instruction set loaded onto the FPGA hardware. The HLS accelerator module designs multiple IP cores to accelerate different operations, including: PWConv: accelerated point-wise convolution; DWConv: accelerated depth-wise convolution; conv: speed up the convolution operation of the first layer of the network; fc: accelerates the global average pooling layer and the fully connected layer; The DW convolution operation is divided into three stages: loading data, calculating convolution, and writing back the results. The ping-pong operation masks the data transmission time. PW convolution operations include increasing the interface bit width, increasing the burst transmission length, and increasing the number of interfaces; Finally, block design is performed in Vivado to generate instruction sets that are loaded onto FPGA hardware; S6 stores the trained final model parameters into DDR3, and FPGA connects to DDR3 through the AXI bus controller and performs data exchange; S7 uses pictures for result testing.

2. The method for designing a hardware accelerated embedded system for face recognition according to claim 1, characterized in that: The face dataset in S1 is built based on the CASIS-FaceV5 dataset. The image size is (640, 480), and the validation set uses LFW.

3. The method for designing a hardware accelerated embedded system for face recognition according to claim 1, characterized in that: The output of high-frequency attention and the output of low-frequency attention in S2.2 are concatenated on the channel as follows: The output of the high-frequency attention and the output of the low-frequency attention are concatenated on the channel and restored to the same shape as the input: Y∈R H*W*2C =Concat(And h ,AND L ) Y∈R N*2C) =reshape(Y)。 4. The method for designing a hardware accelerated embedded system for face recognition according to claim 1, characterized in that: The S7 test steps include: sending the result obtained by convolution calculation to the result buffer and then to the result processing module, performing residual addition, pooling operation, ReLu and quantization operations on it in turn, and finally sending it to the output buffer and transmitting it to the computer through the AXI bus for verification.

Images