An embedded fingerprint recognition method based on lightweight network

Abstract

The present invention discloses an embedded fingerprint recognition method based on a lightweight network, belonging to the field of computer networks, and comprising the following steps: S1: obtaining fingerprint sample data, feeding the sample data into a fingerprint recognition network model composed of a depthwise separable convolutional structure, a grouped convolutional structure, and an asymmetric convolutional structure for iterative training, thereby obtaining a secondary fingerprint recognition network model; the secondary fingerprint recognition network model is a lightweight convolutional neural network model; S2: quantizing and compressing the iteratively trained fingerprint recognition network model to a target memory size, thereby obtaining a target model; S3: deploying the target model to an embedded terminal, where the target model is used by the embedded terminal to perform forward prediction of a target fingerprint, extract fingerprint graphic features, and identify the target fingerprint. By using a lightweight convolutional neural network model and quantizing the trained model, the present invention significantly reduces the size of the model, reduces computational effort, and reduces resource consumption, making the fingerprint recognition network model suitable for embedded platforms.

Description

Technical Field

[0001] The present invention relates to the technical field of fingerprint recognition, and in particular to an embedded fingerprint recognition method based on a lightweight network. Background Art

[0002] Fingerprint recognition technology, due to its uniqueness and stability, has been widely used in our daily lives, such as fingerprint attendance, fingerprint door locks, fingerprint luggage, mobile phone fingerprint login, and mobile phone fingerprint payment. Its applications can be seen everywhere. As fingerprint recognition technology becomes more widely used, the market has placed higher requirements on the speed, performance, and fingerprint collection area of ​​fingerprint recognition technology.

[0003] Fingerprint recognition technology typically builds and trains a deep neural network based on ResNet. However, this type of neural network requires a large model and large computational load, making it difficult to apply to micro-embedded devices and limiting its scope of use. Summary of the Invention

[0004] The technical problem of the present invention is to provide an embedded fingerprint recognition method based on a lightweight network, reduce the fingerprint recognition network model, and apply fingerprint recognition technology to the embedded end while ensuring fingerprint recognition accuracy and fingerprint recognition rate.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An embedded fingerprint recognition method based on a lightweight network includes the following steps: S1 obtains fingerprint sample data, and feeds the sample data into a fingerprint recognition network model composed of a depth-separable convolution structure, a group convolution structure, and an asymmetric convolution structure for iterative training to obtain a secondary fingerprint recognition network model; wherein the secondary fingerprint recognition network model is a lightweight convolutional neural network model; S2 quantizes and compresses the iteratively trained fingerprint recognition network model to a target memory size to obtain a target model; S3 deploys the target model to an embedded end, and the target model is used by the embedded end to perform forward prediction of a target fingerprint, extract fingerprint graphic features, and recognize the target fingerprint.

[0007] The fingerprint recognition network model includes a depth-wise separable convolution structure, an intermediate structure, a global depth-wise separable convolution structure and a grouped convolution structure; the input data enters the intermediate structure through the depth-wise separable convolution structure, enters the global depth-wise separable convolution structure through the intermediate structure, enters the grouped convolution structure through the global depth-wise separable convolution structure and is output; the intermediate structure includes at least one lightweight network module; the convolution kernel of the grouped convolution structure is 1*1.

[0008] The lightweight network module includes: two 1*1 group convolution structures, an n*1 depth-separable convolution structure and a 1*n depth-separable convolution structure; the input data enters the n*1 depth-separable convolution structure through a 1*1 group convolution structure, enters the 1*n depth-separable convolution structure through the n*1 depth-separable convolution structure, enters another 1*1 group convolution structure through the 1*n depth-separable convolution structure and is output.

[0009] A BN normalization processing module is set after each convolution structure of the fingerprint recognition network model.

[0010] S1 includes: S11 organizes and classifies the acquired fingerprint sample data for storage, wherein the classification storage is performed in the following manner: fingerprint images corresponding to the same finger are stored in the same folder, and fingerprint images corresponding to different fingers are stored in different folders; S12 pre-processes the fingerprint sample data and feeds it into the fingerprint recognition network model, and trains the fingerprint recognition network model through the mxnet framework; wherein the pre-processing includes: adjusting the size of the fingerprint sample data according to requirements, and converting the fingerprint sample data into a data format supported by the mxnet framework training; S13 adjusts the hyperparameters of the fingerprint recognition network model, and updates the parameters of each convolutional layer in the fingerprint recognition network model until the loss value stabilizes, thereby obtaining a secondary fingerprint recognition network model.

[0011] S2 includes: in the secondary fingerprint recognition network model, the 1*1 group convolution structure jointly calculates the quantization parameters for the feature maps in the same group; wherein, the group refers to the channel grouping in the secondary fingerprint recognition network model; in the secondary fingerprint recognition network model, the convolutions other than the 1*1 group convolution structure calculate the quantization parameters for the feature maps separately; the feature map refers to the output value of the secondary fingerprint recognition network model; the target model is obtained by quantizing and compressing the secondary fingerprint recognition network model to the target memory size.

[0012] S3 includes: S31 the embedded end reads and saves the parameters of the target model; S32 inputs the target fingerprint from the embedded end, and uses the hardware computing device to perform forward calculations on each layer of the lightweight convolutional neural network on the target fingerprint to obtain the fingerprint graphic features of the target fingerprint; S33 calculates the distance between the fingerprint graphic features of the target fingerprint and the fingerprint graphic features in the fingerprint database. If the distance between the fingerprint graphic features of the target fingerprint and a fingerprint graphic feature in the database is less than a preset threshold, it is determined that the target fingerprint and the fingerprint in the database are from the same finger. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is an architectural diagram of the fingerprint recognition network model provided by the present invention;

[0014] Figure 2 This is an architectural diagram of the lightweight network block provided by the present invention;

[0015] Figure 3 This is a flow chart of an embedded fingerprint recognition method based on a lightweight network provided by the present invention. DETAILED DESCRIPTION

[0016] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0017] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0018] Since the existing deep learning models used in the field of fingerprint recognition technology are large in size and difficult to apply to resource-constrained embedded platforms, the present invention provides an embedded fingerprint recognition method based on a lightweight network, such as Figure 3 As shown, first obtain fingerprint sample data and select a large number of small-area fingerprint images collected by the sensor for training. Multiple images of the same finger should be placed in the same folder, while images of different fingers should be placed in different folders. The number of fingerprint image samples in each folder should be similar, and should cover as many situations as possible, such as images collected from dry fingers, normal fingers, and wet fingers, to avoid overfitting and poor model generalization. Furthermore, to improve model performance, different samples of the same finger should have a certain degree of rotation and overlap.

[0019] After preprocessing, the fingerprint sample data is fed into the fingerprint recognition network model for training. The fingerprint recognition network model is a lightweight convolutional neural network model. To reduce model size and computational complexity, the fingerprint recognition network model no longer uses standard convolution. Instead, it uses depthwise separable convolution, intermediate convolution, grouped convolution, and asymmetric convolution. After each convolution, the output distribution is forced back to a standard normal distribution with a mean of 0 and a variance of 1 through BN normalization. The output is then passed through the Relu6 activation function to increase the network's nonlinearity.

[0020] Specifically, if Figure 1 and Figure 2 As shown, the fingerprint recognition network model includes a depth-separable convolution structure, an intermediate structure, a global depth-separable convolution structure and a group convolution structure. The convolution kernel of the group convolution network is 1*1. The intermediate structure includes at least one lightweight network module. The lightweight network module includes: two 1*1 group convolution structures, an n*1 depth-separable convolution structure and a 1*n depth-separable convolution structure.

[0021] Depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution is responsible for filtering, with a size of h*w*1, a total of c1, and acts on each input channel; pointwise convolution is responsible for transforming channels, with a size of 1*1*c1, a total of c2, and acts on the output feature map of the depthwise convolution. The number of parameters for depthwise convolution is h*w*1*c1, while the number of parameters for pointwise convolution is 1*1*c1*c2, which is 1 / c2 + 1 / (h*w) of the standard convolution. Depthwise convolution is a grouped convolution with g=c1=c2. The difference is that the g groups of results are not directly spliced ​​together, but instead are spliced ​​together through pointwise convolution.

[0022] Grouped convolution divides the input feature maps into g groups based on the number of channels. Each group of input feature maps has a size of H*W*(c1 / g), the corresponding convolution kernel size is h*w*(c1 / g), and the size of each output feature map is H*W*(c2 / g). The g groups of results are concatenated to obtain a final output feature map of size H*W*c2. The parameter count for grouped convolution is: h*w*(c1 / g)*(c2 / g)*g = h*w*c1*c2*(1 / g), which is 1 / g of the parameter count for standard convolution. To further improve network performance, feature maps of different channels from different groups are shuffled before being passed to the next layer. This ensures that each feature map input to the next layer comes from a different group of the previous layer's output. This shuffling of feature maps increases information flow between groups, ensuring the network has stronger representational capabilities and ultimately improving network performance.

[0023] Global depth-wise separable convolution is a special case of depth-wise separable convolution. When the size of the convolution kernel is equal to the size of the feature map, the depth-wise separable convolution is called global depth-wise separable convolution.

[0024] Asymmetric convolution decomposes the n*n convolution into a cascade of n*1 convolution and 1*n convolution, so that the amount of calculation is reduced from n*n multiplication to 2*n multiplication. The larger n is, the more the amount of calculation is reduced.

[0025] In the fingerprint recognition network model, input data enters the depthwise separable convolutional structure for computation and is then output to the intermediate structure. Within the intermediate structure, the input data passes through a 1*1 grouped convolutional structure, enters an n*1 depthwise separable convolutional structure, passes through the n*1 depthwise separable convolutional structure, enters a 1*n depthwise separable convolutional structure, passes through the 1*n depthwise separable convolutional structure, enters another 1*1 grouped convolutional structure, and is output to the global depthwise separable convolutional structure. From the global depthwise separable convolutional structure, the input data enters the 1*1 grouped convolutional structure of the output layer, which outputs a 128-dimensional feature map. The structure of this fingerprint recognition network model is not fixed and can be adjusted based on actual conditions. For example, the size of the convolution kernel, the number of channels, and the number of lightweight network modules are all adjustable. This example is just one example.

[0026] The present invention uses the mxnet framework to train the fingerprint recognition network model. Therefore, before inputting the fingerprint sample data into the fingerprint recognition network model, it is necessary to adjust the size of the fingerprint sample data to the required fixed size, and then convert the fingerprint sample data into the data format required for mxnet framework training, that is, preprocess the fingerprint sample data.

[0027] The team then continuously adjusted hyperparameters of the fingerprint recognition network model, such as the learning rate, batch size, and adjustable parameters of the loss function, and updated the parameters of the convolutional layer to continuously reduce the loss value. When the loss value stabilized and no longer decreased, training was terminated, resulting in a secondary fingerprint recognition network model suitable for fingerprint image recognition. This secondary fingerprint recognition network model is less than 1MB in size, making it a lightweight model.

[0028] In order to further compress the fingerprint recognition network model, reduce the amount of calculation, and improve the forward reasoning speed, the secondary fingerprint recognition network model is quantized, that is, the parameters in the secondary fingerprint recognition network model are quantized from 32 bits to 8 bits, and the secondary fingerprint recognition network model is compressed to one-fourth of the original size. The final quantization bit number here is not fixed and can be adjusted according to actual conditions. The specific principle of model quantization is that model quantization bridges floating point and fixed point, establishes an effective data mapping relationship, and obtains better benefits at the cost of a smaller loss of precision. The floating point to fixed point formula is as follows: Q = R / S + Z

[0029] Where R represents the input floating-point data, Q represents the quantized fixed-point data, Z represents the value of Zero Point, and S represents the value of Scale. We can determine the mapping relationship based on these two parameters. The specific calculation method of S and Z is as follows: S = (Rmax-Rmin) / (Qmax-Qmin)

[0030] Z=Qmax-Rmax / S

[0031] Where: Rmax represents the maximum value of the input floating-point data, Rmin represents the minimum value of the input floating-point data, Qmax represents the maximum fixed-point value, and Qmin represents the minimum fixed-point value. In particular, because some feature maps may exhibit singular values ​​that deviate far from the original value during the quantization process, resulting in some values ​​with particularly large max values ​​or particularly small min values, the present invention calculates S and Z using a histogram truncation method. This method discards a certain percentage of data by truncating the histogram, for example, using the value at the first 1% cutoff point as the max value and the value at the next 1% cutoff point as the min value to calculate the quantization parameter.

[0032] In order to reduce the accuracy loss caused by quantization, the present invention uses a grouped convolution structure to jointly calculate the quantization parameters for the feature maps in the same group. Grouping refers to the channel grouping of the secondary fingerprint recognition model. Other convolution structures calculate the quantization parameters separately for the feature maps. The feature maps are the 128-dimensional features output by the secondary fingerprint recognition network model.

[0033] Specifically, the parameters in each group or feature map are read, their maximum and minimum values ​​are counted, S and Z are calculated, and the parameters are quantized to a range between -128 and 127 according to the above formula. Similarly, the activation values ​​are quantized. Since the range of the activation layer is not easy to obtain in advance, it needs to be calculated during the forward reasoning process or roughly predicted based on the model. After quantization, the 32-bit model parameters are converted to 8 bits, and the model is compressed to one-quarter of the previous size. After quantization, the model size is controlled within 256KB. If further compression of the model is required, the number of bits of the quantized model parameters can be reduced to less than 8 bits to obtain the target model.

[0034] Finally, the target model is deployed to the embedded platform, and the target model parameters are saved as a file. The file is then read using C language to read the model parameters. After inputting the target fingerprint, the C language is used to implement the calculations of each layer of the lightweight convolutional neural network until the entire forward prediction process is completed, outputting the 128-dimensional features of the target fingerprint. Because embedded platforms cannot use forward reasoning acceleration frameworks, the present invention introduces hardware acceleration during forward reasoning, performing some of the calculations in hardware, significantly improving reasoning speed. Furthermore, by calculating the distance between the 128-dimensional fingerprint features of the target fingerprint and the fingerprint features in the fingerprint database, it is possible to determine whether the two features are from the same finger. If the distance between the fingerprint features of the target fingerprint and a fingerprint feature in the database is less than a preset threshold, the target fingerprint and the fingerprint in the database are determined to be from the same finger; if the distance between the features is greater than the preset threshold, the two features are from different fingers. Testing has shown that the recognition accuracy and speed meet practical application requirements.

[0035] The model is deployed entirely in C, performing model parameter reading, network layer calculations, and the entire forward reasoning process. It does not rely on any forward reasoning acceleration framework. To increase reasoning speed, some calculations are performed in hardware. The introduction of hardware acceleration ensures that recognition accuracy and speed meet practical application requirements even under resource constraints.

[0036] Compared with the existing deep learning models applied in the field of fingerprint recognition, the present invention adopts a self-designed lightweight fingerprint recognition network model for feature extraction, and uses depth-separable convolution, group convolution and asymmetric convolution to design the network instead of standard convolution. The network has fewer layers, smaller model parameters and computational complexity, lower resource consumption, and is more suitable for deployment on the embedded side. Quantizing the trained model greatly reduces the computational complexity, reduces the model size, and greatly shortens the forward reasoning time. The quantized model is particularly suitable for deployment on resource-constrained embedded platforms. Moreover, when the model is deployed on a resource-constrained embedded platform, it does not rely on other forward reasoning acceleration frameworks, and can realize model reading and forward reasoning. Combined with the platform's own hardware acceleration, it can meet real-time requirements while ensuring performance. It is suitable for any occasion where a fingerprint recognition module can be applied for recognition, such as fingerprint attendance, fingerprint payment and fingerprint door locks. It can be seen that the present invention has a broad application prospect.

[0037] The above describes the preferred embodiments of the present invention; it should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can make many possible changes and modifications without departing from the technical solution of the present invention, or modify them into equivalent embodiments with equivalent changes, which does not affect the essential content of the present invention; therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solution of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. An embedded fingerprint recognition method based on a lightweight network, characterized in that: The following steps are involved: S1 obtains fingerprint sample data and feeds the sample data into a fingerprint recognition network model composed of a depthwise separable convolution structure, a grouped convolution structure, and an asymmetric convolution structure for iterative training to obtain a secondary fingerprint recognition network model; wherein the secondary fingerprint recognition network model is a lightweight convolutional neural network model; S2 quantizes and compresses the fingerprint recognition network model after iterative training to a target memory size to obtain a target model; S3 deploys the target model to the embedded end, where the target model is used by the embedded end to perform forward prediction on the target fingerprint, extract fingerprint graphic features, and identify the target fingerprint; The fingerprint recognition network model includes a depth-separable convolution structure, an intermediate structure, a global depth-separable convolution structure and a grouped convolution structure; Input data enters the intermediate structure through the depth-wise separable convolution structure, enters the global depth-wise separable convolution structure through the intermediate structure, enters the grouped convolution structure through the global depth-wise separable convolution structure and is output; The intermediate structure includes at least one lightweight network module; The convolution kernel of the grouped convolution structure is 1*1; The lightweight network module includes: two 1*1 grouped convolution structures, an n*1 depth-separable convolution structure and a 1*n depth-separable convolution structure; The input data enters the n*1 depth-separable convolution structure through one of the 1*1 grouped convolution structures, enters the 1*n depth-separable convolution structure through the n*1 depth-separable convolution structure, enters another 1*1 grouped convolution structure through the 1*n depth-separable convolution structure and is output; A BN normalization processing module is provided after each convolution structure of the fingerprint recognition network model.

2. The method of embedded fingerprint recognition based on a lightweight network according to claim 1, characterized in that: Said S1 comprises: S11 organizes and classifies the acquired fingerprint sample data, wherein the classification storage adopts the following method: fingerprint images corresponding to the same finger are stored in the same folder, and fingerprint images corresponding to different fingers are stored in different folders; S12 pre-processes the fingerprint sample data and then feeds it into the fingerprint recognition network model, and trains the fingerprint recognition network model through the mxnet framework; wherein the pre-processing includes: adjusting the size of the fingerprint sample data according to requirements, and converting the fingerprint sample data into a data format supported by the mxnet framework training; S13 adjusts the hyperparameters of the fingerprint recognition network model and updates the parameters of each convolutional layer in the fingerprint recognition network model until the loss value is stable, thereby obtaining a secondary fingerprint recognition network model.

3. The method of embedded fingerprint recognition based on a lightweight network according to claim 2, characterized in that: The S2 includes: In the secondary fingerprint recognition network model, the 1*1 group convolution structure jointly calculates the quantization parameters of the feature maps in the same group; wherein the group refers to the channel group in the secondary fingerprint recognition network model; In the secondary fingerprint recognition network model, quantization parameters are calculated separately for all convolution feature maps except for the 1*1 group convolution structure; The feature map refers to the output value of the secondary fingerprint recognition network model; The target model is obtained by quantizing and compressing the secondary fingerprint recognition network model to the target memory size.

4. The method of embedded fingerprint recognition based on a lightweight network according to claim 3, characterized in that: The S3 includes: S31 The embedded end reads and saves the parameters of the target model; S32 inputs the target fingerprint from the embedded end, and uses the hardware computing device to perform forward calculations on each layer of the lightweight convolutional neural network on the target fingerprint to obtain the fingerprint graphic features of the target fingerprint; S33 calculates the distance between the fingerprint graphic feature of the target fingerprint and the fingerprint graphic feature in the fingerprint database. If the distance between the fingerprint graphic feature of the target fingerprint and a fingerprint graphic feature in the database is less than a preset threshold, it is determined that the target fingerprint and the fingerprint in the database are from the same finger.

Images