A fast fingerprint retrieval method based on self-supervised neural network

Abstract

A fingerprint fast retrieval method based on a self-supervised neural network, constructing a fingerprint feature extractor model and pre-training; the main body of the model adopts a structure of a linear projection layer plus a Transformer encoder; the pre-trained fingerprint feature extractor model is transferred to a fingerprint image and trained again in a self-supervised learning manner; a fingerprint image reconstructor model is constructed, and then connected with the fingerprint feature extractor model obtained by self-supervised learning, and the fingerprint feature extractor model is further improved in its ability to extract fingerprint features by using a fingerprint reconstruction task; the fingerprint feature extractor model trained by the fingerprint image reconstruction task is used to extract features from fingerprint images in a fingerprint library one by one, and a fingerprint index library is established; a candidate fingerprint sequence is retrieved from the fingerprint index library using a query fingerprint. The present invention can directly use the original fingerprint image for automatic feature extraction in large-scale fingerprint retrieval, and can improve the accuracy and efficiency of fingerprint retrieval.

Description

Technical Field

[0001] The invention belongs to the technical field of biometric identification, relates to image processing and fingerprint retrieval, and particularly to a fingerprint rapid retrieval method based on a self-supervisory neural network. Background Art

[0002] Fingerprint is a widely used biometric feature in the field of biometric identification technology. Due to the excellent characteristics of fingerprints such as uniqueness and invariance, automatic fingerprint recognition systems have been widely used in the field of identity authentication. However, as the application of automatic fingerprint recognition technology becomes wider and wider, and the population covered is increasing, the scale of fingerprint databases is also expanding. The capacity of some fingerprint databases even reaches the billion level. The challenge that follows is to find fingerprints with high similarity to the query fingerprint from hundreds of millions of fingerprints, that is, the retrieval problem of large-scale fingerprint images. Taking the fingerprint system of the police system as an example, if every citizen has to register and store fingerprints in its database, the amount of data will be calculated in billions. Such a huge number of fingerprints will inevitably reduce the speed of fingerprint retrieval. Therefore, designing an efficient and accurate fingerprint retrieval algorithm for large-scale fingerprint databases is of great significance in the field of identity authentication.

[0003] Traditional fingerprint image retrieval methods are based on manually designed feature extraction algorithms, which extract features from fingerprint images and then use the features to retrieve and compare fingerprints. The more commonly used manually designed fingerprint features are "minutiae" features. Minutiae points are generally stored in a ternary format: (x, y, θ), where x is the horizontal coordinate of the minutiae point, y is the vertical coordinate of the minutiae point, and θ is the direction of the grain line at the minutiae point. This type of manually designed feature will have limitations due to subjective factors such as the designer's prior knowledge, thereby ignoring some fingerprint features, and these ignored features are likely to be features that will help fingerprint retrieval. Summary of the invention

[0004] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a fingerprint fast retrieval method based on a self-supervised neural network, so as to directly use the original fingerprint image for automatic feature extraction in large-scale fingerprint retrieval and improve the accuracy and efficiency of fingerprint retrieval.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A fingerprint fast retrieval method based on a self-supervised neural network comprises the following steps:

[0007] Step 1, construct a fingerprint feature extractor model, and then pre-train it on the ImageNet dataset; the main body of the fingerprint feature extractor model adopts a linear projection layer plus a Transformer encoder structure, and the output of the first position of the Transformer encoder is used as the input of the fully connected layer classifier; during pre-training, each image in the ImageNet dataset is cut into image blocks, which are input into the linear projection layer as a time series, and then the loss is calculated according to the output category of the fully connected layer classifier and the true category of the image, and the model parameters are updated;

[0008] Step 2: Transfer the fingerprint feature extractor model pre-trained in step 1 to the fingerprint image, and retrain it on the fingerprint image using self-supervised learning. During training, each fingerprint image in the dataset is cut into image blocks and input into the linear projection layer as a time series.

[0009] Step 3: construct a fingerprint image reconstructor model, and then connect it with the fingerprint feature extractor model obtained by self-supervised learning in step 2, and use the fingerprint reconstruction task to further improve the fingerprint feature extraction capability of the fingerprint feature extractor model;

[0010] Step 4, using the fingerprint feature extractor model trained by the fingerprint image reconstruction task in step 3, extract features from the fingerprint images in the fingerprint library one by one, and use the extracted feature vectors to establish a fingerprint index library;

[0011] Step 5: Use the query fingerprint to perform a quick search in the fingerprint index library established in step 4 to obtain a candidate fingerprint sequence.

[0012] Preferably, in step 1, during pre-training, collect and organize image data sets with category labels on ImageNet; cut each image in ImageNet into n 16×16 pixel image blocks, sort them from top to bottom and from left to right, input these n image blocks into the linear projection layer in sequence, and output n token vectors; add special characters before these n tokens <cls>Token, a total of n+1 tokens are input to the Transformer decoder; Transformer <cls>The output of the position is used as the input of the fully connected layer classifier, which represents the semantic expression of the entire image.

[0013] Preferably, in step 2, the self-supervised learning has two tasks:

[0014] a. Randomly select an image block from the cut fingerprint image, randomly rotate the image block, record the rotation angle, and then let the fingerprint feature extractor model predict the rotation angle of the image block;

[0015] b. Randomly select an image block from the cut fingerprint image, cover the image block, and then let the fingerprint feature extractor model predict and generate the image block based on other areas of the fingerprint image;

[0016] Preferably, according to task a of self-supervised learning, the fingerprint image is processed, a window area in the fingerprint image is randomly selected and randomly rotated by a certain angle, and a label of the rotation angle is generated; each obtained fingerprint image is cut into n 16×16 pixel image blocks; the n image blocks are sorted from top to bottom and from left to right, the sorted n image block sequences are input into the linear projection layer, and n token vectors are output; special characters are added before the n tokens <rot>The token represents the rotation angle, and a total of n+1 tokens are input into the Transformer decoder; the output of the Transformer <rot>The output angle corresponding to the position and the label of the generated rotation angle are used to calculate the loss, and then the parameters are updated through backpropagation;

[0017] According to the task b of self-supervised learning, the fingerprint image is processed and each original fingerprint image is cut into n 16×16 pixel image blocks; these n image blocks are sorted from top to bottom and from left to right, and the sorted n image block sequences are input into the linear projection layer to output n token vectors; special characters are added before these n tokens <cov>The token indicates that the current task has been switched to rotation prediction. A random token is selected from these n tokens. <mask>The character is replaced, and the image block at that position is used as the label, and then a total of n+1 tokens are input into the Transformer decoder; the output of the Transformer <mask>The output corresponding to the position and the obtained label calculate the loss, and then backpropagate to update the parameters.

[0018] Preferably, in step 3, the fingerprint reconstruction task is to cut the fingerprint image into image blocks and input them into the fingerprint feature extractor model as a time series. According to the feature vector extracted by the fingerprint feature extractor model, the fingerprint image reconstructor model will output an image, calculate the structural similarity SSIM between the image and the original fingerprint image, judge their similarity, and then update the model parameters.

[0019] Preferably, the fingerprint image reconstructor model adopts a fully connected layer, and the output corresponding to the first token of the fingerprint feature extractor model is connected to the fingerprint image reconstructor model; the fingerprint reconstruction task is to cut each original fingerprint image into n 16×16 pixel image blocks, and then input them into the fingerprint feature extractor model, and use the extracted image semantic expression as a feature vector, which is input into the fingerprint image reconstructor model to output a fingerprint image; the structural similarity index SSIM is calculated between the output fingerprint image and the original image, and the parameters of the fingerprint feature extractor model and the fingerprint image reconstructor model are updated according to the SSIM, so that the output of the fingerprint image reconstructor model is closer and closer to the original fingerprint image.

[0020] Preferably, the structural similarity SSIM calculation formula is as follows:

[0021]

[0022] where μ x is the mean value of x, μ y is the mean value of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y, c 1 =(k 1 L) 2 , c 2 =(k 2 L) 2 is a constant used to maintain stability, L is the dynamic range of pixel values, and k 1 =0.01, k 2 =0.03, the range of structural similarity SSIM is -1 to 1. When two pictures are exactly the same, the value of SSIM is equal to 1. x and y represent the two pictures to be compared for similarity.

[0023] Preferably, in step 4, each fingerprint in the fingerprint library corresponds to a list of detail points of unfixed length, each fingerprint image in the entire fingerprint library is input into the fingerprint feature extractor, and the output of the first token position of the fingerprint feature extractor is used as the feature vector of the fingerprint, the feature vector of the fingerprint is used as the key, and the list of detail points corresponding to the fingerprint is used as the value, and a fingerprint index library is constructed offline; the dimension of the feature vector extracted from each fingerprint is n, and the Euclidean distance d is calculated between the n-dimensional feature vector and the coordinate origin O={0,0,…,0} in the n-dimensional space, and then sorted from small to large to obtain a fingerprint index library S, d, key and value together constitute an element in the fingerprint index library S, so that in the retrieval stage, a part of the fingerprints can be quickly eliminated according to the size of the distance.

[0024] Preferably, in step 5, a first screening is performed by calculating the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the coordinate origin in the n-dimensional space, and then the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the n-dimensional feature vectors of other fingerprints is calculated, and the candidate fingerprint sequence is finally screened out according to the threshold U.

[0025] Preferably, in step 5, the candidate fingerprint sequences are quickly retrieved using the fingerprint index library, and the steps are as follows:

[0026] (1) When a query fingerprint is input, the query fingerprint is input into the fingerprint feature extractor model to obtain the feature vector A of the fingerprint. 1 ,a 2 ,…,a n };

[0027] (2) Calculate the distance between the feature vector A and the origin O as d, define U as the threshold, and let R 1 =dU,R 2 =d+U;

[0028] (3) According to R 1 and R 2 The fingerprint index library S is screened by removing the fingerprints whose d is less than R in S. 1 , and then remove the ones whose d is greater than R in S 2 fingerprints;

[0029] (4) Then, the Euclidean distance is calculated one by one between the feature vector A and the remaining fingerprint feature vectors in the fingerprint index library S. If the distance is less than the threshold U, the fingerprint is placed in the candidate sequence, otherwise the next fingerprint is compared.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] First, unlike traditional manually designed fingerprint features, the present invention uses neural networks to automatically learn and extract fingerprint features in a self-supervised learning manner. Two self-supervised learning tasks are designed based on the characteristics of fingerprint images: a. predicting the rotation angle of part of the fingerprint image area; b. predicting the covered fingerprint graphic area.

[0032] Second, the strategy of reconstructing the fingerprint image using the extracted fingerprint feature vector can improve the accuracy of the model in extracting fingerprint features.

[0033] Thirdly, a storage format of fingerprint index library is designed by using fingerprint feature vectors extracted by neural network, and the fingerprint index can be used for fast fingerprint retrieval.

[0034] Fourth, a new fingerprint retrieval strategy was designed, which uses the distance between the fingerprint feature and the origin in n-dimensional space to perform the first screening of fingerprints, and then uses the distance between the query fingerprint and the remaining fingerprints to perform the second screening. These two screening strategies greatly reduce the time required for fingerprint retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the present invention.

[0036] Figure 2 It is the principle diagram of self-supervised task a.

[0037] Figure 3 It is the schematic diagram of the self-supervised task b.

[0038] Figure 4 This is the principle diagram of the fingerprint image reconstruction task.

[0039] Figure 5 This is the storage structure diagram of the fingerprint index library.

[0040] Figure 6 This is the retrieval principle diagram of fingerprint two-dimensional feature vector. DETAILED DESCRIPTION

[0041] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0042] The present invention is a fingerprint fast retrieval method based on a self-supervised neural network, in which there are mainly two neural network models. The first model is used to extract the feature vector of the fingerprint image, which is called the fingerprint feature extractor model, and the extracted feature vector is used as the index for subsequent fingerprint retrieval. The function of the second model is to reconstruct the fingerprint image according to the feature vector obtained by the fingerprint feature extractor model, which is called the fingerprint image reconstructor model, and its purpose is to improve the fingerprint feature extractor's ability to express fingerprint image features. After the two models are jointly trained, the fingerprint feature extractor model has the best feature extraction ability for the fingerprint, and then an index is established for the fingerprint database in an offline manner based on the extracted fingerprint feature vector, so that when the query fingerprint is input, a candidate fingerprint sequence similar to it can be quickly found, and then used for careful comparison one by one. Thereby, fast and accurate retrieval of large-scale fingerprints is achieved.

[0043] like Figure 1 As shown, the present invention mainly comprises the following steps:

[0044] Step 1: Construction and training of fingerprint feature extractor model.

[0045] The main body of the fingerprint feature extractor model in the present invention adopts the structure of a linear projection layer plus a Transformer encoder. During pre-training, a fully connected layer classifier is connected after the Transformer encoder, and the output of the first position of the Transformer encoder is used as the input of the fully connected layer classifier. The pre-training data set uses the ImageNet data set. During pre-training, each picture in the ImageNet data set is cut into several image blocks, which are input into the linear projection layer as a time series, and then the loss is calculated according to the output category of the fully connected layer classifier and the true category of the picture, and the model parameters are updated.

[0046] More specifically, during pre-training:

[0047] 1) First collect and organize the image dataset with category labels in ImageNet.

[0048] 2) In view of the fact that the Transformer encoder can only process sequence data, each image in ImageNet is cut into n 16×16 pixel image blocks, sorted from top to bottom and from left to right, and these n image blocks are input into the linear projection layer in sequence, and n token vectors are output.

[0049] 3) Add special characters before these n tokens <cls>Token, a total of n+1 tokens are input to the Transformer decoder.

[0050] 4) Transformer <cls>The output of the position is used as the input of the fully connected layer classifier, which represents the semantic expression of the entire image.

[0051] 5) Pre-train the fingerprint feature extractor model on the ImageNet dataset so that the model can initially characterize the features of the image.

[0052] Step 2: Model migration.

[0053] The fingerprint feature extractor model pre-trained in step 1 is transferred to the fingerprint image, and re-trained on the fingerprint image using self-supervised learning. Similarly, during training, each fingerprint image in the dataset is cut into image blocks and input into the linear projection layer as a time series.

[0054] For example, the self-supervised learning in this step has two tasks:

[0055] a. Reference Figure 2 , randomly select an image block from the cut fingerprint image, specifically, randomly select a window with a length and width of h pixels, and the image area covered by the window is an image block. Randomly rotate the image block and record the rotation angle, and then let the fingerprint feature extractor model predict the rotation angle of the image block.

[0056] b. Reference Figure 3 , randomly select an image block from the cut fingerprint image, cover the image block, and then let the fingerprint feature extractor model predict and generate the image block based on other areas of the fingerprint image.

[0057] More specifically, the process of processing the fingerprint image according to the task a of self-supervised learning is:

[0058] 1) Randomly select a window area in the fingerprint image and randomly rotate it by a certain angle to generate a label of the rotation angle.

[0059] 2) Cut each fingerprint image into n 16×16 pixel image blocks.

[0060] 3) Sort the n image blocks in order from top to bottom and from left to right, input the sorted n image block sequence into the linear projection layer, and output n token vectors.

[0061] 4) Add special characters before these n tokens <rot>The token represents the rotation angle, and a total of n+1 tokens are input into the Transformer decoder.

[0062] 5) Transformer output <rot>The output angle corresponding to the position and the label of the generated rotation angle are used to calculate the loss, and then the parameters are updated by backpropagation.

[0063] The process of processing fingerprint images according to task b of self-supervised learning is:

[0064] 1) Cut each original fingerprint image into n 16×16 pixel image blocks.

[0065] 2) Sort the n image blocks in order from top to bottom and from left to right, input the sorted n image block sequence into the linear projection layer, and output n token vectors.

[0066] 3) Add special characters before these n tokens <cov>The token indicates that the current task has been switched to rotation prediction. A random token is selected from these n tokens. <mask>The character is replaced and the image block at that position is used as the label, and then a total of n+1 tokens are input into the Transformer decoder.

[0067] 4) Transformer output <mask>The output corresponding to the position and the obtained label calculate the loss, and then backpropagate to update the parameters.

[0068] Step 3: Construction of the fingerprint image reconstructor model and its association with the fingerprint feature extractor model.

[0069] In the present invention, the structure of the fingerprint image reconstructor model is a fully connected layer, and the fingerprint image is reconstructed as follows Figure 4 As shown, the fingerprint image reconstructor model is connected to the fingerprint feature extractor model obtained by self-supervised learning in step 2. Specifically, the output corresponding to the first token of the fingerprint feature extractor model obtained in step 2 is connected to the fingerprint image reconstructor model. Thus, the fingerprint reconstruction task can be used to further improve the fingerprint feature extraction capability of the fingerprint feature extractor model.

[0070] The fingerprint reconstruction task in this step is to cut the fingerprint image into image blocks and input them into the fingerprint feature extractor model as a time series. According to the feature vector extracted by the fingerprint feature extractor model, the fingerprint image reconstructor model will output an image, calculate the structural similarity SSIM between the image and the original fingerprint image, determine their similarity, and then update the model parameters.

[0071] More specifically, in the fingerprint reconstruction task of this step:

[0072] 1) Cut each original fingerprint image into n 16×16 pixel image blocks, and then input them into the fingerprint feature extractor model as in step 2. The extracted image semantic expression is used as a feature vector and input into the fingerprint image reconstructor model to output a fingerprint image.

[0073] 2) Calculate the structural similarity index SSIM of the output fingerprint image and the original image, and update the parameters of the fingerprint feature extractor model and the fingerprint image reconstructor model according to SSIM, so that the output of the fingerprint image reconstructor model is closer and closer to the original fingerprint image.

[0074] For example, given two images, their structural similarity SSIM calculation formula is as follows:

[0075]

[0076] where μ x is the mean value of x, μ y is the mean value of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y, c 1 =(k 1 L) 2 , c 2 =(k 2 L) 2 is a constant used to maintain stability, L is the dynamic range of pixel values, and k 1 =0.01, k 2 =0.03, the range of structural similarity SSIM is -1 to 1. When two pictures are exactly the same, the value of SSIM is equal to 1. x and y represent the two pictures to be compared for similarity.

[0077] Step 4: Offline establishment of fingerprint index library.

[0078] In the present invention, the fingerprint feature extractor model trained by the fingerprint image reconstruction task in step 3 is used to extract features from the fingerprint images in the fingerprint library one by one, and the fingerprint index library can be established offline using the extracted feature vectors.

[0079] Furthermore, the dimension of the extracted feature vector is n, and the fingerprints are sorted from small to large based on the Euclidean distance between the n-dimensional feature vector and the coordinate origin in the n-dimensional space, so that some fingerprints can be quickly eliminated according to the size of the distance in the retrieval stage.

[0080] More specifically, the process can be described as follows:

[0081] 1) Each fingerprint image in the fingerprint database has had its detail point features extracted using traditional methods, and each fingerprint corresponds to a detail point list of indefinite length.

[0082] 2) Input each fingerprint image in the entire fingerprint library into the fingerprint feature extractor, and use the output of the first token position of the fingerprint feature extractor as the feature vector of the fingerprint.

[0083] 3) Use the feature vector of the fingerprint as the key and the list of minutiae corresponding to the fingerprint as the value to build the fingerprint index library offline.

[0084] 4) The dimension of the feature vector extracted from each fingerprint is n. The Euclidean distance d is calculated between the n-dimensional feature vector and the coordinate origin O={0,0,…,0} in the n-dimensional space, and then sorted from small to large to obtain the fingerprint index library S. d, key and value together constitute an element in the fingerprint index library S, so that some fingerprints can be quickly eliminated according to the size of the distance in the retrieval stage. The storage structure of the fingerprint index library S is as follows: Figure 5 As shown. The Euclidean distance calculation formula in n-dimensional space is as follows:

[0085]

[0086] where x i With y i They represent the lengths of two points in the i-th dimension in the n-dimensional space.

[0087] Step 5: Use the query fingerprint to perform a quick search on the fingerprint index library established in step 4 to obtain a candidate fingerprint sequence.

[0088] Furthermore, the first screening is performed by calculating the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the coordinate origin in the n-dimensional space, and then the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the n-dimensional feature vectors of other fingerprints is calculated, and the candidate fingerprint sequence is finally screened out according to the threshold U.

[0089] More specifically, the fingerprint index library is used to quickly retrieve candidate fingerprint sequences. The steps are as follows:

[0090] 1) When a query fingerprint is input, the query fingerprint is input into the fingerprint feature extractor model to obtain the feature vector A of the fingerprint = {a 1 ,a 2 ,…,a n };

[0091] 2) Calculate the distance between the feature vector A and the origin O as d, define U as the threshold, and let R 1 =dU,R 2 =d+U;

[0092] 3) According to R 1 and R 2 The fingerprint index library S is screened by removing the fingerprints whose d is less than R in S. 1 , and then remove the ones whose d is greater than R in S 2 fingerprints;

[0093] 4) Then use the feature vector A to calculate the Euclidean distance with the remaining fingerprint feature vectors in the fingerprint index library S one by one. If the distance is less than the threshold U, the fingerprint is put into the candidate sequence, otherwise the next fingerprint is compared.

[0094] 5) The threshold U is a hyperparameter and needs to be adjusted experimentally. The physical meaning of this screening process is to map each fingerprint into an n-dimensional space according to the n-dimensional feature vector of the fingerprint. 1 With R 2 are the radii of the two n-dimensional spheres in the n-dimensional space. Use U to calculate R 1 With R 2 , we can first remove most of the fingerprints that are not very similar in the fingerprint index library S. Then, A={a 1 ,a 2 ,…,a n } is the center of the sphere, and U is the radius to make a small n-dimensional sphere. The fingerprints outside the n-dimensional sphere are removed, and the fingerprints inside the n-dimensional sphere are included in the fingerprint candidate sequence. The schematic diagram of the screening process after reducing the n-dimensional to 2-dimensional is as follows Figure 6 shown. Figure 6 In the figure, the small triangles represent all the fingerprints in the index library, and the small circles represent the query fingerprints. 1 With R 2 Most of the irrelevant fingerprints in the fingerprint database can be quickly eliminated from the remaining fingerprints, and the final candidate fingerprints can be screened out using U.< / mask> < / mask> < / cov> < / rot> < / rot> < / cls> < / cls> < / mask> < / mask> < / cov> < / rot> < / rot> < / cls> < / cls>

Claims

1. A fingerprint fast retrieval method based on self-supervised neural network, characterized in that: The steps include: Step 1, construct a fingerprint feature extractor model, and then pre-train it on the ImageNet dataset; the main body of the fingerprint feature extractor model adopts a linear projection layer plus a Transformer encoder structure, and the output of the first position of the Transformer encoder is used as the input of the fully connected layer classifier; during pre-training, each image in the ImageNet dataset is cut into image blocks, which are input into the linear projection layer as a time series, and then the loss is calculated according to the output category of the fully connected layer classifier and the true category of the image, and the model parameters are updated; Step 2: Transfer the fingerprint feature extractor model pre-trained in step 1 to the fingerprint image, and retrain it on the fingerprint image using self-supervised learning. During training, each fingerprint image in the dataset is cut into image blocks and input into the linear projection layer as a time series. Step 3: construct a fingerprint image reconstructor model, and then connect it with the fingerprint feature extractor model obtained by self-supervised learning in step 2, and use the fingerprint reconstruction task to further improve the fingerprint feature extraction capability of the fingerprint feature extractor model; Step 4, using the fingerprint feature extractor model trained by the fingerprint image reconstruction task in step 3, extract features from the fingerprint images in the fingerprint library one by one, and use the extracted feature vectors to establish a fingerprint index library; Step 5: Use the query fingerprint to perform a quick search in the fingerprint index library established in step 4 to obtain a candidate fingerprint sequence.

2. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1 is characterized in that: In the step 1, during pre-training, collect and organize image data sets with category labels on ImageNet; cut each image in ImageNet into n 16×16 pixel image blocks, sort them from top to bottom and from left to right, input these n image blocks into the linear projection layer in sequence, and output n token vectors; add special characters before these n tokens <cls>Token, a total of n+1 tokens are input to the Transformer decoder; Transformer <cls> The output of the position is used as the input of the fully connected layer classifier, which represents the semantic expression of the entire image.< / cls> < / cls> 3. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1 is characterized in that: In step 2, self-supervised learning has two tasks: a. Randomly select an image block from the cut fingerprint image, randomly rotate the image block, record the rotation angle, and then let the fingerprint feature extractor model predict the rotation angle of the image block; b. Randomly select an image block from the cut fingerprint image, cover the image block, and then let the fingerprint feature extractor model predict and generate the image block based on other areas of the fingerprint image.

4. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 3 is characterized in that: According to task a of self-supervised learning, the fingerprint image is processed, a window area in the fingerprint image is randomly selected and randomly rotated by a certain angle, and a label of the rotation angle is generated; each fingerprint image is cut into n 16×16 pixel image blocks; these n image blocks are sorted from top to bottom and from left to right, and the sorted n image block sequences are input into the linear projection layer, and n token vectors are output; special characters are added before these n tokens <rot>The token represents the rotation angle, and a total of n+1 tokens are input into the Transformer decoder; the output of the Transformer <rot> The output angle corresponding to the position and the label of the generated rotation angle are used to calculate the loss, and then the parameters are updated through backpropagation;< / rot> < / rot> According to the task b of self-supervised learning, the fingerprint image is processed and each original fingerprint image is cut into n 16×16 pixel image blocks; these n image blocks are sorted from top to bottom and from left to right, and the sorted n image block sequences are input into the linear projection layer to output n token vectors; special characters are added before these n tokens <cov>The token indicates that the current task has been switched to rotation prediction. A random token is selected from these n tokens. <mask>The character is replaced, and the image block at that position is used as the label, and then a total of n+1 tokens are input into the Transformer decoder; the output of the Transformer <mask> The output corresponding to the position and the obtained label calculate the loss, and then backpropagate to update the parameters.< / mask> < / mask> < / cov> 5. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1 is characterized in that: In step 3, the fingerprint reconstruction task is to cut the fingerprint image into image blocks and input them into the fingerprint feature extractor model as a time series. According to the feature vector extracted by the fingerprint feature extractor model, the fingerprint image reconstructor model will output an image, calculate the structural similarity SSIM between the image and the original fingerprint image, judge their similarity, and then update the model parameters.

6. The fingerprint fast retrieval method based on self-supervised neural network according to claim 1 is characterized in that: The fingerprint image reconstructor model adopts a fully connected layer, and the output corresponding to the first token of the fingerprint feature extractor model is connected to the fingerprint image reconstructor model; the fingerprint reconstruction task is to cut each original fingerprint image into n 16×16 pixel image blocks, and then input them into the fingerprint feature extractor model, and use the extracted image semantic expression as a feature vector, which is input into the fingerprint image reconstructor model to output a fingerprint image; the structural similarity index SSIM is calculated between the output fingerprint image and the original image, and the parameters of the fingerprint feature extractor model and the fingerprint image reconstructor model are updated according to the SSIM, so that the output of the fingerprint image reconstructor model is closer and closer to the original fingerprint image.

7. The fingerprint fast retrieval method based on self-supervised neural network according to claim 6 is characterized in that: The structural similarity SSIM calculation formula is as follows: where μ x is the mean value of x, μ y is the mean value of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y, c1=(k1L) 2 , c2=(k2L) 2 It is a constant used to maintain stability, L is the dynamic range of pixel values, k1=0.01, k2=0.03, the range of structural similarity SSIM is -1 to 1, when two pictures are exactly the same, the value of SSIM is equal to 1, x and y represent two pictures to be compared for similarity.

8. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1 is characterized in that: In step 4, each fingerprint in the fingerprint library corresponds to a list of detail points of unfixed length, each fingerprint image in the entire fingerprint library is input into the fingerprint feature extractor, and the output of the first token position of the fingerprint feature extractor is used as the feature vector of the fingerprint, the feature vector of the fingerprint is used as the key, and the list of detail points corresponding to the fingerprint is used as the value, and a fingerprint index library is constructed offline; the dimension of the feature vector extracted from each fingerprint is n, and the Euclidean distance d is calculated between the n-dimensional feature vector and the coordinate origin O={0,0,…,0} in the n-dimensional space, and then sorted from small to large to obtain a fingerprint index library S, d, key and value together constitute an element in the fingerprint index library S, so that a part of the fingerprints can be quickly eliminated according to the size of the distance in the retrieval stage.

9. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1 is characterized in that: In step 5, the first screening is performed by calculating the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the coordinate origin in the n-dimensional space, and then the Euclidean distance between the n-dimensional feature vector of the query fingerprint and the n-dimensional feature vectors of other fingerprints is calculated, and the candidate fingerprint sequence is finally screened out according to the threshold U.

10. The fingerprint rapid retrieval method based on self-supervised neural network according to claim 1, characterized in that: In step 5, the candidate fingerprint sequence is quickly retrieved using the fingerprint index library, and the steps are as follows: (1) When a query fingerprint is input, the query fingerprint is input into the fingerprint feature extractor model to obtain the feature vector A of the fingerprint = {a1, a2, …, a n }; (2) Calculate the distance between the feature vector A and the origin O as d, define U as the threshold, and let R1 = dU, R2 = d + U; (3) The fingerprint index library S is screened according to R1 and R2. Specifically, the fingerprints whose d is less than R1 in S are eliminated, and the fingerprints whose d is greater than R2 in S are eliminated; (4) Then, the Euclidean distance is calculated one by one between the feature vector A and the remaining fingerprint feature vectors in the fingerprint index library S. If the distance is less than the threshold U, the fingerprint is placed in the candidate sequence, otherwise the next fingerprint is compared.

Images