Fingerprint identification method, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses a fingerprint identification method, electronic equipment and a computer readable medium. The method comprises the following steps: acquiring a bottom library fingerprint image set and a to-be-identified fingerprint image; and inputting the base library fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model comprising an image encoder, a feature fusion module and a classifier. According to the embodiment, the accuracy of fingerprint identification is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer technology, and in particular, to a fingerprint identification method, an electronic device, and a computer readable medium. BACKGROUND

[0002] As an important branch of biometric identification, fingerprint identification technology has been widely used in scenarios such as smart terminal unlocking, identity authentication, and access control systems. Due to the limited feature expression capability of traditional image processing-based fingerprint identification methods, more and more scenarios use end-to-end fingerprint identification methods based on deep learning.

[0003] In the prior art, an end-to-end fingerprint identification method usually extracts and matches the features of a fingerprint image to be identified and a pre-stored database fingerprint image one by one. If the features of the fingerprint image to be identified match those of a certain database fingerprint image, it is determined that the fingerprint identification is passed. However, since a single database fingerprint image has less information, the accuracy of this fingerprint identification method is low. SUMMARY

[0004] Embodiments of the present application provide a fingerprint identification method, an electronic device, and a computer readable medium to solve the technical problem of low accuracy of fingerprint identification in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a fingerprint identification method, which includes: obtaining a database fingerprint image set and a fingerprint image to be identified; inputting the database fingerprint image set and the fingerprint image to be identified into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model including an image encoder, a feature fusion module, and a classifier.

[0006] In a second aspect, an embodiment of the present application provides an electronic device, which includes: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method described in the first aspect.

[0007] In a third aspect, an embodiment of the present application provides a computer readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer program product including a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0009] The fingerprint identification method, the electronic device and the computer readable medium provided by the embodiment of the present application first acquire a base library fingerprint image set and a to-be-identified fingerprint image; then input the base library fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model comprising an image encoder, a feature fusion module and a classifier. Since the base library fingerprint images in the base library fingerprint image set can provide rich and comprehensive fingerprint features, by inputting the base library fingerprint image set into the fingerprint identification model, the fingerprint identification model can make full use of the complementary information of multiple fingerprint images in the base library for fingerprint identification, thereby enhancing the comprehensive understanding of the user's fingerprint and improving the accuracy of fingerprint identification. BRIEF DESCRIPTION OF DRAWINGS

[0010] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings:

[0011] Figure 1 is a flowchart of an embodiment of the fingerprint identification method according to the present application;

[0012] Figure 2 is a structural schematic diagram of a model to be trained in the fingerprint identification method according to the present application;

[0013] Figure 3 is a structural schematic diagram of an embodiment of the fingerprint identification device according to the present application;

[0014] Figure 4 is a structural schematic diagram of an electronic device for implementing the embodiment of the present application. DETAILED DESCRIPTION

[0015] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0016] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0017] It should be noted that all actions of acquiring signals, information or data in the present application are carried out under the premise of complying with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0018] In recent years, biometric technology has been widely applied to various terminal devices or electronic devices. Biometric recognition technology includes, but is not limited to, fingerprint recognition, palmprint recognition, vein recognition, iris recognition, face recognition, living body recognition, anti-counterfeiting recognition, and the like. Among them, fingerprint recognition usually includes optical fingerprint recognition, capacitive fingerprint recognition, and ultrasonic fingerprint recognition. With the rise of full-screen technology, the fingerprint recognition module can be arranged in a partial area or an entire area under the display screen, thereby forming under-display optical fingerprint recognition; or, part or all of the optical fingerprint recognition module can be integrated into the display screen of the electronic device, thereby forming in-display optical fingerprint recognition. The display screen can be an organic light emitting diode (OLED) display screen or a liquid crystal display (LCD), etc. The fingerprint recognition method usually includes steps of acquiring, preprocessing, feature extraction, and feature matching of a fingerprint image. Part or all of the above steps can be realized by a traditional computer vision (CV) algorithm or a deep learning algorithm based on artificial intelligence (AI). The fingerprint recognition technology can be applied to portable or mobile terminals such as smartphones, tablet computers, and game devices, and other electronic devices such as smart door locks, cars, and bank automatic teller machines, for fingerprint unlocking, fingerprint payment, fingerprint attendance, identity authentication, and the like.

[0019] At present, due to the problems such as limited feature expression capability of the traditional image processing-based fingerprint recognition method, more and more scenes use an end-to-end fingerprint recognition method based on deep learning. Generally, the end-to-end fingerprint recognition method usually performs feature extraction and matching on the fingerprint image to be recognized and a pre-stored database fingerprint image one by one, and if the feature of the fingerprint image to be recognized matches that of a certain database fingerprint image, it is determined that the fingerprint recognition is passed. However, since the information of a single database fingerprint image is less, the accuracy of this fingerprint recognition mode is low. The present application provides a fingerprint recognition method, and the fingerprint recognition model trained by using the method can improve the accuracy of fingerprint recognition.

[0020] Please refer to Figure 1 which shows a flow 100 of one embodiment of the fingerprint recognition method according to the present application. The fingerprint recognition method includes the following steps:

[0021] Step 101, acquiring a database fingerprint image set and a fingerprint image to be recognized.

[0022] In this embodiment, the gallery fingerprint image set can include at least one gallery fingerprint image. The gallery fingerprint image refers to a fingerprint image pre-recorded by a user. The fingerprint image to be identified can be a fingerprint image collected when the user attempts to unlock, pay, or verify identity.

[0023] In some optional implementations of this embodiment, the gallery fingerprint image set can be obtained in the following steps:

[0024] First, a plurality of recorded fingerprint images are obtained. The recorded fingerprint image refers to an original fingerprint image collected when a user registers or records a fingerprint.

[0025] Then, each recorded fingerprint image in the plurality of recorded fingerprint images is subjected to quality detection to obtain a quality detection result of each recorded fingerprint image. The quality detection refers to a series of evaluation operations on the recorded fingerprint image to determine whether the recorded fingerprint image meets the quality standard. The quality detection can generally include, but is not limited to, detection of image clarity, completeness of the fingerprint area, etc. The quality detection result can be a quality score, a quality level, etc. and can be pre-set as needed, which is not limited here.

[0026] After that, the plurality of recorded fingerprint images are screened based on the quality detection result. Here, the recorded fingerprint images with high quality can be screened according to the quality detection result, for example, recorded fingerprint images with a quality score greater than a threshold value, or recorded fingerprint images with a quality level higher than a pre-set level, etc., which is not limited here.

[0027] Finally, the screened recorded fingerprint images are taken as gallery fingerprint images, and the gallery fingerprint images are collected to obtain a gallery fingerprint image set.

[0028] By screening the recorded fingerprint images with high quality as the gallery fingerprint images through quality detection, the recorded fingerprint images with low quality can be filtered out, noise and interference can be reduced, and the gallery fingerprint images can be ensured to have good feature expression capability, thereby improving the accuracy of fingerprint identification.

[0029] Step 102, inputting the gallery fingerprint image set and the fingerprint image to be identified into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model including an image encoder, a feature fusion module, and a classifier.

[0030] In this embodiment, the fingerprint identification model can be a neural network model with fingerprint identification function, which can be pre-trained through a machine learning method. The fingerprint identification model can include an image encoder, a feature fusion module, and a classifier.

[0031] The image encoder is configured to extract image features, specifically, image features of each gallery fingerprint image in the set of gallery fingerprint images and the to-be-identified fingerprint image. The image encoder can adopt a structure such as a Convolutional Neural Network (CNN) or a Vision Transformer (ViT). The image features contain not only local information but also global information, and can more comprehensively express the features of the gallery fingerprint images and the to-be-identified fingerprint image.

[0032] The feature fusion module is configured to fuse the image features extracted by the image encoder to obtain fused features. For example, the feature fusion module can include a multi-layer attention mechanism, such as a self-window attention mechanism and a cross-attention mechanism, to realize interaction and fusion between features. By fusing the image features of each gallery fingerprint image, the information of multiple gallery fingerprint images can be fully utilized, thereby improving the accuracy of fingerprint identification.

[0033] The classifier is configured to classify the image features fused by the feature fusion module and output a fingerprint identification result. The fingerprint identification result is a binary classification result, which indicates whether the to-be-identified fingerprint image and the set of gallery fingerprint images are images of the same finger. In practice, if the fingerprint identification model determines that the to-be-identified fingerprint image and the set of gallery fingerprint images represent the same finger, the binary classification result is yes, i.e., the fingerprint identification is successful; if the to-be-identified fingerprint image and the set of gallery fingerprint images represent different fingers, the binary classification result is no, i.e., the fingerprint identification fails.

[0034] The method provided by the above embodiments of the present application first acquires a set of gallery fingerprint images and a to-be-identified fingerprint image, and then inputs the set of gallery fingerprint images and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result. The fingerprint identification model includes an image encoder, a feature fusion module, and a classifier. Since the gallery fingerprint images in the set of gallery fingerprint images can provide rich and comprehensive fingerprint features, by inputting the set of gallery fingerprint images into the fingerprint identification model, the fingerprint identification model can fully utilize the complementary information of multiple fingerprint images in the gallery to perform fingerprint identification, thereby enhancing the comprehensive understanding of the user's fingerprint and improving the accuracy of fingerprint identification.

[0035] In some optional embodiments, step 102 can be performed according to the following sub-steps:

[0036] In sub-step 1021, the set of gallery fingerprint images and the to-be-identified fingerprint image are input into the image encoder to obtain first image features of each gallery fingerprint image in the set of gallery fingerprint images and second image features of the to-be-identified fingerprint image. The first image features of each gallery fingerprint image and the second image features can be represented in the form of feature maps.

[0037] In sub-step 1022, the first image features and the second image features of each gallery fingerprint image are input into a feature fusion module to obtain fused features. The feature fusion module can include a multi-layer attention mechanism, such as a self-window attention mechanism and a cross-attention mechanism, to realize the interaction and fusion between features.

[0038] In sub-step 1023, the fused features are input into a classifier to obtain a fingerprint recognition result. Since the image encoder, the feature fusion module, and the classifier sequentially have the ability to extract high-quality features from fingerprint images, feature fusion, and accurate classification, the fingerprint recognition model obtained by integrating these network modules can ensure that the fingerprint recognition model can efficiently and accurately complete the fingerprint recognition task in actual application.

[0039] In some optional embodiments, the feature fusion module can further include a first branch network, a second branch network, and a fusion network. On this basis, in sub-step 1022 above, the first image features of each gallery fingerprint image can be first input into the first branch network to obtain gallery fused features. Then, the second image features are input into the second branch network to obtain third image features. Finally, the gallery fused features and the third image features of the fingerprint image to be recognized are input into the fusion network to obtain fused features.

[0040] The first branch network is used to further process the first image features of each gallery fingerprint image, such as attention mechanism processing, and fuse the processing results to obtain gallery fused features. The second branch network is used to further process the second image features of the fingerprint image to be recognized, such as attention mechanism processing, to obtain third image features. The fusion network is used to fuse the gallery fused features and the third image features to obtain the final fused features.

[0041] Since multiple gallery fingerprint images provide rich feature information, by fusing the features of multiple gallery fingerprint images, the complementary information of multiple fingerprint images in the gallery can be fully utilized, the comprehensive learning and understanding of the model for the gallery fingerprint images can be enhanced, more discriminative gallery fused features can be generated, the feature expression ability can be improved, and the robustness of the model can be improved. By further fusing the gallery fused features and the third image features of the fingerprint image to be recognized, the complementary information of the gallery fingerprint images and the fingerprint image to be recognized can be fully utilized, more discriminative fused features can be generated, and the discriminative ability of the model can be improved.

[0042] In some optional embodiments, the first branch network comprises at least one first self-window attention layer and a first cross-attention layer. The gallery fingerprint image set comprises at least one gallery fingerprint image, and the at least one first self-window attention layer is in one-to-one correspondence with the at least one gallery fingerprint image. The second branch network can comprise a second self-window attention layer. The fusion network can comprise a second cross-attention layer.

[0043] The self-window attention layer is a feature processing module using a self-window attention mechanism. The self-window attention mechanism is a mechanism for limiting global self-attention calculation within a local window to improve local feature expression capability. Specifically, under the self-window attention mechanism, the input feature map can be evenly divided into multiple smaller, non-overlapping local windows, and a standard self-attention is independently calculated within each window, so that each element within the window only focuses on other elements within the same window. After each window independently calculates the self-attention, the calculation results are spliced back to the original spatial position to form an updated representation of the entire feature map. The self-attention is a basic module of the Transformer architecture, and the image blocks in the image update their own representation according to the importance of all other elements in the image.

[0044] The cross-attention layer is a feature processing module using a cross-attention mechanism, and is one of the core components of the Transformer neural network architecture. It can be used to process information interaction between different images. Through the cross-attention mechanism, dynamic associations between different images can be established to achieve feature fusion. Through the cross-attention mechanism, one image can be focused on the features that are missing in itself but exist in another image, thereby achieving information complementation. In addition, through the cross-attention mechanism, the relationship between the semantic corresponding regions in the two images, such as matching key points and similar objects, can be automatically established, thereby achieving dynamic alignment of the images.

[0045] On this basis, the gallery fusion feature can be extracted by the following steps: first, input the first image features of each gallery fingerprint image into the corresponding first self-window attention layer to obtain the fourth image features of each gallery fingerprint image. Then, input the obtained fourth image features into the first cross-attention layer to obtain the gallery fusion feature.

[0046] Since the first branch network comprises the self-window attention layer, the first image features of each gallery fingerprint image can be updated, so that the output image features can focus on the key areas in the gallery fingerprint image. On this basis, the image features are fused through the cross-attention mechanism, the complementary information of multiple fingerprint images in the gallery can be fully utilized, and more discriminative gallery fusion features are generated, thereby improving the feature expression capability. Since the second branch network comprises the self-window attention layer, the second image features of the extracted to-be-identified fingerprint image can be updated, so that the output image features can focus on the key areas in the to-be-identified fingerprint image, and the model's understanding of the details of the to-be-identified fingerprint image is enhanced. The gallery fingerprint fusion features and the image features of the to-be-identified fingerprint image are finally fused through the second cross-attention layer of the fusion network, the complementary information of the gallery fingerprint image and the to-be-identified fingerprint image can be fully utilized, and more discriminative fusion features are generated, thereby improving the discriminative ability of the model.

[0047] In some optional embodiments, the first branch network can further comprise a third self-window attention layer, which can be connected with the first cross-attention layer to receive the output of the first cross-attention layer. After the fourth image features are input into the first cross-attention layer to obtain the gallery fusion features, the gallery fusion features can be input into the third self-window attention layer to obtain updated gallery fusion features. Through the third self-window attention layer, the gallery fusion features output by the first cross-attention layer can be further fused and updated, so that the output gallery fusion features can focus on the key areas in the gallery fingerprint image, thereby improving the accuracy of the gallery fusion features and further improving the discriminative ability of the model.

[0048] In some optional embodiments, the fingerprint recognition model can be obtained through the following steps:

[0049] Step 201, obtaining a set of gallery fingerprint sample images and a to-be-identified fingerprint sample image.

[0050] Step 202, inputting the set of gallery fingerprint sample images and the to-be-identified fingerprint sample image into a to-be-trained model to obtain the fingerprint recognition result output by the to-be-trained model, a predicted fingerprint image and a predicted gallery fingerprint splicing image aligned with the predicted fingerprint image. Referring to Figure 2 The to-be-trained model comprises an image encoder, a feature fusion module, a classifier and an image decoder, and the predicted fingerprint image and the predicted gallery fingerprint splicing image are output by the image decoder.

[0051] The to-be-trained model refers to a fingerprint recognition model that has not completed training in the training stage. The model can employ a deep neural network. The model can extract fingerprint features of the gallery fingerprint images in the gallery fingerprint image set and fingerprint features of the to-be-identified fingerprint image, and then perform fusion and other processing of the fingerprint features, and further perform fingerprint recognition, generation of a predicted fingerprint image, and generation of a predicted gallery fingerprint splicing image.

[0052] The predicted fingerprint image refers to a to-be-identified fingerprint image predicted by the model. The predicted gallery fingerprint splicing image refers to a splicing image of the gallery fingerprint image predicted by the model and spatially aligned with the predicted fingerprint image. The predicted fingerprint image and the predicted gallery fingerprint splicing image can have the same size and the same fingerprint region.

[0053] The classifier can be used to classify the fused features and output a fingerprint recognition result. The fingerprint recognition result can be used to indicate whether the to-be-identified fingerprint image matches the gallery fingerprint image, and thus can be used to improve the accuracy of fingerprint recognition of the model.

[0054] The image decoder is used to decode the fused features to generate the predicted fingerprint image and the predicted gallery fingerprint splicing image. The image decoder can include network structures such as up-sampling and de-convolution, and can output the predicted fingerprint image and the predicted gallery fingerprint splicing image with a resolution consistent with that of the to-be-identified fingerprint image. The predicted fingerprint image and the predicted gallery fingerprint splicing image can be used to improve feature alignment and spatial consistency.

[0055] By outputting the fingerprint recognition result through the classifier and outputting the predicted fingerprint image and the predicted gallery fingerprint splicing image through the image decoder, multi-task learning of the model can be achieved, so that the model not only has accurate classification ability, i.e., fingerprint recognition ability, but also has feature alignment ability. This multi-task learning method can improve the generalization ability and robustness of the model, making it perform more stably in different application scenarios.

[0056] In step 203, the to-be-trained model is trained based on the fingerprint recognition result, the predicted fingerprint image, and the predicted gallery fingerprint splicing image output by the to-be-trained model, and a fingerprint recognition model is obtained.

[0057] Specifically, the loss value of the model can be calculated based on the fingerprint recognition result, the predicted fingerprint image, the predicted gallery fingerprint splicing image, the gallery fingerprint image set, and the to-be-identified fingerprint image. The loss value is the value of a loss function, which is a non-negative real-valued function and can be used to represent the difference between the detection result and the true result. Generally, the smaller the loss value, the better the robustness of the model. The loss function can be set according to actual needs.

[0058] In practice, the loss value can include two parts, i.e., a classification loss value and an image prediction loss value. The classification loss value can be used to improve the model's fingerprint recognition accuracy. The classification loss value can be determined based on the model output fingerprint recognition result, the preset annotation information, and the preset loss function used to calculate the classification loss value. The annotation information can be used to represent the ideal fingerprint recognition result, which can be obtained in advance by manual annotation or the like. The image prediction loss value can be used to improve the feature alignment capability and the spatial consistency between images. The image prediction loss value can be determined based on the predicted fingerprint image, the predicted database fingerprint splicing image, the database fingerprint image set, the to-be-identified fingerprint image, and the preset loss function used to calculate the image prediction loss value.

[0059] After the loss value of the model is calculated, the parameters of the model can be updated based on the loss value, thereby completing one model training. The model training process can be iteratively performed multiple times until the training is completed. In practice, whether the model training is completed can be determined in various ways. For example, when the accuracy of the model output fingerprint recognition result reaches a preset value, it can be determined that the training is completed. For another example, if the number of model training times is equal to a preset number, it can be determined that the training is completed. For another example, when the loss value of the model converges, it can be determined that the training is completed. After it is determined that the model training is completed, the trained image encoder, feature fusion module, and classifier can be integrated to obtain a fingerprint recognition model. The trained image encoder, feature fusion module, and classifier already have the ability to extract high-quality features from fingerprint images, fuse features, and accurately classify. Integrating these network modules to obtain a fingerprint recognition model can ensure that the fingerprint recognition model can efficiently and accurately complete the fingerprint recognition task in actual application.

[0060] Since the task of predicting the fingerprint image and the predicted database fingerprint splicing image is added in the training process, the model can pay attention to the spatial consistency between the to-be-identified fingerprint image and the database fingerprint image in the training process, so that the model can more accurately extract and align the fingerprint features, thereby further improving the accuracy of fingerprint recognition.

[0061] In addition, the above-mentioned end-to-end training manner makes the training process of the entire model from input to output continuous, without the need for manual design and adjustment of intermediate steps. This training manner not only improves the training efficiency, but also makes the model more easily deployed and expanded. For example, the model can be conveniently deployed on different fingerprint recognition terminal devices to adapt to different application scenarios. In actual application, the model can be deployed on various intelligent devices, such as smart phones, tablet computers, or access control systems, etc.

[0062] In some optional embodiments, step 203 can further include the following sub-steps:

[0063] In substep 2031, a first loss value of the to-be-trained model is determined based on the first loss function, the fingerprint recognition result, and the fingerprint recognition label. The first loss function can be used to measure the difference between the model fingerprint recognition result and the preset label information. For example, a binary classification loss can be used to optimize the classification performance of the model.

[0064] In substep 2032, a second loss value of the to-be-trained model is determined based on the second loss function, the predicted fingerprint image, and the to-be-recognized fingerprint image. The second loss function can be used to measure the difference between the predicted fingerprint image output by the model and the to-be-recognized fingerprint image input to the model. For example, a loss function such as mean square error can be used to optimize the image prediction performance of the model. The setting of the second loss function requires the model to accurately predict the to-be-recognized fingerprint image based on the extracted features, which enables the model to retain the original spatial information and detailed features of the to-be-recognized fingerprint image as completely and accurately as possible in the feature extraction stage, avoids losing information important to the spatial structure of the to-be-recognized fingerprint image in the feature extraction process, and thus guarantees the integrity and spatial consistency of the image features of the to-be-recognized fingerprint image.

[0065] In substep 2033, a database fingerprint splicing sample image is obtained, and a third loss value of the to-be-trained model is determined based on the third loss function, the predicted database fingerprint splicing image, and the database fingerprint splicing sample image. The database fingerprint splicing sample image can be generated by splicing the database fingerprint images in the database fingerprint image set and aligning them with the to-be-recognized fingerprint image in advance. The third loss function can be used to measure the difference between the predicted database fingerprint splicing image generated by the model and the target database splicing fingerprint image. For example, a loss function such as mean square error can be used to optimize the feature alignment and spatial consistency of the model. By setting the third loss function, the model can be guided to generate a predicted database fingerprint splicing image aligned with the to-be-recognized fingerprint image based on the extracted features. To achieve this goal, the model needs to deeply learn the spatial transformation relationship, feature correspondence relationship, etc. between multiple database fingerprint images and the to-be-recognized fingerprint image, thereby effectively reducing the difference between the database fingerprint images and the to-be-recognized fingerprint image in terms of spatial position, etc., strengthening the alignment effect between the two, enabling the features of the two to be more accurately compared and matched in a unified spatial coordinate system, and thus improving the fingerprint recognition performance of the model.

[0066] Optionally, the base library fingerprint splicing sample image can be generated by the following steps: first, splicing the base library fingerprint images in the base library fingerprint image set to obtain an initial base library fingerprint spliced image. Then, aligning the initial base library fingerprint spliced image with the to-be-recognized fingerprint image, and based on the size of the to-be-recognized fingerprint image, cutting the base library fingerprint spliced sample image from the initial base library fingerprint spliced image after alignment with the to-be-recognized fingerprint image. The base library fingerprint spliced sample image is the region in the aligned initial base library fingerprint spliced image corresponding to the to-be-recognized fingerprint image. In this way, an accurate base library fingerprint spliced sample image can be obtained. Taking it as the label of the predicted base library fingerprint spliced image, an accurate third loss value can be calculated, providing an accurate data basis for subsequent model training.

[0067] In substep 2034, the total loss value of the to-be-trained model is determined based on the first loss value, the second loss value, and the third loss value. Here, the first loss value, the second loss value, and the third loss value can be weighted and summed to obtain the total loss value of the to-be-trained model. It should be noted that the weights can be adjusted according to different application scenarios to balance the classification and image prediction tasks.

[0068] In substep 2035, the parameters of the to-be-trained model are updated based on the total loss value to obtain the fingerprint recognition model. Specifically, after the total loss value is calculated, the parameters of the model can be updated based on the total loss value, thereby completing one model training. The model training process can be iteratively performed multiple times until the training is completed.

[0069] By setting the first loss function, the classification performance of the model can be optimized to ensure that the model can accurately determine whether the fingerprints match. By setting the second loss function, the image prediction performance of the model can be optimized to ensure that the model can generate high-quality predicted fingerprint images. By setting the third loss function, the feature alignment and spatial consistency of the model can be optimized to ensure that the model can generate high-quality predicted base library fingerprint spliced images. By combining the above three losses, the model can simultaneously improve the classification, image prediction, and feature alignment performance, thereby improving the overall fingerprint recognition performance.

[0070] It should be noted that additional auxiliary losses, such as fingerprint region segmentation loss, can also be added as needed to further improve the alignment accuracy and robustness of the model, which are not limited here.

[0071] Further reference Figure 3 As an implementation of the method shown in the above figures, the present application provides an embodiment of a fingerprint recognition device, which corresponds to the method embodiment shown in Figure 1 The device can be applied in various electronic devices.

[0072] As Figure 3As shown, the fingerprint identification device 300 of the embodiment includes: an acquisition unit 301, configured to acquire a database fingerprint image set and a to-be-identified fingerprint image; an input unit 302, configured to input the database fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, wherein the fingerprint identification model includes an image encoder, a feature fusion module and a classifier.

[0073] In some optional implementation manners of the embodiment, the input unit 302 is further configured to: input the database fingerprint image set and the to-be-identified fingerprint image into the image encoder to obtain first image features of each database fingerprint image in the database fingerprint image set and second image features of the to-be-identified fingerprint image; input the first image features of each database fingerprint image and the second image features into the feature fusion module to obtain fused features; and input the fused features into the classifier to obtain the fingerprint identification result.

[0074] In some optional implementation manners of the embodiment, the feature fusion module includes a first branch network, a second branch network and a fusion network; and the input unit 302 is further configured to: input the first image features of each database fingerprint image into the first branch network to obtain database fused features; input the second image features into the second branch network to obtain third image features; and input the database fused features and the third image features into the fusion network to obtain the fused features.

[0075] In some optional implementation manners of the embodiment, the database fingerprint image set includes at least one database fingerprint image; the first branch network includes at least one first self-window attention layer and a first cross-attention layer, the at least one first self-window attention layer corresponds to the at least one database fingerprint image in a one-to-one manner; the second branch network includes a second self-window attention layer, and the fusion network includes a second cross-attention layer; and the input unit 302 is further configured to: input the first image features of each database fingerprint image into a corresponding first self-window attention layer to obtain fourth image features of each database fingerprint image; and input the fourth image features into the first cross-attention layer to obtain the database fused features.

[0076] In some optional implementation manners of the embodiment, the first branch network further includes a third self-window attention layer; and the input unit 302 is further configured to: input the database fused features into the third self-window attention layer to obtain updated database fused features.

[0077] In some optional implementations of the present embodiment, the fingerprint recognition model is trained by the following steps: obtaining a base fingerprint sample image set and a to-be-recognized fingerprint sample image; inputting the base fingerprint sample image set and the to-be-recognized fingerprint sample image into a to-be-trained model to obtain a fingerprint recognition result output by the to-be-trained model, a predicted fingerprint image and a predicted base fingerprint splicing image aligned with the predicted fingerprint image, the to-be-trained model comprising the image encoder, the feature fusion module, the classifier and an image decoder, the predicted fingerprint image and the predicted base fingerprint splicing image being output by the image decoder; training the to-be-trained model based on the fingerprint recognition result output by the to-be-trained model, the predicted fingerprint image and the predicted base fingerprint splicing image to obtain a fingerprint recognition model.

[0078] In some optional implementations of the present embodiment, the training of the to-be-trained model based on the fingerprint recognition result output by the to-be-trained model, the predicted fingerprint image and the predicted base fingerprint splicing image to obtain a fingerprint recognition model comprises: determining a first loss value of the to-be-trained model based on a first loss function, the fingerprint recognition result output by the to-be-trained model and a fingerprint recognition label; determining a second loss value of the to-be-trained model based on a second loss function, the predicted fingerprint image and the to-be-recognized fingerprint image; obtaining a base fingerprint splicing sample image and determining a third loss value of the to-be-trained model based on a third loss function, the predicted base fingerprint splicing image and the base fingerprint splicing sample image; determining a total loss value of the to-be-trained model based on the first loss value, the second loss value and the third loss value; updating parameters of the to-be-trained model based on the total loss value to obtain a fingerprint recognition model.

[0079] In some optional implementations of the present embodiment, the obtaining of the base fingerprint splicing sample image comprises: splicing a base fingerprint image in the base fingerprint image set to obtain an initial base fingerprint splicing image; aligning the initial base fingerprint splicing image with the to-be-recognized fingerprint image and, based on a region of the to-be-recognized fingerprint image in the initial base fingerprint splicing image, cutting a base fingerprint splicing sample image from the initial base fingerprint splicing image.

[0080] In some optional implementations of the present embodiment, the obtaining unit 301 is further configured to: obtain a plurality of enrolled fingerprint images; perform quality detection on each enrolled fingerprint image in the plurality of enrolled fingerprint images to obtain a quality detection result of each enrolled fingerprint image; perform screening on the plurality of enrolled fingerprint images based on the quality detection result; take the screened enrolled fingerprint image as a base fingerprint image, and aggregate the base fingerprint images to obtain a base fingerprint image set.

[0081] The device provided by the above embodiments of the present application first acquires the base library fingerprint image set and the to-be-identified fingerprint image, and then inputs the base library fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model comprising an image encoder, a feature fusion module and a classifier. Since the base library fingerprint images in the base library fingerprint image set can provide rich and comprehensive fingerprint features, by inputting the base library fingerprint image set into the fingerprint identification model, the fingerprint identification model can make full use of the complementary information of multiple fingerprint images in the base library for fingerprint identification, thereby enhancing the comprehensive understanding of the user's fingerprint and improving the accuracy of fingerprint identification.

[0082] The embodiments of the present application also provide an electronic device, comprising one or more processors, a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the above fingerprint identification method.

[0083] Reference will be made to the following description Figure 4 which shows a structural schematic diagram of an electronic device for implementing some embodiments of the present application. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0084] As shown in Figure 4 , the electronic device 400 can include a processing device (such as a central processor, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 402 or loaded into a random access memory (RAM) 403 from a storage device 408. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402 and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0085] Generally, the following devices can be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic disk, a hard disk, etc.; and a communication device 409. The communication device 409 can allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or have all the shown devices. More or less devices can alternatively be implemented or possessed. Figure 4Each block shown in the flowchart can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instruction" or "instructions" can be taken to mean any minute part of the computer program). It should also be noted that each block of the flowchart illustrations, and / or any other combinations of blocks in the flowchart illustrations, can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions.

[0086] The embodiment of the present application further provides a computer program product, comprising a computer program which, when executed by a processor, implements the above-mentioned fingerprint identification method.

[0087] In particular, according to some embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, some embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the method shown in the flowcharts. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present application are executed.

[0088] The embodiment of the present application further provides a computer readable medium, having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned fingerprint identification method.

[0089] Note that the computer-readable medium in some embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus, or device to function or operate. In some embodiments of the present application, the computer-readable signal medium can include a computer-readable program code in a baseband or propagated as a carrier wave in a propagated signal, where the computer-readable program code can be used by or in connection with an instruction execution system, apparatus, or device. The propagated signal can take any of a variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF (radio frequency), and the like, or any suitable combination of the foregoing.

[0090] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0091] The computer readable medium can be included in the electronic device; or can exist separately from the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquire a base library fingerprint image set and a to-be-identified fingerprint image; input the base library fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, the fingerprint identification model comprising an image encoder, a feature fusion module and a classifier.

[0092] Computer program code for carrying out operations of some embodiments of the application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0093] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0094] The units described in some embodiments of the present application can be implemented by means of software, or by means of hardware. The described units can also be implemented in a processor, for example, it can be described as: a processor comprising a first determining unit, a second determining unit, a selecting unit and a third determining unit. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0095] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0096] The above description is merely some of the preferred embodiments of the present application and a description of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features can be replaced with technical features disclosed in the embodiments of the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A method of fingerprint recognition, characterized by, The method comprises: obtaining a fingerprint image set and a to-be-identified fingerprint image; inputting the fingerprint image set and the to-be-identified fingerprint image into a pre-trained fingerprint identification model to obtain a fingerprint identification result, wherein the fingerprint identification model comprises an image encoder, a feature fusion module and a classifier.

2. The method of claim 1, wherein, The method comprises: inputting the fingerprint image set and the to-be-identified fingerprint image into the image encoder to obtain first image features of each fingerprint image in the fingerprint image set and second image features of the to-be-identified fingerprint image; inputting the first image features of each fingerprint image and the second image features into the feature fusion module to obtain fusion features; inputting the fusion features into the classifier to obtain the fingerprint identification result.

3. The method of claim 2, wherein, The feature fusion module comprises a first branch network, a second branch network and a fusion network; the method comprises: inputting the first image features of each fingerprint image into the first branch network to obtain fingerprint fusion features; inputting the second image features into the second branch network to obtain third image features; inputting the fingerprint fusion features and the third image features into the fusion network to obtain fusion features.

4. The method of claim 3, wherein, The fingerprint image set comprises at least one fingerprint image; the first branch network comprises at least one first self-window attention layer and a first cross-attention layer, the at least one first self-window attention layer corresponds to the at least one fingerprint image one-to-one; the second branch network comprises a second self-window attention layer, and the fusion network comprises a second cross-attention layer; The method comprises: inputting the first image features of each fingerprint image into the corresponding first self-window attention layer to obtain fourth image features of each fingerprint image; inputting the fourth image features into the first cross-attention layer to obtain the fingerprint fusion features.

5. The method of claim 4, wherein, The first branch network further comprises a third self-window attention layer; after the fourth image features are inputted into the first cross-attention layer to obtain the fingerprint fusion features, the method further comprises: inputting the fingerprint fusion features into the third self-window attention layer to obtain updated fingerprint fusion features.

6. The method of claim 1, wherein, The fingerprint identification model is trained through the following steps: obtaining a fingerprint sample image set and a to-be-identified fingerprint sample image; inputting the fingerprint sample image set and the to-be-identified fingerprint sample image into a to-be-trained model to obtain a fingerprint recognition result, a predicted fingerprint image and a predicted database fingerprint splicing image aligned with the predicted fingerprint image output by the to-be-trained model, the to-be-trained model comprising the image encoder, the feature fusion module, the classifier and an image decoder, the predicted fingerprint image and the predicted database fingerprint splicing image being output by the image decoder; training the to-be-trained model based on the fingerprint recognition result, the predicted fingerprint image and the predicted database fingerprint splicing image output by the to-be-trained model to obtain a fingerprint recognition model.

7. The method of claim 6, wherein, The training the to-be-trained model based on the fingerprint recognition result, the predicted fingerprint image and the predicted database fingerprint splicing image output by the to-be-trained model to obtain a fingerprint recognition model comprises: determining a first loss value of the to-be-trained model based on a first loss function, the fingerprint recognition result output by the to-be-trained model and a fingerprint recognition label; determining a second loss value of the to-be-trained model based on a second loss function, the predicted fingerprint image and the to-be-identified fingerprint image; obtaining a database fingerprint splicing sample image and determining a third loss value of the to-be-trained model based on a third loss function, the predicted database fingerprint splicing image and the database fingerprint splicing sample image; determining a total loss value of the to-be-trained model based on the first loss value, the second loss value and the third loss value; updating parameters of the to-be-trained model based on the total loss value to obtain a fingerprint recognition model.

8. The method of claim 7, wherein, The obtaining a database fingerprint splicing sample image comprises: splicing database fingerprint images in the database fingerprint image set to obtain an initial database fingerprint splicing image; aligning the initial database fingerprint splicing image with the to-be-identified fingerprint image and based on a region of the to-be-identified fingerprint image in the initial database fingerprint splicing image, cutting a database fingerprint splicing sample image from the initial database fingerprint splicing image.

9. The method of claim 1, wherein, The obtaining a database fingerprint image set comprises: obtaining a plurality of enrolled fingerprint images; performing quality detection on each enrolled fingerprint image in the plurality of enrolled fingerprint images to obtain a quality detection result of each enrolled fingerprint image; performing screening on the plurality of enrolled fingerprint images based on the quality detection result; taking the screened enrolled fingerprint image as a database fingerprint image and collecting the database fingerprint images to obtain a database fingerprint image set.

10. An electronic device, comprising: comprises: a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor to implement the method of any one of claims 1-9.

11. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the method of any one of claims 1-9.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-9.

Images