Fingerprint foreign matter detection method and device, electronic equipment and storage medium

Abstract

The invention provides a fingerprint-based foreign matter detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first fingerprint image and a second fingerprint image, the first fingerprint image being a fingerprint image provided during unlocking, and the second fingerprint image being an input fingerprint image; performing spatial alignment processing on the first fingerprint image and the second fingerprint image, taking the first fingerprint image and the second fingerprint image after spatial alignment as input images, and performing foreign matter detection based on the input images; according to the invention, the stability of foreign matter detection can be improved, and the generalization of unknown foreign matter patterns is enhanced.

Description

Technical Field

[0001] The present application relates to the field of fingerprint detection, and in particular to a fingerprint foreign body detection method, device, electronic device and storage medium. Background Art

[0002] The False Accept (FA) test is a key component of fingerprint recognition system security testing. This test typically involves attaching patterns of non-human fingerprints, such as hair, transparent cases, scotch tape, or scratches, to the phone screen to verify the security of the fingerprint unlocking system. Specifically, the fingerprint unlocking system must ensure that the same finger can unlock the phone even with the same pattern, but that different fingers with the same pattern cannot unlock the phone.

[0003] Currently, common foreign object detection technologies primarily rely on image classification models. These models typically use a single image as input to determine whether it contains a foreign object pattern. While single-image image classification techniques can detect foreign object patterns to a certain extent, they suffer from a wide variety of foreign object types, insufficient data diversity, poor generalization, and limited stability. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a fingerprint foreign body detection method, device, electronic device and storage medium, which can improve the stability of foreign body detection and enhance the generalization of unknown foreign body patterns.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a fingerprint foreign body detection method, the method comprising:

[0007] Acquire a first fingerprint image and a second fingerprint image, wherein the first fingerprint image is a fingerprint image provided during unlocking, and the second fingerprint image is a pre-recorded fingerprint image used to verify the matching degree of the first fingerprint image;

[0008] The first fingerprint image and the second fingerprint image are spatially aligned, and the spatially aligned first fingerprint image and the second fingerprint image are used as input images, and foreign body detection is performed based on the input images.

[0009] In a second aspect, an embodiment of the present application further provides a fingerprint foreign body detection device, the device comprising:

[0010] an acquisition module, configured to acquire a first fingerprint image and a second fingerprint image, wherein the first fingerprint image is a fingerprint image provided during unlocking, and the second fingerprint image is a pre-recorded fingerprint image used to verify a matching degree with the first fingerprint image;

[0011] The detection module is used to perform spatial alignment processing on the first fingerprint image and the second fingerprint image, and use the spatially aligned first fingerprint image and the second fingerprint image as input images, and perform foreign body detection based on the input images.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to execute the fingerprint foreign body detection method described in any one of the first aspects.

[0013] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the fingerprint foreign body detection method described in any one of the first aspects is executed.

[0014] The embodiments of the present application have the following beneficial effects:

[0015] By simultaneously inputting the real-time fingerprint image provided by the user during unlocking (the first fingerprint image) and the fingerprint image pre-registered by the system (the second fingerprint image), a richer data base is provided, making it easier to distinguish the presence of foreign object interference during detection. Specifically, the two fingerprint images are first spatially aligned to ensure their precise correspondence during comparison. Subsequently, foreign object detection analysis is performed based on the aligned images. Because dual-image input provides more feature information, it can more accurately identify differences between fingerprints or foreign object patterns. This improved difference recognition capability enables rapid and accurate judgment in various complex and changing unlocking scenarios, effectively preventing false recognition or rejection caused by foreign object interference. In addition, the dual-image input strategy helps learn more feature information about fingerprints and foreign objects, further improving resolution and generalization performance. This enables strong adaptability and accuracy even when dealing with unknown or new foreign object patterns. Therefore, the application of the dual-image input strategy in fingerprint recognition technology, especially in foreign object detection, significantly improves accuracy, resolution, and generalization performance, providing a strong guarantee for the security and reliability of fingerprint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 101 - 102 is a flowchart of steps S101-S102 provided in an embodiment of the present application;

[0018] Figure 2 2 is a flow chart of steps S201-S203 provided in an embodiment of the present application;

[0019] Figure 3 Schematic diagram of the process of steps S301-S302 provided in the embodiment of the present application;

[0020] Figure 4 4 is a flow chart of steps S401-S403 provided in an embodiment of the present application;

[0021] Figure 5 Schematic diagram of the structure of the fingerprint foreign body detection device provided in an embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the composition structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0024] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0026] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0027] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0029] See also Figure 1 , Figure 1 This is a flow chart of steps S101-S102 of the fingerprint foreign body detection method provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S102 are shown for explanation.

[0030] In step S101, a first fingerprint image and a second fingerprint image are obtained, wherein the first fingerprint image is a fingerprint image provided when unlocking, and the second fingerprint image is a pre-recorded fingerprint image used to verify the matching degree of the first fingerprint image.

[0031] Here, the first fingerprint image refers to the fingerprint image captured in real time by the fingerprint sensor when the user attempts to unlock or verify their identity. The second fingerprint image refers to the fingerprint template image previously entered and stored by the user in the fingerprint base library (also known as the fingerprint template library) as the comparison reference.

[0032] In step S102, spatial alignment is performed on the first fingerprint image and the second fingerprint image, and the spatially aligned first fingerprint image and the second fingerprint image are used as input images, and foreign body detection is performed based on the input images.

[0033] Because factors such as the angle and position of the acquisition can cause deviations between the two captured fingerprint images, spatial alignment of the two images is necessary. This process aims to ensure that the fingerprint feature points in the two images coincide as closely as possible, enabling accurate subsequent comparison. Foreign object detection is performed based on the two spatially aligned images. The key approach is to analyze the differences between the two images and determine whether these differences are caused by foreign objects.

[0034] The above method builds a more comprehensive and rich data comparison foundation by synchronously introducing the real-time fingerprint image (first fingerprint image) of the user when unlocking and the pre-stored fingerprint template (second fingerprint image). This dual-image input strategy first ensures that the two fingerprint images are precisely spatially aligned, making the comparison process more rigorous; on this basis, foreign object detection analysis can more keenly capture the subtle differences between fingerprints and foreign object patterns. Because the dual images provide more feature information, judgments can be made more quickly and accurately when faced with complex unlocking scenarios, effectively avoiding misjudgments caused by foreign object interference. At the same time, it also enhances the system's adaptability and resolution capabilities to new foreign object patterns, further improving the overall accuracy and reliability of fingerprint recognition.

[0035] In some embodiments, see Figure 2 , Figure 2 This is a flow chart of steps S201-S203 provided in an embodiment of the present application. The spatial alignment processing of the first fingerprint image and the second fingerprint image can be achieved through steps S201-S203, which will be explained in combination with each step.

[0036] In step S201, key points in the first fingerprint image and the second fingerprint image are obtained, and descriptors are generated for the key points.

[0037] In some embodiments, obtaining key points in the first fingerprint image and the second fingerprint image and generating descriptors for the key points includes:

[0038] Acquire key points in the first fingerprint image and the second fingerprint image using a key point detection algorithm, wherein the key point detection algorithm includes at least one of SIFT and ORB;

[0039] For the key point, local features of the key point are obtained through a descriptor algorithm, and the descriptor is generated based on the local features.

[0040] In the first and second fingerprint images, keypoints are those points with distinctive features that remain relatively stable across different images, such as ridge intersections and endpoints. The Scale-Invariant Feature Transform (SIFT) is a keypoint detection algorithm that detects local feature points in an image and generates a unique descriptor for each feature point. SIFT features are invariant to image rotation, scaling, and brightness changes, making them well-suited for fingerprint image processing. However, the SIFT algorithm is computationally intensive and can require significant processing time. ORB (Oriented FAST and Rotated BRIEF) is a faster keypoint detection algorithm than SIFT, combining the advantages of the FAST (Features from Accelerated Segment Test) keypoint detector and the BRIEF (Binary Robust Independent Elementary Features) descriptor. The ORB algorithm first layers the image into pyramidal layers. Then, the FAST algorithm is used to detect keypoints at each layer, assigning them directional characteristics through orientation calculation. Finally, the BRIEF algorithm is used to describe these directional keypoints, generating binary descriptors. The ORB algorithm greatly improves the processing speed while ensuring a certain degree of accuracy. In practical applications, SIFT, ORB or a combination of them can be selected to detect key points in fingerprint images based on specific needs and resource conditions.

[0041] A descriptor is a vector or string that describes the image information surrounding a keypoint. After obtaining the keypoints, a descriptor algorithm is used to extract the local features of each keypoint and generate a descriptor based on these features. For each keypoint, the descriptor algorithm considers information such as the surrounding pixel values, gradient direction, and texture to extract local features that uniquely characterize the keypoint. Based on the extracted local features, the descriptor algorithm generates a fixed-length vector or string as a descriptor. This descriptor is used in the subsequent keypoint matching process.

[0042] The above approach uses keypoint detection algorithms (such as SIFT and ORB) to extract stable feature points from fingerprint images and generate precise descriptors, ensuring the accuracy and reliability of matching keypoint pairs. This provides a solid foundation for achieving high-precision spatial alignment between the first and second fingerprint images, effectively reducing fingerprint recognition errors caused by inaccurate alignment. Combining multiple keypoint detection algorithms and descriptor generation methods, such as the robustness of SIFT and the efficiency of ORB, allows for flexible selection based on different application scenarios and resource conditions, thereby enhancing the algorithm's adaptability and flexibility.

[0043] In step S202, matching key point pairs in the first fingerprint image and the second fingerprint image are determined based on the descriptor.

[0044] Here, a descriptor matching algorithm (such as FLANN and BFMatcher) can be used to compare the descriptors of all key points in the first and second fingerprint images, finding the most similar point pairs as the matching key point pairs. To improve matching accuracy, optimization techniques such as the Lowe's ratio test can also be used. This requires that the ratio of the nearest neighbor distance to the next nearest neighbor distance of a matching point be less than a certain threshold for the match to be considered valid. In addition, algorithms such as RANSAC (Random Sample Consensus Algorithm) can be used to further eliminate incorrect matching points.

[0045] The above approach, using efficient descriptor matching algorithms (such as FLANN, BFMatcher) and optimization techniques (such as ratio test, RANSAC), can quickly and accurately find matching key point pairs, significantly improving the efficiency of the alignment process and reducing computational costs.

[0046] In step S203, spatial alignment processing is performed on the first fingerprint image and the second fingerprint image based on the key point pair, so that the spatial positions of the first fingerprint image and the second fingerprint image are consistent.

[0047] In some embodiments, the spatially aligning the first fingerprint image and the second fingerprint image based on the key point pair includes:

[0048] Calculating a transformation matrix from the first fingerprint image to the second fingerprint image based on the key point pairs, wherein the transformation matrix is used to perform an affine transformation or a perspective transformation;

[0049] The transformation matrix is applied to the first fingerprint image.

[0050] Here, based on the matched keypoint pairs, a transformation matrix can be calculated that describes the geometric transformation relationship from the first fingerprint image to the second fingerprint image. This transformation can be an affine transformation or a perspective transformation, depending on the difference between the images and the required accuracy.

[0051] Affine transformation includes linear transformations such as rotation, translation, scaling, and flipping, but does not include perspective deformation. In fingerprint image alignment, if the deformation between two images is mainly rotation and translation, then affine transformation can be used.

[0052] Perspective transformation can handle perspective distortion between images. If the fingerprint images have large angle deviations or depth variations when they are collected, perspective transformation can be used to align the images. The transformation matrix is typically calculated using least squares or other optimization algorithms to minimize the positional error between matching keypoint pairs.

[0053] Once the transformation matrix is calculated, it can be applied to the first fingerprint image to generate a new image that is aligned with the second fingerprint image. This process involves resampling and interpolation of the image to ensure that the transformed image remains visually smooth and continuous. Resampling involves calculating the corresponding position of each pixel in the new image in the first fingerprint image based on the transformation matrix. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation.

[0054] The above method can realize affine transformation or perspective transformation by calculating the transformation matrix and applying it to the first fingerprint image, effectively handling complex deformations such as rotation, translation, scaling and perspective distortion between fingerprint images, and further improving the accuracy and robustness of alignment.

[0055] In some embodiments, see Figure 3 , Figure 3 This is a flow chart of steps S301-S303 provided in an embodiment of the present application. The first fingerprint image and the second fingerprint image after spatial alignment are used as input images, and foreign body detection is performed based on the input images. This can be achieved through steps S301-S303, which will be explained in combination with each step.

[0056] In step S301, the input image is input into a target model.

[0057] In step S302, ridge alignment result information is obtained based on the target model; wherein the ridge alignment result information is a binary classification result, and the binary classification result represents whether the fingerprint ridges of the first fingerprint image and the second fingerprint image are aligned or not.

[0058] In step S303, foreign matter detection is performed based on the ridge line alignment result information.

[0059] Here, the input images are first prepared. The spatially aligned first fingerprint image (the fingerprint provided in real time) and the second fingerprint image (the pre-recorded fingerprint) are used as a set of input images to be fed into the subsequent processing flow. This set of input images is then fed into a pre-trained target model. This target model can be an algorithm based on deep learning, machine learning, or other image processing techniques, specifically designed to handle fingerprint image alignment and foreign object detection tasks. The target model first determines whether the fingerprint lines of the first and second fingerprint images meet the alignment criteria. This determination process is based on various techniques such as image matching and feature point alignment. The alignment result information is output as a binary classification result: "aligned" or "misaligned." This binary classification result helps us determine whether the two sets of fingerprint images have achieved a sufficient degree of spatial matching.

[0060] If the alignment result information is "aligned", it indicates that the two sets of fingerprint images have been well matched in terms of spatial position, and foreign object detection can be further performed. Foreign object detection involves comparing the detailed features of the two sets of fingerprint images, such as the continuity, width, and direction of the ridges, to identify any possible foreign objects or anomalies. If a foreign object or anomaly is detected in the aligned image (such as a tampered fingerprint or the addition of a foreign object), the system can reject the unlock request or authentication request for that fingerprint.

[0061] The above method, through the alignment judgment of the target model, can ensure the precise alignment of the two sets of fingerprint images before the subsequent foreign object detection, thereby improving the accuracy of foreign object detection. Foreign object detection based on alignment can be more efficient and accurate because any minor anomalies or foreign objects can be more easily identified in the precisely aligned images.

[0062] In some embodiments, obtaining ridge line alignment result information based on the target model includes:

[0063] extracting a first feature point from the first fingerprint image, and extracting a second feature point from the second fingerprint image;

[0064] Determining a feature vector based on the first feature point and the second feature point; wherein the feature vector represents an alignment state between the first fingerprint image and the second fingerprint image;

[0065] Classification processing is performed on the feature vector, and the grain line alignment result information is output based on the classification result.

[0066] Feature points are usually the intersections, endpoints, or other locations with significant features of the ridges in the fingerprint image.

[0067] Based on these two sets of feature points, a feature vector can be determined. This feature vector can contain various information, such as the location, direction, and distance of the feature points, and is used to characterize the alignment between the first and second fingerprint images. Determining the feature vector involves matching feature points and calculating a transformation matrix to ensure that the two sets of feature points can achieve optimal alignment under a certain transformation.

[0068] Next, ResNet (residual network) can be used for classification processing. ResNet is a deep convolutional neural network with powerful feature extraction and classification capabilities. In the embodiment of this application, it is used to process the alignment judgment task of fingerprint images.

[0069] When the feature vector is input into ResNet, ResNet performs forward propagation calculations based on the input feature vector and outputs an alignment result information. This alignment result information is a binary classification result, namely "aligned" or "misaligned".

[0070] The above-mentioned method uses a method of extracting feature points from fingerprint images and performing classification processing based on feature vectors using a deep convolutional neural network (such as ResNet), which significantly improves the accuracy and robustness of fingerprint image alignment judgment, while improving processing efficiency and reducing dependence on hardware resources. It is not only suitable for the field of fingerprint recognition, but also has the potential to be expanded to other image alignment and classification processing application scenarios.

[0071] In some embodiments, see Figure 4 , Figure 4 This is a flow chart of steps S401-S403 provided in an embodiment of the present application. When the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, the method further includes steps S401-S403, which will be explained in combination with each step.

[0072] In step S401, an image region in the first fingerprint image that is not aligned with the second fingerprint image is determined based on the ridge alignment result information.

[0073] In step S402, a target pattern is determined based on the image area.

[0074] In step S403, the target pattern is compared with the foreign object database. If there is a matching comparison result in the foreign object database, the comparison result is determined as the foreign object pattern; if there is no matching comparison result in the foreign object database, the target pattern is recorded.

[0075] Here, in the fingerprint recognition system, when possible foreign object interference is detected in the first fingerprint image, in order to further analyze and process this situation, the system needs to identify and extract the fingerprint line parts that are not aligned with the second fingerprint image (template) due to foreign object interference. These misaligned line areas are likely to contain evidence of foreign object coverage or tampering. The system can use image processing techniques (such as edge detection, morphological operations, etc.) to highlight these areas and use them as target patterns for further analysis. Next, the system will compare the extracted target pattern with a pre-established foreign object database. This foreign object database should contain patterns and features of various known foreign objects (such as tape, stains, scratches, etc.). By comparing the target pattern with the entries in the database, the system can try to find the type of foreign object that matches it.

[0076] If a match is found, the system identifies it as a foreign object pattern and may take appropriate action, such as issuing a warning, denying the fingerprint recognition request, or logging the abnormality. If no match is found, the system records the target pattern and may add it to a candidate list in the foreign object database for future analysis and updates. The system may also take other actions to further verify or investigate the unknown target pattern.

[0077] Regardless of whether a matching foreign object pattern is found, the system should record the results of the detection and may generate a report for subsequent analysis or auditing. If a new foreign object type or pattern is discovered, the system should consider updating the foreign object database so that it can more accurately identify and handle similar situations in the future. In some cases, the system will also need to notify the user of the foreign object interference detection result and may require the user to provide a new fingerprint sample or take other identity verification measures.

[0078] The above approach, by identifying misaligned image areas based on ridge alignment results, can precisely locate portions of the fingerprint image that may be affected by foreign object interference. This step not only improves foreign object detection accuracy but also reduces subsequent processing costs caused by false positives, thereby enhancing the efficiency of the entire fingerprint recognition system. Secondly, by determining target patterns based on image regions and comparing them with a foreign object database, known foreign object types can be quickly identified. This step not only facilitates rapid response and resolution of foreign object interference but also provides strong evidence for subsequent analysis and audits. Furthermore, when no matching comparison results exist in the foreign object database, the target pattern is recorded and included in the candidate list, facilitating analysis and identification of new foreign object types. This step not only enhances scalability and flexibility but also provides a foundation for continuous optimization and updates. Finally, regardless of whether a matching foreign object pattern is found, the detection results are recorded and a report is generated, facilitating subsequent analysis and audits. Furthermore, updating the foreign object database when new foreign object types are discovered ensures more accurate identification and resolution of similar cases in the future, thereby improving the system's adaptability and robustness.

[0079] In some embodiments, the performing foreign body detection based on the grain alignment result information includes:

[0080] When the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, the foreign matter extraction model is used to determine whether there is a foreign matter in the first fingerprint image. If so, the foreign matter in the first fingerprint image is extracted based on the foreign matter extraction model to obtain the first fingerprint image after the foreign matter is removed; wherein the foreign matter extraction model is trained based on a foreign matter database and normal fingerprint images.

[0081] Here, whether the fingerprint lines of the first fingerprint image and the second fingerprint image are aligned is determined based on the line alignment result information. If the lines are not aligned, the foreign body detection process is triggered, that is, the previously trained foreign body extraction model is used for further judgment. The foreign body extraction model is used to determine whether there is a foreign body in the first fingerprint image. If the model determines that there is a foreign body, the model is further used to extract the foreign body in the first fingerprint image.

[0082] After removing the foreign object, the first fingerprint image needs to be repaired to fill the blank areas left by the foreign object to ensure the accuracy of fingerprint recognition. Finally, fingerprint recognition is performed based on the first fingerprint image after removing the foreign object and repairing it to verify whether it matches the pre-recorded second fingerprint image.

[0083] The foreign body extraction model is trained using a foreign body database (containing fingerprint images of various known foreign bodies) and normal fingerprint images as training data. The goal of the model is to be able to identify whether there are foreign bodies in the fingerprint image and to accurately extract foreign bodies when they are present.

[0084] The above approach triggers the foreign object detection process using the ridge alignment results, accurately locating fingerprint images that may contain foreign objects, avoiding unnecessary detection and processing, and thus improving the efficiency of foreign object detection. Secondly, using a trained foreign object extraction model for foreign object identification and extraction effectively identifies foreign objects in fingerprint images and accurately extracts the foreign object area. This not only improves the accuracy of foreign object detection but also provides strong support for subsequent fingerprint image restoration. Furthermore, restoration processing is performed on the fingerprint image after the foreign object is removed to fill the blank areas left by the foreign object, ensuring the accuracy of fingerprint recognition. Furthermore, the foreign object extraction model is trained based on a foreign object database and normal fingerprint images, which gives it strong generalization and adaptability. With the continuous updating and expansion of the foreign object database, the model can identify and process more types of foreign objects, thereby improving the robustness and scalability of the system. Finally, fingerprint recognition is performed based on the fingerprint image after the foreign object is removed and restored, ensuring recognition accuracy and reliability. This step not only improves the success rate of fingerprint recognition but also provides strong support for subsequent identity verification and authorization.

[0085] In summary, the embodiments of the present application have the following beneficial effects:

[0086] By simultaneously inputting the real-time fingerprint image provided by the user during unlocking (the first fingerprint image) and the fingerprint image pre-registered by the system (the second fingerprint image), a richer data base is provided, making it easier to distinguish the presence of foreign object interference during detection. Specifically, the two fingerprint images are first spatially aligned to ensure their precise correspondence during comparison. Subsequently, foreign object detection analysis is performed based on the aligned images. Because dual-image input provides more feature information, it can more accurately identify differences between fingerprints or foreign object patterns. This improved difference recognition capability enables rapid and accurate judgment in various complex and changing unlocking scenarios, effectively preventing false recognition or rejection caused by foreign object interference. In addition, the dual-image input strategy helps learn more feature information about fingerprints and foreign objects, further improving resolution and generalization performance. This enables strong adaptability and accuracy even when dealing with unknown or new foreign object patterns. Therefore, the application of the dual-image input strategy in fingerprint recognition technology, especially in foreign object detection, significantly improves accuracy, resolution, and generalization performance, providing a strong guarantee for the security and reliability of fingerprint recognition.

[0087] Based on the same inventive concept, the embodiments of the present application also provide a fingerprint foreign body detection device corresponding to the fingerprint foreign body detection method in the first embodiment. Since the principle of solving the problem by the device in the embodiments of the present application is similar to that of the above-mentioned fingerprint foreign body detection method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0088] like Figure 5 As shown, Figure 5 : is a schematic diagram of the structure of the fingerprint foreign body detection device 500 provided in an embodiment of the present application. The fingerprint foreign body detection device 500 includes:

[0089] An acquisition module 501 is configured to acquire a first fingerprint image and a second fingerprint image, wherein the first fingerprint image is a fingerprint image provided during unlocking, and the second fingerprint image is a pre-recorded fingerprint image used to verify a matching degree with the first fingerprint image;

[0090] The detection module 502 is configured to perform spatial alignment processing on the first fingerprint image and the second fingerprint image, and use the spatially aligned first fingerprint image and the second fingerprint image as input images, and perform foreign body detection based on the input images.

[0091] Those skilled in the art should understand that Figure 5 The functions implemented by each unit in the fingerprint foreign body detection device 500 shown can be understood by referring to the relevant description of the aforementioned fingerprint foreign body detection method. Figure 5 The functions of the various units in the fingerprint foreign matter detection device 500 shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0092] In a possible implementation, the detection module 502 performs spatial alignment processing on the first fingerprint image and the second fingerprint image, including:

[0093] Acquire key points in the first fingerprint image and the second fingerprint image, and generate descriptors for the key points;

[0094] determining matching key point pairs in the first fingerprint image and the second fingerprint image based on the descriptor;

[0095] Based on the key point pair, spatial alignment processing is performed on the first fingerprint image and the second fingerprint image, so that the spatial positions of the first fingerprint image and the second fingerprint image are consistent.

[0096] In a possible implementation, the detection module 502 uses the spatially aligned first fingerprint image and the second fingerprint image as input images, and performs foreign object detection based on the input images, including:

[0097] Inputting the input image into a target model;

[0098] Obtaining ridge alignment result information based on the target model; wherein the ridge alignment result information is a binary classification result, and the binary classification result indicates whether the fingerprint ridges of the first fingerprint image and the second fingerprint image are aligned or not;

[0099] Foreign matter detection is performed based on the grain line alignment result information.

[0100] In a possible implementation, the detection module 502 obtains the ridge alignment result information based on the target model, including:

[0101] extracting a first feature point from the first fingerprint image, and extracting a second feature point from the second fingerprint image;

[0102] Determining a feature vector based on the first feature point and the second feature point; wherein the feature vector represents an alignment state between the first fingerprint image and the second fingerprint image;

[0103] Classification processing is performed on the feature vector, and the grain line alignment result information is output based on the classification result.

[0104] In a possible implementation, when the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, the detection module 502 further includes:

[0105] determining an image area in the first fingerprint image that is not aligned with the second fingerprint image based on the ridge alignment result information;

[0106] determining a target pattern based on the image area;

[0107] The target pattern is compared with a foreign body database. If a matching comparison result exists in the foreign body database, the comparison result is determined as a foreign body pattern. If a matching comparison result does not exist in the foreign body database, the target pattern is recorded.

[0108] In a possible implementation, the detection module 502 obtains key points in the first fingerprint image and the second fingerprint image, and generates descriptors for the key points, including:

[0109] Acquire key points in the first fingerprint image and the second fingerprint image using a key point detection algorithm, wherein the key point detection algorithm includes at least one of SIFT and ORB;

[0110] For the key point, local features of the key point are obtained through a descriptor algorithm, and the descriptor is generated based on the local features.

[0111] In a possible implementation, the detection module 502 performs spatial alignment processing on the first fingerprint image and the second fingerprint image based on the key point pair, including:

[0112] Calculating a transformation matrix from the first fingerprint image to the second fingerprint image based on the key point pairs, wherein the transformation matrix is used to perform an affine transformation or a perspective transformation;

[0113] The transformation matrix is applied to the first fingerprint image.

[0114] In a possible implementation, the detection module 502 performs foreign body detection based on the grain alignment result information, including:

[0115] A foreign body extraction model is trained based on a foreign body database and a normal fingerprint image; wherein the foreign body extraction model is used to determine whether a foreign body is present in the first fingerprint image, and to extract the foreign body when a foreign body is present;

[0116] When the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, the foreign matter extraction model is used to determine whether there is a foreign matter in the first fingerprint image. If so, the foreign matter in the first fingerprint image is extracted based on the foreign matter extraction model to obtain the first fingerprint image after the foreign matter is removed.

[0117] The fingerprint foreign body detection device has the following beneficial effects:

[0118] By simultaneously inputting the real-time fingerprint image provided by the user during unlocking (the first fingerprint image) and the fingerprint image pre-registered by the system (the second fingerprint image), a richer data base is provided, making it easier to distinguish the presence of foreign object interference during detection. Specifically, the two fingerprint images are first spatially aligned to ensure their precise correspondence during comparison. Subsequently, foreign object detection analysis is performed based on the aligned images. Because dual-image input provides more feature information, it can more accurately identify differences between fingerprints or foreign object patterns. This improved difference recognition capability enables rapid and accurate judgment in various complex and changing unlocking scenarios, effectively preventing false recognition or rejection caused by foreign object interference. In addition, the dual-image input strategy helps learn more feature information about fingerprints and foreign objects, further improving resolution and generalization performance. This enables strong adaptability and accuracy even when dealing with unknown or new foreign object patterns. Therefore, the application of the dual-image input strategy in fingerprint recognition technology, especially in foreign object detection, significantly improves accuracy, resolution, and generalization performance, providing a strong guarantee for the security and reliability of fingerprint recognition.

[0119] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present application. The electronic device 600 includes:

[0120] A processor 601, a storage medium 602, and a bus 603. The storage medium 602 stores machine-readable instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the storage medium 602 via the bus 603. The processor 601 executes the machine-readable instructions to perform the steps of the fingerprint foreign body detection method described in the embodiment of the present application.

[0121] In actual application, the various components in the electronic device 600 are coupled together via bus 603. It is understood that bus 603 is used to realize the connection and communication between these components. In addition to the data bus, bus 603 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus 603.

[0122] The electronic device has the following beneficial effects:

[0123] By simultaneously inputting the real-time fingerprint image (first fingerprint image) provided by the user when unlocking the phone and the fingerprint image pre-registered by the system (second fingerprint image), a richer data base can be provided, making it easier to distinguish whether there is a foreign object interference during detection. Specifically, the two fingerprint images are first spatially aligned to ensure that they can accurately correspond when compared. Subsequently, foreign object detection analysis is performed based on the aligned images. Because dual-image input provides more feature information, it can more accurately identify the differences between fingerprints or foreign object patterns. This improved difference recognition ability enables rapid and accurate judgment in various complex and changing unlocking scenarios, effectively preventing misidentification or rejection caused by foreign object interference. In addition, the dual-image input strategy helps to learn more feature information about fingerprints and foreign objects, further improving resolution and generalization performance. This enables strong adaptability and accuracy even when facing unknown or new foreign object patterns. Therefore, the application of the dual-image input strategy in fingerprint recognition technology, especially in foreign object detection, significantly improves accuracy, resolution, and generalization performance, providing a strong guarantee for the security and reliability of fingerprint recognition.

[0124] The embodiment of the present application further provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by at least one processor 601, the fingerprint foreign body detection method described in the embodiment of the present application is implemented.

[0125] In some embodiments, the storage medium may be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), or a programmable Memory), Erasable Programmable Read-Only Memory (EPROM, ErasableProgrammable Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Memory), Flash Memory, Magnetic Surface Storage, Optical Disc, or CD-ROM ( Compact Disc Memory) and other memories; it can also be various devices including one or any combination of the above memories.

[0126] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0127] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0128] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0129] The computer-readable storage medium has the following advantages:

[0130] By simultaneously inputting the real-time fingerprint image provided by the user during unlocking (the first fingerprint image) and the fingerprint image pre-registered by the system (the second fingerprint image), a richer data base is provided, making it easier to distinguish the presence of foreign object interference during detection. Specifically, the two fingerprint images are first spatially aligned to ensure their precise correspondence during comparison. Subsequently, foreign object detection analysis is performed based on the aligned images. Because dual-image input provides more feature information, it can more accurately identify differences between fingerprints or foreign object patterns. This improved difference recognition capability enables rapid and accurate judgment in various complex and changing unlocking scenarios, effectively preventing false recognition or rejection caused by foreign object interference. In addition, the dual-image input strategy helps learn more feature information about fingerprints and foreign objects, further improving resolution and generalization performance. This enables strong adaptability and accuracy even when dealing with unknown or new foreign object patterns. Therefore, the application of the dual-image input strategy in fingerprint recognition technology, especially in foreign object detection, significantly improves accuracy, resolution, and generalization performance, providing a strong guarantee for the security and reliability of fingerprint recognition.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0132] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0134] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, platform server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0135] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A fingerprint foreign body detection method, characterized in that: The method comprises: Acquire a first fingerprint image and a second fingerprint image, wherein the first fingerprint image is a fingerprint image provided during unlocking, and the second fingerprint image is a pre-recorded fingerprint image used to verify the matching degree of the first fingerprint image; The first fingerprint image and the second fingerprint image are spatially aligned, and the spatially aligned first fingerprint image and the second fingerprint image are used as input images, and foreign body detection is performed based on the input images.

2. The method according to claim 1, characterized in that The performing spatial alignment processing on the first fingerprint image and the second fingerprint image includes: Acquire key points in the first fingerprint image and the second fingerprint image, and generate descriptors for the key points; determining matching key point pairs in the first fingerprint image and the second fingerprint image based on the descriptor; Based on the key point pair, spatial alignment processing is performed on the first fingerprint image and the second fingerprint image, so that the spatial positions of the first fingerprint image and the second fingerprint image are consistent.

3. The method according to claim 1 or 2, characterized in that The step of using the spatially aligned first fingerprint image and the second fingerprint image as input images and performing foreign body detection based on the input images includes: Inputting the input image into a target model; Grain alignment result information is obtained based on the target model; wherein the grain alignment result information is a binary classification result, and the binary classification result represents whether the fingerprint grains of the first fingerprint image and the second fingerprint image are aligned or not; and foreign body detection is performed based on the grain alignment result information.

4. The method according to claim 3, characterized in that The obtaining of ridge line alignment result information based on the target model includes: extracting a first feature point from the first fingerprint image, and extracting a second feature point from the second fingerprint image; Determining a feature vector based on the first feature point and the second feature point; wherein the feature vector represents an alignment state between the first fingerprint image and the second fingerprint image; Classification processing is performed on the feature vector, and the grain line alignment result information is output based on the classification result.

5. The method according to claim 3, characterized in that When the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, the method further includes: determining an image area in the first fingerprint image that is not aligned with the second fingerprint image based on the ridge alignment result information; determining a target pattern based on the image area; The target pattern is compared with a foreign body database. If a matching comparison result exists in the foreign body database, the comparison result is determined as a foreign body pattern. If a matching comparison result does not exist in the foreign body database, the target pattern is recorded.

6. The method according to claim 2, characterized in that The acquiring key points in the first fingerprint image and the second fingerprint image, and generating descriptors for the key points, includes: Acquire key points in the first fingerprint image and the second fingerprint image using a key point detection algorithm, wherein the key point detection algorithm includes at least one of SIFT and ORB; For the key point, local features of the key point are obtained through a descriptor algorithm, and the descriptor is generated based on the local features.

7. The method according to claim 2, characterized in that The performing spatial alignment processing on the first fingerprint image and the second fingerprint image based on the key point pair includes: Calculating a transformation matrix from the first fingerprint image to the second fingerprint image based on the key point pairs, wherein the transformation matrix is used to perform an affine transformation or a perspective transformation; The transformation matrix is applied to the first fingerprint image.

8. The method according to claim 3, characterized in that The performing foreign body detection based on the grain line alignment result information includes: When the ridge alignment result information indicates that the fingerprint ridges of the first fingerprint image and the second fingerprint image are not aligned, a foreign matter extraction model is used to determine whether there is a foreign matter in the first fingerprint image. If so, the foreign matter in the first fingerprint image is extracted based on the foreign matter extraction model to obtain the first fingerprint image after the foreign matter is removed; wherein the foreign matter extraction model is trained based on a foreign matter database and normal fingerprint images.

9. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the fingerprint foreign body detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the fingerprint foreign body detection method according to any one of claims 1 to 8 is executed.

Images