An under-screen fingerprint identification method and device, an electronic device and a storage medium

Abstract

Embodiments of the present application disclose a screen-under fingerprint identification method and device, electronic equipment and storage medium. The method comprises: receiving an image to be identified sent by a client; inputting the image to be identified into a pre-trained fingerprint separation model to obtain a target image corresponding to the image to be identified; and sending the target image to the client to enable the client to identify the target image. The method of the embodiments of the present application can use the trained fingerprint separation model to quickly and accurately obtain a high-quality fingerprint enhanced image, improve the efficiency and accuracy of the client in identifying user fingerprint information, and further improve the user experience.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent question answering, and more particularly to an under-display fingerprint recognition method, device, electronic device, and storage medium. Background Technology

[0002] Under-display fingerprint technology, also known as "invisible fingerprint technology," places the fingerprint sensor under the screen. Common under-display fingerprint technologies include optical fingerprint technology, ultrasonic fingerprint technology, and capacitive fingerprint technology. Optical fingerprint technology includes under-display fingerprint technology for organic light-emitting diode (OLED) display panels and under-display fingerprint technology for traditional liquid crystal display panels.

[0003] Existing under-display fingerprint technology utilizes the self-emissive properties of organic light-emitting diode (OLED) display panels to individually control the illumination of each pixel, generating different fingerprint images by passing through the gaps in the self-emissive pixel array. However, this method has high requirements for screen materials, and the quality of the extracted fingerprint images is relatively low, resulting in inaccurate fingerprint recognition by the device and reducing the efficiency of under-display fingerprint recognition. Summary of the Invention

[0004] This invention provides an under-display fingerprint recognition method, device, electronic device, and storage medium, which can accurately output high-quality fingerprint images from images to be recognized using a trained fingerprint separation model, thereby improving the efficiency of under-display fingerprint recognition.

[0005] In a first aspect, embodiments of the present invention provide an under-display fingerprint recognition method, the method comprising:

[0006] Receive the image to be recognized sent by the client;

[0007] The image to be identified is input into a pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified.

[0008] The target image is sent to the client, enabling the client to recognize the target image.

[0009] Secondly, embodiments of the present invention also provide an under-display fingerprint recognition device, the device comprising:

[0010] The receiving module is used to receive the image to be recognized sent by the client;

[0011] The input module is used to input the image to be identified into a pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified;

[0012] The recognition module is used to send the target image to the client, so that the client can recognize the target image.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Memory, used to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the under-display fingerprint recognition method provided in any embodiment of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the under-display fingerprint recognition method as provided in any embodiment of the present invention.

[0018] In this embodiment of the invention, an image to be identified is received from a client; the image to be identified is input into a pre-trained fingerprint separation model to obtain a target image corresponding to the image to be identified; the target image is sent to the client, enabling the client to identify the target image. That is, in this embodiment of the invention, a trained fingerprint separation model can be used to quickly and accurately obtain a high-quality fingerprint enhancement image, improving the efficiency and accuracy of the client in identifying user fingerprint information, and further enhancing the user experience. Attached Figure Description

[0019] Figure 1 This is a flowchart of the under-display fingerprint recognition method provided in an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of the training method for the fingerprint separation model provided in this embodiment of the invention;

[0021] Figure 3 This is a schematic diagram of an image synthesis mechanism provided in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram illustrating the process of obtaining output information as provided in an embodiment of the present invention;

[0023] Figure 5 This is a front view of the POS machine provided in an embodiment of the present invention;

[0024] Figure 6 This is a left view of the POS machine provided in an embodiment of the present invention;

[0025] Figure 7 This is a rear view of the POS machine provided in an embodiment of the present invention;

[0026] Figure 8This is a schematic diagram of the detachable invoice printing plug for the POS machine provided by the present invention.

[0027] Figure 9 This is a schematic diagram of the structure of the under-display fingerprint recognition device provided in an embodiment of the present invention;

[0028] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0030] Figure 1 This is a flowchart of an under-display fingerprint recognition method provided in this embodiment of the invention. The method in this embodiment can accurately output a high-quality fingerprint image from the image to be recognized using a trained fingerprint separation model, thus improving the efficiency of under-display fingerprint recognition. This method can be executed by the under-display fingerprint recognition device in this embodiment of the invention. This device can be integrated into an electronic device, which can be a server. The method can be implemented using software and / or hardware. The under-display fingerprint recognition method provided in this embodiment specifically includes the following steps:

[0031] Step 101: Receive the image to be recognized sent by the client.

[0032] The client device can be a mobile phone, POS machine, tablet computer, or other device with an organic light-emitting diode (OLED) display screen. The image to be identified is an image carrying the user's fingerprint information. A fingerprint sensor is a sensing device, belonging to the semiconductor fingerprint sensing device category of optical fingerprint sensing technology, and is a key component for achieving automatic fingerprint acquisition. In one optional implementation, the client device is equipped with a fingerprint sensor, which sequentially captures fingerprint image strips through a wireless fingerprint sensor to obtain an image carrying the user's fingerprint information, i.e., the image to be identified. After identifying the image to be identified, the client device can send it to the server, which can then receive the image.

[0033] Step 102: Input the image to be identified into the pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified.

[0034] The target image is the image to be processed by the fingerprint separation model. The fingerprint separation model is a pre-trained model used to process the image to be identified.

[0035] Specifically, the fingerprint separation model adopts a "fingerprint-background-mask" modeling approach. The image to be identified is considered to consist of three parts: a background image, a fingerprint image, and a mask image. By superimposing each pixel in the fingerprint image and the background image according to different mask values, the pixel value in the image to be identified can be obtained. The superposition formula is shown in formula (1):

[0036] I(x)=m(x)y1(x)+(1-m(x))y2(x) (1)

[0037] Where I(x) represents the image to be identified, m(x) represents the mask image, y1(x) represents the fingerprint image, and y2(x) represents the background image. The goal of the fingerprint separation model is to make the target image output by the model as close as possible to the image to be processed.

[0038] In this embodiment of the solution, optionally, obtaining the trained fingerprint separation model includes the following steps A1-A2:

[0039] Step A1: If the fingerprint separation model does not meet the pre-set convergence conditions, then extract a sample from the image sample library as the current sample.

[0040] The current sample includes a background image, a fingerprint image, and a mask image. The convergence condition is a pre-set condition that the fingerprint separation model must meet based on the experimental environment and specific requirements. In this scheme, the convergence condition of the fingerprint separation model is that the error value between the target image and the image to be processed is less than a pre-set error value. The image sample library stores a large number of images with various fingerprint information and various background images. The current sample is an image randomly selected from the image sample library. When the fingerprint separation model meets the pre-set convergence condition, it means that the synthesized image output by the fingerprint separation model is close enough to the input image. When the fingerprint separation model does not meet the convergence condition, it means that the synthesized image output by the fingerprint separation model is not close enough to the input image, i.e., the fingerprint separation model does not meet the pre-set convergence condition. In this case, an image is randomly selected from the image sample library as the current sample, and the current sample is input into the untrained fingerprint separation model.

[0041] Step A2: Train the fingerprint separation model using the current sample. Repeat the above operation until the fingerprint separation model meets the convergence condition.

[0042] The fingerprint separation model comprises two image prior networks. The image prior network can generate information not present in the current sample using its own generated information and the current sample. The main idea of ​​the image prior network is to perform "enhancement-operation-subtraction" on the image. Through the image prior network, the fingerprint image and mask image corresponding to the current sample can be obtained from the current sample.

[0043] In one optional implementation, the image prior network includes at least a deconvolution layer. The current sample is input to the deconvolution layer, which upsamples the current sample to obtain a sampled image corresponding to the current sample. The sampled image is then denoised using a pre-determined denoising function to obtain a candidate fingerprint image and a candidate mask image corresponding to the current sample. The candidate fingerprint image and candidate mask image are then synthesized with the current sample to obtain a fingerprint-enhanced sample and a mask-enhanced sample corresponding to the current sample. The current sample is then removed from the fingerprint-enhanced sample and mask-enhanced sample to obtain the fingerprint image and mask image corresponding to the current sample. Finally, the mask image, fingerprint image, and a pre-determined background image are synthesized to obtain a synthesized image corresponding to the current sample, and this synthesized image is determined as the output information corresponding to the current sample.

[0044] Furthermore, the reconstruction loss function of the fingerprint separation model is determined based on the synthesized image and the current sample; the internal loss function of the fingerprint separation model is determined based on the mask image; the rejection loss function of the fingerprint separation model is determined based on the fingerprint image and the background image; and the target loss function of the fingerprint separation model is determined based on the reconstruction loss function, the internal loss function, and the rejection loss function. After calculating the target loss function of the fingerprint separation model, the model parameters in the text location determination model are adjusted based on the target loss function until the fingerprint separation model meets the pre-set convergence conditions.

[0045] In the above steps, the fingerprint separation model can be repeatedly trained until it reaches the convergence condition, which reduces the error between the synthetic image output by the fingerprint separation model and the current sample, laying the foundation for the client to quickly identify the user's fingerprint information.

[0046] Step 103: Send the target image to the client so that the client can recognize the target image.

[0047] The target image is the image processed by the fingerprint separation model. Specifically, when the client needs to perform fingerprint recognition on a user, the user presses their finger at the fingerprint sensor. The client obtains the image to be processed containing the user's fingerprint information and sends it to the server. The server inputs the image to be processed into the trained fingerprint separation model, which performs an "enhancement-computation-synthesis" operation on the image to obtain the target image, which is then returned to the client. After receiving the target image, the client identifies the fingerprint information within it and obtains the recognition result.

[0048] In this embodiment, the technical solution involves receiving an image to be identified from a client; inputting the image to be identified into a pre-trained fingerprint separation model to obtain a target image corresponding to the image to be identified; and sending the target image to the client so that the client can identify the target image. This embodiment utilizes a pre-trained fingerprint separation model to quickly and accurately obtain high-quality fingerprint enhancement images, improving the efficiency and accuracy of the client's fingerprint recognition and further enhancing the user experience.

[0049] Figure 2 This is a flowchart of the training method for the fingerprint separation model provided in this embodiment of the invention, as shown below. Figure 2 As shown, the method mainly includes the following steps:

[0050] Step 201: Extract a sample from the image sample library as the current sample, and perform denoising processing on the current sample according to the predetermined image denoising algorithm to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample.

[0051] Image denoising algorithms are used to reduce noise in digital images. Commonly used image denoising algorithms include filter-based denoising algorithms, model-based denoising algorithms, and learning-based denoising algorithms.

[0052] In one alternative implementation, the fingerprint separation model includes an image prior network. An image prior network is a network that improves image quality by bridging the gap between local image patches and the global repaired image. The image prior network can reduce the difference between local image patches and the global image, thus improving image quality. The image prior network considers the difference in denoising results between individual image patches and the global image as a "difference." By reducing this difference, the image prior network increases the number of identical image patches and uses them as new input to the iterative algorithm, thereby sharing local results, reducing the difference between local and global images, and compensating for missing image details, as shown in formula (2).

[0053]

[0054] in, Let f(·) represent the denoised image at the k-th iteration, and f(·) represent the image denoising algorithm. This represents the result of combining the input image with the denoised image.

[0055] Specifically, a sample is extracted from the image sample library as the current sample. The current sample is input into the image prior network, which performs denoising processing on the current sample according to a pre-determined image denoising algorithm to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample. In this embodiment, optionally, performing denoising processing on the current sample according to the pre-determined image denoising algorithm to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample includes: inputting the current sample into a deconvolution layer, upsampling the current sample using the deconvolution layer to obtain a sampled image corresponding to the current sample; and denoising the sampled image based on a pre-determined denoising function to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample.

[0056] The deconvolutional layer, also known as the transposed convolutional layer, can enlarge the size of the current sample. Upsampling refers to restoring the resolution of an image to the resolution of the original image.

[0057] In one alternative implementation, the image prior network uses convolutional layers as encoders and deconvolutional layers as decoders. In this scheme, the decoder in the deep image prior network is considered an image reconstruction module, and the image upsampling process is considered an image enhancement (denoising) process. Based on the idea of ​​the image prior network, in the image denoising of the k-th layer, the features q of the current sample from the previous layer are first processed... k+1 Upsampling is performed to obtain (q) k+1 ) ↑2 Then use the latent features p of the same level in the encoder k Enhance it. Denoising the sampled image based on the denoising function can be expressed as: It indicates that it has a parameter θ k The denoising algorithm, where ↑2 represents an upsampling operator with a scaling factor of 2.

[0058] In the above steps, the image to be processed can be denoised to obtain candidate fingerprint samples and candidate mask samples, which lays the foundation for subsequent image synthesis steps.

[0059] Step 202: Combine the candidate fingerprint image and the candidate mask image with the current sample to obtain the fingerprint enhancement sample and mask enhancement sample corresponding to the current sample.

[0060] Image synthesis maximizes the extraction of useful information from each channel, ultimately synthesizing a high-quality image. By synthesizing the candidate fingerprint image and candidate mask image with the current sample, fingerprint enhancement samples and mask enhancement samples corresponding to the current sample can be obtained.

[0061] In one alternative implementation, the image prior network comprises a residual set consisting of three residual blocks. The latent features p in the encoder k The image contains differential information lost during downsampling. This differential information can be gradually fused into the latent features of the decoder through upsampling to compensate for the lost spatial information. Image prior networks can compensate for lost spatial information by reducing the differences between local and global images, and gradually fused these differences back into the network's latent features, which can better improve network performance. For example, while preserving fingerprint ridges, it can compensate for the absence of fingerprint ridges and valleys and suppress the generation of error artifacts.

[0062] For example, Figure 3 This is a schematic diagram of an image synthesis mechanism provided in an embodiment of the present invention. Figure 3 As shown, the structure includes 5 convolutional layers, 4 residual groups, 5 deconvolutional layers, and 4 lifting layers. The filter size of the convolutional and deconvolutional layers is set to 3×3 with a stride of 2. The convolutional kernel size in the lifting layers and residual groups is 3×3 with a stride of 1. The lifting layer is located between two deconvolutional layers in the decoder. In this scheme, dilated convolutions are used instead of ordinary convolutions used in traditional residual networks, with a dilation factor set to 2 to reduce resolution loss and better aggregate overall fingerprint information. For example, fine fingerprint texture information is preserved in the deep network, and the sampled image is denoised to obtain candidate fingerprint images and candidate mask images corresponding to the current sample. Further, the candidate fingerprint images and candidate mask images are synthesized with the current sample using the 4 residual groups to obtain fingerprint-enhanced samples and mask-enhanced samples corresponding to the current sample.

[0063] Step 203: Delete the current sample from the fingerprint enhancement sample and the mask enhancement sample to obtain the fingerprint image and mask image corresponding to the current sample.

[0064] Specifically, after obtaining the fingerprint enhancement sample and mask enhancement sample corresponding to the current sample, in order to reduce the loss of resolution and better aggregate the overall fingerprint information, the current sample can be deleted from the fingerprint enhancement sample and mask enhancement sample to obtain the image after extracting important information, namely the fingerprint image and mask image corresponding to the current sample.

[0065] Step 204: Combine the mask image, fingerprint image and pre-determined background image to obtain the composite image corresponding to the current sample, and determine the composite image as the output information corresponding to the current sample.

[0066] The background image can be predetermined based on the actual application scenario.

[0067] For example, when the client is a POS machine, the background images of in-display fingerprints on the same POS machine have similarities. The initial background image can be obtained by the user pressing the fingerprint on the machine 50 times evenly and smoothly using different angles and fingers. Further, the average of these 50 images is calculated to obtain an average background image. Specifically, a fingerprint image can be composed of a fingerprint image and a background image. Each press produces a different fingerprint pattern, but the background image remains the same. Therefore, after multiple presses, the fingerprint texture can evenly cover the entire acquisition area, and the average is calculated to obtain an average background image. In this solution, to make the background image more suitable for each client, a comprehensive fingerprint scoring function is used to score the quality of the extracted fingerprint. If the fingerprint quality score exceeds 80 points (out of 100), the background image corresponding to that fingerprint is mixed again with the original fingerprint image stored in the POS machine. This allows each POS machine to make certain modifications based on its own characteristics during use.

[0068] After determining the background image, the mask image, fingerprint image, and pre-determined background image are combined to obtain the composite image corresponding to the current sample, and the composite image is determined as the output information corresponding to the current sample. Figure 4 This is a schematic diagram illustrating the obtained output information provided in an embodiment of the present invention. For example... Figure 4 As shown, the current sample is input into the image prior network to obtain the fingerprint image y1 and mask image m corresponding to the current sample, where y2 is a pre-determined background image. The mask image, fingerprint image, and pre-determined background image are combined to obtain the synthetic image corresponding to the current sample, and the synthetic image is determined as the output information corresponding to the current sample.

[0069] Step 205: Calculate the target loss function of the fingerprint separation model based on the output information corresponding to the current sample and the current sample, and adjust the model parameters in the text location determination model based on the loss function.

[0070] The loss function is a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. Loss functions are often used as learning criteria in relation to optimization problems; that is, the model is solved and evaluated by minimizing the loss function. For example, it is used for parameter estimation in statistics and machine learning. In this embodiment, optionally, the target loss function of the fingerprint separation model is calculated based on the output information corresponding to the current sample and the current sample, including the following steps B1 to B4:

[0071] Step B1: Determine the reconstruction loss function of the fingerprint separation model based on the synthetic image and the current sample.

[0072] Specifically, the reconstruction loss function is used to determine the distance between the input image and the synthesized image, so that the synthesized image reassembled from the separated image layers is as similar as possible to the input image. The reconstruction loss function is shown in formula (3):

[0073]

[0074] Among them, L re Let I be the reconstruction loss function, and let I be the synthesized image. This is the current sample.

[0075] Step B2: Determine the internal loss function of the fingerprint separation model based on the mask image.

[0076] Specifically, the internal loss function is used to detect the autocorrelation within the mask image, thus ensuring the mask image is smooth and continuous. The internal loss function increases the self-similarity within the mask image by minimizing the norm of its Laplacian operator. The internal loss function is shown in equation (4):

[0077]

[0078] Among them, L in (m) is the internal loss function. This indicates that filtering is applied to the synthesized image.

[0079] Step B3: Determine the rejection loss function of the fingerprint separation model based on the fingerprint image and background image.

[0080] Specifically, the repulsion loss is used to determine the distance between the generated fingerprint image B and the input background image P. The repulsion loss function results in lower similarity between the separated layers. The repulsion loss function is shown in formula (5):

[0081]

[0082] in, λ B and λ P These are the normalization parameters. ||·|| is the Frobenius norm. ⊙ represents pixel multiplication, f represents network downsampling, n represents downsampling at the nth layer, and N is the total number of downsampling layers.

[0083] Step B4: Determine the target loss function of the fingerprint separation model based on the reconstruction loss function, the internal loss function, and the exclusion loss function.

[0084] Specifically, the internal loss function and the repulsion loss function are combined with parameters, and then combined with the reconstruction loss function to form the target loss function. The target loss function is shown in formula (6):

[0085]

[0086] Where α = 0.1 and β = 0.05. Furthermore, the model parameters of the fingerprint separation model are adjusted based on the calculation results of the target loss function.

[0087] In the above steps, the reconstruction loss function is used to make the synthesized image reassembled from the separated image layers as similar as possible to the input image. The internal loss function makes the mask image smooth and continuous, and the repulsion loss function reduces the similarity between the separated layers. The target loss function, composed of the internal loss function, repulsion loss function, and reconstruction loss function, improves the accuracy of the target loss function calculation results and further enhances the quality of the synthesized image output by the fingerprint separation model.

[0088] In this solution, the client is optionally a POS machine. Figure 5 This is a front view of the POS machine provided in an embodiment of the present invention, such as... Figure 5 As shown, the front view of the POS machine includes a display screen and a power button. The display screen includes a touch sensor, a fingerprint sensor, and an electronic screen. Figure 6 This is a left view of the POS machine provided in an embodiment of the present invention. Figure 6 As shown, the left view of the POS machine includes a card slot and a power interface. The card slot is used to read bank card information. Figure 7 This is a rear view of the POS machine provided in an embodiment of the present invention. Figure 7 As shown, the rear view of the POS machine includes the interface for the invoice plugin, the slot for the invoice plugin, the QR code scanning camera, and the supplementary lighting strip. The slot for the invoice plugin ensures its stability. The supplementary lighting strip intelligently adjusts its brightness according to the current lighting conditions. Figure 8 This is a schematic diagram of the detachable invoice printing plug for a POS machine provided by [Invention Name]. Figure 8 As shown, the invoice plugin includes a USB interface, an invoice paper outlet, and an invoice paper replacement port. The USB interface can be connected to the invoice plugin's interface. The POS machine's controller can read bank cards, collect user fingerprints via the screen, and automatically generate electronic invoices. The POS machine includes a wireless sensing unit, which includes wireless communication devices such as Bluetooth. The POS machine can connect to the server via the wireless communication unit and send the collected data to the server.

[0089] The main operating scenarios for the POS machine provided in this solution are as follows:

[0090] Scenario 1: The user chooses to pay with a bank card.

[0091] After the user enters the payment amount on the display screen and inserts their bank card into the card slot, the controller verifies the user's identity. The display then shows the password entry page and a virtual keypad, below which is an in-display fingerprint reader. The user can choose to enter their password via the touchscreen or directly press their fingerprint. After successful authentication, the POS machine sends the payment request to the corresponding server for processing. The user can choose to receive an electronic invoice via SMS or print a paper receipt.

[0092] Scenario 2: User selects to enter bank number for payment

[0093] When a user chooses to pay by entering their bank account number, they can manually enter their bank card number and real name, and the POS machine will send the user information to the server.

[0094] Scenario 3: User chooses to pay with personal payment code

[0095] When a user makes a payment using their personal payment code, after setting the payment amount, the POS machine's QR code scanner scans the user's payment code and then deducts the payment.

[0096] The fingerprint separation model training method provided in this embodiment of the invention can extract a sample from an image sample library as the current sample, perform denoising processing on the current sample according to a pre-determined image denoising algorithm, and obtain candidate fingerprint images and candidate mask images for the current sample. The candidate fingerprint images and mask images are then synthesized with the current sample to obtain fingerprint enhancement samples and mask enhancement samples corresponding to the current sample. The current sample is then removed from the fingerprint enhancement samples and mask enhancement samples to obtain fingerprint images and mask images corresponding to the current sample. The mask image, fingerprint image, and a pre-determined background image are then synthesized to obtain a synthesized image corresponding to the current sample, and this synthesized image is determined as the output information corresponding to the current sample. Based on the output information corresponding to the current sample and the current sample, the target loss function of the fingerprint separation model is calculated, and the model parameters in the text location determination model are adjusted based on the loss function. The technical solution of this embodiment can reduce the difference between local image patches and the global image, compensate for missing image details, improve image quality, and improve the accuracy of the target loss function calculation results by using an internal loss function, a rejection loss function, and a reconstruction loss function, thereby further improving the processing efficiency of the fingerprint separation model for the images to be processed.

[0097] Figure 9 This is a schematic diagram of the structure of an under-display fingerprint recognition device provided in an embodiment of the present invention. The present invention provides an under-display fingerprint recognition device, the device comprising:

[0098] The receiving module 901 is used to receive the image to be recognized sent by the client;

[0099] Input module 902 is used to input the image to be identified into a pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified;

[0100] The recognition module 903 is used to send the target image to the client, so that the client can recognize the target image.

[0101] Optionally, before receiving the image to be identified sent by the client, the input module 902 is specifically used to: if the fingerprint separation model does not meet the pre-set convergence condition, extract a sample from the image sample library as the current sample; wherein, the current sample includes a background image, a fingerprint image, and a mask image;

[0102] The fingerprint separation model is trained using the current sample, and the above operation is repeated until the fingerprint separation model meets the convergence condition.

[0103] Optionally, the input module 902 is further configured to: input the current sample into the fingerprint separation model to obtain the output information corresponding to the current sample;

[0104] The target loss function of the fingerprint separation model is calculated based on the output information corresponding to the current sample and the current sample, and the model parameters in the text location determination model are adjusted based on the target loss function.

[0105] Optionally, the input module 902 is further configured to: perform denoising processing on the current sample according to a predetermined image denoising algorithm to obtain a candidate fingerprint image and a candidate mask image corresponding to the current sample;

[0106] The candidate fingerprint image and the candidate mask image are respectively combined with the current sample to obtain the fingerprint enhancement sample and the mask enhancement sample corresponding to the current sample;

[0107] The current sample is deleted from the fingerprint enhancement sample and the mask enhancement sample to obtain the fingerprint image and mask image corresponding to the current sample.

[0108] Optionally, the input module 902 is further configured to: input the current sample into the deconvolution layer, and use the deconvolution layer to upsample the current sample to obtain a sampling map corresponding to the current sample;

[0109] The sampled image is denoised based on a predetermined denoising function to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample.

[0110] Optionally, the input module 902 is further configured to: determine the reconstruction loss function of the fingerprint separation model based on the synthesized image and the current sample;

[0111] The internal loss function of the fingerprint separation model is determined based on the mask image;

[0112] The rejection loss function of the fingerprint separation model is determined based on the fingerprint image and the background image;

[0113] Based on the reconstruction loss function, the internal loss function, and the rejection loss function, the target loss function of the fingerprint separation model is determined.

[0114] The under-display fingerprint recognition device provided in this embodiment of the invention can execute the under-display fingerprint recognition method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0115] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, with reference to... Figure 10 It shows a schematic diagram of the structure of a computer system 12 suitable for implementing an electronic device according to embodiments of the present invention.

[0116] Figure 10 The illustrated electronic device is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0117] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0118] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0119] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 10 Not shown; usually referred to as a "hard drive"). Although Figure 10 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0120] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0121] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, in this embodiment, electronic device 12 and display 24 are not separate entities, but are embedded in a mirror, so that when the display surface of display 24 is not displayed, the display surface of display 24 and the mirror surface visually blend together. Moreover, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that although... Figure 10 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0122] The processing unit 16 executes various functional applications and under-display fingerprint recognition by running programs stored in the system memory 28. For example, it implements an under-display fingerprint recognition method provided in this embodiment of the invention: receiving an image to be recognized sent by a client; inputting the image to be recognized into a pre-trained fingerprint separation model to obtain a target image corresponding to the image to be recognized; and sending the target image to the client so that the client can recognize the target image.

[0123] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an under-display fingerprint recognition method as provided in all embodiments of this invention: receiving an image to be recognized sent by a client; inputting the image to be recognized into a pre-trained fingerprint separation model to obtain a target image corresponding to the image to be recognized; and sending the target image to the client, enabling the client to recognize the target image. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of a computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0124] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0125] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0126] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for under-screen fingerprint recognition, characterized in that, The method includes: Receive the image to be recognized sent by the client; The image to be identified is input into a pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified. The target image is sent to the client, enabling the client to recognize the target image; Before receiving the image to be identified from the receiving client, the method further includes: If the fingerprint separation model does not meet the pre-set convergence condition, a sample is extracted from the image sample library as the current sample; wherein, the current sample includes the background image, the fingerprint image, and the mask image; The fingerprint separation model is trained using the current sample. The process of repeatedly extracting a sample from the image sample library as the current sample and training the fingerprint separation model using the current sample continues until the fingerprint separation model meets the convergence condition. The step of training the fingerprint separation model using the current sample includes: The current sample is input into the fingerprint separation model to obtain the output information corresponding to the current sample; The target loss function of the fingerprint separation model is calculated based on the output information corresponding to the current sample and the current sample, and the model parameters in the fingerprint separation model are adjusted based on the target loss function. The fingerprint separation model includes an image prior network. The step of inputting the current sample into the fingerprint separation model to obtain the output information corresponding to the current sample includes: The current sample is input into the image prior network to obtain the fingerprint image and mask image corresponding to the current sample; The mask image, the fingerprint image, and the predetermined background image are combined to obtain the composite image corresponding to the current sample, and the composite image is determined as the output information corresponding to the current sample.

2. The method of claim 1, wherein, The step of inputting the current sample into the image prior network to obtain the fingerprint image and mask image corresponding to the current sample includes: The current sample is denoised according to a predetermined image denoising algorithm to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample. The candidate fingerprint image and the candidate mask image are respectively combined with the current sample to obtain the fingerprint enhancement sample and the mask enhancement sample corresponding to the current sample; The current sample is deleted from the fingerprint enhancement sample and the mask enhancement sample to obtain the fingerprint image and mask image corresponding to the current sample.

3. The method of claim 2, wherein, The image prior network includes deconvolution layers. The step of denoising the current sample according to a pre-determined image denoising algorithm to obtain the candidate fingerprint image and mask image corresponding to the current sample includes: The current sample is input into the deconvolution layer, and the current sample is upsampled using the deconvolution layer to obtain the sampling map corresponding to the current sample; The sampled image is denoised based on a predetermined denoising function to obtain the candidate fingerprint image and candidate mask image corresponding to the current sample.

4. The method of claim 1, wherein, The step of calculating the target loss function of the fingerprint separation model based on the output information corresponding to the current sample and the current sample includes: The reconstruction loss function of the fingerprint separation model is determined based on the synthesized image and the current sample; The internal loss function of the fingerprint separation model is determined based on the mask image; The rejection loss function of the fingerprint separation model is determined based on the fingerprint image and the background image; Based on the reconstruction loss function, the internal loss function, and the rejection loss function, the target loss function of the fingerprint separation model is determined.

5. An under-screen fingerprint identification device, characterized in that, The device includes: The receiving module is used to receive the image to be recognized sent by the client; The input module is used to input the image to be identified into a pre-trained fingerprint separation model to obtain the target image corresponding to the image to be identified; The recognition module is used to send the target image to the client, so that the client can recognize the target image; Before receiving the image to be identified sent by the client, the input module is specifically used to: if the fingerprint separation model does not meet the pre-set convergence condition, extract a sample from the image sample library as the current sample; wherein, the current sample includes the background image, the fingerprint image and the mask image; The fingerprint separation model is trained using the current sample. The process of repeatedly extracting a sample from the image sample library as the current sample and training the fingerprint separation model using the current sample continues until the fingerprint separation model meets the convergence condition. The input module is also used to: input the current sample into the fingerprint separation model to obtain the output information corresponding to the current sample; The target loss function of the fingerprint separation model is calculated based on the output information corresponding to the current sample and the current sample, and the model parameters in the fingerprint separation model are adjusted based on the target loss function. The fingerprint separation model includes an image prior network, and the input module is further configured to: input the current sample into the image prior network to obtain the fingerprint image and mask image corresponding to the current sample; The mask image, the fingerprint image, and the predetermined background image are combined to obtain the composite image corresponding to the current sample, and the composite image is determined as the output information corresponding to the current sample.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the under-display fingerprint recognition method as described in any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the under-display fingerprint recognition method as described in any one of claims 1-4.

Images