Data processing method and device of ultrasonic fingerprint image and ultrasonic fingerprint device

Abstract

This specification provides a data processing method, apparatus, and ultrasonic fingerprint device for ultrasonic fingerprint images. Before implementation, based on preset training rules and using fingerprint standard samples, a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image containing neither fixed nor dynamic pattern noise but simulating the style of a real ultrasonic fingerprint image can be constructed. Then, the first, second, and third sample images are used in combination to train a preset ultrasonic fingerprint denoising model. During implementation, after directly acquiring the first ultrasonic fingerprint image using the ultrasonic fingerprint device, the preset ultrasonic fingerprint denoising model can be used to process the first ultrasonic fingerprint image to remove fixed and / or dynamic pattern noise, resulting in a high-quality, high-precision ultrasonic fingerprint image.

Description

Technical Field

[0001] This manual pertains to the field of fingerprint recognition technology, and particularly relates to data processing methods, apparatus, and ultrasonic fingerprint devices for ultrasonic fingerprint images. Background Technology

[0002] The ultrasonic fingerprint images acquired by existing ultrasonic fingerprint acquisition equipment often contain a lot of noise, which affects the accuracy of subsequent fingerprint image data processing.

[0003] Based on existing methods, in order to deal with the aforementioned noise, users are often required to lift their fingers to perform an air map scan before fingerprint recognition. Then, the ultrasonic fingerprint image collected during fingerprint recognition is denoised based on the scanned air map. This results in problems such as cumbersome and complicated user operation and unsatisfactory denoising effect.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides a data processing method, apparatus, and ultrasonic fingerprint device for ultrasonic fingerprint images. By training and utilizing a preset ultrasonic fingerprint denoising model to process the directly acquired first ultrasonic fingerprint image, it can effectively simplify user operation and efficiently and accurately remove related noises such as fixed pattern noise and dynamic pattern noise from the ultrasonic fingerprint image, resulting in a high-quality and high-precision ultrasonic fingerprint image.

[0006] This specification provides a data processing method for ultrasonic fingerprint images, including: Acquire the first ultrasonic fingerprint image; The first ultrasonic fingerprint image is processed using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, so as to obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

[0007] In one embodiment, processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements includes: Acquire the acquisition association information related to the first ultrasonic fingerprint image; The target combined data is obtained by combining the first ultrasonic fingerprint image and the acquired associated information; Using the preset ultrasonic fingerprint denoising model based on the target combination data, fixed pattern noise and dynamic pattern noise are removed from the first ultrasonic fingerprint image to obtain a second ultrasonic fingerprint image that meets the requirements.

[0008] In one embodiment, the collected associated information includes at least one of the following: the collected temperature, the membrane material information of the ultrasonic fingerprint device, the ultrasonic emission frequency, and the ultrasonic receiving window; The ultrasonic emission frequency and ultrasonic receiving window are determined based on the acquisition temperature and / or the membrane material information of the ultrasonic fingerprint device.

[0009] In one embodiment, the preset ultrasonic fingerprint denoising model includes: a main network model based on a U-net network, and a physical auxiliary model related to acoustic wave coupling integrated into the main network model.

[0010] In one embodiment, the main network model includes multiple sub-denoising networks; Accordingly, the step of processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements includes: The acoustic coupling mode of the first ultrasonic fingerprint image is determined by using a pre-defined ultrasonic fingerprint denoising model and a physical auxiliary model. Using a preset ultrasonic fingerprint denoising model, a matching target sub-denoising network is determined from multiple sub-denoising networks based on the acoustic coupling mode of the first ultrasonic fingerprint image. The first ultrasonic fingerprint image is denoised using a target sub-denoising network to obtain a second ultrasonic fingerprint image that meets the requirements.

[0011] In one embodiment, the method further includes: Multiple fingerprint standard samples are constructed according to preset training rules; among them, at least one of the fingerprint standard samples is different in terms of texture, material, and size. Using an ultrasonic fingerprint device, multiple fingerprint standard samples are scanned and imaged under different experimental environments to obtain multiple first sample images of the multiple fingerprint standard samples; wherein, one experimental environment corresponds to a combination of acquired associated information; the first sample images contain fixed pattern noise; Add at least one interfering agent to multiple fingerprint standard samples to obtain fingerprint standard samples with added interfering agents; Using an ultrasonic fingerprint device, the fingerprint standard samples with added interference were scanned and imaged under different experimental environments to obtain multiple second sample images of multiple fingerprint standard samples; wherein, the second sample images contain fixed pattern noise and dynamic pattern noise; Obtain a standard sample image of a fingerprint standard sample; and perform degradation simulation processing on the standard sample image to simulate the style of a real ultrasonic fingerprint image to obtain multiple third sample images; Multiple sample image groups are constructed based on the first sample image, the second sample image, and the third sample image; Using multiple sample image groups, an initial ultrasonic fingerprint denoising model is trained to obtain a preset ultrasonic fingerprint denoising model that meets the requirements.

[0012] In one embodiment, the sample image group is further labeled with a combination of corresponding acquisition-related information.

[0013] In one embodiment, the interfering agent includes at least one of the following: dust, hair, or hand cream.

[0014] In one embodiment, the degradation simulation processing of the standard sample image includes: The standard sample image is processed using a preset style transfer model to simulate the style of a real ultrasonic fingerprint image, resulting in a simulated standard sample image. Gaussian blurring is performed on the simulated standard sample image to obtain the corresponding third sample image.

[0015] In one embodiment, after processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements, the method further includes: Based on the second ultrasonic fingerprint image, corresponding fingerprint data processing is performed.

[0016] This specification also provides a data processing device for ultrasonic fingerprint images, including: The acquisition module is used to acquire the first ultrasonic fingerprint image; The noise reduction module is used to process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint noise reduction model, so as to remove at least fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

[0017] This specification also provides an ultrasonic fingerprint device, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the steps of the ultrasonic fingerprint image data processing method.

[0018] This specification also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the ultrasonic fingerprint image data processing method.

[0019] This specification also provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the ultrasonic fingerprint image data processing method.

[0020] Based on the ultrasonic fingerprint image data processing method, apparatus, and ultrasonic fingerprint device provided in this specification, before implementation, a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image simulating the style of a real ultrasonic fingerprint image can be constructed using fingerprint standard samples according to preset training rules. Then, the first, second, and third sample images are used in combination to train a preset ultrasonic fingerprint denoising model that meets the requirements. In specific implementation, after directly acquiring the first ultrasonic fingerprint image using the ultrasonic fingerprint device, the preset ultrasonic fingerprint denoising model can be used to process the first ultrasonic fingerprint image to remove fixed pattern noise and / or dynamic pattern noise, resulting in a second ultrasonic fingerprint image that meets the requirements. This effectively simplifies user operation, efficiently and accurately removes noise such as fixed pattern noise and dynamic pattern noise from the ultrasonic fingerprint image, obtaining a high-quality and high-precision ultrasonic fingerprint image, enabling accurate subsequent fingerprint image data processing. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart of a data processing method for ultrasonic fingerprint images provided in one embodiment of this specification; Figure 2 This is a schematic diagram of an embodiment of the ultrasonic fingerprint image data processing method provided in the embodiments of this specification, applied in a scenario example. Figure 3 This is a schematic diagram of an embodiment of the ultrasonic fingerprint image data processing method provided in the embodiments of this specification, applied in a scenario example. Figure 4 This is a schematic diagram of an embodiment of the ultrasonic fingerprint image data processing method provided in the embodiments of this specification, applied in a scenario example. Figure 5 This is a schematic diagram of the structural composition of an ultrasonic fingerprint device provided in one embodiment of this specification; Figure 6 This is a schematic diagram of the structural composition of a data processing device for ultrasonic fingerprint images provided in one embodiment of this specification; Figure 7 This is a schematic diagram of an embodiment of the ultrasonic fingerprint image data processing method provided in the embodiments of this specification, applied in a scenario example. Figure 8 This is a schematic diagram illustrating an embodiment of the ultrasonic fingerprint image data processing method provided in this specification, applied in a scenario example. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0024] It should be noted that the information and data related to users involved in the embodiments of this specification are all information and data authorized by the user or fully authorized by the relevant parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, and necessary confidentiality measures have been taken. They do not violate public order and good morals, and corresponding operation entry points are provided for users or relevant parties to choose to authorize or refuse.

[0025] It should also be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0026] See Figure 1 As shown in the embodiments of this specification, a data processing method for ultrasonic fingerprint images is provided, wherein the method is specifically applied to one side of an ultrasonic fingerprint device. In specific implementation, the method may include the following: S101: Acquire the first ultrasonic fingerprint image; S102: The first ultrasonic fingerprint image is processed using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, so as to obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

[0027] Specifically, the aforementioned ultrasonic fingerprint image data processing method can be applied to ultrasonic fingerprint devices. These ultrasonic fingerprint devices can be understood as electronic devices that acquire user fingerprint images using ultrasonic waves, such as ultrasonic fingerprint sensors.

[0028] The aforementioned first ultrasonic fingerprint image can be specifically understood as a fingerprint image directly acquired by the ultrasonic fingerprint module in an ultrasonic fingerprint device.

[0029] The aforementioned second ultrasonic fingerprint image can be understood as an ultrasonic fingerprint image obtained after at least removing fixed pattern noise and dynamic pattern noise from the first ultrasonic fingerprint image.

[0030] The aforementioned preset ultrasonic fingerprint denoising model can be understood as a neural network model that is trained in advance according to preset training rules, using a first sample image, a second sample image, and a third sample image. It is adapted to ultrasonic fingerprint images and can automatically remove fixed pattern noise and dynamic pattern noise from ultrasonic fingerprint images as a whole.

[0031] The aforementioned pre-defined ultrasonic fingerprint denoising model includes at least a main network model based on the U-net network. Specifically, the U-net network refers to a symmetric fully convolutional CNN (Convolutional Neural Network) network structure oriented towards pixel-level image segmentation. Compared to other network structures, the U-net network is used to construct and train the pre-defined ultrasonic fingerprint denoising model. On the one hand, the characteristics of this network structure can be utilized to fully consider the mechanistic features of ultrasonic fingerprint imaging. It suppresses patchy noise images globally through downsampling and locally preserves fingerprint features such as subtle ridges through skip connections, making it suitable for handling mixed complex noise and achieving overall removal of both fixed and dynamic pattern noise. On the other hand, the network structure's simple and lightweight nature results in a relatively small number of model parameters, allowing it to be embedded and deployed in ultrasonic fingerprint devices for real-time processing of acquired ultrasonic fingerprint images.

[0032] Specifically, for example, the main network model based on the aforementioned U-net network can include an improved encoder and an improved decoder; wherein, the improved encoder includes multiple layers of upsampling modules in series, each upsampling module including at least two convolutional layers and a normalization layer, and each upsampling module is also configured with SE (Squeeze-and-Excitation) channel attention; the improved decoder includes multiple layers of downsampling modules in series, the number of downsampling modules being equal to the number of upsampling modules, and each downsampling module including at least a bilinear upsampling layer and a convolutional layer; furthermore, skip connections are also set between the same-level upsampling modules and downsampling modules in the improved encoder and improved decoder.

[0033] The first sample image can be understood as a sample ultrasonic fingerprint image containing only fixed pattern noise; the second sample image can be understood as a sample ultrasonic fingerprint image containing both fixed pattern noise and dynamic pattern noise; the third sample image can be understood as an ultrasonic fingerprint image that does not contain fixed pattern noise or dynamic pattern noise, but can simulate the style of a real ultrasonic fingerprint image.

[0034] The first sample image can be used to train the model to learn to distinguish the image features of fingerprint images, the image features of fixed pattern noise, and the relationship between the two in the image. The second sample image can be combined with the first sample image to train the model to further learn to distinguish the image features of fingerprint images, the image features of fixed pattern noise, the image features of dynamic pattern noise, and the relationship between the three in the image. The third sample image can be used to train the model to learn the image features of real ultrasonic fingerprint images that do not have fixed pattern noise and dynamic pattern noise. Then, it can be combined with the first sample image and the second sample image to train the model to learn to adapt to real ultrasonic fingerprint images and use the image features of fingerprint images, the image features of fixed pattern noise, the image features of dynamic pattern noise, and the relationship between the three to remove fixed pattern noise and dynamic pattern noise in the image as a whole.

[0035] Specifically, when using an ultrasonic fingerprint device to acquire ultrasonic fingerprint images through an ultrasonic fingerprint module, firstly, due to inherent factors such as its own non-uniformity, acoustic crosstalk, encapsulation, and structural reflections, a fixed background pattern noise, i.e., fixed pattern noise, will be formed during the imaging process. This fixed pattern noise often severely interferes with the ultrasonic fingerprint image, affecting the recognition of ultrasonic fingerprints.

[0036] Secondly, in addition to the fixed pattern noise mentioned above, ultrasonic fingerprint images acquired by an ultrasonic fingerprint module are also susceptible to external factors such as dust, screen defects, and foreign matter residue on the finger. This results in interference noise in the obtained fingerprint image, namely dynamic pattern noise. For example, if a user has a speck of dust on their finger, a ring centered on that speck of dust will appear in the fingerprint image directly acquired by the ultrasonic fingerprint module. This dynamic pattern noise also interferes with the ultrasonic fingerprint image, affecting ultrasonic fingerprint recognition.

[0037] To address the aforementioned noise, based on existing technologies, one approach involves using conventional image filtering and enhancement methods to remove noise from the image. However, this method cannot effectively remove dynamic pattern noise. Furthermore, it is not well-suited for ultrasonic fingerprint images and cannot effectively remove fixed pattern noise. Another approach requires the user to lift their finger for an air map scan; then, the ultrasonic fingerprint image is denoised based on the scanned air map. While this method can remove fixed pattern noise to some extent, it increases the user's workload and impacts the user experience. Moreover, it inevitably requires storing and updating air maps for different users under various conditions, increasing the device's data storage burden. Additionally, this method cannot simultaneously remove dynamic pattern noise from ultrasonic fingerprint images.

[0038] Having noted the aforementioned problems and their root causes, the applicant, before implementing the proposed solution, could construct, based on pre-defined training rules and using fingerprint standard samples, a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain either fixed or dynamic pattern noise but simulates the style of a real ultrasonic fingerprint image. Furthermore, an initial ultrasonic fingerprint denoising model, including at least a main network model based on a U-net network, could be constructed. Then, the first, second, and third sample images could be used in combination to train the initial ultrasonic fingerprint denoising model. This would allow the model to learn and master the image features of the fingerprint image, fixed pattern noise, and dynamic pattern noise in the ultrasonic fingerprint image, as well as their interaction relationships. This would result in an algorithm model adapted to ultrasonic fingerprint images, capable of automatically analyzing and, based on the image features and interaction relationships of the fingerprint image, fixed pattern noise, and dynamic pattern noise in the ultrasonic fingerprint image, simultaneously removing both fixed and dynamic pattern noise from the image. This would serve as the pre-defined ultrasonic fingerprint denoising model that meets the requirements.

[0039] In practical implementation, when it is necessary to collect and process a user's fingerprint data, the user only needs to place their finger normally in the recognition area (Active Area, abbreviated as AA, or scanning area) of the ultrasonic fingerprint recognition device. There is no need to lift the finger for an air image scan first, nor is it necessary to store a large number of air images locally on the ultrasonic fingerprint device. The ultrasonic fingerprint device can emit ultrasonic waves through the ultrasonic fingerprint module to acquire a first ultrasonic fingerprint image containing noise. This first ultrasonic fingerprint image is then input into a preset ultrasonic fingerprint denoising model. The model is then run, utilizing the knowledge learned during its previous training to denoise the first ultrasonic fingerprint model, thereby removing fixed pattern noise and / or dynamic pattern noise from the image, resulting in a denoised ultrasonic fingerprint image, which serves as a second ultrasonic fingerprint image with higher accuracy and smaller error.

[0040] Furthermore, the ultrasonic fingerprint device can use the second ultrasonic fingerprint image for further fingerprint image data processing. For example, it can authenticate the user based on the second ultrasonic fingerprint image; if the authentication is successful, it can then perform the corresponding unlocking operation.

[0041] Based on the above embodiments, by training and using a preset ultrasonic fingerprint denoising model, it is possible to efficiently and comprehensively remove both fixed pattern noise and dynamic pattern noise in ultrasonic fingerprint images, resulting in high-quality denoised ultrasonic fingerprint images with small errors. Furthermore, it can effectively simplify user operations and reduce the storage burden of ultrasonic fingerprint devices.

[0042] In some embodiments, the first ultrasonic fingerprint image can be directly input into a preset ultrasonic fingerprint denoising model. Using the preset ultrasonic fingerprint denoising model, the first ultrasonic fingerprint image is processed through a main network model to extract image features of the fingerprint image, image features of fixed pattern noise, and image features of dynamic pattern noise from the first ultrasonic fingerprint image. Furthermore, the relationships between the image features of the fingerprint image, the image features of fixed pattern noise, and the image features of dynamic pattern noise are determined. Then, using previously learned knowledge, based on the aforementioned relationships, the image features of the fingerprint image, the image features of fixed pattern noise, the image features of dynamic pattern noise, and their interrelationships, the fixed pattern noise and dynamic pattern noise in the first ultrasonic fingerprint image are completely removed to obtain a second ultrasonic fingerprint image that meets the requirements.

[0043] In some embodiments, in order to better adapt to the ultrasonic fingerprint image and more accurately remove fixed pattern noise and / or dynamic pattern noise in the image, acquisition association information related to the first ultrasonic fingerprint image can be further obtained; then, the acquisition association information and the first ultrasonic fingerprint image are jointly processed using a preset ultrasonic fingerprint denoising model to obtain a second ultrasonic fingerprint image with relatively better denoising effect.

[0044] Specifically, the aforementioned collected related information can be understood as the information of the collected elements that have a major impact on the fixed pattern noise in the ultrasonic fingerprint image collected by the ultrasonic fingerprint module.

[0045] Specifically, a large number of ultrasonic sample fingerprint images can be used in advance to conduct correlation analysis on various collection element information; based on the correlation analysis results, the collection element information that meets the requirements can be screened out as collection association information.

[0046] Specifically, the aforementioned collected information may include at least one of the following: collection temperature, membrane material information of the ultrasonic fingerprint device, ultrasonic emission frequency, and ultrasonic receiving window, etc.

[0047] The membrane material information of the aforementioned ultrasonic fingerprint device specifically refers to the material type of the membrane material set on the surface of the recognition area in the ultrasonic fingerprint module.

[0048] Furthermore, the ultrasonic emission frequency and ultrasonic receiving window can be determined based on the acquisition temperature and / or the membrane material information of the ultrasonic fingerprint device.

[0049] It should be noted that the collection-related information listed above is only illustrative. In actual implementation, depending on the specific circumstances, the collection-related information may also include other collection element information. This specification does not limit this.

[0050] In some embodiments, see Figure 2 As shown, the above-described ultrasonic fingerprint image is processed using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, thereby obtaining a second ultrasonic fingerprint image that meets the requirements. In specific implementations, this may include the following: S2-1: Obtain acquisition association information related to the first ultrasonic fingerprint image; S2-2: Combine the first ultrasonic fingerprint image and the acquired associated information to obtain the target combined data; S2-3: Using the preset ultrasonic fingerprint denoising model based on the target combination data, remove fixed pattern noise and dynamic pattern noise from the first ultrasonic fingerprint image to obtain a second ultrasonic fingerprint image that meets the requirements.

[0051] In practice, the collected associated information can be encoded according to the preset encoding rules to obtain the encoded field of the collected associated information; then the encoded field is added to the first ultrasonic fingerprint image to obtain the target combination data.

[0052] In specific implementation, a preset ultrasonic fingerprint denoising model can be used to parse the encoded field of the collected associated information and the first ultrasonic fingerprint image from the target combination data. Then, based on the encoded field, the target environment type corresponding to the first ultrasonic fingerprint image can be determined. Based on the target environment type, a matching target model operation rule can be determined from multiple preset model operation rules learned and mastered previously. Based on the target model operation rule, the first ultrasonic fingerprint image can be processed through the main network model to extract the image features of the fingerprint image, the image features of fixed pattern noise, and the image features of dynamic pattern noise that match the target environment type. Furthermore, the interrelationship between the image features of the fingerprint image, the image features of fixed pattern noise, and the image features of dynamic pattern noise can be determined. Based on the target model operation rule, the image features of the fingerprint image, the image features of fixed pattern noise, and the image features of dynamic pattern noise, as well as their interrelationships, can be used in combination to remove the fixed pattern noise and / or dynamic pattern noise in the first ultrasonic fingerprint image as a whole, so as to obtain a second ultrasonic fingerprint image that meets the requirements.

[0053] Among them, the above-mentioned preset model operation rules are compiled based on the knowledge learned and mastered during previous model training, and each preset model operation rule corresponds to at least one environment type.

[0054] Based on the above embodiments, by introducing acquisition association information related to the first ultrasonic fingerprint image and using a preset ultrasonic fingerprint denoising model to jointly process the first ultrasonic fingerprint image and the acquisition association information, it is possible to remove fixed pattern noise and dynamic pattern noise in the image more accurately and effectively, and obtain a second ultrasonic fingerprint image with relatively better results.

[0055] In some embodiments, the preset ultrasonic fingerprint denoising model may specifically include: a main network model based on a U-net network, and a physical auxiliary model related to acoustic wave coupling integrated into the main network model.

[0056] Specifically, the aforementioned physical auxiliary model can be understood as a physical constraint model constructed based on the point spread function of ultrasonic wave transmission. The point spread function based on ultrasonic wave transmission can be denoted as: ,in, This refers to the wavelength of the ultrasound wave. d This represents the distance the ultrasonic wave travels.

[0057] It should be noted that the point spread function based on ultrasonic transmission is mainly affected by the wavelength and propagation distance of the ultrasonic wave. Furthermore, for the same material, the wavelength of the ultrasonic wave is also affected by the acquisition temperature. Therefore, through derivation, the above physical auxiliary model can be further improved into a model affected by the acquisition temperature and the propagation distance of the ultrasonic wave.

[0058] Furthermore, the aforementioned main network model may specifically include multiple sub-denoising networks; wherein each sub-denoising network corresponds to a sound wave coupling mode, and each sound wave coupling mode corresponds to at least one combination of sound wave propagation distance range and acquisition temperature range.

[0059] Based on the pre-defined ultrasonic fingerprint denoising model with the above structure, the main network model can first use the knowledge learned previously to perform a first denoising process on the ultrasonic fingerprint image based on fixed pattern noise and dynamic pattern noise to obtain the corresponding intermediate image. Then, the physical auxiliary model can use the acoustic coupling effect during ultrasonic fingerprint acquisition and the corresponding physical constraints to perform further denoising and recovery processing on the intermediate image to obtain a second ultrasonic fingerprint image with relatively better denoising effect and relatively higher quality.

[0060] In some embodiments, the main network model may specifically include multiple sub-denoising networks; wherein each sub-denoising network corresponds to an acoustic coupling mode; Accordingly, see Figure 3As shown, the first ultrasonic fingerprint image is processed using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, resulting in a second ultrasonic fingerprint image that meets the requirements. In specific implementation, this may include the following: S3-1: Using a preset ultrasonic fingerprint denoising model and a physical auxiliary model, the acoustic coupling mode of the first ultrasonic fingerprint image is determined. S3-2: Using a preset ultrasonic fingerprint denoising model, a matching target sub-denoising network is determined from multiple sub-denoising networks based on the acoustic coupling mode of the first ultrasonic fingerprint image. S3-3: The first ultrasonic fingerprint image is denoised using a target sub-denoising network to obtain a second ultrasonic fingerprint image that meets the requirements.

[0061] In practice, a pre-defined ultrasonic fingerprint denoising model can be used first, with the aid of a physical model, to determine the corresponding acoustic wave propagation distance based on the ultrasonic emission frequency and acquisition temperature in the acquired associated information. Then, based on the acquisition temperature and acoustic wave propagation distance, the acoustic wave propagation distance range and acquisition temperature range to which the first ultrasonic fingerprint image belongs can be determined. Furthermore, based on the acoustic wave propagation distance range and acquisition temperature range, the target acoustic wave coupling mode corresponding to the first ultrasonic fingerprint image can be determined through mapping. Next, a sub-denoising network matching the target acoustic wave coupling mode can be selected from multiple sub-denoising networks as the target sub-denoising network. Finally, the target sub-denoising network in the main network model can be called to denoise the first ultrasonic fingerprint image, obtaining a corresponding second ultrasonic fingerprint image that meets the requirements.

[0062] In specific implementation, the first ultrasonic fingerprint image can be denoised using a target sub-denoising network to obtain a corresponding intermediate image. Then, a preset ultrasonic fingerprint denoising model can be used with a physical auxiliary model to perform denoising and restoration processing on the intermediate image based on the target coupling mode, according to the target acoustic wave coupling model and the corresponding acquisition association information. Based on the physical constraints of the relevant mode, some details of the denoised intermediate image can be restored in a targeted manner to obtain a second ultrasonic fingerprint image with relatively better quality.

[0063] In some embodiments, the above-described processing of the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements may further include the following: using the preset ultrasonic fingerprint denoising model and a physical auxiliary model to determine the acoustic coupling mode of the first ultrasonic fingerprint image; using the preset ultrasonic fingerprint denoising model and a main network model to denoise the first ultrasonic fingerprint image according to the acoustic coupling mode, obtaining a corresponding intermediate image; using the preset ultrasonic fingerprint denoising model and a physical auxiliary model to denoise and restore the intermediate image according to the acoustic coupling mode and corresponding acquisition association information, obtaining a second ultrasonic fingerprint image that meets the requirements.

[0064] In specific implementation, for example, a preset ultrasonic fingerprint denoising model can be used to extract ultrasonic image features from the first ultrasonic fingerprint image. Then, a physical auxiliary model can be used to determine the acoustic coupling mode of the first ultrasonic fingerprint image based on the extracted ultrasonic image features. Next, the preset ultrasonic fingerprint denoising model, through a main network model, can be used to denoise the first ultrasonic fingerprint image in the acoustic coupling mode using a matching method to obtain a corresponding intermediate image. Finally, the preset ultrasonic fingerprint denoising model, through a physical auxiliary model, can be used to denoise and restore the intermediate image based on the acoustic coupling mode, the corresponding acquisition association information, and the corresponding physical constraints to obtain a second ultrasonic fingerprint image that meets the requirements.

[0065] Based on the above embodiments, by further introducing and integrating a physical auxiliary model related to acoustic wave coupling into the preset ultrasonic fingerprint denoising model, the influence of acoustic wave coupling on the ultrasonic fingerprint image can be fully considered, and the overall removal of complex noise in the ultrasonic fingerprint image can be achieved more precisely, resulting in a second ultrasonic fingerprint image with relatively higher quality and better effect.

[0066] In some embodiments, see Figure 4 As shown, in specific implementations, the method may also include the following: S4-1: Construct multiple fingerprint standard samples according to preset training rules; among them, at least one of the texture, material, and size of different fingerprint standard samples is different; S4-2: Using an ultrasonic fingerprint device (or a sample ultrasonic fingerprint device), the multiple fingerprint standard samples are scanned and imaged under different experimental environments to obtain multiple first sample images of the multiple fingerprint standard samples; wherein, one experimental environment corresponds to a combination of collected associated information; the first sample image contains fixed pattern noise; S4-3: Add at least one interfering agent to multiple fingerprint standard samples to obtain fingerprint standard samples with added interfering agents; S4-4: Using an ultrasonic fingerprint device, the fingerprint standard sample with added interference is scanned and imaged under different experimental environments to obtain multiple second sample images of multiple fingerprint standard samples; wherein, the second sample images contain fixed pattern noise and dynamic pattern noise; S4-5: Obtain a standard sample image of the fingerprint standard sample; and perform degradation simulation processing on the standard sample image to simulate the style of a real ultrasonic fingerprint image to obtain multiple third sample images; S4-6: Construct multiple sample image groups based on the first sample image, the second sample image, and the third sample image; S4-7: Using multiple sample image groups, train the initial ultrasonic fingerprint denoising model to obtain a preset ultrasonic fingerprint denoising model that meets the requirements.

[0067] Specifically, the aforementioned fingerprint standard sample can be a silicone fingerprint standard sample.

[0068] In practice, multiple different fingerprint standard samples can be prepared based on big data analysis results. These different fingerprint standard samples should differ in at least one of the following: fingerprint pattern, material, or size. This way, by introducing and using multiple different fingerprint standard samples, a wide range of common fingerprint patterns can be comprehensively covered.

[0069] When preparing fingerprint standard samples, first determine the size information of the ultrasonic fingerprint module (e.g., the area of ​​the recognition region of the ultrasonic fingerprint module); based on the size information of the ultrasonic fingerprint module, determine the size information of the fingerprint standard sample; then, based on the size information of the fingerprint standard sample, prepare a matching fingerprint standard sample. The size of the fingerprint standard sample is larger than the size of the ultrasonic fingerprint module, and the difference between the two is less than or equal to a preset size threshold. In this way, by introducing and using fingerprint standard samples that are slightly larger than the ultrasonic fingerprint module, edge effects during scanning and imaging can be effectively avoided, resulting in relatively better sample images.

[0070] In practice, different experimental environments can be constructed by adjusting the combination of different acquired correlation information according to preset training rules. Then, under different experimental environments, an ultrasonic fingerprint device is used to perform ultrasonic scanning imaging on the aforementioned fingerprint standard sample through an ultrasonic fingerprint module, obtaining multiple sample images as the first sample image. Specifically, multiple different experimental environments can be constructed by adjusting the acquisition temperature, transmission frequency, and receiving window according to preset training rules. This allows for the generation of first sample images containing only fixed pattern noise and excluding dynamic pattern noise under various experimental environments. Each experimental environment corresponds to one combination of acquired correlation information.

[0071] Specifically, fingerprint standard samples and ultrasonic fingerprinting devices can be aligned and placed in a temperature-controlled experimental chamber; the membrane material information of the ultrasonic fingerprinting device can be obtained; based on the membrane material information, multiple matching experimental temperatures can be determined; then, for each experimental temperature, multiple matching experimental scanning control parameters can be determined in conjunction with the membrane material information; among them, the experimental scanning control parameters include at least: ultrasonic transmission frequency and ultrasonic receiving window; furthermore, different experimental temperatures can be set using the experimental chamber, and at different experimental temperatures, based on multiple experimental scanning control parameters matching the experimental temperature, the ultrasonic fingerprinting device can be controlled to perform ultrasonic scanning imaging to obtain multiple first sample images corresponding to different experimental environments.

[0072] Before performing ultrasonic scanning imaging, the fingerprint standard sample and the ultrasonic fingerprint module can be aligned according to preset training rules. For example, the central area of ​​the fingerprint standard sample can be aligned with the recognition area of ​​the ultrasonic fingerprint module in a specified direction according to preset training rules.

[0073] Then, according to the preset training rules, one or more interfering agents are added to multiple fingerprint standard samples to simulate the interference of external contamination on the fingerprint model, resulting in fingerprint standard samples with added interfering agents. Next, under different experimental environments, an ultrasonic fingerprint device is used to perform ultrasonic scanning imaging on the fingerprint standard samples with added interfering agents through an ultrasonic fingerprint module, obtaining multiple sample images as second sample images. In this way, a second sample image containing both fixed pattern noise and dynamic pattern noise can be obtained.

[0074] The aforementioned interfering substances may specifically include at least one of the following: dust, hair, hand cream, etc. It should be noted that the interfering substances listed above are merely illustrative. In actual implementation, depending on the specific circumstances and processing requirements, the aforementioned interfering substances may also include other types of interfering substances such as water droplets and dirt. This instruction manual does not limit this.

[0075] Meanwhile, according to the preset training rules, fingerprint images of fingerprint standard samples are acquired using a standard fingerprint image acquisition device as standard images; the standard images contain both fixed pattern noise and dynamic pattern noise; then, by performing degradation simulation processing on the standard sample images to simulate the style of real ultrasonic pen fingerprint images, a corresponding fingerprint image that can accurately simulate the style of real ultrasonic fingerprint images is obtained, which is used as the third sample image.

[0076] Furthermore, according to preset training rules, the first sample image, the second sample image, and the third sample image corresponding to the same standard fingerprint can be combined to obtain multiple sample image groups; each sample image group can correspond to one standard fingerprint. These sample image groups can also be labeled with combinations of corresponding acquisition association information. For example, each sample image group can contain combinations of acquisition association information corresponding to the first sample image and combinations of acquisition association information corresponding to the second sample image. Specifically, each sample image group includes at least one third sample image, which can be used as a label sample.

[0077] Simultaneously, an initial ultrasonic fingerprint denoising model is constructed; wherein the initial ultrasonic fingerprint denoising model includes at least a main network model based on the U-net network; then, multiple sample image groups are used to perform multiple rounds of iterative training on the initial ultrasonic fingerprint denoising model until the convergence condition is met, and a preset ultrasonic fingerprint denoising model that meets the requirements is obtained.

[0078] After training a preset ultrasonic fingerprint denoising model that meets the requirements, the model parameters of the preset ultrasonic fingerprint denoising model that meets the requirements can be obtained; and the model parameters can be stored locally on the ultrasonic fingerprint device, for example, stored in the fingerprint chip of the ultrasonic fingerprint device.

[0079] Accordingly, the ultrasonic fingerprint device can deploy a preset ultrasonic denoising model within the device based on the aforementioned model parameters. In practice, when the ultrasonic fingerprint device acquires the first ultrasonic fingerprint image through the ultrasonic fingerprint module, the preset ultrasonic denoising model can be invoked immediately to process the first ultrasonic fingerprint image, thereby achieving real-time denoising of the ultrasonic fingerprint image.

[0080] Based on the above embodiments, according to the preset training rules, by constructing and utilizing the first sample image, the second sample image, and the third sample image, a preset ultrasonic fingerprint denoising model that is adapted to ultrasonic fingerprint images and can simultaneously remove both fixed pattern noise and dynamic pattern noise with good results can be efficiently trained.

[0081] In some embodiments, the sample image group may further be labeled with a combination of corresponding acquisition-related information.

[0082] In some embodiments, the interfering object may specifically include at least one of the following: dust, hair, hand cream, etc.

[0083] In some embodiments, the degradation simulation processing of the standard sample image described above may include the following: S1: The standard sample image is processed using a preset style transfer model to simulate the style of a real ultrasonic fingerprint image, and a simulated standard sample image is obtained. S2: Perform Gaussian blurring on the simulated standard sample image to obtain the corresponding third sample image.

[0084] Specifically, the aforementioned preset style transfer model can be a neural network model that has been trained using a large number of real ultrasonic fingerprint images obtained from ultrasonic fingerprints. This model learns and masters knowledge about the style of real ultrasonic fingerprint images and can also use this knowledge to perform image style transfer adjustment on the input ultrasonic fingerprint image in order to simulate the style of real ultrasonic fingerprint images.

[0085] Based on the above embodiments, by introducing and utilizing a preset style transfer model, the style of the standard sample image can be effectively simulated to resemble the real ultrasonic fingerprint image, and a third sample image that is close to or even the same as the style of the real ultrasonic fingerprint image can be quickly obtained.

[0086] In some embodiments, after processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements, the method may further include the following: Based on the second ultrasonic fingerprint image, corresponding fingerprint data processing is performed.

[0087] Specifically, for example, the system can query the user's fingerprint database based on the second ultrasonic fingerprint image to perform fingerprint matching for user authentication. If the fingerprint match is successful, the user's authentication is confirmed, allowing login to the user account or the unlocking process. If the fingerprint match fails, the user's authentication is confirmed to have failed, and an error message regarding the verification failure can be displayed.

[0088] It should be noted that the fingerprint data processing described above is only illustrative. In actual implementation, other types of fingerprint data processing can be performed based on the second ultrasonic fingerprint image, depending on the specific circumstances and processing requirements. This specification does not limit this.

[0089] Based on the above embodiments, by using the second ultrasonic fingerprint image instead of the first ultrasonic fingerprint image that was originally directly acquired, the relevant fingerprint data processing can be completed efficiently and accurately.

[0090] As can be seen from the above, based on the ultrasonic fingerprint image data processing method provided in the embodiments of this specification, before specific implementation, a first sample image containing only fixed pattern noise, a second sample image containing both fixed pattern noise and dynamic pattern noise, and a third sample image containing neither fixed pattern noise nor dynamic pattern noise but simulating the style of a real ultrasonic fingerprint image can be constructed using fingerprint standard samples according to preset training rules. Then, the first, second, and third sample images are used in combination to train a preset ultrasonic fingerprint denoising model that meets the requirements. In specific implementation, after directly acquiring the first ultrasonic fingerprint image using an ultrasonic fingerprint device, the preset ultrasonic fingerprint denoising model can be used to process the first ultrasonic fingerprint image to remove fixed pattern noise and / or dynamic pattern noise as a whole, obtaining a second ultrasonic fingerprint image that meets the requirements. This effectively simplifies user operation, efficiently and accurately removes relevant noise from the ultrasonic fingerprint image as a whole, and obtains a high-quality, high-precision ultrasonic fingerprint image, enabling accurate subsequent fingerprint image data processing.

[0091] This specification provides an ultrasonic fingerprint device, see the embodiments below. Figure 5 As shown. The ultrasonic fingerprint device includes a network communication port 501, a processor 502, and a memory 503. These structures are connected by internal cables so that they can perform specific data interaction.

[0092] Specifically, the network communication port 501 can be used to acquire the first ultrasonic fingerprint image.

[0093] The processor 502 can specifically be used to process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements; wherein, the preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed pattern noise and dynamic pattern noise, and a third sample image that does not contain fixed pattern noise and dynamic pattern noise but simulates the style of a real ultrasonic fingerprint image.

[0094] The memory 503 can be used to store the corresponding instruction program and related intermediate data.

[0095] Based on the above method, the relevant structural performance of ultrasonic fingerprint equipment can be effectively utilized to improve the data processing speed of ultrasonic fingerprint equipment and efficiently realize the data processing of ultrasonic fingerprint images.

[0096] In this embodiment, the network communication port 501 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0097] In this embodiment, the processor 502 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0098] In this embodiment, the memory 503 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0099] This specification also provides a computer-readable storage medium based on the above-described ultrasonic fingerprint image data processing method. The computer-readable storage medium stores computer program instructions that, when executed, implement: acquiring a first ultrasonic fingerprint image; processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, thereby obtaining a second ultrasonic fingerprint image that meets the requirements; wherein the preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by jointly using a first sample image containing only fixed pattern noise, a second sample image containing both fixed pattern noise and dynamic pattern noise, and a third sample image that does not contain fixed pattern noise and dynamic pattern noise but simulates the style of a real ultrasonic fingerprint image.

[0100] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0101] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0102] This specification also provides a computer program product, comprising at least a computer program, which, when executed by a processor, implements the following method steps: acquiring a first ultrasonic fingerprint image; processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, thereby obtaining a second ultrasonic fingerprint image that meets the requirements; wherein the preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by jointly using a first sample image containing only fixed pattern noise, a second sample image containing both fixed pattern noise and dynamic pattern noise, and a third sample image that does not contain fixed pattern noise and dynamic pattern noise but simulates the style of a real ultrasonic fingerprint image.

[0103] See Figure 6 As shown, at the software level, this specification also provides a data processing device for ultrasonic fingerprint images, which may specifically include the following structural modules: The acquisition module 601 is specifically used to acquire a first ultrasonic fingerprint image; The denoising module 602 can be used to process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model, so as to remove at least fixed pattern noise and / or dynamic pattern noise, and obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

[0104] In some embodiments, when the denoising module 602 is specifically implemented, it can process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model in the following manner to at least remove fixed pattern noise and / or dynamic pattern noise, and obtain a second ultrasonic fingerprint image that meets the requirements: acquiring acquisition association information related to the first ultrasonic fingerprint image; combining the first ultrasonic fingerprint image and the acquisition association information to obtain target combined data; using the preset ultrasonic fingerprint denoising model according to the target combined data to remove fixed pattern noise and dynamic pattern noise in the first ultrasonic fingerprint image, and obtain a second ultrasonic fingerprint image that meets the requirements.

[0105] In some embodiments, the collected associated information may specifically include at least one of the following: collected temperature, membrane material information of the ultrasonic fingerprint device, ultrasonic emission frequency, and ultrasonic receiving window, etc. The ultrasonic emission frequency and ultrasonic receiving window can be specifically determined based on the acquisition temperature and / or the membrane material information of the ultrasonic fingerprint device.

[0106] In some embodiments, the preset ultrasonic fingerprint denoising model may specifically include: a main network model based on a U-net network, and a physical auxiliary model related to acoustic wave coupling integrated into the main network model.

[0107] In some embodiments, the main network model includes multiple sub-denoising networks; correspondingly, when the denoising module 602 is specifically implemented, it can also process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model in the following manner to at least remove fixed pattern noise and / or dynamic pattern noise, and obtain a second ultrasonic fingerprint image that meets the requirements: using the preset ultrasonic fingerprint denoising model to determine the acoustic coupling mode of the first ultrasonic fingerprint image through a physical auxiliary model; using the preset ultrasonic fingerprint denoising model to determine a matching target sub-denoising network from multiple sub-denoising networks according to the acoustic coupling mode of the first ultrasonic fingerprint image; using the target sub-denoising network to denoise the first ultrasonic fingerprint image to obtain a second ultrasonic fingerprint image that meets the requirements.

[0108] In some embodiments, the above-described device can also be used to: construct multiple fingerprint standard samples according to preset training rules; wherein at least one of the texture, material, and size of different fingerprint standard samples is different; scan and image the multiple fingerprint standard samples using an ultrasonic fingerprint device under different experimental environments to obtain multiple first sample images of the multiple fingerprint standard samples; wherein one experimental environment corresponds to a combination of acquired associated information; the first sample images contain fixed pattern noise; add at least one interfering agent to the multiple fingerprint standard samples to obtain fingerprint standard samples with added interfering agent; scan and image the fingerprint standard samples with added interfering agent under different experimental environments using an ultrasonic fingerprint device to obtain multiple second sample images of the multiple fingerprint standard samples; wherein the second sample images contain fixed pattern noise and dynamic pattern noise; acquire standard sample images of the fingerprint standard samples; and perform degradation simulation processing on the standard sample images to simulate the style of real ultrasonic fingerprint images to obtain multiple third sample images; construct multiple sample image groups based on the first sample images, second sample images, and third sample images; and train an initial ultrasonic fingerprint denoising model using the multiple sample image groups to obtain a preset ultrasonic fingerprint denoising model that meets the requirements.

[0109] In some embodiments, the sample image group may further be labeled with a combination of corresponding acquisition-related information.

[0110] In some embodiments, the interfering object may specifically include at least one of the following: dust, hair, hand cream, etc.

[0111] In some embodiments, when the above-described apparatus is specifically implemented, the standard sample image can be subjected to degradation simulation processing in the following manner: the standard sample image is processed using a preset style transfer model to simulate the style of a real ultrasonic fingerprint image, thereby obtaining a simulated standard sample image; the simulated standard sample image is subjected to Gaussian blurring to obtain a corresponding third sample image.

[0112] In some embodiments, after processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements, the above-described device can also be used to: perform corresponding fingerprint data processing based on the second ultrasonic fingerprint image.

[0113] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0114] As can be seen from the above, the ultrasonic fingerprint image data processing device provided in the embodiments of this specification can effectively simplify user operation, efficiently and accurately remove relevant noise from the ultrasonic fingerprint image as a whole, and obtain ultrasonic fingerprint images with better quality and higher accuracy, so that subsequent fingerprint image data processing can be completed accurately.

[0115] In a specific scenario example, the data processing method for ultrasonic fingerprint images provided in this manual can be used to remove noise from the fixed pattern of ultrasonic fingerprints. For detailed implementation procedures, please refer to the following content.

[0116] In this scenario example, we consider that, firstly, ultrasonic fingerprint sensors (e.g., ultrasonic fingerprint devices) are affected by factors such as their own non-uniformity, acoustic crosstalk, packaging, and structural reflections, resulting in fixed background pattern noise during imaging. Furthermore, this fixed pattern noise varies with temperature, ultrasonic scanning frequency, and the receiving window. This fixed pattern noise severely interferes with ultrasonic fingerprint images, affecting both the recognition rate and the false recognition rate.

[0117] Secondly, in addition to the fixed pattern noise mentioned above, ultrasonic modules (e.g., ultrasonic fingerprint modules) are also affected by dynamic pattern noise caused by dust, screen defects, and foreign objects remaining on the finger. For example, the ultrasonic pattern noise of a single speck of dust is a circular ring. These noises can also severely affect the recognition rate.

[0118] Existing ultrasonic fingerprint image denoising methods typically remove noise through filtering and enhancement, but these methods can only handle random interference noise and cannot remove pattern noise.

[0119] Another existing ultrasonic fingerprint image denoising method is described in [reference needed]. Figure 7As shown, the user needs to press and release their finger to scan an empty image, thus accumulating and updating the air background. The next press uses the previously updated background to remove noise from the fixed pattern. This method requires storing accumulated backgrounds under different temperatures, different film application states, and different delays, and can only remove fixed background noise, resulting in complex and cumbersome user operations and a heavy data storage burden.

[0120] To address the aforementioned problems and their root causes, this scenario example proposes an improved method. This method requires collecting sufficient training data during the experimental phase. This data includes not only fixed pattern noise data but also dynamic pattern noise data generated by foreign objects such as dust, hand cream, and water droplets. After training a neural network model offline using this data, the model (e.g., a pre-defined ultrasonic fingerprint denoising model) is embedded into the carrier device. When the user uses the device, they can directly invoke this neural network model to remove various pattern noises. The removal of fixed and dynamic pattern noise can share the same process. For detailed implementation procedures, please refer to [link to relevant documentation]. Figure 8 As shown, it includes the following content.

[0121] First, the silicone fingerprint standard sample (e.g., a fingerprint standard sample) is physically aligned with the ultrasonic fingerprint module. This involves pre-calibrating the scanning area of ​​the ultrasonic fingerprint module and then aligning the silicone fingerprint standard sample with this scanning area in a predetermined direction. To avoid edge effects, the size of the silicone standard sample is slightly larger than the physical size of the ultrasonic fingerprint module. During alignment, the center area of ​​the silicone standard sample is aligned with the fingerprint membrane.

[0122] In a laboratory environment, the aligned ultrasonic fingerprint module and the silicone standard sample are placed in a temperature-controlled test chamber. The test chamber is adjusted to the corresponding temperature according to multiple preset test temperature points. The temperature points can be generated by sampling at 2℃ intervals from [-20℃ to 60℃].

[0123] After the temperature inside the test chamber stabilizes at the corresponding temperature point (ensuring that the temperature of the ultrasonic module is also at the corresponding temperature point), the external controller sends an ultrasonic scanning command to the ultrasonic module carrier (mobile phone or other terminal) via Bluetooth or wireless LAN through a pre-set control program.

[0124] Upon receiving a scanning command, the ultrasonic fingerprint module carrier controls the ultrasonic fingerprint module to scan the current silicone fingerprint standard sample according to a preset set of ultrasonic fingerprint module scanning parameters. These scanning parameters mainly control the ultrasonic emission frequency and the receiving window. This allows for the acquisition of ultrasonic images of the silicone fingerprint standard sample with fixed pattern noise at different ultrasonic emission frequencies and receiving windows at a defined temperature.

[0125] Foreign matter residues such as dust, hair, and hand cream are added to various silicone standard samples. The previous step is repeated to obtain ultrasonic images of silicone fingerprint standard samples with dynamic pattern noise.

[0126] After obtaining ultrasonic images of the silicone fingerprint standard sample under the corresponding parameters, the preset program stores these images in the file system of the ultrasonic fingerprint module carrier and names these files according to different temperatures and scanning parameters. To ensure a sufficient number of images, the scanning parameter interval is set relatively small to obtain more combinations of scanning parameters.

[0127] The above process is repeated for all silicone standard samples (this can also be done simultaneously if there are enough ultrasonic fingerprint modules), resulting in a large number of ultrasonic fingerprint images of silicone standard samples with different fixed pattern noise under different temperatures, ultrasonic emission frequencies, and ultrasonic receiving windows. Simultaneously, images of the alignment area between the silicone fingerprint standard samples and the ultrasonic fingerprint modules are easily obtained. The standard sample images are then degraded to simulate the style of real ultrasonic fingerprint images. Degradation can be achieved by adding noise, Gaussian blurring, or training a style transfer network.

[0128] The degraded standard sample image and the corresponding ultrasonic scan image with fixed pattern noise are fed into a neural network model for training. The ultrasonic scan image with fixed pattern noise serves as the input, and the corresponding degraded standard sample image serves as the label. The network model can be a U-net structure. After the model converges, the model parameters are extracted, thus completing the network training.

[0129] The obtained model parameters are stored in the program for removing fixed-pattern noise from ultrasonic fingerprints, so that the ultrasonic fingerprint fixed-pattern noise removal neural network can read them. For the input real fingerprint image with fixed-pattern noise, the ultrasonic fingerprint fixed-pattern noise removal neural network is called, and the output is the real fingerprint image with the fixed-pattern noise removed. This achieves the purpose of removing fixed-pattern noise from ultrasonic fingerprints.

[0130] The above scenario examples verify the data processing method for ultrasonic fingerprint images provided in this specification. By training and using the corresponding neural network model, it can effectively address the problem of fixed pattern noise that is difficult to remove due to various temperature changes, scanning frequency fluctuations, and changes in the receiving window. It has good coverage and generalization. It does not require the terminal to perform other operations to update the background noise, nor does it require storing multiple background noises, thus reducing power consumption and memory consumption.

[0131] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0132] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0133] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0134] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0135] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0136] Although this specification has been described by way of examples, those skilled in the art will know that this application has many variations and modifications, and it is intended that the text described herein include these variations and modifications without departing from the spirit of this specification.

Claims

1. A data processing method for ultrasonic fingerprint images, characterized in that, Applications in ultrasonic fingerprint devices include: Acquire the first ultrasonic fingerprint image; The first ultrasonic fingerprint image is processed using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, so as to obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

2. The method according to claim 1, characterized in that, The step of processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements includes: Acquire the acquisition association information related to the first ultrasonic fingerprint image; The target combined data is obtained by combining the first ultrasonic fingerprint image and the acquired associated information; Using the preset ultrasonic fingerprint denoising model based on the target combination data, fixed pattern noise and dynamic pattern noise are removed from the first ultrasonic fingerprint image to obtain a second ultrasonic fingerprint image that meets the requirements.

3. The method according to claim 2, characterized in that, The collected associated information includes at least one of the following: collection temperature, membrane material information of the ultrasonic fingerprint device, ultrasonic emission frequency, and ultrasonic receiving window; The ultrasonic emission frequency and ultrasonic receiving window are determined based on the acquisition temperature and / or the membrane material information of the ultrasonic fingerprint device.

4. The method according to claim 1, characterized in that, The preset ultrasonic fingerprint denoising model includes: a main network model based on the U-net network, and a physical auxiliary model related to acoustic wave coupling integrated into the main network model.

5. The method according to claim 4, characterized in that, The main network model includes multiple sub-denoising networks; Accordingly, the step of processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements includes: The acoustic coupling mode of the first ultrasonic fingerprint image is determined by using a pre-defined ultrasonic fingerprint denoising model and a physical auxiliary model. Using a preset ultrasonic fingerprint denoising model, a matching target sub-denoising network is determined from multiple sub-denoising networks based on the acoustic coupling mode of the first ultrasonic fingerprint image. The first ultrasonic fingerprint image is denoised using a target sub-denoising network to obtain a second ultrasonic fingerprint image that meets the requirements.

6. The method according to claim 1, characterized in that, The method further includes: Multiple fingerprint standard samples are constructed according to preset training rules; among them, at least one of the fingerprint standard samples is different in terms of texture, material, and size. Using an ultrasonic fingerprint device, multiple fingerprint standard samples are scanned and imaged under different experimental environments to obtain multiple first sample images of the multiple fingerprint standard samples; wherein, one experimental environment corresponds to a combination of acquired associated information; the first sample images contain fixed pattern noise; Add at least one interfering agent to multiple fingerprint standard samples to obtain fingerprint standard samples with added interfering agents; Using an ultrasonic fingerprint device, the fingerprint standard samples with added interference were scanned and imaged under different experimental environments to obtain multiple second sample images of multiple fingerprint standard samples; wherein, the second sample images contain fixed pattern noise and dynamic pattern noise; Obtain a standard sample image of a fingerprint standard sample; and perform degradation simulation processing on the standard sample image to simulate the style of a real ultrasonic fingerprint image to obtain multiple third sample images; Multiple sample image groups are constructed based on the first sample image, the second sample image, and the third sample image; Using multiple sample image groups, an initial ultrasonic fingerprint denoising model is trained to obtain a preset ultrasonic fingerprint denoising model that meets the requirements.

7. The method according to claim 6, characterized in that, The sample image group is also marked with a combination of corresponding acquisition-related information.

8. The method according to claim 6, characterized in that, The interfering substances include at least one of the following: dust, hair, or hand cream.

9. The method according to claim 6, characterized in that, The degradation simulation processing of the standard sample image includes: The standard sample image is processed using a preset style transfer model to simulate the style of a real ultrasonic fingerprint image, resulting in a simulated standard sample image. Gaussian blurring is performed on the simulated standard sample image to obtain the corresponding third sample image.

10. The method according to claim 1, characterized in that, After processing the first ultrasonic fingerprint image using a preset ultrasonic fingerprint denoising model to at least remove fixed pattern noise and / or dynamic pattern noise, and obtaining a second ultrasonic fingerprint image that meets the requirements, the method further includes: Based on the second ultrasonic fingerprint image, corresponding fingerprint data processing is performed.

11. A data processing device for ultrasonic fingerprint images, characterized in that, include: The acquisition module is used to acquire the first ultrasonic fingerprint image; The noise reduction module is used to process the first ultrasonic fingerprint image using a preset ultrasonic fingerprint noise reduction model, so as to remove at least fixed pattern noise and / or dynamic pattern noise to obtain a second ultrasonic fingerprint image that meets the requirements. The preset ultrasonic fingerprint denoising model is trained in advance according to preset training rules by using a first sample image containing only fixed pattern noise, a second sample image containing both fixed and dynamic pattern noise, and a third sample image that does not contain fixed and dynamic pattern noise but simulates the style of real ultrasonic fingerprint images.

12. An ultrasonic fingerprint device, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images