Fingerprint image processing method and apparatus, and electronic device

Abstract

The embodiments of the present disclosure belong to the technical field of fingerprint recognition. Provided are a fingerprint image processing method and apparatus, and an electronic device. The fingerprint image processing method comprises: acquiring a first fingerprint image generated by means of a fingerprint sensor; and using global information of the first fingerprint image as an input to predict fingerprint image extension information beyond at least part of the boundary of the first fingerprint image, and using the fingerprint image extension information to perform corresponding image growth, so as to generate a second fingerprint image. By means of the embodiments of the present disclosure, a fingerprint image generated by a fingerprint sensor can be fully utilized to acquire a more accurate prediction result, thereby facilitating an improvement in the correct recognition rate of fingerprint recognition tasks. Moreover, a process can be simplified, and the efficiency can be improved.

Description

Fingerprint image processing method, device and electronic equipment

[0001] This disclosure claims priority to Chinese patent application number 2024101238167, filed with the Patent Office of China on January 29, 2024, entitled “Fingerprint Image Processing Method, Device and Electronic Device,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of fingerprint recognition technology, and in particular to a fingerprint image processing method, device and electronic device. Background Art

[0003] Fingerprint sensors used in consumer electronics and other products are seeing their information collection area shrink. This trend is driven by two main considerations: first, to accommodate the continuous introduction of new products. For example, for fingerprint sensors installed on the side of a mobile phone, the size of the sensor typically matches the thickness of the phone. Second, for cost considerations, reducing the fingerprint sensor's collection area can lower costs. The fingerprint recognition rate is related to the amount of fingerprint information that can be collected. Generally speaking, the smaller the size of the captured fingerprint image, the less information is available, and the recognition rate decreases accordingly.

[0004] In the related art, there are schemes for improving the recognition rate by expanding the useful information of the fingerprint image. For example, by splicing multiple small fingerprint images, a larger image with richer information is obtained; however, this method is not applicable to fingerprint images that do not have a splicing relationship. In other words, the above method cannot be used when the fingerprint images cannot be spliced. For another example, by statistically analyzing the distribution of feature points of the fingerprint image, a binary image is generated after parameter estimation, and the binary image is fused with the background image to obtain a new expanded fingerprint image; however, this method is cumbersome, and it is usually difficult to fit the correct expanded information through traditional parameter estimation methods when the amount of data is insufficient. For another example, the edge area of ​​the fingerprint image is used to predict the expanded information, but this scheme requires the image to be expanded multiple times in each edge area. The method is cumbersome, and there is an error accumulation problem in the corner area of ​​the fingerprint image, and the actual prediction effect is poor.

[0005] In summary, for fingerprint sensors with a small information collection area, how to obtain more fingerprint information to improve the fingerprint recognition rate has not yet been proposed. Summary of the Invention

[0006] In view of this, embodiments of the present disclosure provide a fingerprint image processing method, apparatus, and electronic device that can solve the above-mentioned problems.

[0007] According to one aspect of an embodiment of the present disclosure, a fingerprint image processing method is provided, comprising: acquiring a first fingerprint image generated by a fingerprint sensor; using global information of the first fingerprint image as input to predict fingerprint image extension information outside at least part of a boundary of the first fingerprint image, and performing corresponding image growth using the fingerprint image extension information to generate a second fingerprint image.

[0008] Optionally, the prediction of the fingerprint image extension information and the corresponding image growth are performed by a fingerprint prediction model; wherein, the second fingerprint image includes an original area corresponding to the first fingerprint image, and a growth area obtained by the fingerprint prediction model performing image growth outside at least part of the boundary of the original area based on the fingerprint image extension information.

[0009] Optionally, the fingerprint image processing method further includes: pre-defining a growth area of ​​the second fingerprint image outside at least part of the boundary of the first fingerprint image; filling predetermined pixel information in the growth area to obtain an initial fingerprint extended image; inputting the initial fingerprint extended image into the fingerprint prediction model, and performing prediction of the fingerprint image extended information and corresponding image growth through the fingerprint prediction model to form fingerprint pixel information corresponding to the fingerprint image extended information in the growth area.

[0010] Optionally, the fingerprint prediction model is a codec model, which includes a coding block sequence and a decoding block sequence, wherein the prediction of the fingerprint image extended information and the corresponding image growth include: using the coding block sequence to perform multiple encoding processes on the initial fingerprint extended image and obtain a fingerprint encoded image, wherein the encoding process includes performing multiple convolution operations and corresponding average pooling downsampling processes on the initial fingerprint extended image in sequence; using the decoding block sequence to perform decoding processes on the fingerprint encoded image and obtain a fingerprint decoded image, wherein the decoding process includes performing multiple convolution operations and corresponding upsampling convolution magnification and merging processes on the fingerprint encoded image, and the fingerprint decoded image is used as the second fingerprint image output by the codec model.

[0011] Optionally, the coding block sequence includes multiple coding blocks connected in sequence, the output of the previous coding block serves as the input of the next coding block, and each coding block includes a coding convolution layer and a downsampling layer; wherein, the step of using the coding block sequence to perform multiple coding processes on the initial fingerprint extended image includes: receiving the initial fingerprint extended image, and performing multiple coding convolution operations and corresponding average pooling downsampling processes on the initial fingerprint extended information through the coding convolution layers and downsampling layers of the multiple coding blocks to obtain the fingerprint coded image.

[0012] Optionally, the decoding block sequence includes multiple decoding blocks connected in sequence, the output of the previous decoding block serves as the input of the next decoding block, each decoding block corresponds to one of the coding blocks in the coding block sequence, and includes an upsampling layer, a skip connection layer and a decoding convolution layer, and the skip connection layer is used to combine the decoding block with the feature information of the corresponding coding block in the coding block sequence; wherein, the step of using the decoding block sequence to decode the fingerprint coded image includes: receiving the fingerprint coded image output by the coding block sequence, and performing multiple decoding convolution operations and upsampling convolution amplification processing on the fingerprint coded image through the upsampling layers and decoding convolution layers of the multiple decoding blocks, and at the same time combining it with the feature information of the corresponding coding block in the coding block sequence to obtain the fingerprint decoded image.

[0013] Optionally, in the fingerprint image processing method, before performing the prediction of the fingerprint image extension information and the corresponding image growth, it also includes: estimating the quality of the first fingerprint image and generating a quality score based on the image quality index of the first fingerprint image, wherein the image quality index includes at least one of the fingerprint pattern clarity and signal-to-noise ratio of the first fingerprint image; based on the quality score, determining whether to perform image growth on the first fingerprint image.

[0014] Optionally, the fingerprint image processing method further includes: performing equalization processing on the growth area of ​​the second fingerprint image to enhance at least one of contrast and brightness of the growth area; wherein the equalization processing includes: after the second fingerprint image is generated, calculating equalized pixel values ​​of the growth area of ​​the second fingerprint image according to the following transformation function, and processing fingerprint pixels of the growth area based on the pixel values ​​to obtain a equalized second fingerprint image;

[0015] Wherein, m represents the total number of fingerprint pixels of the second fingerprint image, and s represents the total number of gray levels of the second fingerprint image, m i Indicates gray level g i The number of fingerprint pixels, P(g i ) represents the gray level g of the second fingerprint image j Probability of occurrence.

[0016] Optionally, the fingerprint image processing method further includes: in the fingerprint matching and verification stage, extracting fingerprint feature information from the original area and the growth area of ​​the second fingerprint image, and matching and verifying the fingerprint feature information extracted from the original area with the reference fingerprint feature information of the pre-stored fingerprint template to determine whether the user's input fingerprint is a legal fingerprint, and performing the fingerprint recognition task when the input fingerprint is a legal fingerprint; wherein the fingerprint feature information of the growth area of ​​the second fingerprint image is not directly used for determining the similarity between the second fingerprint image and the pre-stored fingerprint template.

[0017] Optionally, the fingerprint feature information extracted from the growth area of ​​the second fingerprint image is used to perform image positioning and overlapping area calculation between the second fingerprint image and the pre-stored fingerprint template to find a matching relationship between the two, wherein the similarity judgment of the fingerprint feature information is calculated based on the overlapping area between the second fingerprint image and the fingerprint template.

[0018] Optionally, the fingerprint image processing method further includes: in the fingerprint registration stage, extracting fingerprint feature information from the original area and the growth area of ​​the second fingerprint image respectively as baseline fingerprint feature information and storing it as an independent fingerprint template in the fingerprint template library; or, in the fingerprint matching verification stage, when it is determined that the user's input fingerprint is a legal fingerprint, executing the fingerprint template self-learning function, and using the fingerprint feature information extracted from the original area and the growth area of ​​the second fingerprint image respectively as baseline fingerprint feature information, and storing it as a new fingerprint template in the fingerprint template library or replacing another fingerprint template in the fingerprint template library.

[0019] Optionally, the fingerprint image processing method further includes: collecting the user's input fingerprint through a fingerprint sensor and generating the first fingerprint image; wherein the fingerprint sensor is a capacitive fingerprint sensor arranged on the side of the electronic device frame and having a narrow and long rectangular structure, and it includes a sensing array, the sensing array includes a plurality of sensing electrodes distributed in an array, the capacitive fingerprint sensor uses the sensing array to collect the user's input fingerprint and generates the first fingerprint image based on the input fingerprint.

[0020] According to another aspect of an embodiment of the present disclosure, a fingerprint image processing device is provided, comprising: an acquisition module for acquiring a first fingerprint image generated by a fingerprint sensor; a processing module for using global information of the first fingerprint image as input to a fingerprint prediction model, and predicting fingerprint image extension information outside at least part of the boundary of the first fingerprint image through the fingerprint prediction model, and performing image growth using the fingerprint image extension information to generate a second fingerprint image; wherein the second fingerprint image includes an original area corresponding to the first fingerprint image, and a growth area obtained by performing image growth by the fingerprint prediction model outside at least part of the boundary of the original area according to the fingerprint image extension information.

[0021] Optionally, the fingerprint prediction model is used to predefine the growth area outside at least part of the boundary of the first fingerprint image and fill in predetermined pixel information to obtain an initial fingerprint extended image, and perform prediction of the fingerprint image extended information and corresponding image growth based on the input initial fingerprint extended image to form fingerprint pixel information corresponding to the fingerprint image extended information in the growth area.

[0022] Optionally, the fingerprint prediction model is a codec model, which includes a coding block sequence and a decoding block sequence; the coding block sequence includes multiple coding blocks connected in sequence, the output of the previous coding block serves as the input of the next coding block, and each coding block includes a coding convolution layer and a downsampling layer; the coding block sequence is used to sequentially perform multiple convolution operations and corresponding average pooling downsampling processing on the initial fingerprint extended image through the coding convolution layer and downsampling layer of the multiple coding blocks to obtain a fingerprint coding image; the decoding block sequence includes multiple decoding blocks connected in sequence, the output of the previous decoding block serves as the input of the next decoding block, each decoding block corresponds to one of the coding blocks in the sequence, and includes an upsampling layer, a jump connection layer and a decoding convolution layer, and the jump connection layer is used to combine the decoding block with the feature information of the corresponding coding block in the coding block sequence; the decoding block sequence is used to use the multiple decoding blocks to perform multiple convolution operations and corresponding upsampling convolution amplification and merging processing on the fingerprint coding image and obtain a fingerprint decoding image, wherein the fingerprint decoding image is used as the second fingerprint image output by the codec model.

[0023] Optionally, the processing module is also used to: estimate the quality of the first fingerprint image and generate a quality score based on the image quality index of the first fingerprint image, and determine whether to perform image growth on the first fingerprint image based on the quality score; wherein the image quality index includes at least one of the fingerprint pattern clarity and signal-to-noise ratio of the first fingerprint image.

[0024] Optionally, the processing module is further used to extract fingerprint feature information from the original area and the growth area of ​​the second fingerprint image during the fingerprint matching and verification stage, and match and verify the fingerprint feature information extracted from the original area with the reference fingerprint feature information of the pre-stored fingerprint template to determine whether the user's input fingerprint is a legal fingerprint, and perform a corresponding fingerprint recognition task when the input fingerprint is a legal fingerprint; wherein the fingerprint feature information of the growth area of ​​the second fingerprint image is not directly used for the similarity judgment between the second fingerprint image and the pre-stored fingerprint template.

[0025] Optionally, the fingerprint feature information extracted from the growth area of ​​the second fingerprint image is used to perform image positioning and overlapping area calculation between the second fingerprint image and the pre-stored fingerprint template to determine the matching relationship between the two, wherein the processing module performs similarity calculation between the second fingerprint image and the fingerprint template based on the overlapping area between the two.

[0026] Optionally, the processing module is further used to: in the fingerprint registration stage, extract fingerprint feature information from the original area and the growth area of ​​the second fingerprint image respectively, as baseline fingerprint feature information and store it as an independent fingerprint template in the fingerprint template library; or, in the fingerprint matching verification stage, when it is determined that the user's input fingerprint is a legal fingerprint, execute the fingerprint template self-learning function, and use the fingerprint feature information extracted from the original area and the growth area of ​​the second fingerprint image respectively as baseline fingerprint feature information, and store it as a new fingerprint template in the fingerprint template library or replace another fingerprint template in the fingerprint template library.

[0027] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a fingerprint sensor for collecting a user's input fingerprint and generating a first fingerprint image; a fingerprint image processing device, connected to the fingerprint sensor, for acquiring the first fingerprint image generated by the fingerprint sensor, and using global information of the first fingerprint image as input to predict fingerprint image extension information outside at least part of the boundary of the first fingerprint image, and using the fingerprint image extension information to perform image growth to generate a second fingerprint image.

[0028] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, including: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, enable the processor to perform any of the above-mentioned fingerprint image processing methods.

[0029] According to another aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any of the above-mentioned fingerprint image processing methods.

[0030] One or more technical solutions provided in the embodiments of the present disclosure, after acquiring a first fingerprint image generated by a fingerprint sensor, use the global information of the first fingerprint image as input to predict the extended information of the fingerprint image outside at least part of the boundary of the first fingerprint image, and use the extended information of the fingerprint image to perform image growth to generate a second fingerprint image. Since the prediction of the extended fingerprint information and the corresponding image growth utilize the global information of the entire fingerprint image, more accurate prediction results can be obtained, which helps to improve the correct recognition rate of the fingerprint recognition task. In addition, the various technical solutions provided in the embodiments of the present disclosure can realize the expansion of the fingerprint image directly collected from the fingerprint sensor into the full image after the fingerprint image is grown, realizing end-to-end processing, which can simplify the process and improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0032] FIG1A shows a schematic diagram of an electronic device that may be applicable according to an exemplary embodiment of the present disclosure;

[0033] FIG1B shows a system block diagram of an electronic device to which exemplary embodiments of the present disclosure may be applied;

[0034] FIG2 shows a flow chart of a fingerprint image processing method according to an exemplary embodiment of the present disclosure;

[0035] FIG3 is a schematic diagram showing a fingerprint image generated by a fingerprint sensor according to an exemplary embodiment of the present disclosure;

[0036] FIG4 shows a schematic diagram of fingerprint image growth using a codec model according to an exemplary embodiment of the present disclosure;

[0037] FIG5 shows another schematic diagram of fingerprint image growth using a codec model according to an exemplary embodiment of the present disclosure;

[0038] FIG6 shows a schematic diagram of another method of growing a fingerprint image using a codec model according to an exemplary embodiment of the present disclosure;

[0039] FIG7 shows a schematic block diagram of a codec model according to an exemplary embodiment of the present disclosure;

[0040] FIG8 shows a schematic diagram of fingerprint image growth using a codec model according to an exemplary embodiment of the present disclosure;

[0041] FIG9 shows a flowchart of a fingerprint image processing method according to another exemplary embodiment of the present disclosure;

[0042] FIG10 shows a structural block diagram of a fingerprint image processing apparatus according to an exemplary embodiment of the present disclosure;

[0043] FIG11 shows a block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure. DETAILED DESCRIPTION

[0044] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0045] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0046] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0047] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0048] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0049] FIG1A shows a schematic diagram of an electronic device in which the various solutions described herein may be implemented according to an exemplary embodiment of the present disclosure. As shown in FIG1A , the electronic device 100 may include a device body 101 and a fingerprint sensor 102. The fingerprint sensor 102 may capture a fingerprint image for fingerprint recognition. The fingerprint sensor 102 may include, but is not limited to, a capacitive fingerprint sensor, an optical fingerprint sensor, an ultrasonic fingerprint sensor, etc., and the specific location of the fingerprint sensor 102 in the electronic device 100 may be set on the side, back, or below the display screen on the front of the device body 101 according to actual product design requirements.

[0050] In some embodiments, the electronic device 100 may be a portable electronic device, such as a smartphone, a tablet computer, a laptop computer, a personal digital assistant, etc. In other embodiments, the electronic device 100 may also be a smart wearable device, and the disclosed embodiments do not limit the type of the electronic device 100.

[0051] In some embodiments, the fingerprint sensor 102 can be specifically set on the side of the device body 101 of the electronic device 100; as smartphones or other portable electronic devices develop towards being lighter or foldable, the thickness of the electronic device 100 is getting smaller and smaller, which causes the fingerprint sensor 102 set on the side of the device body 101 to become narrower and narrower.

[0052] Referring to Figure 1A , as a typical embodiment, the device body 101 includes a display screen 10 and a midframe 20. The display screen 10 is located on the front of the device body 101, displaying images and providing a user interface. The midframe 20 is generally located between the display screen 10 and the back cover of the electronic device, supporting the display screen 101 and housing various functional components within the device body 101, such as the motherboard, battery, camera, speaker, microphone, and various sensor units. In a specific embodiment, the midframe 20 comprises a bezel located around the periphery of the device body 101. The bezel may include multiple sides and may carry a power button, volume button, or other function buttons. The fingerprint sensor 102 may be located on one of the sides of the bezel and have a sensing area 108. In a specific embodiment, the fingerprint sensor 102 may be a fingerprint recognition chip or a fingerprint module having a fingerprint recognition chip. The fingerprint sensor 102 may be integrated above the power button or volume button on the side of the bezel, embedded in a predetermined area on the side of the bezel, or attached to the inner surface of the side of the bezel, allowing the user to input a fingerprint and implement the side fingerprint function of the electronic device 100.

[0053] Referring to FIG. 1B , the fingerprint sensor 102 includes a sensing array 103, an output module 104, an interface module 105, and a driver module 106. The sensing array 103 is configured to couple with the user's finger to collect fingerprint information when the user presses the fingerprint sensor 102 to input a fingerprint. Specifically, the sensing array 103 includes a plurality of sensing electrodes arranged in an array. The area where the sensing array 103 is located, or its effective fingerprint collection area, constitutes the sensing area 108 of the fingerprint sensor 102. The driver module 106 and the output module 104 are connected to the sensing array 103 and the interface module 105, respectively. The driver module 106 is configured to drive the sensing array 103 to perform a fingerprint scan to collect fingerprint information from the user. The output module 104 is configured to generate a corresponding first fingerprint image based on the fingerprint information collected by the sensing array 103 and output the first fingerprint image to the main processor 120 of the electronic device via the interface module 105. The interface module 105 may be a serial peripheral interface (SPI). The main processor 120 can be specifically a central processing unit CPU or other processing unit or control unit with processing capabilities inside the electronic device 100, such as a microcontroller (MCU). It is connected to the interface module 106 and includes a fingerprint image processing device, which is mainly used to control the working state of the fingerprint sensor 102, and process the first fingerprint image output by the fingerprint sensor 102 and perform fingerprint matching verification to determine whether the currently collected fingerprint image is a legal fingerprint, and unlock the electronic device 100 or perform other functions related to fingerprint recognition based on the judgment result.

[0054] Due to the thickness limitation of the electronic device 100 itself, the fingerprint sensor 102 located on the side of the frame needs to be designed as a narrow strip structure, resulting in a narrow sensing area 108 of the fingerprint sensor 102. The fingerprint information collected by the fingerprint sensor 102 corresponds to the sensing area 108. Therefore, in the above typical application scenario, the fingerprint sensor 102 directly collects only the relatively narrow local data of the part of the finger covering the sensing area 108, resulting in low fingerprint recognition differentiation and affecting the fingerprint recognition performance.

[0055] An embodiment of the present disclosure provides a fingerprint image processing method, which can be applied to the electronic device 100 shown in FIG1A to improve its fingerprint recognition performance.

[0056] FIG2 shows a flowchart of a fingerprint image processing method according to an exemplary embodiment of the present disclosure. As shown in FIG2 , the fingerprint image processing method according to an exemplary embodiment of the present disclosure can be applied to a fingerprint registration stage or a fingerprint verification stage, and specifically includes the following steps.

[0057] Step 200: collecting a user's input fingerprint through a fingerprint sensor and generating a first fingerprint image;

[0058] In an embodiment of the present disclosure, in an electronic device 100, the fingerprint sensor 102 can detect contact with a user's finger or activate a fingerprint collection function according to an instruction from the main processor 120 of the electronic device 100, and collect fingerprint information input by the user by pressing the fingerprint sensor 102 through its sensing array 103, and generate a first fingerprint image based on the fingerprint information input by the user. For example, when the fingerprint sensor 102 is a capacitive fingerprint sensor, the multiple sensing electrodes of the sensing array 103 will form different coupling capacitances with the ridges and valleys of the user's finger. The fingerprint sensor 102 can collect fingerprint information of the part of the user's finger pressed on the sensing area 108 by driving the sensing array 103 to detect the capacitance signals formed by the ridges and valleys and the sensing electrodes, and generate a first fingerprint image based on the fingerprint information; the first fingerprint image is specifically a digital image formed by integrating the fingerprint information of the corresponding position of the finger collected by all the sensing electrodes of the sensing array 103. Optionally, if the currently collected first fingerprint image does not meet the requirements, the fingerprint sensor 102 can discard the currently collected fingerprint image, and re-collect and generate a new first fingerprint image; on the other hand, the fingerprint sensor 102 can also perform some processing on the first fingerprint image it generates, and temporarily store the first fingerprint image inside it to wait for the main processor 120 of the electronic device 100 to obtain it.

[0059] In the embodiment of the present disclosure, as shown in FIG1A , the fingerprint sensor 102 needs to be designed as a narrow strip structure due to the width limitation of the side frame of the electronic device 100 , resulting in a smaller area of ​​its sensing area 108 . When the user presses the fingerprint sensor 102 to input a fingerprint, only the fingerprint information of the finger portion covering the sensing area 108 of the fingerprint sensor 102 can be collected by the fingerprint sensor 102 . Therefore, the first fingerprint image collected and generated by the fingerprint sensor 102 is actually only a part or even only a small part of the user's actual fingerprint. For example, if the fingerprint sensor 102 uses a capacitive fingerprint sensor with a sensing area 108 of 6.6mm*1.6mm or 8mm*1.8mm, and the sensing area 108 specifically includes a sensing array 103 formed by 132*32 or 160*36 sensing electrodes, the first fingerprint image generated by the fingerprint sensor 102 corresponds to the position where the finger contacts the sensing area 108. That is, the size of the first fingerprint image is only approximately 6.6mm*1.6mm, while the effective fingerprint area of ​​the main area of ​​an ordinary user's finger is approximately 20mm*10mm. It can be seen that the first fingerprint image is not only very narrow, but also the image area is much smaller than the actual area of ​​the user's fingerprint.

[0060] For example, as shown in FIG3 , fingerprint image 302a, fingerprint image 302b, and fingerprint image 302c are fingerprint images captured by fingerprint sensor 102, each of which is merely a portion of the user's actual fingerprint 301. It should be understood that fingerprint image 302a, fingerprint image 302b, and fingerprint image 302c are merely exemplary fingerprint images, and fingerprint sensor 102 can capture and generate different fingerprint images depending on the position of the finger applied to sensing area 108 of fingerprint sensor 102. In other embodiments, if fingerprint sensor 102 employs an optical fingerprint sensor, an ultrasonic fingerprint sensor, or another type of sensor, the first fingerprint image actually captured and generated may be larger than the size of sensing area 103, but still only represents a portion of the user's fingerprint.

[0061] Step S201: Acquire a first fingerprint image generated by a fingerprint sensor.

[0062] In an embodiment of the present disclosure, the main processor 120 may send a control instruction to the fingerprint sensor 120 to control the output module 104 of the fingerprint sensor 120 to transmit the first fingerprint image generated by the fingerprint sensor 102 to the fingerprint image processing device of the main processor 120 via the interface module 105. The first fingerprint image obtained by the fingerprint image processing device from the fingerprint sensor 102 is actually the raw data of the corresponding fingerprint image. The fingerprint image processing device may perform pre-processing such as image self-calibration, image quantization, and image enhancement on the first fingerprint image to improve the image quality or clarity of the first fingerprint image.

[0063] As mentioned above, due to the influence of the hardware structure characteristics of the fingerprint sensor 102, the first fingerprint image generated by the fingerprint sensor 102 is relatively narrow and the image area is small, so the first fingerprint image itself may carry relatively little fingerprint feature information. In particular, in the application scenario where the electronic device 100 adopts a side fingerprint, when the user holds the electronic device 100 and presses the fingerprint sensor 102 located on the side of the frame to input the fingerprint, there is some uncertainty in the position of the finger pressing. When the user presses the edge of the finger on the sensing area 103 of the fingerprint sensor 102, since the fingerprint feature points and fingerprint pattern information at the edge of the finger are relatively small, the fingerprint feature information carried in the first fingerprint image generated by the fingerprint sensor 102 will be even less. If the above-mentioned first fingerprint image is used to directly extract fingerprint feature information to generate a fingerprint template or perform fingerprint matching verification, it may cause the fingerprint verification of the legitimate user to fail, thereby affecting the fingerprint recognition performance of the electronic device 100 and reducing the user experience.

[0064] Therefore, the fingerprint image processing method provided by the embodiment of the present disclosure can execute step S202 to predict the fingerprint image extension information and corresponding image growth of the above-mentioned first fingerprint image after the fingerprint image processing device of the main processor 120 obtains the first fingerprint image from the fingerprint sensor 102, thereby increasing the fingerprint feature information of the first fingerprint image.

[0065] Step S202: using the global information of the first fingerprint image as input to predict fingerprint image extension information outside at least part of the boundary of the first fingerprint image, and performing image growth using the fingerprint image extension information to generate a second fingerprint image.

[0066] In the embodiment of the present disclosure, the second fingerprint image includes the original area corresponding to the first fingerprint image and the growth area obtained by image growth in step S202. The growth area is formed by image growth outside at least part of the boundary of the first fingerprint image based on the fingerprint image extension information predicted in step S202.

[0067] In the embodiment of the present disclosure, taking the first fingerprint image as a rectangular image as an example, the global information of the first fingerprint image can be used in step S202 to predict the fingerprint image extension information outside at least one edge of the first fingerprint image at one time.

[0068] As shown in Figures 1A and 1B , the fingerprint sensor 102 is located on the side of the frame of the electronic device 100. Typically, the sensing area 108 of the fingerprint sensor 102 is a narrow, long rectangle. For example, the fingerprint sensor 120 includes a sensing array 103 formed by 160*36 sensing electrodes and measuring 8mm*1.8mm. Therefore, the first fingerprint image generated by the fingerprint sensor 102 has a resolution of 160*36 narrow, long rectangle, i.e., the first fingerprint image only includes 160*36 fingerprint pixels. Because the short side of the first fingerprint image is small, it is difficult to carry sufficient fingerprint feature information for subsequent fingerprint matching verification. As an embodiment, in step S202, extended fingerprint image information beyond the two long sides of the first fingerprint image can be predicted in a single image prediction. This increases the number of fingerprint pixels in the short side direction of the first fingerprint image and improves the resolution of the fingerprint image. For example, the resolution of the second fingerprint image generated by image growing can be increased to 160*48 or other resolutions.

[0069] In this embodiment, as shown in FIG4 , the global information of the first fingerprint image 401 is used as input. The encoder-decoder model provided by the disclosed embodiment can predict the extended fingerprint image information beyond the two long sides of the first fingerprint image 401, and use the extended fingerprint image information to perform corresponding image growth to generate a second fingerprint image 402. The second fingerprint image 402 includes an original region corresponding to the first fingerprint image 401 and a first growth region 403 and a second growth region 404, respectively located beyond the two long sides of the first fingerprint image 401. In this embodiment, the encoder-decoder model is a specific optional fingerprint prediction model configured within the fingerprint image processing device of the main processor 120. The encoder-decoder model can use the global information of the first fingerprint image 401 to predict the extended fingerprint image information in at least two directions at one time and perform corresponding image growth, thereby effectively increasing the number of fingerprint pixels in the second fingerprint image 402. Since this embodiment does not require multiple expansions in different directions, it can simplify the processing process and improve fingerprint recognition performance.

[0070] As another embodiment, using step S202, the fingerprint image extended information outside the four sides of the first fingerprint image can be predicted at one time in a single image prediction, so as to increase the fingerprint pixels of the first fingerprint image in the short side direction to improve the resolution of the fingerprint image. For example, the resolution of the second fingerprint image generated by image growth can be increased to 172*48 or other.

[0071] In this embodiment, exemplarily, as shown in FIG5 , the global information of the first fingerprint image 501 is used as input. The encoder-decoder model provided by the embodiment of the present disclosure can simultaneously predict the fingerprint image extension information of the first fingerprint image 501 other than the two long sides and the two short sides and the corresponding four corner areas, and use the above-mentioned fingerprint image extension information to perform corresponding image growth to generate a second fingerprint image 502. The second fingerprint image 502 is a rectangular image including a growth area 503, wherein the growth area 503 is an annular area surrounding the original area corresponding to the first fingerprint image 501. In this embodiment, the encoder-decoder model also serves as a specific fingerprint prediction model. The encoder-decoder model can use the global information of the first fingerprint image 501 to predict the fingerprint image extension information in four directions and the corresponding four corner areas at one time and perform corresponding image growth, thereby effectively increasing the fingerprint pixels of the second fingerprint image 502. Since there is no need to expand the four sides of the first fingerprint image 501 separately, this embodiment can simplify the processing process and effectively avoid performing image growth separately in the expansion areas of the four boundary fingerprints of the first fingerprint image 501, resulting in secondary image expansion in the four corner areas, effectively reducing error accumulation and improving the accuracy of the fingerprint image expansion information prediction results.

[0072] In summary, through the disclosed embodiments, the first fingerprint image 401 or 501 generated by the fingerprint sensor 102 is input into the encoder-decoder model. The global information of the first fingerprint image 401 or 501 is used to predict the fingerprint image extension information outside at least a portion of the first fingerprint image 401 or 501 boundary at one time. Image growth is then performed using this fingerprint image extension information to increase the number of fingerprint pixels, thereby generating a second fingerprint image 402 or 502 with higher image resolution. The fingerprint image processing method provided by the disclosed embodiments can fully utilize the fingerprint image generated by the fingerprint sensor 102 to obtain more accurate prediction results, thereby helping to improve the accuracy of fingerprint recognition tasks. Furthermore, the global information of the first fingerprint image 401 or 501 is input into the encoder-decoder model, which performs a one-time fingerprint image extension information prediction and image growth, and outputs the grown second fingerprint image 402 or 502. This achieves end-to-end processing, simplifies the process, and improves efficiency. It should be understood that the encoder-decoder model shown in Figures 4 and 5 is only a specific and preferred fingerprint prediction model in the embodiments of the present disclosure. In other alternative embodiments, the encoder-decoder model may also be replaced by other fingerprint extension models with similar fingerprint image prediction functions, and the embodiments of the present disclosure do not make specific limitations here.

[0073] Furthermore, in some embodiments of the fingerprint image processing method provided by the present disclosure, before executing step S202, the necessity of image growth (or image information expansion) can be judged based on the image quality of the first fingerprint image to determine whether image growth or image information expansion is required for the first fingerprint image.

[0074] For example, the fingerprint image processing device of the main processor 120 can first perform a quality estimation based on image quality index parameters such as image clarity, signal-to-noise ratio, image effective signal volume, and structural similarity, and generate a quality score to comprehensively determine whether the image quality of the first fingerprint image obtained from the fingerprint sensor 102 meets a preset image quality threshold. If the quality score of the first fingerprint image obtained from the fingerprint sensor 102 fails to meet the preset image quality threshold, such as due to insufficient image clarity, excessive image noise, or insufficient image effective signal volume (e.g., lack of obvious fingerprint lines), it is foreseeable that the image growth region obtained by directly predicting image extension information based on the first fingerprint image and performing corresponding image growth will still have problems of low image quality or poor reliability. In this case, step S202 for performing image growth or image information expansion on the first fingerprint image can be abandoned. Through this embodiment, fingerprint images with better quality can be grown to obtain reliable prediction results, while avoiding growing images with poor quality, which may lead to unreliable prediction results. Optionally, if the image quality of the first fingerprint image is too poor, the fingerprint image processing device of the main processor 120 may further control the fingerprint sensor 102 to re-collect the user's fingerprint information and re-output the corresponding first fingerprint image to the fingerprint image processing device.

[0075] Specifically, the fingerprint image processing method provided by the embodiment of the present disclosure may further include: estimating the quality of the first fingerprint image and generating a quality score based on the image quality index of the first fingerprint image before inputting the first fingerprint image into the encoder-decoder model for fingerprint image extended information prediction and corresponding image growth; determining whether to input the first fingerprint image into the encoder-decoder model for fingerprint image extended information prediction and corresponding image growth based on the quality score; wherein the image quality index includes at least one of the fingerprint pattern clarity and signal-to-noise ratio of the first fingerprint image.

[0076] For ease of description, the embodiment shown in Figure 5 is used as an example. As an implementation method, estimating the quality of first fingerprint image 501 and generating a quality score may specifically include: determining the fingerprint ridge clarity of first fingerprint image 501; determining the signal-to-noise ratio of first fingerprint image 501; and determining the quality score of the first fingerprint image based on the signal-to-noise ratio and the fingerprint ridge clarity. For example, the quality score may be a weighted average of the signal-to-noise ratio and fingerprint ridge clarity of first fingerprint image 501. This implementation ensures that the encoder-decoder model only predicts fingerprint image extended information for fingerprint images with clear ridges and high signal-to-noise ratios, thereby obtaining highly reliable prediction results.

[0077] As an example, determining the signal-to-noise ratio of the first fingerprint image 501 may include: determining the mean and standard deviation of pixel values ​​of at least a portion of the first fingerprint image 501, and determining the ratio of the mean to the standard deviation as the signal-to-noise ratio of the first fingerprint image 501.

[0078] In some embodiments, the growth region and the original region of the second fingerprint image may have a certain imbalance in brightness and contrast. Based on the second fingerprint image, the growth region can be equalized to enhance the contrast and brightness of the growth region. This embodiment can enhance the brightness and contrast of the growth region without affecting the overall brightness and contrast of the second fingerprint image.

[0079] Taking the embodiment shown in FIG5 as an example, as an optional implementation, histogram equalization can be performed on the growth area 503 of the second fingerprint image 502. For example, assuming that the total number of fingerprint pixels in the second fingerprint image 502 is m, and the second fingerprint image 502 has a total of s gray levels g1 to g2. s , m j Refers to the gray level g j The number of fingerprint pixels, then in the second fingerprint image 502, the gray level g j The probability of occurrence P(g j ) can be expressed as formula (1).

[0080] By using the cumulative distribution function as the transformation function T(r), the second fingerprint image 502 can be converted into a balanced fingerprint image with a uniform grayscale distribution and a uniform probability density.

[0081] In this embodiment, after the second fingerprint image 502 is generated, the pixel values ​​of the growth area 503 of the second fingerprint image 502 after equalization can be further calculated based on the transformation function of formula (2), and the fingerprint pixels of the growth area 503 of the second fingerprint image 502 can be processed based on the above pixel values ​​to obtain the above-mentioned equalized fingerprint image.

[0082] In the disclosed embodiments, the above-mentioned fingerprint image processing method can be applied to the fingerprint matching verification stage. Specifically, the second fingerprint image generated by the above-mentioned fingerprint image processing method can be applied to fingerprint unlocking, fingerprint payment or other fingerprint recognition tasks. Since the growth area of ​​the second fingerprint image is the fingerprint pixels grown according to the predicted fingerprint image extension information, in order to reduce the risk of false recognition, in some optional implementations, in the fingerprint matching verification stage, the fingerprint information of the growth area of ​​the second fingerprint image is not directly used for fingerprint similarity judgment, but is mainly used only for performing image processing before fingerprint similarity judgment (such as determining matching relationships and overlapping areas, etc.).

[0083] For example, referring to FIG5 , the fingerprint information of the growth area 503 can be used to evaluate or precisely locate the positional relationship between the second fingerprint image 502 and the pre-stored fingerprint template and calculate the overlapping area between the two, so as to find the fingerprint feature information matching relationship or matching matrix between the second fingerprint image 502 and the fingerprint template, while the fingerprint similarity judgment between the second fingerprint image 502 and the fingerprint template can still only use the fingerprint information of the original area 501 of the second fingerprint image 502 (corresponding to the first fingerprint image 501, and the reference numeral 501 is used for ease of understanding). That is, during the fingerprint matching and verification phase, the main processor 120 of the electronic device 100 can extract fingerprint feature information corresponding to the fingerprint pixels of the original region 501 based on the original region 501 and the growth region 503 of the second fingerprint image 502; perform fingerprint similarity judgment based on the fingerprint feature information of the original region 501 and the reference fingerprint feature information in the pre-stored fingerprint template, and perform the corresponding fingerprint recognition task according to the judgment result; wherein the fingerprint feature information may include fingerprint feature point information (such as end points, breakpoints, intersections, turning points), fingerprint pattern information, texture detail information, and other relevant information that can characterize fingerprint features. In other alternative embodiments, during the fingerprint matching and verification phase, the main processor 120 can also simultaneously extract fingerprint feature information of the original region 501 and the growth region 502 of the second fingerprint image 502, but when performing fingerprint similarity judgment, only the fingerprint feature information of the original region 501 is used to match and compare with the reference fingerprint feature information in the fingerprint template to determine whether the similarity between the two meets the preset similarity judgment condition.

[0084] In other embodiments of the present disclosure, the above-mentioned fingerprint image processing method can also be applied to the fingerprint registration stage. For example, in the fingerprint registration stage, after collecting and generating the first fingerprint image 501 and performing image growth based on the fingerprint image extension information obtained by a one-time prediction of the global information of the first fingerprint image 501 to obtain the second fingerprint image 502, fingerprint feature information can be extracted from the original area 501 and the growth area 502 of the second fingerprint image 501 at the same time, and the extracted fingerprint feature information can be stored as an independent fingerprint template in the fingerprint template library for fingerprint similarity judgment in the fingerprint matching and verification stage. The fingerprint feature information can include fingerprint feature point information, fingerprint pattern information, and other information that can characterize fingerprint features. Alternatively, in other embodiments, in the fingerprint template registration stage, fingerprint feature information can be extracted only from the original area 501 of the second fingerprint image 502 to generate the corresponding fingerprint template.

[0085] Furthermore, the fingerprint sensor 102 may also support a fingerprint template self-learning function, and after the above-mentioned second fingerprint image 502 passes the matching verification in the fingerprint matching verification stage, the fingerprint feature information extracted from the original area 501 and the growth area 503 of the second fingerprint image 502 may also be stored in the fingerprint template library as a new independent fingerprint template or replace other fingerprint templates in the fingerprint template library according to the saturation level of the fingerprint template library, so that the fingerprint template library inside the electronic device 100 can automatically update the baseline fingerprint feature information of the legitimate user's finger to the latest state or add the baseline fingerprint feature information of the legitimate user's finger.

[0086] In some embodiments, step S202 may specifically include: defining a growth region outside at least a portion of the boundary of the first fingerprint image, and filling the growth region with preset pixel information to obtain an initial fingerprint extension image; inputting the initial fingerprint extension image as a whole into a codec model, predicting fingerprint image extension information of the growth region through the codec model, and performing image growth using the fingerprint image extension information to generate an output second fingerprint image. The codec model, as a fingerprint prediction model, may be consistent with the codec model shown in FIG4 or FIG5 , and the size of the growth region may be preset or dynamically set, which is not limited in the present embodiment.

[0087] Exemplarily, as shown in FIG6 , a growth region 602 is first defined outside the four boundaries of the first fingerprint image 601, and preset pixel information is filled in the growth region 602 to obtain an initial fingerprint extended image 603, wherein the preset pixel information may have a specific pixel value, such as a pixel value of 0; then, the initial fingerprint extended image 603 is input as a whole into the encoder-decoder model, and the encoder-decoder model predicts the fingerprint image extended information within the growth region 602 based on the global information of the first fingerprint image 601; further, the encoder-decoder model uses the fingerprint image extended information to perform image growth in the growth region 602 outside the first fingerprint image 601, thereby obtaining and outputting a second fingerprint image 604, as shown in FIG6 , the second fingerprint image 604 includes an original region corresponding to the first fingerprint image 601 and a growth region 602 obtained by performing image growth outside the original region, wherein, after being processed by the encoder-decoder, the preset pixel information filled in the growth region 602 of the initial fingerprint extended image 603 is changed to fingerprint pixel information corresponding to the fingerprint image extended information.

[0088] In some implementations, the fingerprint prediction models of the various embodiments described above can be implemented using a codec model 700 as shown in FIG7 . The codec 700 may include a coding block sequence 710 and a decoding block sequence 720. The codec 700 extracts global fingerprint information from the first fingerprint image using the coding block sequence 710 and decodes the global fingerprint information using the decoding block sequence 720 to output a second fingerprint image. Through this implementation, the codec 700 can extract global fingerprint information from the entire fingerprint image generated by the fingerprint sensor 102 and accurately predict fingerprint information from the growth area of ​​the second fingerprint image, such as feature point information and texture distribution information of fingerprint ridges and valleys.

[0089] As shown in FIG7 , a coding block sequence 710 may include a plurality of sequentially connected coding blocks C1 to C p The first coding block C1 can receive the first fingerprint image, and the coding blocks C2 to C p The preceding coding blocks C1 to C p-1 The output of is input. The decoding block sequence 720 may include a plurality of sequentially connected decoding blocks D1 to D q The first decoding block D1 can receive the fingerprint coding signal output by the coding block sequence 710, and the decoding blocks D2 to D q , blocks D1 to D can be decoded by their preambles respectively q-1 The output of is the input.

[0090] As an implementation manner, as shown in FIG7 , any one coding block C in the coding block sequence 710 i(1≤i≤p) and the coded block C in the decoded block sequence 720 i Symmetric decoding block D j (1≤j≤q) jump connections to connect the coding block C i Feature information and corresponding decoding block D j By fusing feature information at different levels, the prediction capability of the encoder / decoder model 700 can be improved, ensuring that the fingerprint pixels in the growth area obtained by using the encoder / decoder model 700 for fingerprint image extension information prediction and image growth are more reasonable, which helps to improve the correct recognition rate of subsequent fingerprint recognition tasks.

[0091] As an implementation method, as shown in FIG7 , at least one of the above-mentioned coding blocks C i Including encoding convolution layer and downsampling layer, at least one of the above decoding blocks D j It includes upsampling layer, skip connection layer and decoding convolution layer, in which the coding block C is connected to the decoder through the skip connection layer. i Feature information and corresponding decoding block D j The encoder / decoder model of this embodiment has a simple structure and can quickly process the first fingerprint image and output the second fingerprint image. For example, the downsampling layer can be a pooling layer, and the upsampling layer can be a deconvolution layer.

[0092] As an embodiment, the fingerprint image received by the coding block sequence 710 may be the first fingerprint image 601 shown in FIG6 , or may be the initial fingerprint extended image 603 obtained after the first fingerprint image 601 is filled with preset pixel information. Alternatively, the coding block sequence 710 may also include a preset pixel information filling module, which is configured to receive the first fingerprint image 601 and fill the growth region 602 outside at least one boundary of the first fingerprint image 601 with preset pixel information to obtain the initial fingerprint extended image 603 and output it to the first coding block C1; alternatively, the preset pixel information filling module may also be implemented within the first coding block C1.

[0093] In order to better understand the fingerprint image processing method provided by the embodiment of the present disclosure, the following, in combination with Figure 8, specifically introduces the fingerprint image extension information prediction and the corresponding image growth process of the encoder-decoder model provided by the embodiment of the present disclosure; it should be understood that the following description is only an optional implementation scheme of the embodiment of the present disclosure.

[0094] First, the encoder-decoder model 700 fills predetermined pixels outside at least one image boundary of the first fingerprint image 601 to obtain an initial fingerprint extended image 603 .

[0095] Specifically, the encoder-decoder model 700 can obtain a first fingerprint image 601 generated by the fingerprint sensor 102. The first fingerprint image 601 is a fingerprint image with a height of n and a width of m generated based on the fingerprint information collected by the sensing area 108 of the fingerprint sensor 102. Furthermore, the encoder-decoder model 700 defines a growth region 602 outside at least one image boundary of the first fingerprint image 601 and fills the growth region 602 with predetermined pixel information to obtain an initial fingerprint expansion image 603. For example, the number of channels of the first fingerprint image 601 is 1. In this embodiment, the growth area 602 is defined as a predetermined area outside the upper, lower, left, and right boundaries of the first fingerprint image 601. Based on the predetermined growth area 602, the encoder-decoder model 700 can fill k predetermined pixel information (such as pixel 0) outside the upper, lower, left, and right boundaries of the first fingerprint image 601, thereby obtaining an input tensor with a height of n+2*k, a width of m+2*k, and a channel of 1 (that is, an input tensor of (n+2*k)*(m+2*k)*1), which is the initial fingerprint extended image 603.

[0096] Next, the initial fingerprint extended image 603 is encoded multiple times using the encoding block sequence 710 of the encoder-decoder model 700 to obtain a fingerprint encoded image, wherein the encoding process includes sequentially performing multiple convolution operations and corresponding average pooling downsampling processes on the initial fingerprint extended image 603.

[0097] As shown in FIG8 , in this embodiment, the encoder-decoder model 700 may first receive the initial fingerprint extended image 603 through the first encoding block C1, and perform a first encoding convolution operation on the initial fingerprint extended image 603 using the encoding convolution layer of the first encoding block C1, and then perform a first average pooling sampling process on the result of the first encoding convolution operation using the downsampling layer of the first encoding block C1 to obtain a first encoding output tensor. For example, the first encoding block C1 may perform a first encoding convolution operation with a convolution kernel size of 3*3 and a channel of 1 on the initial fingerprint extended image 603 through the encoding convolution layer to obtain a new (n+2*k)*(m+2*k)*1 tensor, and then obtain a (n+2*k) / 2*(m+2*k) / 2*1 tensor after the first average pooling downsampling process of the downsampling layer of the first encoding block C1, i.e., the first encoding output tensor.

[0098] The first encoding output tensor can be further output to the second encoding block C2, and the second encoding convolution operation and corresponding average pooling downsampling processing are performed on it through the encoding convolution layer and downsampling layer of the second encoding block C2, respectively, to obtain a second encoding output tensor. For example, the second encoding block C2 can perform a second encoding convolution operation with a convolution kernel size of 3*3 and a channel of 4 on the initial fingerprint expansion image 603 through its internal encoding convolution layer to obtain a new (n+2*k) / 2*(m+2*k) / 2*4 tensor, and then after the second average pooling downsampling processing of the downsampling layer of the second encoding block C2, a (n+2*k) / 4*(m+2*k) / 4*4 tensor is obtained, i.e., the second encoding output tensor.

[0099] Similarly, the encoding convolution layer and downsampling layer of the third encoding block C3 can respectively perform a third encoding convolution operation and corresponding average pooling downsampling processing on the second encoding output tensor to obtain a third encoding output tensor; for example, the second encoding output tensor can be subjected to an encoding convolution operation with a convolution kernel size of 3*3 and a channel number of 8 by the encoding convolution layer of the third encoding block C3 to obtain a (n+2*k) / 4*(m+2*k) / 4*8 tensor, and then subjected to a third average pooling downsampling processing by the downsampling layer of the third encoding block C3 to obtain a (n+2*k) / 8*(m+2*k) / 8*8 tensor, that is, the third encoding output tensor.

[0100] In this embodiment, the encoding block sequence 710 performs four encoding processes on the fingerprint image as an example (i.e., p=4). The fourth encoding block C4 further uses its encoding convolution layer to perform a fourth encoding convolution operation on the third encoding output tensor to obtain a fourth encoding output tensor. For example, the third encoding output tensor can be further subjected to an encoding convolution operation with a convolution kernel size of 3*3 and a number of channels of 16 by the encoding convolution layer of the fourth encoding block C4 to obtain a (n+2*k) / 8*(m+2*k) / 8*16 tensor, which can be used as the fourth encoding output tensor.

[0101] At this point, all operations of the encoding block sequence 710 are completed, and the fourth encoding block C4 can use the fourth encoded output tensor obtained as the fingerprint encoded image and output it to the decoder sequence 720. It can be seen that in this embodiment, the encoder-decoder 700 encodes the n*m*1 input (i.e., the first fingerprint image 601) into a (n+2*k) / 8*(m+2*k) / 8*16 tensor, i.e., the fingerprint encoded image, through a series of convolution and downsampling operations of its encoding block sequence 710.

[0102] Finally, the fingerprint encoded image is decoded using the decoding block sequence 720 of the encoder-decoder model 700 to obtain a fingerprint decoded image, wherein the decoding process includes performing multiple convolution operations on the fingerprint encoded image and corresponding upsampling convolution magnification and merging processing, and the fingerprint decoded image can be output as the image growth result of the encoder-decoder model 700, that is, the output is the above-mentioned second fingerprint image 604.

[0103] Specifically, as shown in Figure 8, in this embodiment, the encoder-decoder model 700 can first receive the fingerprint encoded image output by the encoding block sequence 710 through the first decoding block D1 of the decoding block sequence 720, and use the decoding convolution layer of the first decoding block D1 to perform a first decoding convolution operation on the fingerprint encoded image, and then use the upsampling layer of the first decoding block D1 to perform a first upsampling convolution amplification process on the first decoding convolution result, and at the same time, by using the jump connection layer of the first decoding block D1 and the corresponding encoding block in the encoding block sequence 710 to merge the tensors of the same dimension, a first decoding output tensor after feature fusion is obtained. For example, the decoding convolution layer of the first decoding block D1 can perform a first decoding convolution operation on the (n+2*k) / 8*(m+2*k) / 8*16 tensor output by the encoding block sequence 710 with a restrictive convolution kernel size of 3*3 and a channel number of 8 to obtain a new (n+2*k) / 8*(m+2*k) / 8*8 tensor, and then after the first 2*2 upsampling convolution amplification through the upsampling layer, obtain (n+2*k) / 4*(m+2*k) / 4*8. At the same time, the feature information of the tensors of the same dimension in the corresponding encoding blocks (such as the fourth encoding block C4) in the encoding and decoding model 700 is merged or combined to obtain a feature-fused (n+2*k) / 4*(m+2*k) / 4*16 tensor as the first decoding output tensor of the first decoding block D1.

[0104] Furthermore, after the first decoding output tensor (n+2*k) / 4*(m+2*k) / 4*16 enters the second decoding block D2, it undergoes a second decoding convolution operation through the decoding convolution layer of the second decoding block D2 to obtain a new (n+2*k) / 4*(m+2*k) / 4*4 tensor, and then undergoes a second upsampling through the upsampling layer of the second decoding block D2. At the same time, the feature information is merged or combined with the tensor of the same dimension in the corresponding encoding block in the encoding / decoding model 700 (such as the third encoding block C3) to obtain a feature-fused (n+2*k) / 2*(m+2*k) / 2*8 tensor as the second decoding output tensor of the second decoding block D2. The second decoding output tensor is then subjected to a third decoding convolution operation by the decoding convolution layer of the third decoding block D3 to obtain a new (n+2*k) / 2*(m+2*k) / 2*1 tensor, and then subjected to a third upsampling by the upsampling layer of the third decoding block D3. At the same time, the feature information is merged or combined with the tensor of the same dimension in the corresponding coding block in the coding-decoder model 700 (such as the second coding block C2) to obtain a feature-fused (n+2*k)*(m+2*k)*2 tensor as the third decoding output tensor of the third decoding block D3.

[0105] In this embodiment, since the encoding block sequence 710 of the encoder-decoder model 700 undergoes four encoding processes, the decoding block sequence 720 correspondingly performs four decoding processes on the output of the encoding block sequence 710. Specifically, the third decoded output tensor is finally subjected to the fourth decoding convolution process by the fourth decoding block D4, resulting in a fourth decoded output tensor of the target size (n+2*k)*(m+2*k)*1. After further 1*1 convolution and activation of the Sigmoid function, the final output, i.e., the decoded fingerprint image, is obtained. At this point, the decoding block sequence 720 completes the decoding process of the fingerprint encoded image output by the encoding block sequence 710, and the resulting decoded fingerprint image can be used as the second fingerprint image 604 described above. It can be seen that after the fingerprint image extension information prediction and image growth processing of the encoder-decoder model 700, the output second fingerprint image 604 grows a total of (n+m)*2k+4k*k fingerprint pixels in the top, bottom, left and right directions on the basis of the first fingerprint image 601. That is, according to the predicted fingerprint image extension information, k rows or k columns of fingerprint pixels are added outside the four boundaries of the top, bottom, left and right directions on the basis of the original area of ​​the first fingerprint image 601.

[0106] The specific process of the fingerprint image processing can be summarized by the flowchart shown in Figure 9. Figure 9 shows a flowchart of a fingerprint image processing method according to an exemplary embodiment of the present disclosure. As shown in Figure 9, the fingerprint image processing method of the embodiment of the present disclosure includes steps S901 to S907.

[0107] Step S901: Collect fingerprint information through a fingerprint sensor and generate a first fingerprint image.

[0108] Step S902: perform quality estimation on the first fingerprint image and generate a quality score, and determine whether to perform image growth on the first fingerprint image based on the quality score. If yes, proceed to step S903.

[0109] Step S903: Filling preset pixels outside at least a portion of the boundary of the first fingerprint image to form a growth area, thereby obtaining an initial fingerprint expansion image.

[0110] Step S904: input the initial fingerprint extended image into the fingerprint prediction model. The fingerprint prediction model predicts the fingerprint image extended information of the growth area based on the global information of the first fingerprint image, and uses the fingerprint image extended information to perform image growth in the growth area to generate and output a second fingerprint image.

[0111] In some embodiments, step S904 may be implemented by a fingerprint prediction model, such as using the encoder-decoder model 700 shown in FIG. 7 and FIG. 8 to predict the fingerprint image extension information and perform corresponding image growth.

[0112] Step S905: performing equalization processing on the growth area of ​​the second fingerprint image to enhance at least one of the contrast and brightness of the growth area.

[0113] Step S906: performing a corresponding fingerprint recognition task based on the equalized second fingerprint image.

[0114] The fingerprint image processing method shown in FIG9 can be applied to the electronic device 100 shown in FIG1 , and the implementation process of each step of FIG9 can refer to the detailed description in the above specification, which will not be repeated here.

[0115] The fingerprint image processing method provided by the embodiments of the present disclosure can utilize the global information of the fingerprint image to expand the fingerprint pixel area in the four directions of the top, bottom, left, and right of the fingerprint image at one time based on the fingerprint image collected by the fingerprint sensor. This eliminates the need for four separate expansions, greatly simplifying the process and avoiding the problem of error accumulation in the four corner areas of the fingerprint image. Furthermore, by utilizing the global information of the entire fingerprint image for predicting fingerprint expansion information and corresponding image growth, rather than just using the edge areas of the image, more accurate prediction results can be obtained. Furthermore, by using the encoder-decoder model provided by the embodiments of the present disclosure for predicting fingerprint image expansion information and image growth, the model's prediction capability can be enhanced by fusing fingerprint feature information at different levels. The growth area formed around the periphery of the fingerprint image is more reasonable, helping to improve the accuracy of subsequent fingerprint recognition tasks. Furthermore, the fingerprint image processing method provided by the embodiments of the present disclosure can directly expand the fingerprint image collected by the fingerprint sensor into the full image after fingerprint image growth. The end-to-end design can simplify the expansion task process and improve efficiency.

[0116] The present disclosure also provides an embodiment of a fingerprint image processing device, which can form a fingerprint recognition system within the electronic device 100 together with the fingerprint sensor 102 shown in Figures 1A and 1B. The fingerprint image processing device can be specifically the fingerprint image processing device 120 shown in Figure 1B, which can be configured in the main processor 120 (such as the central processing unit CPU) of the electronic device 100 to execute the main steps of the fingerprint image processing method described in the above embodiments. In other alternative embodiments, the fingerprint image processing device 120 can also be implemented using other processing units or control units with image processing capabilities (such as a microcontroller MCU). As shown in Figure 10, the fingerprint image processing device provided by an embodiment of the present disclosure may include:

[0117] An acquisition module 1001, which can be connected to an interface module of the fingerprint sensor 102, is used to acquire a first fingerprint image generated by the fingerprint sensor 102;

[0118] The processing module 1002 is configured to use the global information of the first fingerprint image as input to predict fingerprint image extension information outside at least a portion of the boundary of the first fingerprint image, and to perform image growth using the fingerprint image extension information to generate a second fingerprint image, wherein the second fingerprint image includes an original area corresponding to the first fingerprint image and a growth area obtained by image growth outside at least a portion of the boundary of the first fingerprint image.

[0119] In some embodiments, the processing module 1002 may include a fingerprint prediction model, such as the encoder-decoder model shown in Figures 7 and 8, which is mainly used to perform prediction of fingerprint image extension information and corresponding image growth to fill the predicted fingerprint pixels in the growth area and obtain a second fingerprint image.

[0120] In some embodiments, the processing module 1002 may also be configured to perform quality estimation on the first fingerprint image and generate a quality score before performing image growth on the first fingerprint image; and determine whether to input the first fingerprint image into the fingerprint prediction model for image growth based on the quality score.

[0121] As an implementation, the processing module 1002 may be specifically configured to: determine fingerprint ridge clarity of the first fingerprint image; determine a signal-to-noise ratio of the first fingerprint image; and calculate a quality score of the first fingerprint image based on the signal-to-noise ratio and the fingerprint ridge clarity.

[0122] In some implementations, the processing module 1002 may also be configured to perform equalization processing on the growth region of the second fingerprint image to enhance at least one of the contrast and brightness of the growth region.

[0123] In some embodiments, the processing module 1002 can also be used to perform a corresponding fingerprint recognition task based on the second fingerprint image, where the fingerprint recognition task includes performing a similarity judgment between the second fingerprint image and a pre-stored fingerprint template to confirm whether the fingerprint information collected by the fingerprint sensor 102 is from the fingerprint of a valid user, where the similarity judgment is performed using the fingerprint information of the original area of ​​the second fingerprint image.

[0124] In some embodiments, the first fingerprint image and the second fingerprint image are rectangular, and the growth area is a ring surrounding the first fingerprint image.

[0125] As an embodiment, the processing module 1002 can be specifically used to: fill preset pixels outside at least part of the boundary of the first fingerprint image to form a growth area to obtain an initial fingerprint extension image; input the initial fingerprint extension image into the fingerprint prediction model to predict the fingerprint image extension information of the growth area through the fingerprint prediction model and perform corresponding image growth to generate and output a second fingerprint image.

[0126] In some embodiments, the encoder-decoder model serving as the above-mentioned neural network model may be as shown in FIG7 and FIG8 , which includes an encoding block sequence 710 and a decoding block sequence 720 . The specific structure and function of the encoding block sequence 710 and the decoding block sequence 720 may refer to the specific description of the above embodiments.

[0127] The present disclosure also provides an electronic device 100, comprising a device body 101 and the aforementioned fingerprint sensor 102 disposed on the device body 101. In some embodiments, the electronic device 100 may be a portable electronic device, such as a smartphone, tablet computer, laptop computer, personal digital assistant, etc. Alternatively, the electronic device 100 may be a smart wearable device, which is not limited in the present disclosure.

[0128] The electronic device 100 provided by the embodiment of the present disclosure may further include: a main processor 120; and a memory for storing programs, wherein the program includes instructions, and when the instructions are executed by the main processor 120, the processor 120 executes the method of the above embodiment, such as the method shown in Figures 2 to 9.

[0129] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the main processor 120 of the electronic device 100 to execute the method of the above embodiment, such as the method shown in Figures 2 to 9.

[0130] With reference to Figure 11, it is a block diagram of the structure of the electronic device 1100 provided in an embodiment of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device 1100 may include: a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0131] Multiple components within electronic device 1100 are connected to I / O interface 1105, including an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. Input unit 1106 can be any type of device capable of inputting information into electronic device 1100. Input unit 1106 can receive input numeric or character information and generate key input signals related to user settings and / or function control of the electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 1108 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0132] The computing unit 1101 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described in this disclosure. For example, in some embodiments, the fingerprint image processing method of the embodiments of the present disclosure can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the method of this embodiment by any other appropriate means (e.g., by means of firmware).

[0133] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The above are merely preferred embodiments of the present disclosure and are not intended to limit the present disclosure in any form. Although the present disclosure has been disclosed as above with preferred embodiments, they are not intended to limit the present disclosure. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present disclosure. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present disclosure without departing from the content of the technical solution of the present disclosure are still within the scope of the technical solution of the present disclosure.

Claims

1. A fingerprint image processing method, characterized in that: include: Acquire a first fingerprint image generated by a fingerprint sensor; The global information of the first fingerprint image is used as input to predict fingerprint image extension information outside at least part of the boundary of the first fingerprint image, and the fingerprint image extension information is used to perform corresponding image growth to generate a second fingerprint image.

2. The method according to claim 1, wherein The prediction of the fingerprint image extension information and the corresponding image growth are performed by a fingerprint prediction model; wherein, the second fingerprint image includes an original area corresponding to the first fingerprint image, and a growth area obtained by the fingerprint prediction model performing image growth outside at least part of the boundary of the original area based on the fingerprint image extension information.

3. The method according to claim 2, wherein Also includes: pre-defining a growth area of the second fingerprint image outside at least a portion of the boundary of the first fingerprint image; Filling predetermined pixel information in the growth area to obtain an initial fingerprint extension image; The initial fingerprint extended image is input into the fingerprint prediction model, and the fingerprint prediction model is used to predict the fingerprint image extended information and perform corresponding image growth to form fingerprint pixel information corresponding to the fingerprint image extended information in the growth area.

4. The method according to claim 3, wherein The fingerprint prediction model is a codec model, which includes a coding block sequence and a decoding block sequence, wherein the prediction of the fingerprint image extended information and the corresponding image growth include: Performing multiple encoding processes on the initial fingerprint extended image using the encoding block sequence to obtain a fingerprint encoded image, wherein the encoding process includes sequentially performing multiple convolution operations and corresponding average pooling downsampling processes on the initial fingerprint extended image; The fingerprint encoded image is decoded using the decoding block sequence to obtain a fingerprint decoded image, wherein the decoding process includes performing multiple convolution operations and corresponding upsampling convolution magnification and merging processes on the fingerprint encoded image, and the fingerprint decoded image is used as a second fingerprint image output by the encoder-decoder model.

5. The method according to claim 4, wherein The coding block sequence includes a plurality of coding blocks connected in sequence, the output of the previous coding block serves as the input of the next coding block, and each coding block includes a coding convolution layer and a downsampling layer; Among them, the step of using the coding block sequence to perform multiple encoding processing on the initial fingerprint extended image includes: receiving the initial fingerprint extended image, and performing multiple encoding convolution operations and corresponding average pooling downsampling processing on the initial fingerprint extended information through the encoding convolution layer and downsampling layer of the multiple encoding blocks to obtain the fingerprint encoded image.

6. The method according to claim 5, wherein The decoding block sequence includes a plurality of decoding blocks connected in sequence, the output of a previous decoding block serves as the input of a subsequent decoding block, each decoding block corresponds to one of the coding blocks in the coding block sequence, and includes an upsampling layer, a skip connection layer, and a decoding convolution layer, wherein the skip connection layer is used to combine the feature information of the decoding block with the corresponding coding block in the coding block sequence; Among them, the step of using the decoding block sequence to decode the fingerprint encoded image includes: receiving the fingerprint encoded image output by the encoding block sequence, and performing multiple decoding convolution operations and upsampling convolution amplification processing on the fingerprint encoded image through the upsampling layer and decoding convolution layer of the multiple decoding blocks, and at the same time combining it with the feature information of the corresponding encoding block in the encoding block sequence to obtain the fingerprint decoded image.

7. The method according to claim 1, wherein Before performing the prediction of the fingerprint image extension information and the corresponding image growth, the method further includes: Estimating the quality of the first fingerprint image according to an image quality indicator of the first fingerprint image and generating a quality score, wherein the image quality indicator comprises at least one of fingerprint ridge clarity and a signal-to-noise ratio of the first fingerprint image; Based on the quality score, it is determined whether to perform image growth on the first fingerprint image.

8. The method according to claim 1, wherein Also includes: performing equalization processing on the growth area of the second fingerprint image to enhance at least one of contrast and brightness of the growth area; The equalization processing includes: after the second fingerprint image is generated, calculating the pixel values of the growth area of the second fingerprint image after equalization according to the following transformation function, and processing the fingerprint pixels of the growth area based on the pixel values to obtain the equalized second fingerprint image; Wherein, m represents the total number of fingerprint pixels of the second fingerprint image, and s represents the total number of gray levels of the second fingerprint image, m i Indicates gray level is g i The number of fingerprint pixels, P(g i ) represents the gray level g of the second fingerprint image j Probability of occurrence.

9. The method according to claim 2, wherein Also includes: In the fingerprint matching and verification stage, fingerprint feature information is extracted from the original area and the growth area of the second fingerprint image, and the fingerprint feature information extracted from the original area is matched and verified with the reference fingerprint feature information of the pre-stored fingerprint template to determine whether the user's input fingerprint is a legal fingerprint, and the fingerprint recognition task is performed when the input fingerprint is a legal fingerprint; wherein the fingerprint feature information of the growth area of the second fingerprint image is not directly used for determining the similarity between the second fingerprint image and the pre-stored fingerprint template.

10. The method according to claim 9, wherein The fingerprint feature information extracted from the growth area of the second fingerprint image is used to perform image positioning and overlapping area calculation between the second fingerprint image and the pre-stored fingerprint template to find a matching relationship between the two, wherein the similarity judgment of the fingerprint feature information is calculated based on the overlapping area between the second fingerprint image and the fingerprint template.

11. The method according to claim 9, wherein Also includes: In the fingerprint registration stage, fingerprint feature information is extracted from the original area and the growth area of the second fingerprint image respectively, as reference fingerprint feature information and stored as an independent fingerprint template in the fingerprint template library; or, In the fingerprint matching verification stage, when it is determined that the user's input fingerprint is a valid fingerprint, the fingerprint template self-learning function is executed, and the fingerprint feature information extracted from the original area and the growth area of the second fingerprint image are used as the reference fingerprint feature information, and stored as a new fingerprint template in the fingerprint template library or replaces another fingerprint template in the fingerprint template library.

12. The method according to claim 1, wherein Also includes: Collecting a user's input fingerprint through a fingerprint sensor and generating the first fingerprint image; Among them, the fingerprint sensor is a capacitive fingerprint sensor arranged on the side of the electronic device frame and has a narrow and long rectangular structure, and it includes a sensing array, which includes a plurality of sensing electrodes distributed in an array. The capacitive fingerprint sensor uses the sensing array to collect the user's input fingerprint and generates the first fingerprint image based on the input fingerprint.

13. A fingerprint image processing device, comprising: An acquisition module, configured to acquire a first fingerprint image generated by a fingerprint sensor; A processing module is used to use the global information of the first fingerprint image as the input of a fingerprint prediction model, and to predict the fingerprint image extension information outside at least part of the boundary of the first fingerprint image through the fingerprint prediction model, and to use the fingerprint image extension information to perform image growth to generate a second fingerprint image; wherein, the second fingerprint image includes an original area corresponding to the first fingerprint image, and a growth area obtained by the fingerprint prediction model performing image growth outside at least part of the boundary of the original area according to the fingerprint image extension information.

14. The fingerprint image processing device according to claim 13, wherein: The fingerprint prediction model is used to predefine the growth area outside at least part of the boundary of the first fingerprint image and fill it with predetermined pixel information to obtain an initial fingerprint extended image, and to predict the fingerprint image extended information and perform corresponding image growth based on the input initial fingerprint extended image to form fingerprint pixel information corresponding to the fingerprint image extended information in the growth area.

15. The fingerprint image processing device according to claim 14, wherein: The fingerprint prediction model is a codec model, and the codec model includes a coding block sequence and a decoding block sequence; The coding block sequence includes a plurality of sequentially connected coding blocks, wherein the output of a previous coding block serves as the input of a subsequent coding block, and each coding block includes a coding convolution layer and a downsampling layer; the coding block sequence is used to sequentially perform multiple convolution operations and corresponding average pooling downsampling processes on the initial fingerprint extended image through the coding convolution layers and downsampling layers of the plurality of coding blocks to obtain a fingerprint coded image; The decoding block sequence includes multiple decoding blocks connected in sequence, wherein the output of the previous decoding block serves as the input of the next decoding block, each decoding block corresponds to one of the encoding blocks in the sequence, and includes an upsampling layer, a skip connection layer and a decoding convolution layer, wherein the skip connection layer is used to combine the decoding block with the feature information of the corresponding encoding block in the encoding block sequence; the decoding block sequence is used to use the multiple decoding blocks to perform multiple convolution operations and corresponding upsampling convolution amplification and merging processing on the fingerprint encoding image and obtain a fingerprint decoded image, wherein the fingerprint decoded image serves as the second fingerprint image output by the encoding-decoder model.

16. The fingerprint image processing device according to claim 13, wherein: The processing module is further configured to: perform a quality estimation on the first fingerprint image according to an image quality index of the first fingerprint image and generate a quality score, and determine whether to perform image growth on the first fingerprint image based on the quality score; wherein the image quality index comprises at least one of fingerprint ridge clarity and a signal-to-noise ratio of the first fingerprint image.

17. The fingerprint image processing device according to claim 13, wherein: The processing module is further configured to extract fingerprint feature information from the original area and the growth area of the second fingerprint image during a fingerprint matching and verification phase, and to match and verify the fingerprint feature information extracted from the original area with the reference fingerprint feature information of a pre-stored fingerprint template to determine whether the user's input fingerprint is a legitimate fingerprint, and to perform a corresponding fingerprint recognition task when the input fingerprint is a legitimate fingerprint; wherein the fingerprint feature information of the growth area of the second fingerprint image is not directly used for determining the similarity between the second fingerprint image and the pre-stored fingerprint template.

18. The fingerprint image processing device according to claim 17, wherein: The fingerprint feature information extracted from the growth area of the second fingerprint image is used to perform image positioning and overlapping area calculation between the second fingerprint image and the pre-stored fingerprint template to determine a matching relationship between the two, wherein the processing module calculates the similarity between the second fingerprint image and the fingerprint template based on the overlapping area between the two.

19. The fingerprint image processing device according to claim 17, wherein: The processing module is further configured to: during the fingerprint registration phase, extract fingerprint feature information from the original region and the growth region of the second fingerprint image, respectively, as reference fingerprint feature information and store it as an independent fingerprint template in the fingerprint template library; or, In the fingerprint matching verification stage, when it is determined that the user's input fingerprint is a valid fingerprint, the fingerprint template self-learning function is executed, and the fingerprint feature information extracted from the original area and the growth area of the second fingerprint image are used as the reference fingerprint feature information, and stored as a new fingerprint template in the fingerprint template library or replaces another fingerprint template in the fingerprint template library.

20. An electronic device, characterized in that: include: A fingerprint sensor, configured to collect a user's input fingerprint and generate a first fingerprint image; A fingerprint image processing device is connected to the fingerprint sensor, and is used to obtain a first fingerprint image generated by the fingerprint sensor, and use global information of the first fingerprint image as input to predict fingerprint image extension information outside at least part of the boundary of the first fingerprint image, and use the fingerprint image extension information to perform image growth to generate a second fingerprint image.

21. An electronic device comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 12.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause a computer to execute the method according to any one of claims 1-12.

Images