A non-contact palmprint image quality assessment method, device and smart terminal

Abstract

This invention discloses a non-contact palmprint image quality assessment method, device, and smart terminal. The method includes: acquiring a palmprint image and obtaining a palm region based on the palmprint image; assessing the quality of the palm region to obtain a palmprint quality index; and obtaining the palmprint image quality based on the palmprint quality index. This invention can assess the influence of open ambient lighting and hand posture on palmprint quality in non-contact palmprint recognition, and can assess the flatness of the palm on the imaging plane based on hand posture. It considers both the open ambient lighting characteristic of non-contact palmprint recognition and the flexible hand posture, meeting the needs of non-contact palmprint quality assessment under open ambient lighting in natural scenes. It achieves a comprehensive consideration of palmprint-related quality factors and has high robustness.

Description

Technical Field

[0001] This invention relates to the field of image recognition, specifically to a non-contact palmprint image quality assessment method, device, and smart terminal. Background Technology

[0002] Palmprint recognition technology can automatically identify users based on the biometric features of the human palm surface (such as epidermal lines, wrinkles, spots, and subcutaneous vein distribution), offering advantages such as fast verification speed and objective accuracy. The technique using subcutaneous vein information for identification within palmprint recognition is also called palm vein recognition. Compared to contact-based palmprint recognition, non-contact palmprint recognition technology offers advantages such as natural interaction and hygiene, and is currently widely used in the field of biometric identification. However, the non-contact, free-hand palm recognition method introduces problems such as palm tilt, uneven palm surfaces, and distance issues; simultaneously, the open recognition environment presents complex external lighting and background problems, significantly impacting the specific steps of palmprint recognition, such as region of interest (ROI) localization, palmprint feature extraction, feature matching, and recognition. Therefore, image quality assessment is crucial.

[0003] However, existing image quality assessment methods typically use the mean and variance of image pixel gray values, as well as features such as the area and perimeter of local regions for quality assessment. These methods have too few quality features, are easily affected by noise, and cannot meet the robustness requirements of global quality assessment, resulting in poor performance in real-world scenarios.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a non-contact palmprint image quality assessment method, device and smart terminal to address the above-mentioned defects of the prior art, and to solve the problems of insufficient quality features, susceptibility to noise and inability to meet the robustness requirements of global quality assessment in the prior art.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a non-contact palmprint image quality assessment method, wherein the method includes:

[0008] Acquire a palmprint image and determine the palm region based on the palmprint image;

[0009] The palm area is subjected to quality assessment to obtain palm print quality indicators;

[0010] The palmprint image quality is obtained based on the palmprint quality index.

[0011] In one implementation, acquiring a palmprint image and obtaining a palm region based on the palmprint image includes:

[0012] The palm print image is input into an image semantic segmentation network for semantic segmentation to obtain the palm region.

[0013] In one implementation, the step of performing quality assessment based on the palm region to obtain palmprint quality indicators includes:

[0014] A pixel-level quality assessment of the palm region is performed to obtain brightness and sharpness assessment indices.

[0015] Extract the skeletal joints in the palm region and evaluate the palm posture based on the skeletal joints to obtain the palm center region integrity index, palm center region resolution evaluation index, finger extension index, and palm flatness index.

[0016] The palmprint quality index is obtained based on the brightness evaluation index, sharpness evaluation index, palm center area integrity index, palm center area resolution evaluation index, finger extension index, and palm flatness index.

[0017] In one implementation, the step of performing a palm pixel-level quality assessment on the palm region to obtain a brightness assessment index and a sharpness assessment index includes:

[0018] The palm region is divided into several image blocks;

[0019] Obtain the average pixel value for each image patch;

[0020] The mean of the pixel values ​​of all image blocks is calculated to obtain the illumination level evaluation index.

[0021] Subtract the average pixel value of each image block from the illumination level evaluation index to obtain the difference, and obtain the illumination variation uniformity index based on the difference;

[0022] The brightness evaluation index is obtained based on the illumination level evaluation index and the illumination variation uniformity index.

[0023] The weighted pixel sum of the eight pixels adjacent to each pixel in the palm region is calculated based on preset weighting coefficients. The preset weighting coefficients are: horizontal and vertical adjacent pixels have a weight of 1, and non-horizontal and non-vertical adjacent pixels have a weight of [missing value].

[0024] The sharpness evaluation index is obtained by calculating the average of all weighted pixels.

[0025] In one implementation, the step of extracting skeletal joints in the palm region and evaluating palm posture based on the skeletal joints to obtain a palm center region integrity index, a palm center region resolution evaluation index, a finger extension index, and a palm flatness index includes:

[0026] Extract the skeletal joints in the palm region;

[0027] Based on the number and location of the skeletal joints, a set of real skeletal key points is obtained;

[0028] Based on the inclusion relationship between the actual set of skeletal key points and the preset set of skeletal key points, the integrity index of the central area of ​​the palm is obtained.

[0029] Extract a first set of joints from the skeletal joints and obtain a convex region enclosed by the joints in the first set of joints, wherein the first set of joints includes the wrist root joint, the five finger root joints and the first joint of the thumb.

[0030] The resolution evaluation index of the central area of ​​the palm is obtained based on the ratio between the area of ​​the palm print image and the area of ​​the convex region.

[0031] The inter-joint distance is obtained based on the location of the skeletal joints;

[0032] The inter-joint distance is normalized to obtain the finger extension index;

[0033] Calculate the average depth coordinates of all bone keypoints in the bone keypoint set;

[0034] The variance of the depth coordinates of all skeletal key points in the set of skeletal key points is calculated based on the average value of the depth coordinates of all skeletal key points to obtain the palm flatness index.

[0035] In one implementation, obtaining the palmprint image quality based on the palmprint quality index includes:

[0036] Acquire a recognition scene, wherein the recognition scene includes a scene with a reference image and a scene without a reference image;

[0037] If the recognition scenario is a scenario without a reference image, the palmprint image quality is obtained through a palmprint quality assessment depth model;

[0038] If the recognition scenario is a scenario with a reference image, then the similarity is calculated based on the cosine value of the angle between the palm print quality index of the palm print region and the palm print quality index of the reference image.

[0039] The quality of the palmprint image is obtained based on the similarity.

[0040] In one implementation, before evaluating the palmprint image quality using a palmprint quality assessment depth model, the method further includes:

[0041] Construct a deep network for palmprint quality assessment and obtain a palmprint quality sample database;

[0042] Obtain palm images from the palm print quality sample database that are identical to the palm print image to obtain a first set of palm images;

[0043] Obtain palm images that are different from the palm image in the palm print quality sample database to obtain a second set of palm images;

[0044] Based on the first set of palm images, calculate the similarity of the palm print image with the first palm print quality index of each image in the first set of palm images, and calculate the intra-class distance distribution based on the similarity of the first palm print quality index.

[0045] Based on the second set of palm images, calculate the similarity of the palm print image with the second palm print quality index of each image in the second set of palm images, and calculate the inter-class distance distribution based on the similarity of the second palm print quality index.

[0046] The combined distance is obtained by calculating the distance between the intra-class distance distribution and the inter-class distance distribution of the palmprint image using Wasserstein distance;

[0047] Based on the weighted fusion of the mean of the intra-class distance distribution, the mean of the inter-class distance distribution, and the combined distance, a pseudo-label for the palm image quality is obtained.

[0048] The palmprint quality assessment depth network is trained based on the pseudo-labels of the palm image quality to obtain the palmprint quality assessment depth model.

[0049] Secondly, embodiments of the present invention also provide a non-contact palmprint image quality assessment device, wherein the device includes:

[0050] A palm region acquisition module is used to acquire a palm print image and obtain the palm region based on the palm print image;

[0051] The evaluation index acquisition module is used to evaluate the quality of the palm area and obtain palmprint quality indexes.

[0052] The palmprint image quality acquisition module is used to obtain the palmprint image quality based on the palmprint quality index.

[0053] Thirdly, embodiments of the present invention also provide a smart terminal, wherein the smart terminal includes a memory, a processor, and a non-contact palmprint image quality assessment program stored in the memory and executable on the processor, wherein when the processor executes the non-contact palmprint image quality assessment program, it implements the steps of the non-contact palmprint image quality assessment method as described in any of the above.

[0054] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a non-contact palmprint image quality assessment program, and when the non-contact palmprint image quality assessment program is executed by a processor, it implements the steps of the non-contact palmprint image quality assessment method as described in any of the above claims.

[0055] Beneficial Effects: Compared with existing technologies, this invention provides a non-contact palmprint image quality assessment method. First, a palmprint image is acquired, and the palm region is obtained from the image. By extracting the palm region, interference from abnormal areas such as overly bright or dark background areas can be avoided in subsequent palmprint image quality assessment. Then, the palm region is quality assessed to obtain palmprint quality indicators. By evaluating the quality of the palm at the pixel level, the influence of ambient lighting on the palmprint in non-contact palmprint image processing is addressed. By evaluating the palm posture, the flatness of the palm imaging plane is assessed, thus meeting the needs of non-contact palmprint quality assessment in natural scenes, comprehensively considering palmprint-related quality factors. Finally, based on the palmprint quality indicators, the palmprint image quality is obtained, achieving palmprint image quality assessment. This solves the current problems in palmprint recognition, such as the lack of publicly available palmprint data libraries with quality labels or the need for extensive manual annotation in training deep models for palmprint quality assessment. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of the non-contact palmprint image quality assessment method provided in the embodiments of the present invention.

[0058] Figure 2 This is a palm print image provided in an embodiment of the present invention.

[0059] Figure 3 This is a schematic diagram of the palm print region after semantic segmentation provided in an embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram of the even division of a palm image provided in an embodiment of the present invention.

[0061] Figure 5 This is a schematic diagram of skeletal joints and their numbering provided in an embodiment of the present invention.

[0062] Figure 6 This is an example diagram of skeletal joint point extraction provided in an embodiment of the present invention.

[0063] Figure 7 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0064] Figure 8 This is a schematic diagram of the non-contact palmprint image quality assessment device provided in the embodiments of the present invention.

[0065] Figure 9 This is a block diagram illustrating the internal structure of a smart terminal provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0067] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0068] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0069] Palmprint recognition technology can automatically identify users based on the biometric features of the human palm surface, offering advantages such as fast verification speed and objective accuracy. However, the non-contact, free palm recognition method introduces problems such as palm tilt, unevenness, and distance issues. Furthermore, the open recognition environment presents complex external lighting and background problems, significantly impacting the specific steps of palmprint recognition, such as region of interest (ROI) localization, palmprint feature extraction, feature matching, and recognition. Therefore, image quality assessment is crucial. However, existing image quality assessment methods typically use the mean and variance of image pixel grayscale values, as well as features such as the area and perimeter of local regions. These methods offer insufficient quality features, are easily affected by noise, and fail to meet the robustness requirements of global quality assessment, resulting in poor performance in real-world scenarios.

[0070] This invention addresses the problem that current palmprint image quality assessment relies on local and one-sided information, failing to comprehensively reflect image quality and robustness. It proposes a non-contact palmprint image quality assessment method, device, and smart terminal. First, a deep neural network is used to segment the palm region of the palmprint image. Then, for the obtained palm region, further segmentation is performed, and pixel-level quality assessment and palmprint flatness (reflected by palm geometry) are conducted for each region, resulting in a comprehensive quality assessment of the entire palm. Image-level quality assessment includes brightness and sharpness. The palmprint flatness quality assessment, reflecting palm geometry, first requires obtaining skeletal information of the palm through a deep neural network and extracting key points of the finger joints. Then, the spatial coordinates of these key points are used to calculate and assess the extension and rotation of the palm, as well as the flatness of the palm on the imaging plane. Finally, by fusing the above-mentioned relevant quality assessment indicators, the global quality information of the palmprint is obtained. The palmprint quality values ​​generated by the above scheme can be used as image quality pseudo-labels for training a deep learning network framework based on unsupervised palmprint quality assessment. This solves the problem that there are currently no publicly available palmprint datasets with quality labels for supervised model learning in the field of palmprint recognition, avoids the heavy work of manual annotation, and enhances the generalization ability of the palmprint quality assessment model.

[0071] Exemplary methods

[0072] This embodiment provides a non-contact palmprint image quality assessment method. For example... Figure 1 As shown, the method includes the following steps:

[0073] Step S100: Obtain a palm print image and obtain the palm region based on the palm print image;

[0074] Specifically, palm prints refer to the lines on the palm, and the collected palm print images are as follows: Figure 2As shown. Palm prints consist of thick "lines" and thin "textures." The "lines" are formed naturally in the womb and are not easily changed; they reflect the body's innate condition. After acquiring the palm print image, because the image contains background objects, it is necessary to extract the palm region to avoid interference from abnormal areas such as overly bright or dark background areas on the subsequent palm print region image quality assessment. The extracted palm region is shown below. Figure 3 As shown.

[0075] In one implementation, step S100 of this embodiment includes the following steps:

[0076] S101. Input the palm print image into an image semantic segmentation network for semantic segmentation to obtain the palm region.

[0077] Specifically, in this embodiment, the palm region is extracted from the acquired palmprint image using an image segmentation algorithm. One implementation uses image segmentation algorithms such as HRNet, Grabcut, and UNet. To achieve more accurate palm segmentation, this embodiment can fine-tune the image segmentation algorithm network model. Figure 2 The non-contact palmprint image in the image is obtained after semantic segmentation, as shown below. Figure 3 The sample image of the palm print area shown.

[0078] Step S200: Perform a quality assessment on the palm area to obtain palm print quality indicators;

[0079] Specifically, Image Quality Assessment (IQA) is one of the fundamental techniques in image processing. It primarily involves analyzing the characteristics of an image to evaluate its quality (degree of distortion). IQA plays a crucial role in image processing systems for algorithm analysis and comparison, and system performance evaluation. Based on the existence of an ideal image as a reference, image quality assessment can be categorized into three types: Full-Reference (FR), Reduced-Reference (RR), and No-Reference (NR). In this embodiment, as... Figure 7 As shown, by evaluating the image quality of the palm region, the order of quality of each palm region's image can be determined, and a corresponding evaluation value can be given.

[0080] In one implementation, step S200 of this embodiment includes the following steps:

[0081] Step S201: Perform a pixel-level quality assessment of the palm image in the palm region to obtain a brightness assessment index and a sharpness assessment index.

[0082] In one implementation, step S201 of this embodiment includes the following steps:

[0083] Step S2011: Divide the palm region into several image blocks;

[0084] Step S2012: Obtain the average pixel value of each image block;

[0085] Specifically, the palm region is divided into m×n small image blocks for individual evaluation. This avoids overexposure or underexposure in individual areas directly affecting palmprint recognition performance. Using the original image directly for evaluation would result in the loss of local information, impacting the overall image quality. Considering the balance between computational load and evaluation performance, the granularity of image block division can be determined based on actual application requirements, such as... Figure 4 The image shown is an example of a hand region divided into 4x4 sections. Let the hand region be I(x,y), and the image patch be I... p (x,y), where p∈[1,...,m×n]. Let j be the number of pixel rows and k be the number of columns in the image patch. By segmenting the palm region image into several uniformly sized image patches and evaluating them individually, the ability to focus on local low-quality palmprint regions can be enhanced.

[0086] Step S2013: Calculate the mean of the pixel average of all image blocks to obtain the illumination level evaluation index;

[0087] Step S2014: Subtract the average pixel value of each image block from the illumination level evaluation index to obtain the difference, and obtain the illumination change uniformity index based on the difference;

[0088] Step S2015: Obtain the brightness evaluation index based on the illumination level evaluation index and the illumination change uniformity index;

[0089] Specifically, the average pixel value of the image patch within each region is calculated as follows: Where j is the number of pixel rows in the image patch, k is the number of columns, the palm region is I(x,y), and the image patch is denoted as I. p (x,y), where p∈[1,...,m×n], and m×n is the number of image patches. The overall image illumination level can be evaluated based on the average pixel value of each image patch. and uniformity of light variation index in For each M p The average value. Then, based on the illumination level assessment index and the illumination change uniformity index, we obtain the brightness assessment index that can reflect overexposure and underexposure problems.

[0090] Step S2016: Calculate the weighted pixel sum of the 8 pixels adjacent to each pixel in the palm region according to preset weighting coefficients, wherein the preset weighting coefficients are: the weight of adjacent pixels in the horizontal and vertical directions of each pixel is 1, and the weight of adjacent pixels in the non-horizontal and non-vertical directions is 0.

[0091] Step S2017: Calculate the average value of all weighted pixels to obtain the sharpness evaluation index.

[0092] Specifically, this embodiment uses the DEAV sharpness evaluation index based on point sharpness to evaluate the sharpness of the palm region. The DEAV algorithm has advantages such as monotonicity, sensitivity, content independence, robustness to noise, and low time complexity. The specific steps are as follows: for each pixel I(x,y) in the palm region, calculate the weighted sum NB(x,y) of its 8 neighboring pixels I(i,j). The weights of its neighboring pixels are set as follows: the weights of pixels located in its horizontal and vertical directions are set to 1, and the weights of pixels located in the other four directions are set to... The specific formula for calculating the weighted sum is as follows: Based on this weighting method, the sum of all pixels in the palm region is calculated, and the ratio of this sum to the total number of pixels is obtained. This yields the sharpness evaluation index based on DEAV. It should be noted that, in one implementation, image sharpness assessment algorithms such as Brenner, DCT, and Laplacian can also be used.

[0093] Step S202: Extract the skeletal joints in the palm region and evaluate the palm posture based on the skeletal joints to obtain the palm center region integrity index, palm center region resolution evaluation index, finger extension index, and palm flatness index.

[0094] In one implementation, step S202 of this embodiment includes the following steps:

[0095] Step S2021: Extract the skeletal joints in the palm region;

[0096] Step S2022: Obtain the set of real skeletal key points based on the number and position of the skeletal joints;

[0097] Specifically, the hand joints include the wrist joint, mid-carpal joint, carpometacarpal joint, metacarpophalangeal joint, and interphalangeal joints. For image recognition of the palm region, evaluation metrics can be obtained through the coordinates and numbers of the joint points. In this embodiment, based on the MMPose pose estimation algorithm library of OpenMMLab, 21 skeletal joints in the palm region are detected as key points, such as... Figure 5The image shows an example of 21 skeletal joints of the hand, along with their numbering. Let the coordinates of each keypoint be P. i (x i ,y i ,z i ), where i∈{0,1,...,20}. These 21 key points are: 1 point at the wrist root, 4 points related to the thumb (thumb root, first joint, second joint, fingertip), 4 points related to the index finger (index finger root, first joint, second joint, fingertip), 4 points related to the middle finger (middle finger root, first joint, second joint, fingertip), 4 points related to the ring finger (ring finger root, first joint, second joint, fingertip), and 4 points related to the little finger (little finger root, first joint, second joint, fingertip). The bone joint point extraction effect is as follows: Figure 6 As shown.

[0098] It should be noted that, in one implementation, deep learning frameworks such as HuggingFace and MediaPipe can also be used for keypoint extraction. These frameworks all have multi-scenario functions such as scene segmentation, object detection, face recognition, and human pose estimation, and can be applied to the detection of keypoints in the hand skeleton.

[0099] Step S2023: Based on the inclusion relationship between the actual set of skeletal key points and the preset set of skeletal key points, obtain the integrity index of the central area of ​​the palm.

[0100] Specifically, this embodiment determines whether the central area of ​​the palm is within the imaging range by checking for missing key points on the palm. Let P be the set of extracted skeletal key points. palm Because existing palmprint ROI extraction techniques generally require the presence of adjacent finger bone points on the palm for locating the ROI (Region of Interest), a necessary set of real skeletal keypoints P is required. key It must include: 1 point at the base of the wrist, 3 points at the base of the thumb and its first and second joints, 2 points at the base of the index finger and its first joint, 2 points at the base of the middle finger and its first joint, 2 points at the base of the ring finger and its first joint, and 2 points at the base of the little finger and its first joint. Based on the inclusion relationship between the actual set of skeletal key points and the preset set of skeletal key points, the completeness markings of the central area of ​​the palm are obtained as follows:

[0101]

[0102] Step S2024: Extract the first set of joints from the bone joints and obtain the convex area enclosed by the joints in the first set of joints, wherein the first set of joints includes the wrist root joint, the five finger root joints and the first joint of the thumb.

[0103] Step S2025: Obtain the resolution evaluation index of the palm center area based on the ratio between the area of ​​the palm print image and the area of ​​the convex region;

[0104] Specifically, this embodiment evaluates whether the resolution of the palm center area meets the recognition criteria by measuring the size of the palmprint ROI region extracted from several palm key points, i.e., obtaining the palm center area resolution evaluation metric. Let S be the area of ​​the palmprint image. a Let S be the set of first joint points formed by one point at the base of the wrist, two points at the base of the thumb and its first joint, one point at the base of the index finger, one point at the base of the middle finger, one point at the base of the ring finger, and one point at the base of the little finger. The area of ​​the convex region enclosed by these joint points is S. c Since existing palmprint recognition algorithms generally require a resolution of at least 128*128 pixels for the extracted palmprint ROI region, the evaluation index for the resolution of the palm center area is as follows:

[0105]

[0106] Step S2026: Obtain the inter-joint distance based on the position of the bone joints;

[0107] Step S2027: Normalize the inter-joint distance to obtain the finger extension index;

[0108] Specifically, the smoothness of the palm print texture in the ROI area of ​​the palm is closely related to the degree of extension of each finger. Insufficient finger extension will result in skin wrinkles in the palm print texture, affecting the recognition accuracy. For the thumb, the distance from the base of the index finger (number 5) to the base of the middle finger (number 9, and so on) is recorded as l. 59 The distance from the first joint of the thumb to the base of the index finger is l 25 The distance from the second joint of the thumb to the base of the index finger is l 35 Based on the characteristics of a naturally extended palm, l 25 / l 59 and l 35 / l 59 The finger extension relative to the palm can be measured, hence the corresponding normalized finger extension index is called IQA. tb1 and IQA tb2 The degree of extension of the index, middle, ring, and little fingers also affects the smoothness of the palm print ROI area. Based on the characteristics of palm extension, the distance between the base points of adjacent fingers from the index to the middle finger should be less than the distance between the corresponding first joint points. Accordingly, the relevant normalized finger extension index is denoted as IQA. if Similarly, the finger extension index from the middle finger to the ring finger is denoted as IQA. mf The finger extension from the ring finger to the little finger is marked as IQA.rf .

[0109] Step S2028: Calculate the average depth coordinates of all bone key points in the bone key point set;

[0110] Step S2029: Based on the average depth coordinates of all the skeletal key points, calculate the variance of the depth coordinates of all the skeletal key points in the set of skeletal key points to obtain the palm flatness index.

[0111] Specifically, actions such as bending the fingers towards the palm can affect the flatness of the palm. This embodiment uses geometric constraints on key points of the thumb, index finger, middle finger, ring finger, and little finger to determine the degree of extension of each finger, thereby assessing whether the palm print is flat, i.e., obtaining the palm flatness index. For the necessary set of skeletal key points P... key The depth coordinates of each key point can be used to assess the flatness of the palm surface. Let P be... j (x j ,y j ,z j )∈P key , where j∈{0,1,...,11}. The corresponding depth coordinate variance can be obtained as: in Let IQA be the mean of the depth coordinates for each key point. The normalized palm flatness index is denoted as IQA. 3d .

[0112] Step S203: Based on the brightness evaluation index, sharpness evaluation index, palm center area integrity index, palm center area resolution evaluation index, finger extension index, and palm flatness index, the palm print quality index is obtained.

[0113] Specifically, different dimensions of indicators for evaluating palmprint image quality were obtained, namely: the brightness assessment index IQA. lt IQA (Index to the uniformity of illumination variation) sd Image sharpness index (IQA) deav Palm Integrity Index (IQA) pset Palm center area resolution assessment index (IQA) area Finger extension index (IQA) tb1 IQA tb2 IQA if IQA mf and IQA rf Palm flatness index (IQA) 3d Let the vector characteristics of the comprehensive evaluation index, i.e., the palmprint quality index, be:

[0114] F = (IQA) ltIQA sd IQA deav IQA pset IQA area IQA tb1 IQA tb2 IQA if IQA mf IQA rf IQA 3d ).

[0115] Step S300: Obtain the palmprint image quality based on the palmprint quality index.

[0116] In one implementation, step S300 of this embodiment includes the following steps:

[0117] Step S301: Obtain the recognition scene, wherein the recognition scene includes a scene with a reference image and a scene without a reference image;

[0118] Step S302: If the recognition scenario is a scenario without a reference image, the palmprint image quality is obtained through the palmprint quality assessment depth model;

[0119] Step S303: If the recognition scenario is a scenario with a reference image, then calculate the similarity based on the cosine value of the angle between the palm print quality index of the palm print region and the palm print quality index of the reference image.

[0120] Step S304: Obtain the palmprint image quality based on the similarity.

[0121] Specifically, this embodiment addresses the application scenario of quality assessment with reference images. It can employ an evaluation index such as the cosine of the included angle, using the quality feature vector of the optimal image as a reference to obtain the quality assessment value of the sample image. For quality assessment scenarios with reference images, in palmprint verification (palmprint 1:1, also known as comparison), when a registered palmprint image is available as the optimal reference image quality, the similarity between the sample to be compared and the registered image can be directly calculated using palmprint quality calculation steps. The degree of similarity can be represented by the cosine of the included angle.

[0122]

[0123] F here a and F b These represent the palmprint quality index of the palmprint region and the palmprint quality index of the reference image, respectively. After normalization, we can obtain the overall quality value V of the palmprint image.

[0124] In one implementation, step S302 of this embodiment includes the following steps:

[0125] Step S3021: Construct a palmprint quality assessment deep network and obtain a palmprint quality sample database;

[0126] Step S3022: Obtain palm images in the palm print quality sample database that are identical to the palm print image, to obtain a first set of palm images;

[0127] Step S3023: Obtain palm images that are different from the palm image in the palm print quality sample database to obtain a second set of palm images;

[0128] Step S3024: Based on the first set of palm images, calculate the similarity of the first palm print quality index between the palm print image and each image in the first set of palm images, and calculate the intra-class distance distribution based on the similarity of the first palm print quality index.

[0129] Step S3025: Based on the second palm image set, calculate the similarity of the palm print image with the second palm print quality index of each image in the second palm image set, and calculate the inter-class distance distribution based on the similarity of the second palm print quality index.

[0130] Step S3026: Calculate the distance between the intra-class distance distribution and the inter-class distance distribution of the palmprint image using Wasserstein distance to obtain the combined distance;

[0131] Step S3027: Based on the weighted fusion of the mean of the intra-class distance distribution, the mean of the inter-class distance distribution, and the combined distance, a pseudo-label for the palm image quality is obtained;

[0132] Step S3028: Train the palmprint quality assessment depth network based on the palm image quality pseudo-labels to obtain the palmprint quality assessment depth model.

[0133] Specifically, this embodiment is for the application scenario of no reference image quality assessment, namely palmprint recognition (palmprint 1:N) scenario. This embodiment adopts the relative image quality label assessment method and the unsupervised training strategy.

[0134] For this palm image, the quality index of the extracted palm prints from identical palms is calculated, and the intra-class distance distribution and mean V are calculated using the cosine similarity of the included angle. inmer .

[0135] This hand image is used to calculate the quality index of extracted palm prints from different hands. The inter-class distance distribution and mean V are calculated using the cosine similarity of the included angle. inter .

[0136] The distance V between the intra-class and inter-class distance distributions of the palm region is calculated using Wasserstein distance. wdSpecifically, for palmprint images, their image quality distribution should be similar to that of within-class samples and far from that of between-class samples. Correspondingly, the Wasserstein distance can be used to measure quality similarity, and the relevant formula is:

[0137] Among them, Π(P) a ,P b Let I be the set of all possible joint distributions combining the within-class and between-class quality distributions of image I. In one implementation, KL divergence or JS divergence can also be used as the distribution measure.

[0138] Based on the mean V of the intraclass distance distribution inner Mean V of inter-class distance distribution inter Wasserstein distance V inter The weights of the three are combined to obtain the pseudo-label V of the palm image quality;

[0139] The palmprint quality assessment depth network is trained using the supervised training pseudo-labels. After obtaining the palmprint quality assessment depth model, the palmprint region can be input into the model to evaluate the palmprint image quality. This scheme solves the problem that there is currently no publicly available palmprint dataset with quality labels for supervised model learning in the field of palmprint recognition, avoiding the arduous work of manual annotation and enhancing the generalization ability of the palmprint quality assessment model.

[0140] Exemplary device

[0141] like Figure 8 As shown in the illustration, this embodiment also provides a non-contact palmprint image quality assessment device, the device comprising:

[0142] The palm region acquisition module 10 is used to acquire a palm print image and obtain the palm region based on the palm print image;

[0143] The evaluation index acquisition module 20 is used to evaluate the quality of the palm area and obtain palm print quality index.

[0144] The palmprint image quality acquisition module 30 is used to obtain the palmprint image quality based on the palmprint quality index.

[0145] In one implementation, the palm region acquisition module 10 includes:

[0146] The semantic segmentation unit is used to input the palm print image into the image semantic segmentation network for semantic segmentation to obtain the palm region.

[0147] In one implementation, the evaluation index acquisition module 20 includes:

[0148] A pixel-level quality assessment unit is used to perform a pixel-level quality assessment of the palm region to obtain a brightness assessment index and a sharpness assessment index.

[0149] The palm posture assessment unit is used to extract the skeletal joints in the palm region and perform palm posture assessment based on the skeletal joints to obtain the palm central area integrity index, palm central area resolution assessment index, finger extension index, and palm flatness index.

[0150] The palmprint quality index acquisition unit is used to obtain the palmprint quality index based on the brightness evaluation index, clarity evaluation index, palm center area integrity index, palm center area resolution evaluation index, finger extension index, and palm flatness index.

[0151] In one implementation, the pixel-level quality assessment unit includes:

[0152] An image segmentation subunit is used to divide the palm region into several image blocks;

[0153] The average value calculation unit is used to obtain the average pixel value of each image block;

[0154] The illumination level assessment index acquisition subunit is used to calculate the mean of the pixel average of all image blocks to obtain the illumination level assessment index.

[0155] The illumination variation uniformity index acquisition subunit is used to subtract the average pixel value of each image block from the illumination level evaluation index to obtain the difference, and obtain the illumination variation uniformity index based on the difference.

[0156] The brightness evaluation index acquisition subunit is used to obtain the brightness evaluation index based on the illumination level evaluation index and the illumination change uniformity index.

[0157] The weighted sum calculation subunit is used to calculate the weighted pixel sum of the eight pixels adjacent to each pixel in the palm region according to preset weight coefficients, wherein the preset weight coefficients are 1 for adjacent pixels in the horizontal and vertical directions of each pixel, and 0 for adjacent pixels in the non-horizontal and non-vertical directions.

[0158] The sharpness evaluation index acquisition sub-unit is used to calculate the average value of all weighted pixels to obtain the sharpness evaluation index.

[0159] In one implementation, the hand posture evaluation unit includes:

[0160] A skeletal joint extraction subunit is used to extract skeletal joints in the palm region;

[0161] The real skeleton key point set acquisition subunit is used to obtain the real skeleton key point set based on the number and position of the skeleton joints;

[0162] The integrity index acquisition subunit is used to obtain the integrity index of the palm center area based on the inclusion relationship between the real skeletal key point set and the preset skeletal key point set.

[0163] A convex region acquisition subunit is used to extract a first set of joints from the skeletal joints and acquire a convex region enclosed by the joints in the first set of joints, wherein the first set of joints includes the wrist root joint, the five finger root joints and the first joint of the thumb.

[0164] The resolution evaluation index acquisition subunit is used to obtain the resolution evaluation index of the palm center area based on the ratio between the area of ​​the palm print image and the area of ​​the convex region.

[0165] The inter-joint distance acquisition subunit is used to obtain the inter-joint distance based on the position of the skeletal joint points;

[0166] The extension index acquisition subunit is used to normalize the inter-joint distance to obtain the finger extension index.

[0167] The depth coordinate average calculation subunit is used to calculate the average depth coordinates of all bone key points in the bone key point set;

[0168] The flatness index acquisition subunit is used to calculate the variance of all bone key points in the bone key point set based on the average depth coordinates of all bone key points, and obtain the palm flatness index.

[0169] In one implementation, the palmprint image quality acquisition module 30 includes:

[0170] A scene acquisition unit is used to acquire a scene to be recognized, wherein the scene to be recognized includes a scene with a reference image and a scene without a reference image;

[0171] The first palmprint image quality assessment unit is used to obtain the palmprint image quality through a palmprint quality assessment depth model if the recognition scenario is a scenario without a reference image.

[0172] The similarity calculation unit is used to calculate the similarity based on the cosine value of the angle between the palm print quality index of the palm print region and the palm print quality index of the reference image if the recognition scenario is a scenario with a reference image.

[0173] The second palmprint image quality assessment unit is used to obtain the palmprint image quality based on the similarity.

[0174] In one implementation, the palmprint image quality acquisition module 30 further includes:

[0175] The network building unit is used to construct a deep network for palmprint quality assessment and obtain a palmprint quality sample database.

[0176] The first palm image acquisition unit is used to acquire palm images that are the same as the palm image in the palm print quality sample database, and obtain a first palm image set;

[0177] The second palm image acquisition unit is used to acquire palm images that are different from the palm image in the palm print quality sample database, and obtain a second palm image set.

[0178] The intra-class distance distribution calculation unit is used to calculate the similarity of the palm print image with the first palm print quality index of each image in the first palm print image set based on the first palm print image set, and to calculate the intra-class distance distribution based on the similarity of the first palm print quality index.

[0179] The inter-class distance distribution calculation unit is used to calculate the similarity of the second palm print quality index between the palm print image and each image in the second palm print image set based on the second palm print image set, and to calculate the inter-class distance distribution based on the similarity of the second palm print quality index.

[0180] The combined distance calculation unit is used to calculate the distance between the intra-class distance distribution and the inter-class distance distribution of the palmprint image using Wasserstein distance, and obtain the combined distance.

[0181] The palm image quality pseudo-label acquisition unit is used to obtain palm image quality pseudo-labels by fusing the weights of the mean of the intra-class distance distribution, the mean of the inter-class distance distribution, and the combined distance.

[0182] The model training unit is used to train the palmprint quality assessment depth network based on the palm image quality pseudo-labels to obtain the palmprint quality assessment depth model.

[0183] Based on the above embodiments, the present invention also provides a smart terminal, the principle block diagram of which can be as follows: Figure 9As shown, the smart terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a non-contact palmprint image quality assessment method. The display screen of the smart terminal can be a liquid crystal display (LCD) or an e-ink display.

[0184] Those skilled in the art will understand that Figure 9 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the smart terminal to which the present invention is applied. A specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0185] In one embodiment, a smart terminal is provided, comprising a memory, a processor, and a non-contact palmprint image quality assessment program stored in the memory and executable on the processor. When the processor executes the non-contact palmprint image quality assessment program, it implements the following operation instructions:

[0186] Acquire a palmprint image and determine the palm region based on the palmprint image;

[0187] The palm area is subjected to quality assessment to obtain palm print quality indicators;

[0188] The palmprint image quality is obtained based on the palmprint quality index.

[0189] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0190] In summary, this invention discloses a non-contact palmprint image quality assessment method, device, and smart terminal. The method includes: acquiring a palmprint image and obtaining a palm region based on the palmprint image; performing quality assessment on the palm region to obtain a palmprint quality index; and obtaining the palmprint image quality based on the palmprint quality index. The method of this invention can handle the quality assessment of the influence of open ambient lighting and free palm posture on palmprint imaging effects in non-contact palmprint image processing, meeting the needs of non-contact palmprint quality assessment in natural scenes. It comprehensively considers palmprint-related quality factors and has high robustness.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-contact palmprint image quality assessment method, characterized in that, The method comprises: acquiring a palmprint image and obtaining a palm region according to the palmprint image; performing quality assessment on the palm region to obtain a palmprint quality index, wherein the palmprint quality index is used to reflect brightness, definition and palmprint flatness of the palm region; obtaining palmprint image quality according to the palmprint quality index; the quality assessment on the palm region to obtain the palmprint quality index comprises: performing image pixel level quality assessment on the palm region to obtain brightness assessment index and definition assessment index; extracting skeletal joint points in the palm region and performing palm posture assessment according to the skeletal joint points to obtain palm center area integrity index, palm center area resolution assessment index, finger stretch degree index and palm flatness index; obtaining the palmprint quality index according to the brightness assessment index, definition assessment index, palm center area integrity index, palm center area resolution assessment index, finger stretch degree index and palm flatness index; the extraction of the skeletal joint points in the palm region and the palm posture assessment according to the skeletal joint points to obtain the palm center area integrity index, the palm center area resolution assessment index, the finger stretch degree index and the palm flatness index comprises: extracting the skeletal joint points in the palm region; obtaining a real skeletal key point set according to the number and position of the skeletal joint points; obtaining the palm center area integrity index according to the real skeletal key point set and the containing relationship of a preset skeletal key point set; extracting a first joint point set from the skeletal joint points and acquiring a convex region surrounded by joint points in the first joint point set, wherein the first joint point set comprises a wrist root joint point, five finger root joint points and a first joint of the thumb; obtaining the palm center area resolution assessment index according to the ratio between the area of the palmprint image and the area of the convex region; obtaining the inter-joint distance according to the position of the skeletal joint points; normalizing the inter-joint distance to obtain the finger stretch degree index; calculating the average value of the depth coordinates of all skeletal key points in the skeletal key point set; calculating the palm flatness index by calculating the variance of the depth coordinates of all skeletal key points in the skeletal key point set according to the average value of the depth coordinates of all skeletal key points; the obtaining of the palmprint image quality according to the palmprint quality index comprises: acquiring an identification scene, wherein the identification scene comprises a reference image scene and a non-reference image scene; if the identification scene is a non-reference image scene, obtaining the palmprint image quality through a palmprint quality assessment depth model; if the identification scene is a reference image scene, calculating the similarity according to the included angle cosine value between the palmprint quality index of the palm region and the palmprint quality index of the reference image; obtaining the palmprint image quality according to the similarity.

2. The non-contact palmprint image quality assessment method according to claim 1, wherein the acquisition of the palmprint image and the obtaining of the palm region according to the palmprint image comprise: inputting the palmprint image into an image semantic segmentation network for semantic segmentation to obtain the palm region. 3.The non-contact palmprint image quality assessment method of claim 1, wherein, The image pixel level quality evaluation on the palm region is performed to obtain a brightness evaluation index and a definition evaluation index, and the method comprises the following steps: The palm region is divided into a plurality of image blocks; The average value of pixels of each image block is obtained; The average value of the average values of pixels of all image blocks is calculated to obtain an illumination degree evaluation index; The illumination degree evaluation index is subtracted from the average value of pixels of each image block to obtain a difference value, and an illumination change uniformity index is obtained according to the difference value; The brightness evaluation index is obtained according to the illumination degree evaluation index and the illumination change uniformity index; The weighted pixel sum of the pixel points adjacent to each pixel point in the palm region is calculated according to a preset weight coefficient; The average value of all weighted pixel sums is calculated to obtain the definition evaluation index. 4.The non-contact palmprint image quality assessment method of claim 1, wherein, Before the palmprint image quality is obtained through the palmprint quality evaluation deep model, the method further comprises the following steps: A palmprint quality evaluation deep network is constructed, and a palmprint quality sample database is obtained; A palm image identical to the palm of the palmprint image is obtained from the palmprint quality sample database to obtain a first palm image set; A palm image different from the palm of the palmprint image is obtained from the palmprint quality sample database to obtain a second palm image set; According to the first palm image set, a first palmprint quality index similarity between the palmprint image and each image in the first palm image set is calculated, and an intra-class distance distribution is calculated according to the first palmprint quality index similarity; According to the second palm image set, a second palmprint quality index similarity between the palmprint image and each image in the second palm image set is calculated, and an inter-class distance distribution is calculated according to the second palmprint quality index similarity; The distance between the intra-class distance distribution and the inter-class distance distribution of the palmprint image is calculated through the Wasserstein distance to obtain a combined distance; A palm image quality pseudo label is obtained through weight fusion of the mean value of the intra-class distance distribution, the mean value of the inter-class distance distribution, and the combined distance; The palmprint quality evaluation deep network is trained according to the palm image quality pseudo label to obtain the palmprint quality evaluation deep model.

5. A non-contact palmprint image quality assessment device, characterized by, The device comprises: A palm region acquisition module is configured to acquire a palmprint image and obtain a palm region according to the palmprint image; An evaluation index acquisition module is configured to perform quality evaluation on the palm region to obtain a palmprint quality index; The quality evaluation on the palm region to obtain the palmprint quality index comprises the following steps: An image pixel level quality evaluation is performed on the palm region to obtain a brightness evaluation index and a definition evaluation index; Skeletal joint points in the palm region are extracted, and palm posture evaluation is performed according to the skeletal joint points to obtain a palm center area integrity index, a palm center area resolution evaluation index, a finger stretch degree index, and a palm flatness index; The palmprint quality index is obtained according to the brightness evaluation index, the definition evaluation index, the palm center area integrity index, the palm center area resolution evaluation index, the finger stretch degree index, and the palm flatness index. The palm region is extracted, and a palm posture evaluation is performed according to the skeleton joint points to obtain a palm center area integrity index, a palm center area resolution evaluation index, a finger stretch degree index, and a palm flatness index, including: extracting the skeleton joint points in the palm region; obtaining a real skeleton key point set according to the number and position of the skeleton joint points; obtaining the palm center area integrity index according to the real skeleton key point set and a preset inclusion relationship of the skeleton key point set; extracting a first joint point set from the skeleton joint points, and obtaining a convex region surrounded by joint points in the first joint point set, wherein the first joint point set includes a wrist root joint point, five finger root joint points, and a first joint of the thumb; obtaining the palm center area resolution evaluation index according to a ratio between the area of the palm print image and the area of the convex region; obtaining an inter-joint distance according to the position of the skeleton joint points; normalizing the inter-joint distance to obtain the finger stretch degree index; calculating an average value of depth coordinates of all skeleton key points in the skeleton key point set; calculating a variance of depth coordinates of all skeleton key points in the skeleton key point set according to the average value of depth coordinates of all skeleton key points to obtain the palm flatness index; a palm print image quality acquisition module configured to obtain a palm print image quality according to the palm print quality index; the palm print image quality is obtained according to the palm print quality index, including: obtaining an identification scene, wherein the identification scene includes a reference image scene and a non-reference image scene; if the identification scene is a non-reference image scene, the palm print image quality is obtained through a palm print quality evaluation depth model; if the identification scene is a reference image scene, a similarity is calculated according to a cosine value of an included angle between the palm print quality index of the palm region and a palm print quality index of a reference image; the palm print image quality is obtained according to the similarity.

6. A smart terminal, characterized by The intelligent terminal includes a memory, a processor, and a non-contact palm print image quality evaluation program stored in the memory and executable on the processor. When the processor executes the non-contact palm print image quality evaluation program, the steps of the non-contact palm print image quality evaluation method according to any one of claims 1-4 are implemented.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a non-contact palm print image quality evaluation program. When the non-contact palm print image quality evaluation program is executed by the processor, the steps of the non-contact palm print image quality evaluation method according to any one of claims 1-4 are implemented.

Images