Non-contact palm vein image quality evaluation method, authentication method, program product

Abstract

The present application belongs to palm vein image quality evaluation method, in order to solve the technical problems that the hardware and software of non-contact authentication system in the existing palm vein biometric authentication system are relatively complex, and the existing authentication method does not consider ROI quality evaluation, provide palm vein image quality evaluation method, biometric authentication method, program product, adopt non-contact hardware device to obtain palm vein image, respectively using palm posture judgment model and palm ROI region target detection model, judge whether the collected palm vein image meets the requirements, and obtain palm vein image, then through ROI image quality evaluation model, the quality of the obtained ROI region is evaluated, the quality of palm vein image can be quickly and conveniently judged, the quality of the intercepted ROI image is evaluated before authentication, thereby improving the reliability of authentication.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of palm vein image quality evaluation method, and particularly relates to a non-contact palm vein image quality evaluation method, a biological authentication method and a program product. BACKGROUND

[0002] Biometric authentication technology refers to a method of identifying personal identity through physiological characteristics (such as fingerprints, irises, faces, voiceprints, etc.) and behavioral characteristics (such as gait, signature, etc.) inherent to the human body, and is the most convenient and secure identification technology at present, which has important application value in many fields such as economy, security and people's livelihood. Among them, palm vein recognition is a biometric authentication technology that authenticates through the image features of the palm vein blood vessels of a person. It has been confirmed in medicine that the palm vein of each person is unique. Compared with other commonly used biometric authentication technologies (such as fingerprints, faces, etc.), palm vein recognition has the advantages of high security, strong privacy, inherent living body detection and active identification, etc.

[0003] The principle of palm vein biometric authentication is to collect palm vein near-infrared images and automatically extract palm image features using computer image processing and artificial intelligence algorithms as identity authentication marks, and match and authenticate with a pre-established registration database. Hemoglobin in blood vessels lacking oxygen can absorb near-infrared light. When near-infrared light is irradiated to the palm, a palm vein image can be obtained by photographing through a camera with near-infrared sensitivity. As shown in Figure 1 The existing palm vein biometric authentication system often includes the following parts: palm vein image acquisition hardware device, region of interest (ROI) extraction unit, ROI region feature calculation unit and palm vein authentication unit, and feature matching with the authentication database. After obtaining the palm vein image, the effective area rich in vein patterns in the palm is extracted for calculating image features for authentication. This area is the ROI area (generally a rectangular area in the palm center), as shown in Figure 2 The rectangular area shown in

[0004] For the problem of palm vein biometric authentication, the research of the academic and industrial circles mainly focuses on image acquisition (hardware design), ROI extraction and ROI region feature calculation and matching. There are still the following deficiencies:

[0005] (1)Most of the palm vein authentication systems use contact authentication, and the existing non-contact authentication methods are relatively complex. In the contact authentication system, in order to fix the palm position, additional hardware design is often needed, making the whole system large and complex. For example, in the degree thesis "Research and Implementation of Palm Vein Recognition System", Zhang Wen-hui designed and implemented a typical contact palm vein authentication system. In order to minimize the user's palm shift and ensure the consistency of the images collected from the same palm, a finger-shaped recess was designed on the edge of the shell of the collection device. The recess was designed with a circular arc to prevent injury to the palm. In addition, a light shield was designed on the top of the shell to prevent interference from the background above during imaging and to reduce the influence of the external environment on the image. The above design can make the image background single and improve the image quality. However, such design not only increases the system cost, but also greatly limits the application scenarios of the system. Liang X et al. in "A novel multicamera system for high-speed touchless palm recognition" proposed a four-lens non-contact palm vein authentication prototype system based on RGB-D depth camera and infrared camera, which is a multi-camera-based non-contact authentication system. The system uses an RGB-D depth camera, a grayscale optical camera, and an infrared camera. The multi-camera system obtains multi-source image data, which must consider the problems of multi-image alignment and data fusion, and the algorithm design is relatively complex. Therefore, the hardware and software are relatively complex, which increases the system cost and identity authentication response time, reduces the system reliability, and limits its application.

[0006] (2)The existing methods propose ROI acquisition methods, but do not consider ROI quality evaluation methods. However, ROI quality evaluation is also a guarantee for high-precision authentication. In the palm vein authentication system, after obtaining the ROI, its quality should be evaluated, that is, to judge whether the imaging quality of the obtained ROI image can meet the needs of subsequent authentication. Zhang Wen-hui compared the performance of "self-correcting ROI extraction method" and "region fixed value method" in the degree thesis "Research and Implementation of Palm Vein Recognition System", and explained that the proposed "self-correcting ROI extraction method" is better through the SSIM evaluation index. However, he did not evaluate the quality of the ROI region extracted by the "self-correcting ROI extraction method". Liang X et al. in "A novel multicamera system for high-speed touchless palm recognition" did not discuss this problem at all. SUMMARY

[0007] The application provides a non-contact palm vein image quality evaluation method, a biological authentication method and a program product to solve the technical problems that the hardware and software of a non-contact authentication system are relatively complex in a palm vein biological authentication system, and that an existing authentication method does not consider ROI quality evaluation.

[0008] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0009] The palm vein image quality evaluation method has the following steps:

[0010] S1, inputting a palm vein image acquired through a non-contact hardware device into a palm posture judgment model, if the palm posture is a standard palm posture, executing step S2, otherwise, reacquiring the palm vein image until the palm posture is the standard palm posture;

[0011] The palm posture judgment model is obtained by training a classifier model with palm vein images under multiple palm postures of multiple people, and the standard posture is that the palm directly faces the non-contact palm vein image acquisition device, and the distance between the palm and the palm vein image acquisition device is a preset distance.

[0012] S2, acquiring an ROI region through a palm ROI region target detection model;

[0013] The palm ROI region target detection model is obtained by labeling the ROI region of the palm vein image acquired under the standard palm posture, obtaining a palm vein image-ROI region data set, and training a target detection Faster-RCNN deep neural network through the palm vein image-ROI region data set.

[0014] S3, judging the ROI region acquired in step S2 through an ROI image quality evaluation model, if the ROI region meets a preset requirement, judging that the palm vein image meets the requirement, otherwise, reacquiring the palm vein image;

[0015] The ROI image quality evaluation model is obtained by calculating the visual features of each ROI region in the palm vein image-ROI region data set in step S2, scoring the quality of each ROI region, obtaining an ROI region-visual feature-quality score data set, and training a support vector machine classification model through the ROI region-visual feature-quality score data set.

[0016] Further, the palm posture judgment model in step S1 is obtained by the following method:

[0017] S1-1, using ResNet-18 deep network and DenseNet-121 deep network respectively to extract the palm vein image depth features of the palm vein images of multiple people in multiple palm postures, corresponding to obtain 1024-dimensional DenseNet-121 features and 512-dimensional ResNet-18 features;

[0018] S1-2, fusing the DenseNet-121 features and the ResNet-18 features to obtain fused depth features;

[0019] S1-3, training a linear kernel SVM classifier based on the fused depth features obtained in step S1-2 to obtain a palm posture judgment model.

[0020] Further, in step S1-3, the evaluation index when training the linear kernel SVM classifier is the equal error rate.

[0021] Further, in step S1, the multiple palm postures include standard palm postures of left hand palm and right hand palm, palm horizontal lifting posture, palm wrist lifting posture, palm finger lifting posture, palm overall forward posture and palm overall backward posture.

[0022] Further, in step S3, the visual features include sharpness, information entropy, contrast and equivalent visual number.

[0023] Further, the sharpness is obtained by the following method:

[0024] The Canny operator is used to extract the vein edge of the ROI region to obtain the edge image of the ROI region, the number of edge image pixels E is counted, and the sharpness C is obtained by the following formula:

[0025]

[0026] Wherein, S is the size of the ROI region;

[0027] The information entropy H is obtained by the following formula:

[0028]

[0029] Wherein, p m is the probability value of the pixel with pixel gray value m;

[0030] The contrast B is obtained by the following formula:

[0031]

[0032] Wherein, x i is the gray value of the i-th pixel, xmean M is the average value of the pixel gray value, and M is the total number of pixel points in the ROI region;

[0033] The equivalent view number D is obtained by the following formula:

[0034]

[0035] Wherein, P std is the gray standard deviation of the ROI region, P avg is the average value of the ROI region gray.

[0036] Further, in step S2, the backbone network of the target detection Faster-RCNN deep neural network uses a Resnet-50 architecture;

[0037] The convolution layer parameters in the Resnet-50 architecture are: the convolution kernel size is 3, the padding width is 1, and the step is 1;

[0038] The pooling layer parameters in the Resnet-50 architecture are: the convolution kernel size is 2, the padding width is 0, and the step is 2;

[0039] The epoch of the target detection Faster-RCNN deep neural network during training is 100, and the minimum learning rate and the maximum learning rate are 1e -4 and 1e -6 , respectively, the confidence is 0.5, the non-maximum suppression value is 0.3, and the prior box size is [8, 16, 32].

[0040] Based on the above palm vein image quality evaluation method, the application further provides a palm vein image biometric authentication method, which is characterized by comprising the following steps:

[0041] S1, obtaining the palm vein image meeting the requirements through the above palm vein image quality evaluation method;

[0042] S2, performing feature calculation on the ROI region of the palm vein image meeting the requirements, and combining a pre-set palm vein feature registration database to complete palm vein image biometric authentication.

[0043] In addition, the application further provides a computer program product, comprising a computer program, which is characterized by the steps of the above palm vein image quality evaluation method or palm vein image biometric authentication method when the program is executed by a processor.

[0044] Compared with the prior art, the application has the following beneficial effects:

[0045] 1.The palm vein image quality evaluation method is proposed, a non-contact hardware device is used to obtain the palm vein image, a palm posture judgment model and a palm ROI region target detection model are used to judge whether the collected palm vein image meets the requirements, and the palm vein image is obtained, and then the ROI image quality evaluation model is used to evaluate the quality of the obtained ROI region, so that the palm vein image quality can be quickly and conveniently evaluated. Compared with the existing palm vein biometric authentication system, the palm vein image quality is evaluated, and the method is more flexible, the cost is lower, it is more sanitary and safe, and it is also convenient to realize through human-computer interaction.

[0046] 2.The evaluation method of the present application judges the palm posture, extracts the ROI region, and evaluates the quality of the RIO region based on the machine learning framework, which can accurately judge the correct palm posture of the user and improve the interactivity and reliability of the subsequent authentication.

[0047] 3.The ROI region extraction method in the present application has good robustness, that is, even if the palm vein image contains the palm surrounding environment image, the ROI region can be accurately extracted.

[0048] 4.Based on the above-mentioned palm vein image quality evaluation method, the present application also proposes a palm vein image biometric authentication method, which considers the ROI quality evaluation, ROI image visual feature calculation and support vector machine ROI image quality evaluation method compared with the existing authentication method, which can evaluate the quality of the intercepted ROI image before authentication, thereby improving the reliability of the authentication.

[0049] 5.The present application also provides two kinds of computer program products, which are based on the above-mentioned palm vein image quality evaluation method and palm vein biometric authentication method respectively, and can popularize the application of the method of the present application, and realize the corresponding quality judgment and biometric authentication on the corresponding hardware device. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 It is a schematic diagram of the existing palm vein biometric authentication system;

[0051] Figure 2 It is an ROI region example diagram of the palm palm;

[0052] Figure 3 It is a flowchart of the palm vein image biometric authentication method embodiment of the present application;

[0053] Figure 4 It is a user's left palm vein image collected in the palm vein image quality evaluation method embodiment of the present application;

[0054] Figure 5The image shows a user's right palm vein, collected in an embodiment of the palm vein image quality evaluation method of the present invention.

[0055] Figure 6 This is a schematic diagram of the interactive guidance on the interactive interface in an embodiment of the computer program product of the present invention;

[0056] Figure 7 This is a schematic diagram of six hand postures when a user collects palm vein images without contact in an embodiment of the palm vein image quality evaluation method of the present invention; wherein (a) is the standard state, (b) the palm is raised horizontally, (c) the palm and wrist are raised, (d) the palm and fingers are raised, (e) the palm is moved forward as a whole, and (f) the palm is moved backward as a whole.

[0057] Figure 8 This is a schematic diagram illustrating the principle of training a linear kernel SVM classifier in an embodiment of the palm vein image quality evaluation method of the present invention;

[0058] Figure 9 This is a schematic diagram illustrating the principle of the palm ROI region target detection model in an embodiment of the palm vein image quality evaluation method of the present invention;

[0059] Figure 10 This is a training data sample used when training the palm ROI region detection model in an embodiment of the palm vein image quality evaluation method of the present invention;

[0060] Figure 11 This is a sample of test data used when training the palm ROI region detection model in an embodiment of the palm vein image quality evaluation method of the present invention;

[0061] Figure 12 In this embodiment of the palm vein image quality evaluation method of the present invention, an example group 1 of ROI regions obtained by detecting palm vein images using a palm ROI region detection model is provided.

[0062] Figure 13 In an embodiment of the palm vein image quality evaluation method of the present invention, an example two of the ROI regions (including the surrounding environment image of the palm) obtained by detecting palm vein images using a palm ROI region detection model is provided. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0064] The overall concept of the present application is directed to the palm vein non-contact authentication problem, and proposes a non-contact palm vein image quality evaluation and palm vein biometric authentication method based on machine learning, to distinguish the palm posture and ROI region image quality when collecting palm vein images. The basic steps of the palm vein image quality evaluation method are as follows:

[0065] (1) Collect palm vein images of different placement positions of the user during identity authentication by a hardware device, and train a palm posture judgment model by machine learning technology. The hardware device is any existing non-contact palm vein image acquisition equipment, and the structure of the hardware device will not be described here.

[0066] (2) When the user performs non-contact authentication, the palm posture of the user is judged according to the palm posture judgment model constructed in the foregoing step (1), and only the correct palm posture can continue to the subsequent link; otherwise, the user is prompted to adjust the palm position and posture.

[0067] (3) A palm ROI region target detection model is trained by machine learning technology according to a pre-constructed palm ROI region data set. The palm image ROI region obtained by the above step (2) is detected by using the palm ROI region target detection model.

[0068] (4) A palm ROI image quality evaluation model is trained by machine learning technology according to a pre-constructed "ROI image-visual feature-quality score" data set. The palm image ROI region quality obtained by the above step (3) is calculated by using the palm ROI image quality evaluation model. If the ROI quality passes the evaluation, the algorithm ends, otherwise the user is prompted that the palm image quality does not meet the requirements, and returns to step (2) to require the user to collect the image again.

[0069] The palm vein image quality evaluation method mainly includes three stages, i.e. "palm posture judgment stage", "palm ROI region extraction stage" and "ROI image quality evaluation stage". The "palm posture judgment stage" executes the foregoing steps (1) and (2), and uses a pre-trained machine learning model to judge the posture of the obtained palm image, only the palm image meeting the requirements can enter the next stage for processing; the "palm ROI region extraction stage" executes the foregoing step (3), and uses a pre-trained deep learning target detection Faster-RCNN model to detect the palm image output by the "palm posture judgment stage" to obtain the ROI region thereof; the "ROI image quality evaluation stage" executes the foregoing step (4), and uses a pre-trained machine learning model to evaluate the ROI image quality obtained by the "palm ROI region extraction stage" to determine whether it can be used for the subsequent authentication process. The steps are as follows:

[0070] I. Palm posture judgment stage:

[0071] (A) Palm image acquisition

[0072] The palm vein image of the user to be authenticated is collected by a non-contact hardware device (a collection device based on a near-infrared camera). The input signal of the non-contact hardware device is the palm of the user during authentication, and the output signal is the palm vein image collected by the near-infrared camera.

[0073] (B) Palm posture judgment model training

[0074] A classifier model is pre-trained according to various palm postures (correct palm posture, incorrect palm posture) that the user may place. The input signal of the classifier model is the palm vein image obtained by placing the correct palm posture and the incorrect palm posture of different test users respectively through the palm image acquisition stage, and the output signal is the classifier model trained through the above input signal, which is used as the palm posture judgment model.

[0075] (C) Palm posture judgment: The palm posture judgment model is used to judge the posture of the palm vein image obtained by the user to be authenticated through the "palm image acquisition" stage. Only the image that meets the requirements can enter the next step for processing. Since the present application is directed to a non-contact palm vein authentication system, the present application can also adopt an interactive mode to guide the user to place the palm position correctly to speed up the authentication. After the user places the palm, the palm posture judgment model is used to judge whether the user's palm posture is correct in this stage. The input signal of this part is the palm vein image of the user to be authenticated obtained through the "palm image acquisition" stage, and the output signal is the palm vein image corresponding to the correct palm posture of the user after judgment.

[0076] II. Palm ROI region extraction stage:

[0077] (A) Palm ROI region annotation: First, manually annotate the ROI region (i.e. "target" in the palm vein image) of a large number of palm vein images, then train a target detection deep neural network using the annotated data, and use the palm ROI region target detection model to detect the ROI region of the palm image of the user to be authenticated. The input signal of this part is the palm vein image of the correct palm posture of different users to be authenticated, and the output is the palm ROI region after manual annotation.

[0078] (B) Palm ROI region detection model training: According to the data pair composed of the palm ROI region labeled in the "palm ROI region labeling" stage and the corresponding palm vein image, a target detection Faster-RCNN deep neural network is trained as a palm ROI region target detection model. The input signal is the labeled palm ROI region and the corresponding palm vein image, and the output signal is the Faster-RCNN deep neural network model trained by the above input signal as the palm ROI region target detection model.

[0079] (C) Palm ROI region detection: The palm ROI region target detection model is used to detect the ROI region of the palm vein image of the user to be authenticated obtained in the "palm posture judgment stage". The input signal of this step is the palm vein image of the user to be authenticated obtained in the "palm posture judgment stage", and the trained Faster-RCNN deep neural network model obtained in the "palm ROI region detection model training" stage, and the output signal is the ROI region of the palm vein image of the user to be authenticated.

[0080] III. ROI image quality evaluation stage:

[0081] (A) ROI image quality score labeling: The image quality of the output signal (i.e. the labeled palm image ROI region) of the "palm ROI region labeling" stage in the "palm ROI region extraction stage" is evaluated by subjective evaluation and objective evaluation, and an evaluation score (between 0.0-1.0) is set. Thus, the "image-quality score" training data set is formed to prepare the training data for the subsequent "ROI image quality evaluation model training" stage. The input signal of this part is the output signal of the "palm ROI region labeling" stage in the "palm ROI region extraction stage", i.e. the palm image ROI region. The output signal is the labeled "image-quality score" training data set.

[0082] (B) ROI image feature calculation: Calculate the various visual features of the palm ROI region image. The input signal of this part is the output signal of the "palm ROI region labeling" stage in the "palm ROI region extraction stage", i.e. the palm ROI region. The output signal is the "image-visual feature" data set calculated by the feature.

[0083] (C) ROI image quality evaluation model training: a support vector machine classification model is trained according to the signal output in the "ROI image feature calculation" stage. The input signal in this part is the "ROI image-visual feature-quality score" data set obtained according to the "image-quality score" training data set and the "image-visual feature" output in the "ROI image feature calculation" stage, and the output signal is the support vector machine classification model trained by the above input signal, which is used as the palm ROI image quality evaluation model.

[0084] (D) ROI image quality evaluation: the support vector machine classification model obtained in the "ROI image quality evaluation model training" is used to evaluate the image quality of the ROI region of the palm vein image of the user to be authenticated obtained in the "palm ROI region extraction stage". The input signal in this part is the ROI region of the palm vein image of the user to be authenticated obtained in the "palm ROI region extraction stage" and the trained support vector machine classification model obtained in the "ROI image quality evaluation model training", and the output signal is the ROI region quality evaluation score value (which can be between 0.0 and 1.0).

[0085] Only the ROI region with the ROI region quality evaluation score value meeting the preset requirement is allowed to continue the biometric authentication, otherwise, the palm vein image is re-collected to ensure the reliability of the biometric authentication. As shown in the flowchart of the palm vein image biometric authentication (which can also be registration) method of the application: Figure 3

[0086] (1) a non-contact hardware device is used to collect the palm vein image of the user;

[0087] (2) the palm posture judgment model is used to judge whether the palm posture of the user is accurate when the image is collected, if yes, the subsequent steps are executed, otherwise, the palm vein image of the user is re-collected in (1);

[0088] (3) the palm ROI region target detection model is used to extract the ROI region in the palm vein image;

[0089] (4) the palm ROI image quality evaluation model is used to judge whether the image quality of the ROI region meets the preset requirement, if yes, the subsequent steps are continued, otherwise, the palm vein image of the user is re-collected in (1);

[0090] (5) the ROI region is subjected to feature calculation, and the palm vein image is authenticated in combination with the externally preset palm vein feature registration database, or the ROI region is subjected to feature calculation and registered, the result of the ROI region subjected to feature calculation is sent to the palm vein feature registration database, which is the registration process, and the result of the ROI region subjected to feature calculation is judged by the palm vein feature registration database, which is the authentication process. ​

[0091] (6) judging whether the registration or authentication is successful, and completing the authentication or registration with the judgment result as prompt information.

[0092] The following is a specific embodiment of the present application:

[0093] 1. Palm vein image acquisition

[0094] The palm vein image of the user to be authenticated is collected by a non-contact hardware device. As shown in Figure 4 and Figure 5 are the collected palm vein images of the left and right hands of the user to be authenticated, respectively.

[0095] 2. Interactive guidance collection of palm vein images

[0096] In a contact palm vein authentication system, there are usually hand supports or contact panels and other devices to assist image collection, and the palm posture of the user to be authenticated is usually fixed. In non-contact authentication, the distance between the user's palm and the camera, as well as the palm posture and placement position, are all variable, so corresponding algorithms must be designed to ensure that the posture and position of the collected palm image are correct. The present application proposes an interactive guidance method to facilitate the user to correctly place the palm position in the non-contact palm vein authentication system, thereby ensuring the quality of the collected image. As shown in Figure 6 , the program product corresponding to the biometric authentication method of the present application dynamically displays the palm shape on the interactive interface according to the distance between the palm and the camera, and the user can adjust the position, posture and distance of his own palm to adapt to the palm shape, thereby ensuring that the system collects the correct palm vein image.

[0097] 3. Training of palm posture judgment model

[0098] In addition to the interactive guidance collection method, the present application also proposes a palm posture classifier model based on machine learning to judge the palm posture of the user. Since the palm posture of the user can be flexibly changed in non-contact authentication, it is necessary to first judge whether the palm posture of the user is in a flat state in non-contact authentication, and only the palm image obtained in the flat state can be used for subsequent authentication. As shown in Figure 7 , through experiments, the present application classifies the palm posture of the user in non-contact authentication into 6 categories, namely standard (i.e. correct palm posture), palm raised horizontally (high), palm raised with wrist (Raised back), palm raised with fingers (Front raised), palm moved forward (forward), and palm moved backward (backward), as shown in Figure 7 (a)-(f), respectively. The present application uses a non-contact hardware collection device to collect 6 palm posture images of the left and right hands of each of multiple persons as training image data.

[0099] The palm posture collection scheme is as follows:

[0100] (1) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, and the distance from the camera is about 12 cm (experimental data, the same below), and shoot the "standard state" posture.

[0101] (2) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, and the distance from the camera is more than 15 cm, and shoot the "palm horizontal lifting" posture.

[0102] (3) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, the wrist is lifted, the finger part remains unchanged, the distance from the camera is about 12 cm, and shoot the "palm wrist lifting" posture.

[0103] (4) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, the finger part is lifted, the wrist remains unchanged, the distance from the camera is about 12 cm, and shoot the "palm finger lifting" posture.

[0104] (5) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, and the whole palm moves forward by a part, the distance from the camera is about 12 cm, and shoot the "palm whole body forward" posture.

[0105] (6) Place the palm of the person to be collected horizontally above the camera, with the palm facing down, the five fingers naturally spread out, place the palm, keep the whole palm stable, and the whole palm moves backward by a part, the distance from the camera is about 12 cm, and shoot the "palm whole body backward" posture.

[0106] The above training image data is used to train the classifier model to obtain a palm posture judgment model, which is used to judge whether the palm posture of the user to be authenticated meets the requirements in the subsequent process.

[0107] As Figure 8As shown, the principle of the palm posture classifier model is: the deep features of the palm vein image are extracted using a ResNet-18 deep network and a DenseNet-121 deep network respectively, then the extracted 1024-dimensional DenseNet-121 features and 512-dimensional ResNet-18 features are fused, and then a linear kernel SVM classifier serving as the palm posture classifier model is trained based on the fused deep features. Equal Error Rate (EER) can be used as an evaluation index. EER is an evaluation index widely used in biometric authentication. EER is the value of the intersection point of the ROC curve (Receiver Operating Characteristic Curve) with the straight line y=x, that is, the error rate when the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) are equal. As shown in Table 1, the experimental performance of different features and different classifiers is compared, and the percentage in Table 1 represents the error rate when the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) are equal. As can be seen from Table 1, the fused features and the linear kernel SVM classifier used in the present application can achieve the best performance.

[0108] Table 1 Palm posture recognition ERR comparison table

[0109]

[0110] 4. Training the palm ROI region detection model

[0111] ROI region detection is a key link of palm vein image authentication. If the ROI region is regarded as a target in the palm vein image, a target detection algorithm can be used to detect the ROI region. In recent years, deep learning frameworks represented by Faster R-CNN models have achieved great success in image target detection applications. The present application uses a Faster R-CNN model to automatically detect the ROI region in the palm vein image, and achieves good experimental results.

[0112] (1) The initial image size can be arbitrary, according to the actual image size, that is, Figure 9 The values of P and Q in the formula can be arbitrary. The input picture size is 600*600, which means that the original image is converted to M=600, N=600 size after image scaling, and then input into Figure 9 the Backbone backbone network shown in the formula;

[0113] (2) The Backbone backbone network in the present application uses Resnet-50 architecture. Resnet-50, named deep residual network, is a classic deep network, which is widely used in image classification tasks.Figure 9 The role of the FeatureMap in the input image is to calculate the feature map. Resnet-50 is also a widely used deep network with good application effect. For specific principles, please refer to the statement of Kaiming He et al. in “Deep Residual Learning for Image Recognition” published in 2016 IEEE Conference on Computer Vision and Pattern Recognition. This will not be described in detail here.

[0114] The size of the Feature Map generated by Resnet-50 is (M / 16)*(N / 16). The parameters of Resnet-50 when generating the Feature Map are as follows:

[0115] All conv layers (convolution layers) are: kernel_size = 3, pad = 1, stride = 1

[0116] All pooling layers (pooling layers) are: kernel_size = 2, pad = 0, stride = 2

[0117] kernel_size: convolution kernel size

[0118] pad: padding width (set for the convenience of processing image boundary data)

[0119] stride: step

[0120] The role of the convolution layer is to extract information in the input image. These information is called image features, which are embodied by each pixel in the image through combination or independent means. Convolution is achieved through a filter. The pooling layer is connected after the convolution layer, and its purpose is to compress the size of the image to speed up the operation of the neural network, while retaining the features. Pooling is achieved through downsampling (the kernel of the pooling is only responsible for framing the image range required for the pooling calculation).

[0121] (3) After the Feature Map is generated, the RPN (Region Proposal Network) will be trained. The RPN network processes candidate regions (that is, possible target regions). Since the Feature Map and the original image have a correspondence, one pixel in the Feature Map corresponds to a 16*16 pixel block in the original image. Therefore, when sliding on the Feature Map, multiple anchors (anchor areas, i.e., target candidate areas, which can frame a rectangular area in the original image, which may be the target or the background) will be generated for the corresponding area in the original image. Since the generation of anchors is fixed, after the Feature Map is generated, the position and size of all anchors mapped to the original image are fixed and known. Figure 9 In the algorithm, a 3×3 convolution is performed on the feature map, followed by two separate 1×1 convolutions. One 1×1 convolution has 9 channels, and the other has 36 channels. The RPN network can be trained using pre-labeled target locations on a training dataset. The output of the RPN network is the probability that the network predicts a given anchor as foreground or background, along with the bounding box regression parameters for that anchor.

[0122] (4) The Proposal calculated from the feature map and the RPN prediction box is used to obtain a fixed-size prediction target feature map through ROI Pooling, that is, the target is obtained from the feature map using the prediction box.

[0123] bbox_pred: Adjusts the size of the output target suggestion box;

[0124] Softmax classification: This method uses a normalized exponential function to solve multi-class regression problems, often in conjunction with the cross-entropy loss function.

[0125] cls_pred: The probability of predicting something as foreground or background, including the type of object;

[0126] The parameters for ROI region detection using the Faster R-CNN model are as follows: the backbone network uses the ResNet-50 architecture, the original image is scaled to 600*600 pixels, the training epoch is 100, and the maximum and minimum learning rates during training are 1e. -4 and 1e -6 The confidence level for prediction is 0.5, the nonmaximum suppression value is 0.3, and the prior box size is [8, 16, 32].

[0127] use Figure 10 The data samples shown are used as training data samples. Figure 11The data sample shown is used as a test data sample during training, and the Faster R-CNN model is trained to obtain a palm ROI region detection model.

[0128] The palm ROI region detection model is used for detecting the palm vein image to obtain Figure 12 and Figure 13 The ROI region example shown in Figure 13 The image also contains an environment image, and Figure 12 and Figure 13 It can be seen that the palm ROI region detection model has good robustness and can accurately detect the palm ROI region in various environments.

[0129] 5. ROI region feature calculation

[0130] The visual features of the ROI region image are calculated to construct an "ROI image-visual feature" data set, and then a palm ROI image quality evaluation model is trained through machine learning technology to automatically evaluate the ROI region quality. Through a large number of experiments, the following visual features are selected for quality evaluation:

[0131] (1) Sharpness: This feature is used to calculate the "sharpness" of the ROI image vein edge, and the algorithm is as follows:

[0132] Step 1: Use the Canny operator to extract the vein edge in the ROI image to obtain the edge image;

[0133] Step 2: Count the number of edge image pixels E;

[0134] Step 3: Take the ratio of E and the size S of the ROI image as the image sharpness C, as shown in the following formula:

[0135]

[0136] (2) Information entropy: Calculate the information entropy of the ROI image, as shown in the following formula:

[0137]

[0138] Where p m represents the probability value of the pixel gray value m, and its calculation method is the ratio of the number of pixels with pixel gray value m and the total pixel value of the ROI image.

[0139] (3) Contrast: The image sharpness can be judged by contrast. The larger the contrast, the higher the image quality and sharpness. The contrast B is defined as shown in the following formula:

[0140]

[0141] where x i is the pixel gray value, x mean is the average value of the pixel gray value, and M is the total number of pixels in the ROI image.

[0142] (4) Equivalent Numbers of Looks: Equivalent Numbers of Looks (ENL) is one of the most commonly used image quality evaluation criteria, and a lower equivalent number of looks indicates that the image has better contrast. The equivalent number of looks D is defined as the ratio of the image gray standard deviation P std to the image gray mean P avg , as shown in the following formula:

[0143]

[0144] 6. Training the ROI image quality evaluation model

[0145] Based on the ROI image visual features calculated in 5, the ROI image quality evaluation model is constructed using a traditional classifier algorithm. Experiments show that the support vector machine classifier with a linear kernel can be used for ROI image quality evaluation. When training the model, each ROI region corresponding image needs to be scored in advance, and the visual features and score of each ROI region image can be used to construct the model.

[0146] The scoring rules can be as follows: the score is defined between 0.0-1.0, and the higher the score, the better the image quality. The existing ROI region images can be evaluated subjectively by multiple experts to give scores. Two thresholds are set as t1=0.6 and t2=0.8, and the score below t1 is considered to be a low-quality image, the score between t1-t2 is considered to be a medium-quality image, and the score higher than t2 is considered to be a high-quality image. Experimental results show that the accuracy of the quality evaluation model proposed in the present application can reach 94.57%.

[0147] Correspondingly, the present application also provides a computer program product, including a computer program, which is executed by a processor to realize the steps of the palm vein image quality evaluation method or the palm vein image biometric authentication method described above. The method described above is formed into a computer program product, so that the method can be applied to a terminal device. The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the steps of the palm vein image quality evaluation method or the palm vein image biometric authentication method of the present application. The terminal device here can be a computer, a notebook, a palm computer, and various cloud servers and other computing devices, and the processor can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, or other programmable logic devices.

[0148] The above merely describes the preferred embodiments of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of evaluating the quality of a palm vein image, characterized by, The method comprises the following steps: S1, inputting a palm vein image acquired by a non-contact hardware device into a palm posture judgment model, if it is a standard palm posture, executing step S2, otherwise, reacquiring the palm vein image until it is a standard palm posture; The palm posture judgment model is obtained by the following method: training a classifier model using palm vein images under multiple palm postures of multiple people including a standard palm posture to obtain the palm posture judgment model; the standard palm posture is that the palm directly faces a non-contact palm vein image acquisition device, and the distance between the palm and the palm vein image acquisition device is a preset distance; S2, acquiring an ROI region by a palm ROI region target detection model; The palm ROI region target detection model is obtained by the following method: labeling an ROI region of a palm vein image acquired under a standard palm posture to obtain a palm vein image-ROI region data set, and training a target detection Faster-RCNN deep neural network through the palm vein image-ROI region data set to obtain the palm ROI region target detection model; S3, judging the ROI region acquired in step S2 by an ROI image quality evaluation model, if it meets a preset requirement, judging that the palm vein image meets the requirement, otherwise, reacquiring the palm vein image; The ROI image quality evaluation model is obtained by the following method: calculating a visual feature of each ROI region in the palm vein image-ROI region data set in step S2, and scoring the quality of each ROI region to obtain an ROI region-visual feature-quality score data set, and training a support vector machine classification model through the ROI region-visual feature-quality score data set to obtain the ROI image quality evaluation model.

2. The palm vein image quality evaluation method according to claim 1, characterized by, In step S1, the palm posture judgment model is obtained by the following method: S1-1, using a ResNet-18 deep network and a DenseNet-121 deep network to extract deep features of palm vein images under multiple palm postures of multiple people respectively to correspondingly obtain 1024-dimensional DenseNet-121 features and 512-dimensional ResNet-18 features; S1-2, fusing the DenseNet-121 features and the ResNet-18 features to obtain fused deep features; S1-3, training a linear kernel SVM classifier based on the fused deep features obtained in step S1-2 to obtain the palm posture judgment model.

3. The palm vein image quality evaluation method according to claim 2, characterized by, In step S1-3, the evaluation index for training the linear kernel SVM classifier is the equal error rate.

4. The palm-vein image quality evaluation method according to any one of claims 1 to 3, characterized by, In step S1, the multiple palm postures include standard palm postures of left and right hands, palm horizontal lifting postures, palm wrist lifting postures, palm finger lifting postures, palm overall forward postures, and palm overall backward postures.

5. The palm-vein image quality evaluation method according to claim 4, characterized by: In step S3, the visual features include sharpness, information entropy, contrast, and equivalent visual number.

6. The palm-vein image quality evaluation method according to claim 5, characterized by: The sharpness is obtained by the following method: The venous edge of the ROI region is extracted by using a Canny operator to obtain an edge image of the ROI region, the number of pixel points E of the edge image is counted, and the definition C is obtained by the following formula: Wherein, S is the size of the ROI region; The information entropy H is obtained by the following formula: wherein p m is the probability value of a pixel having a pixel gray value of m; The contrast B is obtained by the following formula: wherein x i is the gray value of the i-th pixel, x mean is the average value of the pixel gray values, and M is the total number of pixel points in the ROI region. The equivalent visual number D is obtained by the following formula: Wherein, P std is the gray standard deviation of the ROI region, P avg is the gray mean of the ROI region.

7. The palm-vein image quality evaluation method according to claim 5, characterized by: In step S2, the backbone network of the target detection Faster-RCNN deep neural network uses a Resnet-50 architecture; The convolution layer parameters in the Resnet-50 architecture are: the convolution kernel size is 3, the padding width is 1, and the step is 1; The pooling layer parameters in the Resnet-50 architecture are: the convolution kernel size is 2, the padding width is 0, and the step is 2; The epoch of the target detection Faster-RCNN deep neural network during training is 100, the minimum learning rate and the maximum learning rate are 1e -4 and 1e -6 , the confidence is 0.5, the non-maximum suppression value is 0.3, and the prior box size is [8, 16, 32].

8. A computer program product comprising a computer program, characterized in that: The program is executed by the processor to realize the steps of the palm vein image quality evaluation method of any one of claims 1 to 7.

9. A palm vein image biometric authentication method characterized by, Comprising the following steps: S1, obtaining a palm vein image meeting the requirements by the palm vein image quality evaluation method of any one of claims 1 to 7; S2, performing feature calculation on the ROI region of the palm vein image meeting the requirements, combining a pre-set palm vein feature registration database, and completing palm vein image biological authentication.

10. A computer program product comprising a computer program, characterized in that: The program is executed by the processor to realize the steps of the palm vein image biological authentication method of claim 9.

Images