Palm print and palm vein camera evaluation method based on deep neural network

Abstract

The invention discloses a palm print and palm vein camera evaluation method based on a deep neural network. The method comprises the following steps: constructing a palm print and palm vein imaging camera quality evaluation interface; collecting a palm print and palm vein image, generating a palm print and palm vein region of interest by using a HandLandmarker model, and preprocessing the palm print and palm vein region of interest to obtain a preprocessed image; inputting the preprocessed image into an improved MobileFace model to construct a standard feature vector library, and obtaining a to-be-detected feature vector; performing similarity calculation on the to-be-measured feature vector and a standard feature vector library to generate an imaging quality evaluation conclusion; the evaluation information is displayed on an evaluation interface, and an evaluation log is generated; and executing abnormal detection of image blurring, insufficient number of palm key points, model loading failure and camera connection failure. According to the method, automatic and standardized evaluation of the quality of the palm print and palm vein imaging camera is realized, and the method can be used for production detection and quality control scenes of palm print and palm vein recognition equipment.

Description

Technical Field

[0001] This invention relates to the field of biometric recognition and intelligent detection technology, and in particular to a palm vein camera evaluation method based on deep neural networks. Background Technology

[0002] With the rapid popularization of biometric technology in industries such as security, finance, and attendance, palmprint and palm vein recognition, as a bimodal biometric feature that integrates epidermal texture and subcutaneous vascular structure, has attracted widespread attention due to its uniqueness, stability, and non-contact nature. The core front-end device of a palmprint and palm vein recognition system is a palmprint and palm vein imaging camera, whose imaging quality directly determines the reliability of subsequent feature extraction, identity comparison, and recognition. Current quality assessments of palmprint and palm vein imaging cameras mainly rely on manual visual observation, traditional image quality indicators, or simple feature matching methods, but these methods generally suffer from the following problems in practical applications:

[0003] Image quality assessment suffers from high subjectivity and low stability: Traditional methods, which rely on manual assessment of palm print and palm vein images for clarity, texture discernibility, or illumination uniformity, are heavily influenced by experience, resulting in significant discrepancies between assessment results and making it difficult to establish quantifiable and repeatable quality judgment standards. Traditional image metrics fail to accurately reflect the biometric characteristics of palm prints and palm veins: metrics such as resolution, contrast, and signal-to-noise ratio only describe visual quality and cannot depict the deeper biological information of palm print texture and vein structure, often resulting in situations where image quality metrics are acceptable but actual recognition accuracy is low. Inaccurate palm region localization leads to unstable assessments: Existing methods typically obtain the palm region through simple segmentation or region cropping, which cannot accurately reflect the palm region's biometric characteristics. Adapting to changes in hand posture, rotation, and background interference can easily lead to mis-cropping or missing key texture regions, thus affecting the accuracy of feature extraction. Existing deep learning feature extraction models are not optimized for palm print and palm vein structures: traditional methods using general face or natural image feature extraction networks lack the ability to specifically model the directional texture of palm prints and the linear structure of veins, resulting in insufficient expression of key biological textures and poor sensitivity to weakly contrasting vein structures. There is a lack of systematic and automated evaluation processes: most existing solutions do not combine visualization interfaces, real-time result display, and log management, which is not conducive to batch testing and production quality inspection, and still relies on manual recording methods, which are inefficient and prone to errors.

[0004] Therefore, how to provide a palm vein camera evaluation method based on deep neural networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a palmprint and palm vein camera evaluation method based on deep neural networks. This invention constructs an image acquisition and evaluation interface, utilizes the HandLandmarker model for precise palm region localization, employs an improved MobileFace model to extract deep feature vectors of the palmprint and palm veins, and quantitatively evaluates the camera imaging quality based on feature vector similarity. Simultaneously, it incorporates an anomaly detection mechanism to improve the stability of the evaluation process. This method fully leverages the feature representation capabilities of deep learning and automated image processing technology to achieve automated, standardized, and high-precision evaluation of palmprint and palm vein camera quality, possessing advantages such as high evaluation objectivity, strong recognition stability, and applicability to batch quality inspection.

[0006] A palm vein camera evaluation method based on a deep neural network according to an embodiment of the present invention includes the following steps: Step 1: Build a quality assessment interface for palm vein imaging cameras based on PyQt, and complete the binding of interface components with backend processing functions so that the interface can execute image acquisition commands, registration commands, threshold setting commands, assessment trigger commands, and result display. Step 2: Acquire palm print and palm vein images, obtain palm key points based on the HandLandmarker model, generate the minimum bounding rectangle of the palm based on the palm key points, crop the palm print and palm vein region of interest, and perform preprocessing to generate a preprocessed image. Step 3: Input the preprocessed image into the improved MobileFace model to extract deep feature vectors, build a standard feature vector library based on the registered image, and use the feature vector corresponding to the image to be tested as the feature vector to be tested; Step 4: Calculate the similarity between the feature vector to be tested and the feature vectors in the standard feature vector library, and generate an evaluation conclusion on the imaging quality of the palm vein imaging camera based on the set threshold. Step 5: Display the registered image, the image to be tested, the similarity value, the evaluation conclusion, and the evaluation time on the evaluation interface, and export it as an evaluation log; Step 6: Perform anomaly detection and provide anomaly alerts for issues such as blurry images, insufficient number of palm key points, failure to load the improved MobileFace model, and failure to connect the palm vein imaging camera.

[0007] Optionally, the palmprint and palm vein imaging camera quality assessment interface is an interactive interface for controlling palmprint and palm vein image acquisition, registration, threshold setting, assessment triggering, and result display. The image acquisition command is an operation command for starting the palmprint and palm vein image acquisition process. The registration command is an operation command for acquiring and registering images and updating the standard feature vector library. The threshold setting command is an operation command for inputting or adjusting the similarity threshold. The assessment command is an operation command for starting the palmprint and palm vein imaging camera quality assessment process. The result display is the process of presenting images and assessment data on the assessment interface.

[0008] Optionally, step two specifically includes: By calling a palm print and palm vein imaging camera, palm print and palm vein images are acquired at a preset resolution and preset frame rate. The palm print and palm vein images are color images that include palm print texture structure and palm vein texture structure. The palm print and palm vein image is input into the HandLandmarker model, which is a hand key point detection model built based on the MediaPipe framework. Key point inference is performed to generate a set of two-dimensional key point coordinates representing the wrist, finger base and fingertip positions. The boundary points of the palm region are determined based on the coordinates of the two-dimensional key points. The smallest bounding rectangle containing the palm region is calculated based on the boundary points and used as the location box of the palm vein region of interest. A cropping operation is performed on the palm print and palm vein image according to the positioning box to obtain the palm print and palm vein region of interest. The region of interest is an image that covers the central area of ​​the palm and includes the main lines of the palm print texture and the main path of the palm vein. Noise reduction is performed on the region of interest by using a bilateral filtering method based on pixel neighborhood differences to preserve the edges of palm vein texture while removing Gaussian noise and salt-and-pepper noise. Perform contrast enhancement processing on the region of interest and equalize the pixel brightness histogram using the equalizeHist function; Perform pixel normalization processing to redistribute the image pixel values ​​according to a preset range, and adjust the image to a preset size to generate a preprocessed image.

[0009] Optionally, the improved MobileFace model includes a direction encoding module, a vein enhancement module, an enhanced Bottleneck trunk module, and a feature vector generation module: The directional encoding module performs multi-directional texture convolution on the preprocessed image and generates directional feature tensors based on different directional responses. The multi-directional texture convolution includes performing convolution operations on the local gradients of the preprocessed image through a preset directional filtering kernel. The vein enhancement module calculates the Hessian matrix and obtains the eigenvalues ​​of the preprocessed image, and generates a vein feature tensor through a nonlinear mapping based on the exponential function of the vein structure. The orientation feature tensor, vein feature tensor, and preprocessed image are spliced ​​along the channel direction to generate an enhanced input tensor, which is then input into the enhanced Bottleneck backbone module. The enhanced Bottleneck backbone module sequentially performs depthwise separable convolution, pointwise convolution, batch normalization, and activation function operations on the enhanced input tensor. A channel-space joint attention module is added to each Bottleneck unit. The channel attention part performs global average pooling on the enhanced input tensor to obtain channel statistical vectors, performs linear transformation and activation function transformation on the channel statistical vectors to generate channel weights, and multiplies the channel weights with the enhanced input tensor channel by channel. The spatial attention part performs max pooling and average pooling on the enhanced input tensor and concatenates them. Convolution is performed on the concatenated result to generate spatial weights, and the spatial weights are multiplied with the enhanced input tensor pixel by pixel. The weighted result is used as the output of the enhanced Bottleneck backbone module, forming the backbone feature map. The feature vector generation module performs second-order statistical pooling on the backbone feature map to generate statistical features, performs linear mapping on the statistical features to obtain feature vectors in the target feature space, and performs normalization operation on the feature vectors to obtain deep feature vectors. Write the deep feature vector corresponding to the registered image into the standard feature vector library, and use the deep feature vector corresponding to the image to be tested as the feature vector to be tested.

[0010] Optionally, step four specifically includes: The vector to be tested is multiplied with each standard feature vector in the standard feature vector library. The vector inner product is obtained by performing element-wise multiplication on the vector elements of the corresponding dimensions and summing the product results. The vector norm is calculated for the eigenvector to be tested and each standard eigenvector. The corresponding vector norm value is obtained by summing the squares of each vector element and performing a square root operation on the summation result. Based on the vector inner product result and the norm value of the feature vector to be tested and the norm value of the corresponding standard feature vector, the cosine similarity between the feature vector to be tested and each standard feature vector is calculated. The cosine similarity values ​​are averaged to obtain the final similarity. The final similarity is compared with a set threshold. When the final similarity is greater than or equal to the set threshold, an evaluation conclusion of qualified imaging quality is generated. When the final similarity is less than the set threshold, an evaluation conclusion of unqualified imaging quality is generated.

[0011] Optionally, step six specifically includes: Image quality detection is performed on the image under test. The pixel gray-level variance of the image under test is calculated and compared with a preset variance threshold. When the pixel gray-level variance is less than the preset variance threshold, an image blur anomaly label is generated. Perform palm region integrity detection on the image to be tested, count the number of key points detected, and generate a palm region missing anomaly marker when the number of key points is less than a preset key point number threshold. Perform loading integrity checks on the improved MobileFace model by performing file hash verification on the improved MobileFace model file and comparing the verification result with a preset hash value. If the comparison is inconsistent, generate a model loading anomaly flag. When calling the palm print and palm vein imaging camera, a camera connection detection is performed. This involves sending a handshake command to the communication port of the palm print and palm vein imaging camera and monitoring the return signal within a preset time window. If no return signal is detected, a camera connection error flag is generated. When any exception identifier is generated, the corresponding exception type is written into the exception record unit, and an exception prompt message is generated in the evaluation interface.

[0012] The beneficial effects of this invention are: This invention establishes an automated, standardized method for evaluating the quality of palm print and palm vein imaging cameras, which possesses depth feature discrimination capabilities, by constructing a quality evaluation interface for palm print and palm vein imaging cameras, a precise positioning module for the palm region, an improved MobileFace depth feature extraction model, and a similarity quantification evaluation mechanism.

[0013] First, this invention employs a deep neural network as the core feature representation method, utilizing the HandLandmarker model to analyze key points on the palm. This allows for the acquisition of a stable palm region even under conditions of varying illumination, pose shifts, and background interference, improving the accuracy of region-of-interest (ROI) cropping. Second, this invention introduces a directional encoding module, a vein enhancement module, an enhanced Bottleneck backbone module, and a second-order statistical pooling embedding layer into the MobileFace model. This enables the model to simultaneously capture the directional texture of palm prints and the linear structural features of palm veins, enhancing its representation ability for weak-contrast vein textures and complex palm print textures. Compared to the original model, this invention exhibits higher feature discrimination and stability. In terms of methodology, the feature extraction process is made more targeted and robust by performing directional convolution, Hessian matrix analysis, nonlinear exponential mapping, channel-space attention weighting, and second-order statistical feature generation on preprocessed images. Imaging quality is quantified by vector inner product, norm calculation, and cosine similarity averaging to ensure that the evaluation results are consistent with the actual recognition performance. An anomaly detection mechanism is used to promptly identify blurred images, missing key points, model loading failures, and camera connection anomalies, ensuring the reliability and engineering controllability of the evaluation process.

[0014] In summary, this invention significantly improves the accuracy, objectivity, and automation of palm vein camera quality assessment, and is applicable to scenarios such as production quality inspection, equipment calibration, and deployment of biometric systems. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0016] Figure 1 This is a flowchart of a palm vein camera evaluation method based on deep neural networks proposed in this invention. Figure 2 This is a schematic diagram of a palm vein camera evaluation method based on deep neural networks proposed in this invention. Figure 3 This is a framework diagram of the improved MobileFace model in the palm print and palm vein camera evaluation method based on deep neural networks proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figure 1-3A palm vein camera evaluation method based on deep neural networks includes the following steps: Step 1: Build a quality assessment interface for palm vein imaging cameras based on PyQt, and complete the binding of interface components with backend processing functions so that the interface can execute image acquisition commands, registration commands, threshold setting commands, assessment trigger commands, and result display. Step 2: Acquire palm print and palm vein images, obtain palm key points based on the HandLandmarker model, generate the minimum bounding rectangle of the palm based on the palm key points, crop the palm print and palm vein region of interest, and perform preprocessing to generate a preprocessed image. Step 3: Input the preprocessed image into the improved MobileFace model to extract deep feature vectors, build a standard feature vector library based on the registered image, and use the feature vector corresponding to the image to be tested as the feature vector to be tested; Step 4: Calculate the similarity between the feature vector to be tested and the feature vectors in the standard feature vector library, and generate an evaluation conclusion on the imaging quality of the palm vein imaging camera based on the set threshold. Step 5: Display the registered image, the image to be tested, the similarity value, the evaluation conclusion, and the evaluation time on the evaluation interface, and export it as an evaluation log; Step 6: Perform anomaly detection and provide anomaly alerts for issues such as blurry images, insufficient number of palm key points, failure to load the improved MobileFace model, and failure to connect the palm vein imaging camera.

[0019] In this embodiment, the palmprint and palm vein imaging camera quality assessment interface is an interactive interface for controlling palmprint and palm vein image acquisition, registration, threshold setting, assessment triggering, and result display. The image acquisition command is an operation command for starting the palmprint and palm vein image acquisition process. The registration command is an operation command for acquiring and registering images and updating the standard feature vector library. The threshold setting command is an operation command for inputting or adjusting the similarity threshold. The assessment command is an operation command for starting the palmprint and palm vein imaging camera quality assessment process. The result display is the process of presenting images and assessment data on the assessment interface.

[0020] This implementation method constructs a quality assessment interface for palm vein imaging cameras and unifies the binding of image acquisition commands, registration commands, threshold setting commands, and assessment trigger commands. This achieves visualization of the assessment process and integrated management of the operation logic, enabling users to complete the entire assessment process through a single interface. This reduces operational complexity, improves the controllability and consistency of the assessment process, and avoids data disorder and assessment deviations caused by scattered operations of multiple modules, thereby improving the standardization and execution efficiency of quality assessment.

[0021] In this embodiment, step two specifically includes: By calling a palm print and palm vein imaging camera, palm print and palm vein images are acquired at a preset resolution and preset frame rate. The palm print and palm vein images are color images that include palm print texture structure and palm vein texture structure. The palm print and palm vein image is input into the HandLandmarker model, which is a hand key point detection model built based on the MediaPipe framework. Key point inference is performed to generate a set of two-dimensional key point coordinates representing the wrist, finger base and fingertip positions. The boundary points of the palm region are determined based on the coordinates of the two-dimensional key points. The smallest bounding rectangle containing the palm region is calculated based on the boundary points and used as the location box of the palm vein region of interest. A cropping operation is performed on the palm print and palm vein image according to the positioning box to obtain the palm print and palm vein region of interest. The region of interest is an image that covers the central area of ​​the palm and includes the main lines of the palm print texture and the main path of the palm vein. Noise reduction is performed on the region of interest by using a bilateral filtering method based on pixel neighborhood differences to preserve the edges of palm vein texture while removing Gaussian noise and salt-and-pepper noise. Contrast enhancement is performed on the region of interest, and the pixel brightness histogram is equalized by the equalizeHist function to improve the brightness level of the palm vein texture. Perform pixel normalization processing to redistribute the image pixel values ​​according to a preset range, and adjust the image to a preset size to generate a preprocessed image.

[0022] This implementation uses the HandLandmarker model to obtain key points of the palm and crop the smallest bounding rectangle of the palm. At the same time, it performs noise reduction, equalization and normalization on the region of interest to generate a preprocessed image. It can stably extract the palm region under the conditions of pose displacement, illumination change and background interference, and enhance the contrast of palm print and palm vein texture, reduce the impact of noise on feature extraction, and provide reliable input for subsequent deep feature expression, thereby significantly improving the stability and accuracy of the evaluation results.

[0023] In this embodiment, the improved MobileFace model includes a directional encoding module, a vein enhancement module, an enhanced Bottleneck trunk module, and a feature vector generation module: The directional encoding module performs multi-directional texture convolution on the preprocessed image and generates directional feature tensors based on different directional responses. The multi-directional texture convolution includes performing convolution operations on the local gradients of the preprocessed image through a preset directional filtering kernel. The vein enhancement module calculates the Hessian matrix and obtains the eigenvalues ​​of the preprocessed image, and generates a vein feature tensor through a nonlinear mapping based on the exponential function of the vein structure. The orientation feature tensor, vein feature tensor, and preprocessed image are spliced ​​along the channel direction to generate an enhanced input tensor, which is then input into the enhanced Bottleneck backbone module. The enhanced Bottleneck backbone module sequentially performs depthwise separable convolution, pointwise convolution, batch normalization, and activation function operations on the enhanced input tensor. A channel-space joint attention module is added to each Bottleneck unit. The channel attention part performs global average pooling on the enhanced input tensor to obtain channel statistical vectors, performs linear transformation and activation function transformation on the channel statistical vectors to generate channel weights, and multiplies the channel weights with the enhanced input tensor channel by channel. The spatial attention part performs max pooling and average pooling on the enhanced input tensor and concatenates them. Convolution is performed on the concatenated result to generate spatial weights, and the spatial weights are multiplied with the enhanced input tensor pixel by pixel. The weighted result is used as the output of the enhanced Bottleneck backbone module, forming the backbone feature map. The feature vector generation module performs second-order statistical pooling on the backbone feature map to generate statistical features, performs linear mapping on the statistical features to obtain feature vectors in the target feature space, and performs normalization operation on the feature vectors to obtain deep feature vectors. Write the deep feature vector corresponding to the registered image into the standard feature vector library, and use the deep feature vector corresponding to the image to be tested as the feature vector to be tested.

[0024] This implementation adds a directional encoding module, a vein enhancement module, an enhanced Bottleneck backbone module, and a second-order statistical pooling embedding layer to the MobileFace model. This enables the model to enhance palm print and palm vein texture features from multiple angles, improving the model's ability to express directional palm print textures and low-contrast palm vein structures. It also enhances feature discrimination and robustness, making the similarity of different images in the feature space more accurately reflect the real imaging quality and significantly improving evaluation accuracy.

[0025] In this embodiment, step four specifically includes: The vector to be tested is multiplied with each standard feature vector in the standard feature vector library. The vector inner product is obtained by performing element-wise multiplication on the vector elements of the corresponding dimensions and summing the product results. The vector norm is calculated for the eigenvector to be tested and each standard eigenvector. The corresponding vector norm value is obtained by summing the squares of each vector element and performing a square root operation on the summation result. Based on the vector inner product result and the norm value of the feature vector to be tested and the norm value of the corresponding standard feature vector, the cosine similarity between the feature vector to be tested and each standard feature vector is calculated. The cosine similarity values ​​are averaged to obtain the final similarity. The final similarity is compared with a set threshold. When the final similarity is greater than or equal to the set threshold, an evaluation conclusion of qualified imaging quality is generated. When the final similarity is less than the set threshold, an evaluation conclusion of unqualified imaging quality is generated.

[0026] This implementation method calculates the dot product, norm, and cosine similarity between the feature vector to be tested and the standard feature vector, and uses the average similarity combined with a threshold to determine the imaging quality. This provides a quantifiable and repeatable method for measuring imaging quality, ensuring that the evaluation results are not affected by human judgment errors, guaranteeing the objectivity and consistency of the evaluation standards, and maintaining a high correlation between the evaluation results and the actual recognition performance, thereby improving the overall system reliability.

[0027] In this embodiment, step six specifically includes: Image quality detection is performed on the image under test. The pixel gray-level variance of the image under test is calculated and compared with a preset variance threshold. When the pixel gray-level variance is less than the preset variance threshold, an image blur anomaly label is generated. Perform palm region integrity detection on the image to be tested, count the number of key points detected, and generate a palm region missing anomaly marker when the number of key points is less than a preset key point number threshold. Perform loading integrity checks on the improved MobileFace model by performing file hash verification on the improved MobileFace model file and comparing the verification result with a preset hash value. If the comparison is inconsistent, generate a model loading anomaly flag. When calling the palm print and palm vein imaging camera, a camera connection detection is performed. This involves sending a handshake command to the communication port of the palm print and palm vein imaging camera and monitoring the return signal within a preset time window. If no return signal is detected, a camera connection error flag is generated. When any exception identifier is generated, the corresponding exception type is written into the exception record unit, and an exception prompt message is generated in the evaluation interface.

[0028] This implementation method achieves multi-type anomaly recognition by performing grayscale variance detection on images, integrity detection on the number of key points, hash verification on model files, and handshake response detection on camera ports. It can promptly alert users when there are abnormal image quality, incomplete palm area, damaged model, or device connection failure, thus avoiding the participation of erroneous images or abnormal models in the evaluation, improving the stability and security of system operation, and ensuring the reliability of the evaluation process.

[0029] Example 1: To verify the feasibility of this invention in practice, it was applied to the camera quality inspection process of a palmprint and palm vein recognition device manufacturer. On the manufacturer's palmprint and palm vein recognition product production line, each camera module must undergo image quality verification after assembly to ensure that the acquired palmprint and palm vein images meet the requirements of the recognition system in terms of texture visibility, vein visibility, contrast, and consistency. However, traditional quality inspection methods mainly rely on manual observation or simple image index judgments, resulting in poor stability of detection results between different batches of cameras and failing to directly reflect the device's usability in actual recognition scenarios. Therefore, this embodiment deploys the method of this invention at the quality inspection station, establishing a quantifiable, objective, and traceable image quality evaluation process through deep feature similarity.

[0030] In actual verification scenarios, quality inspectors connect a palmprint and palm vein imaging camera to the verification computer. They then select the corresponding camera device number in the visual evaluation interface built with PyQt and activate the image acquisition function. The acquired image is displayed in real-time on the left side of the interface to assist quality inspectors in adjusting their hand position. Before the evaluation begins, the system first requires the acquisition of three registered palmprint and palm vein images as standard samples. Each image, after being detected by HandLandmarker and cropped to the smallest bounding rectangle of the palm, undergoes preprocessing steps such as bilateral filtering, histogram equalization, and normalization before being input into the improved MobileFace model to calculate deep feature vectors. These features are automatically written into the standard feature vector library as a comparison benchmark for subsequent quality assessments. Subsequently, quality inspectors acquire three test images from the camera under test with the same hand posture, and each image also generates a test feature vector following the same process.

[0031] To demonstrate the practical evaluation results, this embodiment constructs a test scenario involving ten palmprint and palm vein cameras. Standard images are captured from the same hand at a distance of approximately 12 cm, with ambient illumination maintained at around 350 lx to ensure consistency of the standard samples. Images captured by the cameras under test may be affected by sensor noise, lens cleanliness, uneven illumination, and differences in spectral response in real-world scenarios. Therefore, the images may exhibit blurred textures, weakened veins, and low contrast. This embodiment specifically selects ten cameras with different imaging performance characteristics as test subjects to demonstrate the resolution and stability of the method at different quality levels.

[0032] Through the input and comparison process of deep feature vectors, this embodiment obtains the average similarity value corresponding to each camera, and uses the system threshold of 0.57 as the judgment criterion. When the average cosine similarity between the feature vector to be tested of a camera and the samples in the standard feature vector library is greater than or equal to 0.57, the imaging quality of that camera is considered qualified. This embodiment records the evaluation results in Table 1 according to the camera number, and retains all similarity values, evaluation conclusions, and automatically judged abnormal information.

[0033] Table 1 Camera Image Quality Assessment Data Table

[0034] As shown in Table 1, the depth feature vector similarity method can accurately reflect the actual imaging capabilities of different cameras. For example, the mean similarity values ​​of C001, C002, and C010 are all higher than 0.75, indicating that the palm vein textures they collected are highly consistent with the standard samples in terms of structural integrity, vein response intensity, and texture clarity. The corresponding fuzzy variances are all greater than 95, and the number of keypoints reaches 21 complete keypoints, indicating good image quality, accurate hand localization, and meeting expectations.

[0035] Although the average similarity of C003 is 0.607, which is only slightly higher than the threshold of 0.57, the image quality indicators show a blur variance of 82 and a key point count of 20, indicating that the image is slightly blurred but still maintains the main texture structure. The depth feature determination result is consistent with the visual judgment.

[0036] Both C004 and C005 were deemed unqualified, but for different reasons. C004 had a fuzzy variance of only 47, a typical image blurring problem, resulting in the inability to effectively extract deep texture information, thus its mean similarity was as low as 0.543. While C005 had a fuzzy variance of 90, it only had 17 keypoints, indicating incomplete detection of the palm region, leading to insufficient cropping or cropping offset, resulting in a lower similarity score. This demonstrates that the anomaly detection mechanism of this invention can accurately distinguish different anomaly types, ensuring the interpretability of the evaluation results.

[0037] The fuzzy variance of C009 reaches 120, indicating a clear image, but the similarity is only 0.407. Further observation reveals that the camera's lighting is too cool, significantly reducing vein contrast. Therefore, although the image sharpness is acceptable, the characteristics of the palm vein texture are weakened, leading to a mismatch in deep features. Traditional sharpness or contrast metrics often fail to detect this situation, while this invention expresses biological texture information through depth features, accurately reflecting the effective performance of the imaging device in the recognition scene.

[0038] A comprehensive analysis of all samples shows that the method of this invention has sensitivity and discriminative power for different types of imaging degradation, and can quantitatively reflect the core imaging capabilities of the palm vein camera, rather than just basic image quality indicators.

[0039] This embodiment fully demonstrates the feasibility and advantages of the present invention in a real-world palmprint and palm vein camera quality inspection scenario. The present invention, through deep feature vectors, eliminates reliance on subjective human observation. Instead, it extracts the deep feature structures of palmprints and palm veins using an improved MobileFace model, and measures image quality using cosine similarity, ensuring that the evaluation results are highly consistent with the actual performance of the recognition system. The present invention applies MediaPipe to accurately locate the palm region, effectively eliminating background interference and focusing feature extraction on truly meaningful biological texture areas. Furthermore, the present invention also provides blur detection, keypoint integrity detection, model loading detection, and camera connectivity detection, making the entire evaluation process automated, standardized, and traceable.

[0040] This invention can accurately identify various imaging quality problems such as image blurring, palm detection errors, and weak vein visualization, exhibiting good robustness and engineering applicability. In a production quality inspection environment, this invention significantly improves evaluation consistency, bringing the differences in evaluation results between different batches and different personnel close to zero, effectively enhancing the reliability of equipment outgoing pass rate management. Furthermore, the evaluation interface supports one-click export of Excel logs, making quality inspection data management more standardized and providing a basis for quality tracking throughout the production process.

[0041] This invention is not only applicable to the production and quality inspection of palm vein recognition devices, but can also be further extended to application scenarios such as equipment online maintenance and operational status monitoring, and has broad engineering value and industrial significance.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A palm vein camera evaluation method based on deep neural networks, characterized in that, Includes the following steps: Step 1: Build a quality assessment interface for palm vein imaging cameras based on PyQt, and complete the binding of interface components with backend processing functions so that the interface can execute image acquisition commands, registration commands, threshold setting commands, assessment trigger commands, and result display. Step 2: Acquire palm print and palm vein images, obtain palm key points based on the HandLandmarker model, generate the minimum bounding rectangle of the palm based on the palm key points, crop the palm print and palm vein region of interest, and perform preprocessing to generate a preprocessed image. Step 3: Input the preprocessed image into the improved MobileFace model to extract deep feature vectors, build a standard feature vector library based on the registered image, and use the feature vector corresponding to the image to be tested as the feature vector to be tested; Step 4: Calculate the similarity between the feature vector to be tested and the feature vectors in the standard feature vector library, and generate an evaluation conclusion on the imaging quality of the palm vein imaging camera based on the set threshold. Step 5: Display the registered image, the image to be tested, the similarity value, the evaluation conclusion, and the evaluation time on the evaluation interface, and export it as an evaluation log; Step 6: Perform anomaly detection and provide anomaly alerts for issues such as blurry images, insufficient number of palm key points, failure to load the improved MobileFace model, and failure to connect the palm vein imaging camera.

2. The palmprint and palm vein camera evaluation method based on deep neural networks according to claim 1, characterized in that, The palmprint and palm vein imaging camera quality assessment interface is an interactive interface used to control palmprint and palm vein image acquisition, registration, threshold setting, assessment triggering, and result display. The image acquisition command is an operation command used to start the palmprint and palm vein image acquisition process. The registration command is an operation command used to acquire and register images and update the standard feature vector library. The threshold setting command is an operation command used to input or adjust the similarity threshold. The assessment command is an operation command used to start the palmprint and palm vein imaging camera quality assessment process. The result display is the process of presenting images and assessment data on the assessment interface.

3. The palmprint and palm vein camera evaluation method based on deep neural networks according to claim 1, characterized in that, Step two specifically includes: By calling a palm print and palm vein imaging camera, palm print and palm vein images are acquired at a preset resolution and preset frame rate. The palm print and palm vein images are color images that include palm print texture structure and palm vein texture structure. The palm print and palm vein image is input into the HandLandmarker model, which is a hand key point detection model built based on the MediaPipe framework. Key point inference is performed to generate a set of two-dimensional key point coordinates representing the wrist, finger base and fingertip positions. The boundary points of the palm region are determined based on the coordinates of the two-dimensional key points. The smallest bounding rectangle containing the palm region is calculated based on the boundary points and used as the location box of the palm vein region of interest. A cropping operation is performed on the palm print and palm vein image according to the positioning box to obtain the palm print and palm vein region of interest. The region of interest is an image that covers the central area of ​​the palm and includes the main lines of the palm print texture and the main path of the palm vein. Noise reduction is performed on the region of interest by using a bilateral filtering method based on pixel neighborhood differences to preserve the edges of palm vein texture while removing Gaussian noise and salt-and-pepper noise. Perform contrast enhancement processing on the region of interest and equalize the pixel brightness histogram using the equalizeHist function; Perform pixel normalization processing to redistribute the image pixel values ​​according to a preset range, and adjust the image to a preset size to generate a preprocessed image.

4. The palmprint and palm vein camera evaluation method based on deep neural networks according to claim 1, characterized in that, The improved MobileFace model includes a direction encoding module, a vein enhancement module, an enhanced Bottleneck trunk module, and a feature vector generation module. The directional encoding module performs multi-directional texture convolution on the preprocessed image and generates directional feature tensors based on different directional responses. The multi-directional texture convolution includes performing convolution operations on the local gradients of the preprocessed image through a preset directional filtering kernel. The vein enhancement module calculates the Hessian matrix and obtains the eigenvalues ​​of the preprocessed image, and generates a vein feature tensor through a nonlinear mapping based on the exponential function of the vein structure. The orientation feature tensor, vein feature tensor, and preprocessed image are spliced ​​along the channel direction to generate an enhanced input tensor, which is then input into the enhanced Bottleneck backbone module. The enhanced Bottleneck backbone module sequentially performs depthwise separable convolution, pointwise convolution, batch normalization, and activation function operations on the enhanced input tensor. A channel-space joint attention module is added to each Bottleneck unit. The channel attention part performs global average pooling on the enhanced input tensor to obtain channel statistical vectors, performs linear transformation and activation function transformation on the channel statistical vectors to generate channel weights, and multiplies the channel weights with the enhanced input tensor channel by channel. The spatial attention part performs max pooling and average pooling on the enhanced input tensor and concatenates them. Convolution is performed on the concatenated result to generate spatial weights, and the spatial weights are multiplied with the enhanced input tensor pixel by pixel. The weighted result is used as the output of the enhanced Bottleneck backbone module, forming the backbone feature map. The feature vector generation module performs second-order statistical pooling on the backbone feature map to generate statistical features, performs linear mapping on the statistical features to obtain feature vectors in the target feature space, and performs normalization operation on the feature vectors to obtain deep feature vectors. Write the deep feature vector corresponding to the registered image into the standard feature vector library, and use the deep feature vector corresponding to the image to be tested as the feature vector to be tested.

5. The palm vein and palm print camera evaluation method based on deep neural networks according to claim 1, characterized in that, Step four specifically includes: The vector to be tested is multiplied with each standard feature vector in the standard feature vector library. The vector inner product is obtained by performing element-wise multiplication on the vector elements of the corresponding dimensions and summing the product results. The vector norm is calculated for the eigenvector to be tested and each standard eigenvector. The corresponding vector norm value is obtained by summing the squares of each vector element and performing a square root operation on the summation result. Based on the vector inner product result and the norm value of the feature vector to be tested and the norm value of the corresponding standard feature vector, the cosine similarity between the feature vector to be tested and each standard feature vector is calculated. The cosine similarity values ​​are averaged to obtain the final similarity. The final similarity is compared with a set threshold. When the final similarity is greater than or equal to the set threshold, an evaluation conclusion of qualified imaging quality is generated. When the final similarity is less than the set threshold, an evaluation conclusion of unqualified imaging quality is generated.

6. The palm vein camera evaluation method based on deep neural networks according to claim 1, characterized in that, Step six specifically includes: Image quality detection is performed on the image under test. The pixel gray-level variance of the image under test is calculated and compared with a preset variance threshold. When the pixel gray-level variance is less than the preset variance threshold, an image blur anomaly label is generated. Perform palm region integrity detection on the image to be tested, count the number of key points detected, and generate a palm region missing anomaly marker when the number of key points is less than a preset key point number threshold. Perform loading integrity checks on the improved MobileFace model by performing file hash verification on the improved MobileFace model file and comparing the verification result with a preset hash value. If the comparison is inconsistent, generate a model loading anomaly flag. When calling the palm print and palm vein imaging camera, a camera connection detection is performed. This involves sending a handshake command to the communication port of the palm print and palm vein imaging camera and monitoring the return signal within a preset time window. If no return signal is detected, a camera connection error flag is generated. When any exception identifier is generated, the corresponding exception type is written into the exception record unit, and an exception prompt message is generated in the evaluation interface.

Images