Liveness attendance anti-counterfeiting method and system based on hierarchical fusion of veins and facial features

Abstract

The application discloses a living body attendance anti-fake method and system based on vein and facial feature hierarchical fusion, relates to the field of living body anti-fake identification, collects multiple palm images, optimizes the images through a vein texture enhancement algorithm, acquires a bifurcation point set through an improved U-Net network, completes vein living body inspection and feature matching to obtain a vein user hash; generates random instructions and collects multiple facial images, completes facial living body inspection and feature matching, and the facial user hash needs to be consistent with the vein user hash; dynamically determines a confidence weight ratio, screens optimal palm and facial images for consistency inspection, and integrates the user hash, a time period and the confidence weight ratio into an attendance record to upload a database, thereby guaranteeing the authenticity and security of the attendance result.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of living body anti-counterfeiting identification, in particular to a living body attendance anti-counterfeiting method and system based on hierarchical fusion of vein and facial features. BACKGROUND

[0002] In the attendance management of enterprises and institutions, traditional passwords, magnetic cards and other means are easy to be lost and used by others, and single-modal biometric attendance technology also faces the risk of forgery, which is difficult to meet the high security requirements. Therefore, it is crucial to develop accurate living body identification and efficient multi-modal fusion attendance anti-counterfeiting technology to improve the reliability of the system.

[0003] The existing living body attendance anti-counterfeiting technology has the following defects:

[0004] Vein detection only relies on static texture, lacks physiological dynamic feature verification, facial detection uses fixed action verification, lacks random instruction dynamic micro-expression verification, and is easy to be bypassed by counterfeit means;

[0005] The priority of vein and facial verification is not divided, and fixed weight fusion is used, which cannot adapt to the fluctuation of feature quality, and the verification accuracy is low;

[0006] The consistency of vein and facial features is not verified, two single-modal features forged at different times cannot be identified, leading to identity misjudgment. SUMMARY

[0007] The present application proposes a living body attendance anti-counterfeiting method and system based on hierarchical fusion of vein and facial features to effectively improve the reliability of living body identification, improve the accuracy and efficiency of attendance verification, reduce the risk of identity misjudgment caused by different single-modal feature forgery, and ensure the authenticity and security of attendance results.

[0008] The technical solution for achieving the present application is as follows:

[0009] The living body attendance anti-counterfeiting method based on hierarchical fusion of vein and facial features comprises the following steps:

[0010] Collecting the palm part image of each frame and optimizing to obtain the refined palm part image of each frame, using an improved U-Net network to obtain the bifurcation point set of each frame and referring to the bifurcation point displacement to perform vein living body verification, and after the vein living body verification passes, extracting the vein features of each frame and encrypting the vein features of each frame and performing vein matching in the database, wherein the total number of frames is

[0011] After the vein matching passes, waiting for the first frame of face collection, generating a random instruction, collecting The face map of the frame is subjected to face liveness detection based on micro-expression changes, and after the face liveness detection, local binary and joint coding extraction are used The face features of the frame are encrypted The face features of the frame are subjected to face matching in the database

[0012] After the face matching, the user hash and the time period are confirmed, and the attendance record of the same user hash in the historical time period in the database is called, the confidence weight ratio of the vein features and the face features is dynamically confirmed based on an attention time sequence network, and the frame evaluation network is used in combination The vein features of the frame and The face features of the frame are respectively screened to determine the optimal palm map and the optimal face map, consistency detection is performed according to the spectral features of the optimal palm map and the optimal face map in the same wave band and in combination with the confidence weight ratio, and after the consistency detection, the user hash, the time period and the confidence weight ratio are integrated into the attendance record and uploaded to the database.

[0013] Further, the vein texture enhancement algorithm includes the following steps:

[0014] The palm map is subjected to gray scale normalization to obtain a standard palm map;

[0015] The standard palm map is subjected to convolution operation by using a Gaussian filter kernel and is restored by multiplying the maximum gray scale value to obtain a filtered palm map;

[0016] The filtered palm map is uniformly divided into non-overlapping pixel blocks, the gray scale histogram of each pixel block is counted, the cumulative pixel number of each gray scale value in the pixel block is calculated and is divided by the total pixel number of the pixel block to obtain the gray scale cumulative distribution value of each gray scale value, each gray scale value in the pixel block is replaced by the product of the corresponding gray scale cumulative distribution value and 255 to obtain an equalized pixel block and is restored to an equalized palm map;

[0017] The Laplace operator is used to perform convolution operation on each pixel in the equalized palm map to obtain the pixel gradient value of each pixel and the absolute value of the pixel gradient value is used to replace the corresponding pixel to generate an enhanced palm map, the gray scale mean value And the gray scale variance of the enhanced palm map are counted The gray scale threshold value is accurately determined according to the Leite The gray scale value greater than or equal to the gray scale threshold value is replaced by 0.1 times the difference between the gray scale threshold value and the gray scale threshold value The gray scale value less than the gray scale threshold value is replaced by the gray scale threshold value minus the gray scale threshold value 0.1 times of the difference, generate the refined palm part graph.

[0018] Further, the vein living body test includes the following steps: The refined palm part graph of each frame is input into the improved U-Net network respectively to obtain the vein region probability graph of each frame. The bifurcation point set in the refined palm part graph of each frame includes the following steps:

[0019] The normalized refined palm part graph obtains the standard refined palm part graph, which is sequentially down-sampled by 4 layers of down-sampling layers to generate a set of multi-scale feature maps;

[0020] The set of multi-scale feature maps is sequentially up-sampled by 4 layers of up-sampling layers and connected by jumping to obtain the vein region probability graph;

[0021] The vein region probability graph is binarized according to the probability threshold to obtain the region segmentation mask graph;

[0022] The gradient amplitude of each pixel in the vein region of the region segmentation mask graph is calculated by using the Sobel operator The average gradient amplitude and the gradient variance of the vein region in the region segmentation mask graph of each frame are counted The gradient threshold of the region segmentation mask graph of each frame is obtained by superimposing the average gradient amplitude by 1.5 times the gradient variance The pixels in the vein region whose gradient amplitude is greater than or equal to the gradient threshold are integrated into the potential point pixel set of each frame.

[0023] A 3x3 neighborhood is constructed with each potential point pixel in the potential point pixel set of each frame as the center, the number of connected branches in the neighborhood is counted, and the potential point pixel whose connected branch number is greater than or equal to the branch number threshold is taken as the bifurcation point. The pixel coordinates of all bifurcation points are recorded and integrated to obtain the bifurcation point set of each frame.

[0024] Further, the vein living body test includes the following steps:

[0025] The bifurcation point coordinates set of each frame is coordinate-normalized with the geometric center of the refined palm part graph of each frame as the origin, the neighborhood average gray of the 3x3 local neighborhood of each bifurcation point in the bifurcation point coordinate set of each frame is calculated, and the bifurcation point relative coordinates and the neighborhood average gray of each bifurcation point of each frame are spliced to obtain the corresponding bifurcation point feature.

[0026] With each bifurcation point and the corresponding bifurcation point feature in the first frame of vein collection as the reference, the FLANN matcher is used to determine the corresponding same bifurcation point in the bifurcation point coordinate set of each subsequent frame according to the feature distance, and the bifurcation point feature of each bifurcation point in the bifurcation point coordinate set of each frame is calculated The branch point relative coordinates of each same branch point in the frame are spliced into a relative coordinate sequence of each branch point;

[0027] The mean value of the relative coordinate sequence of each branch point is subjected to fast Fourier transform and the main peak frequency is extracted, the frequency proportion falling in the human pulse frequency interval is counted and compared with the frequency proportion threshold value, and only when the frequency proportion is greater than or equal to the frequency proportion threshold value, it is determined that the vein liveness test passes.

[0028] Further, after the vein liveness test passes, the refined palm part graph and the branch point set of each frame are obtained, the Euclidean distance and the included angle between any two branch points in the branch point set of each frame are calculated and integrated into the distance vector and the included angle vector of each frame, the branch points and the vein segments are taken as nodes and edges respectively, the number of nodes and the number of edges of each frame are counted, the energy, the contrast, the correlation and the entropy of the gray level co-occurrence matrix in the 5x5 large neighborhood of each branch point of each frame are calculated and spliced into the texture feature vector of each frame, and the distance vector, the included angle vector, the number of nodes, the number of edges and the texture feature vector of each frame are spliced to obtain the vein feature of each frame.

[0029] Further, when performing vein matching, the vein feature of each frame is one-way mapped into a vein hash through a biohashing algorithm, and all the recorded vein hashes in the database are retrieved, the average cosine similarity between the vein hash of each frame and each recorded vein hash is calculated and taken as the vein matching degree, only when there is a recorded vein hash with a vein matching degree greater than or equal to the matching degree threshold value in the database, it is determined that the vein matching passes, and the recorded vein hash corresponding to the maximum vein matching degree is anchored and the vein user hash is confirmed.

[0030] Further, the face liveness test comprises the following steps:

[0031] A random instruction is generated, and a face graph of each frame is collected, The face graph components of each wave band channel in the face graph of each frame are averaged to obtain The average face graph of each frame is normalized in gray scale to obtain The standard face graph of each frame;

[0032] A Haar feature classifier is used to mark four core regions in the standard face graph of each frame to obtain a core region set of each frame, an ORB feature point inspection algorithm is used to extract feature points in each core region of each frame, and a FLANN matcher is used to establish a pixel matching relationship between the same core regions of adjacent two frames;

[0033] An optical flow method is used to calculate the face displacement vectors of corresponding pixels in each same core region in adjacent two frames, and the face displacement vector sets of adjacent two frames are integrated to obtain The face displacement vector set of each frame group, is the number of adjacent frame groups;​

[0034] Calculate the standard motion vectors in the database based on random instructions. The average similarity between all facial displacement vectors in the set of facial displacement vectors and the standard motion vectors is used as the instruction matching degree. Facial liveness detection is only considered to be passed when the instruction matching degree is greater than or equal to the instruction matching degree threshold.

[0035] Furthermore, local binary and joint encoding are used to extract... The facial features of the frame include the following steps:

[0036] Get The four core regions in the standard facial image of the frame are cropped to obtain the corresponding standard facial images of the same frame. The core region sub-map of the frame;

[0037] Local binary encoding is performed on each core region sub-image of each frame. A 3×3 neighborhood is constructed with each pixel in the core region sub-image as the center. The normalized gray values ​​of the 8 pixels in the neighborhood are compared with those of the center pixel. The encoding value of the pixels that are greater than or equal to the normalized gray value of the center pixel is recorded as 1, and the encoding value of the pixels that are less than the normalized gray value of the center pixel is recorded as 0. The 8-bit encoding values ​​are concatenated in sequence to obtain the binary encoding of the center pixel. The binary encoding map of each core region sub-image of each frame is generated and integrated into the binary encoding map set of each frame.

[0038] Each binary coded image in each frame is evenly divided into non-overlapping sub-blocks. The frequency of occurrence of 1 and 0 in each sub-block is counted to obtain the sub-block histogram. The sub-block histograms are arranged according to the sub-block positions to obtain the binary coded histogram corresponding to each binary coded image in each frame.

[0039] The binary encoded histograms corresponding to the four core regions of each frame are sequentially concatenated to obtain the encoded concatenated histogram of each frame. Energy, contrast, and entropy are then calculated. The encoded concatenated histogram of each frame is flattened and concatenated with the corresponding energy, contrast, and entropy values, and then normalized to obtain... Facial features of the frame.

[0040] Furthermore, during face matching, the facial features of each frame are mapped to a facial hash using a biometric hashing algorithm, and all recorded facial hashes are retrieved from the database for calculation. The face hash of a frame is compared with the mean cosine similarity of each recorded face hash as the face matching degree. The face matching is considered successful only when there is a recorded face hash in the database with a face matching degree greater than or equal to the matching degree threshold, and the face user hash of the recorded face hash corresponding to the maximum face matching degree is the same as the vein user hash.

[0041] Further, the confidence weight ratio of the vein feature and the face feature is dynamically confirmed based on the attention time sequence network, including the following steps:

[0042] Confirming the user hash and the time period and calling the historical attendance records of the same user hash in the most adjacent historical time periods in the database, extracting the historical attendance time and the historical confidence weight ratio in the historical attendance records of the historical time periods, as reference numbers;

[0043] The historical interval of the historical attendance time of the historical time periods from the attendance time of the current attendance is spliced to obtain a historical interval vector, and the historical interval vector is converted into a time length encoding vector through sine and cosine coding;

[0044] The historical confidence weight ratio of the historical time periods is spliced in sequence to obtain a historical confidence weight ratio vector, and the historical confidence weight ratio vector is spliced with the time length encoding vector of dimensions to obtain a value matrix, the value matrix is transformed into a query matrix and a key matrix respectively by linear modulation, and a dynamic weight vector of dimensions is obtained through a dynamic attention mechanism, wherein when the dynamic attention mechanism calculates the attention score of each dimension in the query vector and the key vector of each dimension in the query matrix and the key matrix, the attention score of each dimension is subtracted by the product of the decay coefficient and the historical interval of the same dimension in the historical interval vector, to generate the dynamic attention score of each dimension and obtain the dynamic weight vector through splicing and normalization;

[0045] The dynamic weight of each dimension in the dynamic weight vector is multiplied by the value vector of the same dimension in the value matrix and summed to obtain a context representation and perform time sequence reasoning through a fully connected layer to obtain the confidence weight ratio of the current attendance.

[0046] Further, the consistency test includes the following steps:

[0047] The vein feature of the frame and the face feature of the frame are respectively input into the frame evaluation network through linear modulation in sequence to obtain the palm image score of the frame and the face image score of the frame, and the optimal palm image with the highest palm image score and the optimal face image with the highest face image score are anchored, wherein the frame evaluation network is realized based on a multilayer perceptron;

[0048] Extract the optimal facial image component from the band channel corresponding to the infrared spectral band in the optimal facial image. Based on the region segmentation mask map corresponding to the frame where the optimal palm image is located, identify the non-vein region of the optimal palm image and calculate the mean gray value, variance, and entropy of the palm skin color in the non-vein region. Then, stitch them together to obtain the palm skin color spectral features. Based on the cheeks concentrated in the core region corresponding to the frame where the optimal facial image is located, identify the cheek region component in the optimal facial image component and calculate the mean gray value, variance, and entropy of the facial skin color in the cheek region component. Then, stitch them together to obtain the facial skin color spectral features.

[0049] Based on the confidence weight ratio, the palm image score of the best palm image and the face image score of the best face image are weighted, summed, and normalized to obtain the comprehensive image score. The cosine similarity between the spectral features of the palm skin color and the spectral features of the face skin color is calculated and used as the spectral similarity score. The comprehensive image score and the spectral similarity score are weighted and summed according to the reference weight to obtain the consistency score. The consistency test is considered to have passed only when the consistency score is greater than or equal to the consistency score threshold.

[0050] The live attendance anti-counterfeiting system based on the hierarchical fusion of vein and facial features includes a data acquisition module, a vein matching module, a facial verification module, a joint verification module, and a storage module.

[0051] Data acquisition module The palm image of the frame is taken, and the process continues until the first frame of the face is captured. Random instructions are then generated to capture the image. A frame of facial images;

[0052] The vein matching module is optimized using a vein texture enhancement algorithm. The refined palm image of each frame is obtained using an improved U-Net network. A set of bifurcation points is then used to perform vein liveness detection based on the bifurcation point displacements. Vein features are extracted and encrypted for vein matching in the database only when the vein liveness detection is successful. The vein user hash is sent only when the vein matching is successful. Total number of frames;

[0053] The facial verification module starts when it receives the vein user hash and analyzes it. Micro-expression changes in facial images of frames are used for facial liveness detection. Only after passing the facial liveness detection is local binary and joint encoding used to extract... The facial features of the frame are encrypted for face matching in the database, and the user hash and time period are only confirmed when the face match is successful;

[0054] The joint verification module references attendance records of the same user hash from the same historical period in the database, and dynamically confirms the confidence weight ratio of vein features and facial features based on an attention-based temporal network. Frame vein features and The face features of the frame are linearly converted respectively and then input into the frame evaluation network in sequence to screen the optimal palm feature map and the optimal face feature map, the spectral features of the optimal palm feature map and the optimal face feature map in the infrared spectral band are extracted, and consistency test is performed by combining the confidence weight ratio, and only when the consistency test passes, it is determined that the attendance is successful, the user hash, time period and confidence weight ratio of this attendance are integrated into the attendance record and sent to the storage module.

[0055] The storage module stores a database and adds the received attendance record in the database.

[0056] Compared with the prior art, the relative coordinate sequence of the vein bifurcation point is analyzed in the frequency domain to match the human pulse characteristics, and the displacement vector of the core area of the face is calculated according to the random instruction to verify the dynamic micro-expression, effectively solving the problem that the vein depends on the static texture and the face depends on the fixed action and is easy to be forged, and significantly improving the reliability of the living body identification; the layered process of first vein matching and then face matching is adopted, and the face matching needs to be consistent with the user hash of the vein matching, and the double-modal confidence weight ratio is dynamically adjusted by combining the user historical attendance data, solving the problems of process redundancy caused by no verification priority and fixed weight unable to adapt to the quality fluctuation of the characteristics, and greatly improving the efficiency and precision of the attendance verification; by screening the optimal palm and face images, extracting the skin color spectral features of the two, and calculating the consistency score, the identity misjudgment caused by the single-modal characteristics of different time forgery is avoided, and the authenticity and security of the attendance result are further ensured. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 It is a flow chart of the living body attendance anti-forgery method based on layered fusion of vein and face features.

[0058] Figure 2 It is an execution flow chart of the improved U-Net network.

[0059] Figure 3 It is a face matching flow chart.

[0060] Figure 4 It is a schematic diagram of the living body attendance anti-forgery system based on layered fusion of vein and face features. DETAILED DESCRIPTION

[0061] The application will be further described in detail below in combination with the drawings and embodiments.

[0062] Embodiment 1

[0063] As shown in the drawings, Figure 1 one specific embodiment of the application discloses a living body attendance anti-forgery method based on layered fusion of vein and face features, comprising the following steps:

[0064] acquiring the vein image of the user through an infrared vein camera The palm part image of each frame is obtained by using a vein texture enhancement algorithm to optimize the gray difference between the veins and the tissues in the palm part image. The refined palm part image of each frame is obtained by using an improved U-Net network to obtain the bifurcation point set in the refined palm part image of each frame, and the bifurcation point displacement of the bifurcation point set of each frame is used to perform a vein living body test. After the vein living body test passes, the vein features of each frame are extracted, encrypted The vein features of each frame are encrypted and matched in the database, The total number of frames is obtained.

[0065] After the vein matching passes, the first frame of the face is collected, a random instruction is generated, and the multispectral imaging device is used to start collecting The face image of each frame is obtained, and a face living body test is performed by analyzing micro-expression changes. After the face living body test passes, the local binary and joint coding are used to extract The face features of each frame are encrypted The face features of each frame are encrypted and matched in the database. The first frame of the face collection and the first frame of the vein collection of the infrared vein camera are always delayed by a frame number. The delay frame number is pre-set to ensure that the vein matching is completed.

[0066] After the face matching passes, the user hash and the time period are confirmed, the attendance records of the same user hash in the historical time period in the database are referred to, the confidence weight ratio of the vein features and the face features is dynamically confirmed based on an attention time sequence network, and The vein features of each frame and The vein features of each frame and

[0067] Further, the vein texture enhancement algorithm includes the following steps:

[0068] The original gray value of each pixel in the palm part image is obtained and divided by the maximum gray value 255, and the gray of each pixel is normalized to obtain a standard palm part image to ensure the calculation stability of the Gaussian filter.

[0069] A Gaussian filter kernel with a size of 3x3 and a kernel standard deviation of 0.8 is used to perform convolution operation on the standard palm part image and multiplied by 255 to restore to obtain a filtered palm part image. While smoothing the random noise area, the vein structure is avoided from being excessively blurred, and the interference of noise on the gray difference between the veins and the tissues is reduced.

[0070] The filtered palm part image is evenly divided according to a preset total number of pixel blocks, ensuring that the pixel blocks are non-overlapping and completely cover the filtered palm part image, the number of pixels of each gray value in the gray interval of 0 to 255 is counted for each pixel block, a gray histogram of each pixel block is obtained, the cumulative number of pixels of each gray value in the gray interval of 0 to 255 is calculated, wherein the cumulative number of pixels of the gray value is equal to the sum of the number of pixels of the gray value 0 to the gray value , and the cumulative number of pixels of each gray value is divided by the total number of pixels of the pixel block to obtain a gray cumulative distribution value of each gray value, which must be in the interval of 0 to 1, for each pixel block, each gray value in the pixel block is replaced by the product of the corresponding gray cumulative distribution value and 255, adaptive histogram equalization of the pixel block is realized, the gray contrast in the pixel block is enhanced, the gray difference between the vein texture and the surrounding tissue is clarified, an equalized pixel block is obtained, and an equalized palm part image is restored according to the pixel block distribution.

[0071] A Laplacian operator with a size of 3x3 is used to perform convolution operation on each pixel in the equalized palm part image to obtain a pixel gradient value of each pixel and replace the corresponding pixel with the absolute value of the pixel gradient value, to strengthen the gray change of the vein edge and generate an enhanced palm part image with clearer vein texture contour, the gray mean value and the gray variance of the enhanced palm part image are counted, the gray mean value is superimposed with 3 times the gray variance to obtain a gray threshold value , for the gray value of each pixel in the enhanced palm part image, if the gray value is greater than or equal to the gray threshold value , the gray threshold value is added by 0.1 times the difference between the gray threshold value and the gray threshold value as a new gray value and is replaced correspondingly, if the gray value is less than the gray threshold value , the gray threshold value is subtracted by 0.1 times the difference between the gray threshold value and the gray value as a new gray value and is replaced correspondingly, to generate a refined palm part image, quantitatively reduce the gray fluctuation of the non-vein area, further enlarge the gray gap between the vein and the tissue, and reduce the interference of pseudo texture on vein feature extraction.

[0072] As shown in Figure 2 , further, the refined palm part images of the frames are respectively input into the improved U-Net network to obtain the bifurcation point set in the refined palm part image of the frame, including the following steps:

[0073] Normalizing the refined palm image of each frame yields a standard refined palm image. Through four downsampling layers, sequential downsampling is performed to obtain first-level, second-level, third-level, and fourth-level feature maps, generating a multi-scale feature map set. The downsampling layer includes two 3×3 convolutional kernels with a stride of 1 and a pooling operation with a size of 2×2 and a stride of 2. The low-level feature maps focus on details such as vein edges and texture, while the high-level feature maps focus on the semantic information of the vein region.

[0074] The vein region probability map for each frame is obtained by successively upsampling the multi-scale feature map set through four upsampling layers and skip connections. The vein region probability map accurately reflects the vein attributes of each pixel in the refined palm image of each frame, providing a quantitative basis for the segmentation mask. The upsampling layer includes a transposed convolution kernel with a size of 3×3, a stride of 2, and padding of 1. When the dimension of the high-level feature map after one upsampling is the same as the dimension of the lower-level feature map in the multi-scale feature map set, the upsampling result of the high-level feature map and the lower-level feature map can be concatenated by skip connections. The dimension of the vein region probability map is the same as that of the refined palm image.

[0075] Based on the preset probability threshold, the probability map of the vein region in each frame is binarized. The pixel value of the pixel with the vein probability greater than or equal to the probability threshold is set to 1, and the pixel value of the pixel with the vein probability less than the probability threshold is set to 0. This yields the region segmentation mask map for each frame, thus achieving the segmentation and marking of the vein region with a pixel value of 1 and the non-vein region with a pixel value of 0.

[0076] The Sobel operator with a size of 3×3 is used to calculate the region segmentation mask image of each frame for each pixel within the vein region. Directional gradient value and Directional gradient value The gradient magnitude of each pixel is obtained by taking the square root of the sum of squares. Calculate the average gradient magnitude of the vein region in the region segmentation mask image of each frame. and gradient variance The average gradient magnitude Superimposed gradient variance of 1.5 times Obtain the gradient threshold of the region segmentation mask map for each frame. Gradient amplitude within the vein region Greater than or equal to the gradient threshold The pixels are labeled as potential point pixels, and the potential point pixel set of each frame is obtained;

[0077] For each potential point pixel in the potential point pixel set of each frame, a 3x3 neighborhood is constructed by covering the surrounding 8 pixels in the region segmentation mask map of each frame centered on the potential point pixel, and the number of connected branches in the neighborhood is counted, wherein the connected branch is defined as an independent region composed of adjacent pixels with a pixel value of 1 in the neighborhood, and adjacent refers to 8 directions of up, down, left, right and diagonal, if the number of connected branches in the neighborhood of the potential point pixel is greater than or equal to the branch number threshold, the potential point pixel is determined to be a bifurcation point, the pixel coordinates of the bifurcation point are recorded, and the bifurcation point set of each frame is obtained by traversing the potential point pixel set of each frame. According to the connectivity analysis, false bifurcation points such as isolated edge pixels and vein endpoints can be excluded, and the accurate positioning of the bifurcation points of each frame is realized.

[0078] Further, the vein living body test comprises the following steps:

[0079] The bifurcation point coordinate set of each frame is subjected to coordinate relativization, all bifurcation point coordinates in the bifurcation point coordinate set of the same frame are converted into bifurcation point relative coordinates with the geometric center of the refined palm part graph of each frame as the origin, and a 3x3 local neighborhood in the refined palm part graph of the same frame is determined with each bifurcation point in the bifurcation point coordinate set of each frame as the center and the neighborhood average gray value of the local neighborhood is calculated. The bifurcation point relative coordinates and the neighborhood average gray value of each bifurcation point of each frame are spliced to obtain the bifurcation point features of each bifurcation point of each frame.

[0080] Since the palm part graph is always collected by the infrared vein camera corresponding to the palm part of the same user, the total number of bifurcation points in the bifurcation point coordinate set of each frame is the same and has a one-to-one correspondence. Each bifurcation point and the corresponding bifurcation point feature in the first frame of vein collection are sequentially taken as the reference point and the corresponding reference point feature, and the same bifurcation point as the reference point is sequentially determined in the bifurcation point coordinate set of each subsequent frame by using the FLANN matcher according to the feature distance, and the bifurcation point feature of each same bifurcation point in the frame is spliced to obtain the bifurcation point feature sequence of each same bifurcation point. The bifurcation point relative coordinates of each same bifurcation point in the frame are spliced according to the time sequence to obtain the relative coordinate sequence of each bifurcation point.

[0081] The average relative coordinate sequence is obtained by taking the mean value of the relative coordinate sequence of each bifurcation point, and the fast Fourier transform is performed to generate the frequency domain power spectrum and extract the main peak frequency. The frequency proportion falling in the human pulse frequency interval is counted, wherein the human pulse frequency interval is 1Hz to 1.67Hz. The frequency proportion of each bifurcation point is compared with the preset frequency proportion threshold.

[0082] If the frequency proportion is greater than or equal to the frequency proportion threshold, it is determined that the vein living body test is passed, and if the frequency proportion is less than the frequency proportion threshold, it is determined that the vein living body test fails, and the attendance is immediately stopped. Based on the frequency domain analysis of the point displacement of the bifurcation point, the periodicity of the pulse wave of the vein is effectively identified to distinguish the living body.

[0083] Furthermore, after the vein liveness test is passed, a detailed palm image and a set of bifurcation points for each frame are obtained. The Euclidean distance and angle between any two bifurcation points in the set of bifurcation points for each frame are calculated and sequentially integrated to obtain the distance vector and angle vector for each frame. The bifurcation points of the set of bifurcation points for each frame are used as nodes, and the vein segments between bifurcation points with adjacent coordinates are used as edges to construct the vein topology graph for each frame. The number of nodes and edges in each frame are counted. Taking each bifurcation point in the set of bifurcation points for each frame as the center and taking a 5×5 large neighborhood, the energy, contrast, correlation, and entropy of the gray-level co-occurrence matrix in the large neighborhood of each bifurcation point are calculated and sequentially concatenated to the texture feature vector of each frame. The distance vector, angle vector, number of nodes, number of edges, and texture feature vector of each frame are concatenated again to obtain the vein features of each frame.

[0084] Furthermore, during vein matching, the vein features of each frame are unidirectionally mapped to vein hashes using a bio-hashing algorithm, and all recorded vein hashes are retrieved from the database for calculation. The vein hash of a frame is compared with the mean cosine similarity of each recorded vein hash as the vein matching degree. When there is no recorded vein hash in the database with a vein matching degree greater than or equal to the matching degree threshold, the vein matching is determined to be unsuccessful and attendance is immediately stopped. When there is a recorded vein hash in the database with a vein matching degree greater than or equal to the matching degree threshold, the recorded vein hash corresponding to the maximum vein matching degree is anchored and the vein user hash is confirmed based on the database. The recorded vein hash is obtained by mapping the recorded vein features extracted by the user's palm image based on the vein texture enhancement algorithm and the improved U-Net network through the bio-hashing algorithm. The user hash is obtained by mapping the user's entered employee ID through the ordinary hash algorithm. The ordinary hash algorithm outputs completely different results when there are slight differences in the input, while the bio-hashing algorithm has tolerance for changes in biological features. When there are slight differences in the input, the outputs are different but have extremely high similarity.

[0085] Furthermore, facial biopsy includes the following steps:

[0086] Random commands are generated, including blinking, raising the corners of the mouth, and turning the head. The process continues until the first frame of the face is captured, and then multispectral imaging equipment is used to continuously capture images starting from that first frame. The facial image of each frame is obtained by averaging the facial image components of each band channel in the facial image of each frame. The average facial image of the frame is divided by the maximum gray value of 255 and normalized to obtain the result. A standard facial image of a frame;

[0087] A Haar feature classifier is used to label all core regions in the standard facial image of each frame, resulting in a core region set for each frame. The core regions are the areas in the standard facial image where micro-expressions are most likely to occur. The core regions include the area around the eyes, nose, mouth, and cheeks. That is, there are 4 core regions in the standard facial image of each frame. The ORB feature point detection algorithm is used to extract feature points in each core region of each frame. The FLANN matcher is combined to establish the pixel matching relationship of the same core regions in adjacent frames. The Haar feature classifier and the ORB feature point detection algorithm are existing technologies and are not the focus of the technical solution of this application, so they will not be described in detail.

[0088] The optical flow method is used to calculate the facial displacement vectors of corresponding pixels within the same core region in two adjacent frames, and these vectors are integrated into a set of facial displacement vectors for the two adjacent frames, resulting in a total of... Group of facial displacement vector sets, The number of adjacent frame groups is equal to the total number of frames. Subtract 1, and the number of vectors in the facial displacement vector set of two adjacent frames is 4. The facial displacement vector is used to quantify the motion trajectory and intensity of the core region in two adjacent frames, avoiding the error of subjective judgment by the human eye.

[0089] Calculate the standard motion vectors in the database based on random instructions. The average similarity between all facial displacement vectors in the set of facial displacement vectors and the standard motion vectors is used as the instruction matching degree. The instruction matching degree is compared with the instruction matching degree threshold. If the instruction matching degree is greater than or equal to the instruction matching degree threshold, the facial liveness detection is deemed to have passed. If the instruction matching degree is less than the instruction matching degree threshold, the facial liveness detection is deemed to have failed, and attendance is immediately stopped.

[0090] Furthermore, local binary and joint encoding are used to extract... The facial features of the frame include the following steps:

[0091] In the process of obtaining liveness testing All core regions in the standard facial image of a frame are cropped based on the core regions of each frame to obtain a core region sub-image set for each frame. The number of core region sub-images in the core region sub-image set for each frame is also 4.

[0092] Local binary encoding is performed on each core region sub-image of each frame. A 3×3 neighborhood is constructed with each pixel in the core region sub-image as the center. The normalized grayscale value of the 8 pixels in the neighborhood is compared with that of the center pixel. If the normalized grayscale value of the neighboring pixels is greater than or equal to that of the center pixel, the corresponding pixel's encoding value is recorded as 1. If the normalized grayscale value of the neighboring pixels is less than that of the center pixel, the corresponding pixel's encoding value is recorded as 0. Starting from the top left pixel, 8-bit encoding values ​​are concatenated in a clockwise order to obtain the binary encoding of the center pixel. This generates the binary encoded image corresponding to each core region sub-image of each frame and integrates them into a binary encoded image set for each frame. The number of binary encoded images in the binary encoded image set of each frame is equal to the number of core regions in the standard facial image. Local texture features of the core regions are extracted through local binary encoding to enhance the differentiated features of facial textures of different users.

[0093] Each binary coded image in each frame is evenly divided according to the preset total number of sub-blocks, ensuring that the sub-blocks do not overlap and completely cover the corresponding binary coded image. The frequency of occurrence of 1 and 0 in each sub-block is counted to generate a sub-block histogram. Based on the arrangement position of the sub-blocks in the binary coded image, the binary coded histogram corresponding to each binary coded image in each frame is restored, realizing the transformation of local texture features into statistical features, reducing the impact of single pixel noise on feature stability, and obtaining the binary coded histograms corresponding to the four core regions of each frame to obtain the binary coded histogram set of each frame.

[0094] The binary encoded histograms corresponding to the four core regions of each frame are concatenated in the order of periorbital, nose, mouth, and cheek to obtain the encoded concatenated histogram of each frame. Energy, contrast, and entropy are then calculated. The encoded concatenated histogram of each frame is flattened and concatenated with the corresponding energy, contrast, and entropy values ​​to obtain the facial encoded vector of each frame. The face encoding vectors of the frames are obtained by Z-score normalization. Facial features of the frame.

[0095] like Figure 3 As shown, further, during face matching, the facial features of each frame are unidirectionally mapped to a face hash using a biometric hashing algorithm, and all recorded face hashes are retrieved from the database for calculation. The facial hash of a frame is compared with the mean cosine similarity of each recorded facial hash as the facial matching degree. When there is no recorded facial hash in the database with a facial matching degree greater than or equal to the matching degree threshold, the facial matching is deemed to have failed and attendance is immediately stopped. When there is a recorded facial hash in the database with a facial matching degree greater than or equal to the matching degree threshold, the recorded facial hash corresponding to the maximum facial matching degree is anchored and the facial user hash is confirmed by searching the database. It is then determined whether the vein user hash and the facial user hash are the same. If they are different, the facial matching is deemed to have failed and attendance is immediately stopped. If they are the same, the facial matching is deemed to have passed. The recorded facial hash is obtained by mapping the recorded facial features extracted from the user's facial image based on local binary and joint encoding through a bio-hashing algorithm.

[0096] Furthermore, the confidence weight ratio between vein features and facial features is dynamically determined based on an attention-based temporal network, including the following steps:

[0097] Confirm the user hash and the time period of this attendance session, and use this as a keyword to search the database. Then, retrieve the nearest match for the same user hash from the database. Extract historical attendance records from the same historical period. The historical attendance times and historical confidence weights in historical attendance records from the same historical period, among which... For reference, the confidence weight ratio is equal to the ratio of the confidence weight of veins to the confidence weight of face;

[0098] Will The historical intervals between the historical attendance times of the same historical period and the attendance time of the current attendance are pieced together to obtain... The historical interval vector is processed using sine and cosine coding in the Transformer architecture to convert each historical interval into a corresponding duration code to capture the non-uniformity of interval duration, and then integrated to generate... A time-encoded vector of dimensions;

[0099] Will The historical confidence weights of each historical period are concatenated sequentially to obtain the following: Historical confidence weights of the dimension are compared with the vector and The time-encoded vectors of dimension 1 are concatenated row-wise to obtain a value matrix. This value matrix is ​​then transformed into a query matrix and a key matrix using two different linear modulations. Finally, a dynamic attention mechanism is employed to obtain... The dynamic weight vector is defined as follows: The dynamic attention mechanism introduces a learnable decay coefficient into the attention mechanism calculation. When the query vector and key vector of each dimension in the query matrix and key matrix calculate the attention score for each dimension according to the existing attention mechanism, the attention score for each dimension is subtracted by the product of the decay coefficient and the historical interval of the same dimension in the historical interval vector, generating the dynamic attention score for each dimension, which is then concatenated. The dynamic attention score vector of dimension is normalized into a dynamic weight vector using the Softmax function;

[0100] The dynamic weights of each dimension in the dynamic weight vector are multiplied by the value vectors of the same dimension in the value matrix and then summed to obtain the context representation. The context representation is dynamically fused based on historical intervals. The historical confidence weight ratio information of each historical period is used to perform temporal reasoning on the context representation using a fully connected layer, and the confidence weight ratio of this attendance is mapped to obtain the confidence weight ratio of this attendance.

[0101] Furthermore, the consistency check includes the following steps:

[0102] Will Frame vein features and The facial features of each frame are linearly modulated to adjust their dimensions to match the input dimensions of the frame evaluation network, generating... Frame vein modulation characteristics and The facial modulation features of the frames are sequentially input into the frame evaluation network to obtain... Palm image rating of the frame and The frame evaluation system scores facial images, anchoring the best palm image with the highest palm image score and the best facial image with the highest facial image score. The frame evaluation network is based on a multilayer perceptron, pre-collecting palm and facial images and evaluating image quality through manual annotation. Vein modulation features for each palm image are obtained using a vein texture enhancement algorithm, an improved U-Net network, and linear dimension tuning. Facial modulation features for each facial image are obtained using local binary and joint encoding and linear dimension tuning. The Adam optimizer is used for training, with the minimum mean square error between the network output score and the corresponding manually annotated score as the loss. It is worth noting that the best palm image and best facial image do not have a frame number correspondence; they are only used to evaluate the image quality. The frame with the best image quality in the palm view of the frame and The frame with the best image quality in the facial images;

[0103] The infrared spectral bands of the infrared vein camera are determined, and the optimal facial image component in the band channel corresponding to the infrared spectral band is extracted from the optimal facial image. Based on the region segmentation mask corresponding to the frame where the optimal palm image is located, the non-vein region of the optimal palm image is identified, that is, the region marked with 0 pixels in the region segmentation mask. The mean gray value, variance, and entropy of the palm skin color in the non-vein region are calculated and concatenated to obtain the palm skin color spectral features. Based on the cheeks in the core region of the frame where the optimal facial image is located, the cheek region component in the optimal facial image component is identified. The mean gray value, variance, and entropy of the facial skin color in the cheek region component are calculated and concatenated to obtain the facial skin color spectral features.

[0104] Based on the confidence weight ratio, 1 is split into vein confidence weight and face confidence weight. Based on the vein confidence weight and face confidence weight, the palm image score of the best palm image and the face image score of the best face image are weighted and summed, and normalized using the Sigmoid function to obtain the comprehensive image score. The cosine similarity between the palm skin color spectral features and the face skin color spectral features is calculated and used as the spectral similarity score. The comprehensive image score and the spectral similarity score are weighted and summed again according to the preset reference weight to obtain the consistency score and compare it with the consistency score threshold.

[0105] If the consistency score is less than the consistency score threshold, the consistency test is deemed to have failed and attendance is immediately stopped. If the consistency score is greater than or equal to the consistency score threshold, the consistency test is deemed to have passed.

[0106] Example 2

[0107] like Figure 4 As shown, a specific embodiment of the present invention discloses a live attendance anti-counterfeiting system based on the layered fusion of vein and facial features, including a data acquisition module, a vein matching module, a facial verification module, a joint verification module, and a storage module.

[0108] The acquisition module acquires data via an infrared vein camera. The palm image of the frame is captured and sent to the vein matching module. After acquiring the first frame of the palm image, the system waits for the first frame of the face image to be acquired, generates random instructions, and uses a multispectral imaging device to acquire the image. The facial image of the frame is sent to the facial verification module;

[0109] The vein matching module uses a vein texture enhancement algorithm to optimize the grayscale difference between veins and tissues in the palm image, resulting in... The refined palm portion image of each frame is obtained using an improved U-Net network, which is used to obtain the set of bifurcation points in the refined palm portion image of each frame. The bifurcation point displacements of the frame's bifurcation point set are used for vein liveness detection. If the vein liveness detection fails, attendance is stopped. If the vein liveness detection passes, the vein features of each frame are extracted and encrypted. The system analyzes the vein features of the frame and performs vein matching in the database. If the vein matching fails, attendance is stopped; if the vein matching succeeds, the vein user hash is sent to the face verification module. Total number of frames;

[0110] The facial verification module starts upon receiving the vein user hash and analyzes it. Micro-expression changes in the facial images of the frames are used for facial liveness detection. If the facial liveness detection fails, attendance is suspended. If the facial liveness detection passes, local binary encoding and joint coding are used to extract the facial liveness signal. Facial features of the frame, encrypted The system detects facial features in the frame and performs facial matching in the database. If facial matching fails, attendance is stopped. If facial matching succeeds, the user hash and time period are confirmed and sent to the joint verification module.

[0111] The joint verification module references attendance records of the same user hash from the same historical period in the database, and dynamically confirms the confidence weight ratio of vein features and facial features based on an attention-based temporal network. Frame vein features and The facial features of each frame are linearly dimension-tuned and then sequentially input into the frame evaluation network. The optimal palm image and the optimal face image are selected and filtered respectively. The spectral features of the optimal palm image and the optimal face image are extracted in the acquisition band of the infrared vein camera and combined with the confidence weight ratio for consistency check. If the consistency check fails, attendance is stopped. If the consistency check passes, attendance is considered successful. The user hash, time period and confidence weight ratio of this attendance are integrated into the attendance record and sent to the storage module.

[0112] The storage module stores the database, receives attendance records, and adds them to the database.

[0113] This invention discloses a live attendance anti-counterfeiting method and system based on layered fusion of vein and facial features. It involves acquiring multiple frames of palm images, optimizing them using a vein texture enhancement algorithm to obtain a refined palm image, extracting a set of bifurcation points using an improved U-Net network, and performing vein liveness detection based on pulse frequency analysis of the relative coordinate sequence of the bifurcation points. After successful vein matching, vein features are extracted and encrypted using bio-hash matching to obtain a vein user hash. Following successful vein matching, a random command is generated to acquire frames of facial images. Facial liveness detection is performed using dynamic displacement verification through micro-expressions. Facial features are extracted and encrypted for matching; a match is only considered successful if the facial user hash matches the vein user hash. The system retrieves the user's historical attendance records for the same period, dynamically determining the confidence weight ratio of vein and facial features using an attention-based temporal network, selecting the optimal palm and facial images using a frame evaluation network, and performing a consistency check by combining their spectral features. After successful verification, the user hash, time period, and confidence weight ratio are integrated into the attendance record and uploaded to the database. This effectively improves the reliability, accuracy, and security of liveness detection and attendance, while reducing the risk of forgery.

[0114] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A live attendance anti-counterfeiting method based on layered fusion of vein and facial features, characterized in that, Includes the following steps: collection The palm image of the frame is optimized using a vein texture enhancement algorithm and combined with an improved U-Net network to obtain... The set of bifurcation points in the frame is used to perform vein liveness detection based on the displacement of bifurcation points in adjacent frames. Once the vein liveness detection passes, the vein is extracted. Frame vein features, encryption Frame vein features and perform vein matching. Total number of frames; After vein matching is successful, wait until the first frame of facial image capture is completed, generate random instructions, and capture the image. The facial image of the frame is analyzed, and facial liveness detection is performed based on micro-expression changes. After the facial liveness detection is passed, local binary encoding and joint coding are used to extract the facial image. Facial features of the frame, encrypted Frame facial features and perform face matching; After facial matching is successful, the user hash and time period are confirmed. Referencing historical attendance records of users with the same hash during the same time period, a confidence weight ratio is determined based on an attention-based temporal network. Finally, a frame evaluation network is used in conjunction with... Frame vein features and The optimal palm image and the optimal face image are determined by the facial feature screening of the frame. The consistency test is carried out based on the spectral characteristics of the optimal palm image and the optimal face image in the same band and combined with the confidence weight ratio. The attendance is considered successful after the consistency test is passed. The consistency check includes: The vein and facial features are linearly dimensionally tuned and sequentially input into the frame evaluation network to obtain palm and facial image scores, anchoring the optimal palm and facial images. The corresponding band components of the optimal facial image are extracted, and the non-vein regions of the optimal palm image are determined by combining the region segmentation mask image, and the skin color spectral features of the palm are calculated. The cheek region components of the optimal facial image are determined, and the skin color spectral features of the face are calculated. The comprehensive image score is calculated based on the confidence weight ratio, and the spectral similarity score is calculated. The comprehensive image score and the spectral similarity score are weighted and summed to obtain the consistency score. When the score meets the preset threshold, the consistency test is considered passed.

2. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 1, characterized in that, The vein texture enhancement algorithm includes: normalizing the grayscale of the palm image to obtain a standard palm image; using Gaussian filtering convolution and restoring to obtain a filtered palm image; dividing the image into blocks, statistically analyzing the cumulative grayscale distribution values, and equalizing them to obtain an equalized palm image; after enhancement by the Laplacian operator, setting the grayscale threshold according to the Leyte criterion, and adjusting the pixel grayscale of the equalized palm image to generate a refined palm image.

3. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 2, characterized in that, The bifurcation point set obtained using the improved U-Net network includes: normalizing and thinning the palm image to obtain a standard thinned palm image; generating a multi-scale feature map set by successively downsampling through 4 downsampling layers; obtaining a vein region probability map by successively upsampling through 4 upsampling layers and skip connections; binarizing the vein region probability map to obtain a region segmentation mask map; calculating the gradient magnitude of the vein region pixels in the region segmentation mask map and filtering to obtain a set of potential point pixels; constructing a neighborhood with the potential point pixels as the center, counting the number of connected branches in the neighborhood, determining the bifurcation points and recording their coordinates, and integrating them to obtain the bifurcation point set.

4. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 2, characterized in that, The vein liveness detection process includes: using the geometric center of the refined palm image as the origin, relativizing the coordinates of the bifurcation point coordinate set, calculating the average gray level of the neighborhood of the bifurcation point, and splicing them together to obtain the bifurcation point features; using each bifurcation point and its features in the first frame of vein acquisition as a benchmark, matching the same bifurcation points in subsequent frames, and splicing them together to obtain the relative coordinate sequence of each bifurcation point; performing a fast Fourier transform on the mean of the relative coordinate sequence and extracting the main peak frequency, and statistically analyzing the proportion of frequencies falling within the human pulse frequency range. When this proportion meets a preset threshold, the vein liveness detection is considered passed.

5. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 2, characterized in that, The extraction of vein features includes: obtaining a refined palm image and a set of bifurcation points; calculating the Euclidean distance and angle between bifurcation points; integrating these to obtain distance and angle vectors; treating bifurcation points as nodes and vein segments as edges; counting the number of nodes and edges; calculating the energy, contrast, correlation, and entropy of the gray-level co-occurrence matrix within the large neighborhood of each bifurcation point; and concatenating these to obtain a texture feature vector; and concatenating the distance vector, angle vector, number of nodes, number of edges, and texture feature vector to obtain the vein features.

6. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 1, characterized in that, Facial liveness detection includes: generating random commands, acquiring facial images, averaging the facial image components for each band channel to obtain an average facial image, and normalizing the grayscale to obtain a standard facial image; using a Haar feature classifier to label core regions to obtain a core region set, extracting feature points using the ORB feature point detection algorithm, and establishing pixel matching relationships for the same core regions in adjacent frames using a FLANN matcher; calculating the facial displacement vector set using optical flow, and calculating the command matching degree using standard motion vectors. When the matching degree meets a preset threshold, the facial liveness detection is considered passed.

7. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 1, characterized in that, The method for extracting facial features using local binary and joint encoding includes: obtaining the core region from the standard facial image and cropping it to obtain a core region sub-image set; performing local binary encoding on each core region sub-image to generate a binary encoded image and integrating them to obtain a binary encoded image set; dividing the binary encoded image into sub-blocks, counting the frequency of 1 and 0 occurrences within each sub-block to obtain a sub-block histogram, and arranging them by position to obtain a binary encoded histogram; concatenating the core region binary encoded histograms, calculating energy, contrast, and entropy, flattening and standardizing them to obtain facial features.

8. The live attendance anti-counterfeiting method based on layered fusion of vein and facial features as described in claim 1, characterized in that, The dynamic confirmation of confidence weight ratios based on attention-based temporal networks includes: confirming user hashes and time periods; retrieving historical attendance records for the same user hashes during the same time period from the database; extracting historical attendance times and historical confidence weight ratios; concatenating historical intervals to obtain a historical interval vector; converting it into a duration encoding vector using sine and cosine encoding; concatenating historical confidence weight ratios to obtain a historical confidence weight ratio vector; concatenating it row-wise with the duration encoding vector to obtain a value matrix; converting the value matrix into a query matrix and a key matrix using linear modulation; obtaining a dynamic weight vector using a dynamic attention mechanism; calculating the context representation; and obtaining the confidence weight ratio through temporal inference in a fully connected layer.

9. A live attendance and anti-counterfeiting system based on layered fusion of vein and facial features, characterized in that, It includes a data acquisition module, a vein matching module, a facial verification module, and a combined testing module; Data acquisition module The palm image of the frame is taken, and the process continues until the first frame of the face is captured. Random instructions are then generated for the capture. A frame of facial images; The vein matching module is optimized using a vein texture enhancement algorithm. The palm part of the frame is obtained using an improved U-Net network. The set of bifurcation points in a frame is used to perform vein liveness detection based on the displacement of bifurcation points in adjacent frames, and vein liveness detection is only performed when the vein liveness detection is successful. The vein features of the frame are encrypted and then used for vein matching. The vein user hash is sent only when the vein match is successful. Total number of frames; The facial verification module starts when it receives the vein user hash and analyzes it. Micro-expression changes in facial images of frames are used for facial liveness detection. Local binary and joint encoding are used to extract liveness data only when the facial liveness detection is passed. The facial features of the frame are encrypted and then used for face matching. The user hash and time period are only confirmed when the face match is successful. The joint verification module references attendance records of the same user hash within the same historical time period, and dynamically confirms the confidence weight ratio based on an attention-based temporal network. Frame vein features and The facial features of each frame are linearly tuned and then sequentially input into the frame evaluation network to filter and obtain the optimal palm image and the optimal face image. The spectral features of the optimal palm image and the optimal face image in the same band are extracted and combined with the confidence weight ratio for consistency testing. Attendance is considered successful only when the consistency test is passed. The consistency check includes: The vein and facial features are linearly dimensionally tuned and sequentially input into the frame evaluation network to obtain palm and facial image scores, anchoring the optimal palm and facial images. The corresponding band components of the optimal facial image are extracted, and the non-vein regions of the optimal palm image are determined by combining the region segmentation mask image, and the skin color spectral features of the palm are calculated. The cheek region components of the optimal facial image are determined, and the skin color spectral features of the face are calculated. The comprehensive image score is calculated based on the confidence weight ratio, and the spectral similarity score is calculated. The comprehensive image score and the spectral similarity score are weighted and summed to obtain the consistency score. When the score meets the preset threshold, the consistency test is considered passed.

Images