Employee attendance system based on face recognition technology

Abstract

The invention discloses an employee attendance system based on a face recognition technology, and belongs to the technical field of attendance. A first face skeleton extraction mapping module; a filling module; a classification module; an identification segmentation module; a second acquisition module; a second face skeleton extraction mapping module; an analysis module; an attendance recording module; according to the system, identity recognition and attendance checking are carried out on the basis of comparison of facial skeleton structures, the influence of factors such as light and shadow and makeup on identity recognition can be eliminated, meanwhile, the distinguished facial skeleton structure image and the facial skeleton structure image are compared only through coordinates during comparison, the recognition data volume of the system is reduced, and the recognition efficiency is improved. And the attendance recognition efficiency of the system is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of attendance, and in particular relates to an employee attendance system based on face recognition technology. Background Art

[0002] Employee attendance is an important task in the daily management of an enterprise. It is not only related to employee attendance and salary calculation, but also closely related to the normal operation and efficiency of the enterprise.

[0003] Through effective employee attendance management, companies can ensure that employees arrive on time, complete their work tasks, and improve production efficiency and quality. At the same time, a reasonable attendance system can also help maintain the company's labor discipline and employee work order, and improve the company's overall image and competitiveness.

[0004] Chinese Patent No. 202110699382.1 discloses an attendance method, system, electronic device and storage medium based on face recognition, which consists of an acquisition module, an expansion module, a training module and a recognition module. Before attendance, the corrected face pictures of the personnel to be checked are collected as a sample set, and the pre-trained 3D deformable generative adversarial network is used to generate face pictures of any angle for the corrected face pictures to expand the sample set, and the face recognition model is trained with the expanded sample set. During attendance, the trained face recognition model is used to recognize the face pictures of the on-site attendance personnel at any angle, and when the recognition is successful, an attendance record is generated.

[0005] The inventors of this application have discovered that the above-mentioned existing attendance method based on face recognition is based on a trained face recognition model that can recognize faces from multiple angles to complete attendance. However, the facial information of attendance employees at multiple angles varies greatly under different lighting environments, and the facial information of some attendance employees after putting on makeup varies greatly. These problems will bring difficulties to attendance and reduce the success rate of attendance based on face recognition. Summary of the invention

[0006] To solve the problems raised in the above background technology, the present invention provides an employee attendance system based on face recognition technology, which can eliminate the influence of factors such as light and shadow and makeup on identity recognition, while reducing the amount of recognition data of the system and improving the efficiency of the system in recognizing attendance.

[0007] To achieve the above purpose, the present invention provides the following technical solution: an employee attendance system based on face recognition technology, comprising:

[0008] The first acquisition module acquires the correct face image of the employee to be checked in a specific background;

[0009] The first facial skeleton extraction and mapping module uses a deep learning algorithm to extract the skeleton joint point sequence of the employee's corrected face image to generate a facial skeleton structure image;

[0010] A filling module generates a multi-angle facial bone structure image of the employee to be checked in a specific background based on the input facial bone structure image of the employee to be checked in and preset viewing angle data through a pre-trained adversarial network image generation model;

[0011] A classification module classifies the multiple facial bone structure images according to the same angle;

[0012] The recognition and segmentation module recognizes the difference between multiple facial bone structure images at the same angle through a pre-trained image recognition model, and segments multiple different facial bone structure images with coordinate information retained through a pre-trained image segmentation model;

[0013] A saving module is used to save the plurality of images of facial bone structures segmented above;

[0014] The second acquisition module acquires the face images of employees to be checked in a specific background;

[0015] The second facial skeleton extraction and mapping module uses a deep learning algorithm to extract the skeleton joint point sequence of the collected facial image of the employee to be checked in and generate a facial skeleton structure image;

[0016] The analysis module determines the angle of the facial bone structure image of the employee to be checked in based on a specific background. After the angle is determined, the facial bone structure image of the employee to be checked in is compared with multiple different facial bone structure images saved at the adapted angle to analyze the identity of the employee to be checked in.

[0017] Attendance record module, the analysis module automatically generates attendance records after analyzing the identity of the employee to be checked in, and the attendance is successful.

[0018] Furthermore, the specific background in the first acquisition module refers to the static background behind the employee to be checked, and the static background contains at least one relatively static object including a corner of a wall that cannot be moved or blocked at will.

[0019] Furthermore, the deep learning algorithms in the first facial skeleton extraction and mapping module and the second facial skeleton extraction and mapping module are Openpose deep learning algorithms.

[0020] Furthermore, the pre-training steps of the adversarial network image generation model in the filling module are:

[0021] Establish an initial model for adversarial network image generation;

[0022] Collect multi-angle face images and annotate the categories and angles of the face images, annotate them as real annotations, and obtain a real face image training data set;

[0023] The generator generates multi-angle face images and annotates the categories and angles of the face images, annotating them as false annotations, and obtaining a false face image training data set;

[0024] The real face image training data set and the fake face image training set are input into the discriminator for discrimination, the discrimination results are compared with the annotation results, and the generator loss function and the discriminator loss function are optimized until the model converges to obtain the adversarial network image generation model.

[0025] Furthermore, the multiple angles of the real face image of the adversarial network image generation model in the filling module are distinguished based on the angle between a relatively static object including a corner of a wall in a specific background that cannot be moved or blocked at will and a certain bone joint point of the facial bone structure image.

[0026] Furthermore, the angle classification of the classification module is also based on the angle distinction between a relatively stationary object including a wall corner in a specific background that cannot be moved or blocked at will and a certain bone joint point of the facial bone structure image.

[0027] Furthermore, the pre-training steps of the image recognition model and the image segmentation model in the recognition and segmentation module are similar to the pre-training steps of the adversarial network image generation model, both of which include establishing an initial model, training through a training data set, and optimizing convergence.

[0028] Furthermore, in the analysis module, the specific steps of comparing the facial bone structure image of the employee to be checked in with multiple distinguishing facial bone structure images saved at the adaptation angle are: comparing the multiple distinguishing facial bone structure images with the images with the same coordinate information of the facial bone structure image of the employee to be checked in sequence according to the coordinate information; if the facial bone structure image of the employee to be checked in is the same as a certain distinguishing facial bone structure image, the identity of the employee to be checked in is analyzed.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention collects a straight face image of an employee to be checked in a specific background, extracts a bone joint point sequence based on a deep learning algorithm to generate a facial bone structure image, then generates a multi-angle facial bone structure image based on a preset angle, classifies and compares the facial bone structure images of multiple persons to be checked in based on the angle, and segments different images of multiple facial bone structure images at the same angle for storage. When checking in, the face image of the person to be checked in is collected and a bone joint point sequence is extracted to generate a facial bone structure image, and then the angle of the collected facial bone structure image is analyzed and multiple segmented facial bone structure images at the same angle are extracted for comparison to quickly analyze the information of the person to be checked in, perform attendance record generation and complete the attendance. The system performs identity recognition and then checks in based on the comparison of facial bone structures, and can screen out the influence of factors such as light and shadow and makeup on identity recognition. At the same time, when comparing, only the difference facial bone structure image needs to be compared with the facial bone structure image through coordinates, thereby reducing the recognition data volume of the system and improving the recognition and attendance efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0032] In the figure: 1. The first acquisition module; 2. The first facial skeleton extraction and imaging module; 3. The filling module; 4. The classification module; 5. The recognition and segmentation module; 6. The saving module; 7. The second acquisition module; 8. The second facial skeleton extraction and imaging module; 9. The analysis module; 10. The attendance record module. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 The present invention provides the following technical solution: an employee attendance system based on face recognition technology, comprising:

[0035] The first acquisition module 1 acquires a corrected face image of the employee to be checked in a specific background, wherein the specific background refers to a static background behind the employee to be checked in, and there is at least one relatively static object including a corner in the static background that cannot be moved or blocked at will, and the static background is determined before the system has not checked in. If the system can continue to check in, but the static background is changed during the attendance process, the system needs to clear the data, re-collect the corrected face image of the employee to be checked in the specific background, and perform subsequent processing;

[0036] The first facial skeleton extraction and mapping module 2 uses the Openpose deep learning algorithm to extract the skeleton joint point sequence of the employee's corrected face image to generate a facial skeleton structure image;

[0037] Filling module 3, generates a multi-angle facial bone structure image of the employee to be checked in a specific background based on the input facial bone structure image of the employee to be checked in and the preset viewing angle data through a pre-trained adversarial network image generation model, wherein the pre-training steps of the adversarial network image generation model are:

[0038] Establish an initial model for adversarial network image generation;

[0039] Collect multi-angle face images and annotate the categories and angles of the face images, annotate as real annotations, and obtain a real face image training data set, wherein the multi-angle is based on the angle between a relatively static object including a wall corner in a specific background that cannot be moved or blocked at will and a preset bone joint point in the facial bone structure image, that is, one side of the object that cannot be moved or blocked at will is taken as edge A, and the connection between the object that cannot be moved or blocked at will and a preset bone joint point is taken as edge B. The lengths of edge A and edge B are determined based on pixel blocks, thereby determining the angle. The calculation formula is:

[0040]

[0041] The generator generates multi-angle face images and annotates the categories and angles of the face images, annotating them as false annotations, and obtaining a false face image training data set;

[0042] The real face image training data set and the fake face image training set are input into the discriminator for discrimination, the discrimination result is compared with the annotation result, and the generator loss function and the discriminator loss function are optimized until the model converges to obtain the adversarial network image generation model;

[0043] The objective function of the adversarial network model is:

[0044]

[0045] Where: min represents the minimum value, max represents the maximum value, G represents the image generator, D represents the image discriminator, V represents the training perspective, x represents the training image, G(z,v) represents the angle image generated by the noise z of the generator, pdata represents the probability distribution of the training image, pv represents the probability distribution of the training perspective data, D(x) and D(G(z,v)) represent the discriminator output probabilities, λ represents the preset balance factor, and L represents the cross entropy loss function;

[0046] Classification module 4, classifying the above-mentioned multiple facial bone structure images according to the same angle, where the angle is distinguished based on the angle between a relatively static object including a wall corner in a specific background that cannot be moved or blocked at will and a certain set bone joint point of the facial bone structure image;

[0047] The recognition and segmentation module 5 recognizes the difference between multiple facial bone structure images at the same angle through a pre-trained image recognition model, and segments multiple different facial bone structure images with coordinate information retained through the pre-trained image segmentation model, wherein the pre-training steps of the image recognition model are:

[0048] Establish an initial model for image recognition;

[0049] Collect a facial bone structure image, and block and mark different positions of the facial bone structure image to obtain a training data set;

[0050] Input the training data set into the initial image recognition model for recognition, compare the recognition result with the annotation result, optimize the loss function until the model converges, and obtain the image recognition model;

[0051] The objective function of the image recognition model is:

[0052]

[0053] Where: θ represents the learning parameter of the F model, L represents the model loss function, X represents the input image set, including complete facial bone structure images and occluded facial bone structure images, and y represents the correct label;

[0054] The pre-training steps for the image segmentation model are:

[0055] Establish an initial model for image segmentation;

[0056] Collect multiple facial bone structure images as image sample data sets;

[0057] Calibrate the corresponding pairwise constraint set according to the image sample data set;

[0058] Assign weights to each pair of sample pixels in the image sample data set according to preset rules to obtain a weight coefficient matrix;

[0059] The cross entropy of the initial image segmentation model is calculated according to the weight coefficient matrix, and the model is optimized according to the cross entropy until the model converges to obtain the image segmentation model;

[0060] A storage module 6 is used to store the plurality of images of facial bone structure that are segmented;

[0061] The second acquisition module 7 acquires the face image of the employee to be checked in a specific background;

[0062] The second facial skeleton extraction and mapping module 8 extracts the skeleton joint point sequence of the collected facial image of the employee to be checked in through the Openpose deep learning algorithm to generate a facial skeleton structure image, wherein the second facial skeleton extraction and mapping module 8 and the first facial skeleton extraction and mapping module 2 can be the same facial skeleton extraction and mapping module;

[0063] The analysis module 9 determines the angle of the facial bone structure image of the employee to be checked based on a specific background, compares the facial bone structure image of the employee to be checked with a plurality of different facial bone structure images saved at an adaptation angle after the angle determination, and compares the plurality of different facial bone structure images with the same coordinate information images of the facial bone structure image of the employee to be checked in sequence according to the coordinate information, and if the facial bone structure image of the employee to be checked is the same as a certain different facial bone structure image, the identity of the employee to be checked is analyzed;

[0064] The attendance recording module 10 and the analysis module 9 automatically generate the attendance record after analyzing the identity of the employee to be checked on, and the attendance is successful.

[0065] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Employee attendance system based on face recognition technology, It is characterized in that include: The first acquisition module (1) acquires a corrected face image of the employee to be checked in a specific background; The first facial skeleton extraction and mapping module (2) extracts the skeleton joint point sequence of the corrected face image of the employee to be checked in through a deep learning algorithm to generate a facial skeleton structure image; A filling module (3) generates a multi-angle facial bone structure image of the employee to be checked in a specific background based on the input facial bone structure image of the employee to be checked in and preset viewing angle data through a pre-trained adversarial network image generation model; A classification module (4) classifies the plurality of facial bone structure images according to the same angle; A recognition and segmentation module (5) is used to recognize the difference between multiple facial bone structure images at the same angle through a pre-trained image recognition model, and to segment multiple different facial bone structure images with coordinate information retained through a pre-trained image segmentation model; A storage module (6) stores the plurality of images of facial bone structure that are segmented; The second acquisition module (7) acquires the facial image of the employee to be checked in a specific background; The second facial skeleton extraction and mapping module (8) extracts the skeleton joint point sequence of the collected facial image of the employee to be checked in through a deep learning algorithm to generate a facial skeleton structure image; An analysis module (9) determines the angle of the facial bone structure image of the employee to be checked based on a specific background, compares the facial bone structure image of the employee to be checked with a plurality of different facial bone structure images saved at the adapted angle, and analyzes the identity of the employee to be checked; The attendance recording module (10) and the analysis module (9) automatically generate an attendance record after analyzing the identity of the employee to be checked on, and the attendance is successful.

2. The employee attendance system based on face recognition technology according to claim 1, Features: The specific background in the first acquisition module (1) refers to the static background behind the employee to be checked, and the static background contains at least one relatively static object including a corner of a wall that cannot be moved or blocked at will.

3. The employee attendance system based on face recognition technology according to claim 1, Features: The deep learning algorithm in the first facial skeleton extraction and mapping module (2) and the second facial skeleton extraction and mapping module (8) is an Openpose deep learning algorithm.

4. The employee attendance system based on face recognition technology according to claim 1, Features: The pre-training steps of the adversarial network image generation model in the filling module (3) are: Establish an initial model for adversarial network image generation; Collect multi-angle face images and annotate the categories and angles of the face images, annotate them as real annotations, and obtain a real face image training data set; The generator generates multi-angle face images and annotates the categories and angles of the face images, annotating them as false annotations, and obtaining a false face image training data set; The real face image training data set and the fake face image training set are input into the discriminator for discrimination, the discrimination results are compared with the annotation results, and the generator loss function and the discriminator loss function are optimized until the model converges to obtain the adversarial network image generation model.

5. The employee attendance system based on face recognition technology according to claim 4, Features: The multiple angles of the real face image of the adversarial network image generation model in the filling module (3) are distinguished based on the angle between a relatively static object including a wall corner in a specific background that cannot be moved or blocked at will and a certain bone joint point of the facial bone structure image.

6. The employee attendance system based on face recognition technology according to claim 1, Features: The angle classification of the classification module (4) is also based on the angle distinction between a relatively static object including a wall corner in a specific background that cannot be moved or blocked at will and a certain bone joint point in the facial bone structure image.

7. The employee attendance system based on face recognition technology according to claim 1, Features: The pre-training steps of the image recognition model and the image segmentation model in the recognition and segmentation module (5) are similar to the pre-training steps of the adversarial network image generation model, both of which include establishing an initial model, training through a training data set, and optimizing convergence.

8. The employee attendance system based on face recognition technology according to claim 1, Features: In the analysis module (9), the specific steps of comparing the facial bone structure image of the employee to be checked with a plurality of distinguishing facial bone structure images stored at the adaptation angle are: comparing the plurality of distinguishing facial bone structure images in sequence according to the coordinate information with the image with the same coordinate information of the facial bone structure image of the employee to be checked; if the facial bone structure image of the employee to be checked is the same as a certain distinguishing facial bone structure image, the identity of the employee to be checked is analyzed.

Images