Sign-in method, device, electronic device and storage medium based on face recognition

Abstract

The face recognition-based sign-in method, device, electronic device and storage medium provided by the embodiment of the present invention are applied to the field of information technology, and can obtain an image to be identified; perform face detection on the image to be identified to obtain a face region image and corresponding face feature information; transform the face region image according to the face feature information to obtain a face image of preset specifications; match the face image of preset specifications with preset face images corresponding to multiple preset persons to obtain a matching preset face image; and mark the preset person corresponding to the matching preset face image as having signed in. Thus, by transforming the face region image according to the face feature information to obtain a face image of preset specifications, and then performing recognition and sign-in through the face image of preset specifications, not only the efficiency of sign-in can be improved, but also the accuracy of sign-in can be improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a sign-in method, device, electronic device and storage medium based on face recognition. Background Art

[0002] At present, the sign-in method based on face recognition has been widely used. By extracting and analyzing the feature values ​​of the facial graphics of different people in the captured video, and comparing them with the face similarity threshold stored in the database, it can be concluded whether the target person is present or signed in.

[0003] However, the inventors have found that the current sign-in method based on face recognition often has high requirements for the captured face image. When the angle of the captured person is offset, the side face is too far away, the face image is small, etc., the recognition accuracy is often low. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a sign-in method, device, electronic device and storage medium based on face recognition, so as to improve the accuracy of sign-in based on face recognition. The specific technical solution is as follows:

[0005] In a first aspect of an embodiment of the present application, a sign-in method based on face recognition is first provided, comprising:

[0006] Obtain an image to be recognized;

[0007] Performing face detection on the image to be recognized to obtain a face region image and corresponding face feature information;

[0008] Transforming the face region image according to the face feature information to obtain a face image of preset specifications;

[0009] Matching the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image;

[0010] The preset person corresponding to the matched preset face image is marked as having signed in.

[0011] Optionally, the facial feature information includes location information of facial feature points, and the detecting of the face on the image to be identified to obtain the facial region image and the corresponding facial feature information includes:

[0012] Performing face detection on the image to be identified by using a preset network model to obtain the face area image;

[0013] Identify facial feature points in the facial region image and obtain position information of the facial feature points.

[0014] Optionally, transforming the face region image according to the face feature information to obtain a face image of preset specifications includes:

[0015] Calculate the corresponding affine matrix according to the position information of the facial feature points;

[0016] The face region image is transformed by the affine matrix to obtain a face image of preset specifications.

[0017] Optionally, obtaining the image to be identified includes:

[0018] Obtain video images of the preset check-in period;

[0019] Frame extraction is performed on the video image of the preset check-in time period according to a first frame extraction rate to obtain a plurality of images to be identified.

[0020] Optionally, after marking the person corresponding to the designated facial image as having signed in, the method further includes:

[0021] If there is a person who is not marked as having signed in among the multiple preset persons, frame sampling is performed on the video image of the preset sign-in period according to a second frame sampling rate to obtain a plurality of secondary frame sampling images, wherein the second frame sampling rate is greater than the first frame sampling rate;

[0022] The image to be identified is replaced with the secondary frame image, and the step of performing face detection on the image to be identified is returned to obtain a face area image and corresponding face feature information and the detection is repeated until a preset stop condition is met.

[0023] Optionally, the method further includes:

[0024] Acquire video images of the preset sign-out period;

[0025] Extract frames of the video image of the preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified;

[0026] Performing face detection on the sign-out image to be identified to obtain a face region image and corresponding face feature information;

[0027] Transforming the face region image according to the face feature information to obtain a face image of preset specifications;

[0028] Matching the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image;

[0029] The preset person corresponding to the matched preset facial image is marked as having signed out.

[0030] According to a second aspect of the embodiment of the present application, a check-in device based on face recognition is provided, comprising:

[0031] An image acquisition module, used for acquiring an image to be identified;

[0032] A face detection module is used to detect faces in the image to be identified, and obtain a face region image and corresponding face feature information;

[0033] An image transformation module, used to transform the face region image according to the face feature information to obtain a face image of preset specifications;

[0034] An image matching module is used to match the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain matching preset face images;

[0035] The personnel marking module is used to mark the preset personnel corresponding to the matched preset face image as having signed in.

[0036] Optionally, the facial feature information includes location information of facial feature points, and the face detection module includes:

[0037] A model detection submodule, used to detect the face of the image to be identified by using a preset network model to obtain the face area image;

[0038] The feature point recognition module is used to recognize the facial feature points in the facial region image and obtain the position information of the facial feature points.

[0039] Optionally, the image transformation module includes:

[0040] A matrix calculation submodule, used to calculate the corresponding affine matrix according to the position information of the facial feature points;

[0041] The image transformation submodule is used to transform the face area image through the affine matrix to obtain a face image of preset specifications.

[0042] Optionally, the image acquisition module includes:

[0043] A video acquisition submodule is used to acquire video images during a preset period of time;

[0044] The first frame extraction submodule is used to extract frames of the video images in the preset time period according to a first frame extraction rate to obtain a plurality of images to be identified.

[0045] Optionally, the device further comprises:

[0046] A secondary frame extraction module, configured to extract frames of the video images of the preset time period according to a second frame extraction rate to obtain a plurality of secondary frame extraction images if there are persons who are not marked as having signed in among the plurality of preset persons;

[0047] The secondary detection module is used to replace the image to be identified with the secondary frame image, and return to the step of performing face detection on the image to be identified to obtain the face area image and the corresponding face feature information and re-detect until a preset stop condition is met.

[0048] Optionally, the device further comprises:

[0049] A sign-out video acquisition module is used to acquire video images during a preset sign-out period;

[0050] A sign-out image acquisition module, configured to extract frames of the video image of the preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified;

[0051] A sign-out video acquisition module is used to detect the face of the sign-out image to be identified, and obtain a face area image and corresponding face feature information;

[0052] A sign-out image transformation module, used to transform the face region image according to the face feature information to obtain a face image of preset specifications;

[0053] A sign-out face matching module is used to match the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image;

[0054] The sign-out personnel marking module is used to mark the preset personnel corresponding to the matched preset facial image as having signed out.

[0055] Another aspect of the embodiment of the present application further provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0056] Memory, used to store computer programs;

[0057] The processor is used to implement any of the above-mentioned steps of the face recognition-based sign-in method when executing the program stored in the memory.

[0058] In another aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned steps of the face recognition-based check-in method is implemented.

[0059] In another aspect of an embodiment of the present application, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned steps of the check-in method based on face recognition.

[0060] Beneficial effects of the embodiments of the present invention:

[0061] The face recognition-based sign-in method, device, electronic device and storage medium provided in the embodiment of the present invention can obtain an image to be identified; perform face detection on the image to be identified to obtain a face region image and corresponding face feature information; transform the face region image according to the face feature information to obtain a face image of preset specifications; match the face image of preset specifications with preset face images corresponding to multiple preset persons to obtain matching preset face images; mark the preset person corresponding to the matching preset face image as having signed in. Thus, by transforming the face region image according to the face feature information to obtain a face image of preset specifications, and then performing recognition and sign-in through the face image of preset specifications, not only the efficiency of sign-in can be improved, but also the accuracy of sign-in can be improved.

[0062] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0064] Figure 1 A flowchart of a face recognition-based sign-in method provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of a process for detecting a face in an image to be recognized provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of a network model provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a process for transforming a face region image provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of an example of a sign-in method based on face recognition provided in an embodiment of the present application;

[0069] Figure 6A schematic diagram of a process for obtaining an image to be identified provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of a process flow of a secondary sign-in provided in an embodiment of the present application;

[0071] Figure 8 A flowchart of a sign-out method based on face recognition provided in an embodiment of the present application;

[0072] Figure 9a Another example schematic diagram of a sign-in method based on face recognition provided in an embodiment of the present application;

[0073] Figure 9b A schematic diagram of another example of a sign-in method based on face recognition provided in an embodiment of the present application;

[0074] Fig.10 A structural diagram of a sign-in device based on face recognition provided in an embodiment of the present application;

[0075] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0077] In a first aspect of an embodiment of the present application, a sign-in method based on face recognition is first provided, comprising:

[0078] Obtain an image to be recognized;

[0079] Perform face detection on the image to be recognized to obtain the face area image and the corresponding face feature information;

[0080] Transform the face region image according to the face feature information to obtain a face image of preset specifications;

[0081] Matching a facial image of a preset specification with preset facial images corresponding to a plurality of preset persons to obtain a matching preset facial image;

[0082] The preset person corresponding to the matching preset face image is marked as having signed in.

[0083] It can be seen that the method of the embodiment of the present application can transform the face area image according to the face feature information to obtain a face image of preset specifications, and then use the face image of preset specifications for recognition and sign-in, which can not only improve the efficiency of sign-in, but also improve the accuracy of sign-in.

[0084] For details, see Figure 1 , Figure 1 A flowchart of a face recognition-based sign-in method provided in an embodiment of the present application includes:

[0085] Step S11, obtaining an image to be recognized.

[0086] The method of the embodiment of the present application is applied to face check-in. Specifically, it can be achieved by collecting a face image, and then matching the newly added face image with the face images of multiple preset persons to be checked-in. If more than one person to be checked-in is matched, the person to be checked-in successfully signs in. The image to be recognized in the embodiment of the present application can be an image directly obtained for face check-in, or it can be an image for face check-in obtained by obtaining a video for face check-in and then extracting frames from the video.

[0087] The method of the embodiment of the present application is applied to an intelligent terminal and can be implemented by the intelligent terminal. Specifically, the intelligent terminal can be an intelligent terminal for training a network model, or the intelligent terminal can be used only for image enhancement, and after data enhancement, the enhanced image can be imported into a device for training a network model for training. The intelligent terminal can be a computer, a server, etc.

[0088] Step S12, performing face detection on the image to be recognized to obtain a face region image and corresponding face feature information.

[0089] Among them, the face detection of the image to be recognized can be performed by a pre-trained network model. Specifically, the model can be a variety of types of models, for example, a Retinaface (face detection network) model, etc. By performing face detection on the image to be recognized, an optimal face detection frame can be obtained, and feature information of multiple preset face features can be marked. Specifically, the feature information can be feature information of the multiple preset features. For example, the feature information corresponding to five target points on the face, such as the eyes, nose, eyebrows, and both sides of the mouth, can be calibrated.

[0090] Step S13, transforming the face region image according to the face feature information to obtain a face image of preset specifications.

[0091] Among them, the face area image is transformed according to the face feature information, and the key feature points of the face can be automatically located according to the face feature information, and a set of feature points of the face can be obtained. Then, the affine transformation of openCV (OpenSource Computer Vision Library, a cross-platform computer vision and machine learning software library) is used to map the key point coordinates to the fixed coordinates to achieve face alignment and obtain a face image of preset specifications.

[0092] Step S14, matching the face image of preset specifications with the preset face images corresponding to a plurality of preset persons to obtain matching preset face images.

[0093] The facial image of preset specifications is matched with the preset facial images corresponding to the multiple preset persons, and a preset image library of persons to be signed in can be used, and the image library can include facial images of multiple persons to be signed in. The facial image of preset specifications is matched with the preset facial images corresponding to the multiple preset persons, and the facial image of preset specifications can be matched with the facial image of each person to be signed in respectively.

[0094] Specifically, a face image of preset specifications is matched with preset face images corresponding to multiple preset persons, and FaceNet (a face recognition system) can be used to map the face image of preset specifications to a 128-dimensional feature vector. FaceNet is a model for acquiring facial features. The feature vector of each face can be obtained by passing the previously obtained face image of preset specifications into the FaceNet model. Then, FaceNet uses the CNN+Triplet loss (convolutional neural network and triplet loss) method to directly learn the mapping from the face image to the compact Euclidean space and extract the embedded features. Then, using the feature vector of the face, the distance between the face vector corresponding to the preset face images corresponding to multiple preset persons is calculated to obtain the serial number of the face most similar to the face image of the preset specifications. And determine whether the distance corresponding to the serial number is less than the threshold. If it is less than, it is determined to be a match.

[0095] Step S15, marking the preset person corresponding to the matched preset face image as having signed in.

[0096] After matching a face image of preset specifications with preset face images corresponding to multiple preset persons, the preset person corresponding to the preset face image that matches the face image of preset specifications can be selected, and then the preset person is marked as having signed in. For example, when the face image of preset specifications matches the preset face image of Zhang San, Zhang San is marked as having signed in. In actual use, the time when the image to be identified corresponding to the face image of preset specifications is obtained can also be recorded, and then the check-in time of the preset person can be marked according to the time, and the specific information can also include the name of the person corresponding to the face, the entry time, the entry person, the status and other information.

[0097] It can be seen that the method of the embodiment of the present application can transform the face area image according to the face feature information to obtain a face image of preset specifications, and then use the face image of preset specifications for recognition and sign-in, which can not only improve the efficiency of sign-in, but also improve the accuracy of sign-in.

[0098] Optional, see Figure 2 The facial feature information includes the location information of the facial feature points. Step S12 detects the face of the image to be recognized to obtain a facial region image and corresponding facial feature information, including:

[0099] Step S121, detecting a face on the image to be recognized by using a preset network model to obtain a face region image;

[0100] Step S122, identifying facial feature points in the face region image, and obtaining position information of the facial feature points.

[0101] The face detection is performed on the image to be recognized by the preset network model, and the face area image can be detected by the pre-trained network model. Specifically, the model can be a variety of types of models, such as Retinaface model, etc. For details, see Figure 3, the Retinaface model adds the SSH (Su-Schrieffer-Heeger) network detection module to form an mnet (deep learning) structure that uses the feature pyramid technology. The feature extraction is enhanced through the FPN (feature pyramid) pyramid and the SSH detection network to achieve the fusion of multi-scale information. The detection frame prediction results are obtained from the features through the Class Head module, Box Head module and Landmark Head module in Retinaface, and the low-score face area is ignored. Then, the redundant bounding boxes are eliminated through decoding and non-maximum suppression, and then the optimal detection result is obtained. Among them, the ClassHead module is used to calculate the probability that the image in the calibration frame belongs to the face, the Box Head module is used to predict the position of the calibration frame, and the Landmark Head module is used to calibrate the position of the facial feature points. The detection of the facial key point Landmark is added to achieve the learning of multiple target points. By detecting the face of the image to be recognized, the optimal face detection frame can be obtained, and the facial feature points in the face area image can be identified. The feature information of multiple preset facial features can be marked. Specifically, the feature information can be the feature information of the multiple preset features. For example, the feature information corresponding to five target points on the face, such as the eyes, nose, eyebrows, and both sides of the mouth, can be calibrated.

[0102] Optional, see Figure 4 Step S13 transforms the face region image according to the face feature information to obtain a face image of preset specifications, including:

[0103] Step S131, calculating the corresponding affine matrix according to the position information of the facial feature points;

[0104] Step S132, transforming the face region image by using an affine matrix to obtain a face image of preset specifications.

[0105] Among them, see Figure 5 , the corresponding affine matrix is ​​calculated according to the position information of the facial feature points; the face area image is transformed by the affine matrix to obtain a face image of preset specifications. Please refer to the prior art and will not repeat it here. Optionally, the coordinates corresponding to the eyes, nose, eyebrows and both sides of the mouth in the face image of preset specifications can be: [[38.2946, 51.6963], [73.5318, 51.5014], [56.0252, 71.7366], [41.5493, 92.3655], [70.7299, 92.2041]].

[0106] Optional, see Figure 6 Step S11 obtains the image to be identified, including:

[0107] Step S111, obtaining a video image of a preset check-in period;

[0108] Step S112: extract frames of the video images of the preset check-in period according to a first frame extraction rate to obtain a plurality of images to be identified.

[0109] Among them, the video images of the preset check-in time period can be obtained by using a video acquisition device, such as a camera, to obtain the video images of the area where personnel are waiting to check in. The video images of the preset check-in time period are framed according to a first frame rate to obtain multiple images to be identified. The video images of the preset check-in time period can be framed at multiple frame rates. For example, the preset frame rate of 0.8 frames / second is used to extract images within the first 5 minutes between 8:30 and 14:00; and the preset frame rate of 0.5 frames / second is used to extract images within the last 5 minutes after 17:30, to obtain multiple images to be identified. In actual use, the image to be identified can also be preprocessed. Specifically, the preprocessing method includes at least one of the following: grayscale, binarization, and image super-resolution preprocessing.

[0110] Optional, see Figure 7 After marking the person corresponding to the specified face image as having signed in, the method further includes:

[0111] Step S71, if there is a person who is not marked as having signed in among the multiple preset persons, frame sampling is performed on the video image of the preset sign-in period according to a second frame sampling rate to obtain a plurality of secondary frame sampling images, wherein the second frame sampling rate is greater than the first frame sampling rate;

[0112] Step S72, replace the image to be identified with the secondary frame image, and return to the step of performing face detection on the image to be identified, and re-detect the step of obtaining the face area image and the corresponding face feature information until the preset stop condition is met.

[0113] If there are persons who are not marked as signed in among the multiple preset persons, the video images of the preset sign-in period can be framed according to the second frame rate to obtain multiple secondary framed images, and then the image to be identified is replaced by the secondary framed images, and then the process returns to the above step S12 to perform face detection on the image to be identified, obtain a face area image and corresponding face feature information, re-detect, and re-execute steps S12 to S15 to continue. Until all persons have signed in or the time has exceeded the preset time point.

[0114] Optional, see Figure 8 , the above method further comprises:

[0115] Step S81, obtaining a video image of a preset sign-out period;

[0116] Step S82, extracting frames of the video image of the preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified;

[0117] Step S83, performing face detection on the sign-out image to be recognized to obtain a face region image and corresponding face feature information;

[0118] Step S84, transforming the face region image according to the face feature information to obtain a face image of preset specifications;

[0119] Step S85, matching the face image of the preset specification with the preset face images corresponding to the plurality of preset persons to obtain a matching preset face image;

[0120] Step S86: Mark the preset person corresponding to the matched preset facial image as having signed out.

[0121] Among them, the steps S81-S86 for signing out are similar to the steps for signing in, and can be referred to the steps for signing in, which will not be repeated here.

[0122] In order to illustrate the method of the embodiment of the present application, the following is described in conjunction with specific embodiments. Figure 9a and Figure 9b ,include:

[0123] 1. Obtain the target video of the area where personnel are waiting to sign in;

[0124] 2. Extract the image at the preset frame rate S within the pre-order time threshold range Tn of each time point from the preset 1st time point to the n-1th time point; extract the image at the preset frame rate S within the post-order time threshold range Tm of the nth time point, and transmit it to the face detection and recognition module;

[0125] 3. The video sequence within the preceding time threshold range Tn of each time point from the 1st time point to the n-1th time point in step 2 and the video sequence within the following time threshold range Tm of the nth time point are stored in the video storage module;

[0126] 4. The face detection and recognition module receives and preprocesses the image in step 2;

[0127] 5. Perform face detection based on Retinaface+SSH network to obtain the optimal face detection frame;

[0128] 6. Transform the feature points to a preset position and calculate an affine matrix. Finally, transform the face area image using the affine matrix and scale it to the same size to achieve face alignment.

[0129] 7. Make the face encodings of the same category more cohesive and the face encodings of different categories more dispersed, so as to encode the face images after the above face alignment into low-dimensional vectors.

[0130] 8. Using the obtained facial feature vector, calculate the distance between the facial vector and the facial vector in the reference facial database, and obtain the serial number of the most similar face in the reference database for each face. Determine whether the facial vector distance corresponding to the serial number is less than a threshold value.

[0131] If it is less than, the face match is successful, and the "Whether to sign in flag" of the successfully matched face in the sign-in result database is updated to "yes".

[0132] If the face vector distance is greater than the threshold, the match fails, and the "whether to sign in flag" of the successfully matched face in the sign-in result database is updated to "no".

[0133] 9. The face detection and recognition module traverses the check-in result database at a preset time point. If there is a non-check-in mark within a period of time before the time point, execute step 10; if within a period of time before the time point, all personnel have signed in or the time has exceeded the preset time point, execute step 12.

[0134] 10. Call the video sequence stored in the video storage module in step 3, and extract the video sequence into images at a frame rate higher than the frame rate in step 2.

[0135] 11. Repeat steps 4 to 8.

[0136] 12. The sign-in result information is displayed in a visual way through mobile client, WEB interface, etc.

[0137] In a second aspect of the embodiment of the present application, a check-in device based on face recognition is provided, see Fig.10 ,include:

[0138] An image acquisition module 1001 is used to acquire an image to be identified;

[0139] The face detection module 1002 is used to detect faces in the image to be recognized, and obtain a face region image and corresponding face feature information;

[0140] The image transformation module 1003 is used to transform the face region image according to the face feature information to obtain a face image of preset specifications;

[0141] The image matching module 1004 is used to match a face image of preset specifications with preset face images corresponding to a plurality of preset persons to obtain matching preset face images;

[0142] The person marking module 1005 is used to mark the preset person corresponding to the matched preset face image as having signed in.

[0143] Optionally, the facial feature information includes location information of facial feature points, and the face detection module 1002 includes:

[0144] The model detection submodule is used to detect the face of the image to be recognized through a preset network model to obtain a face area image;

[0145] The feature point recognition module is used to recognize the facial feature points in the face area image and obtain the position information of the facial feature points.

[0146] Optionally, the image transformation module 1003 includes:

[0147] The matrix calculation submodule is used to calculate the corresponding affine matrix according to the position information of the facial feature points;

[0148] The image transformation submodule is used to transform the face area image through an affine matrix to obtain a face image of preset specifications.

[0149] Optionally, the image acquisition module 1001 includes:

[0150] A video acquisition submodule is used to acquire video images during a preset period of time;

[0151] The first frame extraction submodule is used to extract frames of video images in a preset time period according to a first frame extraction rate to obtain a plurality of images to be identified.

[0152] Optionally, the above device further includes:

[0153] A secondary frame extraction module is used to extract frames of the video images of the preset time period according to a second frame extraction rate to obtain multiple secondary frame extraction images if there are persons who are not marked as having signed in among the multiple preset persons;

[0154] The secondary detection module is used to replace the image to be identified with the secondary frame image, and return to the image to be identified for face detection, and re-detect the step of obtaining the face area image and the corresponding face feature information until the preset stop condition is met.

[0155] Optionally, the above device further includes:

[0156] A sign-out video acquisition module is used to acquire video images during a preset sign-out period;

[0157] A sign-out image acquisition module, configured to extract frames of the video image of a preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified;

[0158] The sign-out video acquisition module is used to detect the face of the sign-out image to be recognized, and obtain the face area image and the corresponding face feature information;

[0159] The sign-out image transformation module is used to transform the face area image according to the face feature information to obtain a face image of preset specifications;

[0160] A sign-out face matching module is used to match a face image of preset specifications with preset face images corresponding to multiple preset persons to obtain matching preset face images;

[0161] The sign-out personnel marking module is used to mark the preset personnel corresponding to the matching preset face image as having signed out.

[0162] It can be seen that, through the device of the embodiment of the present application, the face area image can be transformed according to the face feature information to obtain a face image of a preset specification, and then the face image of the preset specification can be used for recognition and sign-in, which can not only improve the efficiency of sign-in, but also improve the accuracy of sign-in.

[0163] The embodiment of the present invention further provides an electronic device, such as Fig.11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104.

[0164] Memory 1103, used for storing computer programs;

[0165] The processor 1101 is used to implement the following steps when executing the program stored in the memory 1103:

[0166] Obtain an image to be recognized;

[0167] Perform face detection on the image to be recognized to obtain the face area image and the corresponding face feature information;

[0168] Transform the face region image according to the face feature information to obtain a face image of preset specifications;

[0169] Matching a facial image of a preset specification with preset facial images corresponding to a plurality of preset persons to obtain a matching preset facial image;

[0170] The preset person corresponding to the matching preset face image is marked as having signed in.

[0171] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0172] The communication interface is used for communication between the above electronic device and other devices.

[0173] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0174] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0175] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned check-in methods based on face recognition are implemented.

[0176] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes any of the face recognition-based sign-in methods in the above embodiments.

[0177] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0178] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0179] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0180] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A sign-in method based on face recognition, characterized in that: include: Obtain an image to be recognized; Performing face detection on the image to be recognized to obtain a face region image and corresponding face feature information; Transforming the face region image according to the face feature information to obtain a face image of preset specifications; Matching the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image; Marking the preset person corresponding to the matched preset face image as having signed in; The step of obtaining an image to be identified comprises: Obtain video images of the preset check-in period; Extract frames of the video image of the preset check-in time period according to a first frame extraction rate to obtain a plurality of images to be identified; After marking the preset person corresponding to the matched preset face image as having signed in, the method further includes: If there is a person who is not marked as having signed in among the multiple preset persons, frame sampling is performed on the video image of the preset sign-in period according to a second frame sampling rate to obtain a plurality of secondary frame sampling images, wherein the second frame sampling rate is greater than the first frame sampling rate; The image to be identified is replaced with the secondary frame image, and the step of performing face detection on the image to be identified is returned to obtain a face area image and corresponding face feature information and the detection is repeated until a preset stop condition is met.

2. The method according to claim 1, characterized in that The facial feature information includes location information of facial feature points, and the face detection is performed on the image to be identified to obtain a face region image and corresponding facial feature information, including: Performing face detection on the image to be identified by using a preset network model to obtain the face area image; Identify facial feature points in the facial region image and obtain position information of the facial feature points.

3. The method according to claim 2, characterized in that The step of transforming the face region image according to the face feature information to obtain a face image of preset specifications includes: Calculate the corresponding affine matrix according to the position information of the facial feature points; The face region image is transformed by the affine matrix to obtain a face image of preset specifications.

4. The method according to claim 1, characterized in that The method further comprises: Acquire video images of the preset sign-out period; Extract frames of the video image of the preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified; Performing face detection on the sign-out image to be identified to obtain a face region image and corresponding face feature information; Transforming the face region image according to the face feature information to obtain a face image of preset specifications; Matching the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image; The preset person corresponding to the matched preset facial image is marked as having signed out.

5. A sign-in device based on face recognition, characterized in that: include: An image acquisition module, used for acquiring an image to be identified; A face detection module is used to detect faces in the image to be identified, and obtain a face region image and corresponding face feature information; An image transformation module, used to transform the face region image according to the face feature information to obtain a face image of preset specifications; An image matching module is used to match the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain matching preset face images; A personnel marking module, used to mark a preset person corresponding to the matched preset face image as having signed in; The image acquisition module comprises: A video acquisition submodule is used to acquire video images during a preset period of time; A first frame extraction submodule, configured to extract frames of the video images in the preset time period according to a first frame extraction rate to obtain a plurality of images to be identified; A secondary frame extraction module, configured to extract frames of the video images of the preset time period according to a second frame extraction rate to obtain a plurality of secondary frame extraction images if there are persons who are not marked as having signed in among the plurality of preset persons; The secondary detection module is used to replace the image to be identified with the secondary frame image, and return to the step of performing face detection on the image to be identified to obtain the face area image and the corresponding face feature information and re-detect until a preset stop condition is met.

6. The device according to claim 5, characterized in that The facial feature information includes location information of facial feature points, and the face detection module includes: The model detection submodule is used to detect the face of the image to be recognized through a preset network model, and obtain The face area image; The feature point recognition module is used to recognize the facial feature points in the facial region image and obtain the position information of the facial feature points.

7. The device according to claim 6, characterized in that The image transformation module comprises: A matrix calculation submodule, used to calculate the corresponding affine matrix according to the position information of the facial feature points; The image transformation submodule is used to transform the face area image through the affine matrix to obtain a face image of preset specifications.

8. The device according to claim 5, characterized in that The device also includes: A sign-out video acquisition module is used to acquire video images during a preset sign-out period; A sign-out image acquisition module, configured to extract frames of the video image of the preset sign-out period according to a first frame extraction rate to obtain a plurality of sign-out images to be identified; A sign-out video acquisition module is used to detect the face of the sign-out image to be identified, and obtain a face area image and corresponding face feature information; A sign-out image transformation module, used to transform the face region image according to the face feature information to obtain a face image of preset specifications; A sign-out face matching module is used to match the face image of the preset specification with the preset face images corresponding to a plurality of preset persons to obtain a matching preset face image; The sign-out personnel marking module is used to mark the preset personnel corresponding to the matched preset facial image as having signed out.

9. An electronic device, characterized in that: The invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the method steps described in any one of claims 1 to 4 when executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images