Face recognition method and device

Abstract

The disclosed embodiment provides a face recognition method and device, wherein the method comprises: determining a target face image sequence of a target face, wherein the target face image sequence comprises a plurality of face region images of the target face; performing face recognition on at least one face region image in the target face image sequence, and determining the target face region image in the target face image sequence based on the confidence of the face recognition result of the at least one face region image; and determining the identity information of the target face based on the face recognition result of the target face region image. The method improves the recognition success rate of the face region image and reduces the cost of face recognition.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to Singapore patent application number 10202110328W, filed on September 19, 2021, with invention name “Face Recognition Method and Device”, the full text of which is incorporated herein by reference. Technical Field

[0003] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to a face recognition method and device. Background Art

[0004] Face recognition is the most basic and important part of intelligent video analysis. It is necessary to track the target object in the video for a long time, and when receiving the recognition request from the upper-layer application, perform face recognition and identity determination on the tracked target object. Summary of the invention

[0005] In view of this, embodiments of the present disclosure provide at least one face recognition method and device.

[0006] In a first aspect, a face recognition method is provided, the method comprising:

[0007] Determine a target facial image sequence of a target face, wherein the target facial image sequence includes a plurality of facial region images of the target face;

[0008] Performing face recognition on at least one face region image in the target face image sequence, and determining a target face region image in the target face image sequence based on a confidence level of a face recognition result of the at least one face region image;

[0009] Based on the face recognition result of the target facial area image, the identity information of the target face is determined.

[0010] In a second aspect, a face recognition device is provided, the device comprising:

[0011] A target facial image sequence determination module, used to determine a target facial image sequence of a target face, wherein the target facial image sequence includes a plurality of facial region images of the target face;

[0012] a target facial region image determination module, configured to perform face recognition on at least one facial region image in the target facial image sequence, and determine the target facial region image in the target facial image sequence based on the confidence of the face recognition result of the at least one facial region image;

[0013] The identity information determination module is used to determine the identity information of the target face based on the face recognition result of the target face area image.

[0014] According to a third aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions executable on the processor, and the processor is used to implement the face recognition method described in the first aspect when executing the computer instructions.

[0015] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the face recognition method described in the first aspect is implemented.

[0016] In the face recognition method provided by the technical solution of the embodiment of the present disclosure, face recognition is performed on the face area images in the target face image sequence of the target face to determine the target face area image, and then determine the identity information of the target face, and use the target face image sequence as a quality selection model. When one of the face area images cannot be recognized, other face area images in the target face image sequence can continue to be recognized, which greatly improves the recognition success rate of the face area images, that is, the recall rate; compared with the method of only recognizing high-quality face area images selected by the face quality model, the method described in the present disclosure does not need to perform high-cost large-scale data collection to train the face quality model in order to exhaust all low-quality face situations, thereby reducing the cost of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in one or more embodiments of the present disclosure or related technologies, the drawings required for use in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 is a flow chart of a face recognition method shown in an embodiment of the present disclosure;

[0019] Figure 1A is a flow chart of a method for determining a target facial image sequence of a target face shown in an embodiment of the present disclosure;

[0020] Figure 2 is a flow chart of another face recognition method shown in an embodiment of the present disclosure;

[0021] Figure 3 is a block diagram of a face recognition device shown in an embodiment of the present disclosure;

[0022] Figure 4 is a block diagram of another face recognition device shown in an embodiment of the present disclosure;

[0023] Figure 5 It is a schematic diagram of the hardware structure of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] For face recognition based on video tracking, the key step is how to select high-quality face images for face recognition from a large number of face image sequences in the video. If the quality of the face image to be recognized is too poor, it cannot be matched with any identity information in the face database, and an erroneous result is obtained, that is, the face image cannot be recalled.

[0028] Traditional methods collect a large amount of data to train a face quality model in order to select high-quality face images from a face image sequence, and then use the selected high-quality face images for face recognition. However, in actual applications, the environment in which the target object is located is complex, and the target object can have many movements, such as turning the head, lowering the head, covering the face with hands, looking in the mirror, or wearing a mask, etc. It is difficult to exhaust all scenarios to train a face quality model through data collection.

[0029] During the entire video tracking process, in addition to low-quality facial images in certain time periods due to the target object turning its head, covering its head with its hands, or due to the presence of mirrored faces in reflective objects such as glass, mirrors, and tiles, there are also sufficient high-quality facial images. The method of selecting high-quality facial images only through the face quality model, especially the single face image selection method, cannot guarantee that high-quality facial images required for face recognition will be selected, such as face images that meet the requirements of clear face, no obstruction, and no rotation. If no result is recognized for the selected image, the face recognition is directly declared to have failed and cannot be remedied. Therefore, in order to improve the success rate of face recognition, a large amount of data needs to be collected to train the face quality model, but the large amount of data collection requires high costs.

[0030] To this end, the embodiments of the present disclosure provide a face recognition method, which can reduce the impact of low-quality face images without collecting a large amount of data, greatly improve the face recognition accuracy, and ensure the recall rate of face images.

[0031] like Figure 1 As shown, Figure 1 : is a flow chart of a face recognition method shown in an embodiment of the present disclosure, the method comprising the following steps:

[0032] In step 102, a target facial image sequence of a target face is determined, wherein the target facial image sequence includes a plurality of facial region images of the target face.

[0033] In this step, the facial region image may be an image containing the target face of the target object detected from the tracking video when the target object is being face tracked, or may be an image containing the target face of the target object obtained by photographing the target object. Here, the target face is the face of a designated or undesignated person to be identified.

[0034] The face recognition method of this embodiment may be executed by a face recognition device, for example, by a terminal device or a server or other processing device, wherein the terminal device may be a user device, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant, a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the face recognition method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0035] This embodiment does not limit the specific method of determining the target facial image sequence. For example, the target facial image sequence corresponding to the target face may be selected from facial image sequences of multiple different faces pre-maintained by the device; or the target facial image sequence may be directly obtained from other devices.

[0036] In addition, determining the target facial image sequence in this step can be performed at any time. For example, the method of this embodiment can be performed when a face recognition request message is received, and the face recognition request message includes information about the target face; or the method can be performed when a tracking video or continuously shot images are received.

[0037] In step 104, face recognition is performed on at least one face region image in the target face image sequence, and a target face region image in the target face image sequence is determined based on the confidence of the face recognition result of the at least one face region image.

[0038] The face recognition of the face region image may be performed by sequentially comparing a face region image with a plurality of face images in a face library to obtain comparison results. Each comparison result obtained includes a confidence level, which indicates the probability that the face in the face region image and the face in the image in the face library are the same face of the same object, and the comparison result with the highest confidence level is determined as the face recognition result of the face region image.

[0039] If the confidence of the face recognition result of the facial area image reaches the preset confidence threshold, that is, the face corresponding to the facial area image is found in the face library, then the face recognition is successful, and the facial area image is determined to be the target facial area image; if the confidence of the face recognition result of the facial area image does not reach the preset confidence threshold, that is, the face corresponding to the facial area image is not found in the face library, then the face recognition fails, and another facial area image that has not been selected in the target facial image sequence is continuously selected for face recognition until the face recognition is successful.

[0040] If, after performing face recognition on all facial area images in the target facial image sequence, no face corresponding to the facial area image is found in the face library, the facial area image corresponding to the face recognition result closest to successful recognition can be selected from all face recognition results as the target facial area image, that is, the facial area image corresponding to the face recognition result with the highest confidence level can be selected as the target facial area image, or the face recognition can be terminated.

[0041] In this step, the order of performing face recognition on the facial region images in the target facial image sequence can be arbitrary, such as randomly selecting one image in the target facial image sequence for recognition; or the facial region images in the target facial image sequence can be selected in a preset order, such as from high to low image quality, for face recognition.

[0042] This embodiment does not limit the specific face recognition method. For example, recognition can be performed through a neural network, or through other methods.

[0043] In step 106, based on the face recognition result of the target facial region image, the identity information of the target face is determined.

[0044] The face recognition result of the target face region image includes the face image corresponding to the target face region image in the face database, and the identity information corresponding to the face image is determined as the identity information of the target face. The identity information can be the certificate number, name, registration account, etc. pre-stored in the face database.

[0045] In the face recognition method provided in the present embodiment, face recognition is performed on multiple face area images in a target face image sequence to determine the target face area image, and then determine the identity information of the target face. Although low-quality images may be extracted for recognition from the multiple face area images, when one of the face area images cannot be recognized, the next face area image in the target face image sequence can be used for recognition, which is equivalent to using the target face image sequence as a quality selection model, greatly improving the recognition success rate of the face area image, that is, the recall rate; compared with the method of only recognizing high-quality face area images selected by the face quality model, this method does not need to perform high-cost large-scale data collection to train the face quality model in order to exhaust all low-quality face situations, thereby reducing the cost of face recognition.

[0046] In one embodiment, the method may be used to identify a target face in a video stream, and the target facial image sequence of the target face determined in step 102 may include multiple facial region images of the target face extracted from multiple frames of the video stream.

[0047] The video stream may be a recorded video or a real-time video, and the multiple frames of the video stream may include the target face of the target object. In this embodiment, the video stream may be a tracking video obtained by performing face tracking on the target object.

[0048] In one example, before step 102, a video stream may be acquired in advance. Face detection is performed based on each frame of the acquired video stream to detect faces in the video stream. For each of the multiple faces detected in the video stream, the detected face is tracked based on the video stream to determine the facial region image of the face in the multiple frames of the video stream; and a facial image sequence of the face is generated based on the facial region image of the face in the multiple frames of the video stream. In this way, facial image sequences of multiple different faces appearing in the video stream may be acquired in advance, and in subsequent processing, a target facial image sequence of a target face may be conveniently and quickly determined from the facial image sequences of multiple different faces.

[0049] In this example, for each of the multiple faces detected in the video stream, after determining the face area image of the face in the multiple frames of the video stream, a face identifier of the face can also be generated; based on the face identifier, a facial image sequence consisting of the facial area images corresponding to the face can be quickly determined. Determining the target facial image sequence in this step can be performed after obtaining a face recognition request, and the face recognition request can be sent by an upper-layer application. When a face recognition request message is received, the face identifier in the obtained face recognition request is used as the face identifier of the target face, and the target facial image sequence is determined from multiple facial image sequences based on the face identifier of the target face.

[0050] This example does not restrict the face tracking method used. For example, face tracking can be performed using a Kalman filter or a neural network.

[0051] In one example, step 102 determines a target facial image sequence of a target face, such as Figure 1A As shown, the following steps may be specifically included:

[0052] In step 1021, facial region images of a target face in multiple frames of a video stream are determined as multiple candidate facial region images of the target face.

[0053] For example, when the video stream is a tracking video, the candidate facial region images in the tracking video can be obtained through face detection. One or more candidate facial region images can be detected in each frame of the tracking video, or the candidate facial region images may not be detected for some of the frames. Through face tracking, multiple candidate facial region images corresponding to the same face can be obtained, and multiple candidate facial region images of the target face are determined. In the case where multiple candidate facial region images are detected in one frame, these multiple candidate facial region images may include a physical facial region image and one or more mirrored facial region images.

[0054] In step 1022, the image qualities of the plurality of candidate facial region images are determined.

[0055] Exemplarily, for each of the plurality of candidate facial region images, a pre-trained facial image quality assessment model may be used to assess the quality of the candidate facial region image and determine the image quality of the candidate facial region image. For example, the candidate facial region image may be input into a pre-trained facial image quality assessment model to obtain an image quality assessment result of the candidate facial region image. Image quality may be an assessment of the facial region image based on any of several factors such as comprehensive image intelligibility, brightness, clarity, facial symmetry, and noise. The image quality assessment result may be expressed using grades, scores, etc., or may be expressed in other ways.

[0056] A sequence can be used to represent the image quality evaluation results of multiple candidate facial region images of the target face, such as sequence A = {a n ,n=1,2,3,…,N},a n represents the image quality evaluation result of the nth candidate facial region image, where N represents the total number of frames of the candidate facial region images of the target face.

[0057] It should be noted that the facial image quality assessment model used in this embodiment can be a model of a certain accuracy obtained through conventional training. For example, a binary classifier based on deep learning can be used to train the model through general facial quality data, without having to spend a high cost to collect a large amount of data to train a high-precision facial quality model.

[0058] In step 1023, the plurality of candidate facial region images are sorted according to the image quality evaluation result to obtain a first sequence.

[0059] For example, multiple candidate facial region images may be sorted from high quality to low quality or from low quality to high quality to obtain a first sequence. Continuing with the above example, sequence A is sorted in reverse order according to the image quality scores to obtain sequence B. In this example, sequence B represents the first sequence.

[0060] In step 1024, the target facial image sequence is determined based on the first sequence.

[0061] The first sequence can be directly determined as the target facial image sequence; or the first subsequence of the first sequence can be determined as the target facial image sequence, wherein the first subsequence includes a preset number of candidate facial region images that meet preset image quality requirements. By further screening the images in the first sequence based on image quality and only selecting a subsequence including a preset number of candidate facial region images as the target facial image sequence, the quality of the target facial image sequence can be further improved, and the number of images in the target facial image sequence can be effectively controlled, which helps to improve the accuracy and recognition efficiency of the face recognition result of the target face.

[0062] In some embodiments, the first sequence may be subjected to sparse processing according to the corresponding time sequence of multiple candidate facial region images in the first sequence in the video stream to obtain a second sequence, and the target facial image sequence may be determined according to the second sequence.

[0063] The interval distance between corresponding time sequences of any two adjacent candidate facial region images in the second sequence in the video stream is greater than a preset time interval threshold.

[0064] The following sparse processing method may be used to remove some candidate facial region images in sequence B to obtain sequence C, so that the interval distance between any two adjacent candidate facial region images in sequence C is greater than a preset time interval threshold step:

[0065] a) Traverse the values ​​in sequence B and record the sequence number of the current candidate facial region image as i.

[0066] b) For each i, traverse the values ​​of sequence B starting from i and record the sequence number of the current candidate facial region image as j.

[0067] c) If B i and B j If the interval distance is less than the preset time interval threshold step, then delete B j .

[0068] d) After completing the loop traversal of i, all candidate facial region images in the remaining sequence B are assigned to sequence C.

[0069] e) End.

[0070] In one embodiment, assuming that sequence B includes the following 6 candidate facial region images, B1, B2, B3, B4, B5, B6, and the preset time interval threshold is 0.05 ms, when i is 1, the time intervals between B1 and B2, B3, B4, B5, B6 are calculated in sequence, the time interval between B1 and B2 is 0.02 ms, the time interval between B1 and B3 is 0.06 ms, the time interval between B1 and B4 is 0.15 ms, the time interval between B1 and B5 is 0.04 ms, and the time interval between B1 and B6 is 0.10 ms. In this case, B2 and B5 in sequence B can be deleted, and the remaining elements in sequence B are B1, B3, B4, B6. When i is 3, the time intervals between B3 and B4 and B6 are calculated in turn. The time interval between B3 and B4 is 0.09ms, and the time interval between B3 and B6 is 0.04ms. In this case, B6 in sequence B can be deleted, and the remaining elements in sequence B are B1, B3, and B4. At this time, the loop traversal of i is completed, and the elements in sequence C include B1, B3, and B4.

[0071] In actual implementation, the time sequence can be represented by a time tag. The candidate facial region image is a part of an image frame of a video stream, and the time tag of the candidate facial region image can be the time tag of its corresponding image frame in the video stream.

[0072] For the target object in the tracking video, when the target object makes some actions such as turning the head, covering the head with hands or drinking water, the image captured by the tracking video will not be suitable for face recognition. Generally, these actions will last for a period of time, and the image frames that are continuous in time sequence in the tracking video are usually similar, and the image quality is generally similar. Repeated face recognition of similar images is not valuable. Therefore, the collected candidate facial area images can be processed in time sequence to obtain a second sequence, thereby avoiding multiple face recognition of low-quality images and improving the efficiency of face recognition.

[0073] This embodiment does not limit the specific method of performing the time-series sparse processing, and any method that can achieve that the interval distance between any two adjacent candidate facial region images in the second sequence is greater than the preset time interval threshold can be adopted. For example, multiple time tags can be calculated according to the preset time interval threshold, and the candidate facial region images in the first sequence within the difference range of the above multiple time tags are combined into the second sequence; for another example, the first sequence can be traversed in order, and the adjacent candidate facial region images whose interval distance is less than the preset time interval threshold are removed to obtain the second sequence.

[0074] After the second sequence is obtained, the target facial image sequence is determined according to the second sequence. The second sequence may be directly determined as the target facial image sequence, or a second subsequence may be extracted from the second sequence and determined as the target facial image sequence, wherein the second subsequence includes a preset number of candidate facial region images that meet preset image quality requirements.

[0075] The image quality requirement may be a requirement for image quality ranking. Exemplarily, when the second sequence is candidate facial region images sorted in reverse order from high to low image quality, a subsequence consisting of a first preset number of candidate facial region images in the second sequence may be determined as the target facial image sequence; or, when the second sequence is candidate facial region images sorted in positive order from low to high image quality, a subsequence consisting of a last preset number of candidate facial region images in the second sequence may be determined as the target facial image sequence.

[0076] The preset number can be represented by K and is set by those skilled in the art according to actual needs. The first K candidate facial region images of sequence C are taken as sequence D, that is, the target facial image sequence finally determined. In another example, when the first sequence is sorted from low to high, the last K elements of sequence C can be taken as the target facial image sequence.

[0077] The image quality requirement may also be a requirement for the level or score of the image quality, that is, the image quality is required to meet a preset level or preset score. This embodiment does not limit the image quality requirement. By further selecting a preset number of candidate facial region images from the second sequence as the target facial image sequence through the facial quality requirement, the target facial region images corresponding to the target face can be further streamlined, so that the target facial image sequence finally obtained is a temporally dispersed and high-quality sequence, thereby improving the efficiency of face recognition. After obtaining the target facial image sequence, face recognition can be performed on the facial region images in descending order of image quality.

[0078] In another example, determining the target facial image sequence may be obtaining the target facial image sequence maintained by other devices, where the other devices may be terminal devices, servers or other processing devices, and the other devices may perform Figure 1A Steps 1021 to 1024 are shown to obtain a target facial image sequence.

[0079] Figure 2 A face recognition method according to another embodiment of the present disclosure is provided, and the method may include the following processing, wherein the steps that are the same as the process of the above embodiment will not be described in detail again.

[0080] In step 202, a target facial image sequence of a target face is determined, and image qualities of a plurality of facial region images are determined.

[0081] The target face image sequence includes a plurality of face region images of the target face.

[0082] A pre-trained face image quality assessment model with a predetermined accuracy can be used to assess the image quality of the face area image to obtain an evaluation result.

[0083] For example, after determining the target facial image sequence, the quality of the facial region images in the target facial image sequence may be evaluated to obtain an evaluation result.

[0084] The image quality of the facial region image may also be determined before determining the target facial image sequence. For example, multiple candidate facial region images of the target face may be obtained from multiple frames of the video stream, the quality of the candidate facial region images may be evaluated to obtain evaluation results, and then the facial region images of the target facial image sequence may be determined from the candidate facial region images according to the evaluation results.

[0085] In step 204, when the target facial region image is not determined, facial recognition is performed on the first facial region image in the target facial image sequence that has not been subjected to facial recognition and has the highest image quality.

[0086] In the case that the target facial region image has not been determined, the facial region images in the target facial image sequence may be identified in sequence according to the image quality order to obtain a face recognition result.

[0087] For example, the facial region images in the target facial image sequence D can be sequentially input into the face recognition model as the first facial region images to extract the facial features in the facial region images, and compared in the face database based on the facial features, and the confidence level can be used to represent the comparison result. For each facial region image in the facial image sequence D, when the image is compared in the face database, multiple confidence levels can be obtained, wherein the comparison result with the highest confidence level is determined as the face recognition result of the facial region image.

[0088] In step 206, in response to determining that the confidence level of the face recognition result of the first facial region image exceeds a preset threshold, the first facial region image is determined to be the target facial region image.

[0089] When the confidence level of the face recognition result of the first face area image exceeds a preset threshold, the face recognition is successful, the first face image is determined as the target face area image, and face recognition is no longer performed on other face area images in the target face image sequence.

[0090] When the confidence of the face recognition result of the first face area image does not exceed the preset threshold, and the first face area image is not the last face area image in the target face image sequence, face recognition is continued on the first face area image in the target face image sequence that has not been subjected to face recognition and has the highest image quality.

[0091] In step 208, in response to determining that the confidences of the face recognition results of all the face area images in the target face image sequence are less than a preset threshold, a second face area image corresponding to the maximum value of the confidences of the face recognition results is determined, and the second face area image is used as the target face area image.

[0092] When face recognition is performed on all facial area images in the target facial image sequence, and the confidence levels of the face recognition results of the facial area images are all less than a preset threshold, a second facial area image corresponding to the maximum value of the confidence levels of the face recognition results of all facial area images is taken as the target facial area image.

[0093] In step 210, based on the face recognition result of the target facial region image, the identity information of the target face is determined.

[0094] The identity information associated with the face recognition result of the target facial area image in the face database is used as the identity information of the target face, thereby determining the identity information of the target object through face recognition.

[0095] In the face recognition method provided in the present embodiment, in the process of identifying multiple face area images in the target face image sequence, the face area image with the highest image quality is first identified. If it cannot be identified, the face area image with the highest image quality in the remaining images of the target face image sequence is continued to be used for identification, which is equivalent to using the target face image sequence as an alternative quality selection model, greatly improving the recognition success rate of face area images and the recall rate of high-quality face area images; compared with the method of only identifying high-quality face area images selected by the face quality model, this method does not need to collect a large amount of data at a high cost to train the face quality model in order to exhaust all low-quality face images, thereby reducing the cost of face recognition.

[0096] In one embodiment, the face recognition method provided by the embodiment of the present disclosure can be applied in the environment of a gaming venue. In the scenario of a smart gaming venue, it is necessary to track the seated players for a long time, and when receiving the recognition request sent by the upper-layer application, the face recognition of the tracked players is performed to determine their identities. However, the environment of the gaming venue is very complex, such as large changes in lighting and rich player postures. It is difficult to exhaust all problem scenarios through data collection to train the face quality model, so as to select high-quality face images for face recognition.

[0097] The following describes the application of a face recognition method provided by an embodiment of the present disclosure in a gaming venue.

[0098] First, the face detection model, face image quality assessment model and face recognition model need to be pre-trained.

[0099] For the face detection model, you can use common face detection models, such as Retina Net, YoloV3 or PCN, and use general face data to complete training. You can also use general face data and game venue-specific face data for training to improve model accuracy.

[0100] For the face image quality assessment model, a binary classifier based on deep learning can be used, and training can be completed using general face data, or it can be trained using general face quality data and game venue-specific face data to improve model accuracy. For example, binary cross entropy loss can be used as a loss function during training.

[0101] For face recognition models, you can use common neural networks, such as resnet50, SqueezeNet, etc., and use common face recognition data to complete training. During training, you can use common face recognition loss functions, such as arcface.

[0102] Based on the above prepared model, the following processing can be performed on the tracking video captured in real time by the camera of the game place to maintain a temporally dispersed and high-quality facial image sequence. In other examples, the tracking video captured in advance by the camera of the game place can also be processed.

[0103] Face detection: Input each frame image in the tracking video into the face detection model in turn to obtain the face detection frame in each frame image. The image in the face detection frame is the candidate facial area image.

[0104] Image quality assessment: The candidate facial region images are input into the facial image quality assessment model frame by frame in real time to obtain the image quality scores of the candidate facial region images.

[0105] Face tracking: Through a fast face tracking solution, such as Kalman filtering, the quality scores of all candidate facial area images corresponding to the same player's face can be obtained, which can be represented by a quality score sequence.

[0106] Selection of high-quality facial image sequence: The sequence can be sorted in reverse order of quality score, and then subjected to time-series sparse processing so that the interval between any two adjacent elements in the sequence is greater than a preset time interval threshold, and then the first K elements of the sequence are taken as the final facial image sequence, which contains K facial region images sorted from high to low image quality.

[0107] It should be noted that if there are N players in the tracking video, it is necessary to maintain the corresponding N facial image sequences. When the tracking video grows over time, the maintained facial image sequences will also be updated together. When a face recognition request is received from the upper-layer application, the face recognition of the player specified in the face recognition request is performed.

[0108] The facial region images in the facial image sequence corresponding to the player are input into the face recognition model one by one in order for feature extraction and identity retrieval. If the retrieval confidence of the current facial region image is greater than the preset threshold, the retrieval is successful, and the identity information associated with the retrieval result corresponding to the retrieval confidence in the face database is output. If the retrieval confidence of the current facial region image is less than the preset threshold, the retrieval confidence is recorded and the next facial region image is input into the face recognition model. If the current facial region image is already the last facial region image in the facial image sequence, the identity information associated with the retrieval result corresponding to the highest retrieval confidence among the recorded retrieval confidences is output in the face database.

[0109] In the scenario of a smart gaming venue, if you need to identify the identity of the players captured by the camera, you can use this solution to achieve the goal, which can ensure higher recognition accuracy. Based on the recall rate of high-quality facial area images and the recall rate of player faces, you can better carry out subsequent work, such as game table monitoring and user analysis.

[0110] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0111] like Figure 3 As shown, Figure 3 1 is a block diagram of a face recognition device shown in an embodiment of the present disclosure, wherein the device comprises: a target face image sequence determination module 31 , a target face region image determination module 32 , and an identity information determination module 33 .

[0112] The target facial image sequence determining module 31 is used to determine a target facial image sequence of a target face, wherein the target facial image sequence includes a plurality of facial region images of the target face.

[0113] The target facial region image determination module 32 is used to perform face recognition on at least one facial region image in the target facial image sequence, and determine the target facial region image in the facial image sequence based on the confidence of the face recognition result of the at least one facial region image.

[0114] The identity information determination module 33 is used to determine the identity information of the target face based on the face recognition result of the target face area image.

[0115] In one example, the target facial image sequence includes multiple facial region images of the target face extracted from multiple frames of images in a video stream.

[0116] In an example, Figure 4 As shown, based on the above device embodiment, the device further includes: a facial image sequence generating module 30.

[0117] The facial image sequence generation module 30 is used to perform face detection based on each frame image in the acquired video stream; for each of the multiple faces detected in the video stream, track the detected face based on the video stream, and determine the facial area image of the face in each frame of the multiple frames of the video stream; based on the facial area image of the face in the multiple frames of the video stream, generate a facial image sequence of the face.

[0118] In one example, the target facial image sequence determination module 31 is specifically used to: determine that the facial area images of the target face in the multiple frames of the video stream are multiple candidate facial area images of the target face; determine the image quality of the multiple candidate facial area images; sort the multiple candidate facial area images according to the image quality to obtain a first sequence; and determine the target facial image sequence based on the first sequence.

[0119] In one example, the target facial image sequence determining module 31 is further configured to: determine a first subsequence of the first sequence as the target facial image sequence, the first subsequence comprising a preset number of candidate facial region images meeting preset image quality requirements.

[0120] In one example, the target facial image sequence determination module 31 is further used to: perform a sparse processing on the first sequence according to the corresponding time sequence of the multiple candidate facial area images in the first sequence in the video stream to obtain a second sequence, wherein the interval distance between the corresponding time sequence of any adjacent candidate facial area images in the second sequence in the video stream is greater than a preset time interval threshold; and determine the target facial image sequence according to the second sequence.

[0121] In one example, the target facial image sequence determining module 31 is further configured to: determine a second subsequence of the second sequence as the target facial image sequence, wherein the second subsequence includes a preset number of candidate facial region images that meet preset image quality requirements.

[0122] In one example, the target facial image sequence determination module 31, when used to determine the image quality of the multiple candidate facial area images, is specifically used to: for each of the multiple candidate facial area images, use a pre-trained facial image quality assessment model to perform quality assessment on the candidate facial area image, and determine the image quality evaluation result of the candidate facial area image.

[0123] In one example, the facial image sequence generation module 30 is used to: generate a facial identifier of the face after tracking the detected face in the video stream for each of the multiple faces detected in the video stream and determining the facial region image of the face in multiple frames of the video stream. The target facial image sequence determination module 31 is used to: use the facial identifier in the acquired face recognition request as the facial identifier of the target face; and determine the target facial image sequence in the facial image sequence of the multiple faces according to the facial identifier of the target face.

[0124] In one example, the target facial image sequence determination module 31 is further used to determine the image quality of the multiple facial region images. The target facial region image determination module 32 is specifically used to: in response to determining that the confidence of the face recognition result of the first facial region image exceeds a preset threshold, determine that the first facial region image is the target facial region image.

[0125] In one example, the target facial area image determination module 32 is further used to: in response to determining that the confidences of the face recognition results of all facial area images in the target facial image sequence are less than a preset threshold, determine a second facial area image with a maximum confidence face recognition result; and use the second facial area image as the target facial area image.

[0126] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0127] The present disclosure also provides an electronic device, such as Figure 5 As shown, the electronic device includes a memory 51 and a processor 52, wherein the memory 51 is used to store computer instructions that can be executed on the processor, and the processor 52 is used to implement the face recognition method described in any embodiment of the present disclosure when executing the computer instructions.

[0128] The embodiments of the present disclosure also provide a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the face recognition method described in any embodiment of the present disclosure is implemented.

[0129] The embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the face recognition method described in any embodiment of the present disclosure is implemented.

[0130] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0131] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0133] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0134] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A face recognition method, comprising: Perform face detection based on each frame image in the acquired video stream; For each of the multiple faces detected in the video stream, Tracking the face based on the video stream to determine a facial region image of the face in multiple frames of the video stream; Generate a facial image sequence of the human face based on facial region images of the human face in multiple frames of images of the video stream; Determine that the facial region images of the target face in the multiple frames of the video stream are multiple candidate facial region images of the target face; Determining the image quality of the plurality of candidate facial region images; sorting the plurality of candidate facial region images from high to low according to the image quality to obtain a first sequence; According to the corresponding time sequence of the plurality of candidate facial region images in the first sequence in the video stream, performing time sequence thinning processing on the first sequence to obtain a second sequence, wherein the interval between any two adjacent candidate facial region images in the second sequence is greater than a preset time interval threshold; determining a second subsequence of the second sequence as the target facial image sequence, the second subsequence comprising a preset number of candidate facial region images meeting preset image quality requirements; Performing face recognition on at least one face region image in the target face image sequence; determining a target face region image in the target face image sequence based on the confidence of the face recognition result of the at least one face region image; Determining the identity information of the target face based on the face recognition result of the target face area image; The step of performing face recognition on at least one face region image in the target face image sequence and determining the target face region image in the target face image sequence based on the confidence of the face recognition result of the at least one face region image comprises: In the case where the target facial region image is not determined, performing facial recognition on the first facial region image which has not been subjected to facial recognition and has the highest image quality in the target facial image sequence; When the confidence level of the face recognition result of the first facial region image exceeds a preset threshold, the face recognition is successful, the first facial image is determined as the target facial region image, and face recognition is no longer performed on other facial region images in the target facial image sequence that have not been subjected to face recognition; When the confidence level of the face recognition result of the first face region image does not exceed the preset threshold, and the first face region image is not the last face region image in the target face image sequence, then continue to perform face recognition on the first face region image in the target face image sequence that has not been subjected to face recognition and has the highest image quality, until the confidence level of the face recognition result of the first face region image exceeds the preset threshold; In response to determining that the confidences of the face recognition results of all the facial region images in the target facial image sequence are less than a preset threshold, a second facial region image with the maximum confidence is determined, and the second facial region image is used as the target facial region image.

2. The method according to claim 1, wherein: The determining the image quality of the plurality of candidate facial region images comprises: For each of the plurality of candidate facial region images, The quality of the candidate facial region image is assessed using a pre-trained facial image quality assessment model to determine an image quality assessment result of the candidate facial region image.

3. The method according to claim 1, wherein: For each of the multiple faces detected in the video stream, after tracking the face based on the video stream and determining a facial region image of the face in multiple frames of images in the video stream, the method further includes: generating a facial identifier for the face; The target facial image sequence of the target face is determined, comprising: Using the facial identifier in the obtained face recognition request as the facial identifier of the target face; The target facial image sequence of the target face is determined from the facial image sequences of the multiple faces according to the facial identifier of the target face.

4. A face recognition device, comprising: Facial image sequence generation module for Perform face detection based on each frame image in the acquired video stream; For each of the plurality of faces detected in the video stream, Tracking the face based on the video stream to determine a facial region image of the face in multiple frames of the video stream; Generate a facial image sequence of the human face based on facial region images of the human face in multiple frames of images of the video stream; Target facial image sequence determination module, used for Determine that the facial region images of the target face in the multiple frames of the video stream are multiple candidate facial region images of the target face; Determining the image quality of the plurality of candidate facial region images; sorting the plurality of candidate facial region images from high to low according to the image quality to obtain a first sequence; According to the corresponding time sequence of the plurality of candidate facial region images in the first sequence in the video stream, performing time sequence thinning processing on the first sequence to obtain a second sequence, wherein the interval between any adjacent candidate facial region images in the second sequence is greater than a preset time interval threshold; determining a second subsequence of the second sequence as the target facial image sequence, the second subsequence comprising a preset number of candidate facial region images meeting preset image quality requirements; a target facial region image determination module, configured to perform face recognition on at least one facial region image in the target facial image sequence, and determine the target facial region image in the target facial image sequence based on the confidence of the face recognition result of the at least one facial region image; An identity information determination module, used to determine the identity information of the target face based on the face recognition result of the target face area image; Wherein, the target facial area image determination module is further used for: In the case where the target facial region image is not determined, performing facial recognition on the first facial region image which has not been subjected to facial recognition and has the highest image quality in the target facial image sequence; When the confidence level of the face recognition result of the first facial region image exceeds a preset threshold, the face recognition is successful, the first facial image is determined as the target facial region image, and face recognition is no longer performed on other facial region images in the target facial image sequence that have not been subjected to face recognition; When the confidence level of the face recognition result of the first face region image does not exceed the preset threshold, and the first face region image is not the last face region image in the target face image sequence, then continue to perform face recognition on the first face region image in the target face image sequence that has not been subjected to face recognition and has the highest image quality, until the confidence level of the face recognition result of the first face region image exceeds the preset threshold; In response to determining that the confidences of the face recognition results of all the facial region images in the target facial image sequence are less than a preset threshold, a second facial region image with the maximum confidence is determined, and the second facial region image is used as the target facial region image.

5. An electronic device, comprising a memory and a processor, wherein: The memory is used to store computer instructions executable on a processor, and the processor is used to implement the method according to any one of claims 1 to 3 when executing the computer instructions.

6. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

7. A computer program, comprising computer readable codes, wherein when the computer readable codes are executed on a processor in an electronic device, the processor executes the method according to any one of claims 1 to 3.

Images