Face recognition method and device, embedded device and computer readable storage medium

Abstract

The application discloses a face recognition method, a face recognition device, an embedded device and a nonvolatile computer readable storage medium. The face recognition method comprises the following steps: detecting face information in a collected image to generate a face image; detecting the quality of the face image based on a preset face screening model; and calculating the similarity between a face feature vector of the face image and a preset face feature vector based on a reconfigurable computing unit, so as to perform face recognition. The face recognition method, the face recognition device, the embedded device and the nonvolatile computer readable storage medium can eliminate some face images with poor quality, thereby reducing the number of face images that need to be recognized, and improving the recognition efficiency. The reconfigurable computing unit can reduce the resource occupation rate and time consumption of the embedded device in the face recognition process, so as to meet the operation requirements of the embedded device in other work.

Description

Technical Field

[0001] This application relates to the field of face recognition technology, and more specifically, to a face recognition method, a face recognition device, an embedded device, and a non-volatile computer-readable storage medium. Background Technology

[0002] In recent years, with the rapid improvement of computer performance and the continuous refinement of deep learning methods, significant breakthroughs have been achieved in both pattern recognition and artificial intelligence. Deep learning methods have yielded excellent results in many pattern recognition tasks, and face recognition is no exception. However, in the face recognition process, facial features must first be extracted and searched in a facial feature database. But if the amount of facial feature data is too large, it will lead to low face recognition efficiency. Summary of the Invention

[0003] This application provides a face recognition method, a face recognition device, an embedded device, and a non-volatile computer-readable storage medium.

[0004] The face recognition method according to the present application includes detecting face information in a captured image to generate a face image; detecting the quality of the face image based on a preset face screening model; and calculating the similarity between the face feature vector of a face image whose quality meets a preset condition and a preset face feature vector based on a reconfigurable computing unit, so as to perform face recognition.

[0005] The face recognition device according to this application includes a generation module, a first detection module, and a recognition module. The generation module detects facial information in a captured image to generate a face image. The first detection module detects the quality of the face image based on a preset face screening model. The recognition module calculates the similarity between the face feature vector of a face image meeting preset quality conditions and a preset face feature vector using a reconfigurable computing unit, to perform face recognition.

[0006] The embedded device according to embodiments of this application includes a processor. The processor is used to detect facial information in a captured image to generate a facial image; to detect the quality of the facial image based on a preset facial screening model; and to calculate the similarity between the facial feature vector of a facial image whose quality meets preset conditions and a preset facial feature vector based on a reconfigurable computing unit, so as to perform facial recognition.

[0007] The non-volatile computer-readable storage medium of this application includes a computer program that, when executed by one or more processors, causes the processors to perform the following face recognition method: detecting face information in a captured image to generate a face image; detecting the quality of the face image based on a preset face screening model; and calculating the similarity between the face feature vector of a face image whose quality meets preset conditions and a preset face feature vector based on a reconfigurable computing unit, so as to perform face recognition.

[0008] In the face recognition method, face recognition device, embedded device, and non-volatile computer-readable storage medium of the embodiments of this application, before face recognition is performed on a face image, the quality of the face image is detected based on a preset face screening model. Only when the quality of the face image meets the preset conditions will face recognition be performed based on the reconfigurable computing unit. That is, before face recognition is performed on the face image, some face images with poor quality are eliminated, thereby reducing the number of face images that need to be recognized, thus improving recognition efficiency. Moreover, the reconfigurable computing unit can reduce the workload of the processor, thereby reducing the resource occupancy and time consumption of the embedded device in the face recognition process, thereby meeting the operating requirements of the embedded device in other tasks.

[0009] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0010] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0011] Figure 1 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0012] Figure 2 This is a schematic diagram of a face recognition device according to certain embodiments of this application;

[0013] Figure 3 This is a plan view of an embedded device according to certain embodiments of this application;

[0014] Figure 4 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0015] Figure 5 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0016] Figure 6 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0017] Figure 7 This is a schematic diagram of a face recognition method according to certain embodiments of this application.

[0018] Figure 8 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0019] Figure 9 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0020] Figure 10 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0021] Figure 11 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0022] Figure 12 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0023] Figure 13 This is a flowchart illustrating a face recognition method according to certain embodiments of this application;

[0024] Figure 14 This is a schematic diagram illustrating the connection state of a non-volatile computer-readable storage medium and a processor in certain embodiments of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0026] Please see Figure 1 This application provides a face recognition method. The face recognition method includes the following steps:

[0027] 01: Detect facial information in the acquired images to generate facial images;

[0028] 03: Based on a preset face screening model, detect the quality of face images; and

[0029] 05: Based on the reconfigurable computing unit, calculate the similarity between the facial feature vector of a face image that meets the preset quality conditions and the preset facial feature vector, in order to perform face recognition.

[0030] Please see Figure 2This application provides a face recognition device 10. The face recognition device 10 includes a generation module 11, a first detection module 12, and a recognition module 13. The face recognition method of this application can be applied to the face recognition device 10. The generation module 11, the first detection module 12, and the recognition module 13 are respectively used to execute steps 01, 03, and 05. That is, the generation module 11 is used to detect facial information in the acquired image to generate a face image. The first detection module 12 is used to detect the quality of the face image based on a preset face screening model. The recognition module 13 is used to calculate the similarity between the face feature vector of the face image that meets the preset quality conditions and a preset face feature vector based on a reconfigurable computing unit, in order to perform face recognition.

[0031] Please see Figure 3 This application also provides an embedded device 100. The face recognition method of this application can be applied to the embedded device 100. The embedded device 100 includes a processor 20. The processor 20 is used to execute steps 01, 03, and 05. That is, the processor 20 is used to detect face information in the acquired image to generate a face image; detect the quality of the face image based on a preset face screening model; and calculate the similarity between the face feature vector of the face image that meets the preset quality conditions and the preset face feature vector based on a reconfigurable computing unit, so as to perform face recognition.

[0032] The embedded device 100 also includes a housing 30 and a camera 40. The housing 30 is used to mount functional modules of the embedded device 100, such as display devices, imaging devices, power supply devices, and communication devices, providing dustproof, drop-proof, and waterproof protection for these modules. The camera 40 is used to capture images. The embedded device 100 can be a mobile phone, digital camera, smartwatch, head-mounted display device, game console, robot, etc. Figure 3 As shown, the embodiments of this application are described using a mobile phone as an example of embedded device 100. It can be understood that the specific form of embedded device 100 is not limited to a mobile phone.

[0033] Specifically, after the camera 40 captures an image, the processor 20 can detect facial information in the captured image to generate a facial image. Facial information may include the location of the face in the image and the locations of key facial points (such as the eyes, mouth, nose, and ears). The processor 20 can employ a face detection algorithm to obtain the facial information in the image. The face detection algorithm can be a FaceBoxes face detection network or a RetinaFace face detection network. Thus, the processor 20 can generate a facial image based on the obtained facial information.

[0034] Next, the processor 20 can detect the quality of face images according to a preset face screening model. The face screening model can employ a 16-layer convolutional neural network, which may include convolutional layers, activation layers, normalization network (BN) layers, and a loss function layer. The loss function layer uses a robust regression loss function (Huber regression loss function).

[0035] The preset face selection model is a pre-trained selection model. The training samples in the face selection model can include both high-quality and low-quality face samples.

[0036] More specifically, high-quality face samples are those with stable and easily identifiable facial feature points, while low-quality face samples are those with difficult-to-identify facial feature points. Therefore, when the processor 20 detects the quality of a face image using a preset face selection model, the face image can be input into the preset face selection model, and the processor 20 can then determine the quality of the face image.

[0037] For example, if the feature points of a facial image are difficult to identify, the preset face filtering model will output a face image of lower quality. Conversely, if the feature points of a facial image are easy to identify, the preset face filtering model will output a face image of higher quality.

[0038] In some implementations, the preset face selection model can also be configured to assign different scores to training samples of different qualities. That is, the preset face selection model can score images corresponding to different numbers of identified feature points based on the number of feature points identified. It can be understood that the more feature points identified, the higher the score of the corresponding face image.

[0039] Thus, when the quality of the face image meets the preset conditions, the processor 20 will perform face recognition based on the face image.

[0040] In one implementation, the preset conditions can be divided into high quality and low quality. Thus, the processor 20 will perform face recognition based on the face image only when the quality of the face image is high quality.

[0041] In another implementation, the preset condition can be a specific score, such as 80 or 90 points on a percentage scale. Then, when the processor 20 detects the quality score of a face image using a preset face screening model, it can compare it with the preset condition. The processor 20 will only perform face recognition based on the face image if the quality score of the face image is greater than or equal to the score of the preset condition.

[0042] It is understandable that the processor 20 will only perform face recognition based on the face image when the quality meets the preset conditions. That is, face recognition will not be performed on low-quality face images, thereby filtering out low-quality face images and reducing the number of face recognitions required. Furthermore, performing face recognition only on high-quality face images can also improve the recognition rate.

[0043] More specifically, when the processor 20 performs face recognition based on a face image, the processor 20 can use a reconfigurable computing unit to extract the face feature vector of the face image that meets the preset quality conditions.

[0044] Next, the processor 20 can input preset facial feature vectors from a preset facial information database to the reconfigurable computing unit for calculation. The reconfigurable computing unit is a device called a Reconfigurable Computing Unit (RCU). By calculating the similarity between the test facial feature vector and the pre-stored feature vectors using the RCU, the workload of the processor 20 can be reduced, thereby reducing the resource consumption and time spent by the embedded device 100 during facial recognition, thus meeting the operational needs of the embedded device 100 in other tasks.

[0045] In the face recognition method, face recognition device 10, and embedded device 100 of this application embodiment, before performing face recognition on a face image, the quality of the face image is detected based on a preset face screening model. Only when the quality of the face image meets the preset conditions will face recognition be performed based on the reconfigurable computing unit. That is, before performing face recognition on a face image, some face images with poor quality are eliminated, thereby reducing the number of face images that need to be recognized, thus improving recognition efficiency. Furthermore, the reconfigurable computing unit can reduce the workload of the processor 20, thereby reducing the resource occupancy and time consumption of the embedded device 100 in the face recognition process, thus meeting the operational needs of the embedded device 100 in other tasks.

[0046] Please see Figure 2 , Figure 3 and Figure 4 In some implementations, step 01: detecting facial information in the acquired image to generate a facial image includes the following steps:

[0047] 011: Based on a preset face detection model, detect the location of faces in the acquired images;

[0048] 012: Generate a face image based on the face position.

[0049] In some embodiments, the generation module 11 is used to perform steps 011 and 012. That is, the generation module 11 is used to detect the position of a face in the acquired image based on a preset face detection model; and generate a face image based on the face position.

[0050] In some embodiments, the processor 20 is used to perform steps 011 and 012. That is, the processor 20 is used to detect the position of a face in the acquired image based on a preset face detection model; and generate a face image based on the face position.

[0051] Specifically, the facial information includes the location of the target face, i.e., the location of the face in the image captured by camera 40. When processor 20 detects the facial information in the captured image to generate a facial image, it can do so by: processor 20 detecting the location of the face in the captured image based on a preset facial detection model. As can be seen from the above, the preset facial detection model can be a FaceBoxes facial detection network, a RetinaFace facial detection network, etc.

[0052] After the processor 20 obtains the image captured by the camera 40, it can first obtain the face position in the captured image according to the preset face detection model, and then generate a face image based on the face position.

[0053] In this way, image data that does not belong to the face in the image captured by camera 40 can be removed, thereby reducing the amount of data required for quality detection of face images and improving recognition efficiency.

[0054] Please see Figure 2 , Figure 3 and Figure 5 The face recognition method according to the embodiments of this application further includes the following steps:

[0055] 07: Based on a pre-defined face detection model, detect the location of facial landmarks in a face image; and

[0056] 09: Align the face image according to the face position and the position of the face key points so that the pose of the target face in the aligned face image is adjusted to the preset pose.

[0057] Partial Step 03: Detecting the quality of the face image, including the following steps:

[0058] 031: Detect the quality of the aligned face image.

[0059] In some embodiments, the face recognition device 10 further includes a second detection module 14 and an alignment module 15. The second detection module 14 is used to perform step 07. The alignment module 15 is used to perform step 09. The second detection module 14 is used to perform step 031. That is, the second detection module 14 is used to detect the positions of facial key points in the face image based on a preset face detection model. The alignment module 15 is used to align the face image according to the face position and the positions of the facial key points, so that the pose of the target face in the aligned face image is adjusted to a preset pose. The first detection module 12 is used to detect the quality of the aligned face image.

[0060] In some embodiments, the processor 20 is used to perform steps 07, 09, and 031. That is, the processor 20 is used to detect the position of facial key points in a face image based on a preset face detection model; align the face image according to the face position and the position of the facial key points, so that the pose of the target face in the aligned face image is adjusted to a preset pose; and detect the quality of the aligned face image.

[0061] Specifically, the facial information also includes facial landmarks. After generating the facial image, the processor 20 can also detect the locations of facial landmarks in the facial image according to a preset facial detection model. These facial landmarks can be located at the eyes, nose, mouth, and ears, among other things.

[0062] Thus, after the processor 20 obtains the face position and the positions of the facial key points, it can align the face image according to the face position and the positions of the facial key points, so that the pose of the target face in the aligned face image is adjusted to a preset pose. The preset pose can be a front view of the face, a side view of the face, etc. Preferably, to ensure that the feature points in the face image are easy to extract, and considering that the face image is used for face recognition, the preset pose is a front view of the face.

[0063] Specifically, the processor 20 can perform an affine transformation on the face image based on the face position and the position of the face key points, so that the face angle is facing forward, that is, the face image is a frontal view of the face.

[0064] Furthermore, when the processor 20 detects the quality of a face image based on a preset face screening model, it detects the quality of the aligned face image.

[0065] In this way, it can be ensured that when the processor 20 detects the quality of face images based on the preset face screening model, the feature points of the face images obtained are more obvious, thereby ensuring high accuracy when detecting the quality of face images.

[0066] Please see Figure 2 , Figure 3 and Figure 6In some implementations, step 011, detecting the location of facial key points in the acquired image, further includes the following steps:

[0067] 0111: Based on the size of the acquired image, generate multiple candidate boxes of preset sizes and output the score for each candidate box;

[0068] 0113: Identify candidate bounding boxes with scores greater than the preset score as face bounding boxes;

[0069] 0115: Obtain the overlap ratio between any two face bounding boxes that have overlapping parts, and determine the face bounding box with the highest score among those with an overlap ratio greater than a preset overlap ratio as the target face bounding box; and

[0070] 0117: Output the position of the target face bounding box as the face position.

[0071] In some embodiments, the second detection module 14 is used to perform steps 0111, 0113, 0115, and 0117. The second detection module 14 is used to generate multiple candidate boxes of preset sizes based on the size of the acquired image, and output a score for each candidate box; determine that candidate boxes with scores greater than preset scores are face boxes; obtain the overlap degree between any two face boxes that have overlapping parts, determine that among the face boxes with overlap degrees greater than preset overlap degrees, the face box with the highest score is the target face box; and output the position of the target face box as the face position.

[0072] In some embodiments, the processor 20 is used to execute steps 0111, 0113, 0115, and 0117. That is, the processor 20 is used to generate multiple candidate boxes of preset sizes according to the size of the acquired image, and output a score for each candidate box; determine that the candidate box with a score greater than a preset score is a face box; obtain the overlap degree between any two face boxes that have overlapping parts, determine that the face box with the highest score among the face boxes with an overlap degree greater than a preset overlap degree is the target face box; and output the position of the target face box as the face position.

[0073] Specifically, when the processor 20 detects face locations in an image based on a preset face detection model, the processor 20 can generate multiple candidate boxes of preset sizes according to the size of the acquired image and output a score for each candidate box. The preset size is directly proportional to the size of the acquired image; that is, the larger the size of the acquired image, the larger the size of the candidate box of the preset size.

[0074] like Figure 7 As shown in (a), if the size of the acquired image P1 is 1600*900, then the size of the candidate box S1 is 200*150. Figure 7As shown in (b), if the size of the acquired image P2 is 1920*1080, then the size of the candidate box S2 is 240*200.

[0075] When the processor 20 scores multiple candidate boxes, it can do so based on a pre-trained face detection model. For example, the pre-trained face detection model contains scores corresponding to training samples of different qualities. When a candidate box is input into the pre-trained face detection model, a score for each candidate box is obtained. For instance, the more facial features a candidate box contains, the higher its score.

[0076] Next, the processor 20 can determine face boxes based on the score of each candidate box, identifying those with scores greater than a preset score. The preset score can be a manually set score, such as 80 or 90 points. It can be understood that candidate boxes with scores greater than the preset score contain more facial features from the corresponding image.

[0077] Furthermore, the processor 20 can obtain the overlap degree between any two face boxes that have overlapping parts, so as to determine the target face box based on the overlap degree and the score of the candidate box.

[0078] More specifically, the greater the overlap between any two face bounding boxes, the higher the overlap ratio. When the overlap ratio exceeds a preset overlap ratio, the processor 20 considers the face bounding box with the highest score among the two face bounding boxes to be the target face bounding box. In other words, an overlap ratio greater than the preset overlap ratio indicates that the two face bounding boxes share a larger area.

[0079] The target face bounding box is the face bounding box with the highest score among all face bounding boxes with an overlap greater than the preset overlap determined by the processor 20.

[0080] In this way, the processor 20 can output the position of the target face bounding box as the face location. It can be understood that the position of the target face bounding box is the most accurate face location in the acquired image, thus ensuring relatively accurate face detection.

[0081] Please see Figure 2 , Figure 3 and Figure 8 In some implementations, step 05: Calculating the similarity between the facial feature vector of a face image meeting preset quality conditions and a preset facial feature vector based on the reconfigurable computing unit for face recognition, further includes the following steps:

[0082] 051: Based on a preset feature extraction model, extract features from the face image to generate a face feature vector; and

[0083] 053: Establish a facial information database based on facial feature vectors.

[0084] In some embodiments, the recognition module 13 is used to perform steps 051 and 053. That is, the recognition module 13 is used to extract features of the face image based on a preset feature extraction model to generate a face feature vector; and to establish a face information database based on the face feature vector.

[0085] In some embodiments, the processor 20 is used to perform steps 051 and 053. That is, the processor 20 is used to extract features of a face image based on a preset feature extraction model to generate a face feature vector; and to establish a face information database based on the face feature vector.

[0086] Specifically, when performing face recognition based on face images, the processor 20 can also extract features from the face images based on a preset feature extraction model to generate face feature vectors, and establish a face information database based on the face feature vectors.

[0087] The preset feature extraction model is a pre-trained model that can employ a 72-layer convolutional neural network, including convolutional layers, pooling layers, activation layers, fully connected layers, and loss layers. The loss function used is a weighted sum of softmax-loss and center-loss. Softmax-loss is used to improve the intra-class aggregation of samples in the feature space, while center-loss is used to increase the inter-class distance of samples in the feature space.

[0088] In the face recognition method of this application, the face information database must first store face information capable of face recognition before face recognition can be performed. Therefore, before face recognition, the user needs to register their own face information. That is, an offline face image registration process is required.

[0089] The specific process is as follows: Figure 9 As shown: First, the camera 40 of the embedded device 100 acquires images for detection by the processor 20. The processor 20 detects facial information (face location and facial landmark locations) based on a preset face detection model to align the face image. Next, the processor 20 uses a preset face filtering model to detect the quality of the face. When the quality of the face image meets preset conditions, the processor 20 extracts features from the face image based on a preset feature extraction model to generate a face feature vector and establish a face information database. If the quality of the face image does not meet preset conditions, the camera 40 prompts the user to re-acquire the image, repeating the above steps until the quality of the face image meets the preset conditions.

[0090] It is understandable that the facial information database stores facial images that meet quality standards. In this way, when performing facial recognition, it can be ensured that the facial images that pass the facial recognition are of good quality, thus guaranteeing the accuracy of facial recognition.

[0091] Please see Figure 2 , Figure 3 and Figure 10 In some implementations, step 05: Calculating the similarity between the facial feature vector of a face image meeting preset quality conditions and a preset facial feature vector based on the reconfigurable computing unit for face recognition, further includes the following steps:

[0092] 055: Based on the preset feature extraction model, extract features from the face image to generate the face feature vector to be tested;

[0093] 057: Input pre-stored feature vectors from a preset face information database into the reconstructable computing unit to calculate the similarity between the feature vector of the face to be tested and the pre-stored feature vectors, and determine the target feature vector based on the similarity; and

[0094] 059: If the similarity between the target feature vector and the feature vector of the face to be tested is greater than a preset threshold, the face authentication is determined to be successful.

[0095] In some embodiments, the recognition module 13 is used to perform steps 055, 057, and 059. That is, the recognition module 13 is used to extract features of the face image based on a preset feature extraction model to generate a face feature vector to be tested; input pre-stored feature vectors from a preset face information database into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors, and determine the target feature vector based on the similarity; and determine that face authentication is successful if the similarity between the target feature vector and the face feature vector to be tested is greater than a preset threshold.

[0096] In some embodiments, the processor 20 is used to execute steps 055, 057, and 059. That is, the processor 20 is used to extract features from a face image based on a preset feature extraction model to generate a face feature vector to be tested; input pre-stored feature vectors from a preset face information database into a reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors, and determine the target feature vector based on the similarity; and determine that face authentication is successful if the similarity between the target feature vector and the face feature vector to be tested is greater than a preset threshold.

[0097] Specifically, when the processor 20 performs face recognition based on a face image, the processor 20 can extract features from the face image based on a preset feature extraction model, thereby generating a face feature vector to be tested.

[0098] Next, the processor 20 can input pre-stored feature vectors from a preset face information database to the reconfigurable computing unit for calculation. The reconfigurable computing unit (RCU) is a device. Calculating the similarity between the test face feature vector and the pre-stored feature vectors using the RCU reduces the workload of the processor 20, thereby reducing the resource consumption and time spent by the embedded device 100 during face recognition, thus meeting the operational needs of the embedded device 100 in other tasks.

[0099] More specifically, after the RCU device calculates the similarity between the feature vector of the face to be tested and the pre-stored feature vectors, the processor 20 can determine the target feature vector based on the similarity. When the similarity between the target feature vector and the feature vector of the face to be tested is greater than a preset threshold, the processor 20 can determine that the face authentication is successful. It can be understood that the target feature vector is one or more feature vectors among all the pre-stored feature vectors whose similarity to the feature vector of the face to be tested is greater than the preset threshold.

[0100] The preset threshold can be any value set manually, such as 90%, 95%, 98%, etc. When the similarity between the target feature vector and the feature vector of the face to be tested is greater than the preset threshold, it means that the face image can be recognized and can be successfully unlocked.

[0101] In this way, the processor 20 can calculate the similarity between the feature vector of the face to be tested and the pre-stored feature vector based on the RCU device, thereby relieving the processor 20 of its work, reducing the resource consumption of the face recognition work on the embedded device 100, reducing the time consumption of face recognition, and meeting the operating requirements of the embedded device 100 in other tasks.

[0102] Please see Figure 2 , Figure 3 and Figure 11 In some implementations, step 057, which involves inputting pre-stored feature vectors from a preset face information database into the reconstructable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors, and determining the target feature vector based on the similarity, further includes the following steps:

[0103] 0571: Based on the memory capacity of the reconfigurable computing unit, the pre-stored feature vectors in the face information database are divided into multiple feature sets so that the memory occupied by each feature set is less than the memory capacity.

[0104] 0573: Input each feature set into the reconstructable computing unit in sequence to calculate the similarity between the feature vector of the face to be tested and the pre-stored feature vector of each feature set;

[0105] 0575: Sort the pre-stored feature vectors according to their similarity to determine the pre-sorted pre-stored feature vectors as the target feature vectors.

[0106] In some embodiments, the recognition module 13 is used to perform steps 0571, 0573, and 0575. That is, the recognition module 13 is used to divide the pre-stored feature vectors in the face information database into multiple feature sets according to the memory capacity of the reconfigurable computing unit, so that the memory occupied by each feature set is less than the memory capacity; input each feature set into the reconfigurable computing unit in sequence to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors of each feature set; sort them according to the similarity corresponding to each pre-stored feature vector to determine the pre-sorted pre-stored feature vectors as the target feature vectors.

[0107] In some embodiments, the processor 20 is used to execute steps 0571, 0573, and 0575. That is, the processor 20 is used to divide the pre-stored feature vectors in the face information database into multiple feature sets according to the memory capacity of the reconfigurable computing unit, so that the memory occupied by each feature set is less than the memory capacity; sequentially input each feature set into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors of each feature set; sort them according to the similarity corresponding to each pre-stored feature vector to determine the pre-sorted pre-stored feature vectors as the target feature vectors.

[0108] Specifically, when the reconfigurable computing unit (RCU device) calculates the similarity between the feature vector of the face to be tested and the pre-stored feature vector, the pre-stored feature vector in the face information database can be divided into multiple feature sets according to the memory capacity of the RCU device.

[0109] More specifically, taking a total of N pre-stored feature vectors as an example, the processor 20 can divide the RCU device's memory capacity into M parts (i.e., into M feature sets). The number of pre-stored feature vectors in each part is (N+M-1) / M. In this way, it can be ensured that the RCU device's memory capacity can be divided into M parts, the difference in the number of pre-stored feature vectors in each part will not be large, and the memory occupied by each feature set is less than the memory capacity.

[0110] Next, the processor 20 can sequentially input each feature set into the RCU device to calculate the similarity between the feature vector of the face to be tested and the pre-stored feature vector of each feature set. Since the RCU device has limited memory capacity, it can only calculate the similarity between the pre-stored feature vector within one feature set and the feature vector of the face to be tested during the calculation process. Therefore, sequentially inputting each feature set into the RCU device ensures that it can calculate the similarity between the pre-stored feature vectors within all feature sets and the feature vector of the face to be tested.

[0111] Finally, the processor 20 obtains the similarity corresponding to each pre-stored feature vector, sorts the similarity of the pre-stored feature vectors, and selects the pre-sorted pre-stored feature vectors as target feature vectors. The processor 20 can sort the pre-stored feature vectors from largest to smallest based on their similarity. The preset sorting can be 5, 10, 15, etc. For example, when the preset sorting is 5, the first 5 pre-stored feature vectors are the target feature vectors, meaning the 5 pre-stored feature vectors with the highest similarity are the target feature vectors.

[0112] In some implementations, after the first feature set is input into the RCU device, the RCU device can calculate the similarity between each pre-stored feature vector in the first feature set and the feature vector of the face to be tested, so as to obtain the similarity between K feature vectors of the face to be tested and the pre-stored feature vectors of each feature set. Here, K is the top K similarity scores after sorting the similarity scores of the feature vectors of the face to be tested and the pre-stored feature vectors of each feature set from largest to smallest, that is, the K similarity scores are the K with the largest similarity scores in the feature set.

[0113] Then, the processor inputs the second feature set into the RCU device. The RCU calculates the similarity between each pre-stored feature vector in the second feature set and the feature vector of the face to be tested, to obtain K1 similarities again. These similarities are then compared with the K similarities obtained from the first feature set and updated to obtain K2 similarities. Here, K1 and K2 equal K. It can be understood that these K2 similarities are the K2 similarities with the higher similarity after the similarity calculation between the first and second feature sets.

[0114] Similarly, when the processor inputs a feature set into the RCU device, it calculates the similarity between the feature set and the feature vector of the face to be tested, compares it with the K similarities obtained from the previous feature set, and updates the result. This process continues until all feature sets have been input, thus obtaining the K largest similarities among the pre-stored feature vectors in all feature sets and the feature vector of the face to be tested. In other words, the pre-stored feature vectors corresponding to these K similarities are the target feature vectors.

[0115] Please see Figure 2 , Figure 3 and Figure 12 In some implementations, step 057: calculating the similarity between the feature vector of the face to be tested and the pre-stored feature vector, further includes the following steps:

[0116] 0577: Calculate the Euclidean distance between the feature vector of the face to be tested and the pre-stored feature vector; and

[0117] 0579: Determine the similarity between the feature vector of the face to be tested and the pre-stored feature vector based on Euclidean distance.

[0118] In some embodiments, the recognition module 13 is used to perform steps 0577 and 0579. That is, the recognition module 13 is used to calculate the Euclidean distance between the feature vector of the face to be tested and the pre-stored feature vector; and to determine the similarity between the feature vector of the face to be tested and the pre-stored feature vector based on the Euclidean distance.

[0119] In some embodiments, the processor 20 is used to perform steps 0577 and 0579. That is, the processor 20 is used to calculate the Euclidean distance between the feature vector of the face to be tested and the pre-stored feature vector; and to determine the similarity between the feature vector of the face to be tested and the pre-stored feature vector based on the Euclidean distance.

[0120] Specifically, when the RCU device calculates the similarity between the face feature vector to be tested and the pre-stored feature vector, it can first calculate the Euclidean distance between the face feature vector to be tested and the pre-stored feature vector, and then determine the similarity between the face feature vector to be tested and the pre-stored feature vector based on the Euclidean distance.

[0121] The formulas for calculating Euclidean distance are shown in formulas (1), (2), and (3) below:

[0122]

[0123]

[0124]

[0125] Formula (1) is the calculation formula for two-dimensional space, formula (2) is the calculation formula for three-dimensional space, and formula (3) is the calculation formula for N-dimensional space. Specifically, the corresponding formula is selected for calculation based on the specific numbers of the face feature vector to be tested and the pre-stored feature vector. ρ and d(x, y) are the specific values ​​of the Euclidean distance, (x1, y1) are the coordinates of the face feature vector to be tested, and (x2, y1) are the coordinates of the pre-stored feature vector.

[0126] More specifically, the smaller the Euclidean distance, the greater the similarity between the feature vector of the face to be tested and the pre-stored feature vectors. Thus, after calculating the Euclidean distance between the feature vector of the face to be tested and the pre-stored feature vectors, one or more pre-stored feature vectors with a high similarity to the feature vector of the face to be tested can be identified based on the Euclidean distance.

[0127] In one implementation, the RCU device can sequentially calculate the Euclidean distance between the test face feature vector and the pre-stored feature vector in each feature set, and only store the K test face feature vectors with the smallest Euclidean distance. Next, it sequentially compares the O test face vectors in each feature set to obtain the O test face feature vectors with the smallest Euclidean distance across all feature sets. Here, K and O are any positive integers greater than 0, and K and O can be equal.

[0128] The face recognition method of this application, during the process of a user performing face recognition using the embedded device 100, such as Figure 13 As shown, the processor 20 first acquires the user's real-time image through the camera 40 and performs face detection on the real-time image to obtain a face image. Then, based on the face position and the position of facial key points, the face image is aligned, and the face image is filtered according to a preset face filtering model, i.e., quality detection. When the quality meets the preset conditions, facial features are extracted from the face image; when the quality does not meet the preset conditions, the embedded device 100 will end the face recognition and may also remind the user that the face recognition failed. After extracting the features of the face image according to the preset feature extraction model to generate a face feature vector to be tested, the processor 20 can calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors in the face information database according to the RCU device, thereby obtaining a target feature vector that is more similar to the pre-stored feature vector, i.e., returning the face similarity ID. When the similarity between the target feature vector and the face feature vector to be tested meets the preset threshold, the face authentication is determined to be successful.

[0129] Please see Figure 14 This application also provides a non-volatile computer-readable storage medium 300 containing a computer program 301. When the computer program 301 is executed by one or more processors 20, the one or more processors 20 perform the face recognition method of any of the above embodiments.

[0130] For example, when computer program 301 is executed by one or more processors 20, the processors 20 perform the following face recognition method:

[0131] 01: Detect facial information in the acquired images to generate facial images;

[0132] 03: Based on a preset face screening model, detect the quality of face images; and

[0133] 05: Based on the reconfigurable computing unit, calculate the similarity between the facial feature vector of a face image that meets the preset quality conditions and the preset facial feature vector, in order to perform face recognition.

[0134] For example, when computer program 301 is executed by one or more processors 20, the processors 20 perform the following face recognition method:

[0135] 07: Based on a pre-defined face detection model, detect the location of facial landmarks in a face image; and

[0136] 09: Align the face image according to the face position and the position of the face key points so that the pose of the target face in the aligned face image is adjusted to the preset pose.

[0137] In the description of this specification, the references to terms such as "some embodiments," "in one example," "exemplarily," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0138] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0139] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A face recognition method, characterized in that, include: Detect facial information in the acquired images to generate facial images; The quality of the face image is detected based on a preset face screening model; and Based on the reconfigurable computing unit, the similarity between the facial feature vector of a face image that meets the preset quality conditions and the preset facial feature vector is calculated to perform face recognition. The method based on the reconfigurable computing unit calculates the similarity between the facial feature vector of a facial image that meets preset quality conditions and a preset facial feature vector for face recognition, including: Based on a preset feature extraction model, the features of the face image are extracted to generate a face feature vector to be tested; Input the pre-stored feature vectors from the preset face information database into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors, and determine the target feature vector based on the similarity. If the similarity between the target feature vector and the face feature vector to be tested is greater than a preset threshold, the face authentication is determined to be successful. The process of inputting pre-stored feature vectors from a preset face information database into a reconstructable computing unit to calculate the similarity between the test face feature vector and the pre-stored feature vectors, and determining the target feature vector based on the similarity, includes: Based on the memory capacity of the reconfigurable computing unit, the pre-stored feature vectors in the face information database are divided into multiple feature sets, so that the memory occupied by each feature set is less than the memory capacity. Each feature set is sequentially input into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vector of each feature set; The pre-stored feature vectors are sorted according to their similarity to determine the pre-stored feature vectors in the preset sorting as the target feature vectors.

2. The face recognition method according to claim 1, characterized in that, The facial information includes the location of the target face, and the detection of facial information in the acquired image to generate a facial image includes: Based on a preset face detection model, the location of the face in the acquired image is detected; The face image is generated based on the face location.

3. The face recognition method according to claim 2, characterized in that, The facial information also includes facial key points, and the facial recognition method further includes: Based on a preset face detection model, the positions of the facial key points in the face image are detected; Based on the face position and the position of the face key points, align the face image so that the pose of the target face in the aligned face image is adjusted to a preset pose. The process of detecting the quality of the face image includes: The quality of the aligned face image is detected.

4. The face recognition method according to claim 2, characterized in that, The detection of the face position in the acquired image includes: Based on the size of the acquired image, generate multiple candidate boxes of preset sizes and output a score for each candidate box; Candidate boxes with scores greater than a preset score are identified as face boxes. Obtain the overlap degree between any two face frames that have overlapping parts, and determine the face frame with the highest score among the face frames with the overlap degree greater than a preset overlap degree as the target face frame; Output the position of the target face bounding box as the face position.

5. The face recognition method according to claim 1, characterized in that, The method based on the reconfigurable computing unit calculates the similarity between the facial feature vector of a facial image that meets preset quality conditions and a preset facial feature vector for face recognition, including: Based on a preset feature extraction model, features of the face image are extracted to generate a face feature vector; A facial information database is established based on the facial feature vectors.

6. The face recognition method according to claim 1, characterized in that, The calculation of the similarity between the face feature vector to be tested and the pre-stored feature vector includes: Calculate the Euclidean distance between the feature vector of the face to be tested and the pre-stored feature vector; and The similarity between the feature vector of the face to be tested and the pre-stored feature vector is determined based on the Euclidean distance.

7. A face recognition device, characterized in that, The apparatus is used to perform the face recognition method according to any one of claims 1-6, and the apparatus comprises: A generation module is used to detect facial information in the acquired image to generate a facial image; A first detection module is used to detect the quality of the face image based on a preset face screening model; and The recognition module is used to calculate the similarity between the facial feature vector of a face image that meets the preset quality conditions and a preset facial feature vector based on a reconfigurable computing unit, so as to perform face recognition.

8. An embedded device, characterized in that, The device is used to perform the face recognition method according to any one of claims 1-6. The device includes a processor, which is used to detect face information in the acquired image to generate a face image; detect the quality of the face image based on a preset face screening model; and calculate the similarity between the face feature vector of the face image that meets the preset quality conditions and the preset face feature vector based on a reconfigurable computing unit, so as to perform face recognition.

9. The embedded device according to claim 8, characterized in that, The facial information includes the facial location of the target face. The processor is used to detect the facial location in the facial image based on a preset facial detection model; and to generate the facial image based on the facial location.

10. The embedded device according to claim 9, characterized in that, The face information also includes facial key points. The processor is used to detect the position of the facial key points in the face image based on a preset face detection model; align the face image according to the face position and the position of the facial key points so that the pose of the target face in the aligned face image is adjusted to a preset pose; and detect the quality of the aligned face image.

11. The embedded device according to claim 9, characterized in that, The processor is configured to generate multiple candidate boxes of preset sizes based on the size of the acquired image, and output a score for each candidate box; determine the candidate box with a score greater than a preset score as a face box; obtain the overlap degree between any two face boxes that have overlapping parts, and determine the face box with the highest score among the face boxes with an overlap degree greater than a preset overlap degree as the target face box; and output the position of the target face box as the face position.

12. The embedded device according to claim 8, characterized in that, The processor is used to extract features from the face image based on a preset feature extraction model to generate a collected face feature vector; and to establish a face information database based on the collected face feature vector.

13. The embedded device according to claim 8, characterized in that, The processor is used to extract features from the face image based on a preset feature extraction model to generate a face feature vector to be tested; input pre-stored feature vectors from a preset face information database into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vectors, and determine the target feature vector based on the similarity; if the similarity between the target feature vector and the face feature vector to be tested is greater than a preset threshold, the face authentication is determined to be successful.

14. The embedded device according to claim 13, characterized in that, The processor is used to divide the pre-stored feature vectors in the face information database into multiple feature sets according to the memory capacity of the reconfigurable computing unit, so that the memory occupied by each feature set is less than the memory capacity. Each feature set is sequentially input into the reconfigurable computing unit to calculate the similarity between the face feature vector to be tested and the pre-stored feature vector of each feature set; the pre-stored feature vectors are sorted according to their corresponding similarity to determine the pre-stored feature vectors in the preset order as the target feature vector.

15. The embedded device according to claim 13, characterized in that, The processor is further configured to calculate the Euclidean distance between the face feature vector to be tested and the pre-stored feature vector; and to determine the similarity between the face feature vector to be tested and the pre-stored feature vector based on the Euclidean distance.

16. A non-volatile computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by one or more processors, implements the face recognition method according to any one of claims 1-6.

Images