Face recognition method, device, electronic device and storage medium

Abstract

The present application provides a face recognition method, device, electronic device and storage medium, the face recognition method includes obtaining the main features of a face image to be recognized, the main features being face features; obtaining the main features of each template image; for each template image, based on the main features of the template image and the main features of the face image to be recognized, calculating the auxiliary features between the face image to be recognized and the template image; based on the auxiliary features and the auxiliary recognition threshold corresponding to the template image, determining the recognition result of the face image to be recognized, wherein each template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated according to the recognition record when the template image is successfully recognized in the historical time period. The present application can improve the accuracy of face recognition.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a face recognition method, device, electronic device and storage medium. Background Art

[0002] With the development of image processing technology and artificial intelligence technology, face recognition technology has been widely used, and various face recognition products have been launched one after another. Usually, face recognition products will set a unified global recognition threshold, based on which the current face recognition result can be determined. However, this face recognition method has low accuracy. Summary of the invention

[0003] In view of this, embodiments of the present application provide a face recognition method, device, electronic device and storage medium to solve the problem of how to improve the accuracy of face recognition in the prior art.

[0004] A first aspect of an embodiment of the present application provides a face recognition method, characterized by comprising:

[0005] Acquire main features of the face image to be recognized, wherein the main features are face features;

[0006] Obtain the main features of each template image;

[0007] For each template image, based on the main features of the template image and the main features of the face image to be identified, the auxiliary features between the face image to be identified and the template image are calculated; based on the auxiliary features and the auxiliary recognition threshold corresponding to the template image, the recognition result of the face image to be identified is determined, wherein each template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0008] A second aspect of an embodiment of the present application provides a face recognition device, characterized by comprising:

[0009] An acquisition unit, used for acquiring main features of the face image to be recognized, wherein the main features are face features;

[0010] an auxiliary feature determination unit, configured to obtain the main features of the face image to be identified; obtain the main features of the template image; and for each template image, respectively calculate the auxiliary features between the face image to be identified and the template image based on the main features of the template image and the main features of the face image to be identified;

[0011] A recognition result determination unit is used to determine the recognition result of the face image to be recognized based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, wherein each pre-stored template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the steps of the face recognition method.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, an electronic device implements the steps of the face recognition method.

[0014] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of the face recognition method described in the first aspect.

[0015] The embodiment of the present application determines the recognition result based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, which can realize the personalization of the recognition threshold and more accurately perform face recognition based on the recognition threshold applicable to the current face image to be recognized. Compared with the existing method of setting only a unified recognition threshold, it can improve the accuracy of face recognition. In addition, since the recognition threshold is generated based on the recognition record when the template image is successfully recognized in the historical time period, that is, the recognition threshold is dynamically generated based on the actual recognition situation in the historical time period, it can make the recognition threshold more in line with the actual face recognition situation, further improving the accuracy of face recognition. Furthermore, the embodiment of the present application makes judgments based on auxiliary features rather than main features, and can tolerate a small number of main features that are different, but most of the main features are the same or similar, so as to reduce the recognition error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0017] Figure 1 This is a schematic diagram of an implementation flow of a face recognition method provided in an embodiment of the present application;

[0018] Figure 2This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0020] Figure 4 is a specific implementation flow chart of step S107 in a face recognition method provided in an embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0022] Figure 6 is a schematic diagram of a face recognition device provided in an embodiment of the present application;

[0023] Figure 7 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0025] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0026] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0027] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] Usually, face recognition products will set a unified global recognition threshold, based on which the current face recognition result can be determined. When a person's face is easily misidentified, it is usually solved by raising the global recognition threshold. However, raising the global recognition threshold will easily make it difficult to recognize other people's faces, resulting in a lower success rate of face recognition. Therefore, the method of performing face recognition based on a unified global recognition threshold has a low accuracy.

[0032] In order to solve this technical problem, the embodiments of the present application provide a face recognition method, device, electronic device and storage medium, including: obtaining the main features of a face image to be recognized; obtaining the main features of each template image; for each template image, based on the main features of the template image and the main features of the face image to be recognized, respectively calculating the auxiliary features between the face image to be recognized and the template image; based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, determining the recognition result of the face image to be recognized, wherein each pre-stored template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0033] The embodiment of the present application determines the recognition result based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, which can realize the personalization of the recognition threshold and more accurately perform face recognition based on the recognition threshold applicable to the current face image to be recognized. Compared with the existing method of setting only a unified recognition threshold, it can improve the accuracy of face recognition. In addition, since the recognition threshold is generated based on the recognition record when the template image is successfully recognized in the historical time period, that is, the recognition threshold is dynamically generated based on the actual recognition situation in the historical time period, it can make the recognition threshold more in line with the actual face recognition situation, further improving the accuracy of face recognition. Furthermore, the embodiment of the present application makes judgments based on auxiliary features rather than main features, and can tolerate a small number of main features that are different, but most of the main features are the same or similar, so as to reduce the recognition error rate.

[0034] Embodiment 1:

[0035] Figure 1 The flowchart of a face recognition method provided by an embodiment of the present application is shown, and the execution subject of the face recognition method is an electronic device. The electronic device can be a device with a camera module, such as a face recognition attendance machine, a face access control device, a monitoring device, a mobile phone, a camera, etc.; or, the electronic device can be other computing devices connected to a device with a camera module, such as a computer connected to a monitoring camera, a video camera, etc. Figure 1 The face recognition method shown is detailed as follows:

[0036] In S101, main features of the face image to be recognized are obtained.

[0037] The face image currently captured by the camera module is obtained as the face image to be recognized.

[0038] In one embodiment, after detecting a preset instruction, the camera module can be started to shoot and generate a face image to be identified. Further, after detecting a preset instruction, a user prompt message can be issued to prompt the user to move to a specified position, so that the camera module can accurately shoot the face image and generate an accurate face image to be identified.

[0039] In another embodiment, the camera module may automatically start shooting at preset time intervals to obtain the face image to be identified, thereby achieving continuous capture and tracking of the face.

[0040] The main feature refers to the extraction of facial features from the face image to be identified. In this embodiment, facial feature extraction is implemented using multiple methods: for example, geometric feature methods, algebraic features, statistical learning, Gabor, Fisher-vector, scale-invariant feature transformation, local binary pattern, high-dimensional local binary pattern, deep learning and other methods. The method based on geometric features mainly extracts the geometric shapes and geometric relationships of obvious facial features (such as eyes, nose, mouth, chin) as facial features, and constructs a feature vector that can distinguish facial features. The feature vector carries information such as the Euclidean distance and cosine distance between feature points. The method of facial feature extraction here is only an example and not a limitation.

[0041] In this step, facial features are extracted using the aforementioned method, and the main features obtained may include sub-features related to M (e.g., dozens) of key points of the face, such as eye width, pupil color, nose length, eyebrow distance, etc. Those skilled in the art can set key points according to actual needs. The sub-features of dozens of key points are combined into dozens of vectors, and the main feature is a vector with M dimensions (dozens of dimensions).

[0042] In S102, main features of each template image are obtained.

[0043] In the embodiment of the present application, the face images of authorized persons collected in advance are stored in the storage unit of the electronic device or the storage unit of a third party accessible to the electronic device as template images. The method of obtaining the main features of the template image is the same as that disclosed in S101, so it is not repeated.

[0044] In S103, for each template image, auxiliary features between the face image to be recognized and the template image are calculated based on the main features of the template image and the main features of the face image to be recognized.

[0045] In the embodiment of the present application, for each pre-stored template image, there is at least one corresponding primary recognition threshold. For example, N facial images of authorized persons can be stored in advance as N template images, numbered 1 to N, and their corresponding primary recognition thresholds are T1 to T2. N . Moreover, the main recognition threshold corresponding to each template image is dynamically generated based on the recognition records of the template image being successfully recognized in the historical time period. The historical time period can be a time period of a preset length, such as a day, a week, a month, or from a specific time point in the past to the present. For example, for template image i, its current corresponding recognition threshold T i, can be obtained by calculating (e.g., averaging) the similarities between each face image identified as the template image i in the past week and the template image i. It can be understood that in the initial state where the corresponding recognition record has not yet been generated for each template image, the corresponding main recognition threshold can be an initial value preset in advance.

[0046] In one embodiment, each template image has corresponding identification information, which can be an identity identification number (Identity, ID). The identification information of the template image is stored correspondingly to its main recognition threshold, such as being stored in a mapping table storing the main recognition threshold. The identification information corresponding to each template image is the target identification information. The target identification information is used as an index to query the mapping table storing the main recognition threshold, and the main recognition threshold of each template image can be obtained. In order to correspond to the tens-dimensional vectors of the main feature similarity, the main recognition threshold can be a vector of the same dimension, and each dimension has its corresponding main recognition threshold. In other words, the reference value of the difference of each dimension (i.e., each key point) can be different, which is conducive to customized recognition standards. In one case, for example, in China, most people have black eyes, so the main recognition threshold of the pupil color difference can be set lower, indicating that a slight difference in pupil color can be used to judge that the pupil color is different.

[0047] For the sub-features related to the M key points contained in the main features, the sub-features under the key points contained in the main features of the face image to be identified and the sub-features under the key points contained in the main features of the template image are inner-producted to generate the inner-product results under the key points. The difference between the inner-product result and the corresponding main recognition threshold is judged to determine the intermediate recognition result of the face image to be identified, that is, each vector comparison generates an intermediate recognition result. In more detail, the inner-product result under each key point of the face image to be identified is compared with the corresponding main recognition threshold. When the inner-product result is greater than the main recognition threshold of the template image at the key point, it indicates that the difference is low, and the intermediate recognition result under the key point can be set to a first value (for example, 1). When the inner-product result is not greater than the main recognition threshold of the template image at the key point, it indicates that the difference is high, and the intermediate recognition result under the key point can be set to a second value (for example, 0). The tens-dimensional vector of the face image to be identified will generate a sequence of dozens of intermediate recognition results with values ​​of 0 or 1, where 0 indicates that the difference of the main feature is large, and 1 indicates that the difference of the main feature is small. In other words, under the premise that the main feature is an M-dimensional vector, the auxiliary feature is an M-dimensional binary sequence including the first value and the second value.

[0048] In S104, the recognition result of the face image to be recognized is determined based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, wherein each pre-stored template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is also generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0049] The generation method of the auxiliary recognition threshold is similar to that of the main recognition threshold. Specifically, for each template image, the recognition record of the template image being successfully recognized in the historical time period is obtained, and the recognition record includes the auxiliary features between each face image to be recognized and the template image when it is successfully recognized as the template image; the auxiliary recognition threshold corresponding to the template image is generated according to the auxiliary features contained in the recognition record. Among them, the generation of the auxiliary recognition threshold corresponding to the template image according to the auxiliary features contained in the recognition record includes: according to the recognition record, respectively determining the sub-average values ​​of the auxiliary features contained in each sub-time period; according to each sub-average value, obtaining the target average value; the target average value is the average value of each sub-average value; according to the target average value, the auxiliary recognition threshold corresponding to the template image is generated.

[0050] In this step, it is determined whether the proportion of auxiliary features with a value of 1 is greater than the auxiliary recognition threshold. If it is greater than the auxiliary recognition threshold (for example, 95%), it means that the differences in most of the main features are small, and the recognition result is considered a successful recognition. Otherwise, it is considered a failed recognition.

[0051] Optionally, when the face image to be identified is successfully recognized, the electronic device performs the target action. For example, if the electronic device is a face recognition attendance machine, the target action may be an action of writing the user's attendance record to a storage unit. If the electronic device is a face access control device, the target action may be an action of opening a door. For example, if the electronic device is a mobile phone, the target action may be an action of unlocking the screen, etc.

[0052] In the embodiment of the present application, a face image to be identified is obtained, and the similarity of the main features between the face image to be identified and each pre-stored template image, that is, the auxiliary features, is calculated respectively. Based on the auxiliary features and the auxiliary recognition thresholds corresponding to each template image, the recognition result of the face image to be identified is determined. Since each template image has its own corresponding auxiliary recognition threshold, the method of determining the recognition result based on the auxiliary features and the auxiliary recognition thresholds corresponding to each template image can not only consider the overall difference of the main features, but also realize the personalization of the recognition threshold, and more accurately perform face recognition based on the recognition threshold applicable to the current face image to be identified. Compared with the existing method of setting only a unified recognition threshold, the accuracy of face recognition can be improved. In addition, since the main and auxiliary recognition thresholds are generated based on the recognition records when the template image is successfully recognized in the historical time period, that is, the recognition threshold is dynamically generated based on the actual recognition situation in the historical time period, the two recognition thresholds can be made more in line with the actual face recognition situation, further improving the accuracy of face recognition. Furthermore, it is very easy to change appearance in today's society, such as doing lip augmentation surgery, or doing bone cutting surgery to turn a round face into a melon-seed face, or doing rhinoplasty surgery, etc. The aforementioned means will change at least one of the main features. If the analysis is based solely on the main features, recognition failure may easily occur due to a large difference in one of the main features. The embodiment of the present application makes judgments based on auxiliary features, using the degree of similarity of the main features (without pursuing complete identity of the main features), rather than judging based on the main features. It can tolerate a small number of different main features but the vast majority of the main features being the same or similar, so as to reduce the recognition error rate.

[0053] Figure 2 A schematic flow chart of a face recognition method provided by another embodiment of the present application is shown. After the above step S104, the face recognition method further includes:

[0054] S105: If the recognition result of the face image to be recognized is successful, the auxiliary features are stored corresponding to each template image.

[0055] In an embodiment of the present application, when the recognition result of the face image to be recognized is successful, the auxiliary feature can be stored in correspondence with each template image, so that subsequent query tracking or statistical analysis can be performed based on the auxiliary feature when the recognition is successful. In one embodiment, the auxiliary feature is stored in correspondence with each template image, and specifically, the current recognition time point, the auxiliary feature, and the target identification information corresponding to each template image can be stored in correspondence, such as in a mapping table, for subsequent reference and analysis.

[0056] Figure 3A schematic flow chart of a face recognition method provided by another embodiment of the present application is shown. After the above step S104, the face recognition method further includes:

[0057] S106: for each template image, obtaining a recognition record when the template image is successfully recognized in a historical time period, the recognition record including main features and auxiliary features between each face image to be recognized and the template image when it is successfully recognized as the template image;

[0058] S107: Generate a main recognition threshold and an auxiliary recognition threshold corresponding to the template image according to the main features and auxiliary features included in the recognition record.

[0059] In an embodiment of the present application, during the face recognition process, for each template image, whenever a face image to be recognized is successfully recognized as the template image, the template image, as well as the main features and auxiliary features between the face image to be recognized and the template image are recorded accordingly to obtain a recognition record.

[0060] In one embodiment, the identification information of the template image, the time point when the recognition is successful, and the main features and auxiliary features can be stored correspondingly, such as in a mapping table. For each template image, according to the identification information of the template image and the time start point and time end point corresponding to the historical time period, the mapping table is queried to obtain the storage items whose time points are between the time start point and time end point corresponding to the historical time period and whose identification information is consistent with the identification information of the template image, and obtain the recognition record when the template image is successfully recognized in the historical time period.

[0061] For each template image, the recognition threshold corresponding to the template image can be dynamically generated according to the main features and auxiliary features contained in the recognition record corresponding to the template image, including the main recognition threshold and the auxiliary recognition threshold. In one embodiment, the average value of the main features and auxiliary features contained in the recognition record can be calculated, and the average value is used as the main feature and auxiliary recognition threshold newly generated by the template image. Further, before calculating the average value, the maximum and minimum values ​​of the main features and auxiliary features contained in the recognition record can be deleted to obtain the remaining main features and auxiliary features, and then the average value is calculated based on the remaining main features and auxiliary features to obtain the recognition threshold, thereby avoiding some abnormal extreme similarity values ​​in the recognition record from affecting the accuracy of the recognition threshold. In another embodiment, the main features and auxiliary features that appear the most times in the recognition record (i.e., the mode of the main features and auxiliary features) can also be counted, and then the mode of the main features and auxiliary features is subtracted from the preset value to obtain the main features and auxiliary recognition thresholds.

[0062] In the embodiment of the present application, for each template image, the main features and auxiliary recognition thresholds corresponding to the template image can be generated based on the main features and auxiliary features in the recognition record when it was successfully recognized within the historical time period, that is, the corresponding main features and auxiliary recognition thresholds can be dynamically generated according to the actual face recognition situation in the historical time period. Therefore, the main features and auxiliary recognition thresholds can be made more in line with the actual face recognition situation, thereby improving the accuracy of face recognition.

[0063] Optionally, the historical time period includes a preset number of sub-time periods, and the above step S107, such as Figure 4 As shown, it specifically includes steps S1071 to S1073:

[0064] S1071: Determine the sub-average values ​​of the main features and the auxiliary features included in each sub-time period according to the recognition record;

[0065] S1072: Calculate a target average value based on each sub-average value; the target average value is the average value of each sub-average value;

[0066] S1073: Generate main features and auxiliary recognition thresholds corresponding to the template image according to the target average value.

[0067] In the embodiment of the present application, the historical time period may specifically include a preset number of sub-time periods. For example, the historical time period may be one week, and the sub-time period may be one day, and the historical time period includes 7 sub-time periods.

[0068] After obtaining the recognition record corresponding to the template image, the main features and auxiliary features contained in each sub-time period are distinguished according to the information of the time point contained in the recognition record. In one embodiment, the main features and auxiliary features contained in each sub-time period can be stored corresponding to the sub-time period. For example, according to the time point corresponding to each sub-time period, the main features and auxiliary features in the sub-time period can be stored in a mapping table corresponding to the sub-time period. Different sub-time periods can correspond to different mapping tables, or they can be stored in the same mapping table, which is not limited here.

[0069] For each sub-time period, a weighted average solution operation is performed according to each main feature and auxiliary feature contained in the sub-time period, so that the weight of the sub-time period closer to the current time is higher. For example, in the past four weeks, the weights of the first week (farthest from the current time) and the second week are 1, the weight of the third week is 2, and the weight of the fourth week (closest to the current time) is 4. The average values ​​of the main features and auxiliary features in these sub-time periods are multiplied by the corresponding weights, and then divided by the total weight value 7 (1+2+4) to obtain the weighted average value, which is the sub-average value. Similarly, the maximum value and the minimum value of each main feature and auxiliary feature contained in the sub-time period can be deleted, and then the weighted average solution operation is performed, so that a more accurate sub-average value can be obtained. In one embodiment, since the weight of the sub-time period closer to the current time is higher, the importance of the main and auxiliary features closer to the current time is higher, so that the recognition threshold is closer to the current facial features of the person to be recognized. For each sub-time period, the main features and auxiliary features contained in each sub-time period can be read from the mapping table to obtain the sub-weighted average value corresponding to the sub-time period; then, the identification information of the template image, the information of the sub-time period and the sub-weighted average value are stored correspondingly in the mapping table that stores the sub-average values.

[0070] After obtaining the sub-average values ​​corresponding to each sub-time period included in the historical time period, each sub-average value is added and divided by a preset number to obtain the average value of each sub-average value, which is called the target average value. In one embodiment, the sub-average values ​​corresponding to each sub-time period of the template image in the historical time period can be read from a mapping table storing sub-average values ​​and calculated to obtain the target average value. Exemplarily, assuming that the sub-time period is one day and the historical time period is one week, the sub-average values ​​corresponding to each day in the week are added and divided by 7 to obtain the target average value corresponding to the week.

[0071] Afterwards, the main recognition threshold and auxiliary recognition threshold of the template image are generated according to the target average value. In one embodiment, the target average value can be directly used as the main recognition threshold and auxiliary recognition threshold newly generated for the template image. In another embodiment, the target average value can be multiplied by a certain weight or added with a preset value to serve as the main recognition threshold and auxiliary recognition threshold corresponding to the template image.

[0072] In the embodiment of the present application, since the sub-average values ​​corresponding to each sub-time period in the historical time period can be obtained first, and then the target average value can be obtained based on each sub-average value, the efficiency of solving the average value can be improved, thereby improving the efficiency of generating the main recognition threshold and the auxiliary recognition threshold.

[0073] Figure 5A schematic flow chart of a face recognition method provided by another embodiment of the present application is shown. After the above step S104, the face recognition method further includes:

[0074] S108: Sending a prompt message matching the recognition result.

[0075] In the embodiment of the present application, after the recognition result is determined, a prompt message matching the recognition result can be issued in any form such as text, voice, image, etc.

[0076] In one embodiment, if the current recognition result is a successful recognition, a first prompt message is issued. The first prompt message may include a prompt indicating "successful recognition". Furthermore, the first prompt message may also include the identity information corresponding to each template image. The storage unit stores the correspondence between the template image and its corresponding identity information. The identity information may be the user's account number or work number, or may include any one or more of the user's name, ID number, age, and gender. After determining each template image corresponding to the current face image to be recognized, the identity information corresponding to each template image may be obtained. Afterwards, a first preset prompt message is issued by way of text display or voice broadcast. The first preset prompt message at least includes the identity information corresponding to the target image, so that the management personnel can obtain the information of the personnel who have successfully passed the face recognition, further improving the intelligence of the face recognition.

[0077] In another embodiment, if the current recognition result is a recognition failure, a second prompt message is issued. The second prompt message may include a prompt indicating "recognition failed". Furthermore, the second prompt message may also include instruction information prompting the user to apply for permission, so that the user can enter the permission application information according to the instruction information, so that after the permission application information is verified by the electronic device or the server, the user's facial image is stored as a template image in a preset storage unit, and corresponding personalized primary and auxiliary recognition thresholds are set for the template image.

[0078] In the embodiment of the present application, since a prompt message corresponding to the recognition result can be issued after the recognition result is obtained, the current face recognition result can be fed back to the user or manager in a timely manner, thereby improving the intelligence of face recognition.

[0079] Optionally, each template image has at least two recognition thresholds corresponding to different lighting conditions respectively; determining the recognition result of the face image to be recognized based on the auxiliary features and the recognition threshold corresponding to each template image includes:

[0080] Get the current target lighting conditions;

[0081] Determine a target recognition threshold corresponding to each template image according to the target illumination condition; wherein the target recognition threshold is a recognition threshold matched with the target illumination condition determined from at least two recognition thresholds corresponding to different illumination conditions;

[0082] Based on the auxiliary features and the target recognition threshold, the recognition result of the face image to be recognized is determined.

[0083] In the embodiment of the present application, in the face recognition application scenario, there are at least two lighting conditions. Exemplarily, the lighting conditions may include: sunlight during the day and night light illumination at night. Exemplarily, the sunlight during the day can be further subdivided into: illumination conditions under different weather conditions such as sunny day illumination, cloudy day illumination, and rainy day illumination.

[0084] Since for the same person, the effects of different facial images obtained by photographing the person under different lighting conditions are different, in an embodiment of the present application, the corresponding multiple recognition thresholds under different lighting conditions can be stored in advance for the template image of the same person.

[0085] During the face recognition process, the current lighting condition is obtained as the target lighting condition.

[0086] After determining the auxiliary features and each template image, according to the target illumination condition, a recognition threshold matching the target illumination condition is selected as the target recognition threshold from a plurality of recognition thresholds stored corresponding to the target identification information of each template image.

[0087] Afterwards, based on the auxiliary feature and the target recognition threshold under the target illumination condition, the recognition result of the face image to be recognized under the target illumination condition is determined.

[0088] In the embodiment of the present application, in addition to being able to flexibly obtain the personalized recognition threshold corresponding to the current person, the corresponding target recognition threshold can also be obtained according to the current lighting conditions. Therefore, the accuracy of determining the recognition threshold can be further guaranteed, thereby improving the accuracy of face recognition.

[0089] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0090] Embodiment 2:

[0091] Figure 6 A schematic diagram of the structure of a face recognition device provided in an embodiment of the present application is shown. For ease of explanation, only the part related to the embodiment of the present application is shown:

[0092] The face recognition device comprises: an acquisition unit 61, an auxiliary feature determination unit 62, and a recognition result determination unit 63. Among them:

[0093] The acquisition unit 61 is used to acquire the main features of the face image to be recognized, where the main features are face features.

[0094] The auxiliary feature determination unit 62 is used to obtain the main features of the face image to be identified; obtain the main features of the template image; for each template image, based on the main features of the template image and the main features of the face image to be identified, respectively calculate the auxiliary features between the face image to be identified and the template image.

[0095] The recognition result determination unit 63 is used to determine the recognition result of the face image to be recognized based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, wherein each pre-stored template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0096] Optionally, the main features are extracted using one of geometric feature methods, algebraic features, statistical learning, Gabor, Fisher-vector, scale-invariant feature transform, local binary pattern, high-dimensional local binary pattern, and deep learning to generate sub-features related to M key points of the face, and the sub-features of the M key points are combined into an M-dimensional vector.

[0097] Optionally, the auxiliary feature determination unit is specifically used to perform inner product on the sub-features under the key points contained in the main features of the face image to be identified and the sub-features under the key points contained in the main features of the template image, so as to generate an inner product result under the key points; if the inner product result is greater than the main recognition threshold of the template image at the key points, the intermediate recognition result under the corresponding key points is set to a first value; if the inner product result is not greater than the main recognition threshold of the template image at the key points, the intermediate recognition result under the corresponding key points is set to a second value; based on the intermediate recognition results under each key point, the auxiliary features between the face image to be identified and the template image are determined:

[0098] Optionally, the auxiliary feature is an M-dimensional binary sequence including the first value and the second value.

[0099] Optionally, the face recognition device further includes:

[0100] A recognition record acquisition unit, used to acquire, for each template image, a recognition record when the template image is successfully recognized within a historical time period, wherein the recognition record includes auxiliary features between each face image to be recognized and the template image when the face image to be recognized is successfully recognized as the template image;

[0101] The recognition threshold generation unit is used to generate an auxiliary recognition threshold corresponding to the template image according to the auxiliary features included in the recognition record.

[0102] Optionally, the historical time period includes a preset number of sub-time periods, and the recognition threshold generation unit is specifically used to determine the sub-average values ​​of the auxiliary features contained in each of the sub-time periods according to the recognition records; obtain a target average value based on each of the sub-average values; the target average value is the average value of each of the sub-average values; and generate an auxiliary recognition threshold corresponding to the template image based on the target average value.

[0103] Optionally, the face recognition device further includes:

[0104] The prompt unit is used to issue prompt information matching the recognition result.

[0105] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0106] Embodiment three:

[0107] Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a face recognition program. When the processor 70 executes the computer program 72, the steps in the above-mentioned face recognition method embodiments are implemented, such as Figures 1 to 5 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of the acquisition unit 61 to the recognition result determination unit 63 are shown.

[0108] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the electronic device 7.

[0109] The electronic device 7 may be a computing device such as a face recognition attendance machine, a face access control device, a monitoring device, a mobile phone, a desktop computer, a notebook, a PDA, and a cloud server. The electronic device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation of the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0110] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0111] The memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. The memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the electronic device 7. The memory 71 is used to store the computer program and other programs and data required by the electronic device. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0112] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0113] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0118] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0119] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A face recognition method, characterized in that: include: Acquire main features of the face image to be recognized, wherein the main features are face features; Obtain the main features of each template image; For each template image, based on the main features of the template image and the main features of the face image to be identified, calculate the auxiliary features between the face image to be identified and the template image; as well as Determine the recognition result of the face image to be recognized based on the auxiliary features and the auxiliary recognition threshold corresponding to the template image, wherein each template image has its own corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated according to the recognition record when the template image is successfully recognized in a historical time period; The main features are extracted to generate sub-features related to M key points of the face, and the sub-features related to the M key points are combined into an M-dimensional vector; The step of calculating the auxiliary features between the face image to be identified and the template image based on the main features of the template image and the main features of the face image to be identified comprises: For the sub-features related to the M key points contained in the main features, inner products are performed on the sub-features under the key points contained in the main features of the face image to be identified and the sub-features under the key points contained in the main features of the template image, so as to generate inner product results under the key points; If the inner product result is greater than the main recognition threshold of the template image at the key point, the intermediate recognition result at the corresponding key point is set to a first value; if the inner product result is not greater than the main recognition threshold of the template image at the key point, the intermediate recognition result at the corresponding key point is set to a second value; and Determining auxiliary features between the face image to be recognized and the template image based on the intermediate recognition results at each key point; The auxiliary feature is an M-dimensional binary sequence including the first value and the second value.

2. The face recognition method according to claim 1, characterized in that: The extraction of the main features adopts one of the following methods: geometric feature method, algebraic feature, statistical learning, Gabor, Fisher vector, scale-invariant feature transformation, local binary pattern, high-dimensional local binary pattern, and deep learning.

3. The face recognition method according to claim 1, characterized in that: The method further comprises: For each template image, obtaining a recognition record when the template image is successfully recognized within a historical time period, wherein the recognition record includes auxiliary features between each face image to be recognized and the template image when the face image to be recognized is successfully recognized as the template image; An auxiliary recognition threshold corresponding to the template image is generated according to the auxiliary features included in the recognition record.

4. The face recognition method according to claim 3, characterized in that: The historical time period includes a preset number of sub-time periods, and the generating, according to the auxiliary features included in the recognition record, the auxiliary recognition threshold corresponding to the template image includes: Determining, according to the identification records, the sub-average values ​​of the auxiliary features contained in each of the sub-time periods; According to each of the sub-average values, a target average value is obtained; the target average value is the average value of each of the sub-average values; An auxiliary recognition threshold corresponding to the template image is generated according to the target average value.

5. The face recognition method according to any one of claims 1 to 4, characterized in that: After determining the recognition result of the face image to be recognized based on the auxiliary feature and the auxiliary recognition threshold corresponding to the template image, the method further includes: A prompt message matching the recognition result is issued.

6. A face recognition device, characterized in that: include: An acquisition unit, used for acquiring main features of the face image to be recognized, wherein the main features are face features; an auxiliary feature determination unit, configured to obtain the main features of the face image to be identified; obtain the main features of the template image; and for each template image, respectively calculate the auxiliary features between the face image to be identified and the template image based on the main features of the template image and the main features of the face image to be identified; as well as a recognition result determination unit, configured to determine a recognition result of the face image to be recognized based on the auxiliary features and the auxiliary recognition threshold corresponding to each template image, wherein each pre-stored template image has a corresponding auxiliary recognition threshold, and the auxiliary recognition threshold is generated based on recognition records when the template image is successfully recognized within a historical time period; The step of calculating the auxiliary features between the face image to be identified and the template image based on the main features of the template image and the main features of the face image to be identified includes: For the sub-features related to the M key points contained in the main features, inner products are performed on the sub-features under the key points contained in the main features of the face image to be identified and the sub-features under the key points contained in the main features of the template image, so as to generate inner product results under the key points; If the inner product result is greater than the main recognition threshold of the template image at the key point, the intermediate recognition result at the corresponding key point is set to a first value; if the inner product result is not greater than the main recognition threshold of the template image at the key point, the intermediate recognition result at the corresponding key point is set to a second value; and Determining auxiliary features between the face image to be recognized and the template image based on the intermediate recognition results at each key point; The auxiliary feature is an M-dimensional binary sequence including the first value and the second value.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the electronic device implements the steps of the method according to any one of claims 1 to 5.

Images