Video-based face recognition method, device, electronic device, and storage medium

Abstract

This application is applicable to the field of image processing technology and provides a video-based face recognition method, device, electronic device, and storage medium. The video-based face recognition method includes intercepting multiple facial images of the same target person appearing in multiple video frames in a video; for each facial image to be recognized, calculating the similarity between the facial image to be recognized and each pre-stored template image to determine the target similarity; when the proportion of facial images to be recognized with a target similarity greater than a first threshold corresponding to the target template image in all facial images to be recognized is greater than the first proportion, determining the recognition result of the facial image to be recognized. This application can improve the accuracy of video face recognition.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a video-based face recognition method, device, electronic device and storage medium. Background Art

[0002] With the development of image processing and artificial intelligence technologies, facial recognition technology has been widely used, and various facial recognition products have been launched one after another. Typically, facial recognition products set a unified global first threshold, based on which the current facial recognition result is determined. However, this facial recognition method has low accuracy.

[0003] Especially when recognizing faces in videos, since faces appear at various angles in the video and the resolution of the video is generally low, how to accurately recognize faces in the video is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a video-based face recognition method, device, electronic device, and storage medium to solve the problem of how to improve the accuracy of face recognition in the prior art.

[0005] A first aspect of an embodiment of the present application provides a video-based face recognition method, characterized by comprising:

[0006] intercepting multiple facial images of the same target person to be identified that appear in multiple video frames in the video;

[0007] For each face image to be identified, calculate the similarity between the face image to be identified and each pre-stored template image to determine a target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image with the greatest similarity to the face image to be identified among the pre-stored template images;

[0008] When the proportion of the facial images to be identified whose target similarity is greater than a first threshold corresponding to the target template image is greater than a first proportion in all facial images to be identified, determining a recognition result of the facial images to be identified, wherein each pre-stored template image has a corresponding first threshold, and the first threshold is generated based on recognition records when the template image is successfully recognized within a historical time period;

[0009] When the proportion of facial images to be identified whose target similarity is greater than the first threshold corresponding to the target template image in all facial images to be identified is not greater than the first proportion, and the proportion of facial images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all facial images to be identified is greater than the second proportion, the recognition result is determined, wherein the second threshold is less than the first threshold, and the second threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period.

[0010] A second aspect of the embodiments of the present application provides a video-based face recognition device, characterized by comprising:

[0011] an acquisition unit, configured to capture multiple facial images of the same target person to be identified that appear in multiple video frames in the video;

[0012] a target similarity determination unit, configured to respectively calculate the similarity between the face image to be identified and each pre-stored template image to determine a target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image having the greatest similarity with the face image to be identified among the pre-stored template images;

[0013] a recognition result determination unit, configured to determine a recognition result of the face image to be recognized when the proportion of the face images to be recognized whose target similarity is greater than a first threshold value corresponding to the target template image in all the face images to be recognized is greater than the first proportion, wherein each pre-stored template image has a first threshold value corresponding to each, and the first threshold value is generated based on the recognition record when the template image is successfully recognized within a historical time period; and determine the recognition result when the proportion of the face images to be recognized whose target similarity is greater than the first threshold value corresponding to the target template image in all the face images to be recognized is not greater than the first proportion, and the proportion of the face images to be recognized whose target similarity is greater than a second threshold value corresponding to the target template image in all the face images to be recognized is greater than the second proportion, and the second threshold value is less than the first threshold value;

[0014] Each pre-stored template image has a corresponding first threshold and second threshold respectively, and the first threshold and the second threshold are generated according to the recognition records when the template image is successfully recognized within a historical time period.

[0015] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the electronic device implements the steps of the video-based face recognition method.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the electronic device implements the steps of the video-based face recognition method.

[0017] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of the video-based face recognition method as described in the first aspect.

[0018] Compared with the prior art, the embodiments of the present application have the following advantages: In the embodiments of the present application, a facial image to be recognized is obtained, and the similarities between the facial image to be recognized and each pre-stored template image are calculated. The template image with the greatest similarity to the facial image to be recognized is determined as the target template image, and the similarity corresponding to the target template image is determined as the target similarity. Next, a determination is made as to whether the proportion of facial images to be recognized whose target similarity exceeds a first threshold corresponding to the target template image is greater than a first percentage among all facial images to be recognized; if so, recognition is successful; if not, a determination is made as to whether the proportion of facial images to be recognized whose target similarity exceeds a second threshold corresponding to the target template image is greater than a second percentage among all facial images to be recognized; if so, recognition is successful; if not, recognition fails. Because each template image has its own corresponding first and second thresholds, determining the recognition result based on the target similarity and the first and second thresholds corresponding to the target template image can achieve personalized first and second thresholds, more accurately applying the target similarity to the current facial image to be recognized for face recognition. Compared with the existing method of setting only a unified threshold, this method can improve the accuracy of face recognition. Furthermore, since the first and second thresholds are generated based on the recognition records of successful template image recognition within a historical time period, that is, the first and second thresholds are dynamically generated based on the actual recognition situation during the historical time period, they can be made more consistent with the actual face recognition situation, further improving the accuracy of video face recognition. Finally, the embodiments of the present application solve the problem of only screening similar facial images captured from a video once, which can easily lead to deviations in the recognition results, thereby improving the accuracy of video face image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art.

[0020] Figure 1 This is a schematic diagram of an implementation flow of a face recognition method provided in an embodiment of the present application;

[0021] Figure 2This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0023] Figure 4 This is a flowchart of a specific implementation of step S107 in a face recognition method provided in an embodiment of the present application;

[0024] Figure 5 This is a flowchart for the specific implementation of step S102 in a face recognition method according to an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of an implementation flow of a face recognition method provided by another embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a face recognition device provided in an embodiment of the present application;

[0027] Figure 8 Schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0029] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0030] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0031] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0032] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0033] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0034] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0035] Typically, facial recognition products set a unified global threshold, which is used to determine the current facial recognition result. When a certain person's face is prone to misidentification, this is usually resolved by raising the global threshold. However, raising the global threshold can make it difficult to recognize other faces, resulting in a lower success rate. Therefore, using a unified global threshold for facial recognition results in lower accuracy.

[0036] In order to solve this technical problem, an embodiment of the present application provides a video-based face recognition method, which is characterized in that it includes: intercepting several face images to be identified of the same target person appearing in multiple video frames in the video; based on each face image to be identified, performing the following steps: calculating the similarity between the face image to be identified and each pre-stored template image to determine the target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image with the greatest similarity to the face image to be identified among the pre-stored template images. When the proportion of the face images to be identified whose target similarity is greater than the first threshold corresponding to the target template image in all the face images to be identified is greater than the first proportion, the recognition result of the face images to be identified is determined, wherein each pre-stored template image has its own corresponding first threshold, and the first threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period; when the proportion of the face images to be identified whose target similarity is greater than the first threshold corresponding to the target template image in all the face images to be identified is not greater than the first proportion, and the proportion of the face images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all the face images to be identified is greater than the second proportion, the recognition result is determined, wherein the second threshold is less than the first threshold, and the second threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period.

[0037] Since each template image has its own corresponding first and second thresholds, the method of determining the recognition result based on the target similarity and the first and second thresholds corresponding to the target template image can achieve personalized thresholds, and more accurately perform face recognition based on the first and second thresholds applicable to the current face image to be recognized. Compared with the existing method of setting only a unified threshold, it can improve the accuracy of video face recognition. In addition, since the first and second thresholds are generated based on the recognition records when the template image was successfully recognized in the historical time period, that is, the first and second thresholds are dynamically generated based on the actual recognition situation in the historical time period, it can make the first and second thresholds more consistent with the actual face recognition situation, further improving the accuracy of video face recognition.

[0038] Example 1:

[0039] Figure 1 The flowchart of a video-based face recognition method provided by an embodiment of the present application is shown. The execution subject of the face recognition method is an electronic device. The electronic device can be a device with a camera module, such as a monitoring device, a mobile phone, a camera, etc.; or the electronic device can be other computing devices connected to a device with a camera module, such as a computer connected to a monitoring camera, a camera, etc. Figure 1 The face recognition method shown is detailed as follows:

[0040] In S101, several facial images of the same target person appearing in multiple video frames in the video are captured. For the multiple video frames appearing in the video, facial images whose positions in the multiple video frames are within a reasonable range are captured to form a group of facial images to be recognized. That is, the embodiment of the present application captures a group of facial images to be recognized based on the continuity of the facial images in the video in time and space. The facial images to be recognized are preset as facial images of the same person, and it is necessary to confirm whether they are the same target person and their identity in subsequent steps.

[0041] More specifically, the present invention can obtain facial images of a person that are preset to belong to the same target person in multiple video frames of a video and determine whether the facial position of the person in the previous video frame and the facial position of the person in the next video frame reasonably appear within a specific range. If so, it can be inferred that the two facial images belong to the same target person. In this way, multiple facial images are obtained to form a group of facial images to be identified.

[0042] In terms of the temporal continuity of facial images, in the embodiment of the present application, the facial images of the same target person appearing in at least two video frames played consecutively are exemplarily considered to be temporally continuous. In other cases, the consecutive video frames can be separated by a certain number, for example, one is taken every five video frames. As for the spatial position continuity of facial images in the embodiment of the present application: in the embodiment of the present application, the facial images of the same target person appearing at substantially the same position in two video frames played consecutively are exemplarily considered to be spatially continuous. In other words, when the facial images of the same target person appear in two adjacent video frames, and the distance difference between the position of the facial image of the same target person in the previous video frame and its position in the next video frame is within a reasonable range, the facial images are considered to be continuous in terms of temporal and spatial positions. The above-mentioned facial images of the same target person only represent facial images whose facial features are similar enough to meet the predetermined similarity requirements, and are not confirmed to be the same target person.

[0043] In this embodiment, the reasonable range refers to the range of distances within which a face in the preceding and following frames can reasonably move. If the image is within the reasonable range, indicating that the same target person could have moved that distance within the timeframe of the preceding and following frames, it is classified as a face image to be identified. If it is outside the reasonable range, indicating that the same target person could not have moved such a great distance within the timeframe of the preceding and following frames, in which case the following frame image is excluded from the face image to be identified. Thus, by determining spatiotemporal continuity, the face image to be identified is likely to be the same target person.

[0044] For example, when a video frame is detected where a facial image first appears, a facial image to be identified is created for each facial image in that frame. For example, no facial images appear in frames 1 to 100 of a video, but two facial images appear in frames 101 to 300, and these images are temporally and spatially continuous. They belong to two different individuals, defined here as the first and second facial images. Three facial images appear in frames 301 to 400. Based on temporal and spatial continuity, two of these individuals are considered to be the same person as the two individuals in frames 101 to 300, and are therefore categorized as the first and second facial images, respectively. The first facial image to be identified contains the facial features and key points of the first facial image, while the second facial image to be identified contains the facial features and key points of the second facial image. For the third individual, who appears only in frames 301 to 400, a third facial image to be identified can be created separately.

[0045] At this point, the face images to be recognized are obtained. Then, based on each face image to be recognized, the following steps are performed.

[0046] In S102, the similarity between the face image to be identified and each pre-stored template image is calculated to determine the target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image with the greatest similarity to the face image to be identified among the pre-stored template images.

[0047] In an embodiment of the present application, facial images of various authorized persons collected in advance are pre-stored as template images in a storage unit of the electronic device itself or a third-party storage unit that the electronic device can access.

[0048] After obtaining the face image to be recognized, the similarity between each face image to be recognized and each template image is calculated. For example, for a face image to be recognized, there are currently N template images (N is a positive integer greater than 1), then the similarity between the face image to be recognized and each of the N template images is calculated one by one, thereby obtaining the N similarity values ​​t1~t corresponding to the face image to be recognized. N For N facial images to be recognized, there are N*N similarity values. Then, a maximum similarity value is determined from the N*N similarity values. The template image corresponding to the maximum similarity value is used as the target template image, and the similarity between all facial images to be recognized and the target template image is used as the target similarity.

[0049] In one embodiment, the similarity between the face image to be identified and the template image may be cosine similarity. As a possible implementation, the similarity between the face image to be identified and the template image may be calculated using a pre-trained neural network model.

[0050] In S103, when the proportion of facial images to be identified whose target similarity is greater than the first threshold corresponding to the target template image is greater than the first proportion in all facial images to be identified, the recognition result of the facial image to be identified is determined, wherein each pre-stored template image has its own corresponding first threshold, and the first threshold is generated based on the recognition record when the template image is successfully recognized within a historical time period.

[0051] In the embodiment of the present application, for each pre-stored template image, there is at least one first threshold value corresponding to it. For example, N facial images of authorized persons can be stored in advance as N template images, numbered 1 to N, and their corresponding first threshold values ​​are T1 to T2. N Moreover, the first threshold corresponding to each template image is dynamically generated based on the recognition records of the template image being successfully recognized in the historical time period. The historical time period can be a time period of a preset length, such as one day, one week, or one month. For example, for template image i, its current corresponding first threshold T i , can be obtained by calculating (e.g., averaging) the similarities between each facial image identified as template image i over the past week and template image i. It is understood that in the initial state where no corresponding recognition record has been generated for each template image, the first threshold corresponding to that template image can be a pre-set initial value.

[0052] After determining the target similarity and the target template image, a pre-stored first threshold corresponding to the target template image is obtained. In one embodiment, each template image has corresponding identification information, which may be an identity identification number (ID). The identification information of the template image is stored in correspondence with its first threshold, such as in a mapping table storing the first threshold. The identification information corresponding to the target template image is the target identification information. Using the target identification information as an index, a query is performed in the mapping table storing the first threshold to obtain the first threshold of the target template image.

[0053] The target similarity determined in step S102 is then compared with the first threshold of the currently acquired target template image to determine the recognition result of the facial image to be recognized. In one embodiment, when the proportion of facial images to be recognized having a target similarity greater than the first threshold corresponding to the target template image is greater than the first proportion among all facial images to be recognized, the recognition result of the current facial image to be recognized is determined to be: recognition successful.

[0054] In S104, when the proportion of the face images to be identified whose target similarity is greater than the first threshold corresponding to the target template image in all the face images to be identified is not greater than the first proportion, and the proportion of the face images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all the face images to be identified is greater than the second proportion, the recognition result is determined, wherein the second threshold is less than the first threshold, and the second threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period.

[0055] In the embodiment of the present application, for each pre-stored template image, there is at least one corresponding second threshold value. Similarly, N facial images of authorized persons can be stored in advance as N template images, numbered 1 to N, and their corresponding second threshold values ​​are S1 to S2. N Moreover, the second threshold corresponding to each template image is dynamically generated based on the recognition records of the template image being successfully recognized in the historical time period. The historical time period can be a time period of a preset length, such as one day, one week, or one month. For example, for template image i, its current corresponding second threshold S i , can be obtained by calculating (e.g., averaging) the similarities between each facial image identified as template image i over the past week and template image i. It is understood that in the initial state where no corresponding recognition record has been generated for each template image, the corresponding second threshold value can be a pre-set initial value.

[0056] The facial image to be identified that was not successfully identified in S103 is compared again, this time with the second threshold. In one embodiment, under the premise that the identification was not successful in S103, if the proportion of facial images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all facial images to be identified is greater than the second proportion, then the current recognition result of the facial image to be identified can also be determined as: successful identification. In another embodiment, if the proportion of facial images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all facial images to be identified is not greater than the second proportion, then the current recognition result of the facial image to be identified is determined as: failed identification.

[0057] Since the second threshold is smaller than the first threshold, a facial image to be identified that is not successfully identified based on the first threshold may be successfully identified based on the second threshold.

[0058] For example, if the first proportion is set to 60% and the second proportion is set to 80%. In the first case, if the proportion of face images to be identified with a target similarity greater than the first threshold is 65% of all face images to be identified, and the proportion of face images to be identified with a target similarity greater than the second threshold corresponding to the target template image is 75% of all face images to be identified, since 65% is greater than the first proportion of 60%, the recognition result is a successful recognition, and there is no need to refer to the second threshold. In the second case, if the proportion of face images to be identified with a target similarity greater than the first threshold is 55% of all face images to be identified, and the proportion of face images to be identified with a target similarity greater than the second threshold corresponding to the target template image is 75% of all face images to be identified, since 55% is not greater than the first proportion of 60%, and 75% is not greater than the second proportion of 80%, the recognition result is a failed recognition. In the third case, if the proportion of facial images to be identified with a target similarity greater than the first threshold is 55% of all facial images to be identified, and the proportion of facial images to be identified with a target similarity greater than the second threshold corresponding to the target template image is 90% of all facial images to be identified, since 55% is not greater than the first proportion of 60%, but 90% is greater than the second proportion of 80%, the recognition result is successful. Successful recognition indicates that the facial image to be identified and the target template image are the same person, and failed recognition indicates that the facial image to be identified and the target template image are not the same person.

[0059] Since each template image has its own corresponding first threshold and second threshold, the method of determining the recognition result based on the target similarity and the first threshold and second threshold corresponding to the target template image can realize the personalization of the threshold, and more accurately perform face recognition based on the first threshold and second threshold applicable to the current face image to be recognized. Compared with the existing method of setting only a unified threshold, it can improve the accuracy of video face recognition. Moreover, since the first threshold and the second threshold are generated based on the recognition records when the template image is successfully recognized in the historical time period, that is, the first threshold and the second threshold are dynamically generated based on the actual recognition situation of the historical time period, it can make the first threshold and the second threshold more consistent with the actual face recognition situation, further improving the accuracy of video face recognition. Finally, the embodiment of the present application solves the problem that similar face images intercepted from the video are only screened once, which easily leads to deviations in the recognition results, thereby improving the accuracy of video face image recognition.

[0060] Figure 2 A schematic flow chart of a video face recognition method provided in another embodiment of the present application is shown. After step S104, the video face recognition method further includes:

[0061] S105: If the recognition result of the face image to be recognized is successful, the target similarity and the target template image are stored in correspondence.

[0062] In an embodiment of the present application, in step S104, when the recognition result of the face image to be recognized is successful, the target similarity can be stored in correspondence with the target template image, so that subsequent query tracking or statistical analysis can be performed based on the target similarity when the recognition is successful. In one embodiment, the target similarity is stored in correspondence with the target template image, specifically, the current recognition time point, the target similarity, and the target identification information corresponding to the target template image can be stored in correspondence, such as in a mapping table, for subsequent reference and analysis.

[0063] Figure 3 A flowchart of a face recognition method provided by another embodiment of the present application is shown. After step S104 above, the face recognition method further includes:

[0064] S106: For each template image, obtaining a recognition record of the template image being successfully recognized within a historical time period, the recognition record including a similarity between each face image to be recognized and the template image when the face image is successfully recognized as the template image;

[0065] S107: Generate a first threshold and a second threshold corresponding to the template image according to the similarity included in the recognition record.

[0066] In an embodiment of the present application, during the face recognition process, for each template image, whenever a face image to be recognized is successfully recognized as the template image, the template image and the similarity between the face image to be recognized and the template image are recorded accordingly to obtain a recognition record.

[0067] In one embodiment, the template image's identification information, the time point at which successful recognition occurred, and the similarity can be stored in a corresponding manner, for example, in a mapping table. For each template image, the mapping table is queried based on the template image's identification information and the start and end times of the historical time period. The mapping table retrieves the storage items whose time points fall between the start and end times of the historical time period and whose identification information matches the template image's identification information, thereby obtaining the recognition records of the template image's successful recognition within the historical time period.

[0068] For each template image, the first threshold and second threshold corresponding to the template image can be dynamically generated based on the various similarities contained in the recognition record corresponding to the template image. In one embodiment, the average value of the various similarities contained in the recognition record can be calculated, and the average value can be used as the first threshold and second threshold newly generated for the template image. Furthermore, the similarities contained in the recognition record are sorted from high to low; based on the similarities ranked before the benchmark value after sorting, a first average value is calculated; based on the first average value, the first threshold corresponding to the template image is generated; based on the second average value, the similarities ranked after the benchmark value after sorting are calculated; and based on the second average value, the second threshold corresponding to the template image is generated.

[0069] For example, assume that the identification record includes 10 normalized similarity records, which are 0.71, 0.79, 0.66, 0.93, 0.82, 0.67, 0.66, 0.92, 0.82, and 0.67 respectively. To obtain the first threshold, the similarities contained in the recognition records are first sorted from high to low, i.e., 0.93, 0.92, 0.82, 0.82, 0.79, 0.71, 0.67, 0.67, 0.66, and 0.66. Then, a first average value is calculated based on the similarities before the reference value after sorting. Here, the reference value is set to 50%, that is, the first 50% similarities after sampling and sorting are: 0.93, 0.92, 0.82, 0.82, and 0.79. The average value of these similarities is (0.93+0.92+0.82+0.82+0.79) / 5=0.86, so the first average value is 0.86. Finally, based on the first average value, a first threshold corresponding to the template image is generated. In this embodiment, the first average value is directly set as the first threshold, so the first threshold is 0.86.

[0070] To obtain the second threshold, first calculate the second average value based on the similarities after the sorting and after the benchmark value, that is, calculate the average value of the last 50% of the similarities after sorting, that is, (0.71+0.67+0.67+0.66+0.66) / 5=0.67, so the second average value is 0.67; then, based on the second average value, generate the second threshold corresponding to the template image. In this embodiment, the second average value is directly set as the second threshold, so the second threshold is 0.67.

[0071] In another embodiment, the similarity with the most occurrences in the identification record (ie, the mode of the similarities) may be statistically analyzed, and then a preset value is subtracted from the mode of the similarities to obtain the first threshold and the second threshold.

[0072] In an embodiment of the present application, for each template image, the first threshold and the second threshold corresponding to the template image can be generated based on the similarity in the recognition record when it was successfully recognized within the historical time period, that is, the corresponding first threshold and the second threshold can be dynamically generated according to the actual face recognition situation in the historical time period, so that the first threshold and the second threshold can be made more consistent with the actual face recognition situation, thereby improving the accuracy of face recognition.

[0073] The above step S107 is as follows Figure 4 As shown, it specifically includes steps S1071 to S1075:

[0074] S1071: Sort the similarities contained in the recognition records from high to low;

[0075] S1072: Calculate a first average value based on the similarity between the sorted and the benchmark values;

[0076] S1073: Generate a first threshold corresponding to the template image according to the first average value;

[0077] S1074: Calculate a second average value based on the similarity of the sorted values ​​after the benchmark value;

[0078] S1075: Generate a second threshold corresponding to the template image according to the second average value.

[0079] After obtaining the recognition record corresponding to the template image, the similarities contained in each time period can be stored in correspondence with the time period. For example, the similarities within the time period can be stored in the mapping table corresponding to the time period according to the time point corresponding to each time period. Different time periods can correspond to different mapping tables, or they can be stored in the same mapping table, which is not limited here.

[0080] In the embodiment of the present application, since the first and second average values ​​corresponding to each time period in the historical time period can be obtained first, and then the first threshold and the second threshold are updated according to the first and second average values, the generation efficiency of the first threshold and the second threshold is improved.

[0081] During the face recognition process, the first and second average values ​​are continuously counted, specifically using time periods as the unit length. During this process, when it is detected that the time interval between the current time and the most recently generated time of the first and second threshold values ​​corresponding to the template image reaches a preset time threshold (i.e., the time interval is greater than or equal to the time length corresponding to the historical time period), it indicates that the similarity record for a historical time period has been completed. At this time, the first and second threshold values ​​are updated based on the first and second average values ​​corresponding to each time period within the current historical time period.

[0082] In an embodiment of the present application, since the first / second average value can be calculated when it is detected that the time interval between the current time and the latest generation time of the first threshold and the second threshold reaches the preset time threshold, the dynamic generation of the first threshold and the second threshold can be realized in real time to improve the accuracy of face recognition.

[0083] Alternatively, as Figure 5 As shown, the above step S102 specifically includes steps S1021 to S1022:

[0084] S1021: Calculating the similarity between the face image to be recognized and each pre-stored template image respectively;

[0085] S1022: Determine a maximum similarity based on a preset basic threshold and a similarity with the largest value among the similarities.

[0086] In an embodiment of the present application, the storage unit of this end or a third party also stores a preset basic threshold in advance, and the preset basic threshold is the minimum threshold for face recognition set in advance.

[0087] The similarities between the face image to be identified and each of the pre-stored template images are calculated using a preset algorithm, and then the similarities are checked to see if any of them are greater than a preset basic threshold.

[0088] Specifically, the highest similarity can be determined from the various similarities, and the highest similarity can be compared with the preset basic threshold. If it is determined that the highest similarity is greater than the preset basic threshold, the highest similarity is directly determined as the maximum similarity. Conversely, if the highest similarity is less than the preset basic threshold, it indicates that the similarity between the current facial image to be identified and each of the pre-stored template images is relatively small, and the facial image to be identified may be the facial image of an unauthorized person. In this case, the recognition of the facial image to be identified will be directly determined to have failed.

[0089] In the embodiment of the present application, since a preset basic threshold can be set, the maximum similarity can be further determined efficiently and accurately based on the preset basic threshold and the similarity with the largest value among the similarities, thereby further improving the accuracy and efficiency of face recognition.

[0090] Figure 6 A flowchart of a face recognition method provided by another embodiment of the present application is shown. After step S104 above, the face recognition method further includes:

[0091] S108: issuing a prompt message that matches the recognition result.

[0092] In an embodiment of the present application, after the recognition result is determined, a prompt message matching the recognition result can be issued in any form such as text, voice, image, etc.

[0093] In one embodiment, if the current recognition result is a successful recognition, a first prompt message is issued. The first prompt message may include a prompt indicating "recognition successful". Furthermore, the first prompt message may also include the identity information corresponding to the target template image. The storage unit stores the correspondence between the template image and its corresponding identity information. The identity information may be the user's account number or work number, or may include any one or more of the user's name, ID number, age, and gender. After determining the target template image corresponding to the current facial image to be recognized, the identity information corresponding to the target template image can be obtained. Afterwards, a first preset prompt message is issued through text display or voice broadcast. The first preset prompt message contains at least the identity information corresponding to the target image, so that management personnel can obtain the information of the people in the video, further improving the intelligence of video face recognition.

[0094] In another embodiment, if the current recognition result is a recognition failure, a second prompt message is issued. The second prompt message may include a prompt indicating "recognition failed". Furthermore, the second prompt message may also include instruction information prompting the user to apply for permission, so that the user can enter the permission application information according to the instruction information, so that after the permission application information is verified by the electronic device or the server, the user's facial image is stored as a template image in a preset storage unit, and a corresponding personalized first threshold and second threshold are set for the template image.

[0095] In the embodiment of the present application, since prompt information corresponding to the recognition result can be issued after the recognition result is obtained, the current face recognition result can be fed back to the user or manager in a timely manner, thereby improving the intelligence of video face recognition.

[0096] Optionally, each of the template images has at least two first thresholds and second thresholds corresponding to different lighting conditions, respectively; and determining the recognition result of the face image to be recognized based on the target similarity and the first threshold and the second threshold corresponding to the target template image includes:

[0097] Get the current target lighting conditions;

[0098] Determining a target first threshold and a second threshold corresponding to the target template image according to the target illumination condition; wherein the target first threshold and the second threshold are the first threshold and the second threshold determined from the at least two first thresholds and the second thresholds corresponding to different illumination conditions, and matching the target illumination condition;

[0099] Based on the target similarity and the target first threshold and second threshold, a recognition result of the face image to be recognized is determined.

[0100] In the embodiment of the present application, in the face recognition application scenario, there are at least two lighting conditions. For example, the lighting conditions may include: daytime sunlight and nighttime light. For example, daytime sunlight can be further subdivided into lighting conditions under different weather conditions, such as sunny, cloudy, and rainy days.

[0101] Since for the same target person, the effects of different facial images obtained by shooting the person under each different lighting condition are different, therefore, in an embodiment of the present application, the template image of the same target person can store in advance the corresponding first thresholds and second thresholds under different lighting conditions.

[0102] During the face recognition process, the current lighting condition is obtained as the target lighting condition.

[0103] After determining the target similarity and the target template image, according to the target lighting condition, the first threshold and the second threshold that match the target lighting condition are selected as the target first threshold and the second threshold from the multiple first thresholds and the second thresholds stored corresponding to the target identification information of the target template image.

[0104] Afterwards, based on the target similarity and the target first threshold and the target second threshold under the target illumination condition, a recognition result of the face image to be recognized under the target illumination condition is determined.

[0105] In the embodiment of the present application, in addition to being able to flexibly obtain the personalized first threshold and second threshold corresponding to the current person, it is also possible to obtain the corresponding target first threshold and second threshold according to the current lighting conditions. Therefore, the accuracy of determining the first threshold and the second threshold can be further guaranteed, thereby improving the accuracy of video face recognition.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0107] Example 2:

[0108] Figure 7 The following is a schematic diagram of the structure of a video face recognition device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown:

[0109] The video face recognition device includes: an acquisition unit 71, a target similarity determination unit 72, and a recognition result determination unit 73.

[0110] The acquisition unit 71 is used to capture multiple facial images of the same target person to be identified that appear in multiple video frames in the video.

[0111] The target similarity determination unit 72 is used to respectively calculate the similarity between the face image to be identified and each pre-stored template image to determine the target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image with the greatest similarity to the face image to be identified among the pre-stored template images.

[0112] The recognition result determination unit 73 is used to determine the recognition result of the face image to be recognized when the proportion of the face images to be recognized whose target similarity is greater than the first threshold corresponding to the target template image in all the face images to be recognized is greater than the first proportion, wherein each pre-stored template image has its own corresponding first threshold, and the first threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period; when the proportion of the face images to be recognized whose target similarity is greater than the first threshold corresponding to the target template image in all the face images to be recognized is not greater than the first proportion, and the proportion of the face images to be recognized whose target similarity is greater than the second threshold corresponding to the target template image in all the face images to be recognized is greater than the second proportion, the recognition result is determined, and the second threshold is less than the first threshold; wherein each pre-stored template image has its own corresponding first threshold and second threshold, and the first threshold and second threshold are generated based on the recognition records when the template image is successfully recognized within a historical time period.

[0113] Optionally, the several facial images to be identified of the same target person are determined based on the continuity of the facial images in the video in terms of time and space positions, and the continuity is: the facial image of the same target person appears in two adjacent video frames, and the distance difference between the position of the facial image of the same target person in the previous video frame and the position in the next video frame is within a reasonable range.

[0114] Optionally, the video-based face recognition device further includes:

[0115] The storage unit is configured to store the target similarity and the target template image in correspondence if the recognition result of the face image to be recognized is successful.

[0116] Optionally, the video face recognition device further includes:

[0117] a recognition record acquisition unit, configured to acquire, for each template image, a recognition record of when the template image was successfully recognized within a historical time period, the recognition record including a similarity between each face image to be recognized and the template image when the face image is successfully recognized as the template image;

[0118] The threshold generating unit is configured to generate a first threshold and a second threshold corresponding to the template image according to the similarity contained in the recognition record.

[0119] Optionally, the threshold generation unit is specifically used to sort the similarities contained in the recognition records from high to low; calculate a first average value based on the similarities before the benchmark value after sorting; and generate a first threshold corresponding to the template image based on the first average value.

[0120] Optionally, the threshold generating unit is specifically configured to obtain a second average value based on the similarity after sorting after the reference value; and generate a second threshold corresponding to the template image based on the second average value.

[0121] Optionally, the video-based face recognition device further includes:

[0122] The prompt unit is used to issue prompt information matching the recognition result.

[0123] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0124] Example 3:

[0125] Figure 8 Schematic diagram of an electronic device provided by an embodiment of the present application. Figure 8 As shown, the electronic device 8 of this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a face recognition program. When the processor 80 executes the computer program 82, the steps in the above-mentioned various embodiments of the video-based face recognition method are implemented, such as Figure 1 Alternatively, when the processor 80 executes the computer program 82, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 7 The functions of the acquisition unit 71 to the recognition result determination unit 73 are shown.

[0126] For example, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the electronic device 8.

[0127] The electronic device 8 can be a monitoring device, a mobile phone, a desktop computer, a notebook, a PDA, a cloud server or other computing device. The electronic device can include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that Figure 8 It is only an example of the electronic device 8 and does not constitute a limitation of the electronic device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0128] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0129] The memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. The memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 8. Furthermore, the memory 81 may include both an internal storage unit of the electronic device 8 and an external storage device. The memory 81 is used to store the computer program and other programs and data required by the electronic device. The memory 81 may also be used to temporarily store data that has been output or is about to be output.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0131] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0132] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0136] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0137] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A video-based face recognition method, characterized in that: include: intercepting multiple facial images of the same target person to be identified that appear in multiple video frames in the video, wherein the multiple facial images of the same target person to be identified are determined based on the continuity of the facial images in the video in terms of temporal and spatial positions, wherein the continuity is: the facial image of the same target person appears in two adjacent video frames, and the distance difference between the position of the facial image of the same target person in the previous video frame and the position in the next video frame is within a reasonable range; For each face image to be identified, calculate the similarity between the face image to be identified and each pre-stored template image to determine a target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image with the greatest similarity to the face image to be identified among the pre-stored template images; When the proportion of the facial images to be identified whose target similarity is greater than a first threshold corresponding to the target template image in all facial images to be identified is greater than a first proportion, determining that the recognition result of the facial images to be identified is successful, wherein each pre-stored template image has a corresponding first threshold, and the first threshold is generated based on recognition records when the template image is successfully recognized within a historical time period; When the proportion of facial images to be identified whose target similarity is greater than the first threshold corresponding to the target template image in all facial images to be identified is not greater than the first proportion, and the proportion of facial images to be identified whose target similarity is greater than the second threshold corresponding to the target template image in all facial images to be identified is greater than the second proportion, the recognition result is determined to be a successful recognition, wherein the second threshold is less than the first threshold, and the second threshold is generated based on the recognition records when the template image is successfully recognized within a historical time period.

2. The face recognition method according to claim 1, wherein: The method further comprises: If the recognition result of the face image to be recognized is successful, the target similarity and the target template image are stored in correspondence.

3. The face recognition method according to claim 1, wherein: The method further comprises: For each template image, obtaining recognition records of the template image when it is successfully recognized within a historical time period, the recognition records including the similarity between each face image to be recognized and the template image when it is successfully recognized as the template image; A first threshold and a second threshold corresponding to the template image are generated according to the similarity contained in the recognition record.

4. The face recognition method according to claim 3, wherein: Generating a first threshold corresponding to the template image according to the similarity contained in the recognition record includes: sorting the similarities contained in the identification records from high to low; Calculate the first average value based on the similarity between the sorted values ​​and the benchmark value; A first threshold corresponding to the template image is generated according to the first average value.

5. The face recognition method according to claim 4, wherein: Generating a second threshold corresponding to the template image according to the similarity contained in the recognition record includes: Obtain a second average value based on the similarity of the sorted items after the reference value; A second threshold corresponding to the template image is generated according to the second average value.

6. The face recognition method according to any one of claims 1 to 5, characterized in that: After determining the recognition result, the method further includes: A prompt message matching the recognition result is issued.

7. A video-based face recognition device, characterized in that: include: an acquisition unit, configured to capture multiple facial images of the same target person to be identified that appear in multiple video frames in the video; a target similarity determination unit, configured to respectively calculate the similarity between the face image to be identified and each pre-stored template image to determine a target similarity; wherein the target similarity is the similarity corresponding to the target template image, and the target template image is the template image having the greatest similarity with the face image to be identified among the pre-stored template images; a recognition result determination unit, configured to determine that the recognition result of the face image to be recognized is a successful recognition when the proportion of the face images to be recognized whose target similarity is greater than a first threshold value corresponding to the target template image in all the face images to be recognized is greater than a first proportion, wherein each pre-stored template image has a first threshold value corresponding to each, and the first threshold value is generated based on the recognition record when the template image is successfully recognized within a historical time period; when the proportion of the face images to be recognized whose target similarity is greater than the first threshold value corresponding to the target template image in all the face images to be recognized is not greater than the first proportion, and the proportion of the face images to be recognized whose target similarity is greater than a second threshold value corresponding to the target template image in all the face images to be recognized is greater than a second proportion, determine that the recognition result is a successful recognition, and the second threshold value is less than the first threshold value; Each pre-stored template image has a corresponding first threshold and second threshold respectively, and the first threshold and the second threshold are generated according to the recognition records when the template image is successfully recognized within a historical time period.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the electronic device implements the steps of the video-based face recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the electronic device implements the steps of the video-based face recognition method according to any one of claims 1 to 6.

Images