Facial data archiving method and related equipment

Abstract

The embodiment of the present invention provides a method for archiving face data, the method comprising: obtaining a batch of face data to be archived, the face data comprising face features and face posture features; calculating the pairwise similarities between all face features to obtain the global similarity of each face data; performing threshold screening on the global similarity of each face data according to the face posture features and a preset similarity threshold matrix to obtain a first face data set, the preset similarity threshold matrix being obtained according to the face features and face posture features contained in the archived face data in a preset archive library; archiving the first face data set through the archived face data in the preset archive library. The preset threshold matrix can be used to provide a similarity threshold that is compatible with the face posture features, thereby solving the problem of low archiving rate due to the use of a unified threshold evaluation method in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for archiving face data and related equipment. Background Art

[0002] With the continuous development of artificial intelligence, the related technologies of image recognition are constantly improving, and more and more face recognition applications have been implemented. The archiving of face data is to classify the face data of the same person into the same face file so as to query and process the face data. In the archiving of face data, the archiving method usually adopted is to cluster and archive the face data. However, due to the fact that factors such as the posture, brightness, and clarity of the face have a great influence on the clustering archiving, the archiving rate is low. Among them, when the face posture changes very little, the similarity difference is not large, but after the posture changes to a certain extent, the similarity changes greatly, which in turn affects the archiving rate. Therefore, due to the change of face posture, there is a problem of low archiving rate in the existing face data archiving. Summary of the invention

[0003] An embodiment of the present invention provides a method for archiving face data. The method obtains batch face data from a video stream and extracts corresponding face features for archiving. During the archiving process, a preset threshold matrix is ​​used to perform similarity judgment. Since the preset threshold matrix is ​​obtained based on face posture features of the archived face data, a similarity threshold that is compatible with the face posture features can be provided through the preset threshold matrix, thereby solving the problem of low archiving rate caused by the use of a unified threshold evaluation method in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides a method for archiving face data, the method comprising:

[0005] Acquire batch face data to be archived, wherein the face data includes face features and face posture features;

[0006] Calculate the pairwise similarity between all facial features to obtain the global similarity of each facial data;

[0007] Performing threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library;

[0008] The first face data set is archived using the archived face data in the preset archive library.

[0009] Optionally, the method further includes:

[0010] Obtain archived face data in a preset archive library, and extract facial features and facial posture features of the archived face data;

[0011] The preset threshold matrix is ​​constructed based on the facial features and facial posture features of the archived facial data.

[0012] Optionally, the multiple archived face data sets include a frontal face data set and multiple deflected face data sets, the face posture features include face angle feature values, and the preset threshold matrix is ​​constructed according to the face features and face posture features of the archived face data, including:

[0013] For each archived face file in the preset archive library, the face data contained in the archived face file is divided according to the face angle feature value to obtain archived face data sets in multiple different face angle intervals;

[0014] Calculating the average similarity between the archived face data sets based on the face features of the archived face data, or calculating the average similarity between the frontal face data set and the deflected face data set based on the face features of the archived face data;

[0015] According to the average similarity between the archived face data sets, construct an average similarity matrix of each archived face file, wherein each archived face file corresponds to one average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of the divided intervals;

[0016] The preset threshold matrix is ​​calculated based on the average similarity matrix.

[0017] Optionally, one average similarity matrix unit in the average similarity matrix corresponds to one average similarity value, and the step of calculating the preset threshold matrix according to the average similarity matrix includes:

[0018] For all the average similarity matrices, extract the average similarity values ​​in the corresponding average similarity matrix units, add and average them, and obtain a final similarity matrix;

[0019] The final similarity matrix is ​​used as the preset threshold matrix.

[0020] Optionally, archiving the first face data set by using the archived face data in the preset archive library includes:

[0021] Filtering the first face data set according to the face spatiotemporal features to obtain a second face data set;

[0022] The second face data set is archived using the archived face data in the preset archive library.

[0023] Optionally, the second face relationship graph includes a first face relationship cluster, and archiving the second face data set by using the archived face data in the preset archive library includes:

[0024] Based on the second face data set, generating a first face relationship graph;

[0025] Performing strong connectivity calculation on the first face relationship graph to obtain a second face relationship graph;

[0026] The batch face data is archived based on the second face relationship graph.

[0027] Optionally, the second face relationship graph includes a first face relationship cluster, and archiving the batch face data based on the second face relationship graph includes:

[0028] Optimizing the first face relationship cluster according to a preset community discovery algorithm to obtain a second face relationship cluster;

[0029] The batch face data is archived based on the second face relationship cluster.

[0030] In a second aspect, an embodiment of the present invention provides a device for archiving face data, the device comprising:

[0031] An acquisition module, used to acquire batch face data to be archived, wherein the face data includes face features and face posture features;

[0032] A calculation module is used to calculate the pairwise similarities between all facial features to obtain the global similarity of each facial data;

[0033] A first screening module, configured to perform a threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library;

[0034] The archiving module is used to archive the first face data set using the archived face data in the preset archiving library.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method for archiving facial data provided in an embodiment of the present invention when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method for archiving facial data provided in the embodiment of the invention are implemented.

[0037] In an embodiment of the present invention, a batch of face data to be archived is obtained, the face data includes face features and face posture features; the pairwise similarities between all face features are calculated to obtain the global similarity of each face data; the global similarity of each face data is threshold screened according to the face posture features and a preset similarity threshold matrix to obtain a first face data set, the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library; the first face data set is archived through the archived face data in the preset archive library. After obtaining batch face data from a video stream, the corresponding face features are extracted for archiving. During the archiving process, a preset threshold matrix is ​​used for similarity judgment. Since the preset threshold matrix is ​​obtained according to the face posture features of the archived face data, a similarity threshold suitable for the face posture features can be provided through the preset threshold matrix, thereby solving the problem of low archiving rate due to the use of a unified threshold evaluation method in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a method for archiving face data provided by an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a threshold matrix construction method provided by an embodiment of the present invention;

[0041] Figure 3 It is a structural schematic diagram of a facial data archiving device provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of the structure of another device for archiving face data provided by an embodiment of the present invention;

[0043] Figure 5 is a structural diagram of a building module provided by an embodiment of the present invention;

[0044] Figure 6 is a structural schematic diagram of a second computing submodule provided by an embodiment of the present invention;

[0045] Figure 7 is a structural schematic diagram of an archiving module provided by an embodiment of the present invention;

[0046] Figure 8 is a structural schematic diagram of an archiving submodule provided by an embodiment of the present invention;

[0047] Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 , Figure 1 is a flowchart of a method for archiving face data provided by an embodiment of the present invention. Figure 1 As shown, the method for archiving face data includes the following steps:

[0050] 101. Obtain batch face data to be archived.

[0051] In an embodiment of the present invention, the above-mentioned video stream can be a video stream obtained in real time by a camera, or it can be a video stream uploaded by a user. The above-mentioned camera can be a camera installed in various scenes, such as a camera installed indoors or outdoors.

[0052] Furthermore, the above video stream may be a video stream captured by one camera, or may be a video stream captured by multiple cameras.

[0053] The face detection algorithm can be used to obtain batches of face data from the video stream. The face data includes face features and face posture features. Each face data also includes face image information, face confidence information, and face detection frame ID. In the face detection algorithm, the same face detection frame ID will be assigned to the same target identified by the algorithm.

[0054] Each facial data also includes time information and location information. When the camera captures a video stream, the image frame in the video stream includes time information, and the video stream includes location information. The above time information can be understood as the time point when the image frame is collected, and the above location information can be understood as the location where the video stream is collected. The above location information can also be understood as the installation location of the camera. Each camera will be set with corresponding longitude and latitude information as location information when it is installed.

[0055] The time information corresponding to the image frame can be used to obtain the time information of the corresponding face data, and the position information of the corresponding face data can be obtained through the position information of the video stream.

[0056] The facial features corresponding to the facial data can be extracted from the above facial image information through a facial feature extraction algorithm, the facial posture features corresponding to the facial data can be extracted from the above facial image information through a facial state estimation algorithm, and the facial spatiotemporal features corresponding to the facial data can be extracted from the above time information and position information.

[0057] The above-mentioned facial features can be facial feature vectors. Facial feature vectors map the high-dimensional spatial image data of facial images to low dimensions to form a preset dimensional feature vector (such as a 512-dimensional feature vector) that can represent the characteristics of the image itself. Different facial images can be judged to be similar by comparing the distances between facial feature vectors. The closer the facial feature vectors are, the greater the probability that the facial images are the same person's face. The degree of discreteness of the facial feature vector depends on the selected facial key point positions and the degree of distinction of each feature.

[0058] The above-mentioned facial posture feature may be an angle value of the facial orientation deflection, and the facial orientation deflection may include pitch angle deflection, yaw angle deflection, roll angle deflection, etc.

[0059] 102. Calculate the pairwise similarities between all facial features to obtain the global similarity of each facial data.

[0060] In the embodiment of the present invention, one facial data corresponds to one facial feature, and batch facial data corresponds to batch facial features. For the batch facial features, the similarity between each two is calculated. This is equivalent to calculating the similarity of one facial feature with all facial features except itself, and obtaining the global similarity of the facial feature.

[0061] For example, the batch face data is M face data, and M facial features are extracted accordingly. Each facial feature is calculated for similarity with M-1 facial features other than itself to obtain the corresponding global similarity. At this time, the global similarity corresponding to each facial feature has M-1 similarities.

[0062] In a possible embodiment, for each facial data, the highest TopN similarities among M-1 similarities may be selected as the global similarity of the facial data.

[0063] 103. Perform threshold screening on the global similarity of each face data according to the face posture features and the preset similarity threshold matrix.

[0064] In an embodiment of the present invention, the preset threshold matrix is ​​obtained according to the facial posture features of the archived facial data. Specifically, the preset threshold matrix is ​​provided with thresholds corresponding to different facial posture features, and the corresponding thresholds can be matched in the preset threshold matrix according to different facial posture features, and the global similarity can be compared with the corresponding threshold. If there is a similarity greater than the corresponding threshold in the global similarity, the corresponding facial data can be added to the first facial data set, wherein one facial data corresponds to one first facial data set.

[0065] For example, for a face data A, the face posture feature of face data M-1 is A1, and among the remaining M-1 face data, there is any face data B whose similarity with face data A is S1, and the face posture feature of face data B is B1, then the corresponding threshold C1 can be found in the preset threshold matrix according to (A1, B1), and the similarity S1 is compared with the threshold C1. If the similarity S1 is greater than or equal to the threshold C1, the face data B is added to the first face data set of face data A. If the similarity S1 is less than the threshold C1, no processing is performed. The first face data set of face data A includes face data A.

[0066] 104. Archiving the first facial data set using archived facial data in a preset archiving library.

[0067] In the embodiment of the present invention, the first face data set retains face data with a high similarity to the main face data, and the first face data sets corresponding to the main face data with the same face detection frame can be classified into the same file.

[0068] Optionally, the first face data set is screened according to face temporal and spatial features to obtain a second face data set; and the second face data set is archived using archived face data in the preset archive library.

[0069] The above-mentioned face temporal and spatial features include time information and position information, and the first face data set is screened according to the face temporal and spatial features.

[0070] Further, for a first face data set of face data, the face data is the main face data of the first face data set. If two or more face data with the same time information appear in the first face data set, the face data with the highest similarity to the main face data among the two or more face data is retained, and the remaining face data is deleted from the first face data set. If two or more face data with the same time information but different position information appear in the first face data set, the face data with the highest similarity to the main face data among the two or more face data is retained, and the remaining face data is deleted from the first face data set. It is determined whether the position and time relationship between the main face data and each face data meets the first preset relationship. If the position and time relationship between the main face data and a certain face data does not meet the first preset relationship, the certain face data is deleted from the first face data set. Determine whether the position and time relationship between each two face data, except the master face data, conforms to the second preset relationship. If the position and time relationship between two face data does not conform to the second preset relationship, determine whether the two face data conform to the trajectory data corresponding to the master face data, and delete the face data that does not conform to the trajectory data corresponding to the master face data. The trajectory data corresponding to the master face data can be determined according to the face detection frame ID corresponding to the master face data. Specifically, the face data having the same face detection frame ID as the master face data can form corresponding trajectory data with the master face data.

[0071] For example, the position information of two face data are LA and LB, and the time information are TA and TB, respectively. The expression of the position and time relationship is: ε = |LA-LB| / |TA-TB|. The first preset relationship is ε1. If the position and time relationship ε between the main face data and a certain face data is greater than ε1, it means that the position and time relationship between the main face data and the certain face data does not meet the first preset relationship. The second preset relationship is ε2. If the position and time relationship ε between two face data is greater than ε2, it means that the position and time relationship between two face data does not meet the second preset relationship.

[0072] The first face data set is screened by the above method to obtain a second face data set, wherein one second face data set corresponds to one face data.

[0073] In an embodiment of the present invention, the second facial data set retains facial data with a high degree of similarity to the main facial data, and filters out data that does not conform to the spatiotemporal relationship through facial spatiotemporal data, thereby increasing the probability that the facial data in the second facial data set belongs to the same person.

[0074] Furthermore, the second face data set corresponding to the main face data having the same face detection frame may be grouped into the same file.

[0075] In a possible embodiment, the second face data set is clustered and archived through a portion of archived face data, and the archiving of face data is to archive the unarchived face data into the archive corresponding to this portion of archived face data. Specifically, the archived face data also includes an archive tag, and the archived face data is used as the clustering center to cluster the main face data of all second face data sets. After the clustering is successful, the second face data sets belonging to the same archive tag are deduplicated and archived.

[0076] In a possible embodiment, the second face data set is formed into a face relationship graph, and is archived according to face relationship clusters in the face relationship graph. It should be noted that in the face relationship graph, each node represents a face data, and the edge between two nodes represents the similarity between two face data.

[0077] Optionally, the second face data set may be formed into a first face relationship graph; a strong connectivity calculation may be performed on the first face relationship graph to obtain a second face relationship graph; and the batch face data may be archived based on the second face relationship graph. Wherein, both the first face relationship graph and the second face relationship graph are directed graphs.

[0078] It should be noted that in computer graph theory, strongly connected means that there is a path from v1 to v2 (a path is called a path if the points and edges along the path are not repeated) and a path from v2 to v1 between any two points v1 and v2 in a directed graph G. The strongly connected component of the first face relationship graph can be obtained by using the bidirectional traversal and intersection method, and the time complexity is O(N^2+M). The Kosaraju algorithm or the Tarjan algorithm can also be used to obtain the strongly connected component of the first face relationship graph, and the time complexity of both algorithms is O(N+M).

[0079] By performing strong connectivity calculation on the first face relationship graph, some erroneous face data can be eliminated, making the face relationship cluster more accurate.

[0080] Optionally, the second face relationship map includes a first face relationship cluster, and the first face relationship cluster can be optimized according to a preset community discovery algorithm to obtain a second face relationship cluster; the batch face data can be archived based on the second face relationship cluster.

[0081] It should be noted that the second face relationship graph represents the face relationship network. In the face relationship network, some faces are closely connected, while some faces are sparsely connected. In such a face relationship network, the closely connected part can be regarded as a community, in which the nodes inside are closely connected, while the connection between the two communities is relatively sparse. The community discovery algorithm can realize the division of large-scale networks into communities of different granularities in a short time, and there is no need to specify the number of communities. When the modularity is no longer increased, the iteration stops automatically.

[0082] In the embodiment of the present invention, when the distribution of outdoor scene cameras is not very dense and the cameras have different resolutions, the facial features captured in different scenes are very different. Clustering will carry some erroneous data, so the community discovery algorithm can be used to perform secondary optimization on the cluster group to make the first face relationship cluster, thereby improving the accuracy of face clustering.

[0083] In an embodiment of the present invention, a batch of face data to be archived is obtained, the face data includes face features and face posture features; the pairwise similarities between all face features are calculated to obtain the global similarity of each face data; the global similarity of each face data is threshold screened according to the face posture features and a preset similarity threshold matrix to obtain a first face data set, the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library; the first face data set is archived through the archived face data in the preset archive library. After obtaining batch face data from a video stream, the corresponding face features are extracted for archiving. During the archiving process, a preset threshold matrix is ​​used for similarity judgment. Since the preset threshold matrix is ​​obtained according to the face posture features of the archived face data, a similarity threshold suitable for the face posture features can be provided through the preset threshold matrix, thereby solving the problem of low archiving rate due to the use of a unified threshold evaluation method in the prior art.

[0084] Optional, please continue to refer to Figure 2 , Figure 2 is a flowchart of a method for constructing a threshold matrix provided by an embodiment of the present invention. Figure 2 As shown, in Figure 1 Before step 103 of the embodiment, a preset threshold matrix construction method is also included, which specifically includes the following steps:

[0085] 201. Obtain archived face data in a preset archive library, and extract face features and face posture features of the archived face data.

[0086] In an embodiment of the present invention, multiple archived face files can be obtained, each archived face file corresponds to the archived face data of a person, and the archived face data in the archived face file may include cover face data and filed face data, wherein the cover face data may be front face data, and the face orientation deflection of the front face data may include a pitch angle deflection of 0°, a yaw angle deflection of 0°, and a roll angle deflection of 0°.

[0087] For the same archived face file, there are multiple archived face data, each archived face data includes face image information, face confidence information, face detection frame ID, time information and location information. Face feature extraction algorithm can be used to extract face features from the face image information of the archived face data to obtain the face features of the archived face data. Face posture estimation algorithm can be used to estimate the face state of the face image information of the archived face data to obtain the face posture features of the archived face data.

[0088] 202. Construct a preset threshold matrix based on the facial features and facial posture features of the archived facial data.

[0089] In an embodiment of the present invention, for an archived face file, corresponding face features and face posture features are extracted from each archived face data. The face posture feature difference between two archived face data can be calculated, and the similarity between the two archived face data can be calculated, and the similarity between the two archived face data can be associated with the face posture feature difference between the two archived face data.

[0090] Optionally, the above-mentioned facial posture features include facial angle feature values. For each archived facial file in the preset archive library, the facial data contained in the archived facial file is divided according to the facial angle feature values ​​to obtain archived facial data sets in multiple different facial angle intervals; the average similarity between the archived facial data sets is calculated according to the facial features of the archived facial data, or the average similarity between the frontal facial data set and the deflected facial data set is calculated according to the facial features of the archived facial data; based on the average similarity between the archived facial data sets, an average similarity matrix for each archived facial file is constructed, wherein each archived facial file corresponds to an average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of divided intervals; a preset threshold matrix is ​​calculated based on the average similarity matrix.

[0091] By dividing the facial angle feature values ​​and setting the average similarity corresponding to the divided intervals as the similarity threshold, the problem that a single similarity cannot well determine whether the face is similar when the face angle is deflected can be avoided.

[0092] For example, the facial posture feature is the angle feature value of the face yaw angle. The angle feature value of the face yaw angle can be between 0° and 60°. The angle feature value of the face yaw angle can be divided according to a preset angle number. For example, the angle feature value of the face yaw angle is divided into intervals of 0°, (0, 10°], (10°, 20°], (20°, 30°], (30°, 40°], (40°, 50°], (50°, 60°], etc. The above intervals are formed into a matrix, and the following table 1 is obtained:

[0093] Table 1

[0094] 0° (0，10°] (10°，20°] (20°，30°] (30°，40°] (40°，50°] (50°，60°] 0° (0，10°] (10°，20°] (20°，30°] (30°，40°] (40°，50°] (50°，60°]

[0095] The two archived face data are archived face data A and archived face data B. The angle feature value of the face yaw angle of archived face data A is 35°, which falls within the interval of [30°, 40°]. The angle feature value of the face yaw angle of archived face data B is 20°, which falls within the interval of [10°, 20°]. The similarity SAB between archived face data A and archived face data B is calculated. The obtained similarity SAB corresponds to [30°, 40°] and [10°, 20°]. Table 2 is shown below:

[0096] Table 2

[0097] 0° (0，10°] (10°，20°] (20°，30°] (30°，40°] (40°，50°] (50°，60°] 0° (0，10°] (10°，20°] SAB (20°，30°] (30°，40°] SAB (40°，50°] (50°，60°]

[0098] It should be noted that in Table 1 and Table 2, after removing the first row and the first column, the corresponding matrix form is obtained. When there are multiple similarities corresponding to (30°, 40°] and (10°, 20°], the average similarity is calculated for filling. For example, there are two archived face data, namely, archived face data C and archived face data D. The angle feature value of the face yaw angle of archived face data C is 35°, which falls within the interval of (30°, 40°], and the angle feature value of the face yaw angle of archived face data D is 20°, which falls within the interval of (10°, 20°]. The similarity SCD between the archived face data C and the archived face data D is calculated, and the obtained similarity SCD corresponds to (30°, 40°] and (10°, 20°]. The average value S of SAB and SCD is calculated as the average similarity corresponding to (30°, 40°] and (10°, 20°], and Table 3 is obtained as shown below:

[0099] Table 3

[0100] 0° (0，10°] (10°，20°] (20°，30°] (30°，40°] (40°，50°] (50°，60°] 0° (0，10°] (10°，20°] S (20°，30°] (30°，40°] S (40°，50°] (50°，60°]

[0101] Optionally, the average similarity between the frontal face dataset and the deflected face dataset can be calculated based on the facial features of the archived face data.

[0102] For an archived face file, an average similarity table as shown in Table 4 can be obtained. Table 4 is as follows:

[0103] Table 4

[0104] 0° (0，10°] (10°，20°] (20°，30°] (30°，40°] (40°，50°] (50°，60°] 0° a0 a1 a2 a3 a4 a5 a6 (0，10°] a1 b0 b1 b2 b3 b4 b5 (10°，20°] a2 b1 c0 c1 c2 c3 c4 (20°，30°] a3 b2 c1 d0 d1 d2 d3 (30°，40°] a4 b3 c2 d1 e0 e1 e2 (40°，50°] a5 b4 c3 d2 e1 f0 f1 (50°，60°] a6 b5 c4 d3 e2 f1 g0

[0105] We can keep only a single average similarity, as shown in Table 5:

[0106] Table 5

[0107]

[0108]

[0109] Among them, the above a0, a1, a2, a3, a4, a5, and a6 are the average similarities between the frontal face dataset and the deflected face dataset calculated based on the facial features of the archived face data.

[0110] It should be noted that, in the above Tables 4 and 5, the corresponding average similarity matrices are obtained by removing the first row and the first column.

[0111] For multiple archived face files, there are corresponding average similarity matrices. For all the above average similarity matrices, the average similarity values ​​in the corresponding average similarity matrix units can be extracted and added and averaged to obtain a final similarity matrix; the above final similarity matrix is ​​used as the above preset threshold matrix.

[0112] When the face deflection angle is very small, the similarity difference is not large, but after the deflection reaches a certain degree, the similarity changes greatly. The use of a unified threshold evaluation method cannot solve this large deflection problem well, and the evaluation is more complicated. In the embodiment of the present invention, the similarity threshold matrix of the smoothed moving average similarity is used to fit the similarity under multiple postures as the threshold matrix, which can further improve the clustering effect and thus improve the archiving rate of face data.

[0113] It should be noted that the facial data archiving method provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can archive facial data.

[0114] Optional, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a structure of a face data archiving device provided by an embodiment of the present invention. Figure 3 As shown, the device comprises:

[0115] An acquisition module 301 is used to acquire batch face data to be archived, wherein the face data includes face features and face posture features;

[0116] A calculation module 302 is used to calculate the pairwise similarities between all facial features to obtain the global similarity of each facial data;

[0117] A first screening module 303, configured to perform a threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library;

[0118] The archiving module 304 is used to archive the first face data set using the archived face data in the preset archiving library.

[0119] Optional, such as Figure 4 As shown, the device also includes:

[0120] An extraction module 306 is used to obtain archived face data in a preset archive library, and extract face features and face posture features of the archived face data;

[0121] The construction module 307 is used to construct the preset threshold matrix according to the facial features and facial posture features of the archived facial data.

[0122] Optional, such as Figure 5 As shown, the multiple archived face data sets include a frontal face data set and multiple deflected face data sets, the face posture features include face angle feature values, and the construction module 307 includes:

[0123] The division submodule 3071 is used for dividing the face data contained in each archived face file in the preset archive library according to the face angle feature value to obtain archived face data sets in multiple different face angle intervals;

[0124] A first calculation submodule 3072, configured to calculate the average similarity between the archived face data sets according to the face features of the archived face data, or to calculate the average similarity between the frontal face data set and the deflected face data set according to the face features of the archived face data;

[0125] A construction submodule 3073 is used to construct an average similarity matrix of each archived face file according to the average similarity between the archived face data sets, wherein each archived face file corresponds to one average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of the divided intervals;

[0126] The second calculation submodule 3074 is used to calculate the preset threshold matrix according to the average similarity matrix.

[0127] Optional, such as Figure 6 As shown, one average similarity matrix unit in the average similarity matrix corresponds to one average similarity value, and the archiving module 304 includes:

[0128] The extraction unit 30741 is used to extract the average similarity values ​​in the corresponding average similarity matrix units for all the average similarity matrices, add and average them, and obtain a final similarity matrix;

[0129] The acquisition unit 30742 is configured to use the final similarity matrix as the preset threshold matrix.

[0130] Optionally, the archiving module 305 is further used to filter the first face data set according to the temporal and spatial characteristics of the face to obtain a second face data set; and archive the second face data set using the archived face data in the preset archiving library.

[0131] Optional, such as Figure 7 As shown, the archiving module 305 includes:

[0132] An integration submodule 3051, configured to form a first face relationship graph from the second face data set;

[0133] A third calculation submodule 3052 is used to perform strong connectivity calculation on the first face relationship graph to obtain a second face relationship graph;

[0134] The archiving submodule 3053 is used to archive the batch face data based on the second face relationship graph.

[0135] Optional, such as Figure 8 As shown, the second face relationship graph includes the first face relationship cluster, and the archiving submodule 3053 includes:

[0136] An optimization unit 50531 is used to optimize the first face relationship cluster according to a preset community discovery algorithm to obtain a second face relationship cluster;

[0137] The archiving unit 30532 is used to archive the batch face data based on the second face relationship cluster.

[0138] It should be noted that the facial data archiving device provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can archive facial data.

[0139] The face data archiving device provided in the embodiment of the present invention can implement each process implemented by the face data archiving method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0140] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, it includes: a memory 902, a processor 901, and a computer program of a method for archiving face data stored in the memory 902 and executable on the processor 901, wherein:

[0141] The processor 901 is used to call the computer program stored in the memory 902 and execute the following steps:

[0142] Acquire batch face data to be archived, wherein the face data includes face features and face posture features;

[0143] Calculate the pairwise similarity between all facial features to obtain the global similarity of each facial data;

[0144] Performing threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library;

[0145] The first face data set is archived using the archived face data in the preset archive library.

[0146] Optionally, the method executed by the processor 901 further includes:

[0147] Obtain archived face data in a preset archive library, and extract facial features and facial posture features of the archived face data;

[0148] The preset threshold matrix is ​​constructed based on the facial features and facial posture features of the archived facial data.

[0149] Optionally, the multiple archived face data sets include a frontal face data set and multiple deflected face data sets, the face posture features include face angle feature values, and the processor 901 executes the step of constructing the preset threshold matrix based on the face features and face posture features of the archived face data, including:

[0150] For each archived face file in the preset archive library, the face data contained in the archived face file is divided according to the face angle feature value to obtain archived face data sets in multiple different face angle intervals;

[0151] Calculating the average similarity between the archived face data sets based on the face features of the archived face data, or calculating the average similarity between the frontal face data set and the deflected face data set based on the face features of the archived face data;

[0152] According to the average similarity between the archived face data sets, construct an average similarity matrix of each archived face file, wherein each archived face file corresponds to one average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of the divided intervals;

[0153] The preset threshold matrix is ​​calculated based on the average similarity matrix.

[0154] Optionally, one average similarity matrix unit in the average similarity matrix executed by the processor 901 corresponds to one average similarity value, and the step of calculating the preset threshold matrix according to the average similarity matrix includes:

[0155] For all the average similarity matrices, extract the average similarity values ​​in the corresponding average similarity matrix units, add and average them, and obtain a final similarity matrix;

[0156] The final similarity matrix is ​​used as the preset threshold matrix.

[0157] Optionally, the archiving process of the first face data set by using the archived face data in the preset archive library performed by the processor 901 includes:

[0158] Filtering the first face data set according to the face spatiotemporal features to obtain a second face data set;

[0159] The second face data set is archived using the archived face data in the preset archive library.

[0160] Optionally, the second face relationship graph includes a first face relationship cluster, and the processor 901 performs archiving processing on the second face data set by using the archived face data in the preset archive library, including:

[0161] Based on the second face data set, generating a first face relationship graph;

[0162] Performing strong connectivity calculation on the first face relationship graph to obtain a second face relationship graph;

[0163] The batch face data is archived based on the second face relationship graph.

[0164] Optionally, the second face relationship map includes a first face relationship cluster, and the archiving of the batch face data based on the second face relationship map performed by the processor 901 includes:

[0165] Optimizing the first face relationship cluster according to a preset community discovery algorithm to obtain a second face relationship cluster;

[0166] The batch face data is archived based on the second face relationship cluster.

[0167] The electronic device provided in the embodiment of the present invention can implement each process implemented by the method for archiving face data in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0168] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the various processes of the facial data archiving method or the application-side facial data archiving method provided in the embodiment of the present invention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0169] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0170] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for archiving face data, characterized in that: The following steps are involved: Acquire a batch of face data to be archived, the face data including face features and face posture features, the face features are face feature vectors extracted according to a face feature extraction algorithm, the face posture features are angle values ​​of face orientation deflection extracted according to a face state estimation algorithm, and the face orientation deflection includes pitch angle deflection, yaw angle deflection, and roll angle deflection; Calculate the pairwise similarity between all facial features to obtain the global similarity of each facial data; Performing threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library; Archiving the first face data set using the archived face data in the preset archiving library; Obtain archived face data in a preset archive library, and extract facial features and facial posture features of the archived face data; For each archived face file in the preset archive library, the face data contained in the archived face file is divided according to the angle value of the face orientation deflection to obtain a plurality of archived face data sets in different face angle intervals, wherein the plurality of archived face data sets include a frontal face data set and a plurality of deflected face data sets; Calculating the average similarity between the archived face data sets based on the face features of the archived face data, or calculating the average similarity between the frontal face data set and the deflected face data set based on the face features of the archived face data; According to the average similarity between the archived face data sets, construct an average similarity matrix of each archived face file, wherein each archived face file corresponds to an average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of divided intervals; The preset similarity threshold matrix is ​​calculated based on the average similarity matrix.

2. The method for archiving facial data according to claim 1, characterized in that: One average similarity matrix unit in the average similarity matrix corresponds to one average similarity value, and the step of calculating the preset similarity threshold matrix according to the average similarity matrix includes: For all the average similarity matrices, extract the average similarity values ​​in the corresponding average similarity matrix units, add and average them, and obtain a final similarity matrix; The final similarity matrix is ​​used as the preset similarity threshold matrix.

3. The method for archiving facial data according to claim 1, characterized in that: The archiving process of the first face data set by using the archived face data in the preset archiving library includes: Filtering the first face data set according to the face spatiotemporal features to obtain a second face data set; The second face data set is archived using the archived face data in the preset archive library.

4. The method for archiving face data as claimed in claim 3, characterized in that: The archiving process of the second face data set by using the archived face data in the preset archive library includes: Based on the second face data set, generating a first face relationship graph; Performing strong connectivity calculation on the first face relationship graph to obtain a second face relationship graph; The batch face data is archived based on the second face relationship graph.

5. The method for archiving facial data according to claim 4, characterized in that: The second face relationship graph includes the first face relationship cluster, and archiving the batch face data based on the second face relationship graph includes: Optimizing the first face relationship cluster according to a preset community discovery algorithm to obtain a second face relationship cluster; The batch face data is archived based on the second face relationship cluster.

6. A facial data archiving device, characterized in that: The device comprises: An acquisition module, used to acquire batch face data to be archived, wherein the face data includes face features and face posture features, wherein the face features are face feature vectors extracted according to a face feature extraction algorithm, and the face posture features are angle values ​​of face orientation deflection extracted according to a face state estimation algorithm, wherein the face orientation deflection includes pitch angle deflection, yaw angle deflection, and roll angle deflection; A calculation module is used to calculate the pairwise similarities between all facial features to obtain the global similarity of each facial data; A first screening module, configured to perform threshold screening on the global similarity of each face data according to the face posture feature and a preset similarity threshold matrix to obtain a first face data set, wherein the preset similarity threshold matrix is ​​obtained according to the face features and face posture features contained in the archived face data in a preset archive library; An archiving module, configured to archive the first face data set using the archived face data in the preset archiving library; An extraction module, used to obtain archived face data in a preset archive library, and extract facial features and facial posture features of the archived face data; A construction module is used to divide the face data contained in the archived face file according to the angle value of the face orientation deflection for each archived face file in the preset archive library, so as to obtain archived face data sets in multiple different face angle intervals, wherein the multiple archived face data sets include a front face data set and multiple deflected face data sets; calculate the average similarity between the archived face data sets according to the face features of the archived face data, or calculate the average similarity between the front face data set and the deflected face data set according to the face features of the archived face data; construct an average similarity matrix for each archived face file according to the average similarity between the archived face data sets, wherein each archived face file corresponds to an average similarity matrix, and the dimension of the average similarity matrix is ​​related to the number of divided intervals; and calculate the preset similarity threshold matrix according to the average similarity matrix.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method for archiving facial data as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for archiving facial data as described in any one of claims 1 to 5 are implemented.

Images