An occluded face recognition method based on collaborative representation classification and dynamic matching
The occluded face images are processed by the classification and dynamic image-to-category deformation method based on collaborative representation, and the candidate faces are screened and dynamically matched, the impact of occlusion on the accuracy of face recognition is solved, and the recognition rate and efficiency are improved.
Patent Information
- Application Number
- CN202210430153.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-22
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-04-22
AI Technical Summary
Existing methods of occlusion face recognition, especially classification based on sparse representations and coordinated representations, are difficult to effectively deal with occlusion problems in face images, resulting in a decrease in recognition accuracy.
The classification method based on collaborative representation is used to calculate the representation error and filter the candidate faces, and combine the dynamic image to class deformation method to block and dynamically match the images to reduce the impact of occlusion on the recognition results.
Through dynamic matching of local image processing, the impact of occlusion on recognition results is reduced, the recognition rate is improved, and the calculation amount is reduced, and the recognition efficiency is improved.
Smart Images

Figure CN114708640B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of graphic processing, and relates to an occluded face recognition method based on collaborative representation classification and dynamic matching. Background Art
[0002] Face recognition is one of many identity authentication methods. Compared with other identity authentication methods, face recognition has the advantages of simple image acquisition, numerous image databases, low acquisition cost, and rich information. Therefore, it has gradually become a key research object in the field of image processing. However, in actual application scenarios, face images are usually blocked in various ways, which greatly affects the recognition accuracy of face recognition systems.
[0003] Among the existing occluded face recognition methods, the sparse representation-based classification (SRC) method sparsely encodes the image to be recognized on the training dictionary to obtain a sparse linear combination of training samples, and then classifies the test samples by calculating the value of the representation error. However, the SRC method must meet a premise that there are enough training samples for each type of face, while face recognition is a typical small sample problem. The collaborative representation-based classification (CRC) method uses training samples of all categories to collaboratively represent the test samples, solving the small sample problem of SRC, and using the norm to constrain the reconstruction coefficient, effectively reducing the computational overhead in the solution process. However, this classification method based on global image processing still cannot effectively deal with the occlusion problem in face images. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide an occluded face recognition method based on collaborative representation classification and dynamic matching. The representation error of all category samples is calculated using the collaborative representation based classification (CRC) method, and a certain number of candidate faces are screened out from all training samples according to the error size. The sample images of these candidate faces are then divided into blocks and dynamically matched using the dynamic image-to-class warping (DICW) method based on local image processing, and finally the final classification result is found according to the matching distance cost.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A method for occluded face recognition based on collaborative representation and dynamic matching. The specific technical solution is as follows:
[0007] Step 1: Extract features from all training samples to obtain a feature matrix consisting of feature vectors. Normalize each column of the feature matrix and convert each column into a unit vector to obtain a training dictionary A = [A1, A2, ... A k ,…,A K ], where A k It is the union of the feature vectors of all face images of the k-th type of face (i.e. the k-th person).
[0008] Step 2: Use the CRC method to collaboratively encode the image y on the training dictionary A: in Sparse coding coefficient, ε is the regularization parameter. The above formula can be derived as Let P = (A T A+εK) -1 A T , the mapping matrix can be calculated. When the image y appears, y can be projected onto σ through Py to solve the sparse coding coefficients.
[0009] Step 3: Calculate the representation error of each type of face:
[0010] Step 4: Select the I-type face with the smallest representation error as the candidate face, and then construct this I-type face into a candidate face set C = {c1, c2, ..., c i …,c I}.
[0011] Step 5: Perform preliminary classification by judging the size of the representation error: If (γ is the threshold), then it can be determined that the category of the tested face is consistent with c1, the classification result is output and the subsequent steps are no longer executed; if Then proceed to step six.
[0012] Step 6: Divide the image of the tested face y and the candidate face set C into blocks and extract the first-order differential features: Divide y into M blocks, and the first-order differential features of the tested face are y = {y1, y2, ..., y M Assuming that there are K sample images for each type of face in the candidate face set, and each face image is divided into N blocks, the first-order difference feature of the kth image of the i-th candidate face is The first-order difference features of all sample images of the i-th candidate face are:
[0013] Step 7: Use DICW to dynamically match the image block sequence of the sample under test with the image block sequences of all samples in the candidate face set in the time series {1, 2, …, t, …, T}, and find the class with the minimum matching cost as the final classification result: in The face under test and c i The matching distance of human-like faces, D s(t) is the local matching distance obtained according to the matching path at time t.
[0014] The beneficial effects of the present invention are:
[0015] (1) The CRC method based on global image processing can use all categories of training samples to collaboratively represent a tested sample, but it cannot effectively avoid or process local occlusions in face images. Inspired by the DICW method based on local image processing, the present invention divides the candidate face set and the tested sample screened by CRC into several sub-blocks, and constructs them into a candidate face sub-block sequence set and a tested face sub-block sequence according to the natural order of the face, and then dynamically matches the candidate face sub-block sequence with the tested face sub-block sequence from the time dimension and the intra-class dimension. Although this dynamic matching between image blocks cannot directly remove the occlusion in the image or reconstruct the occluded part, it can avoid the large distance error caused by occlusion and use the unoccluded part for recognition, thereby reducing the influence of occlusion on the final classification result and improving the recognition rate.
[0016] (2) Although the DICW method can deal with the occlusion problem in face images to a certain extent, it is difficult to match accurate classification results from all categories of training samples, and the computational complexity of this method is large. The present invention uses the CRC method to quickly screen out a part of faces to construct a candidate face set, and then uses the DICW method to further classify these candidate face images, which effectively reduces the computational complexity of solving the optimal matching path between image block sequences during the dynamic matching process, and is more efficient than the original DICW method.
[0017] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0019] Figure 1 is a flow chart of the present invention;
[0020] Figure 2 It is the recognition rate of the random position synthetic occlusion experiment under U&O condition;
[0021] Figure 3 It is the recognition rate of the random position synthetic occlusion experiment under O&U conditions;
[0022] Figure 4 It is the recognition rate of the random position synthetic occlusion experiment in the O&O case;
[0023] Figure 5 is the recognition rate of the fixed type object occlusion experiment on the sunglasses occlusion test set;
[0024] Figure 6 is the recognition rate of the fixed type object occlusion experiment on the scarf occlusion test set;
[0025] Figure 7 It is the recognition rate of fixed type object occlusion experiment under O&U situation;
[0026] Figure 8 It is the recognition rate of fixed type object occlusion experiment in O&O case on the sunglasses occlusion test set;
[0027] Fig. 9 It is the recognition rate of the fixed type object occlusion experiment under the O&O condition on the scarf occlusion test set. DETAILED DESCRIPTION
[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.
[0029] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0030] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0031] The present invention is a method for occluded face recognition based on collaborative representation and dynamic matching (CR-D), the flow chart of which is as follows: Figure 1 As shown. Specifically including the following steps:
[0032] Step 1: For all training samples, the embodiment of the present invention uses a 300-dimensional Eigenface feature vector as input to obtain a feature matrix composed of feature vectors. Each column of the feature matrix is normalized and converted into a unit vector to obtain a training dictionary A = [A1, A2, ... A k ,…,A K ], where A k It is the union of the feature vectors of all face images of the k-th type of face (i.e. the k-th person).
[0033] Step 2: Perform l2 norm regularization on the image y and the training sample set A, and use the CRC method to perform collaborative sparse coding on the image y on the training dictionary A: in The sparse coding coefficient and ε are regularization parameters. Considering that the least square solution of CRC may become unstable when the number of face samples is large, the regularization parameter is set to (1.43E-6)·n, where n is the number of training samples. The above collaborative sparse coding model can be derived as Let P = (A T A+εK) -1 A T , the mapping matrix can be calculated. When the image y appears, y can be projected onto σ through Py to solve the sparse coding coefficients.
[0034] Step 3: Calculate the representation error of each type of face:
[0035] Step 4: Select the I-type face with the smallest representation error as the candidate face, and then construct this I-type face into a candidate face set C = {c1, c2, ..., c i …,c I}
[0036] Step 5: Perform preliminary classification by judging the size of the representation error: If (γ is the threshold), then it can be determined that the category of the tested face is consistent with c1, the classification result is output and the subsequent steps are no longer executed; if Then proceed to step six.
[0037] Step 6: If a face image is divided into J blocks, and the pixel value at the coordinate (x, y) in the image is f(x, y), then the pixel value of the jth block of the image is f j (x, y), the first-order difference feature of the j-th image is Δf j (x,y)=f j+1 (x,y)-f j (x, y). The images of the tested face y and the candidate face set C are divided into blocks according to the natural order of facial organs and the first-order differential features are extracted: y is evenly divided into M blocks, and the first-order differential features of the tested face are y = {y1, y2, ..., y M}; Assuming that each type of face in the candidate face set has K sample images, and each face image is divided into N blocks, the first-order difference feature of the kth image of the i-th type of candidate face is The first-order difference features of all sample images of the i-th candidate face are: The feature matrix of the candidate face set is
[0038] Step 7: Use DICW to dynamically match the image block sequence of the sample under test with the image block sequence of all samples of each type of face in the candidate face set in the time series {1, 2, …, t, …, T}, that is, match from both the time dimension and the intra-class dimension. The global matching path of the sample under test and the i-th type of face in the candidate face set in the time series is S = (s(1), …, s(t), …, s(T)). The total matching cost is the sum of the local distances of each matching step. The matching path with the minimum total matching cost is the optimal matching path: in The face under test and c i Minimum matching cost of human-like faces, D s(t) is the local matching distance obtained based on the matching path at time t. Here, Euclidean distance is used as the distance metric. The dynamic programming algorithm is used to calculate the value of the minimum matching distance, and the class with the minimum matching cost is found as the final classification result:
[0039] The effectiveness of the present invention in occluded face recognition is verified by two embodiments, namely, the random position synthetic occlusion experiment and the fixed type physical occlusion experiment. Both embodiments are performed on the AR data set, and can be divided into the following three scenarios according to the distribution of occlusion: the training set has no occlusion and the test set has occlusion (U&O), the training set has occlusion and the test set has no occlusion (O&U), and both the training set and the test set have occlusion (O&O). In addition, the regularization parameter in the two embodiments is set to ε = (1.43E-6) · n (n is the number of training samples), the threshold is set to γ = 0.5, all images are resized to 83 × 60 pixels, and the block size in the dynamic matching stage is adjusted to 5 × 5 pixels.
[0040] Example 1: Random position synthesis occlusion experiment
[0041] Random position synthetic occlusion is to replace each randomly positioned square patch (size is 0% to 50% of the original image) in the original image with a black square. This group of experiments randomly selected 8 unobstructed pictures from 2,600 pictures of 100 subjects in the AR face dataset, and after adding random position synthetic occlusion, each person selected 4 pictures as training sets and test sets, and the number of samples of each type of face in the training samples was set to a fixed value of K = 4. The recognition rates of the random position synthetic occlusion experiment in the three cases of U&O, O&U, and O&O are as follows: Figure 2 , Figure 3 , Figure 4 As shown. When the occlusion ratio is large to a certain extent, the recognition rates of the SRC and CRC methods will drop rapidly as the occlusion ratio increases. This is because a large occlusion ratio causes the key parts of the face to be occluded, making it impossible for the face recognition algorithm to accurately recognize the face. Although the method proposed in the present invention has no obvious advantages over the DICW method, its recognition performance is greatly improved compared with the CRC method, and its time performance has obvious advantages over the DICW method, which will be verified in Example 2.
[0042] Example 2: Fixed type physical object occlusion experiment
[0043] This group of experiments selected some face images from the AR data set as training samples to verify the recognition performance of the present invention in the three cases of U&O, O&U, and O&O, and set the number of samples K of each type of face in the training samples to different values to test the impact of different numbers of training samples on the performance of the method of the present invention. The selected images include unobstructed images and images with two types of occlusion: sunglasses occlusion and scarf occlusion. The recognition rates of the fixed type physical occlusion experiment in the three cases of U&O, O&U, and O&O are as follows: Figures 5 to 9 shown. Figure 5and Figure 6 It can be reflected that, when the test set is occluded by sunglasses, the recognition rate of the method of the present invention is slightly lower than that of the DICW algorithm, and is significantly higher than that of the other two methods. However, in the scarf occlusion test set with a larger occlusion area, the recognition rate of each method decreases to varying degrees, while the recognition rate of the method of the present invention is higher than that of the other methods, indicating that the present invention can effectively reduce the impact of occlusion on recognition. Figure 7 The case where the training set is occluded is more difficult to recognize than the case where the test set is occluded. However, the recognition performance of the method of the present invention is better than that of several other methods as a whole. Figure 8 and Fig. 9 This is the case where both the training set and the test set are occluded. It can be seen that even in this case where recognition is very difficult, the method of the present invention can still maintain a relatively high recognition rate. Table 1 shows the average running time of the method of the present invention and several comparison methods on the AR dataset.
[0044] Table 1 Average running time of each method on the AR dataset
[0045]
[0046] It is not difficult to find that the method of the present invention uses CRC to quickly determine the candidate face set, which reduces the amount of calculation for solving the optimal matching path in dynamic matching. Although the time performance of the method of the present invention is not as good as CRC, it is still significantly better than SRC and the DICW method before improvement. Therefore, the method of the present invention can improve the recognition speed to a certain extent.
[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. A method for occluded face recognition based on collaborative representation classification and dynamic matching, characterized by: The method comprises the following steps: Step 1: Extract features from all training samples to obtain a feature matrix consisting of feature vectors; normalize each column of the feature matrix and convert each column into a unit vector to obtain a training dictionary A = [A1, A2, ... A k ,…,A K ], where A k is the union of the feature vectors of all face images of the k-th type of face, i.e. the k-th person; Step 2: Use the CRC method to collaboratively encode the image y on the training dictionary A: in Sparse coding coefficient, ε is the regularization parameter; the above formula is derived as Let P = (A T A+εK) -1 A T , then calculate the mapping matrix. When the image y appears, project y onto σ through Py to solve the sparse coding coefficients. Step 3: Calculate the representation error of each type of face: Step 4: Select the I-type face with the smallest representation error as the candidate face, and then construct this I-type face into a candidate face set C = {c1, c2, ..., c i …,c I }; Step 5: Perform preliminary classification by judging the size of the representation error: If γ is the threshold, then the category of the face being tested is determined to be consistent with c1, the classification result is output and the subsequent steps are not executed; if Then proceed to step six; Step 6: Divide the image of the tested face y and the candidate face set C into blocks and extract the first-order differential features: Divide y into M blocks, and the first-order differential features of the tested face are y = {y1, y2, ..., y M }; Assuming that each type of face in the candidate face set has K sample images, and each face image is divided into N blocks, the first-order difference feature of the kth image of the i-th candidate face is The first-order difference features of all sample images of the i-th candidate face are: Step 7: Use DICW to dynamically match the image block sequence of the sample under test with the image block sequences of all samples in the candidate face set in the time series {1, 2, …, t, …, T}, and find the class with the minimum matching cost as the final classification result: in The face under test and c i The matching distance of human-like faces, D s(t) is the local matching distance obtained according to the matching path at time t.
