Half-shielded face recognition method, recognition equipment and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to a half-shielded face recognition method, recognition equipment and a storage medium. The half-shielded face recognition method comprises the following steps: firstly, acquiring a to-be-recognized half-shielded face image; and then inputting the to-be-recognized half-shielded face image into a trained face recognition model to obtain face feature information corresponding to the to-be-recognized half-shielded face image. Wherein the face recognition model comprises a feature extraction module, a mask estimation module and a feature suppression processing module. According to the half-shielding face recognition method, shielding face recognition is carried out by adopting a shielding region feature suppression method, a mask estimation module is arranged in a face recognition model, a pixel-level feature mask can be obtained by fusing deep features and shallow features, the accuracy of the mask features is improved, and the recognition accuracy of the half-shielding face is improved. And the recognition accuracy of the half-shielded face is higher.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a semi-occluded face recognition method, a recognition device and a storage medium. Background Art

[0002] Face recognition technology has been widely used in various fields. In some scenarios, it is impossible to obtain a complete face image. In this case, it is also important to recognize facial features of partially occluded or semi-occluded face images.

[0003] For example, in the field of transportation, occluded face recognition has important application value. This technology provides key support for the safety, intelligence and efficiency of the transportation system by identifying occluded faces. In terms of intelligent traffic management, occluded face recognition helps to identify pedestrians and drivers in areas such as intersections and crosswalks. This provides support for the implementation of traffic control strategies such as intelligent traffic light control and pedestrian priority crossing, and improves the efficiency and safety of road traffic. In addition, occluded face recognition can also be applied to security prevention. By analyzing the behavioral characteristics of the occluded area, the system can issue an alarm or take other preventive measures to improve the overall safety of the traffic scene.

[0004] In order to identify occluded face images, the related technologies mainly use the method of occluded area restoration, that is, to restore the occluded face part. With the development of deep neural networks, image restoration technology based on deep learning has gradually become mainstream. The recognition model of the generative adversarial network uses a generative adversarial network pre-trained on unoccluded images, so that the model can automatically detect occluded image areas and correctly restore related image pixels. However, the training of the generative adversarial network is usually more complicated, and it is prone to problems such as training instability and model collapse, resulting in low accuracy of the final recognition results of semi-occluded face images.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a semi-occluded face recognition method, recognition device and storage medium, aiming to solve the technical problem of low accuracy of recognition results for semi-occluded face images in the prior art.

[0007] On the one hand, the present application provides a semi-occluded face recognition method, comprising:

[0008] Obtain a semi-occluded face image to be identified;

[0009] Inputting the semi-occluded face image to be identified into a trained face recognition model to obtain face feature information corresponding to the semi-occluded face image to be identified;

[0010] Wherein, the face recognition model includes a feature extraction module, a mask estimation module and a feature suppression processing module;

[0011] The feature extraction module is used to extract feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features;

[0012] The mask estimation module is used to extract the deep pyramid features to obtain deep feature information; and to fuse the shallow pyramid features with the deep feature information to obtain pixel-level mask features;

[0013] The feature suppression processing module is used to perform feature fusion on the deep pyramid feature and the pixel-level mask feature to obtain the facial feature information.

[0014] In one embodiment, the feature extraction module includes a feature pyramid network, and the feature pyramid network includes a bottom-up model and a top-down model;

[0015] The step of extracting feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features includes:

[0016] The bottom-up model is used to gradually extract features of the semi-occluded face image to be identified from the shallow layer to the deep layer, and at the same time downsample layer by layer from the shallow layer to the deep layer to obtain feature maps of different sizes;

[0017] The Top-down model is used to upsample the feature map obtained in the previous layer, and pass the feature information obtained in the previous layer downward layer by layer to obtain the deep pyramid features and shallow pyramid features.

[0018] In one embodiment, the mask estimation module includes a plurality of convolution modules with different dilation rates and a global average pooling layer;

[0019] The extracting the deep pyramid features to obtain deep feature information includes:

[0020] The convolution modules with different expansion rates and the global average pooling layer respectively extract features from the deep pyramid features to obtain multiple different feature information;

[0021] The multiple different feature information are fused to obtain the deep feature information.

[0022] In one embodiment, the mask estimation module includes a 1×1 convolution module with a dilation rate of 1, a 3×3 convolution module with a dilation rate of 6, a 3×3 convolution module with a dilation rate of 12, a 3×3 convolution module with a dilation rate of 18, a global average pooling layer, a feature fusion module, a channel adjustment module, and an upsampling module.

[0023] In one embodiment, the semi-occluded face recognition method further includes:

[0024] The deep pyramid features are respectively input into a 1×1 convolution module with a dilation rate of 1, a 3×3 convolution module with a dilation rate of 6, a 3×3 convolution module with a dilation rate of 12, a 3×3 convolution with a dilation rate of 18, and a global average pooling layer to obtain five different feature information;

[0025] The feature fusion module performs feature fusion on the five different feature information to obtain first fused feature information;

[0026] The channel adjustment module adjusts the number of channels of the first fused feature information to obtain second fused feature information;

[0027] The upsampling module upsamples the second fused feature information to obtain the deep feature information.

[0028] In one embodiment, before inputting the semi-occluded face image to be recognized into a trained face recognition model, the method further includes:

[0029] Obtaining a semi-occluded face sample image and face feature information corresponding to the semi-occluded face sample image;

[0030] Inputting the semi-occluded face sample image into an initial face recognition model to obtain effective face feature information corresponding to the semi-occluded face sample image;

[0031] The initial face recognition model is trained according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain a trained face recognition model.

[0032] In one embodiment, the face recognition model further includes a loss calculation module;

[0033] The training of the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain the trained face recognition model includes:

[0034] Iteratively updating the model parameters of the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image;

[0035] After each update is completed, the loss calculation module calculates the loss value of the loss function of the updated face recognition model. If the loss value of the loss function meets the conditions, the iterative update is stopped and the updated face recognition model is used as the trained face recognition model; if the loss value of the loss function does not meet the conditions, the iterative training continues.

[0036] In one embodiment, the loss function is a joint loss function; the joint loss function includes a cosine loss function and a classification loss function;

[0037] Wherein, the joint loss function is expressed as:

[0038] L total =L cos +L seg

[0039] The cosine loss function is expressed as:

[0040]

[0041] The classification loss function is expressed as:

[0042]

[0043] In the above formula, L total is the joint loss function, L cos is the cosine loss function, L seg is the classification loss function, M and N are the number of samples, is the category y to which the i-th sample belongs i The corresponding cosine value, θ j is the cosine value of the i-th sample and all other categories j, s is a scaling parameter, and m is the cosine boundary angle parameter; p(x i ) is the probability distribution of the true label, q(x i ) is the predicted value of the model.

[0044] On the other hand, the present invention also provides a semi-occluded face recognition device, which includes: a memory, a processor, and a semi-occluded face recognition program stored in the memory and runnable on the processor, wherein the semi-occluded face recognition program is configured to implement the steps of the semi-occluded face recognition method as described in any one of the above items.

[0045] On the other hand, the present invention also provides a storage medium, on which a semi-occluded face recognition program is stored, and when the semi-occluded face recognition program is executed by a processor, the steps of the semi-occluded face recognition method as described in any one of the above items are implemented.

[0046] According to the semi-occluded face recognition method provided by the present invention, firstly, a semi-occluded face image to be identified is obtained; then, the semi-occluded face image to be identified is input into a trained face recognition model to obtain face feature information corresponding to the semi-occluded face image to be identified. Among them, the face recognition model provided by the present application includes a feature extraction module, a mask estimation module and a feature suppression processing module; the feature extraction module is used to extract feature information of the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features; the mask estimation module is used to extract features from the deep pyramid features to obtain deep feature information; the shallow pyramid features and the deep feature information are fused to obtain pixel-level mask features; the feature suppression processing module is used to fuse the deep pyramid features and the pixel-level mask features to obtain the face feature information. It can be seen that the semi-occluded face recognition method of the present application adopts the occluded area feature suppression method to perform occluded face recognition, and sets a mask estimation module in the face recognition model to make the occluded face recognition more accurate. By fusing deep features and shallow features, a pixel-level feature mask can be obtained, and the accuracy of the mask feature can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the structure of a semi-occluded face recognition device in a hardware operating environment according to an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of a process flow of a semi-occluded face recognition method provided by an embodiment of the present invention;

[0049] Figure 3 One of the schematic diagrams of the face recognition model structure provided by the embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the structure of a feature extraction module provided in an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of a deep feature information acquisition process provided by an embodiment of the present invention;

[0052] Figure 6 A flow chart of a method for acquiring deep feature information provided by an embodiment of the present invention;

[0053] Figure 7 The second schematic diagram of the face recognition model structure provided by the embodiment of the present invention;

[0054] Figure 8 A schematic flow chart of a method for training a face recognition model provided in an embodiment of the present invention.

[0055] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0057] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a semi-occluded face recognition device in the hardware operating environment involved in the embodiment of the present invention.

[0058] like Figure 1 As shown, the semi-occluded face recognition device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art will understand that Figure 1 The structure shown in does not constitute a limitation on the semi-occluded face recognition device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0060] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and a semi-occluded face recognition program.

[0061] exist Figure 1In the semi-obstructed face recognition device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the semi-obstructed face recognition device of the present invention can be set in the semi-obstructed face recognition device, and the semi-obstructed face recognition device calls the semi-obstructed face recognition program stored in the memory 1005 through the processor 1001, and executes the semi-obstructed face recognition method provided by the embodiment of the present invention.

[0062] In order to identify occluded face images, the related technologies mainly use the method of occluded area restoration, that is, to restore the occluded face part. With the development of deep neural networks, image restoration technology based on deep learning has gradually become mainstream. The recognition model of the generative adversarial network uses a generative adversarial network pre-trained on unoccluded images, so that the model can automatically detect occluded image areas and correctly restore related image pixels. However, the training of the generative adversarial network is usually more complicated, and it is prone to problems such as training instability and model collapse, resulting in low accuracy of the final recognition results of semi-occluded face images.

[0063] In order to overcome the deficiencies of the above-mentioned prior art, the present application provides a semi-occluded face recognition method, which is based on the inventive concept of occluded area removal, and can also be understood as the inventive concept of occluded area suppression. The inventive concept of occluded area removal is to ignore the features of the occluded part of the face and focus on the remaining unoccluded facial features for recognition. The semi-occluded face recognition method of the inventive concept can be implemented based on an end-to-end occluded face recognition model, and a model is constructed so that the model can learn accurate feature masks and effectively discard facial features in occluded areas.

[0064] Specifically, in order to accurately and efficiently obtain facial feature information in a semi-occluded face image, the present application inputs the semi-occluded face image to be identified into a trained face recognition model to obtain facial feature information corresponding to the semi-occluded face image to be identified. Among them, the face recognition model provided by the present application includes a feature extraction module, a mask estimation module and a feature suppression processing module; the feature extraction module is used to extract feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features; the mask estimation module is used to extract features from the deep pyramid features to obtain deep feature information; the shallow pyramid features and the deep feature information are fused to obtain pixel-level mask features; the feature suppression processing module is used to fuse the deep pyramid features and the pixel-level mask features to obtain the facial feature information. It can be seen that the semi-occluded face recognition method of the present application adopts the occluded area feature suppression method for occluded face recognition, and sets a mask estimation module in the face recognition model to make the occluded face recognition more accurate. By fusing deep features and shallow features, a pixel-level feature mask can be obtained, and the accuracy of the mask feature can be improved. Feature suppression processing is then performed, and the obtained pixel-level mask feature is multiplied with the deep pyramid feature to suppress the deep damaged feature, thereby further improving the accuracy of occluded face recognition.

[0065] The embodiment of the present invention provides a semi-occluded face recognition method, referring to Figure 2 , Figure 2 A schematic diagram of the flow of a semi-occluded face recognition method provided by an embodiment of the present invention.

[0066] In this embodiment, the semi-occluded face recognition method includes:

[0067] Step S201: Obtain a partially obstructed face image to be identified.

[0068] In one embodiment, a semi-occluded face image in a certain scene can be captured by an image capture device such as a camera, and then the captured semi-occluded face image is sent to a semi-occluded face recognition device. After the semi-occluded face recognition device obtains the semi-occluded face image to be recognized, it performs facial feature information recognition on the semi-occluded face image to be recognized.

[0069] Step S202: input the semi-occluded face image to be identified into a trained face recognition model to obtain face feature information corresponding to the semi-occluded face image to be identified.

[0070] Figure 3 One of the schematic diagrams of the face recognition model structure provided by the embodiment of the present invention, please refer to Figure 3As shown, the face recognition model includes a feature extraction module 301 , a mask estimation module 302 , and a feature suppression processing module 304 .

[0071] Among them, the feature extraction module 301 is used to extract feature information of the input semi-occluded face image to be identified, and obtain deep pyramid features and shallow pyramid features; the mask estimation module 302 is used to extract features of the deep pyramid features and obtain deep feature information, so that effective face features in the image can be better detected and extracted; the shallow pyramid features and the deep feature information are fused to obtain pixel-level mask features, which improves the accuracy of feature mask prediction; the feature suppression processing module 304 is used to fuse the deep pyramid features and the pixel-level mask features to obtain the face feature information.

[0072] When the face recognition model provided in this embodiment recognizes a semi-occluded face image, the mask estimation module 302 is provided, so that the recognized face feature information is more accurate.

[0073] In some examples, the feature extraction module 301 includes a feature pyramid network, which uses the feature pyramid network to perform multi-scale feature extraction on the input semi-occluded face image to be identified, and can obtain feature information of different scales.

[0074] Figure 4 For a schematic diagram of the structure of the feature extraction module provided in the embodiment of the present invention, please refer to Figure 4 As shown, the feature pyramid network includes a Bottom-up model and a Top-down model.

[0075] In one embodiment, please combine Figure 4 The feature extraction module extracts feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features, specifically including:

[0076] The bottom-up model is used to gradually extract features from the shallow layer to the deep layer of the semi-occluded face image to be identified, and at the same time downsample layer by layer from the shallow layer to the deep layer to obtain feature maps of different sizes, for example, Figure 4 The three different scales of feature information x1, x2, and x3 shown in the figure; the Top-down model is used to upsample the feature map obtained in the previous layer, and pass the feature information obtained in the previous layer downward layer by layer to obtain the deep pyramid features and shallow pyramid features, for example Figure 4 The three different scales of feature information x'3, x'2, and x1' are shown in . Among them, x'3 is the obtained deep pyramid feature, and x1' is the obtained shallow pyramid feature.

[0077] Among them, passing the feature information obtained from the next layer upward layer by layer and passing the feature information obtained from the previous layer downward layer by layer can be understood as the process of feature fusion. Before feature fusion, the features of different layers can be upsampled and downsampled to make the feature sizes of adjacent layers the same, and the number of channels can be adjusted through 1×1 convolution. Finally, the features after the fusion of the upper and lower layers are further smoothed using 3×3 convolution to remove interference noise.

[0078] Among them, Figure 4 In the process of step-by-step fusion of the deep pyramid feature x'3 into the shallow pyramid feature x1', the following formula can be used to fuse feature information:

[0079] x'3=Conv 3*3 (Conv 1*1 (x3))(1)

[0080] x'2=Conv 3*3 (Conv 1*1 (x2)+Upsample(Conv 1*1 (x3)))(2)

[0081] x1'=Conv 3*3 (Conv 1*1 (x1)+Upsample(Conv 1*1 (x2)))(3)

[0082] In the above formula, x1', x'2, and x'3 are three types of pyramid features from shallow to deep, Conv 1*1 is a 1×1 convolution operation, Conv 3*3 It is a 3×3 convolution operation, and Upsample is an upsampling operation.

[0083] Finally, the feature extraction module 301 outputs a deep pyramid feature x'3 containing global information and a shallow pyramid feature x1' containing global information and local information.

[0084] In some examples, the mask estimation module includes a plurality of convolution modules with different dilation rates and a global average pooling layer.

[0085] In some examples, extracting the deep pyramid features to obtain deep feature information specifically includes:

[0086] The convolution modules with different expansion rates and the global average pooling layer respectively extract features from the deep pyramid features to obtain a plurality of different feature information; and the plurality of different feature information are fused to obtain the deep feature information.

[0087] Figure 5 For a schematic diagram of the deep feature information acquisition process provided by an embodiment of the present invention, please refer to Figure 5 As shown, the mask estimation module of this embodiment includes a 1×1 convolution module 501 with a dilation rate of 1, a 3×3 convolution module 502 with a dilation rate of 6, a 3×3 convolution module 503 with a dilation rate of 12, a 3×3 convolution module 504 with a dilation rate of 18, a global average pooling layer 505, a feature fusion module 506, a channel adjustment module 507 and an upsampling module 508.

[0088] Figure 6 For a flowchart of the method for obtaining deep feature information provided by an embodiment of the present invention, please refer to Figure 6 As shown, in some examples, the semi-occluded face recognition method further includes:

[0089] Step S601, input the deep pyramid features into a 1×1 convolution module with a dilation rate of 1, a 3×3 convolution module with a dilation rate of 6, a 3×3 convolution module with a dilation rate of 12, a 3×3 convolution with a dilation rate of 18, and a global average pooling layer, respectively, to obtain five different feature information.

[0090] Step S602: The feature fusion module performs feature fusion on the five different feature information to obtain first fused feature information.

[0091] Step S603: The channel adjustment module adjusts the number of channels of the first fused feature information to obtain second fused feature information.

[0092] Exemplarily, a 1×1 convolution may be used to adjust the number of channels of the first fused feature information to obtain the second fused feature information x cov .

[0093] Step S604: The upsampling module upsamples the second fused feature information to obtain the deep feature information x d .

[0094] Exemplarily, an upsampling module may be used to fused the second feature information x cov Upsampling is performed to make the size of the second fusion feature information consistent with the size of the shallow pyramid feature x1', thereby obtaining the deep feature information x d .

[0095] Figure 7 For the second schematic diagram of the face recognition model structure provided by the embodiment of the present invention, please refer to Figure 7 As shown, in one embodiment, the mask estimation module 302 is specifically used to: firstly use 1×1 convolution to adjust the number of channels of the shallow pyramid feature x1' to obtain feature information xl , the adjusted feature information x l With deep feature information x d Fusion; then two depth-separable convolutions are performed on the fused feature information x cat After processing, the output value is finally mapped to between 0 and 1 through the sigmoid function to obtain the pixel-level mask feature x cls .

[0096] See also Figure 7 As shown, in one embodiment, the feature suppression processing module 304 is specifically used to combine the deep pyramid feature x'3 and the pixel-level mask feature x cls Multiply, pass the feature information obtained after multiplication through the fully connected layer, and finally get the face feature information x e .

[0097] It is understandable that the face recognition model needs to be acquired through training, and only after verification that the trained face recognition model meets the face recognition accuracy and speed can the trained face recognition model be used to recognize semi-occluded face images.

[0098] Figure 8 For a flow chart of the training method of the face recognition model provided by the embodiment of the present invention, please refer to Figure 8 As shown, the training method of the face recognition model includes:

[0099] Step S801: Obtain a partially occluded face sample image and facial feature information corresponding to the partially occluded face sample image.

[0100] Exemplarily, effective feature information of a face can be obtained through a model.

[0101] Step S802: input the semi-occluded face sample image into an initial face recognition model to obtain effective face feature information corresponding to the semi-occluded face sample image.

[0102] It should be noted that, in the present embodiment, for a semi-occluded face sample image, an image pair of the same face can be obtained, the image pair including an occluded sample image of the face and an unobstructed sample image. The above-mentioned facial feature information can be obtained based on the unobstructed sample image of the face, and the corresponding effective facial feature information can be obtained through the model for the unobstructed sample image.

[0103] Step S803: training the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain a trained face recognition model.

[0104] Please continue to see Figure 7As shown, in one embodiment, the face recognition model also includes a loss calculation module 305, which is used to calculate the loss function during the initial face recognition model training process to stop the training when the loss value of the loss function meets the conditions, thereby obtaining a trained face recognition model.

[0105] In one embodiment, training the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain the trained face recognition model specifically includes:

[0106] Iteratively updating the model parameters of the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image;

[0107] After each update is completed, the loss calculation module calculates the loss value of the loss function of the updated face recognition model. If the loss value of the loss function meets the conditions, the iterative update is stopped and the updated face recognition model is used as the trained face recognition model; if the loss value of the loss function does not meet the conditions, the iterative training continues.

[0108] In one embodiment, the loss function may be a joint loss function; the joint loss function includes a cosine loss function and a classification loss function.

[0109] Wherein, the joint loss function is expressed as:

[0110] L total =L cos +L seg

[0111] The cosine loss function is expressed as:

[0112]

[0113] The classification loss function is expressed as:

[0114]

[0115] In the above formula, L total is the joint loss function, L cos is the cosine loss function, L seg is the classification loss function, M and N are the number of samples, is the category y to which the i-th sample belongs i The corresponding cosine value, θ j is the cosine value of the i-th sample and all other categories j, S is a scaling parameter, which is used to control the distribution range of the cosine value; m is the cosine boundary angle parameter; p(x i) is the probability distribution of the true label, p(x i ) takes the value of 0 or 1, p(x i ) represents the category to which it belongs. If the sample belongs to the target category, then p(x i ) takes the value of 1, otherwise it takes the value of 0; q(x i ) is the predicted value of the model, q(x i ) is a probability value between 0 and 1, q(x i ) represents the probability that the sample belongs to the target category.

[0116] Please continue to refer to Figure 7 As shown in the figure, during the training of the initial face recognition model, in each iteration, the effective feature information of the recognized face and the upsampled pixel-level mask feature x cls After inputting the loss calculation module 305, the loss calculation module 305 calculates the loss function values ​​of the cosine loss and the mask estimation loss according to the two feature information input, thereby determining the direction of training to quickly complete the model training.

[0117] The face recognition model provided in this embodiment is provided with a loss calculation module 305, which adopts joint loss calculation, specifically adopts cosine loss and mask estimation loss to calculate the difference between the model's recognition result of the occluded face image and the true label, thereby improving the recognition accuracy of the facial feature information of the trained occluded face recognition model.

[0118] According to the semi-occluded face recognition method provided by the present application, an occluded area feature suppression method is adopted to perform occluded face recognition, and a mask estimation module 302 and a feature suppression processing module 304 are set in the face recognition model, which can better detect and extract effective face feature information, and suppress the face information of the occluded part by fusing the shallow pyramid features and the deep feature information, so that the face feature information finally output is more accurate.

[0119] The present application also provides a storage medium, on which a semi-occluded face recognition program is stored, and when the semi-occluded face recognition program is executed by a processor, the steps of the semi-occluded face recognition method described above are implemented. The semi-occluded face recognition method comprises: obtaining a semi-occluded face image to be identified; inputting the semi-occluded face image to be identified into a trained face recognition model to obtain face feature information corresponding to the semi-occluded face image to be identified; wherein the face recognition model comprises a feature extraction module, a mask estimation module and a feature suppression processing module; the feature extraction module is used to extract feature information of the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features; the mask estimation module is used to extract features from the deep pyramid features to obtain deep feature information; the shallow pyramid features and the deep feature information are fused to obtain pixel-level mask features; the feature suppression processing module is used to perform feature fusion on the deep pyramid features and the pixel-level mask features to obtain the face feature information.

[0120] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0121] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0123] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A semi-occluded face recognition method, It is characterized in that include: Obtain a semi-occluded face image to be identified; Inputting the semi-occluded face image to be identified into a trained face recognition model to obtain face feature information corresponding to the semi-occluded face image to be identified; Wherein, the face recognition model includes a feature extraction module, a mask estimation module and a feature suppression processing module; The feature extraction module is used to extract feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features; The mask estimation module is used to extract the deep pyramid features to obtain deep feature information; and fuse the shallow pyramid features with the deep feature information to obtain pixel-level mask features; The feature suppression processing module is used to perform feature fusion on the deep pyramid feature and the pixel-level mask feature to obtain the facial feature information.

2. The semi-occluded face recognition method according to claim 1, It is characterized in that The feature extraction module includes a feature pyramid network, and the feature pyramid network includes a bottom-up model and a top-down model; The step of extracting feature information from the input semi-occluded face image to be identified to obtain deep pyramid features and shallow pyramid features includes: The bottom-up model is used to gradually extract features of the semi-occluded face image to be identified from the shallow layer to the deep layer, and at the same time downsample layer by layer from the shallow layer to the deep layer to obtain feature maps of different sizes; The Top-down model is used to upsample the feature map obtained in the previous layer, and pass the feature information obtained in the previous layer downward layer by layer to obtain the deep pyramid features and shallow pyramid features.

3. The semi-occluded face recognition method according to claim 1, It is characterized in that The mask estimation module includes a plurality of convolution modules with different dilation rates and a global average pooling layer; The extracting the deep pyramid features to obtain deep feature information includes: The convolution modules with different expansion rates and the global average pooling layer respectively extract features from the deep pyramid features to obtain multiple different feature information; The multiple different feature information are fused to obtain the deep feature information.

4. The semi-occluded face recognition method according to claim 3, It is characterized in that The mask estimation module includes a 1×1 convolution module with a dilation rate of 1, a 3×3 convolution module with a dilation rate of 6, a 3×3 convolution module with a dilation rate of 12, a 3×3 convolution module with a dilation rate of 18, a global average pooling layer, a feature fusion module, a channel adjustment module and an upsampling module.

5. The semi-occluded face recognition method according to claim 4, It is characterized in that Also includes: The deep pyramid features are respectively input into a 1×1 convolution module with a dilation rate of 1, a 3×3 convolution module with a dilation rate of 6, a 3×3 convolution module with a dilation rate of 12, a 3×3 convolution with a dilation rate of 18, and a global average pooling layer to obtain five different feature information; The feature fusion module performs feature fusion on the five different feature information to obtain first fused feature information; The channel adjustment module adjusts the number of channels of the first fused feature information to obtain second fused feature information; The upsampling module upsamples the second fused feature information to obtain the deep feature information.

6. The semi-occluded face recognition method according to claim 1, It is characterized in that The step of inputting the half-occluded face image to be identified into the trained face recognition model further includes: Obtaining a semi-occluded face sample image and face feature information corresponding to the semi-occluded face sample image; Inputting the semi-occluded face sample image into an initial face recognition model to obtain effective face feature information corresponding to the semi-occluded face sample image; The initial face recognition model is trained according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain a trained face recognition model.

7. The semi-occluded face recognition method according to claim 5, It is characterized in that The face recognition model also includes a loss calculation module; The training of the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image to obtain the trained face recognition model includes: Iteratively updating the model parameters of the initial face recognition model according to the effective face feature information and face feature information corresponding to the semi-occluded face sample image; After each update is completed, the loss calculation module calculates the loss value of the loss function of the updated face recognition model. If the loss value of the loss function meets the conditions, the iterative update is stopped and the updated face recognition model is used as the trained face recognition model; if the loss value of the loss function does not meet the conditions, the iterative training continues.

8. The semi-occluded face recognition method according to claim 6, It is characterized in that The loss function is a joint loss function; the joint loss function includes a cosine loss function and a classification loss function; Wherein, the joint loss function is expressed as: L total =L cos +L seg The cosine loss function is expressed as: The classification loss function is expressed as: In the above formula, L total is the joint loss function, L cos is the cosine loss function, L seg is the classification loss function, M and N are the number of samples, θ yi is the category y to which the i-th sample belongs i The corresponding cosine value, θ j is the cosine value of the i-th sample and all other categories j, s is a scaling parameter, and m is the cosine boundary angle parameter; p(x i ) is the probability distribution of the true label, q(x i ) is the predicted value of the model.

9. A semi-occluded face recognition device, It is characterized in that The device comprises: a memory, a processor, and a semi-occluded face recognition program stored in the memory and executable on the processor, wherein the semi-occluded face recognition program is configured to implement the steps of the semi-occluded face recognition method as described in any one of claims 1 to 8.

10. A storage medium, It is characterized in that The storage medium stores a semi-occluded face recognition program, and when the semi-occluded face recognition program is executed by the processor, the steps of the semi-occluded face recognition method according to any one of claims 1 to 8 are implemented.

Images