Face recognition methods and devices with occlusion

Abstract

This disclosure relates to the field of face recognition technology, providing a method and apparatus for face recognition with occlusion. The method includes: constructing an occlusion perception network using a global average pooling layer, a fully connected layer, and an activation layer; constructing a right branch network using the occlusion perception network, a scale network, and a Transformer module; constructing a left branch network using an attention mechanism module and a scale network; constructing a peri-ocular attention network using the right and left branch networks; inserting the peri-ocular attention network after the second, third, and fourth stage networks of the residual network model, respectively, to obtain a face recognition model; training the face recognition model; and performing face recognition using the trained face recognition model. By employing the above techniques, the problem of low accuracy in recognizing occluded images using traditional face recognition models in the prior art is addressed.

Description

Technical Field

[0001] This disclosure relates to the field of facial recognition technology, and in particular to a facial recognition method and apparatus for faces with occlusion. Background Technology

[0002] When performing facial recognition, the images to be identified are often occluded. However, because facial recognition systems with occlusion do not adequately consider this situation, the accuracy of facial recognition models in identifying occluded images is low.

[0003] In realizing the concept disclosed herein, the inventors discovered at least the following technical problems in the related technologies: the traditional face recognition model has low accuracy in recognizing images with occlusion. Summary of the Invention

[0004] In view of this, the present disclosure provides a face recognition method, apparatus, electronic device, and computer-readable storage medium with occlusion, to solve the problem of low accuracy of traditional face recognition models in recognizing images with occlusion in the prior art.

[0005] A first aspect of this disclosure provides a face recognition method with occlusion, comprising: constructing an occlusion perception network using a global average pooling layer, a fully connected layer, and an activation layer; constructing a right branch network using the occlusion perception network, a scale network, and a Transformer module; constructing a left branch network using an attention mechanism module and a scale network; constructing a peri-ocular attention network using the right and left branch networks; inserting the peri-ocular attention network after the second, third, and fourth stage networks of the residual network model, respectively, to obtain a face recognition model; training the face recognition model; and performing face recognition using the trained face recognition model.

[0006] A second aspect of this disclosure provides a face recognition device with occlusion, comprising: a first construction module configured to construct an occlusion perception network using a global average pooling layer, a fully connected layer, and an activation layer; a second construction module configured to construct a right branch network using the occlusion perception network, a scale network, and a Transformer module, construct a left branch network using an attention mechanism module and a scale network, and construct a peri-ocular attention network using the constructed right and left branch networks; a third construction module configured to insert the peri-ocular attention network after the second-stage network, the third-stage network, and the fourth-stage network of the residual network model, respectively, to obtain a face recognition model; and a face recognition module configured to train the face recognition model and perform face recognition using the trained face recognition model.

[0007] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The beneficial effects of this disclosed embodiment compared to the prior art are as follows: This disclosed embodiment constructs an occlusion perception network using a global average pooling layer, a fully connected layer, and an activation layer; it constructs a right branch network using an occlusion perception network, a scale network, and a Transformer module; it constructs a left branch network using an attention mechanism module and a scale network; and it constructs a peri-ocular attention network using the right and left branch networks. The peri-ocular attention network is then inserted after the second, third, and fourth stage networks of the residual network model to obtain a face recognition model. The face recognition model is then trained, and the trained face recognition model is used for face recognition. Therefore, by employing the above technical means, the problem of low accuracy in recognizing occluded images by traditional face recognition models in the prior art can be solved, thereby improving the accuracy of the model in recognizing occluded images. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0012] Figure 2 This is a schematic flowchart of a face recognition method with occlusion provided in an embodiment of this disclosure;

[0013] Figure 3 This is a schematic diagram of the structure of a face recognition device with occlusion provided in an embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0016] A face recognition method and apparatus with occlusion according to an embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 101, 102, and 103, server 104, and network 105.

[0018] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays that support communication with server 104, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. Terminal devices 101, 102, and 103 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not impose any limitations on this. Furthermore, various applications can be installed on terminal devices 101, 102, and 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0019] Server 104 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 104 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This embodiment of the disclosure does not impose any limitations on these aspects.

[0020] It should be noted that server 104 can be either hardware or software. When server 104 is hardware, it can be various electronic devices that provide various services to terminal devices 101, 102, and 103. When server 104 is software, it can be multiple software programs or software modules that provide various services to terminal devices 101, 102, and 103, or it can be a single software program or software module that provides various services to terminal devices 101, 102, and 103. This disclosure does not limit the scope of the embodiments.

[0021] Network 105 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc. This disclosure does not limit the scope of the network.

[0022] Users can establish a communication connection with server 104 via network 105 through terminal devices 101, 102, and 103 to receive or send information, etc. It should be noted that the specific types, quantities, and combinations of terminal devices 101, 102, and 103, server 104, and network 105 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any limitations on this.

[0023] Figure 2 This is a schematic flowchart of a face recognition method with occlusion provided in an embodiment of this disclosure. Figure 2 Face recognition methods with occlusion can be derived from Figure 1 The computer or server, or the software on the computer or server, executes the command. For example... Figure 2 As shown, the face recognition method with occlusion includes:

[0024] S201 utilizes a global average pooling layer, a fully connected layer, and an activation layer to construct an occlusion perception network;

[0025] S202, a right branch network is constructed using an occlusion perception network, a scale network, and a Transformer module; a left branch network is constructed using an attention mechanism module and a scale network; and a periocular attention network is constructed using the right and left branch networks.

[0026] S203, an eye-peri-eye attention network is inserted after the second-stage network, the third-stage network and the fourth-stage network of the residual network model respectively to obtain a face recognition model;

[0027] S204, Train the face recognition model and use the trained face recognition model to perform face recognition.

[0028] Practice has shown that face recognition with occlusion does not adequately consider the situation where images are occluded. This is mainly due to the internal structure of traditional face recognition models. In order to better consider the recognition of occluded images, this disclosure constructs an occlusion perception network (the occlusion perception network is used to perform adaptive occlusion perception processing on the feature map of the image, that is, to adaptively perceive the area on the feature map of the image that is covered by a mask). Based on this occlusion perception network, an eye-peripheral attention network is constructed. Then, the eye-peripheral attention network is inserted after the second-stage network, the third-stage network, and the fourth-stage network of the residual network model (the residual network model is the traditional face recognition model). The improved residual network model is used as the face recognition model of this disclosure embodiment.

[0029] The scale network is a network of scale; the Transformer module, also known as the Transformer model, was proposed in the paper "Attention is All You Need" and is one of the commonly used neural network models; the attention mechanism module is the Squeeze-and-Excitation network.

[0030] Residual network models, such as ResNet50, include a zero-stage network (Stage 0), a first-stage network (Stage 1), a second-stage network (Stage 2), a third-stage network (Stage 3), and a fourth-stage network (Stage 4). This disclosure does not improve Stage 0 and Stage 1, but only Stage 2, Stage 3, and Stage 4.

[0031] Training a face recognition model is a common training method, which will not be elaborated further.

[0032] According to the technical solution provided in this disclosure, an occlusion perception network is constructed using a global average pooling layer, a fully connected layer, and an activation layer; a right branch network is constructed using the occlusion perception network, a scale network, and a Transformer module; a left branch network is constructed using an attention mechanism module and a scale network; and an eye-periorbital attention network is constructed using the right and left branch networks. The eye-periorbital attention network is then inserted after the second, third, and fourth stage networks of the residual network model to obtain a face recognition model. The face recognition model is trained, and face recognition is performed using the trained model. Therefore, by employing the above technical means, the problem of low accuracy in recognizing occluded images by traditional face recognition models in the prior art can be solved, thereby improving the accuracy of the model in recognizing occluded images.

[0033] An occlusion perception network is constructed using a global average pooling layer, a fully connected layer, and an activation layer, including: sequentially connecting a global average pooling layer, a first fully connected layer, a first activation layer, a second fully connected layer, and a second activation layer to obtain the occlusion perception network; wherein, the first fully connected layer is used for feature compression, the second fully connected layer is used for feature activation, the fully connected layer includes: a first fully connected layer and a second fully connected layer, and the activation layer includes: a first activation layer and a second activation layer.

[0034] The first activation layer can use ReLU, and the second activation layer can use Sigmoid.

[0035] It should be noted that the global average pooling layer used in this embodiment performs full pooling processing on the channel direction and horizontal direction of the feature map, without involving the height direction of the feature map. The horizontal direction refers to the width direction of the feature map. A feature map is composed of the channel direction, the horizontal direction, and the height direction.

[0036] A right branch network is constructed using an occlusion perception network, a scale network, and a Transformer module; a left branch network is constructed using an attention mechanism module and a scale network; and a periocular attention network is constructed using the right and left branch networks. This process includes: sequentially connecting the occlusion perception network, scale network, and Transformer module to obtain the right branch network; sequentially connecting the attention mechanism module and scale network to obtain the left branch network; and then connecting the right and left branch networks in parallel to obtain the periocular attention network. The output of the periocular attention network is obtained by adding the outputs of the right and left branch networks.

[0037] The residual network model consists of a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network connected sequentially.

[0038] The first feature map output from each stage of the face recognition model is input into the peri-ocular attention network inserted after that stage network. The process includes: using the occlusion perception network in the right branch network to perform adaptive occlusion perception processing on the first feature map to obtain the second feature map; using the scaling network in the right branch network to perform a dot product operation on the first and second feature maps to obtain the third feature map; using the Transformer module in the right branch network to perform attention transformation processing on the third feature map to obtain the fourth feature map; using the attention mechanism module in the left branch network to perform adaptive channel importance adjustment processing on the first feature map to obtain the fifth feature map; using the scaling network in the left branch network to perform a dot product operation on the first and fifth feature maps to obtain the sixth feature map; and adding the fourth and sixth feature maps to obtain the seventh feature map, which is the final output of the peri-ocular attention network.

[0039] The "each stage network" here actually refers to the second, third, and fourth stage networks. Only the second, third, and fourth stage networks are followed by the periocular attention network. The first and second stage networks are not followed by the periocular attention network.

[0040] The face recognition model internally processes feature maps. For ease of explanation, the feature map output by each stage network is referred to as the first feature map (because the processing of the peri-ocular attention network inserted after each stage network is the same). The feature map finally output by the peri-ocular attention network inserted after this stage network is referred to as the seventh feature map.

[0041] It should be noted that the above-mentioned periocular attention network processing can be viewed as either the process of processing the feature maps of training samples during model training, or as the process of using the model to process the feature maps of the image to be recognized after the model has been trained.

[0042] Adaptive occlusion perception processing is the adaptive perception of the regions on the feature map of the image that are occluded by the mask; attention transformation processing can be seen as using the Transformer module to enhance the information of the unoccluded areas and weaken the information of the occluded areas; adaptive channel importance adjustment processing is the adaptive adjustment of the importance of channel information on the feature map (such as increasing the weight of important channel information and decreasing the weight of unimportant channel information, or compressing unimportant channel information and stimulating the weight of important channel information).

[0043] The first feature map is adaptively occlusion-aware processed using the occlusion-aware network in the right branch network to obtain the second feature map. This process includes: pooling the first feature map using a global average pooling layer to obtain the eighth feature map; performing feature compression and activation processing on the eighth feature map sequentially using a first fully connected layer and a first activation layer to obtain the ninth feature map; and performing feature activation and activation processing on the ninth feature map sequentially using a second fully connected layer and a second activation layer to obtain the second feature map. The fully connected layers include a first fully connected layer and a second fully connected layer, and the activation layers include a first activation layer and a second activation layer.

[0044] Pooling, feature compression, feature activation, and activation processing are all commonly used techniques in neural network models and will not be elaborated further.

[0045] When the network at this stage is the second stage network, the seventh feature map is the input to the third stage network; when the network at this stage is the third stage network, the seventh feature map is the input to the fourth stage network; when the network at this stage is the fourth stage network, the seventh feature map is the feature map obtained by the face recognition model in the final processing.

[0046] The face recognition model outputs the recognition result based on the feature map obtained from the final processing.

[0047] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0048] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0049] Figure 3 This is a schematic diagram of a face recognition device with occlusion provided in an embodiment of this disclosure. Figure 3 As shown, the occluded facial recognition device includes:

[0050] The first building module 301 is configured to build an occlusion perception network using a global average pooling layer, a fully connected layer and an activation layer.

[0051] The second construction module 302 is configured to construct a right branch network using an occlusion perception network, a scale network, and a Transformer module; construct a left branch network using an attention mechanism module and a scale network; and construct a peri-ocular attention network using the construction of the right and left branch networks.

[0052] The third building module 303 is configured to insert a peri-ocular attention network after the second-stage network, the third-stage network and the fourth-stage network of the residual network model, respectively, to obtain a face recognition model;

[0053] The face recognition module 304 is configured to train a face recognition model and use the trained face recognition model to perform face recognition.

[0054] Practice has shown that face recognition with occlusion does not adequately consider the situation where images are occluded. This is mainly due to the internal structure of traditional face recognition models. In order to better consider the recognition of occluded images, this disclosure constructs an occlusion perception network (the occlusion perception network is used to perform adaptive occlusion perception processing on the feature map of the image, that is, to adaptively perceive the area on the feature map of the image that is covered by a mask). Based on this occlusion perception network, an eye-peripheral attention network is constructed. Then, the eye-peripheral attention network is inserted after the second-stage network, the third-stage network, and the fourth-stage network of the residual network model (the residual network model is the traditional face recognition model). The improved residual network model is used as the face recognition model of this disclosure embodiment.

[0055] The scale network is a network of scale; the Transformer module, also known as the Transformer model, was proposed in the paper "Attention is All You Need" and is one of the commonly used neural network models; the attention mechanism module is the Squeeze-and-Excitation network.

[0056] Residual network models, such as ResNet50, include a zero-stage network (Stage 0), a first-stage network (Stage 1), a second-stage network (Stage 2), a third-stage network (Stage 3), and a fourth-stage network (Stage 4). This disclosure does not improve Stage 0 and Stage 1, but only Stage 2, Stage 3, and Stage 4.

[0057] Training a face recognition model is a common training method, which will not be elaborated further.

[0058] According to the technical solution provided in this disclosure, an occlusion perception network is constructed using a global average pooling layer, a fully connected layer, and an activation layer; a right branch network is constructed using the occlusion perception network, a scale network, and a Transformer module; a left branch network is constructed using an attention mechanism module and a scale network; and an eye-periorbital attention network is constructed using the right and left branch networks. The eye-periorbital attention network is then inserted after the second, third, and fourth stage networks of the residual network model to obtain a face recognition model. The face recognition model is trained, and face recognition is performed using the trained model. Therefore, by employing the above technical means, the problem of low accuracy in recognizing occluded images by traditional face recognition models in the prior art can be solved, thereby improving the accuracy of the model in recognizing occluded images.

[0059] Optionally, the first building module 301 is further configured to sequentially connect a global average pooling layer, a first fully connected layer, a first activation layer, a second fully connected layer, and a second activation layer to obtain an occlusion perception network; wherein, the first fully connected layer is used for feature compression, the second fully connected layer is used for feature activation, the fully connected layer includes: a first fully connected layer and a second fully connected layer, and the activation layer includes: a first activation layer and a second activation layer.

[0060] The first activation layer can use ReLU, and the second activation layer can use Sigmoid.

[0061] It should be noted that the global average pooling layer used in this embodiment performs full pooling processing on the channel direction and horizontal direction of the feature map, without involving the height direction of the feature map. The horizontal direction refers to the width direction of the feature map. A feature map is composed of the channel direction, the horizontal direction, and the height direction.

[0062] Optionally, the second building module 302 is further configured to sequentially connect the occlusion perception network, the scale network, and the Transformer module to obtain a right branch network; sequentially connect the attention mechanism module and the scale network to obtain a left branch network; and connect the right branch network and the left branch network in parallel to obtain a peri-ocular attention network, wherein the output of the right branch network and the output of the left branch network are added together to obtain the output of the peri-ocular attention network.

[0063] The residual network model consists of a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network connected sequentially.

[0064] Optionally, the face recognition module 304 is further configured to perform adaptive occlusion perception processing on the first feature map using the occlusion perception network in the right branch network to obtain the second feature map; perform a dot product operation on the first and second feature maps using the scaling network in the right branch network to obtain the third feature map; perform attention transformation processing on the third feature map using the Transformer module in the right branch network to obtain the fourth feature map; perform adaptive channel importance adjustment processing on the first feature map using the attention mechanism module in the left branch network to obtain the fifth feature map; perform a dot product operation on the first and fifth feature maps using the scaling network in the left branch network to obtain the sixth feature map; and add the fourth and sixth feature maps to obtain the seventh feature map finally output by the peri-ocular attention network.

[0065] The "each stage network" here actually refers to the second, third, and fourth stage networks. Only the second, third, and fourth stage networks are followed by the periocular attention network. The first and second stage networks are not followed by the periocular attention network.

[0066] The face recognition model internally processes feature maps. For ease of explanation, the feature map output by each stage network is referred to as the first feature map (because the processing of the peri-ocular attention network inserted after each stage network is the same). The feature map finally output by the peri-ocular attention network inserted after this stage network is referred to as the seventh feature map.

[0067] It should be noted that the above-mentioned periocular attention network processing can be viewed as either the process of processing the feature maps of training samples during model training, or as the process of using the model to process the feature maps of the image to be recognized after the model has been trained.

[0068] Adaptive occlusion perception processing is the adaptive perception of the regions on the feature map of the image that are occluded by the mask; attention transformation processing can be seen as using the Transformer module to enhance the information of the unoccluded areas and weaken the information of the occluded areas; adaptive channel importance adjustment processing is the adaptive adjustment of the importance of channel information on the feature map (such as increasing the weight of important channel information and decreasing the weight of unimportant channel information, or compressing unimportant channel information and stimulating the weight of important channel information).

[0069] Optionally, the face recognition module 304 is further configured to perform pooling processing on the first feature map using a global average pooling layer to obtain an eighth feature map; to perform feature compression processing and activation processing on the eighth feature map sequentially using a first fully connected layer and a first activation layer to obtain a ninth feature map; and to perform feature activation processing and activation processing on the ninth feature map sequentially using a second fully connected layer and a second activation layer to obtain a second feature map; wherein the fully connected layer includes: a first fully connected layer and a second fully connected layer, and the activation layer includes: a first activation layer and a second activation layer.

[0070] Pooling, feature compression, feature activation, and activation processing are all commonly used techniques in neural network models and will not be elaborated further.

[0071] When the network at this stage is the second stage network, the seventh feature map is the input to the third stage network; when the network at this stage is the third stage network, the seventh feature map is the input to the fourth stage network; when the network at this stage is the fourth stage network, the seventh feature map is the feature map obtained by the face recognition model in the final processing.

[0072] The face recognition model outputs the recognition result based on the feature map obtained from the final processing.

[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0074] Figure 4 This is a schematic diagram of the electronic device 4 provided in an embodiment of this disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0075] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or different components.

[0076] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0077] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 4. The memory 402 can also include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0078] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0080] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A face recognition method with occlusion, characterized in that, include: An occlusion perception network is constructed using a global average pooling layer, a fully connected layer, and an activation layer. A right branch network is constructed using the occlusion perception network, scale network, and Transformer module; a left branch network is constructed using the attention mechanism module and the scale network; and a periocular attention network is constructed using the constructed right branch network and left branch network. The peri-ocular attention network is inserted after the second-stage network, third-stage network, and fourth-stage network of the residual network model to obtain the face recognition model. The face recognition model is trained, and the trained face recognition model is used for face recognition. The first feature map output by each stage network of the face recognition model is input into the peri-ocular attention network inserted after that stage network; The first feature map is adaptively occlusion-aware processing is performed on the occlusion-aware network in the right branch network to obtain the second feature map. The first feature map and the second feature map are multiplied by the scaling network in the right branch network to obtain the third feature map. The third feature map is processed by attention transformation using the Transformer module in the right branch network to obtain the fourth feature map. The first feature map is adaptively adjusted for channel importance using the attention mechanism module in the left branch network to obtain the fifth feature map; The first feature map and the fifth feature map are multiplied by the scaling network in the left branch network to obtain the sixth feature map; The fourth feature map and the sixth feature map are added together to obtain the seventh feature map, which is the final output of the periocular attention network. Wherein, when the network in this stage is the second stage network, the seventh feature map is the input of the third stage network; When the network at this stage is the third stage network, the seventh feature map is the input to the fourth stage network; When the network at this stage is the fourth stage network, the seventh feature map is the feature map finally obtained by the face recognition model.

2. The method according to claim 1, characterized in that, An occlusion-aware network is constructed using a global average pooling layer, a fully connected layer, and an activation layer, including: The occlusion perception network is obtained by sequentially connecting the global average pooling layer, the first fully connected layer, the first activation layer, the second fully connected layer, and the second activation layer. Wherein, the first fully connected layer is used for feature compression, the second fully connected layer is used for feature activation, the fully connected layer includes: the first fully connected layer and the second fully connected layer, and the activation layer includes: the first activation layer and the second activation layer.

3. The method according to claim 1, characterized in that, A right branch network is constructed using the occlusion perception network, scale network, and Transformer module; a left branch network is constructed using the attention mechanism module and the scale network; and a peri-ocular attention network is constructed using the right and left branch networks, including: The right branch network is obtained by sequentially connecting the occlusion perception network, the scale network, and the Transformer module. The attention mechanism module and the scale network are connected sequentially to obtain the left branch network; The right branch network and the left branch network are connected in parallel to obtain the periocular attention network, wherein the output of the right branch network and the output of the left branch network are added together to obtain the output of the periocular attention network.

4. The method according to claim 1, characterized in that, The residual network model is composed of a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network connected sequentially.

5. The method according to claim 1, characterized in that, The first feature map is adaptively occlusion-aware processed using the occlusion-aware network in the right branch network to obtain a second feature map, including: The first feature map is pooled using the global average pooling layer to obtain the eighth feature map. The eighth feature map is subjected to feature compression and activation processing sequentially using the first fully connected layer and the first activation layer to obtain the ninth feature map. The second feature map is obtained by sequentially performing feature activation processing and activation processing on the ninth feature map using the second fully connected layer and the second activation layer. The fully connected layer includes a first fully connected layer and a second fully connected layer, and the activation layer includes a first activation layer and a second activation layer.

6. A face recognition device with occlusion, characterized in that, include: The first building block is configured to construct an occlusion-aware network using a global average pooling layer, a fully connected layer, and an activation layer. The second construction module is configured to construct a right branch network using the occlusion perception network, scale network, and Transformer module; construct a left branch network using the attention mechanism module and the scale network; and construct a peri-ocular attention network using the constructed right branch network and left branch network. The third construction module is configured to insert the peri-ocular attention network after the second-stage network, the third-stage network, and the fourth-stage network of the residual network model, respectively, to obtain the face recognition model. The face recognition module is configured to train the face recognition model and use the trained face recognition model to perform face recognition. The face recognition module is also configured to input the first feature map output by each stage network of the face recognition model into the peri-ocular attention network inserted after that stage network; The first feature map is adaptively occlusion-aware processing is performed on the occlusion-aware network in the right branch network to obtain the second feature map. The first feature map and the second feature map are multiplied by the scaling network in the right branch network to obtain the third feature map. The third feature map is processed by the Transformer module in the right branch network to obtain the fourth feature map; the first feature map is processed by the attention mechanism module in the left branch network to obtain the fifth feature map. The first feature map and the fifth feature map are multiplied by the scaling network in the left branch network to obtain the sixth feature map; The fourth feature map and the sixth feature map are added together to obtain the seventh feature map, which is the final output of the peri-ocular attention network. Wherein, if the network at this stage is the second stage network, the seventh feature map is the input to the third stage network; if the network at this stage is the third stage network, the seventh feature map is the input to the fourth stage network; and if the network at this stage is the fourth stage network, the seventh feature map is the feature map finally obtained by the face recognition model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images