Mask face recognition method and system for coding image based on prior region supervision

Abstract

The application relates to a mask face recognition method and system for coding and decoding images based on prior region supervision, wherein the method comprises the following steps: acquiring an original face image and a missing face image, dividing the missing face image into a missing region subgraph and a non-missing region subgraph; performing data processing on the obtained subgraphs to respectively obtain missing region embedding vectors and non-missing region embedding vectors, obtaining a non-missing region coding vector through an encoder, decoding the non-missing region coding vector and the missing region embedding vector to obtain a decoding vector; training the encoder and the decoder by minimizing a loss function according to the decoding vector and the original face image, obtaining a trained encoder and a trained decoder; performing data processing on the missing face image through the trained encoder and the trained decoder to obtain a missing region recovery graph, performing Poisson fusion on the missing region recovery graph and a non-missing region face image to recover a face image, and finally performing face recognition according to the recovered face image to improve the mask face recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of face recognition technology, and in particular to a method and system for face recognition of masks based on prior region supervised encoding and decoding images. Background Technology

[0002] Wearing masks obscures most of the face, severely impacting the accuracy of facial recognition devices in public places, leading to problems such as false recognition, inability to pass through check-in areas, and allowing strangers to pass. Therefore, to improve the accuracy of masked facial recognition, a feasible method is to first recover the facial image hidden by the mask and then perform recognition on the recovered image.

[0003] However, traditional image restoration techniques typically rely on low-level features or global statistical features to recover lost content from known neighborhood information. Deep learning-based image inpainting often utilizes Generative Adversarial Networks (GANs), combining global and local discriminators to determine the semantic consistency of the restored image from both global and local regional perspectives. Similarly, in addition to using global and local discriminators, some models utilize facial geometry to assist in face completion. However, these methods suffer from a drawback: distortion occurs when generating the contours of missing regions. Furthermore, filling in missing parts of the image based on user-provided local contours, color maps, and other auxiliary information requires manual intervention, leading to excessively high labor costs.

[0004] Currently, no effective solution has been proposed to address the issue of low accuracy in facial recognition using masks in related technologies. Summary of the Invention

[0005] This application provides a method and system for masked face recognition based on prior region supervised encoding and decoding images, so as to at least solve the problem of low accuracy of masked face recognition in related technologies.

[0006] In a first aspect, embodiments of this application provide a masked face recognition method based on prior region supervised encoding and decoding images, the method comprising:

[0007] Obtain the original face image and the missing face image, and divide the missing face image to obtain a missing region sub-image and a non-missing region sub-image;

[0008] Data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain missing region embedding vector and non-missing region embedding vector. The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The non-missing region encoding vector and the missing region embedding vector are input into the decoder to obtain the decoding vector.

[0009] Based on the decoding vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to obtain a trained encoder and decoder.

[0010] The missing face image is processed by the trained encoder and decoder to obtain a missing region restoration map. The missing region restoration map is then fused with the original non-missing region face image using a Poisson method to obtain a restored face image. The restored face image is then input into a face recognition model for face recognition.

[0011] In some embodiments, training and optimizing the encoder and decoder by minimizing the loss function includes:

[0012] Based on the decoded vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to ensure that the recovered face image is consistent with the original face image. The formula for calculating the minimization loss function is as follows:

[0013] L=λ1L mask +λ2L nonmask +λ3L cos (F(D k (E θ (LP(x′)))), F(x))

[0014] L mask =(D k (LP(x′ mask ))-x mask ) 2

[0015] L nonmask =(D κ (E θ (LP(x′ nonmask )))-x nonmask ) 2

[0016] L cos (F(D κ (E θ (LP(x′)))), F(x))=1-cos(F(D κ (E θ (LP(x′)))), F(x))

[0017] Where λ1, λ2, and λ3 are adjustable parameters, λ1 > λ2, x′ mask For the missing region submap, x mask x′ represents the region corresponding to the missing sub-image in the original image. nonmask For non-missing region submaps, x nonmaskx′ represents the region corresponding to the non-missing sub-image on the original image, x′ represents the missing face image, and x represents the original face image; LP(·) is the linear projection and location coding layer; E θ (·) represents the encoder, and θ represents the encoder parameters; D κ (·) represents the decoder, and κ represents the decoder parameters; L cos (·) represents the identity feature loss, and F(·) is the face recognition model.

[0018] In some embodiments, obtaining the missing face image includes:

[0019] The original face image is detected and aligned by a face detector. A mask is then mapped onto the original face image using a spatial transformation network to obtain a masked face image. The masked area of ​​the masked face image is then segmented to obtain a face image with the masked area missing.

[0020] In some embodiments, data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain missing region embedding vectors and non-missing region embedding vectors, including:

[0021] Linear projection and position encoding are performed on the missing region subgraph and the non-missing region subgraph respectively to obtain the missing region embedding vector and the non-missing region embedding vector.

[0022] In some embodiments, the missing face image is processed using the trained encoder and decoder to obtain a missing region reconstruction map, including:

[0023] Acquire facial images of individuals whose masked areas are missing;

[0024] The face image with missing mask area is divided into missing area sub-image and non-missing area sub-image, and the sub-images are processed by linear projection and position encoding to obtain the embedding vector;

[0025] The embedding vector is sequentially input into the trained encoder and decoder to obtain the decoding vector of the missing region subgraph, and the decoding vector of the missing region subgraph is reshaped into the missing region recovery map.

[0026] Secondly, embodiments of this application provide a mask-wearing face recognition system based on prior region supervised encoding and decoding images, the system comprising:

[0027] The image processing module is used to acquire the original face image and the missing face image, and to divide the missing face image into missing region sub-images and non-missing region sub-images.

[0028] Data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain missing region embedding vector and non-missing region embedding vector. The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The non-missing region encoding vector and the missing region embedding vector are input into the decoder to obtain the decoding vector.

[0029] The training optimization module is used to train and optimize the encoder and decoder by minimizing the loss function based on the decoding vector and the original face image, so as to obtain the trained encoder and decoder.

[0030] The face recognition module is used to process the missing face image through the trained encoder and decoder to obtain a missing area restoration map, perform Poisson fusion of the missing area restoration map with the original non-missing area face image to obtain a restored face image, and input the restored face image into the face recognition model for face recognition.

[0031] In some embodiments, the training optimization module is further configured to train and optimize the encoder and decoder by minimizing a loss function based on the decoding vector and the original face image, so that the recovered face image is consistent with the original face image, wherein the formula for calculating the minimization loss function is as follows:

[0032] L=λ1L mask +λ2L nonmask +λ3L cos (F(D κ (E θ (LP(x′)))), F(x))

[0033] L mask =(D κ (LP(x′ mask ))-x mask ) 2

[0034] L nonmask =(D κ (E θ (LP(x′ nonmask )))-x nonmask ) 2

[0035] L cos (F(D κ (E θ (LP(x′)))), F(x))=1-cos(F(D κ (E θ (LP(x′)))), F(x))

[0036] Where λ1, λ2, and λ3 are adjustable parameters, λ1 > λ2, x′ mask For the missing region submap, x mask x′ represents the region corresponding to the missing sub-image in the original image. nonmask For non-missing region submaps, x nonmask x′ represents the region corresponding to the non-missing sub-image on the original image, x′ represents the missing face image, and x represents the original face image; LP(·) is the linear projection and location coding layer; E θ (·) represents the encoder, and θ represents the encoder parameters; D κ (·) represents the decoder, and κ represents the decoder parameters; L cos (·) represents the identity feature loss, and F(·) is the face recognition model.

[0037] In some embodiments, the image processing module is further configured to detect and align the original face image using a face detector, map a mask onto the original face image using a spatial transformation network to obtain a masked face image, and segment the mask region of the masked face image to obtain a face image with missing mask region.

[0038] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the masked face recognition method based on prior region supervised encoding and decoding images as described in the first aspect above.

[0039] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the mask-wearing face recognition method based on prior region supervised encoding and decoding images as described in the first aspect above.

[0040] Compared to related technologies, the masked face recognition method based on prior region supervised encoding and decoding images provided in this application obtains the original face image and the missing face image. The missing face image is divided into missing region sub-images and non-missing region sub-images. Data processing is performed on the missing region sub-images and non-missing region sub-images respectively to obtain missing region embedding vectors and non-missing region embedding vectors. The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The non-missing region encoding vector and the missing region embedding vector are input into the decoder to obtain the decoding vector. Based on the decoding vector and the original face image, the method minimizes the loss function... The encoder and decoder are trained and optimized to obtain trained encoder and decoder. The missing face image is processed by the trained encoder and decoder to obtain a missing area restoration map. The missing area restoration map is fused with the original non-missing face image by Poisson to obtain the restored face image. The restored face image is input into the face recognition model for face recognition. This solves the problem of low accuracy of face recognition with masks in related technologies. This application can make the restored face image closer to the real image and can ensure that the identity of the person in the restored image is consistent with the original image, making face recognition with masks more accurate. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a flowchart of a method for identifying network user roles according to an embodiment of this application;

[0043] Figure 2 This is a flowchart illustrating the training phase of the mask-wearing face recognition method according to an embodiment of this application.

[0044] Figure 3 This is a flowchart illustrating the testing phase of the mask-wearing face recognition method according to an embodiment of this application.

[0045] Figure 4 This is a structural block diagram of a network user role recognition system according to an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0048] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] It should be noted that the embodiments of this application can be applied not only to face recognition scenarios with masks on, but also to face recognition scenarios where faces are obscured.

[0051] This embodiment provides a mask-wearing face recognition method based on prior region supervised encoding and decoding images. Figure 1 This is a flowchart of a mask-wearing face recognition method based on prior region supervised encoding and decoding images according to an embodiment of this application, as follows: Figure 1 As shown, the process includes the following steps:

[0052] Step S101: Obtain the original face image and the missing face image, divide the missing face image into missing region sub-images and non-missing region sub-images;

[0053] Figure 2 This is a flowchart illustrating the training phase of the mask-wearing face recognition method according to an embodiment of this application, as shown below. Figure 2 As shown. Preferably, in this embodiment, the original face image x is detected and aligned by a face detector, a mask shape is initialized, and the mask is mapped to a reasonable position on the original face image through a spatial transformation network (STNs) to obtain a masked face image. The mask region of the masked face image is segmented so that the region is missing, thus obtaining a face image x′ with the mask region missing.

[0054] Next, the face image x′ with missing mask area is divided into n regular and non-overlapping sub-images. The sub-image containing the missing area is called the missing area sub-image, and the rest are called the non-missing area sub-images.

[0055] It should be noted that by mapping the mask onto a reasonable position in the original face image using Spatial Transform Networks (STNs), the resulting mask-covered facial missing area is the prior region.

[0056] Step S102: Perform data processing on the missing region subgraph and the non-missing region subgraph respectively to obtain the missing region embedding vector and the non-missing region embedding vector. Input the non-missing region embedding vector into the encoder to obtain the non-missing region encoding vector. Input the non-missing region encoding vector and the missing region embedding vector into the decoder to obtain the decoding vector.

[0057] like Figure 2 As shown, in this embodiment, linear projection and positional encoding are performed on the missing region subgraph and the non-missing region subgraph respectively to obtain the missing region embedding vector and the non-missing region embedding vector;

[0058] The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The obtained non-missing region encoding vector and missing region embedding vector are sorted according to position encoding and input into the decoder to obtain the decoding vector. The last layer of the decoder is a linear projection layer, and the number of its output channels is equal to the number of pixel values ​​of the image. The vector predicted by the decoder is the decoding vector in this embodiment.

[0059] It should be noted that this embodiment does not specifically limit the encoder and decoder.

[0060] Step S103: Based on the decoding vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to obtain the trained encoder and decoder.

[0061] like Figure 2 As shown, based on the decoded vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to obtain the trained encoder and decoder. Specifically, this includes: calculating the loss function based on the obtained decoded vector and the original face image, thereby training and optimizing the encoder and decoder so that the recovered face image D... κ (E θ The loss function (LP((x′)))) resembles the original face image x, while simultaneously constraining the restored face to represent the same person as the original face, further ensuring consistency between the restored face image and the original image. The formula for minimizing the loss function is as follows:

[0062] L=λ1L mask +λ2L nonmask +λ3L cos (F(D κ (E θ (LP(x′)))), F(x)) (1)

[0063] L mask =(D κ (LP(x′ mask ))-x mask ) 2 (2)

[0064] L nonmask =(D κ (E θ (LP(x′ nonmask )))-x nonmask ) 2 (3)

[0065] L cos (F(D κ (E θ(LP(x′)))), F(x))=1-cos(F(D κ (E θ (LP(x′)))), F(x)) (4)

[0066] Where λ1, λ2, and λ3 are adjustable parameters that can be freely adjusted according to the actual situation, x′ mask For the missing region submap, x mask x represents the region corresponding to the missing sub-image in the original image. nonmask For non-missing region submaps, x nonmask Let x' be the region corresponding to the non-missing sub-image on the original image, x be the missing face image, and L be the original face image. mask For the missing region loss, L nonmask For non-missing regions, L cos (·) represents the identity feature loss, F(·) is the face recognition model; LP(·) is the linear projection and position encoding layer. By performing linear projection and position encoding on the missing and non-missing sub-images through LP(·), the missing region embedding vector and the non-missing region embedding vector can be obtained; E θ (·) represents the encoder, which encodes the non-missing region embedding vector to obtain the non-missing region encoded vector; θ represents the encoder parameters; D κ (·) represents the decoder, κ represents the decoder parameters, and D κ (·) is used to decode the non-missing region encoding vector and the missing region embedding vector according to their positional encoding order. Through the above processing, the recovered complete face image D can be obtained. κ (E θ (LP((x′)))), recovers the image D of the missing region in the image. κ (LP(x′ mask ), to restore the non-missing regions of the image D κ (E θ (LP(x′ nonmask Finally, through L mask L nonmask L cos (·) Calculate the loss between the restored image and the original image respectively, and through repeated training, minimize the values ​​of the three loss functions to optimize the parameters in the encoder and decoder.

[0067] It should be noted that L mask and L nonmask L2 loss is used to calculate the pixel difference between corresponding regions in the restored image and the original image. Where L... mask L measures the similarity between the restored missing region map and the corresponding missing region submap in the original image. nonmaskThe similarity between the restored non-missing region image and the corresponding region in the original image is measured. Since the missing region subimage loses a lot of facial information, the restored image almost determines the correctness of the face identity. Therefore, it is necessary to increase the penalty for the face image in the missing region, such that λ1 > λ2.

[0068] Furthermore, to ensure that the restored face and the original face belong to the same person, an identity feature loss is also performed in the minimization loss function. Specifically, L cos (·) represents the identity feature loss, and F(·) is the face recognition model. Inputting the original face image into the face recognition model F(·) yields F(x), which is the 512-dimensional identity feature extracted by the face recognition model from the original face image x. Inputting the recovered face image into the face recognition model F(·) yields F(D). κ (E θ (LP((x′))))), which is the value extracted by the face recognition model from the recovered face image. 512 Dimensional identity features. L-value loss is obtained through identity features. cos (·) can minimize the loss of identity features between the restored face and the original face, so that the restored face image has the same identity as the original face image, further ensuring that the restored face image is consistent with the original image.

[0069] After training and optimizing the encoder and decoder through the above process, a well-trained encoder and decoder can be obtained. Using this well-trained encoder and decoder to acquire the recovered face image can improve the accuracy of face recognition for people wearing masks. It should be noted that this embodiment does not specifically limit the face recognition model.

[0070] It should be noted that L2 in this embodiment is a type of loss function, representing least squares error, which is common knowledge. In this embodiment, L... mask and L nonmask Both belong to L2 loss.

[0071] Step S104: The missing face image is processed by the trained encoder and decoder to obtain the missing area restoration map. The missing area restoration map is then fused with the original non-missing face image by Poisson to obtain the restored face image. The restored face image is then input into the face recognition model for face recognition.

[0072] Figure 3 This is a flowchart illustrating the testing phase of the mask-wearing face recognition method according to an embodiment of this application, as shown below. Figure 3As shown. Obtain a face image with missing mask regions; divide this face image into missing region sub-images and non-missing region sub-images, and process the divided sub-images using linear projection and positional encoding to obtain embedding vectors;

[0073] The embedding vector is sequentially input into the trained encoder and decoder. The decoding vector of the missing region subgraph is extracted from the decoder's output, and the decoding vector of the missing region subgraph is reshaped into the missing region recovery map, i.e., the mask part subgraph.

[0074] The reconstructed image of the missing region is stitched together with the original face image of the non-missing region. To achieve a smoother stitching boundary, Poisson fusion is used in this embodiment. While maintaining the fusion boundary pixels of the missing region image, the gradient field of that portion of the non-missing region sub-image is used as a guiding gradient to generate pixels within the fusion region, ensuring that the gradient field of the generated pixels within the fusion region is consistent with the gradient field of the pixels in the fusion portion of the non-missing sub-image. Finally, the reconstructed face image is obtained through stitching and fusion.

[0075] Finally, the recovered face image is input into the face recognition model for face recognition.

[0076] It should be noted that this embodiment does not specifically limit the face recognition model.

[0077] Through the above steps S101 to S104, this embodiment not only makes the restored face image closer to the real image by minimizing the loss function, but also ensures that the identity of the person in the restored image is consistent with that in the original image, making masked face recognition more accurate; in addition, this embodiment also uses Poisson fusion to ensure that the restored face image is more realistic and clear.

[0078] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0079] This embodiment also provides a mask-wearing face recognition system based on prior region supervised encoding and decoding images. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0080] Figure 4This is a structural block diagram of a mask-wearing face recognition system based on prior region supervised encoding and decoding images, according to an embodiment of this application. Figure 4 As shown, the system includes an image processing module 41, a training and optimization module 42, and a face recognition module 43.

[0081] Image processing module 41 is used to acquire the original face image and the missing face image, divide the missing face image into missing region sub-images and non-missing region sub-images, process the missing region sub-images and non-missing region sub-images respectively to obtain missing region embedding vectors and non-missing region embedding vectors, input the non-missing region embedding vector into the encoder to obtain the non-missing region encoding vector, and input the non-missing region encoding vector and the missing region embedding vector into the decoder to obtain the decoding vector; training and optimization module 42 is used to train and optimize the encoder and decoder by minimizing the loss function based on the decoding vector and the original face image to obtain the trained encoder and decoder; face recognition module 43 is used to process the missing face image using the trained encoder and decoder to obtain the missing region restoration map, perform Poisson fusion of the missing region restoration map and the original non-missing region face image to obtain the restored face image, and input the restored face image into the face recognition model for face recognition.

[0082] Through the above system, this embodiment not only makes the restored face image closer to the real image by minimizing the loss function, but also ensures that the identity of the person in the restored image is consistent with that in the original image, making masked face recognition more accurate; in addition, this embodiment also uses Poisson fusion to ensure that the restored face image is more realistic and clear.

[0083] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0084] Furthermore, it should be noted that the aforementioned modules can be either functional modules or program modules, and can be implemented through software or hardware. For modules implemented in hardware, these modules can reside in the same processor; alternatively, they can be located in different processors in any combination.

[0085] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0086] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0087] Furthermore, in conjunction with the masked face recognition method based on prior region supervised encoding and decoding images in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the masked face recognition methods based on prior region supervised encoding and decoding images in the above embodiments.

[0088] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a mask-wearing face recognition method based on prior region supervised encoding and decoding images. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0089] In one embodiment, Figure 5 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 5 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 5 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operation of the operating system and computer programs, and the computer programs, when executed by the processor, implement a mask-wearing face recognition method based on prior region supervised encoding and decoding images. The database stores data.

[0090] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for mask face recognition based on prior region supervised coding of an image, characterized in that, The method includes: Obtain the original face image and the missing face image; divide the missing face image into missing region sub-images and non-missing region sub-images; wherein, obtaining the missing face image includes: The original face image is detected and aligned by a face detector. A mask is then mapped onto the original face image using a spatial transformation network to obtain a masked face image. The masked area of ​​the masked face image is then segmented to obtain a face image with missing mask areas. Data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain missing region embedding vector and non-missing region embedding vector. The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The non-missing region encoding vector and the missing region embedding vector are input into the decoder to obtain the decoding vector. Based on the decoding vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to obtain a trained encoder and decoder. The missing face image is processed by the trained encoder and decoder to obtain a missing region restoration map. The missing region restoration map is then fused with the original non-missing region face image using a Poisson method to obtain a restored face image. The restored face image is then input into a face recognition model for face recognition.

2. The method according to claim 1, characterized in that, Training and optimizing the encoder and decoder by minimizing the loss function includes: Based on the decoded vector and the original face image, the encoder and decoder are trained and optimized by minimizing the loss function to ensure that the recovered face image is consistent with the original face image. The formula for calculating the minimization loss function is as follows: in, , and It is an adjustable parameter. , For the missing sub-map, This represents the region corresponding to the missing sub-image in the original image. For non-missing region submaps, This refers to the region corresponding to the non-missing sub-image on the original image. For the missing face image, Original human face image; For linear projection and position coding layer; For encoder, These are the parameters of the encoder; For decoder, These are the parameters for the decoder; For loss of identity features, It is a facial recognition model.

3. The method according to claim 1, characterized in that, Data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain the missing region embedding vector and the non-missing region embedding vector, including: Linear projection and position encoding are performed on the missing region subgraph and the non-missing region subgraph respectively to obtain the missing region embedding vector and the non-missing region embedding vector.

4. The method according to claim 1, characterized in that, The missing face image is processed using the trained encoder and decoder to obtain a missing region reconstruction map, including: Acquire facial images of individuals whose masked areas are missing; The face image with missing mask area is divided into missing area sub-image and non-missing area sub-image, and the sub-images are processed by linear projection and position encoding to obtain the embedding vector; The embedding vector is sequentially input into the trained encoder and decoder to obtain the decoding vector of the missing region subgraph, and the decoding vector of the missing region subgraph is reshaped into the missing region recovery map.

5. A mask-wearing face recognition system based on prior region supervised encoding and decoding images, characterized in that, The system includes: The image processing module is used to acquire the original face image and the missing face image, and to divide the missing face image into missing region sub-images and non-missing region sub-images. Acquiring the missing face image includes: The original face image is detected and aligned by a face detector. A mask is then mapped onto the original face image using a spatial transformation network to obtain a masked face image. The masked area of ​​the masked face image is then segmented to obtain a face image with missing mask areas. Data processing is performed on the missing region subgraph and the non-missing region subgraph respectively to obtain missing region embedding vector and non-missing region embedding vector. The non-missing region embedding vector is input into the encoder to obtain the non-missing region encoding vector. The non-missing region encoding vector and the missing region embedding vector are input into the decoder to obtain the decoding vector. The training optimization module is used to train and optimize the encoder and decoder by minimizing the loss function based on the decoding vector and the original face image, so as to obtain the trained encoder and decoder. The face recognition module is used to process the missing face image through the trained encoder and decoder to obtain a missing area restoration map, perform Poisson fusion of the missing area restoration map with the original non-missing area face image to obtain a restored face image, and input the restored face image into the face recognition model for face recognition.

6. The system according to claim 5, characterized in that, The training optimization module is further configured to train and optimize the encoder and decoder by minimizing a loss function based on the decoding vector and the original face image, so that the recovered face image is consistent with the original face image. The formula for calculating the minimization loss function is as follows: in, , and It is an adjustable parameter. , For the missing sub-map, This represents the region corresponding to the missing sub-image in the original image. For non-missing region submaps, This refers to the region corresponding to the non-missing sub-image on the original image. For the missing face image, Original human face image; For linear projection and position coding layer; For encoder, These are the parameters of the encoder; For decoder, These are the parameters for the decoder; For loss of identity features, It is a facial recognition model.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the masked face recognition method based on prior region supervised encoding and decoding images as described in any one of claims 1 to 4.

8. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the masked face recognition method based on prior region supervised encoding and decoding images according to any one of claims 1 to 4.

Images