Face semantic recognition method and device, equipment and storage medium

Abstract

The invention provides a face semantic recognition method and device, equipment and a storage medium, and belongs to the field of artificial intelligence, and the method comprises the steps: obtaining a shielded face image, and obtaining a face semantic recognition model; performing image reconstruction on a sheltered area in the sheltered face image through an image reconstruction layer to obtain a target face image; performing face semantic recognition on the target face image through a first face semantic recognition layer to obtain first face semantic information; performing face semantic recognition on the target face image through a second face semantic recognition layer to obtain second face semantic information; and determining target face semantic information of the shielded face image according to the first face semantic information and the second face semantic information. According to the method, the semantic information of the user face can be accurately extracted, and the success rate of face payment verification when the user purchases a financial product in a financial institution is improved. The invention further relates to a block chain technology, and the face semantic recognition model can be stored in the block chain.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for face semantic recognition. Background Art

[0002] With the continuous development of 5G technology, the Internet and the Internet of Things, face recognition has broader application scenarios, for example, tracking fugitive suspects, finding lost elderly people or children, passing access control and paying by face swiping, etc.; in the case of finding lost children, the camera collects images of children in motion. Since children are in motion, the collected images may contain a lot of noise, resulting in large errors in the extracted facial semantic information. For example, in hospitals, doctors need to pass facial recognition to enter some high-security areas (for example, drug warehouses, operating rooms, and conference rooms), but doctors generally wear masks. Therefore, they need to take off the mask every time they pass the face recognition to extract the doctor's facial semantic information, which poses a risk of virus infection.

[0003] For example, when users purchase financial products at financial institutions and verify their payment, they usually use face recognition to prevent the leakage of payment passwords. However, the user may wear a mask or sunglasses, which may result in the inability to accurately extract the semantic information of the user's face, thereby causing facial identity verification to fail, resulting in a poor experience for users when purchasing financial products.

[0004] Therefore, how to accurately generate semantic information of occluded face images is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a facial semantic recognition method, device, equipment and storage medium, aiming to accurately generate semantic information of occluded facial images.

[0006] In a first aspect, the present application provides a method for face semantic recognition, the method comprising the following steps:

[0007] Acquire an occluded face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer, and a second face semantic recognition layer;

[0008] Reconstructing the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image;

[0009] Performing face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information;

[0010] Performing face semantic recognition on the target face image through the second face semantic recognition layer to obtain second face semantic information;

[0011] The target face semantic information of the occluded face image is determined according to the first face semantic information and the second face semantic information.

[0012] In a second aspect, the present application further provides a face semantic recognition device, the face semantic recognition device comprising an acquisition module, an image reconstruction module, a semantic recognition module and a determination module, wherein:

[0013] The acquisition module is used to acquire the blocked face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer;

[0014] The image reconstruction module is used to reconstruct the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image;

[0015] The semantic recognition module is used to perform facial semantic recognition on the target facial image through the first facial semantic recognition layer to obtain first facial semantic information;

[0016] The semantic recognition module is further used to perform facial semantic recognition on the target facial image through the second facial semantic recognition layer to obtain second facial semantic information;

[0017] The determination module is used to determine the target face semantic information of the occluded face image according to the first face semantic information and the second face semantic information.

[0018] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned face semantic recognition method are implemented.

[0019] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the above-mentioned face semantic recognition method are implemented.

[0020] The present application provides a method, device, equipment and storage medium for face semantic recognition. The present application obtains an occluded face image and obtains a face semantic recognition model, which includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer; the image reconstruction layer performs image reconstruction on the occluded area in the occluded face image to obtain a target face image; the first face semantic recognition layer performs face semantic recognition on the target face image to obtain first face semantic information; the second face semantic recognition layer performs face semantic recognition on the target face image to obtain second face semantic information; according to the first face semantic information and the second face semantic information, the target face semantic information of the occluded face image is determined. The present solution can accurately extract the semantic information of the user's face, improve the success rate of face payment verification when the user purchases financial products at a financial institution, greatly improve the efficiency of financial business processing and improve the customer's satisfaction with financial business. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of a face semantic recognition method provided in an embodiment of the present application;

[0023] Figure 2 for Figure 1 A schematic diagram of the sub-step flow chart of the face semantic recognition method in FIG.

[0024] Figure 3 for Figure 1 A schematic diagram of the sub-step flow chart of the face semantic recognition method in FIG.

[0025] Figure 4 The structural intention of the face semantic recognition model provided for implementing this embodiment;

[0026] Figure 5 A schematic block diagram of a facial semantic recognition device provided in an embodiment of the present application;

[0027] Figure 6 for Figure 5 A schematic block diagram of submodules of a face semantic recognition device in;

[0028] Figure 7 for Figure 5 A schematic block diagram of submodules of a face semantic recognition device in;

[0029] Figure 8 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0030] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0033] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0034] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0035] The embodiment of the present application provides a method, apparatus, device and storage medium for face semantic recognition. The face semantic recognition method can be applied to a computer device, which can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device. For example, the computer device is a mobile phone, which obtains an occluded face image and obtains a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer; the occluded area in the occluded face image is reconstructed by the image reconstruction layer to obtain a target face image; the target face image is subjected to face semantic recognition by the first face semantic recognition layer to obtain first face semantic information; the target face image is subjected to face semantic recognition by the second face semantic recognition layer to obtain second face semantic information; the target face semantic information of the occluded face image is determined according to the first face semantic information and the second face semantic information.

[0036] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] Please refer to Figure 1 , Figure 1 A flowchart of a facial semantic recognition method provided in an embodiment of the present application.

[0038] like Figure 1 As shown, the face semantic recognition method includes steps S101 to S105.

[0039] Step S101, obtaining an occluded face image and obtaining a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer.

[0040] Among them, the occluded face image is a face image partially blocked by other objects, resulting in the failure to collect a complete face image. For example, the collected user wears a mask or sunglasses, and the collected image includes the mask or sunglasses. For another example, when the user uses a mobile phone to sample a face image, the fingers holding the mobile phone partially block the camera, resulting in partial occlusion of the collected face image, and an occluded face image is obtained. It should be noted that the occluded face image in the embodiment of the present invention also includes an image in which the face image is partially blurred.

[0041] In one embodiment, an occluded face image is obtained, where the occluded face image is an image whose face semantic information is to be extracted, and the occluded face image includes but is not limited to occluded eyes, occluded mouth, occluded nose, occluded cheeks, etc.

[0042] In one embodiment, a face semantic recognition model is obtained, and the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer, and a second face semantic recognition layer. The image reconstruction layer is used to reconstruct the occluded area in the occluded face image to obtain a target face image that is closer to the real image, the first face semantic recognition layer is used to perform face semantic recognition on the target face image, and the second face semantic recognition layer is used to perform face semantic recognition on the target face image. By obtaining the face semantic recognition model, the target face semantic information of the occluded face image can be accurately identified.

[0043] It should be noted that the model structures of the image reconstruction layer, the first face semantic recognition layer and the second face semantic recognition layer can be selected according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the image reconstruction layer can be a Masked Autoencoders Are Scalable Vision Learners (MAE) model, the first face semantic recognition layer can be a student network of a self-supervised self-distillation with no labels (DINO) model, and the second face semantic recognition layer can be a teacher network of a self-supervised self-distillation with no labels (DINO) model.

[0044] Step S102: reconstruct the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image.

[0045] Among them, the image reconstruction layer is used to reconstruct the pixels of the occluded area in the occluded face image to obtain a face image that is closer to the real image. The image reconstruction layer can be a Masked Autoencoders Are Scalable Vision Learners (MAE) model, and the MAE model includes an encoder and a decoder.

[0046] In one embodiment, if Figure 2 As shown, step S102 includes sub-steps S1021 to S1023.

[0047] Sub-step S1021, segmenting the blocked facial image to obtain a plurality of sub-facial images, and acquiring position information of each of the sub-facial images.

[0048] The occluded face image is segmented to obtain multiple sub-face images, wherein the segmentation granularity can be set according to actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the occluded face image is segmented into 9*9 sub-face images to obtain 81 sub-face images; for another example, the occluded face image is segmented into 16*16 sub-face images to obtain 256 sub-face images.

[0049] In one embodiment, after the occluded face image is segmented to obtain multiple sub-face images, the position information of each sub-face image is obtained, and the position information is the position of the sub-face image in the occluded face image. By obtaining the position information of each sub-face image, the occluded face image can be reconstructed.

[0050] Exemplarily, the occluded facial image is divided into 4*4 sub-facial images to obtain 16 sub-facial images, and the position information of each sub-facial image is obtained, and the position coordinates of each sub-facial image include (0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), (1, 3), (2, 0), (2, 1), (2, 2), (2, 3), (3, 0), (3, 1), (3, 2) and (3, 3).

[0051] Sub-step S1022: encoding each of the sub-face images and the position information of each of the sub-face images by the encoder to generate a first feature vector.

[0052] Each sub-face image is encoded to generate a sub-feature vector corresponding to each sub-face image; the position information of each sub-face image is encoded to generate a position feature vector corresponding to each position information; the sub-feature vector corresponding to each sub-face image is associated with the position feature vector to generate a first feature vector. By encoding the sub-face image and the position information, the sub-feature vector and the position feature vector corresponding to the sub-face image can be accurately obtained, and the sub-feature vector corresponding to each sub-face image can be associated with the position feature vector to accurately generate the first feature vector, which greatly improves the efficiency and accuracy of face semantic recognition.

[0053] Sub-step S1023: decoding the first feature vector by the decoder to obtain the target face image.

[0054] The target face image is an image obtained by interpolating the occluded face image, and the image of the occluded area in the target face image is clearer than the image of the corresponding area of ​​the occluded face image.

[0055] In one embodiment, the pixel value vector of each pixel point in the first feature vector is pixel reconstructed to obtain the target pixel value vector of each pixel point; each target pixel value vector is decoded to generate a target face image. By pixel reconstructing the pixel value vector of each pixel point, an image closer to the real one can be accurately obtained, which greatly improves the accuracy of face semantic recognition.

[0056] In one embodiment, the pixel value vector of each pixel point in the first feature vector is reconstructed to obtain the target pixel value vector of each pixel point by: obtaining the pixel value of each pixel point, and determining whether the pixel value of each pixel point meets the preset pixel condition, and reconstructing the pixel value vector of the pixel point that does not meet the preset pixel condition to obtain the target pixel value vector of each pixel point. Among them, the preset pixel condition can be set according to the actual situation, and the embodiment of the present invention does not make specific limitations on this. For example, the preset pixel condition can be whether the pixel value is greater than or equal to the preset pixel value. By only reconstructing the pixels that do not meet the preset pixel value, the amount of pixel reconstruction calculation can be reduced, which greatly improves the efficiency of semantic recognition of occluded faces.

[0057] In one embodiment, the pixel value vector of each pixel point in the first feature vector is reconstructed pixel by pixel to obtain the target pixel value vector of each pixel point. The pixel value vectors of all pixel points in the first feature vector are reconstructed pixel by pixel to obtain the target pixel value vector corresponding to each pixel point. By reconstructing the pixel value vectors of all pixel points, the overall clarity of the target face image can be improved, and the accuracy of semantic recognition of occluded faces can be greatly improved.

[0058] Step S103: performing facial semantic recognition on the target facial image through the first facial semantic recognition layer to obtain first facial semantic information.

[0059] The target face image is input into the first face semantic recognition layer to obtain the first face semantic information corresponding to the target face image. By performing face semantic recognition on the target face image through the first face semantic recognition layer, the first face semantic information can be accurately obtained, which greatly improves the efficiency and accuracy of face semantic recognition.

[0060] Step S104: performing facial semantic recognition on the target facial image through the second facial semantic recognition layer to obtain second facial semantic information.

[0061] The target face image is input into the second face semantic recognition layer to obtain the second face semantic information corresponding to the target face image. The second face semantic recognition layer performs face semantic recognition on the target face image to accurately obtain the second face semantic information, which greatly improves the efficiency and accuracy of face semantic recognition.

[0062] Step S105: Determine the target face semantic information of the occluded face image according to the first face semantic information and the second face semantic information.

[0063] The target face semantic information is face semantic information that makes the occluded face image closer to the real face image.

[0064] In one embodiment, if Figure 3 As shown, step S105 includes sub-steps S1051 and S1052.

[0065] Sub-step S1051, determining the similarity between the first face semantic information and the second face semantic information;

[0066] The cosine similarity between the first face semantic information and the second face semantic information is calculated to obtain the similarity between the first face semantic information and the second face semantic information. By calculating the cosine similarity between the first face semantic information and the second face semantic information, the similarity between the first face semantic information and the second face semantic information can be accurately obtained.

[0067] Sub-step S1052: when the similarity is greater than a preset similarity, determining the first facial semantic information as the target facial semantic information of the occluded facial image.

[0068] The preset similarity can be set according to the actual situation, and the embodiment of the present invention does not make any specific limitation to this.

[0069] In one embodiment, when it is determined that the similarity between the first face semantic information and the second face semantic information is greater than a preset similarity, it is determined that the face semantic information recognized by the first face semantic recognition layer is relatively accurate, and the first face semantic information is determined as the target face semantic information of the occluded face image. When the similarity is greater than the preset similarity, the first face semantic information is determined as the target face semantic information of the occluded face image. This greatly improves the efficiency and accuracy of face semantic recognition.

[0070] In one embodiment, when the similarity is less than or equal to the preset similarity, the model parameters of the first face semantic recognition layer are adjusted to update the first face semantic recognition layer; the step of performing face semantic recognition on the target face image through the first face semantic recognition layer to obtain the first face semantic information is continued until the similarity between the first face semantic information and the second face semantic information is greater than the preset similarity; the first face semantic recognition layer performs face semantic recognition on the target face image to obtain the first face semantic information, and determines the target face semantic information of the occluded face image. By adjusting the model parameters of the first face semantic recognition layer to update the first face semantic recognition layer, the face semantics recognized by the first face semantic recognition layer is made more accurate.

[0071] For example, Figure 4 As shown, the face semantic recognition model includes an image reconstruction layer 11, a first face semantic recognition layer 12 and a second face semantic recognition layer 13, which obtains an occluded face image, and inputs the obtained occluded face image into the image reconstruction layer 11 to reconstruct the occluded area to obtain a target face image; inputs the target face image into the first face semantic recognition layer 12 to perform face semantic recognition to obtain first face semantic information; inputs the target face image into the second face semantic recognition layer 13 to perform face semantic recognition to obtain second face semantic information; and determines the target face semantic information of the occluded face image based on the first face semantic information and the second face semantic information.

[0072] Exemplarily, when hospital doctors need to pass facial recognition to enter some high-security areas (for example, drug warehouses, operating rooms, and conference rooms, etc.), a facial image of the doctor wearing a mask is obtained, and a facial semantic recognition model is obtained. The occluded area in the facial image of the doctor wearing the mask is reconstructed through the image reconstruction layer to obtain a target facial image; facial semantic recognition is performed on the target facial image through the first facial semantic recognition layer to obtain first facial semantic information; facial semantic recognition is performed on the target facial image through the second facial semantic recognition layer to obtain second facial semantic information; and the target facial semantic information of the doctor is determined based on the first facial semantic information and the second facial semantic information.

[0073] Exemplarily, when a user purchases a financial product at a financial institution and performs payment verification, a facial image of the user is obtained, in which the user wears sunglasses, and an image reconstruction layer is used to reconstruct the occluded area in the facial image of the user to obtain a target facial image; a first facial semantic recognition layer is used to perform facial semantic recognition on the target facial image to obtain first facial semantic information; a second facial semantic recognition layer is used to perform facial semantic recognition on the target facial image to obtain second facial semantic information; and facial semantic information of the user's facial image is determined based on the first facial semantic information and the second facial semantic information. By recognizing the facial image of the user through the facial semantic recognition model, the facial semantic information of the user can be accurately determined, thereby improving the success rate of facial payment verification when the user purchases financial products at a financial institution, greatly improving the efficiency of financial business processing, and improving the customer's satisfaction with financial business.

[0074] The face semantic recognition method provided in the above embodiment obtains an occluded face image and a face semantic recognition model, the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer; the image reconstruction layer performs image reconstruction on the occluded area in the occluded face image to obtain a target face image; the first face semantic recognition layer performs face semantic recognition on the target face image to obtain first face semantic information; the second face semantic recognition layer performs face semantic recognition on the target face image to obtain second face semantic information; according to the first face semantic information and the second face semantic information, the target face semantic information of the occluded face image is determined. This solution performs face semantic recognition on the occluded face image through the face semantic recognition model, and can accurately obtain the target face semantic information of the occluded face image. This solution can accurately extract the semantic information of the user's face, improve the success rate of face payment verification when the user purchases financial products at a financial institution, greatly improve the efficiency of financial business processing and improve the customer's satisfaction with financial business.

[0075] Please refer to 5, Figure 5 A schematic block diagram of a facial semantic recognition device provided in an embodiment of the present application.

[0076] like Figure 5 As shown, the face semantic recognition device 200 includes an acquisition module 210, an image reconstruction module 220, a semantic recognition module 230 and a determination module 240, wherein:

[0077] The acquisition module 210 is used to acquire the blocked face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer;

[0078] The image reconstruction module 220 is used to reconstruct the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image;

[0079] The semantic recognition module 230 is used to perform facial semantic recognition on the target facial image through the first facial semantic recognition layer to obtain first facial semantic information;

[0080] The semantic recognition module 230 is further configured to perform facial semantic recognition on the target facial image through the second facial semantic recognition layer to obtain second facial semantic information;

[0081] The determination module 240 is used to determine the target face semantic information of the occluded face image according to the first face semantic information and the second face semantic information.

[0082] In one embodiment, if Figure 6 As shown, the image reconstruction module 220 includes a segmentation module 221, an acquisition submodule 222 and a generation module 223, wherein:

[0083] The segmentation module 221 is used to segment the blocked face image to obtain multiple sub-face images;

[0084] The acquisition submodule 222 is used to acquire the position information of each of the sub-face images;

[0085] The generating module 223 is used to encode each of the sub-face images and the position information of each of the sub-face images through the encoder to generate a first feature vector;

[0086] The generating module 223 is further configured to decode the first feature vector through the decoder to obtain the target face image.

[0087] In one embodiment, the generating module 223 is further used for:

[0088] Performing encoding processing on each of the sub-face images to generate a sub-feature vector corresponding to each of the sub-face images;

[0089] Encoding the position information of each of the sub-face images to generate a position feature vector corresponding to each of the position information;

[0090] The sub-feature vectors and the position feature vector corresponding to each of the sub-face images are associated to generate a first feature vector.

[0091] In one embodiment, the generating module 223 is further used for:

[0092] Performing pixel reconstruction on the pixel value vector of each pixel point in the first feature vector to obtain a target pixel value vector of each pixel point;

[0093] Each of the target pixel value vectors is decoded to generate the target face image.

[0094] In one embodiment, if Figure 7 As shown, the determination module 240 includes a first determination submodule 241 and a second determination submodule 242, wherein:

[0095] The first determination submodule 241 is used to determine the similarity between the first face semantic information and the second face semantic information;

[0096] The second determination submodule 242 is configured to determine the first facial semantic information as target facial semantic information of the occluded facial image when the similarity is greater than a preset similarity.

[0097] In one embodiment, the determining module 240 is further configured to:

[0098] When the similarity is less than or equal to a preset similarity, adjusting the model parameters of the first face semantic recognition layer to update the first face semantic recognition layer;

[0099] Continue to perform face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information until the similarity between the first face semantic information and the second face semantic information is greater than a preset similarity;

[0100] The first face semantic recognition layer performs face semantic recognition on the target face image to obtain first face semantic information, and determines the target face semantic information of the occluded face image.

[0101] In one embodiment, the first determining submodule 241 is further configured to:

[0102] The cosine similarity between the first face semantic information and the second face semantic information is calculated to obtain the similarity between the first face semantic information and the second face semantic information.

[0103] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-mentioned facial semantic recognition device can refer to the corresponding process in the aforementioned facial semantic recognition method embodiment, and will not be repeated here.

[0104] See also Figure 8 , Figure 8 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0105] like Figure 8 As shown, the computer device 300 includes a processor 302 and a memory 303 connected via a system bus 301 , wherein the memory 303 may include a storage medium and an internal memory.

[0106] The storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any face semantic recognition method.

[0107] The processor 302 is used to provide computing and control capabilities and support the operation of the entire computer device.

[0108] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any facial semantic recognition method.

[0109] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] It should be understood that the processor 302 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] In one embodiment, the processor 302 is used to run a computer program stored in the memory to implement the following steps:

[0112] Acquire an occluded face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer, and a second face semantic recognition layer;

[0113] Reconstructing the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image;

[0114] Performing face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information;

[0115] Performing face semantic recognition on the target face image through the second face semantic recognition layer to obtain second face semantic information;

[0116] The target face semantic information of the occluded face image is determined according to the first face semantic information and the second face semantic information.

[0117] In one embodiment, the processor 302 includes an encoder and a decoder when implementing the image reconstruction layer; when performing image reconstruction on the occluded area in the occluded face image through the image reconstruction layer to obtain the target face image, it is used to implement:

[0118] Segmenting the blocked face image to obtain a plurality of sub-face images, and acquiring position information of each of the sub-face images;

[0119] Encoding each of the sub-face images and the position information of each of the sub-face images by the encoder to generate a first feature vector;

[0120] The first feature vector is decoded by the decoder to obtain the target face image.

[0121] In one embodiment, when implementing the encoding process of each of the sub-face images and the position information of each of the sub-face images by the encoder to generate the first feature vector, the processor 302 is used to implement:

[0122] Performing encoding processing on each of the sub-face images to generate a sub-feature vector corresponding to each of the sub-face images;

[0123] Encoding the position information of each of the sub-face images to generate a position feature vector corresponding to each of the position information;

[0124] The sub-feature vectors and the position feature vector corresponding to each of the sub-face images are associated to generate a first feature vector.

[0125] In one embodiment, when the processor 302 decodes the first feature vector through the decoder to obtain the target face image, it is configured to implement:

[0126] Performing pixel reconstruction on the pixel value vector of each pixel point in the first feature vector to obtain a target pixel value vector of each pixel point;

[0127] Each of the target pixel value vectors is decoded to generate the target face image.

[0128] In one embodiment, when the processor 302 determines the target face semantic information of the occluded face image according to the first face semantic information and the second face semantic information, it is used to implement:

[0129] Determine the similarity between the first face semantic information and the second face semantic information;

[0130] When the similarity is greater than a preset similarity, the first facial semantic information is determined as the target facial semantic information of the occluded facial image.

[0131] In one embodiment, after implementing the determining of the similarity between the first face semantic information and the second face semantic information, the processor 302 is further configured to implement:

[0132] When the similarity is less than or equal to a preset similarity, adjusting the model parameters of the first face semantic recognition layer to update the first face semantic recognition layer;

[0133] Continue to perform face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information until the similarity between the first face semantic information and the second face semantic information is greater than a preset similarity;

[0134] The first face semantic recognition layer performs face semantic recognition on the target face image to obtain first face semantic information, and determines the target face semantic information of the occluded face image.

[0135] In one embodiment, the processor 302, when implementing the determining of the similarity between the first face semantic information and the second face semantic information, is used to implement:

[0136] The cosine similarity between the first face semantic information and the second face semantic information is calculated to obtain the similarity between the first face semantic information and the second face semantic information.

[0137] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned face semantic recognition method embodiment, and will not be repeated here.

[0138] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the facial semantic recognition method of the present application.

[0139] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0140] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0141] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0142] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0143] It should also be understood that the term "and / or" used in the present specification refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0144] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A method for face semantic recognition, characterized in that: include: Acquire an occluded face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer, and a second face semantic recognition layer; Reconstructing the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image; Performing face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information; Performing face semantic recognition on the target face image through the second face semantic recognition layer to obtain second face semantic information; The target face semantic information of the occluded face image is determined according to the first face semantic information and the second face semantic information.

2. The face semantic recognition method according to claim 1, characterized in that: The image reconstruction layer includes an encoder and a decoder; the image reconstruction layer is used to reconstruct the occluded area in the occluded face image to obtain the target face image, including: Segmenting the blocked face image to obtain a plurality of sub-face images, and acquiring position information of each of the sub-face images; Encoding each of the sub-face images and the position information of each of the sub-face images by the encoder to generate a first feature vector; The first feature vector is decoded by the decoder to obtain the target face image.

3. The face semantic recognition method according to claim 2, characterized in that: The encoding process of each of the sub-face images and the position information of each of the sub-face images by the encoder to generate a first feature vector includes: Performing encoding processing on each of the sub-face images to generate a sub-feature vector corresponding to each of the sub-face images; Encoding the position information of each of the sub-face images to generate a position feature vector corresponding to each of the position information; The sub-feature vectors and the position feature vector corresponding to each of the sub-face images are associated to generate a first feature vector.

4. The method for face semantic recognition according to claim 2, characterized in that: The decoding process of the first feature vector by the decoder to obtain the target face image includes: Performing pixel reconstruction on the pixel value vector of each pixel point in the first feature vector to obtain a target pixel value vector of each pixel point; Each of the target pixel value vectors is decoded to generate the target face image.

5. The method for face semantic recognition according to claim 4, characterized in that: The determining, according to the first face semantic information and the second face semantic information, the target face semantic information of the occluded face image comprises: Determine the similarity between the first face semantic information and the second face semantic information; When the similarity is greater than a preset similarity, the first facial semantic information is determined as the target facial semantic information of the occluded facial image.

6. The method for face semantic recognition according to claim 5, characterized in that: After determining the similarity between the first face semantic information and the second face semantic information, the method further includes: When the similarity is less than or equal to a preset similarity, adjusting the model parameters of the first face semantic recognition layer to update the first face semantic recognition layer; Continue to perform face semantic recognition on the target face image through the first face semantic recognition layer to obtain first face semantic information until the similarity between the first face semantic information and the second face semantic information is greater than a preset similarity; The first face semantic recognition layer performs face semantic recognition on the target face image to obtain first face semantic information, and determines the target face semantic information of the occluded face image.

7. The method for face semantic recognition according to claim 5, characterized in that: Determining the similarity between the first face semantic information and the second face semantic information includes: The cosine similarity between the first face semantic information and the second face semantic information is calculated to obtain the similarity between the first face semantic information and the second face semantic information.

8. A facial semantic recognition device, characterized in that: The face semantic recognition device comprises an acquisition module, an image reconstruction module, a semantic recognition module and a determination module, wherein: The acquisition module is used to acquire the blocked face image and acquire a face semantic recognition model, wherein the face semantic recognition model includes an image reconstruction layer, a first face semantic recognition layer and a second face semantic recognition layer; The image reconstruction module is used to reconstruct the occluded area in the occluded face image through the image reconstruction layer to obtain a target face image; The semantic recognition module is used to perform facial semantic recognition on the target facial image through the first facial semantic recognition layer to obtain first facial semantic information; The semantic recognition module is further used to perform facial semantic recognition on the target facial image through the second facial semantic recognition layer to obtain second facial semantic information; The determination module is used to determine the target face semantic information of the occluded face image according to the first face semantic information and the second face semantic information.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the face semantic recognition method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the face semantic recognition method according to any one of claims 1 to 7 are implemented.

Images