Method, device, electronic device and storage medium for locating key points of human face

Abstract

The present invention relates to the field of face recognition technology and provides a method, device, electronic device, and storage medium for locating facial key points. By using a face deocclusion model to obtain an unobstructed face image corresponding to an original face image, and further using a key point positioning model, the facial key point information in the original face image can be accurately obtained, thereby avoiding the uncertainty or noise caused by directly locating facial key points on an occluded face image in the prior art. Furthermore, the method does not require the use of a key point positioning model to locate facial key points on an occluded face image, nor does it require manual labeling of occluded samples to train the key point positioning model. This method can ensure the accuracy of the key point positioning model's positioning of unobstructed face images, thereby reducing positioning costs.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a method, device, electronic device and storage medium for locating key points of a face. Background Art

[0002] Facial landmark localization involves detecting the position of each face element and key points in a facial image, such as the nose, eyes, and mouth. This technology can be used for facial pose correction, gesture recognition, fatigue monitoring, 3D facial reconstruction, facial animation, face recognition, and expression analysis. Incorrect key point localization can lead to facial distortion, making algorithms that accurately extract facial landmarks crucial.

[0003] At present, there are three main facial key point positioning algorithms, namely, Active Appearance Models (AAM) based on generative models, Active Shape Models (AAM) and its extended and expanded positioning methods, positioning methods based on cascade shape regression, and positioning methods based on deep learning.

[0004] All three of the aforementioned methods utilize localization models to locate facial landmarks in existing occluded face images. However, due to the lack of apparent facial features in the occluded regions of these images, the locations of keypoints within the occluded regions can only be estimated using the positions of other visible points and the structural priors of the face itself. When the occluded regions are large, directly estimating the locations of keypoints within them results in significant uncertainty and noise. This also greatly complicates the manual labeling of occluded samples, resulting in a scarcity of occluded samples and the inability to directly train a robust localization model through data-driven approaches. Summary of the Invention

[0005] The present invention provides a method, device, electronic device and storage medium for locating key points of a face, so as to solve the defects in the prior art.

[0006] The present invention provides a method for locating key points of a face, comprising:

[0007] Obtain the original face image for key point positioning;

[0008] Inputting the original face image into a face deocclusion model to obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model;

[0009] Inputting the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model;

[0010] Among them, the face deocclusion model is trained based on the first unobstructed face image sample and the corresponding first occluded face image sample, and the key point positioning model is trained based on the second unobstructed face image sample carrying the first face key point label.

[0011] According to a facial key point positioning method provided by the present invention, the first occluded facial image sample is determined based on the following method:

[0012] Determine first three-dimensional facial information of the first unobstructed facial image sample, and determine a first texture map of a facial region in the first unobstructed facial image sample based on the first three-dimensional facial information;

[0013] Obtaining a second occluded face image sample, determining second three-dimensional facial information of the second occluded face image sample, and determining a second texture map of an occluded area in the second occluded face image sample based on the second three-dimensional facial information;

[0014] The first texture map and the second texture map are fused to obtain a first fusion result, and the first occluded face image sample is determined based on the first three-dimensional face information and the first fusion result.

[0015] According to a facial key point location method provided by the present invention, the method includes inputting the original facial image into a facial deocclusion model to obtain an unobstructed facial image corresponding to the original facial image output by the facial deocclusion model, including:

[0016] Inputting the original face image into the encoder module of the face deocclusion model, and having the encoder module perform feature extraction on the original face image to obtain a feature vector of the original face image output by the encoder module;

[0017] The feature vector is input into the generation module of the face de-occlusion model, and the generation module performs image reconstruction on the feature vector to obtain the unobstructed face image output by the generation module.

[0018] According to a facial key point location method provided by the present invention, the encoder module is trained based on the following method:

[0019] Inputting the first unobstructed face image sample and the first obstructed face image sample into an initial encoder module respectively, and obtaining a first feature vector sample corresponding to the first unobstructed face image sample and a second feature vector sample corresponding to the first obstructed face image sample output by the initial encoder module;

[0020] Based on the first feature vector sample and the second feature vector sample, a feature loss of the initial encoder module is calculated, and based on the feature loss, parameters of the initial encoder module are iterated to obtain the encoder module.

[0021] According to a facial key point location method provided by the present invention, the first occluded face image sample carries a second facial key point label, and the second facial key point label is determined based on the facial key point label carried by the first unoccluded face image sample;

[0022] The generation module is trained based on the following method:

[0023] Inputting the third feature vector sample corresponding to the first occluded face image sample into an initial generation module to obtain a first face image corresponding to the third feature vector sample output by the initial generation module;

[0024] Inputting the first facial image into an auxiliary key point positioning module to obtain auxiliary facial key point information corresponding to the first facial image output by the auxiliary key point positioning module;

[0025] Calculating the key point consistency loss of the initial generation module based on the auxiliary facial key point information and the second facial key point label;

[0026] Based on the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

[0027] According to a facial key point location method provided by the present invention, the method of performing parameter iteration on the initial generation module based on the key point consistency loss to obtain the generation module specifically includes:

[0028] Inputting the fourth feature vector sample corresponding to the first unobstructed face image sample into an initial generation module to obtain a second face image corresponding to the fourth feature vector sample output by the initial generation module;

[0029] Calculating a first image reconstruction loss of the initial generation module based on the first face image and the first unobstructed face image sample, and calculating a second image reconstruction loss of the initial generation module based on the second face image and the first unobstructed face image sample;

[0030] Inputting the first facial image, the second facial image, and the first unobstructed facial image sample into an auxiliary discrimination module, respectively, to obtain a first feature map corresponding to the first facial image, a second feature map corresponding to the second facial image, and a third feature map corresponding to the first unobstructed facial image sample extracted by the auxiliary discrimination module;

[0031] Calculating a first feature matching loss of the initial generation module based on the first feature map and the third feature map, and calculating a second feature matching loss of the initial generation module based on the second feature map and the third feature map;

[0032] Based on the first image reconstruction loss, the second image reconstruction loss, the first feature matching loss, the second feature matching loss and the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

[0033] According to a facial key point location method provided by the present invention, the key point location model is trained based on the following method:

[0034] Inputting the second unobstructed facial image sample into an initial key point positioning model, obtaining a first Gaussian response map centered on the key point corresponding to the second unobstructed facial image sample output by the initial key point positioning model, and determining a second Gaussian response map corresponding to the first facial key point label;

[0035] Calculating a key point positioning loss of the initial key point positioning model based on the first Gaussian response map and the second Gaussian response map;

[0036] Based on the key point positioning loss, parameters of the initial key point positioning model are iterated to obtain a key point positioning model.

[0037] The present invention also provides a facial key point positioning device, comprising:

[0038] An image acquisition module is used to acquire the original face image for key point positioning;

[0039] a deocclusion module, configured to input the original face image into a face deocclusion model, and obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model;

[0040] A key point positioning module is used to input the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model;

[0041] Among them, the face deocclusion model is trained based on the first unobstructed face image sample and the corresponding first occluded face image sample, and the key point positioning model is trained based on the second unobstructed face image sample carrying the first face key point label.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for locating key points of a face as described above is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for locating facial key points.

[0044] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-described methods for locating key points of a face.

[0045] The facial key point localization method, device, electronic device, and storage medium provided by the present invention utilize a facial deocclusion model to obtain an unobstructed facial image corresponding to an original facial image. Furthermore, the facial key point information in the original facial image can be accurately obtained using a key point localization model, thus avoiding the uncertainty or noise caused by directly localizing facial key points on occluded facial images in the prior art. Furthermore, this method does not require the use of a key point localization model to localize facial key points on occluded facial images, nor does it require manual labeling of occluded samples to train the key point localization model. This ensures the key point localization model's accuracy in localizing unobstructed facial images, thereby reducing localization costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without any creative work.

[0047] Figure 1 Schematic diagram of the process of locating key points of a face provided by the present invention;

[0048] Figure 2 This is a schematic diagram of the process of generating the first occluded face image sample in the face key point positioning method provided by the present invention;

[0049] Figure 3 It is a schematic diagram of the process of training the initial model of the face de-occlusion model in the face key point positioning method provided by the present invention;

[0050] Figure 4 It is a structural schematic diagram of the facial key point positioning device provided by the present invention;

[0051] Figure 5It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] Because existing facial landmark localization methods typically utilize a localization model to locate facial landmarks on existing occluded facial images, directly estimating the positions of keypoints within large occluded areas can produce significant uncertainty and noise. This also greatly complicates manual labeling of occluded samples, resulting in a scarcity of occluded samples and the inability to directly train a robust, anti-occlusion localization model through data-driven training. To address these issues, embodiments of the present invention provide a facial landmark localization method.

[0054] Figure 1 FIG. 1 is a flow chart of a method for locating key points of a face provided in an embodiment of the present invention, as shown in FIG. Figure 1 As shown, the method includes:

[0055] S1, obtain the original face image to be located with key points;

[0056] S2, inputting the original face image into a face deocclusion model to obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model;

[0057] S3, inputting the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model;

[0058] Among them, the face deocclusion model is trained based on the first unobstructed face image sample and the corresponding first occluded face image sample, and the key point positioning model is trained based on the second unobstructed face image sample carrying the first face key point label.

[0059] Specifically, the facial key point locating method provided in the embodiment of the present invention is executed by a facial key point locating device, which can be configured in a server. The server can be a local server or a cloud server. The local server can specifically be a computer, etc., and this is not specifically limited in the embodiment of the present invention.

[0060] First, execute step S1 to obtain the original face image for key point positioning. The original face image refers to the image in which the facial key point information of the face area needs to be determined. It can be an unobstructed face image, that is, the face area is not obstructed by any obstruction, or an occluded face image, that is, the face area is obstructed by any obstruction. No specific limitation is made here.

[0061] Then, step S2 is executed, where the original facial image is input into a facial deocclusion model. The facial deocclusion model performs a deocclusion operation on the original facial image to obtain and output an unobstructed facial image corresponding to the original facial image. The deocclusion operation removes obstructions such as masks, sunglasses, hands, and scarves from the facial image.

[0062] Since the original face image can be either an unobstructed face image or an occluded face image, the input of the face deocclusion model can be either an unobstructed face image or an occluded face image, and the output is always an unobstructed face image.

[0063] When the original face image is an unobstructed face image, the deocclusion operation of the face deocclusion model has no actual effect on the original face image. At this time, the unobstructed face image corresponding to the output original face image is itself or the error with the original face image is within a preset range; when the original face image is an occluded face image, the deocclusion operation of the face deocclusion model can remove the occlusion on the face area in the original face image to obtain an unobstructed face image.

[0064] In embodiments of the present invention, the face deocclusion model employed can be obtained by training an initial face deocclusion model using a first unobstructed face image sample and a corresponding first occluded face image sample. The initial face deocclusion model can be a neural network model or other model, which is not specifically limited herein.

[0065] The first unobstructed face image sample refers to a face image sample that fully displays the face area, and can be obtained from a conventional face image library or by photographing. The first obstructed face image sample refers to a face image sample corresponding to the first unobstructed face image sample, but with the face area partially obstructed by an obstructing object. It can be automatically generated from the first unobstructed face image sample or obtained by photographing, without specific limitation herein.

[0066] The first unobstructed face image sample and the first obstructed face image sample are two images of the same person, and the only difference between them is that the face region of the former is not obstructed by an obstruction, while the face region of the latter is obstructed by an obstruction.

[0067] There are multiple first unobstructed face image samples, which can be set as needed. Different first unobstructed face image samples can be face images of the same person in different postures, or face images of different people, which is not specifically limited here.

[0068] Here, the first unobstructed face image sample can be copied to obtain two identical first unobstructed face image samples, and a first sample pair can be formed. One of the first unobstructed face image samples is used as the model input, and the other first unobstructed face image sample is used as the label. The initial face deocclusion model is trained by the first sample pair, so that the obtained face deocclusion model has the function of directly outputting unobstructed face images.

[0069] The first unobstructed face image sample and the corresponding first occluded face image sample can also form a second sample pair, and the first occluded face image sample is used as the model input, and the first unobstructed face image sample is used as the label. The initial face occlusion model is trained by the second sample pair, so that the obtained face deocclusion model has the function of performing deocclusion operations on occluded face images.

[0070] Finally, step S3 is executed, where the unobstructed facial image obtained in step S2 is input into a key point localization model. The key point localization model then obtains and outputs facial key point information from the unobstructed facial image. Facial key point information refers to the location information of key points in the facial region of the unobstructed facial image. The number of key points can include 5, 21, 49, 68, or 92, and is not specifically limited here.

[0071] The key point localization model can be obtained by training the initial key point localization model using a second unobstructed facial image sample that carries the first facial key point labels. The second unobstructed facial image sample can be the same as or different from the first unobstructed facial image sample, without specific limitation. The first facial key point labels are the location information of the key points in the facial region in the second unobstructed facial image sample. The initial key point localization model can be an hourglass network, a U-net network, or the like.

[0072] The facial key point localization method provided in an embodiment of the present invention first obtains an original facial image for key point localization; then, the original facial image is input into a facial deocclusion model to obtain an unobstructed facial image corresponding to the original facial image output by the facial deocclusion model; and finally, the unobstructed facial image is input into a key point localization model to obtain facial key point information in the unobstructed facial image output by the key point localization model. This method uses the facial deocclusion model to obtain an unobstructed facial image corresponding to the original facial image, and further uses the key point localization model to accurately obtain facial key point information in the original facial image, avoiding the uncertainty or noise caused by directly localizing facial key points on occluded facial images in the prior art. Furthermore, this method does not require the use of a key point localization model to localize facial key points on occluded facial images, nor does it require manual labeling of occluded samples to train the key point localization model. This ensures the accuracy of the key point localization model for localizing unobstructed facial images, thereby reducing localization costs.

[0073] In existing technologies, face deocclusion is typically implemented using an object removal model. This model is trained using training samples consisting of a source face image, an occluded face image, and a target output face image. The occluded face images used to train the object removal model are typically obtained by directly capturing them or by directly adding occluder maps, such as masks and scarves, to the target output face image.

[0074] It is difficult to find the corresponding source face image for the occluded face image obtained by direct shooting. Directly adding occluding object maps such as masks and scarves to the target output face image will result in a large difference between the obtained occluded face image and the actual occluded face image, affecting the de-occlusion effect of the de-occlusion model.

[0075] To this end, based on the above embodiment, in the facial key point positioning method provided in the embodiment of the present invention, the first occluded face image sample is determined based on the following method:

[0076] Determine first three-dimensional facial information of the first unobstructed facial image sample, and determine a first texture map of a facial region in the first unobstructed facial image sample based on the first three-dimensional facial information;

[0077] Obtaining a second occluded face image sample, determining second three-dimensional facial information of the second occluded face image sample, and determining a second texture map of an occluded area in the second occluded face image sample based on the second three-dimensional facial information;

[0078] The first texture map and the second texture map are fused to obtain a first fusion result, and the first occluded face image sample is determined based on the first three-dimensional face information and the first fusion result.

[0079] Specifically, in embodiments of the present invention, when determining a first occluded facial image sample, first 3D facial information of a first unobstructed facial image sample can be determined. This first 3D facial information refers to 3D information of the facial region in the first unobstructed facial image sample. The first unobstructed facial image sample can be input into a 3D facial reconstruction network, which performs 3D reconstruction of the facial region in the first unobstructed facial image sample to obtain the first 3D facial information. The 3D facial reconstruction network can be a PRNet or other network.

[0080] Then, the texture of the facial region in the first unobstructed facial image sample can be mapped to a facial texture map space using the first three-dimensional facial information, thereby determining a first texture map of the facial region in the first unobstructed facial image sample. The facial texture map space can be a UV space, where mapping involves unfolding the surface texture of the facial region in the three-dimensional first unobstructed facial image sample and then rendering it on a two-dimensional plane, thereby obtaining a first texture map that is a UV texture map. U and V are coordinates in the facial texture map space, and are used to define the position information of each point in the facial texture map space.

[0081] Afterwards, a second occluded face image sample can be obtained, and the second occluded face image sample can be a real occluded face image obtained by shooting. The second occluded face image sample can be input into a face three-dimensional reconstruction network, and the face area in the second occluded face image sample can be three-dimensionally reconstructed by the face three-dimensional reconstruction network to obtain second face three-dimensional information. Through the second face three-dimensional information, the texture of the occluded area in the second unoccluded face image sample can be mapped to the face texture map space to determine the second texture map of the occluded area in the second occluded face image sample. The occluded area refers to the area covered by the occluder in the face area, which can be obtained by performing occluder segmentation on the second unoccluded face image sample. The second texture map can also be a UV texture map.

[0082] Finally, the first texture map and the second texture map may be fused to obtain a first fusion result, and combined with the first three-dimensional face information, the first fusion result may be three-dimensionally rendered to determine a first occluded face image sample.

[0083] After obtaining the first fusion result, the first fusion result can be subjected to Gaussian filtering and smoothing processing, and then combined with the first three-dimensional face information, the Gaussian filtering and smoothing processing result can be three-dimensionally rendered to determine the first occluded face image sample.

[0084] like Figure 2 , which is a schematic diagram of the generation process of the first occluded face image sample. Figure 2 In this process, the second occluded face image sample is segmented by the occluder to obtain the texture of the occluded area, and then texture mapped to obtain a second UV texture map. Furthermore, the first unobstructed face image sample undergoes 3D reconstruction and texture mapping to obtain a first UV texture map. The first UV texture map is fused with the second UV texture map to obtain a first fusion result, which is then combined with the 3D information of the first face to render the first occluded face image sample.

[0085] In an embodiment of the present invention, the first occluded face image sample corresponding to the first unobstructed face image sample can be determined through the existing second occluded face image sample without direct shooting, so that the occluded area in the first occluded face image sample can be closer to the actual occluded area position, making the occluded area in the first occluded face image sample more in line with the actual situation.

[0086] On the basis of the above embodiment, the facial key point localization method provided in the embodiment of the present invention, wherein the original facial image is input into a facial deocclusion model, and an unobstructed facial image corresponding to the original facial image output by the facial deocclusion model is obtained, includes:

[0087] Inputting the original face image into the encoder module of the face deocclusion model, and having the encoder module perform feature extraction on the original face image to obtain a feature vector of the original face image output by the encoder module;

[0088] The feature vector is input into the generation module of the face de-occlusion model, and the generation module performs image reconstruction on the feature vector to obtain the unobstructed face image output by the generation module.

[0089] Specifically, in embodiments of the present invention, a face deocclusion model may include an encoder module and a generator module. The encoder module may be a mapping network within a convolutional neural network structure, configured to extract features from the input. The generator module may be a style generative adversarial network, which may include a face synthesis network, which may be a fully convolutional neural network, configured to reconstruct an image based on the feature extraction results of the encoder module. It is understood that the feature extraction process may be equivalent to mapping the original face image to the latent space of the style generative adversarial network.

[0090] After the original face image is input into the face deocclusion model, the encoder module can be used to extract features of the original face image to obtain a feature vector of the original face image, which can be a 512-dimensional latent vector.

[0091] After that, the feature vector is input into the generation module of the face deocclusion model. The face synthesis network in the generation module is combined with the adaptive instance normalization method to fuse the feature vector with the noise feature map to obtain the second fusion result. Among them, if the feature vector is w1, the feature vector can be injected into each layer of the face synthesis network through the adaptive instance normalization method to guide the style of the face image generated by the face synthesis network. First, through the affine transformation matrix T i Perform an affine transformation on w1 to obtain y i , that is:

[0092] y i =T i (w1)

[0093] Then, y i Split into y s，i and y b，i Two vectors, y s，i and y b，i The dimensions are equal to the number of channels of the face synthesis network at this layer.

[0094] Afterwards, Gaussian noise is added to the face synthesis network to control the randomness of the synthesized image. And y is calculated by the following method s，i and y b，i and noise characteristic graph x i To integrate, we have:

[0095]

[0096] Among them, x i is the feature map of the ith channel in a layer of the face synthesis network through the Gaussian noise map z, AdaIN(x i ,y i ) is the fusion result corresponding to a certain layer, μ(x i ) and σ(x i ) represent x i The mean and variance of all eigenvalues ​​above.

[0097] Finally, the second fusion result obtained by layer-by-layer fusion can obtain an unobstructed face image.

[0098] In an embodiment of the present invention, the initial model may include an initial encoder module and an initial generation module. When training the initial model, the initial encoder module and the initial generation module may be trained separately. The face synthesis network in the initial generation module may be obtained through pre-training in advance, that is, the face synthesis network in the initial generation module is the same as the face synthesis network in the generation module.

[0099] In the embodiment of the present invention, a specific structure of a face de-occlusion model is provided, and the acquisition of an unobstructed face image corresponding to the original face image is achieved through the collaboration of various modules.

[0100] On the basis of the above embodiment, in the method for locating key points of a face provided in the embodiment of the present invention, the encoder module is trained based on the following method:

[0101] Inputting the first unobstructed face image sample and the first obstructed face image sample into an initial encoder module respectively, and obtaining a first feature vector sample corresponding to the first unobstructed face image sample and a second feature vector sample corresponding to the first obstructed face image sample output by the initial encoder module;

[0102] Based on the first feature vector sample and the second feature vector sample, a feature loss of the initial encoder module is calculated, and based on the feature loss, parameters of the initial encoder module are iterated to obtain the encoder module.

[0103] Specifically, in the embodiment of the present invention, in the process of training the initial encoder module to obtain the encoder module, the first unobstructed face image sample I and the first obstructed face image sample I can be firstly o Input them into the initial encoder module respectively, and obtain the first feature vector sample w corresponding to the first unobstructed face image sample I and the first obstructed face image sample I output by the initial encoder module. o The corresponding second eigenvector sample w o . That is:

[0104] w o =F mapping (I o )

[0105] w=F mapping (I)

[0106] Among them, F mapping Represents the initial encoder module.

[0107] The first eigenvector sample w and the second eigenvector sample w o It can be regarded as the first unobstructed face image sample I and the first obstructed face image sample I oSimultaneously mapped to the same position in the latent space of the initial style generation adversarial module.

[0108] Furthermore, the feature loss L of the initial encoder module can be calculated using the first feature vector sample and the second feature vector sample using the following formula: w :

[0109] L w =||ww o ||2

[0110] The feature loss is the mean square error (MSE) to ensure that the first feature vector sample w and the second feature vector sample w o The positions in the latent space are as consistent as possible.

[0111] Finally, through the above feature loss L w , perform parameter iteration on the initial encoder module, that is, adjust the parameters of the initial encoder module, and on this basis continue to use other first unobstructed face image samples and other first occluded face image samples to calculate the feature loss, and continue to adjust the parameters until the preset number of iterations is reached, or the feature loss converges, and the encoder module is obtained.

[0112] In an embodiment of the present invention, the initial encoder module is trained by combining the first unobstructed face image sample with the first occluded face image sample, so that the trained encoder module has the function of unobstructed encoding of the occluded face image, that is, the obtained feature vector is the feature vector of the unobstructed face image corresponding to the occluded face image, so as to facilitate the combination with the generation module to obtain the unobstructed face image.

[0113] Based on the above embodiment, in the facial key point localization method provided in the embodiment of the present invention, the first occluded face image samples all carry second facial key point labels, and the second facial key point labels are determined based on the facial key point labels carried by the first unoccluded face image samples;

[0114] The generation module is trained based on the following method:

[0115] Inputting the third feature vector sample corresponding to the first occluded face image sample into an initial generation module to obtain a first face image corresponding to the third feature vector sample output by the initial generation module;

[0116] Inputting the first facial image into an auxiliary key point positioning module to obtain auxiliary facial key point information corresponding to the first facial image output by the auxiliary key point positioning module;

[0117] Calculating the key point consistency loss of the initial generation module based on the auxiliary facial key point information and the second facial key point label;

[0118] Based on the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

[0119] Specifically, in an embodiment of the present invention, the first occluded facial image sample may carry a second facial key point label. The second facial key point label may be the facial key point label carried by the first occluded facial image sample when the first occluded facial image sample was generated, while the second occluded facial image sample used in the generation process did not carry the label.

[0120] On this basis, when training the initial generation module, the first occluded face image sample I o The corresponding third feature vector sample w3 is input to the initial generation module to obtain the first face image I corresponding to the third feature vector sample w3 output by the initial generation module. o The third feature vector sample w3 can be generated by a trained encoder module, which is not specifically limited here.

[0121] Then the first face image I o ′ are input to the auxiliary key point positioning module respectively, and the first face image I output by the auxiliary key point positioning module is obtained. o The auxiliary key point positioning module can be a conventional module with key point positioning function, and the role of introducing the auxiliary key point positioning module is to calculate the key point consistency loss of the initial generation module through the facial key point information obtained by the auxiliary key point positioning module.

[0122] The key point consistency loss of the initial generation module can be calculated based on the auxiliary face key point information and the second face key point label, that is:

[0123] L ldmk =||TD ldmk (I o ′)||2

[0124] Among them, T is the second face key point label, which can be the coordinate vector form of the face key point, D ldmk represents the auxiliary key point positioning module, D ldmk (I o ′) represents the auxiliary face key point information, L ldmk represents the keypoint consistency loss.

[0125] It can be understood that the key point consistency loss is the mean square error, so as to ensure that the original face structure of the occluded face image is not destroyed after the deocclusion operation is performed on the occluded face image.

[0126] Finally, according to the key point consistency loss, the parameters of the initial generation module are iterated, that is, the parameters of the initial generation module are adjusted, and on this basis, other first occluded face image samples are used to calculate the key point consistency loss, and the parameters are continued to be adjusted until the preset number of iterations is reached, or the key point consistency loss converges, and the generation module is obtained.

[0127] In the embodiment of the present invention, the initial generation module is trained by combining the first occluded face image sample with the auxiliary key point positioning module, so that the trained generation module can have the image reconstruction function of keeping the face structure unchanged.

[0128] On the basis of the above embodiment, the facial key point localization method provided in the embodiment of the present invention, wherein the initial generation module is subjected to parameter iteration based on the key point consistency loss to obtain the generation module, specifically includes:

[0129] Inputting the fourth feature vector sample corresponding to the first unobstructed face image sample into an initial generation module to obtain a second face image corresponding to the fourth feature vector sample output by the initial generation module;

[0130] Calculating a first image reconstruction loss of the initial generation module based on the first face image and the first unobstructed face image sample, and calculating a second image reconstruction loss of the initial generation module based on the second face image and the first unobstructed face image sample;

[0131] Inputting the first facial image, the second facial image, and the first unobstructed facial image sample into an auxiliary discrimination module, respectively, to obtain a first feature map corresponding to the first facial image, a second feature map corresponding to the second facial image, and a third feature map corresponding to the first unobstructed facial image sample extracted by the auxiliary discrimination module;

[0132] Calculating a first feature matching loss of the initial generation module based on the first feature map and the third feature map, and calculating a second feature matching loss of the initial generation module based on the second feature map and the third feature map;

[0133] Based on the first image reconstruction loss, the second image reconstruction loss, the first feature matching loss, the second feature matching loss and the key point consistency loss, parameter iteration is performed on the initial generation module to obtain a generated module.

[0134] Specifically, in an embodiment of the present invention, when training the initial generation module, in addition to introducing key point consistency loss, image reconstruction loss and feature matching loss may also be introduced.

[0135] The fourth feature vector sample w4 corresponding to the first unobstructed facial image sample I can be input into the initial generation module to obtain the second facial image I′ corresponding to the fourth feature vector sample w4 output by the initial generation module. The fourth feature vector sample w4 can be generated by a trained encoder module, which is not specifically limited here.

[0136] Then, according to the first face image I o ' and the first unobstructed face image sample I, calculate the first image reconstruction loss of the initial generation module, that is:

[0137] L recon =||II o ′||1

[0138] Among them, L recon The first image reconstruction loss is used to ensure that the texture and attributes of the occluded face image are consistent with those of the unoccluded face image after the deocclusion operation is performed on the occluded face image.

[0139] According to the second face image I′ and the first unobstructed face image sample I, the second image reconstruction loss of the initial generation module is calculated, that is,

[0140] L′ recon =||II′||1

[0141] Among them, L′ recon is the second image reconstruction loss, which is used to ensure that the texture, attributes, etc. of the unobstructed face image are consistent with those of the unobstructed face image after the de-occlusion operation is performed on the unobstructed face image.

[0142] Afterwards, the first face image I o ′, the second face image I′ and the first unobstructed face image sample I are respectively input into the auxiliary discrimination module to obtain the first face image I extracted by the auxiliary discrimination module. o ’, the second feature map corresponding to the second face image I’, and the third feature map corresponding to the first unobstructed face image sample I. The structure of the auxiliary discrimination module is the same as that of the face recognition network in the initial style adversarial generation model, both of which include multiple layers, and each layer of the auxiliary discrimination module has a feature extraction function. The auxiliary discrimination module also has the function of discriminating and classifying the input image through the extracted features, that is, the feature maps extracted by the auxiliary discrimination module are all used for the subsequent discrimination and classification of the input image. It can be understood that the discrimination and classification refers to discriminating whether the input image is an occluded face image or an unobstructed face image.

[0143] Furthermore, based on the first feature map and the third feature map, the first feature matching loss of the initial generation module can be calculated, that is:

[0144]

[0145] Among them, D i (I) is the third feature map extracted by the i-th layer of the auxiliary discrimination module, D i (I o ′) is the first feature map extracted by the i-th layer of the auxiliary discrimination module, L FM It is the first feature matching loss, which is used to ensure that the hidden layer features of the occluded face image are consistent with those of the unoccluded face image after the deocclusion operation is performed on the occluded face image.

[0146] According to the second feature map and the third feature map, the second feature matching loss of the initial generation module can be calculated:

[0147]

[0148] Among them, D i (I′) is the second feature map extracted by the i-th layer of the auxiliary discrimination module, L′ FM is the second feature matching loss, which is used to ensure that the hidden layer features of the unobstructed face image are consistent with those of the unobstructed face image after the deocclusion operation is performed on the unobstructed face image.

[0149] Finally, according to the first image reconstruction loss, the second image reconstruction loss, the first feature matching loss, the second feature matching loss and the key point consistency loss, the parameters of the initial generation module are iterated, that is, the parameters of the initial generation module are adjusted, and on this basis, other first occluded face image samples and first unoccluded face image samples are used to calculate the above losses, and the parameters are continued to be adjusted until the preset number of iterations is reached, or all losses converge, and the generation module is obtained.

[0150] In an embodiment of the present invention, the initial generation module is trained by combining the first unobstructed face image sample with the auxiliary discrimination module, so that the trained generation module not only has the image reconstruction function of keeping the face structure unchanged, but also has the image reconstruction function of keeping the image content unchanged and the latent features unchanged.

[0151] Figure 3 This is a schematic diagram of the process of training the initial model of the face deocclusion model in the face key point positioning method provided in an embodiment of the present invention.

[0152] like Figure 3 As shown, the method includes:

[0153] The first occluded face image sample I o and the first unobstructed face image sample I are respectively input into the initial encoder module, and the first obstructed face image sample I output by the initial encoder module is obtained. o The corresponding second eigenvector sample w o And the first feature vector sample w corresponding to the first unobstructed face image sample I, and the feature loss L of the initial encoder module is calculated by the first feature vector sample and the second feature vector sample w , to train the initial encoder module and obtain the encoder module.

[0154] Then, the first occluded face image sample I o and the first unobstructed face image sample I are respectively input to the encoder module, and the first obstructed face image sample I output by the encoder module is obtained. o The third eigenvector sample w3 corresponding to the first unobstructed face image sample I and the fourth eigenvector sample w4 corresponding to the first unobstructed face image sample I are injected layer by layer into the face synthesis network of the initial generation module, and fused with the noise feature maps output by each layer of the face synthesis network. Figure 3 A represents the fusion process, and N represents the introduced Gaussian noise.

[0155] Finally, the face synthesis network can output the first face image I o ' and the second face image I'. Then, based on the auxiliary face key point information and the second face key point label, the key point consistency loss L can be calculated ldmk , through I and I o ′, the first image reconstruction loss L can be calculated recon , through I and I′, the second image reconstruction loss L′ can be calculated recon Combined with the auxiliary discriminant module, the first feature matching loss and the second feature matching loss can be calculated. Combining these losses, the generation module can be trained. At this point, the face deocclusion model is obtained.

[0156] On the basis of the above embodiments, in the embodiment of the present invention, the facial key point location method provided, the key point location model is trained based on the following method:

[0157] Inputting the second unobstructed facial image sample into an initial key point positioning model, obtaining a first Gaussian response map centered on the key point corresponding to the second unobstructed facial image sample output by the initial key point positioning model, and determining a second Gaussian response map corresponding to the first facial key point label;

[0158] Calculating a key point positioning loss of the initial key point positioning model based on the first Gaussian response map and the second Gaussian response map;

[0159] Based on the key point positioning loss, the initial key point positioning model is trained to obtain a key point positioning model.

[0160] Specifically, in an embodiment of the present invention, when training the initial key point positioning model, the second unobstructed facial image sample can be first input into the initial key point positioning model to obtain the first Gaussian response map centered on the key point corresponding to the second unobstructed facial image sample output by the initial key point positioning model, and the second Gaussian response map corresponding to the first facial key point label can be determined.

[0161] Then, the key point positioning loss of the initial key point positioning model is calculated through the first Gaussian response map and the second Gaussian response map, that is:

[0162]

[0163] Among them, L heat represents the key point positioning loss, H(I) represents the first Gaussian response map, Represents a second Gaussian response plot.

[0164] It can be understood that the key point positioning loss is the mean square error to ensure the accuracy of the key point positioning position.

[0165] In summary, embodiments of the present invention propose a method for locating facial key points in complex scenes that is resistant to occlusion. This method utilizes a face deocclusion method based on three-dimensional rendering and style adversarial generative network priors to deocclude occluded facial images, effectively reducing the positioning ambiguity of the key point positioning model. When the original facial image is occluded, the facial features of the occluded area are restored by introducing prior information from the face synthesis module while maintaining the facial structure of the visible area. Ultimately, a high-definition, unobstructed facial image is generated for subsequent facial key point positioning by the facial key point positioning model, thereby improving the robustness of the positioning algorithm.

[0166] like Figure 4 As shown, based on the above embodiment, an embodiment of the present invention provides a facial key point positioning device, including:

[0167] An image acquisition module 41 is used to acquire an original face image for key point location;

[0168] a deocclusion module 42 for inputting the original face image into a face deocclusion model to obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model;

[0169] A key point positioning module 43 is configured to input the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model;

[0170] Among them, the face deocclusion model is trained based on the first unobstructed face image sample and the corresponding first occluded face image sample, and the key point positioning model is trained based on the second unobstructed face image sample carrying the first face key point label.

[0171] On the basis of the above embodiment, the facial key point locating device provided in the embodiment of the present invention further includes a sample generating module for:

[0172] Determine first three-dimensional facial information of the first unobstructed facial image sample, and determine a first texture map of a facial region in the first unobstructed facial image sample based on the first three-dimensional facial information;

[0173] Obtaining a second occluded face image sample, determining second three-dimensional facial information of the second occluded face image sample, and determining a second texture map of an occluded area in the second occluded face image sample based on the second three-dimensional facial information;

[0174] The first texture map and the second texture map are fused to obtain a first fusion result, and the first occluded face image sample is determined based on the first three-dimensional face information and the first fusion result.

[0175] On the basis of the above embodiments, in the facial key point positioning device provided in the embodiments of the present invention, the de-occlusion module is specifically configured to:

[0176] Inputting the original face image into the encoder module of the face deocclusion model, and having the encoder module perform feature extraction on the original face image to obtain a feature vector of the original face image output by the encoder module;

[0177] The feature vector is input into the generation module of the face de-occlusion model, and the generation module performs image reconstruction on the feature vector to obtain the unobstructed face image output by the generation module.

[0178] On the basis of the above embodiment, the facial key point locating device provided in the embodiment of the present invention further includes an encoder training module for:

[0179] Inputting the first unobstructed face image sample and the first obstructed face image sample into an initial encoder module respectively, and obtaining a first feature vector sample corresponding to the first unobstructed face image sample and a second feature vector sample corresponding to the first obstructed face image sample output by the initial encoder module;

[0180] Based on the first feature vector sample and the second feature vector sample, a feature loss of the initial encoder module is calculated, and based on the feature loss, parameters of the initial encoder module are iterated to obtain the encoder module.

[0181] Based on the above embodiment, in the facial key point locating device provided in the embodiment of the present invention, the first occluded face image sample carries a second facial key point label, and the second facial key point label is determined based on the facial key point label carried by the first unoccluded face image sample;

[0182] The apparatus further comprises a generating training module, configured to:

[0183] Inputting the third feature vector sample corresponding to the first occluded face image sample into an initial generation module to obtain a first face image corresponding to the third feature vector sample output by the initial generation module;

[0184] Inputting the first facial image into an auxiliary key point positioning module to obtain auxiliary facial key point information corresponding to the first facial image output by the auxiliary key point positioning module;

[0185] Calculating the key point consistency loss of the initial generation module based on the auxiliary facial key point information and the second facial key point label;

[0186] Based on the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

[0187] On the basis of the above embodiments, in the facial key point locating device provided in the embodiments of the present invention, the generating training module is specifically used for:

[0188] Inputting the fourth feature vector sample corresponding to the first unobstructed face image sample into an initial generation module to obtain a second face image corresponding to the fourth feature vector sample output by the initial generation module;

[0189] Calculating a first image reconstruction loss of the initial generation module based on the first face image and the first unobstructed face image sample, and calculating a second image reconstruction loss of the initial generation module based on the second face image and the first unobstructed face image sample;

[0190] Inputting the first facial image, the second facial image, and the first unobstructed facial image sample into an auxiliary discrimination module, respectively, to obtain a first feature map corresponding to the first facial image, a second feature map corresponding to the second facial image, and a third feature map corresponding to the first unobstructed facial image sample extracted by the auxiliary discrimination module;

[0191] Calculating a first feature matching loss of the initial generation module based on the first feature map and the third feature map, and calculating a second feature matching loss of the initial generation module based on the second feature map and the third feature map;

[0192] Based on the first image reconstruction loss, the second image reconstruction loss, the first feature matching loss, the second feature matching loss and the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

[0193] On the basis of the above embodiment, the facial key point locating device provided in the embodiment of the present invention further includes a key point locating model training module for:

[0194] Inputting the second unobstructed facial image sample into an initial key point positioning model, obtaining a first Gaussian response map centered on the key point corresponding to the second unobstructed facial image sample output by the initial key point positioning model, and determining a second Gaussian response map corresponding to the first facial key point label;

[0195] Calculating a key point positioning loss of the initial key point positioning model based on the first Gaussian response map and the second Gaussian response map;

[0196] Based on the key point positioning loss, parameters of the initial key point positioning model are iterated to obtain a key point positioning model.

[0197] Specifically, the functions of each module in the facial key point locating device provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.

[0198] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the facial key point location method provided in the above embodiments, which includes: obtaining an original facial image to be key point located; inputting the original facial image into a face deocclusion model to obtain an unobstructed facial image corresponding to the original facial image output by the face deocclusion model; inputting the unobstructed facial image into a key point location model to obtain facial key point information in the unobstructed facial image output by the key point location model; wherein the face deocclusion model is trained based on a first unobstructed facial image sample and a corresponding first occluded facial image sample, and the key point location model is trained based on a second unobstructed facial image sample carrying a first facial key point label.

[0199] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0200] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the facial key point positioning method provided in the above embodiments, the method including: obtaining an original facial image to be key-point positioned; inputting the original facial image into a facial deocclusion model to obtain an unobstructed facial image corresponding to the original facial image output by the facial deocclusion model; inputting the unobstructed facial image into a key point positioning model to obtain facial key point information in the unobstructed facial image output by the key point positioning model; wherein, the facial deocclusion model is trained based on a first unobstructed facial image sample and a corresponding first occluded facial image sample, and the key point positioning model is trained based on a second unobstructed facial image sample carrying a first facial key point label.

[0201] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the facial key point positioning method provided in the above-mentioned embodiments, the method comprising: obtaining an original facial image to be key-point positioned; inputting the original facial image into a facial deocclusion model to obtain an unobstructed facial image corresponding to the original facial image output by the facial deocclusion model; inputting the unobstructed facial image into a key point positioning model to obtain facial key point information in the unobstructed facial image output by the key point positioning model; wherein the facial deocclusion model is trained based on a first unobstructed facial image sample and a corresponding first occluded facial image sample, and the key point positioning model is trained based on a second unobstructed facial image sample carrying a first facial key point label.

[0202] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0203] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A facial key point location method, characterized in that: include: Obtain the original face image for key point positioning; Inputting the original face image into a face deocclusion model to obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model; Inputting the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model; The face deocclusion model is trained based on a first unobstructed face image sample and a corresponding first occluded face image sample, and the key point positioning model is trained based on a second unobstructed face image sample carrying a first face key point label; The first occluded face image sample is determined based on the following method: Determine first three-dimensional facial information of the first unobstructed facial image sample, and determine a first texture map of a facial region in the first unobstructed facial image sample based on the first three-dimensional facial information; Obtaining a second occluded face image sample, determining second three-dimensional facial information of the second occluded face image sample, and determining a second texture map of an occluded area in the second occluded face image sample based on the second three-dimensional facial information; The first texture map and the second texture map are fused to obtain a first fusion result, and the first occluded face image sample is determined based on the first three-dimensional face information and the first fusion result.

2. The facial key point positioning method according to claim 1, characterized in that: Inputting the original face image into a face deocclusion model to obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model includes: Inputting the original face image into the encoder module of the face deocclusion model, and having the encoder module perform feature extraction on the original face image to obtain a feature vector of the original face image output by the encoder module; The feature vector is input into the generation module of the face de-occlusion model, and the generation module performs image reconstruction on the feature vector to obtain the unobstructed face image output by the generation module.

3. The facial key point positioning method according to claim 2, characterized in that: The encoder module is trained based on the following method: Inputting the first unobstructed face image sample and the first obstructed face image sample into an initial encoder module respectively, and obtaining a first feature vector sample corresponding to the first unobstructed face image sample and a second feature vector sample corresponding to the first obstructed face image sample output by the initial encoder module; Based on the first feature vector sample and the second feature vector sample, a feature loss of the initial encoder module is calculated, and based on the feature loss, parameter iteration is performed on the initial encoder module to obtain the encoder module.

4. The facial key point positioning method according to claim 2, characterized in that: The first occluded face image sample carries a second face key point label, and the second face key point label is determined based on the face key point label carried by the first unoccluded face image sample; The generation module is trained based on the following method: Inputting the third feature vector sample corresponding to the first occluded face image sample into an initial generation module to obtain a first face image corresponding to the third feature vector sample output by the initial generation module; Inputting the first facial image into an auxiliary key point positioning module to obtain auxiliary facial key point information corresponding to the first facial image output by the auxiliary key point positioning module; Calculating the key point consistency loss of the initial generation module based on the auxiliary facial key point information and the second facial key point label; Based on the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

5. The facial key point positioning method according to claim 4, characterized in that: The step of performing parameter iteration on the initial generation module based on the key point consistency loss to obtain the generation module specifically includes: Inputting the fourth feature vector sample corresponding to the first unobstructed face image sample into an initial generation module to obtain a second face image corresponding to the fourth feature vector sample output by the initial generation module; Calculating a first image reconstruction loss of the initial generation module based on the first face image and the first unobstructed face image sample, and calculating a second image reconstruction loss of the initial generation module based on the second face image and the first unobstructed face image sample; Inputting the first facial image, the second facial image, and the first unobstructed facial image sample into an auxiliary discrimination module, respectively, to obtain a first feature map corresponding to the first facial image, a second feature map corresponding to the second facial image, and a third feature map corresponding to the first unobstructed facial image sample extracted by the auxiliary discrimination module; Calculating a first feature matching loss of the initial generation module based on the first feature map and the third feature map, and calculating a second feature matching loss of the initial generation module based on the second feature map and the third feature map; Based on the first image reconstruction loss, the second image reconstruction loss, the first feature matching loss, the second feature matching loss and the key point consistency loss, parameter iteration is performed on the initial generation module to obtain the generation module.

6. The method for locating facial key points according to any one of claims 1 to 5, characterized in that: The key point positioning model is trained based on the following method: Inputting the second unobstructed facial image sample into an initial key point positioning model, obtaining a first Gaussian response map centered on the key point corresponding to the second unobstructed facial image sample output by the initial key point positioning model, and determining a second Gaussian response map corresponding to the first facial key point label; Calculating a key point positioning loss of the initial key point positioning model based on the first Gaussian response map and the second Gaussian response map; Based on the key point positioning loss, parameters of the initial key point positioning model are iterated to obtain a key point positioning model.

7. A facial key point positioning device, characterized in that: include: An image acquisition module is used to acquire the original face image for key point positioning; a deocclusion module, configured to input the original face image into a face deocclusion model, and obtain an unobstructed face image corresponding to the original face image output by the face deocclusion model; A key point positioning module is used to input the unobstructed face image into a key point positioning model to obtain facial key point information in the unobstructed face image output by the key point positioning model; The face deocclusion model is trained based on a first unobstructed face image sample and a corresponding first occluded face image sample, and the key point positioning model is trained based on a second unobstructed face image sample carrying a first face key point label; The first occluded face image sample is determined based on the following method: Determine first three-dimensional facial information of the first unobstructed facial image sample, and determine a first texture map of a facial region in the first unobstructed facial image sample based on the first three-dimensional facial information; Obtaining a second occluded face image sample, determining second three-dimensional facial information of the second occluded face image sample, and determining a second texture map of an occluded area in the second occluded face image sample based on the second three-dimensional facial information; The first texture map and the second texture map are fused to obtain a first fusion result, and the first occluded face image sample is determined based on the first three-dimensional face information and the first fusion result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for locating facial key points according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for locating facial key points according to any one of claims 1 to 6 is implemented.

Images