A face key point detection method, device and equipment

Abstract

A face key point detection method, device and equipment and a storage medium are disclosed, which include: obtaining a face image data set, the face image data set including unoccluded face images; constructing an adaptive detection model, the adaptive detection model including a first network branch and a second network branch; training the first network branch with the face image data set as first training data, and training the second network branch with the unoccluded face images and pseudo-occluded face images as second training data; and detecting face key points in an occluded face image through the trained adaptive detection model. Stable and accurate face point prediction can be achieved, and the sensitivity to occlusion can be reduced in the case of face occlusion, and the positioning accuracy of face key points in the occluded area can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a face key point detection method, device and equipment. BACKGROUND

[0002] Artificial intelligence products based on deep learning have become an essential part of daily life, among which face recognition intelligent products are widely used. Face key point detection is an essential technology in face recognition systems, but in actual application, diversified face occlusion will affect the integrity of face information, greatly increasing the difficulty of accurate positioning of face key points, which is one of the difficult problems to be solved in face key point detection tasks.

[0003] Existing face key point detection methods mainly include two types of methods: key point regression and heat map regression. The key point regression method mainly uses a model to directly learn the mapping relationship between face features and face key point coordinates. This method usually uses a fully connected layer in the model to directly predict key point coordinates, focusing on the shape correlation between different face points, while ignoring the spatial correlation in face features. The heat map regression method mainly uses a coding-decoding structure to directly regress the coordinate heat map of the face point, and directly predicts the spatial position of the face point based on spatial information, which is a relatively advanced method type in the current academic field. However, for industrial application landing, the model calculation amount and processing speed of the heat map regression method are slightly inferior. Both of the above two methods, whether based on shape correlation or spatial correlation, are easily affected by the prediction accuracy of face points when dealing with occlusion problems. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a face key point detection method, device and equipment, which aims to solve the problem of low prediction accuracy of face points when existing face point detection algorithms deal with occlusion.

[0005] To achieve the above purpose, the present application provides a face key point detection method, which comprises:

[0006] obtaining a face image data set, wherein the face image data set comprises unoccluded face images;

[0007] constructing an adaptive detection model, wherein the adaptive detection model comprises a first network branch and a second network branch;

[0008] training the first network branch using the face image data set as first training data, and training the second network branch using the unoccluded face images and pseudo-occluded face images as second training data, wherein the pseudo-occluded face images are obtained by occlusion gain on each unoccluded face image.

[0009] detecting the facial key points in the occluded face image through the trained adaptive detection model.

[0010] Preferably, the first network branch comprises a first face feature extraction module, a key point detection module and a de-occlusion module; the training of the first network branch with the face image dataset as the first training data comprises:

[0011] extracting features of the input face image dataset through the first face feature extraction module to obtain a face feature map;

[0012] inputting the face feature map into the key point detection module and the de-occlusion module respectively to obtain a face key point prediction result.

[0013] Preferably, the structure of the de-occlusion module comprises two convolution blocks and a CBAM module; the structure of the key point detection module comprises four convolution blocks and a fully connected output layer.

[0014] Preferably, the training of the first network branch with the face image dataset as the first training data further comprises:

[0015] performing weighted calculation on the intermediate feature map obtained by the key point detection module through the CBAM module to assist the key point detection module in focusing on the prediction learning of the face key points in the un-occluded area. Preferably, the second network branch comprises a second face feature extraction module, a reconstruction module and a discrimination module, the discrimination module comprises an image discriminator and a feature discriminator; the training of the second network branch with the un-occluded face image and the pseudo-occluded face image as the second training data comprises:

[0016] inputting the un-occluded face image and the pseudo-occluded face image into the second face feature extraction module and the reconstruction module to obtain a first reconstruction feature map and a second reconstruction feature map respectively, and a reconstruction image corresponding to the pseudo-occluded face image;

[0017] inputting the first reconstruction feature map and the second reconstruction feature map into the feature discriminator for feature discrimination, and inputting the reconstruction image and the un-occluded face image into the image discriminator for image discrimination.

[0018] Preferably, the training of the second network branch with the un-occluded face image and the pseudo-occluded face image as the second training data further comprises:

[0019] optimizing the feature discriminator by minimizing the discrimination loss through a feature discrimination loss formula, the feature discrimination loss formula is

[0020] The image discriminator is optimized by minimizing the discriminative loss through an image discriminative loss formula, which is where p nc represents the first reconstructed feature map, represents the second reconstructed feature map, s nc represents the unoccluded face image, represents the reconstructed image, D f represents the feature discriminator, D g represents the image discriminator.

[0021] Preferably, the method further comprises:

[0022] The reconstruction module and the second face feature extraction module are optimized by minimizing the supervision loss through a supervision loss formula, which is

[0023] To achieve the above object, the present application further provides a face key point detection device, which comprises: an acquisition unit configured to acquire a face image dataset, wherein the face image dataset comprises unoccluded face images;

[0024] a construction unit configured to construct an adaptive detection model, wherein the adaptive detection model comprises a first network branch and a second network branch;

[0025] a training unit configured to train the first network branch by taking the face image dataset as first training data, and train the second network branch by taking the unoccluded face images and pseudo-occluded face images as second training data, wherein the pseudo-occluded face images are obtained by performing occlusion gain on each unoccluded face image;

[0026] a detection unit configured to detect face key points in an occluded face image by using the trained adaptive detection model.

[0027] To achieve the above object, the present application further provides a device, which comprises a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a face key point detection method as described in the above embodiments.

[0028] To achieve the above object, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the face key point detection method according to the above embodiment.

[0029] Beneficial effects:

[0030] The above scheme can realize stable and accurate face point prediction in actual application, reduce the sensitivity to occlusion in various face occlusion situations in daily life, and improve the face key point positioning accuracy in the occluded area.

[0031] The above scheme introduces a de-occlusion module and a reconstruction-discrimination module into the constructed adaptive detection model, which can improve the face key point prediction accuracy in the occluded face image. On the one hand, the de-occlusion module based on the full-quantity face image dataset reduces the negative influence of occlusion ambiguity information on the key point detection module, and improves the occlusion robustness of the model; on the other hand, the reconstruction-discrimination module introduces an image discriminator and a feature discriminator, which guide the network to learn as much effective information as possible in the occluded area of the occluded face image from the image and feature levels, so as to better assist the accurate prediction of the occluded face key points. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0033] Figure 1 A flowchart of a face key point detection method provided by an embodiment of the present application is shown.

[0034] Figure 2 A specific flowchart of a face key point detection method provided by an embodiment of the present application is shown.

[0035] Figure 3 A network structure diagram of an adaptive detection model provided by an embodiment of the present application is shown.

[0036] Figure 4 A calculation flowchart of a CBAM attention module provided by an embodiment of the present application is shown.

[0037] Figure 5 A structure diagram of a face key point detection device provided by an embodiment of the present application is shown.

[0038] Among them, about Figure 3The description of the drawings is as follows:

[0039] Represents a convolution operation with a 3*3*channel number and a step size of 1. Represents a downsampling operation with a downsampling ratio of 2. Represents a sigmoid function calculation. Represents a dot product calculation. Represents a fully connected layer. Represents a network structure of the discriminator (wherein each convolutional block is connected by a downsampling layer in the middle).

[0040] The purposes, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the application. Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the application.

[0042] In the description of the application, the terms “first” and “second” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more of the features.

[0043] The content of the application will be described in detail below in combination with embodiments.

[0044] Referring to Figure 1 Fig. 1 shows a flowchart of a face key point detection method provided by an embodiment of the application.

[0045] In this embodiment, the method comprises:

[0046] S11, acquiring a face image dataset, wherein the face image dataset comprises unoccluded face images.

[0047] In the embodiment, the obtained face image dataset is cropped, specifically, the face images in the face image dataset are cropped according to the corresponding face frames and reduced to a uniform size HxW, and the dataset is denoted as S. Unoccluded face images are selected from S, and the unoccluded face image dataset is denoted as S nc .

[0048] S12, an adaptive detection model is constructed, the adaptive detection model comprising a first network branch and a second network branch.

[0049] S13, the first network branch is trained by taking the face image dataset as first training data, and the second network branch is trained by taking the unoccluded face images and pseudo-occluded face images as second training data, wherein the pseudo-occluded face images are obtained by occlusion gain on each unoccluded face image.

[0050] S14, the face key points in the occluded face image are detected by the trained adaptive detection model.

[0051] The first network branch comprises a first face feature extraction module, a key point detection module, and a de-occlusion module.

[0052] In step S13, the first network branch is trained by taking the face image dataset as first training data, comprising:

[0053] S13-01, the face image dataset is input into the first face feature extraction module for feature extraction to obtain a face feature map;

[0054] S13-02, the face feature map is input into the key point detection module and the de-occlusion module respectively to obtain a face key point prediction result.

[0055] Further, in step S13, the first network branch is trained by taking the face image dataset as first training data, further comprising:

[0056] S13-03, the intermediate feature map obtained by the key point detection module is weighted and calculated by the CBAM module to assist the key point detection module in focusing on the prediction learning of unoccluded area face key points.

[0057] Specifically, the CBAM module is introduced into the de-occlusion module to assist the key point detection module in learning, the intermediate feature map obtained by the key point detection module is weighted and calculated, so that the key point detection module focuses on the prediction learning of unoccluded area face key points.

[0058] Referring toFigure 2 As shown in FIG. 6, by inputting all face images, the face key point detection is guided based on the de-occlusion module. Specifically, the face image dataset S after the cropping processing is input into the face feature extraction module FM to obtain the face feature map f. The face feature extraction module includes 5 3x3xc convolution blocks and 4 average pooling layers. Referring to FIG. 7, after the HxW data is input into the face feature extraction module, the face feature map f of the size of H / 16xW / 16 is obtained. Figure 3

[0059] Further, the face feature map f is input into the key point detection module PM and the de-occlusion module DOM respectively to obtain the face key point prediction result. The structure of the de-occlusion module DOM includes 2 3x3xc convolution blocks and a CBAM attention mechanism module. The CBAM attention module combines the channel attention mechanism and the spatial attention mechanism. The specific calculation process is as shown in FIG. 8. Figure 4 As shown in FIG. 8, the feature map input into the CBAM module is denoted as f', and the feature map output from the CBAM module is denoted as f". The calculation formula of the CBAM module is as shown below.

[0060]

[0061] wherein, represents the multiplication of the corresponding dimension elements, M c represents the channel attention mechanism, M s represents the spatial attention mechanism. M c , M s The calculation formula of the two parts is as follows, and F generally refers to the feature map.

[0062] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F)));

[0063] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]));

[0064] wherein, MLP represents a shared two-layer perceptron structure, AvgPool and MaxPool represent the average pooling operation and the maximum pooling operation in the corresponding channel dimension and spatial dimension respectively, and σ represents the sigmoid function calculation.

[0065] The de-occlusion module DOM makes the network features focus on the effective unoccluded area and ignore the occluded area through the attention mechanism calculation. The feature map output from the de-occlusion module DOM is denoted as f DOM . f DOM is normalized to the interval of 0-1 through the sigmoid function.​

[0066] The key point detection module PM includes four 3x3xc convolution blocks and a fully connected output layer. Denote the feature map output by the first two convolution modules of the key point detection module as f PM2 . Dot product of the above f DOM and f PM2 is performed to increase the learning weight of the unoccluded area and reduce the learning weight of the occluded area. Further, f PM2 is input into the subsequent convolution blocks, the human face key point prediction result is obtained through the output layer, and the Euclidean distance loss calculation is performed between the human face key point prediction result and the preset human face point label (the human face point label is obtained by manually labeling the human face points corresponding to the human face image data set) to update the weight parameters of the first network branch.

[0067] The second network branch includes a second human face feature extraction module, a reconstruction module, and a discrimination module, and the discrimination module includes an image discriminator and a feature discriminator.

[0068] During the training process, the first human face feature extraction module and the second human face feature extraction module share weight parameters.

[0069] In step S13, the unoccluded human face image and the pseudo-occluded human face image are used as the second training data to train the second network branch, including:

[0070] S13-11, the unoccluded human face image and the pseudo-occluded human face image are input into the second human face feature extraction module and the reconstruction module to obtain a first reconstructed feature map and a second reconstructed feature map, and a reconstructed image corresponding to the pseudo-occluded human face image, respectively.

[0071] S13-12, the first reconstructed feature map and the second reconstructed feature map are input into the feature discriminator for feature discrimination, and the reconstructed image and the unoccluded human face image are input into the image discriminator for image discrimination.

[0072] Further, in step S13, the unoccluded human face image and the pseudo-occluded human face image are used as the second training data to train the second network branch, further including:

[0073] S13-13, the feature discriminator is optimized by minimizing the discrimination loss through a feature discrimination loss formula, and the feature discrimination loss formula is

[0074] S13-14, the image discriminator is optimized by minimizing the discrimination loss through an image discrimination loss formula, and the image discrimination loss formula is Where, p nc The first reconstructed feature map, The second reconstructed feature map, s nc This refers to the unobstructed face image. The reconstructed image, D f The feature discriminator, D g This refers to the image discriminator.

[0075] In this embodiment, the dataset S of unobstructed face images nc Each image in the dataset is subjected to occlusion gain, which involves overlaying a non-face image onto any region of an unoccluded face image to obtain a set of pseudo-occluded face image data. Each unoccluded face image has a corresponding pseudo-occluded face image, and their pairing relationship is denoted as follows:

[0076] Reference Figure 2 As shown, by inputting an unoccluded face image and its corresponding matched pseudo-occluded face image, a face discrimination feature extraction is guided by a reconstruction-discrimination module to learn the distribution of face features and face information in the occluded face region. Specifically, the unoccluded face image data s nc and pseudo-occlusion face image data The reconstructed features p are obtained from the face feature extraction module and the reconstruction module, respectively. nc and reconstructed features and images with pseudo-occluded faces Corresponding reconstructed image Reference Figure 3 As shown, the reconstructed feature p nc and reconstructed features With 16 channels, reconstruct the image. The number of channels is 3. Further, the reconstructed feature p... nc and reconstructed features Input Feature Discrimination Module D f Feature discrimination is performed to identify false occlusion images. Reconstructed image and the corresponding unobstructed image s nc Input image discrimination module D g Image discrimination is performed.

[0077] Furthermore, for the feature discrimination module D f The optimization steps are as follows: Let label 1 represent the reconstructed feature p nc Label 0 indicates reconstructed features To minimize the discrimination loss of the feature discrimination module, the feature discrimination module can distinguish the reconstructed features p from the undisturbed image and the occluded image.nc and The feature discrimination loss calculation formula is as follows:

[0078]

[0079] Similarly, the image discrimination module D g The optimization steps are as follows: let label 1 represent the unoccluded face image s nc , and label 0 represent the reconstructed image of the pseudo-occluded face image Minimize the discrimination loss of the image discrimination module, so that the image discrimination module distinguishes the image s nc and The image discrimination loss calculation formula is as follows:

[0080]

[0081] In addition, after updating the parameters of the above image and feature discrimination modules, the parameters of the reconstruction module and the face feature extraction module are updated by minimizing the following supervised loss. Not only does the reconstructed feature of the pseudo-occluded face image as much as possible consistent with the reconstructed feature p nc of the unoccluded face image, but also constrains the reconstructed image of the pseudo-occluded face image as much as possible consistent with the unoccluded face image. The supervised loss is shown in the following formula (where R refers to the reconstruction module and the face feature extraction module):

[0082]

[0083] Based on the updated network parameters, the training of the first network branch is repeated: all face images S are input into the first face feature extraction module, and the network parameters are updated based on the key point prediction of the de-occlusion module. Further, the training of the second network branch is repeated: the unoccluded face image and the corresponding matched pseudo-occluded face image are input into the second face feature extraction module, and the reconstruction-discrimination module guided network learns as much as possible the discrimination feature distribution of the occluded region face and decodes the face information of the occluded region. Repeat the iteration update until the network training loss converges.

[0084] Referring to Figure 5 Fig. 1 shows a structure schematic diagram of a face key point detection device provided by an embodiment of the present application.

[0085] In the embodiment, the device 50 comprises:

[0086] An acquisition unit 51 is configured to acquire a face image dataset, wherein the face image dataset comprises unoccluded face images.

[0087] The constructing unit 52 is configured to construct an adaptive detection model, wherein the adaptive detection model comprises a first network branch and a second network branch.

[0088] The training unit 53 is configured to train the first network branch by taking the face image dataset as first training data, and train the second network branch by taking the unoccluded face image and the pseudo-occluded face image as second training data, wherein the pseudo-occluded face image is obtained by performing occlusion gain on each unoccluded face image.

[0089] The detection unit 54 is configured to detect the face key points in the occluded face image by the trained adaptive detection model.

[0090] Further, the first network branch comprises a first face feature extraction module, a key point detection module and a de-occlusion module; and the training unit 53 comprises:

[0091] The first input unit is configured to perform feature extraction on the input face image dataset by the first face feature extraction module to obtain a face feature map;

[0092] The key point prediction unit is configured to input the face feature map into the key point detection module and the de-occlusion module respectively to obtain a face key point prediction result;

[0093] The first calculation unit is configured to perform weighted calculation on the intermediate feature map obtained by the key point detection module by the CBAM module to assist the key point detection module in focusing on the prediction learning of the face key points in the unoccluded area.

[0094] Further, the second network branch comprises a second face feature extraction module, a reconstruction module and a discrimination module, wherein the discrimination module comprises an image discriminator and a feature discriminator; and the training unit 53 further comprises:

[0095] The second input unit is configured to input the unoccluded face image and the pseudo-occluded face image into the second face feature extraction module and the reconstruction module to obtain a first reconstruction feature map and a second reconstruction feature map, and a reconstruction image corresponding to the pseudo-occluded face image respectively;

[0096] The discrimination unit is configured to input the first reconstruction feature map and the second reconstruction feature map into the feature discriminator for feature discrimination, and input the reconstruction image and the unoccluded face image into the image discriminator for image discrimination;

[0097] The second loss calculation unit is configured to optimize the feature discriminator by minimizing the discrimination loss through a feature discrimination loss formula, wherein the feature discrimination loss formula is

[0098] a third loss calculation unit configured to minimize a discrimination loss of the image discriminator by an image discrimination loss formula, the image discrimination loss formula being wherein p nc represents the first reconstructed feature map, represents the second reconstructed feature map, s nc represents the unoccluded face image, represents the reconstructed image, D f represents the feature discriminator, D g represents the image discriminator.

[0099] Further, the device further comprises:

[0100] a fourth loss calculation unit configured to minimize a supervision loss of the reconstruction module and the second face feature extraction module by a supervision loss formula, the supervision loss formula being

[0101] The various unit modules of the device 50 can respectively perform the corresponding steps in the above method embodiments, so the unit modules will not be described here. For details, please refer to the above description of the corresponding steps.

[0102] The embodiments of the present application also provide a device, which comprises the face key point detection device as described above, wherein the face key point detection device can adopt Figure 5 The structure of the embodiments, which can correspondingly perform Figure 1 The technical solutions of the method embodiments shown in the above description, the implementation principles and technical effects are similar, and details can be referred to the above description of the embodiments, which will not be described here.

[0103] The device comprises a mobile phone, a digital camera, a tablet computer or other devices with a photographing function, or a device with an image processing function, or a device with an image display function. The device can include a memory, a processor, an input unit, a display unit, a power supply and other components.

[0104] The memory can be configured to store software programs and modules, and the processor can execute various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access to the memory for the processor and the input unit.

[0105] The input unit can be configured to receive inputted digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. Specifically, the input unit of the embodiment can include a camera, in addition to a touch-sensitive surface (such as a touch display screen) and other input devices.

[0106] The display unit can be configured to display information inputted by a user or provided to a user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, video and any combination thereof. The display unit can include a display panel, which can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface can cover the display panel, and when the touch-sensitive surface detects a touch operation thereon or in the vicinity thereof, transmits to the processor to determine the type of the touch event, and then the processor provides corresponding visual output on the display panel according to the type of the touch event.

[0107] The embodiment of the present application also provides a computer readable storage medium, which can be a computer readable storage medium contained in the memory in the above-mentioned embodiments; or can exist separately and not be assembled into the device. The computer readable storage medium stores at least one instruction, which is loaded and executed by the processor to implement Figure 1 The face key point detection method shown. The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0108] It should be noted that each of the embodiments described in this specification has a corresponding counterpart embodiment not mentioned explicitly. Therefore, each of the embodiments described in this specification should not be taken as excluding the other. For example, the method embodiments can be implemented by a device or apparatus, and the device or apparatus embodiments can be implemented by a method. Therefore, the device or apparatus embodiments are not mutually exclusive with the method embodiments.

[0109] Also, as used in the description of the embodiments of the application and the appended claims, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component" or "the component" can include a plurality of such components unless the context clearly dictates otherwise.

[0110] The above specification, examples and data provide essential information for constructing and implementing known expedients. The description suffices for implementing by those of ordinary skill in the art. The present application is not limited to the details of the foregoing, since the present application can be implemented in various other combinations, modifications, and environments, and is capable of changes or modifications within the scope of the inventive concept as expressed herein, through the application of the teachings or knowledge of the art. Changes and modifications can be made to the application in its various aspects by those of ordinary skill in the art without departing from the spirit or scope of the present application. The application is not limited to the embodiments described above, but can be implemented in various other combinations, modifications, and environments, and is capable of changes or modifications within the scope of the inventive concept as expressed herein, through the application of the teachings or knowledge of the art. Changes and modifications made to the application in its various aspects by those of ordinary skill in the art without departing from the spirit or scope of the present application are to be deemed within the scope of the present application as set forth in the following claims.

Claims

1. A face key point detection method, characterized in that, The method comprises: obtaining a face image dataset, the face image dataset comprising unoccluded face images; constructing an adaptive detection model, the adaptive detection model comprising a first network branch and a second network branch; wherein the first network branch comprises a first face feature extraction module, a key point detection module, and an unocclusion module; the structure of the unocclusion module comprises two convolution blocks and a CBAM module; the second network branch comprises a second face feature extraction module, a reconstruction module, and a discrimination module, the discrimination module comprising an image discriminator and a feature discriminator; training the first network branch using the face image dataset as first training data, and training the second network branch using the unoccluded face images and pseudo-occluded face images as second training data, wherein the pseudo-occluded face images are obtained by performing occlusion gain on each unoccluded face image; wherein the training of the first network branch using the face image dataset as first training data comprises: extracting features from the input face image dataset through the first face feature extraction module to obtain a face feature map; inputting the face feature map into the key point detection module and the unocclusion module respectively to obtain a face key point prediction result; performing weighted calculation on the intermediate feature map obtained by the key point detection module through the CBAM module to assist the key point detection module in focusing on the prediction learning of unoccluded area face key points; detecting face key points in an occluded face image through the trained adaptive detection model.

2. The method of claim 1, wherein, The structure of the key point detection module comprises four convolution blocks and a fully connected output layer.

3. The method of claim 1, wherein, The training of the second network branch using the unoccluded face images and pseudo-occluded face images as second training data comprises: inputting the unoccluded face images and the pseudo-occluded face images into the second face feature extraction module and the reconstruction module to obtain first and second reconstruction feature maps and a reconstructed image corresponding to the pseudo-occluded face image respectively; inputting the first and second reconstruction feature maps into the feature discriminator for feature discrimination, and inputting the reconstructed image and the unoccluded face image into the image discriminator for image discrimination.

4. The method of claim 3, wherein, The training of the second network branch using the unoccluded face images and pseudo-occluded face images as second training data further comprises: The feature discriminator is optimized by minimizing a feature discrimination loss through a feature discrimination loss formula, which is The image discriminator is optimized by minimizing the discriminative loss through an image discriminative loss formula, which is wherein p nc represents the first reconstructed feature map, represents the second reconstructed feature map, s nc represents the unoccluded face image, represents the reconstructed image, D f represents the feature discriminator, D g represents the image discriminator.

5. The method of claim 4, wherein, further comprising: The optimization of minimizing the supervision loss by a supervision loss formula on the reconstruction module and the second face feature extraction module, the supervision loss formula is 6. An apparatus for face key point detection, the apparatus comprising: The device comprises: an acquisition unit configured to acquire a face image dataset, the face image dataset comprising unoccluded face images; a construction unit configured to construct an adaptive detection model, the adaptive detection model comprising a first network branch and a second network branch; The training unit is configured to train the first network branch by using the face image dataset as first training data, and train the second network branch by using the unoccluded face image and the pseudo-occluded face image as second training data, wherein the pseudo-occluded face image is obtained by performing occlusion gain on each unoccluded face image; The first network branch includes a first face feature extraction module, a key point detection module, and a de-occlusion module; the structure of the de-occlusion module includes two convolution blocks and a CBAM module; the second network branch includes a second face feature extraction module, a reconstruction module, and a discrimination module, the discrimination module includes an image discriminator and a feature discriminator; the training unit includes: The first input unit is configured to perform feature extraction on the input face image dataset by using the first face feature extraction module to obtain a face feature map; The key point prediction unit is configured to input the face feature map into the key point detection module and the de-occlusion module respectively to obtain a face key point prediction result; The first calculation unit is configured to perform weighted calculation on the intermediate feature map obtained by the key point detection module by using the CBAM module to assist the key point detection module to focus on the prediction learning of the face key points in the unoccluded area; The detection unit is configured to detect the face key points in the occluded face image by using the trained adaptive detection model.

7. An apparatus, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the face key point detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the face key point detection method according to any one of claims 1 to 5.

Images