Key point occlusion identification method and device for 2D face

Abstract

One aspect of the present disclosure relates to a key point occlusion identification method and device for a 2D face. One aspect of the present disclosure relates to a training method for a key point occlusion identification model for a 2D face, comprising: inputting a training sample set into a key point positioning model based on a convolutional neural network for training, wherein the key point positioning model comprises a backbone network and a head network; freezing the parameters of the trained key point positioning model, and extracting output features of the backbone network; inputting the extracted output features into a key point occlusion identification model based on a convolutional neural network for training.

Description

Technical Field

[0001] This disclosure generally relates to the field of image algorithms. In particular, this disclosure relates to key point occlusion recognition of 2D faces. Background Technology

[0002] In AI-powered beauty and makeup photography scenarios, 2D facial landmark recognition technology is typically used to locate multiple key points on the face, and then makeup (lip gloss, colored contact lenses, eyebrows, etc.) is rendered according to these points. However, users often cover key facial areas such as the mouth and eyes with their hands or other objects, causing the makeup to be rendered on the obstruction. Furthermore, the location and recognition of key points become inaccurate when obscured, impacting the user experience. Therefore, it is necessary to identify obscured areas of the face and address this issue by removing makeup from obscured areas or increasing the transparency of makeup in obscured areas.

[0003] Existing occlusion recognition technologies typically employ large, complex neural networks (e.g., heatmaps). The larger and more complex the neural network, the more hardware resources are required for training and detection, often limiting their operation to high-performance servers. Mobile devices, constrained by hardware resources and processing power, struggle to run complex network models. Furthermore, beauty and makeup effects are usually applied to mobile devices and require real-time enhancements, making such large-model-based occlusion recognition methods unsuitable. Currently, no technology exists on the market specifically designed for real-time facial occlusion recognition on mobile devices. Summary of the Invention

[0004] The purpose of this invention is to provide a method suitable for recognizing occluded areas of 2D faces in mobile user devices. This method can identify which key points are occluded while locating key points, without introducing excessive power consumption.

[0005] To perform occlusion recognition without introducing excessive power consumption, this invention adds a branch to the head of the 2D keypoint localization model for keypoint occlusion recognition and adopts a model pruning scheme to remove some weight parameters without significantly affecting accuracy, thereby accelerating model operation.

[0006] According to one aspect of this disclosure, a training method for a keypoint occlusion recognition model for 2D faces is provided, comprising: inputting a training sample set into a keypoint localization model based on a convolutional neural network for training, wherein the keypoint localization model includes a backbone network and a head network; freezing the parameters of the trained keypoint localization model and extracting the output features of the backbone network; and inputting the extracted output features into the keypoint occlusion recognition model based on a convolutional neural network for training.

[0007] According to another aspect of this disclosure, a method for keypoint occlusion recognition of 2D faces is provided, comprising: inputting a 2D face image to be detected into the backbone network of a keypoint localization model; inputting the output features of the backbone network of the keypoint localization model into the head network of the keypoint localization model and a keypoint occlusion recognition model; and the keypoint localization model outputting keypoint localization detection results, and the keypoint occlusion recognition model outputting keypoint occlusion recognition results, wherein the keypoint localization model and the keypoint occlusion recognition model are trained using the training method described above.

[0008] According to another aspect of this disclosure, a training apparatus for a keypoint occlusion recognition model for 2D faces is provided, comprising: a memory storing instructions; and a processor configured to execute the instructions stored in the memory to perform the training method described above.

[0009] According to another aspect of this disclosure, a key point occlusion recognition device for 2D faces is provided, comprising: a memory storing instructions; and a processor configured to execute the instructions stored in the memory to perform the recognition method as described above. Attached Figure Description

[0010] Figure 1 The illustration shows an exemplary 2D facial landmark diagram according to an embodiment of the present disclosure.

[0011] Figure 2 The diagram illustrates a block diagram of a key point localization model and a key point occlusion recognition model according to an embodiment of the present disclosure.

[0012] Figure 3 The figure illustrates a model structure diagram of a key point localization model according to an embodiment of the present disclosure.

[0013] Figure 4 The illustration shows a model structure diagram with an added keypoint occlusion model according to an embodiment of the present disclosure.

[0014] Figure 5 The illustration shows a flowchart of a training method for a key point occlusion recognition model according to an embodiment of the present disclosure.

[0015] Figure 6 The illustration shows a schematic diagram of a key point localization model and a key point occlusion recognition model according to an embodiment of the present disclosure.

[0016] Figure 7 The illustration shows a flowchart of a key point occlusion recognition method according to an embodiment of the present disclosure.

[0017] Figure 8 The occluded key points identified according to one embodiment of this disclosure are presented.

[0018] Figure 9 An exemplary configuration of a training apparatus for a keypoint occlusion recognition model according to an embodiment of the present disclosure is illustrated.

[0019] In each figure, similar reference numerals refer to the same figure elements. Detailed Implementation

[0020] In intelligent facial beautification and makeup scenarios, 2D facial landmark localization models are typically used to identify different regions of the face. Facial landmark localization is a fundamental algorithm for face recognition, and its main task is to locate the facial contours and features. This algorithm can locate multiple key points on a 2D face image that reflect the features of various parts of the face, such as the tip of the nose, eyebrows, and corners of the mouth. Currently, the industry widely uses 106 key points for face detection. Figure 1 A facial image labeled with 106 key points is shown, each key point having a corresponding number. Those skilled in the art will understand that... Figure 1 The 106 key points shown are for illustrative purposes only. This invention is not limited to the detection of 106 key points; detection based on other numbers of key points is also applicable to the solutions of this invention.

[0021] 2D face landmark localization models are typically implemented based on convolutional neural networks, and include at least a backbone network and a head network. The backbone network is used to extract features, specifically information from the image. The head network is used to perform detection using the extracted features.

[0022] According to one embodiment of the present invention, occlusion recognition is achieved by adding the keypoint occlusion recognition model as a branch of the head network to the keypoint localization model. Figure 2 The diagram shows the block diagrams of the keypoint localization model and the keypoint occlusion recognition model. As can be seen from the diagram, after the backbone network, a keypoint occlusion recognition module, i.e., the keypoint localization module in the diagram, is added in parallel with the head network of the keypoint localization model. This keypoint occlusion recognition model can be understood as another head network branch parallel to the head network of the keypoint localization model, and they share the same backbone network.

[0023] According to one embodiment, the backbone network of the 2D face landmark localization model of the present invention can be implemented using MobileNet-V1. MobileNet-V1 is a lightweight neural network with smaller size, less computation, and higher accuracy, making it particularly suitable for operation on mobile devices. However, the present invention is not limited to using MobileNet-V1; other lightweight neural networks are also applicable.

[0024] Figure 3 An example model structure diagram of a keypoint localization model 300 based on MobileNet-V1 is shown. Model 300 includes multiple convolutional modules. For illustrative purposes, Figure 2 The diagram shows five convolutional modules: MobileBottleneck1, MobileBottleneck2, MobileBottleneck3, MobileBottleneck4, and MobileBottleneck5. MobileBottleneck1, MobileBottleneck2, and MobileBottleneck3 form the backbone network, while MobileBottleneck4 and MobileBottleneck5 form the head network. In one embodiment, MobileBottleneck3 has a 64-channel output. In another embodiment, MobileBottleneck3 has a 16×16×64 output feature number.

[0025] Each convolutional module includes a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The trained keypoint localization model 300 can detect the input face image and output the coordinates (x, y) of the keypoints for subsequent beautification and makeup operations.

[0026] Those skilled in the art should understand that Figure 3 The network structure shown is only an example; other neural networks with different structures can also be used to implement the key point localization model.

[0027] Figure 4 Figure 400 shows the model structure with an added keypoint occlusion recognition model branch. As shown, after the backbone network, a head network branch is added on the right side, parallel to the original head network. This head network branch is the keypoint occlusion recognition model used to identify keypoint occlusions.

[0028] As shown in the figure, the output features of convolutional module MobileBottleneck3 are simultaneously input into convolutional modules MobileBottleneck4 and MobileBottleneck4_2. After passing through their respective convolutional layers, global average pooling layers, and fully connected layers, the left branch outputs the position coordinates of each keypoint, while the right branch outputs the detection results indicating whether each keypoint is occluded.

[0029] Figure 4The network structure of the keypoint occlusion recognition model shown is the same as the head network structure of the keypoint localization model. For example, the keypoint occlusion model uses two convolutional modules with the same structure as the fourth convolutional module MobileBottleneck4 and the fifth convolutional module MobileBottleneck5 of the keypoint localization model. Figure 4 The examples shown are MobileBottleneck4_2 and MobileBottleneck5_2. However, the invention is not limited to this; the added keypoint occlusion recognition model can also employ a network structure different from the head network structure of the keypoint localization model.

[0030] Figure 5 The reference is shown Figure 2 The flowchart describes the training method for the keypoint occlusion recognition model. Since the keypoint occlusion recognition model is added as a branch of the head network to the original keypoint localization model, its training process will be coordinated with the training process of the keypoint localization model.

[0031] First, in step 501, the training sample set is input into a keypoint localization model based on a convolutional neural network for training, wherein the keypoint localization model includes a backbone network and a head network. After the keypoint localization model is trained, in step 502, the parameters of the trained keypoint localization model are frozen, and the output features of the backbone network are extracted. Then, in step 503, the extracted output features are input into a keypoint occlusion recognition model based on a convolutional neural network for training.

[0032] According to one embodiment, freezing the parameters of a trained keypoint localization model includes keeping the parameters of the keypoint localization model unchanged during the training of the keypoint occlusion recognition model. For example, the parameters of each layer in the backbone and head networks of the keypoint localization model can be fixed so that they are not updated during the training of the keypoint occlusion recognition model.

[0033] In other words, the learnable parameters of the keypoint localization model, including those of the backbone and head networks, are already trained before the keypoint occlusion recognition model is trained. These parameters are frozen and remain unchanged during the training process. For example, after training the keypoint localization model, a trained model is obtained. After loading the trained model, all learnable parameters of the keypoint localization model are frozen, the output features of the backbone network are extracted, and these features are used as input to the keypoint occlusion recognition model before training it again.

[0034] by Figure 4Taking the network structure shown as an example, the five convolutional modules on the left, MobileBottleneck1-MobileBottleneck5, are trained first. After training, the parameters of each layer of MobileBottleneck1-MobileBottleneck5 are frozen, and the output features of MobileBottleneck3 are extracted and input into MobileBottleneck4_2. Then, only the parameters of MobileBottleneck4_2 and MobileBottleneck5_2 are trained until MobileBottleneck4_2 and MobileBottleneck5_2 can accurately perform occlusion detection. During the training of the parameters of MobileBottleneck4_2 and MobileBottleneck5_2, the parameters of each layer of MobileBottleneck1-MobileBottleneck5 remain unchanged.

[0035] Since the keypoint occlusion recognition model is added as a branch to the keypoint localization model as a head network, the weight parameters of the backbone network are reused for both the keypoint localization and keypoint occlusion recognition models. The parameters of the trained keypoint localization model are frozen when training the keypoint occlusion recognition branch; only the added branch is trained. Compared to training the original keypoint localization model, there is not much increase in computation. Therefore, training efficiency is greatly improved. Figure 6 The figure shows a graphical representation of the key point localization model and the key point occlusion recognition model, where the red box represents the reused weight parameters.

[0036] According to one embodiment, the training sample set includes face images in which the coordinates of key points are labeled and whether each key point is occluded.

[0037] According to one embodiment, to further improve model efficiency, this invention performs channel pruning on both the keypoint localization model and the keypoint occlusion recognition model. This optimizes the application scenario without affecting accuracy or increasing processing time and CPU usage.

[0038] Channel pruning is a technique used to reduce the number of parameters in deep neural networks. In channel pruning, certain convolutional channels are removed from the network to reduce the number of parameters and computational complexity. The principle behind channel pruning is that removing certain channels reduces redundancy and makes the network more compact and efficient.

[0039] Channel pruning can be performed after each training task. For example, after the keypoint localization model has finished training, it can be pruned to reduce the number of parameters. For instance, MobileBottleneck3 originally had 64 channels, which can be pruned to 48. Then, the keypoint localization model can be fine-tuned, and the parameters after fine-tuning can be frozen. After the keypoint occlusion recognition module has finished training, the keypoint occlusion recognition model (in...) can be further fine-tuned. Figure 4 The examples demonstrate pruning and fine-tuning training for MobileBottleneck4_2 and MobileBottleneck5_2. According to one embodiment, the pruning rate can be as high as 20%.

[0040] The following reference Figure 7 This paper describes a method for keypoint occlusion recognition using the keypoint occlusion recognition model described above.

[0041] First, in step 701, the 2D face image to be detected is input into the backbone network of the keypoint localization model. Then, in step 702, the output features of the backbone network of the keypoint localization model are input into the head network and the keypoint occlusion recognition model of the keypoint localization model. In step 703, the keypoint localization model outputs the keypoint localization detection result, and the keypoint occlusion recognition model outputs the keypoint occlusion recognition result. The keypoint localization model and the keypoint occlusion recognition model are trained using the training method described above.

[0042] According to one embodiment, the keypoint occlusion recognition result includes a confidence score for whether each keypoint is occluded. Keypoints are identified by serial numbers; for example, in the case of using 106 keypoints, the keypoints are identified by serial numbers from 1 to 106. The confidence score for occlusion ranges from 0 to 1, with a score greater than 0.5 indicating occlusion and a score less than 0.5 indicating no occlusion. Therefore, it is possible to further determine whether each keypoint is occluded based on the output confidence score value. Figure 8 The image shows a face where some keypoints are identified as occluded. The darker points are keypoints with a confidence score greater than 0.5, meaning they are identified as occluded keypoints.

[0043] According to one embodiment, it can be determined whether each region of a 2D face is occluded based on the detection results. The face can be roughly divided into several regions based on a keypoint model, such as the left eye, right eye, nose, and mouth. Each region includes multiple keypoints. A voting process can be performed on the keypoints in these regions. If the proportion of keypoints detected as occluded in a region exceeds a predetermined threshold, then it can be determined that the region is occluded.

[0044] To be applicable on mobile devices, the model of this invention has a very small input resolution and a limited number of parameters, resulting in limited performance in key point occlusion detection. By employing a voting process, the recognition results become more robust and stable.

[0045] The following describes an example of voting processing. In this example, the key points included in each region are shown in the table below:

[0046]

[0047] The mouth, right eye, and left eye areas each contain 20 key points, while the nose area contains 12 key points. With a threshold ratio set to 80%, for the mouth, right eye, and left eye areas, at least 16 key points need to be identified as occluded to classify the entire area as occluded. For the nose area, at least 10 key points need to be identified as occluded to classify it as occluded. For areas identified as occluded, no makeup will be applied during the beautification process.

[0048] For example, the detection results show that keypoints 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 65, 66, 67, 68, 69, 70, 71, 73, and 74 are occluded. Of these occluded keypoints, 18 (i.e., 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 65, 66, 67, 68, 69, 70, and 71) belong to the mouth region, and 2 (i.e., 73 and 74) belong to the nose region. Therefore, it can be determined that the mouth is occluded, while the nose is not. Makeup can be omitted from the mouth during beautification. It is evident that although some keypoints are not correctly identified, such as keypoints 63, 64, 73, and 74, these incorrect identifications do not affect the final detection result. In other words, the voting process significantly increases the stability and robustness of the recognition effect.

[0049] The keypoint occlusion recognition model according to embodiments of this disclosure at least solves the problem of real-time face region occlusion recognition on mobile terminals. The method and apparatus of this disclosure identify occluded keypoints by adding a keypoint occlusion recognition model branch to the original keypoint localization model. Because some convolutional modules of the model are reused and channel pruning is applied, occlusion can be detected without significantly increasing processing time or CPU usage. Furthermore, keypoint voting processing is performed on each region after occlusion recognition, greatly improving the stability and robustness of the detection, thereby enhancing the effect and user experience of intelligent face beautification and makeup.

[0050] Figure 9 An exemplary configuration of a training apparatus 900 for a keypoint occlusion recognition model according to embodiments of the present disclosure is shown.

[0051] Training device 900 is an example of a hardware device capable of applying the above aspects of this disclosure. Training device 900 can be any machine configured to perform processing and / or computation. Training device 900 can be, but is not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal data assistant (PDA), smartphone, in-vehicle computer, or a combination thereof.

[0052] like Figure 9 As shown, the training device 900 may include one or more components that can be connected to or communicate with the bus 902 via one or more interfaces. The bus 902 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. The training device 900 may include, for example, one or more processors 904, one or more input devices 906, and one or more output devices 908. The one or more processors 904 may be any type of processor and may include, but is not limited to, one or more general-purpose processors or dedicated processors (such as dedicated processing chips). The processors 904 are configured, for example, to implement the training methods of this disclosure. The input devices 906 may be any type of input device capable of inputting information to a computing device and may include, but is not limited to, a mouse, keyboard, touchscreen, microphone, and / or remote controller. The output devices 908 may be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer.

[0053] The training device 900 may also include or be connected to a non-transitory storage device 914, which can be any non-transitory storage device capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, compressed disks or any other optical media, cache memory and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions and / or code. The training device 900 may also include random access memory (RAM) 910 and read-only memory (ROM) 912. ROM 912 may store executable programs, utilities, or processes in a non-volatile manner. RAM 910 provides volatile data storage and stores instructions related to the operation of the testing device 800. The training device 900 may also include a network / bus interface 916 coupled to a data link 918. The network / bus interface 916 can be any kind of device or system capable of enabling communication with external devices and / or networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets (such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication facilities, etc.).

[0054] According to one aspect of this disclosure, a keypoint occlusion recognition device for 2D faces is also provided. The keypoint occlusion recognition device can utilize reference to... Figure 9 The training device is implemented with a similar hardware configuration, except that its processor is configured to execute the key point occlusion recognition method described above.

[0055] This disclosure can be implemented as any combination of apparatus, system, integrated circuit, and computer program on a non-transitory computer-readable medium. One or more processors can be implemented as integrated circuits (ICs), application-specific integrated circuits (ASICs), or large-scale integrated circuits (LSIs), system LSIs, super LSIs, or ultra LSI components that perform some or all of the functions described in this disclosure.

[0056] This disclosure includes the use of software, application programs, computer programs, or algorithms. Software, application programs, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to cause a computer, such as one or more processors, to perform the steps described above and in the accompanying drawings. For example, one or more memories may store the software or algorithm in executable instructions, and one or more processors may be associated with executing a set of instructions of the software or algorithm to provide various functionalities according to embodiments described in this disclosure.

[0057] Software and computer programs (also referred to as programs, software applications, applications, components, or code) include machine instructions for programmable processors and can be implemented in high-level procedural languages, object-oriented programming languages, functional programming languages, logic programming languages, assembly languages, or machine languages. The term "computer-readable medium" means any computer program product, apparatus, or device used to provide machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories, and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.

[0058] This disclosure includes the following embodiments:

[0059] 1. A training method for a 2D face keypoint occlusion recognition model, comprising:

[0060] The training sample set is input into the key point localization model based on the convolutional neural network for training, wherein the key point localization model includes a backbone network and a head network.

[0061] Freeze the parameters of the trained keypoint localization model and extract the output features of the backbone network;

[0062] The extracted output features are input into a keypoint occlusion recognition model based on a convolutional neural network for training.

[0063] 2. The training method as described in Example 1, wherein the key point occlusion recognition model is configured as a head network branch parallel to the head network of the key point localization model.

[0064] 3. The training method as described in Example 1, wherein freezing the parameters of the trained key point localization model includes: keeping the parameters of the trained key point localization model unchanged during the training process of the key point occlusion recognition model.

[0065] 4. The training method as described in Example 1, wherein the training sample set includes the coordinates of multiple key points and face images of each key point, indicating whether the key points are occluded.

[0066] 5. The training method as described in Example 1 further includes:

[0067] Channel pruning is performed on the trained keypoint localization model; and

[0068] Channel pruning is performed on the trained keypoint occlusion recognition model.

[0069] 6. The training method as described in Example 5, wherein the pruning rate is 20%.

[0070] 7. The training method as described in Example 1, wherein the backbone network of the key point localization model adopts Mobilenet-V1.

[0071] 8. A method for key point occlusion recognition of 2D faces, comprising:

[0072] The 2D face image to be detected is input into the backbone network of the key point localization model;

[0073] The output features of the backbone network of the keypoint localization model are input into the head network of the keypoint localization model and the keypoint occlusion recognition model, respectively; and

[0074] The keypoint localization model outputs keypoint localization and detection results, and the keypoint occlusion recognition model outputs keypoint occlusion recognition results.

[0075] The key point localization model and the key point occlusion recognition model are trained using the training method described in any one of Examples 1-7.

[0076] 9. The key point occlusion recognition method as described in Example 8, wherein the key point occlusion recognition result includes the confidence level of whether each key point is occluded.

[0077] 10. The key point occlusion recognition method as described in Example 8 further includes: determining whether each region of the 2D face is occluded based on the key point occlusion recognition result.

[0078] 11. The key point occlusion recognition method as described in Example 8, wherein determining whether each region of a 2D face is occluded based on the key point occlusion recognition result includes:

[0079] Determine the ratio of the number of occluded keypoints in each region of a 2D face to the total number of keypoints in that region; and

[0080] When the ratio exceeds a predetermined threshold, it is determined that the area is occluded.

[0081] 12. The key point occlusion recognition method as described in Example 8, wherein the key point occlusion recognition method is applicable to mobile user terminals.

[0082] 13. A training device for a 2D face key point occlusion recognition model, comprising:

[0083] Memory, which stores instructions; and

[0084] The processor is configured to execute instructions stored in the memory to perform the method according to any one of embodiments 1 to 7.

[0085] 14. A key point occlusion recognition device for 2D faces, comprising:

[0086] Memory, which stores instructions; and

[0087] The processor is configured to execute instructions stored in the memory to perform the method according to any one of embodiments 8 to 12.

[0088] The subject matter of this disclosure is provided as examples of apparatus, systems, methods, and programs for performing the features described herein. However, other features or variations are contemplated in addition to those described above. It is anticipated that the components and functions of this disclosure can be implemented using any emerging techniques that may replace any of the above-described implementations.

[0089] Furthermore, the above description provides examples and does not limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various processes or components may be appropriately omitted, substituted, or added in various embodiments. For example, features described with respect to certain embodiments may be combined in other embodiments.

[0090] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring the operations to be performed in the specific order shown or in sequential order, or requiring the execution of all illustrated operations to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A training method for a 2D face keypoint occlusion recognition model, comprising: The training sample set is input into the key point localization model based on the convolutional neural network for training. The key point localization model includes a backbone network and a head network. The training sample set includes the coordinates of multiple key points and face images of each key point, including whether each key point is occluded. Freeze the parameters of the trained keypoint localization model and extract the output features of the backbone network. Freezing the parameters of the trained keypoint localization model includes keeping the parameters of the trained keypoint localization model unchanged during the training process of the keypoint occlusion recognition model. The extracted output features are input into a keypoint occlusion recognition model based on a convolutional neural network for training. The key point occlusion recognition model is configured as a head network branch parallel to the head network of the key point localization model, and shares the same backbone network with the head network, so that the parameters of the backbone network are reused for both the key point localization model and the key point occlusion recognition model.

2. The training method as described in claim 1, further comprising: Channel pruning is performed on the trained keypoint localization model; and Channel pruning is performed on the trained keypoint occlusion recognition model.

3. The training method as described in claim 2, wherein the pruning rate is 20%.

4. The training method as described in claim 1, wherein the backbone network of the key point localization model adopts Mobilenet-V1.

5. A method for key point occlusion recognition of 2D faces, comprising: The 2D face image to be detected is input into the backbone network of the key point localization model; The output features of the backbone network of the keypoint localization model are input into the head network of the keypoint localization model and the keypoint occlusion recognition model, respectively; and The keypoint localization model outputs keypoint localization and detection results, and the keypoint occlusion recognition model outputs keypoint occlusion recognition results. The key point localization model and the key point occlusion recognition model are trained using the training method described in any one of claims 1-4.

6. The key point occlusion recognition method as described in claim 5, wherein, The keypoint occlusion recognition results include the confidence level of whether each keypoint is occluded.

7. The key point occlusion recognition method as described in claim 5 further includes: Based on the key point occlusion recognition results, determine whether each region of the 2D face is occluded.

8. The key point occlusion recognition method as described in claim 5, wherein determining whether each region of a 2D face is occluded based on the key point occlusion recognition result includes: Determine the ratio of the number of occluded keypoints in each region of a 2D face to the total number of keypoints in that region; as well as When the ratio exceeds a predetermined threshold, it is determined that the area is occluded.

9. The key point occlusion recognition method as described in claim 5, wherein the key point occlusion recognition method is applicable to mobile user terminals.

10. A training device for a 2D face keypoint occlusion recognition model, comprising: Memory, which stores instructions; as well as The processor is configured to execute instructions stored in the memory to perform the method according to any one of claims 1 to 4.

11. A key point occlusion recognition device for 2D faces, comprising: Memory, which stores instructions; as well as The processor is configured to execute instructions stored in the memory to perform the method according to any one of claims 5 to 9.

Images