Face occlusion detection method, device and computer equipment

Abstract

The application relates to a face occlusion detection method, device and computer equipment. The method comprises the following steps: inputting a target face image into a pre-trained occlusion feature encoder, extracting a target face feature corresponding to the target face image through the occlusion feature encoder; obtaining a first similarity between the target face feature and a first type face feature stored in a pre-constructed occlusion feature knowledge base; in the case that the first similarity is less than a first preset threshold, obtaining a second similarity between the target face feature and each second type face feature stored in the occlusion feature knowledge base; and in the case that a maximum value in the second similarities is less than a second preset threshold, determining that the target face image is a face image of an unknown occlusion type. The method can realize the recognition of the face image of the unknown occlusion type, thereby improving the accuracy of face occlusion recognition.

Description

Technical Field

[0001] This application relates to the field of face detection technology, and in particular to a face occlusion detection method, apparatus and computer equipment. Background Technology

[0002] With the development of facial detection technology, a new technology has emerged that applies facial detection to driver monitoring systems to improve driving safety. This technology can identify and warn of risky behaviors such as fatigued driving, distracted driving, and inattention by collecting the driver's facial information. Furthermore, if the driver is wearing a mask, sunglasses, or scarf, their face may be obscured, affecting the accuracy of facial detection and thus impacting driving safety. Therefore, facial occlusion detection technology needs to be incorporated into driver monitoring systems to improve driving safety.

[0003] In traditional technologies, face occlusion detection typically relies on training a deep learning model using large-scale labeled data to identify the type of face occlusion. However, these methods struggle to handle unknown types of occlusion. For occlusions not present in the training set, the model is highly susceptible to misclassification as "normal faces" or "no target," leading to the failure of security functions. Therefore, existing face occlusion detection methods are ill-equipped to identify unknown types of occlusion. Summary of the Invention

[0004] Based on this, this application addresses the aforementioned technical problems by providing a method, apparatus, computer device, computer-readable storage medium, and computer program product capable of identifying face occlusion of unknown types.

[0005] Firstly, this application provides a face occlusion detection method, including:

[0006] Acquire a target face image, input the target face image into a pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder;

[0007] Obtain the first similarity between the target face feature and the first type of face features stored in the pre-constructed occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image;

[0008] If the first similarity is less than the first preset threshold, the second similarity between the target face feature and each of the second type face features stored in the occlusion feature knowledge base is obtained; each of the second type face features is a face feature extracted from a face image with different known occlusion types.

[0009] If the maximum value of each of the second similarities is less than the second preset threshold, the target face image is determined to be a face image of unknown occlusion type.

[0010] In the aforementioned face occlusion detection method, a target face image is acquired and input into a pre-trained occlusion feature encoder. The encoder extracts the target face features corresponding to the target face image. A first similarity is obtained between the target face features and a first type of face features stored in a pre-built occlusion feature knowledge base. The first type of face features are face features extracted from unoccluded face images. If the first similarity is less than a first preset threshold, a second similarity is obtained between the target face features and each second type of face feature stored in the occlusion feature knowledge base. Each second type of face feature is a face feature extracted from face images with different known occlusion types. If the maximum value of each second similarity is less than a second preset threshold, the target face image is determined to be a face image with an unknown occlusion type. This application can acquire a target face image and input it into an occlusion feature encoder. The occlusion feature encoder extracts the target face features corresponding to the target face image. Then, the target face features can be compared with the first type of face features and the second type of face features stored in a pre-set occlusion feature knowledge base. If the target face features are not similar to either the first type of face features or the second type of face features, the target face image can be determined to be a face image with an unknown occlusion type. In this way, the recognition of face images with unknown occlusion types can be realized, thereby improving the accuracy of face occlusion recognition.

[0011] In one embodiment, after determining that the target face image is a face image with an unknown occlusion type, the method further includes: acquiring original sample face images corresponding to the unknown occlusion type; the number of original sample face images is a first number; inputting the original sample face images into a pre-trained face image generation model, and generating a synthetic sample face image corresponding to the unknown occlusion type through the face image generation model; the number of synthetic sample face images is a second number, which is greater than the first number; inputting each of the original sample face images and each of the synthetic sample face images into the occlusion feature encoder, and extracting face features corresponding to each of the original sample face images and each of the synthetic sample face images through the occlusion feature encoder; constructing a third type of face features based on the face features corresponding to each of the original sample face images and each of the synthetic sample face images, and updating the occlusion feature knowledge base based on the third type of face features.

[0012] In this embodiment, when an unknown occlusion type is identified, the terminal can also collect a small number of original sample face images corresponding to the unknown occlusion type, and synthesize a large number of synthetic sample face images corresponding to the unknown occlusion type through a pre-trained face image generation model. Then, the occlusion feature encoder extracts the face features of the above images to construct a third type of face features corresponding to the unknown occlusion type, so as to update the occlusion feature knowledge base. This method does not require iterative training of the deployed occlusion feature encoder, and can also identify new types of occlusion, thereby reducing response latency. Furthermore, it can expand the number of sample face images corresponding to the unknown occlusion type through the face image generation model, thereby solving the problem of scarce sample face image data corresponding to the unknown occlusion type, and also protecting user privacy.

[0013] In one embodiment, updating the occlusion feature knowledge base based on the third type of facial features includes: performing pixel averaging on each of the synthetic sample facial images to obtain an average facial image map corresponding to the unknown occlusion type; uploading the average facial image map to a cloud server; the cloud server obtaining the occlusion type label corresponding to the average facial image map and returning the occlusion type label; using the occlusion type label as the occlusion type corresponding to the third type of facial features, and updating the occlusion feature knowledge base using the correspondence between the third type of facial features and the occlusion type.

[0014] In this embodiment, the occlusion feature knowledge base can also be updated by generating an average image that cannot reconstruct the user's identity. This method can update the occlusion feature knowledge base while further protecting user privacy and security.

[0015] In one embodiment, acquiring the original sample face image corresponding to the unknown occlusion type includes: displaying a prompt interface; the prompt interface is used to request the acquisition of a face image at the current moment as the original sample face image; in response to a request authorization operation triggered in the prompt interface, acquiring a first number of face images, and using the first number of face images as the original sample face image.

[0016] In this embodiment, after determining that the target face image is a face image with an unknown occlusion type, a prompt interface can be displayed to request the current face image as the original sample face image. At this time, the user can trigger the authorization request operation in the prompt interface, so that the terminal will collect the user's face image as the original sample face image only after authorization. This method can improve the intelligence of the original sample face image collection.

[0017] In one embodiment, the face image generation model includes an encoder and a decoder; the face image generation model is trained through the following steps: acquiring sample occluded face images, and occlusion type information and occlusion region information corresponding to the sample occluded face images; inputting the sample occluded face images, the occlusion type information, and the occlusion region information into the encoder of the face image generation model to be trained, extracting image features corresponding to the sample occluded face images through the encoder, and fusing the image features with the occlusion type information and the occlusion region information to obtain latent features corresponding to the image features; the latent features are image features that incorporate the occlusion type information and the occlusion region information as conditional constraints; inputting the latent features into the decoder of the face image generation model to be trained, generating predicted occluded face images through the decoder; constructing a first loss value based on the difference between the predicted occluded face image and the sample occluded face image, and constructing a second loss value based on the distribution of the latent features, and training the face image generation model using the first loss value and the second loss value.

[0018] In this embodiment, the face image generation model can be trained using a first loss value and a second loss value. The first loss value is obtained based on the difference between the predicted occluded face image and the sample occluded face image. The predicted occluded face image can be generated by the decoder after obtaining latent features through the encoder. The second loss value is obtained based on the distribution of latent features output by the encoder. This method can improve the accuracy of the face image generation model in generating occluded face images.

[0019] In one embodiment, the occlusion feature encoder is trained through the following steps: acquiring a first sample face image, and corresponding positive and negative sample images; the positive sample image is a face image with the same identity as the person in the first sample face image and the same occlusion type; the negative sample images include face images with the same identity as the person in the first sample face image but different occlusion types, and face images with different identities as the person in the first sample face image but the same occlusion type; inputting the first sample face image, the positive and negative sample images into the occlusion feature encoder to be trained, extracting a first face feature corresponding to the first sample face image, a second face feature corresponding to the positive sample image, and a third face feature corresponding to the negative sample image through the occlusion feature encoder; obtaining a third similarity between the first face feature and the second face feature, and a fourth similarity between the first face feature and the third face feature, obtaining a loss value based on the third and fourth similarities, and training the occlusion feature encoder using the loss value.

[0020] In this embodiment, other samples with the same identity and the same occlusion category can be used as positive samples, while samples with different identities but the same occlusion type, and samples with the same identity but different occlusion types can be used as two types of negative samples. This allows for the training of the occlusion feature encoder through contrastive learning, enabling the encoder to maintain the consistency of facial identity features under different occlusion conditions and enhancing its ability to discriminate occlusion patterns. This method significantly improves the generalization performance and robustness of the feature encoder, allowing it to extract discriminative invariant features even when facing unknown occlusions on the vehicle side, providing a high-quality feature foundation for few-shot learning.

[0021] In one embodiment, the occlusion feature knowledge base is obtained through the following steps: acquiring a second sample face image and the occlusion type corresponding to the second sample face image; the occlusion type includes: no occlusion and different known occlusion types; inputting the second sample face image into the pre-trained occlusion feature encoder, and obtaining sample face features corresponding to the second sample face image through the occlusion feature encoder; for each occlusion type, constructing occlusion type face features corresponding to each occlusion type based on the sample face features of the second sample face image of the same occlusion type; the occlusion type face features include a first type face feature corresponding to the no-occlusion type and a second type face feature corresponding to each known occlusion type; constructing the occlusion feature knowledge base according to the correspondence between each occlusion type and the occlusion type face features.

[0022] In this embodiment, a pre-trained occlusion feature encoder can be used to output sample face features of sample face images of various occlusion types. Based on the sample face features, occlusion type face features of each occlusion type can be constructed, and an occlusion feature knowledge base can be constructed based on the above correspondence. This method can improve the accuracy of the data stored in the occlusion feature knowledge base.

[0023] In one embodiment, after obtaining the first similarity between the target face feature and the first type of face features stored in the pre-built occlusion feature knowledge base, the method further includes: determining that the target face image is an unoccluded face image if the first similarity is greater than or equal to the first preset threshold; after obtaining the second similarity between the target face feature and each of the second type of face features stored in the occlusion feature knowledge base, the method further includes: taking the known occlusion type corresponding to the maximum value of each second similarity as the occlusion type corresponding to the target face image if the maximum value of each second similarity is greater than or equal to the second preset threshold.

[0024] In this embodiment, the identification of face image occlusion type can also be completed by using the first similarity and the second similarity, which can improve the efficiency of face image occlusion type identification.

[0025] Secondly, this application also provides a face occlusion detection device, comprising:

[0026] The face feature extraction module is used to acquire a target face image, input the target face image into a pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder.

[0027] The first similarity acquisition module is used to acquire the first similarity between the target face features and the first type of face features stored in the pre-built occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image;

[0028] The second similarity acquisition module is used to acquire the second similarity between the target face feature and each second type of face feature stored in the occlusion feature knowledge base when the first similarity is less than the first preset threshold; each second type of face feature is a face feature extracted from face images of different known occlusion types.

[0029] The unknown occlusion recognition module is used to determine that the target face image is a face image with unknown occlusion when the maximum value of each of the second similarities is less than a second preset threshold.

[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0031] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0032] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above aspects.

[0033] Regarding the beneficial effects of any of the technical solutions in the second to fifth aspects mentioned above, refer to the beneficial effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of an optional flowchart of a face occlusion detection method in one embodiment;

[0036] Figure 2 This is a schematic diagram of an optional process for updating the occlusion feature knowledge base in one embodiment;

[0037] Figure 3 This is a schematic diagram of an optional process for updating the occlusion feature knowledge base in another embodiment;

[0038] Figure 4 This is a schematic diagram of an optional process for training a face image generation model in one embodiment;

[0039] Figure 5 This is a schematic diagram of an optional process for training an occlusion feature encoder in one embodiment;

[0040] Figure 6 This is a schematic diagram of an optional process for constructing an occlusion feature knowledge base in one embodiment;

[0041] Figure 7 This is a schematic diagram of an optional flowchart of a driver's face occlusion detection method in one embodiment;

[0042] Figure 8 This is a schematic diagram of an optional structure of a lightweight convolutional neural network in one embodiment;

[0043] Figure 9 This is a schematic diagram of an optional structure of the backbone network projection head in one embodiment;

[0044] Figure 10 This is a schematic diagram of an optional structure of the original U-shaped network in one embodiment;

[0045] Figure 11 This is a schematic diagram of an optional structure for depthwise separable convolution in one embodiment;

[0046] Figure 12 This is a schematic diagram of an optional process for real-time occlusion detection in one embodiment;

[0047] Figure 13 This is a schematic diagram of an optional structure of a face occlusion detection device in one embodiment;

[0048] Figure 14 This is a schematic diagram of an optional internal structure of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0050] The terms "first," "second," etc., used in this application may be used to describe various elements, but these elements are not limited by these terms. These terms are used only to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0051] In one embodiment, such as Figure 1 As shown, a face occlusion detection method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0052] Step S101: Obtain the target face image, input the target face image into the pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder.

[0053] Here, the terminal can refer to the vehicle terminal, the target face image refers to the face image that needs to be occlusion category detection, which can be the driver's face image captured by the vehicle camera, the occlusion feature encoder can be a pre-trained encoder for extracting face features, and the target face features are the face features extracted from the target face image.

[0054] Specifically, the vehicle terminal can be equipped with a pre-trained occlusion feature encoder. After the vehicle camera captures the driver's face image, the vehicle terminal can use the driver's face image as the target face image and input it into the pre-trained occlusion feature encoder. The occlusion feature encoder can then extract the corresponding face features of the target face image as the target face features.

[0055] Step S102: Obtain the first similarity between the target face features and the first type of face features stored in the pre-built occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image.

[0056] The occlusion feature knowledge base is a pre-built feature knowledge base that stores facial features corresponding to different occlusion types. The first type of facial feature refers to the facial features in the occlusion feature knowledge base corresponding to the unoccluded type, which can be extracted from an unoccluded facial image. The first similarity refers to the similarity between the target facial feature and the first type of facial feature, which can be calculated using cosine similarity.

[0057] Specifically, after extracting the target face features, the feature similarity between the target face features and the first type of face features corresponding to the unoccluded category stored in the occlusion feature knowledge base can be calculated as the first similarity to determine whether the occlusion category of the target face image is the unoccluded category.

[0058] Step S103: If the first similarity is less than the first preset threshold, obtain the second similarity between the target face feature and each second type of face feature stored in the occlusion feature knowledge base; each second type of face feature is a face feature extracted from face images with different known occlusion types.

[0059] The first preset threshold is a pre-set threshold used to determine whether the occlusion category of the target face image is the unoccluded category. For example, the threshold can be set to 0.45. If the feature similarity between the target face features and the first type of face features is lower than the first preset threshold, it indicates that the target face image state deviates from the normal unoccluded state, and then the second stage of judgment is entered.

[0060] The second type of facial features refers to facial features in the occlusion feature knowledge base corresponding to different known occlusion types. In this embodiment, in addition to storing the first type of facial features corresponding to the no-occlusion category, the occlusion feature knowledge base can also store the second type of facial features corresponding to various known occlusion categories, such as the second type of facial features corresponding to the mouth and nose occlusion category, and the second type of facial features corresponding to the eye occlusion category, etc. The second similarity is the similarity between the target facial feature and each second type of facial feature, which can also be calculated using cosine similarity.

[0061] Specifically, if the first similarity is less than the first preset threshold, the terminal can determine that the state of the target face image deviates from the normal unoccluded state. Therefore, it can further calculate the similarity between the target face features and the different second-type face features stored in the occlusion feature knowledge base, thereby obtaining multiple second similarities.

[0062] Step S104: If the maximum value of each second similarity is less than the second preset threshold, the target face image is determined to be a face image with an unknown occlusion type.

[0063] The second preset threshold is a threshold used to determine whether the occlusion category of the target face image is a known occlusion category. This threshold can be set to 0.35. If the maximum value of each second similarity is less than the second preset threshold, it means that the target face image does not belong to any known occlusion category. Therefore, the terminal can determine that the target face image is a face image with an unknown occlusion type.

[0064] In the aforementioned face occlusion detection method, a target face image is acquired and input into a pre-trained occlusion feature encoder. The encoder extracts the target face features corresponding to the target face image. A first similarity is obtained between the target face features and a first type of face features stored in a pre-built occlusion feature knowledge base. The first type of face features are face features extracted from unoccluded face images. If the first similarity is less than a first preset threshold, a second similarity is obtained between the target face features and each second type of face feature stored in the occlusion feature knowledge base. Each second type of face feature is a face feature extracted from face images with different known occlusion types. If the maximum value of each second similarity is less than a second preset threshold, the target face image is determined to be a face image with an unknown occlusion type. This application can acquire a target face image and input it into an occlusion feature encoder. The occlusion feature encoder extracts the target face features corresponding to the target face image. Then, the target face features can be compared with the first type of face features and the second type of face features stored in a pre-set occlusion feature knowledge base. If the target face features are not similar to either the first type of face features or the second type of face features, the target face image can be determined to be a face image with an unknown occlusion type. In this way, the recognition of face images with unknown occlusion types can be realized, thereby improving the accuracy of face occlusion recognition.

[0065] In one embodiment, such as Figure 2 As shown, after step S104, the following may also be included:

[0066] Step S201: Obtain the original sample face image corresponding to the unknown occlusion type; the number of original sample face images is the first quantity.

[0067] The original sample face image refers to the face image directly captured by the camera and used to update the occlusion feature knowledge base. The occlusion type of the face image can be an unknown occlusion type, and the number can be a first number. In order to optimize the recognition capability for face images with unknown occlusion types, after detecting that the target face image is a face image with unknown occlusion type, a first number of original sample face images corresponding to the unknown occlusion type can be further collected. For example, 3-5 face images of the driver at the current moment can be collected as original sample face images.

[0068] Step S202: Input the original sample face image into the pre-trained face image generation model, and generate a synthetic sample face image corresponding to the unknown occlusion type through the face image generation model; the number of synthetic sample face images is the second number, which is greater than the first number.

[0069] A face image generation model refers to a pre-trained neural network model used to synthesize face images corresponding to unknown occlusion types, i.e., synthetic sample face images. This model can generate a large number of diverse and semantically consistent synthetic images (i.e., a second number of synthetic sample face images) based on a small number of novel occluded face images (i.e., a first number of original sample face images). This approach solves the problem of scarce sample face image data corresponding to unknown occlusion types and also protects driver privacy.

[0070] Specifically, after obtaining the first number of original sample face images, the original sample face images can be input into a pre-trained face image generation model, which generates synthetic sample face images corresponding to unknown occlusion types. The number of these synthetic sample face images is the second number, thereby expanding the number of sample face images corresponding to unknown occlusion types.

[0071] Step S203: Input each original sample face image and each synthetic sample face image into the occlusion feature encoder, and extract the face features corresponding to each original sample face image and each synthetic sample face image through the occlusion feature encoder.

[0072] Step S204: Construct a third type of face feature based on the face features corresponding to each original sample face image and each synthesized sample face image, and update the occlusion feature knowledge base based on the third type of face feature.

[0073] The third type of face feature refers to the face feature corresponding to the unknown occlusion type. This feature can be used to update the occlusion feature knowledge base so that the updated occlusion feature knowledge base can recognize face images with unknown occlusion types.

[0074] Specifically, after obtaining the original sample face images and the synthesized sample face images, these images can be input into a pre-trained occlusion feature encoder on the terminal. The occlusion feature encoder then extracts the corresponding facial features from each original and synthesized sample face image. Subsequently, a third type of facial feature can be constructed based on these features. For example, the arithmetic mean of the various facial features can be calculated as the third type of facial feature, which is then used to update the occlusion feature knowledge base.

[0075] In this embodiment, when an unknown occlusion type is identified, the terminal can also collect a small number of original sample face images corresponding to the unknown occlusion type, and synthesize a large number of synthetic sample face images corresponding to the unknown occlusion type through a pre-trained face image generation model. Then, the occlusion feature encoder extracts the face features of the above images to construct a third type of face features corresponding to the unknown occlusion type, so as to update the occlusion feature knowledge base. This method does not require iterative training of the deployed occlusion feature encoder, and can also identify new types of occlusion, thereby reducing response latency. Furthermore, it can expand the number of sample face images corresponding to the unknown occlusion type through the face image generation model, thereby solving the problem of scarce sample face image data corresponding to the unknown occlusion type, and also protecting user privacy.

[0076] Furthermore, such as Figure 3 As shown, step S204 may further include:

[0077] Step S301: Perform pixel averaging on each synthesized sample face image to obtain the average face image corresponding to the unknown occlusion type.

[0078] Step S302: Upload the average face image to the cloud server; the cloud server is used to obtain the occlusion type label corresponding to the average face image and return the occlusion type label.

[0079] The average face image refers to the average image generated by averaging each synthetic sample face image at the pixel level. Since the average image cannot reconstruct the user's identity, in this embodiment, the terminal can only upload the average image that cannot reconstruct the user's identity to the cloud server, thus further protecting user privacy. The occlusion type label refers to the actual occlusion type label corresponding to the average face image, which can be obtained through annotation.

[0080] Specifically, after the terminal generates synthetic sample face images through the face image generation model, it can also perform pixel-level averaging on each synthetic sample face image to generate an average face image corresponding to an unknown occlusion type, and upload it to the cloud server. After receiving the average face image, the cloud server can label the occlusion type of the average face image, obtain the corresponding occlusion type label, and return it to the terminal.

[0081] Step S303: Use the occlusion type label as the occlusion type corresponding to the third type of face feature, and update the occlusion feature knowledge base using the correspondence between the third type of face feature and the occlusion type.

[0082] After receiving the occlusion type label, the terminal can use the occlusion type label as the occlusion type corresponding to the third type of face feature, thereby storing the correspondence between the occlusion type label and the third type of face feature in the occlusion feature knowledge base, completing the annotation of unknown occlusion types, and updating the occlusion feature knowledge base.

[0083] In this embodiment, the occlusion feature knowledge base can also be updated by generating an average image that cannot reconstruct the user's identity. This method can update the occlusion feature knowledge base while further protecting user privacy and security.

[0084] Additionally, step S201 may further include: displaying a prompt interface; the prompt interface is used to request the acquisition of a face image at the current moment as an original sample face image; in response to the authorization request triggered in the prompt interface, a first number of face images are collected, and the first number of face images are used as original sample face images.

[0085] The prompt interface is used to request user authorization. In this embodiment, the original sample face image is obtained by collecting the driver's face image at the current moment. The driver's face image is used to update the occlusion feature knowledge base, which usually requires the driver's consent. Therefore, it is necessary to display a prompt interface to obtain authorization.

[0086] Specifically, upon detecting that the target face image is a face image with an unknown occlusion type, the terminal can generate a pop-up prompt in the interactive interface, explaining to the user that an unknown occlusion type has been detected, and requesting authorization to collect face images at the current moment for use in optimizing recognition capabilities. If the user then grants authorization and triggers the authorization operation for the aforementioned request, the terminal can collect a first number of face images at the current moment as raw sample face images.

[0087] In this embodiment, after determining that the target face image is a face image with an unknown occlusion type, a prompt interface can be displayed to request the current face image as the original sample face image. At this time, the user can trigger the authorization request operation in the prompt interface, so that the terminal will collect the user's face image as the original sample face image only after authorization. This method can improve the intelligence of the original sample face image collection.

[0088] In one embodiment, such as Figure 4 As shown, the face image generation model includes an encoder and a decoder; the face image generation model is trained through the following steps:

[0089] Step S401: Obtain the sample occluded face image, as well as the occlusion type information and occlusion area information corresponding to the sample occluded face image.

[0090] Occluded face images refer to sample images used to train face image generation models. In these sample images, the face region is occluded. These can be occluded face photographs. The occlusion type information of the sample occluded face image represents the type of occlusion, such as a mask or sunglasses. The occlusion region information represents the occluded area of ​​the face in the sample occluded face image, which can be represented by a black-and-white occlusion region map. When training a face image generation model, sample occluded face images can be collected first, and the occlusion type and occlusion region of each sample occluded face image can be obtained.

[0091] Step S402: Input the sample occluded face image, occlusion type information, and occlusion region information into the encoder of the face image generation model to be trained. Extract the image features corresponding to the sample occluded face image through the encoder, and fuse the image features with the occlusion type information and occlusion region information to obtain the latent features corresponding to the image features. The latent features are image features that incorporate the occlusion type information and occlusion region information as conditional constraints.

[0092] In this embodiment, the face image generation model can be a Conditional Variational Autoencoder (CVAE) model. This model maps the input image to the distribution parameters (mean and variance) of the latent space through the encoder, and then samples and reconstructs the image from this distribution through the decoder. The backbone network of the model adopts a U-shaped network structure, the core of which is a symmetrical encoder-decoder structure. Therefore, the face image generation model can include an encoder and a decoder.

[0093] Image features refer to the features corresponding to the occluded face image of the sample, while latent features are image features that incorporate occlusion type information and occlusion area information as conditional constraints. In this embodiment, occlusion type information and occlusion area information can be used as conditional information to control the generated image content.

[0094] Specifically, the occluded face image, occlusion type information, and occlusion region information can be input into the encoder of the face image generation model to be trained. The encoder extracts the image features corresponding to the occluded face image. Then, the image features can be fused with the occlusion type information and occlusion region information to obtain the corresponding latent vector features.

[0095] Step S403: Input the latent features into the decoder of the face image generation model to be trained, and generate a predicted occluded face image through the decoder.

[0096] Predicting occluded face images refers to the occluded face images generated by the decoder. After obtaining the latent features, the occluded face images can be reconstructed and generated by the decoder of the face image generation model, which serves as the predicted occluded face images.

[0097] Step S404: Construct a first loss value based on the difference between the predicted occluded face image and the sample occluded face image, and construct a second loss value based on the distribution of latent features. Use the first loss value and the second loss value to train the face image generation model.

[0098] Finally, the difference between the predicted occluded face image and the sample occluded face image can be used to construct a first loss value, which can be a reconstruction loss. A second loss value can be constructed using the distribution of latent features, which can be the KL divergence. The two loss values ​​are then combined to complete the training of the face image generation model.

[0099] For example, the loss function of a face image generation model can be shown below:

[0100]

[0101] in, This represents the total loss value of the model. This represents the reconstruction loss, i.e., the first loss value, while... This represents the KL divergence, i.e., the second loss value. It is a hyperparameter, typically set to 0.5, used to control the balance between reconstruction loss and KL divergence.

[0102] Furthermore, reconstruction loss It can be calculated using the mean square error, for example, using the following formula:

[0103]

[0104] in, Indicates the reconstruction loss. The sample image occludes the face. These are predicted occluded face images generated by U-Net, where N is the number of pixels in the image. and These represent the actual value and the predicted value of the i-th pixel, respectively.

[0105] The KL divergence is used to ensure that the distribution of the latent variable approximates a predefined prior distribution, such as the standard normal distribution. The calculation formula is as follows:

[0106]

[0107] in, Denotes KL divergence, and The mean and logarithmic variance of the latent features are represented by J, which represents the number of latent features and is typically taken as 32.

[0108] In this embodiment, the face image generation model can be trained using a first loss value and a second loss value. The first loss value is obtained based on the difference between the predicted occluded face image and the sample occluded face image. The predicted occluded face image can be generated by the decoder after obtaining latent features through the encoder. The second loss value is obtained based on the distribution of latent features output by the encoder. This method can improve the accuracy of the face image generation model in generating occluded face images.

[0109] In one embodiment, such as Figure 5 As shown, the occlusion feature encoder is trained through the following steps:

[0110] Step S501: Obtain a first sample face image, and a positive sample image and a negative sample image corresponding to the first sample face image; the positive sample image is a face image with the same identity as the person in the first sample face image and the same occlusion type; the negative sample image includes a face image with the same identity as the person in the first sample face image but a different occlusion type, and a face image with a different identity as the person in the first sample face image but the same occlusion type.

[0111] The first sample face image refers to the sample image used to train the occlusion feature encoder. This sample image can include unoccluded face images as well as face images with various known occlusion types. Positive sample images refer to face images with the same person identity and occlusion type as the first sample face image. For example, if the first sample face image is the face image of user i under occlusion type c, then the corresponding positive sample image could also be other face images of user i under occlusion type c, such as face images taken from different angles or with different resolutions.

[0112] The negative sample image is composed of two types of negative samples from the first sample face image. Type 1 negative samples are cross-identity negative samples, representing samples with different identities but the same occlusion type, i.e., the face image of user j under occlusion type c. Type 2 negative samples are samples with the same identity but different occlusion types, i.e., the face image of user i under occlusion type c. The image below shows a human face.

[0113] Specifically, when training the occlusion feature encoder, a first sample face image, as well as the corresponding positive and negative sample images, can be obtained.

[0114] Step S502: Input the first sample face image, the positive sample image, and the negative sample image into the occlusion feature encoder to be trained. Extract the first face feature corresponding to the first sample face image, the second face feature corresponding to the positive sample image, and the third face feature corresponding to the negative sample image through the occlusion feature encoder.

[0115] The first face feature refers to the face feature corresponding to the first sample face image, the second face feature refers to the face feature corresponding to the positive sample image, and the third face feature refers to the face feature corresponding to the negative sample image. Specifically, the first sample face image, the positive sample image, and the negative sample image can be input into the occlusion feature encoder to be trained, so that the face features of each sample image can be extracted by the occlusion feature encoder, and the first face feature, the second face feature, and the third face feature can be obtained respectively.

[0116] Step S503: Obtain the third similarity between the first face feature and the second face feature, and the fourth similarity between the first face feature and the third face feature. Obtain the loss value based on the third similarity and the fourth similarity, and use the loss value to train the occlusion feature encoder.

[0117] The third similarity refers to the similarity between the first face feature and each of the second face features, while the fourth similarity refers to the similarity between the first face feature and each of the third face features. After obtaining the first face feature, the second face feature, and the third face feature, the similarity between the first face feature and each of the second face features, and the similarity between the first face feature and each of the third face features can be calculated respectively. Based on the above similarity, the loss value is calculated, and the loss value is used to complete the training of the occlusion feature encoder. For example, it can be to make the similarity between the first sample face image and the positive sample as large as possible, while making the similarity between the first sample face image and the negative sample as small as possible.

[0118] For example, the loss value of the occlusion feature encoder It can be calculated using the following formula:

[0119]

[0120] Where P represents the number of positive samples with the same identity and the same occlusion type; N represents the number of negative samples across identities; and sim represents the similarity between two feature values. Indicates the third similarity. as well as This represents the fourth similarity between two types. Indicates the weight of the occlusion type. This indicates a type-specific temperature parameter. For the uniform temperature parameters of the negative samples, This indicates cross-category weights.

[0121] In this embodiment, other samples with the same identity and the same occlusion category can be used as positive samples, while samples with different identities but the same occlusion type, and samples with the same identity but different occlusion types can be used as two types of negative samples. This allows for the training of the occlusion feature encoder through contrastive learning, enabling the encoder to maintain the consistency of facial identity features under different occlusion conditions and enhancing its ability to discriminate occlusion patterns. This method significantly improves the generalization performance and robustness of the feature encoder, allowing it to extract discriminative invariant features even when facing unknown occlusions on the vehicle side, providing a high-quality feature foundation for few-shot learning.

[0122] In one embodiment, such as Figure 6 As shown, the occlusion feature knowledge base is obtained through the following steps:

[0123] Step S601: Obtain the second sample face image and the occlusion type corresponding to the second sample face image; the occlusion type includes: no occlusion and different known occlusion types.

[0124] The second sample face image refers to the sample image used to build the occlusion feature knowledge base. Specifically, in building the occlusion feature knowledge base, a second sample face image and the occlusion type corresponding to the second sample face image can also be collected. The second sample face image can include unoccluded face images and face images with known occlusion types.

[0125] Step S602: Input the second sample face image into the pre-trained occlusion feature encoder, and obtain the sample face features corresponding to the second sample face image through the occlusion feature encoder.

[0126] The sample face features refer to the face features of the second sample face image extracted by the occlusion feature encoder. After obtaining the second sample face image, each second sample face image can be input into the pre-trained occlusion feature encoder, which will then obtain the face features corresponding to each second sample face image as the sample face features.

[0127] Step S603: For each occlusion type, construct occlusion type face features corresponding to each occlusion type based on the sample face features of the second sample face image of the same occlusion type; the occlusion type face features include the first type face features corresponding to the no-occlusion type, and the second type face features corresponding to each known occlusion type.

[0128] Step S604: Construct an occlusion feature knowledge base based on the correspondence between each occlusion type and the facial features of the occlusion type.

[0129] After obtaining the sample face features corresponding to each second sample face image, the sample face features of the second sample face images contained in each occlusion type can be used to calculate the occlusion type face features corresponding to each occlusion type.

[0130] For example, for the unoccluded type, facial features of each unoccluded face image in the second sample face images can be collected, and the average value of the facial features of each unoccluded face image can be calculated as the first type of facial feature corresponding to the unoccluded type. Similarly, for the mouth and nose occlusion type, facial features of each mouth and nose occlusion type face image in the second sample face images can be collected, and the average value of the above facial features can be calculated as the second type of facial feature corresponding to the mouth and nose occlusion type.

[0131] By using the above methods, the correspondence between each occlusion type and the facial features of the occlusion type can be obtained, and the above correspondence can be stored in the occlusion feature knowledge base to complete the construction of the occlusion feature knowledge base.

[0132] In this embodiment, a pre-trained occlusion feature encoder can be used to output sample face features of sample face images of various occlusion types. Based on the sample face features, occlusion type face features of each occlusion type can be constructed, and an occlusion feature knowledge base can be constructed based on the above correspondence. This method can improve the accuracy of the data stored in the occlusion feature knowledge base.

[0133] In one embodiment, after step S102, the method may further include: if the first similarity is greater than or equal to the first preset threshold, determining that the target face image is an unoccluded face image; after step S103, the method may further include: if the maximum value among the second similarities is greater than or equal to the second preset threshold, using the known occlusion type corresponding to the maximum value as the occlusion type corresponding to the target face image.

[0134] If the first similarity is greater than or equal to the first preset threshold, it means that the target face features corresponding to the target face image are highly similar to the face features extracted from the unoccluded face image. At this time, the target face image is determined to be an unoccluded face image.

[0135] If the maximum value among the second similarities is greater than or equal to the second preset threshold, then the known occlusion type corresponding to the maximum second similarity can be used as the occlusion type corresponding to the target face image. For example, if the target face features have the highest similarity to the face features extracted from the face image with the mouth and nose occlusion type and exceed the second preset threshold, then the mouth and nose occlusion type can be used as the occlusion type of the target face image.

[0136] In this embodiment, the identification of face image occlusion type can also be completed by using the first similarity and the second similarity, which can improve the efficiency of face image occlusion type identification.

[0137] In one embodiment, a driver face occlusion detection method based on generative few-shot augmentation and contrastive learning is also provided. This method can quickly identify novel face occlusions in a local environment using a very small number of samples, balancing recognition accuracy, response speed, privacy, and system real-time performance. First, a dataset containing normal, unoccluded faces and various typical occlusion categories is constructed. On the server side, two core models are jointly trained using a contrastive learning framework: an occlusion feature encoder capable of extracting occlusion-invariant features, and a face image generation model for data augmentation. Simultaneously, an occlusion feature knowledge base covering normal faces and various known occlusion types is established. Subsequently, the trained models and knowledge base are deployed to the vehicle's infotainment system. During vehicle operation, the target face image captured by the camera is fed into the occlusion feature encoder to extract the target face features, which are then matched with various features in the occlusion feature knowledge base to determine the occlusion status. If the occlusion is identified as known or unoccluded, the corresponding safety policy is triggered; if the similarity of multiple consecutive frames is below a set threshold, it is determined to be an unknown occlusion type, and the system displays a pop-up prompt requesting user authorization to collect images. Secondly, after obtaining authorization, the system utilizes a vehicle-deployed facial image generation model to perform local generative enhancement on a small number (3-5 frames) of original sample facial images provided by the user, synthesizing a large number of diverse synthetic sample facial images. Then, an occlusion feature encoder extracts their features and calculates the mean, generating a third type of facial feature, which is then updated to the local knowledge base, achieving "zero-fine-tuning" real-time learning. Finally, the system uploads the pixel-level average image of the generated images to a cloud server for manual semantic annotation and security policy definition. Subsequently, occlusion type labels and policies are synchronized to all vehicles via OTA, forming a closed-loop end-cloud collaborative evolution of "local discovery—rapid learning—privacy uplink—cloud definition—global upgrade," significantly improving the adaptive capabilities, privacy compliance, and swarm intelligence level of the intelligent cockpit system.

[0138] The overall process of this embodiment can be described as follows: Figure 7 As shown, it can be divided into a base model training part and a real-time detection part.

[0139] The training component of the base model trains an occlusion feature encoder and a face image generation model from a constructed dataset of normal, unoccluded faces and faces of various occlusion categories. It also calculates the average feature values ​​of the occlusion feature encoder for normal, unoccluded faces and faces with different occlusion categories, and writes these values ​​into an occlusion feature knowledge base. Subsequently, the occlusion feature encoder, face image generation model, and occlusion feature knowledge base are deployed to the vehicle's infotainment system for real-time detection.

[0140] The real-time monitoring unit acquires target face images from the camera and feeds them into an occlusion feature encoder to obtain target face features. The similarity between these target face features and face features in the occlusion feature knowledge base is calculated to determine the occlusion state category. If there is no occlusion or the occlusion type is known, a predetermined security policy is executed. If the occlusion type is unknown, a pop-up window informs the user that the occlusion belongs to an unknown type and that original sample face images must be collected, and processing is only performed locally. After authorization, 3-5 original sample face images are collected at the current moment. When the vehicle is stationary and the driver is out of the vehicle, and the vehicle's system load is low, a face image generation model is used to generate synthetic sample face images with diverse styles. The occlusion-aware feature encoder obtains the face features of each synthetic sample face image. The average of these face features is used as the third type of face feature for the current occlusion category and updated in the vehicle's occlusion feature knowledge base for subsequent occlusion state recognition of new occlusion categories. Simultaneously, the average value of the synthesized sample face images is calculated, and the average face image map and the third type of face features are uploaded to the cloud. The cloud obtains the corresponding occlusion type label from the average face image map and defines the corresponding security policy. The cloud aggregates new occlusion information from all operating vehicles, adds the third type of face features to the knowledge base, and defines OTA updates to the occlusion feature knowledge base and security policies for all vehicles, completing the closed loop of edge-cloud collaborative evolution of "local discovery - rapid learning - privacy uplink - cloud definition - global upgrade".

[0141] The specific execution steps are as follows:

[0142] Step 1: Base Model Training and Knowledge Base Construction: Base model training is the prerequisite and key foundation for achieving few-shot recognition capabilities. This step aims to train two core models: an occlusion feature encoder and a face image generation model, equipping them with prior knowledge of facial structure, common occlusion patterns, and occlusion semantics. This provides a reliable starting point for incremental few-shot learning on the vehicle-mounted device. A large-scale labeled dataset containing various common occlusion types and unoccluded faces is used to train the feature encoder and face image generation model. The feature encoder learns occlusion invariant features through a contrastive learning framework, while the generation model learns the distribution patterns of facial structure and occluded regions, ensuring that the model possesses basic occlusion recognition and image generation capabilities.

[0143] (1) First, a face dataset containing various common occlusion types was constructed. This dataset covers typical occlusion situations that drivers may encounter in real cockpit environments, including but not limited to: wearing masks (mouth and nose occlusion); wearing sunglasses (eye occlusion); looking down or turning the head, causing parts of the face to be invisible (pose occlusion); and hand occlusion behaviors such as touching the face, making a phone call, or resting the chin on the hand (torso occlusion). At the same time, the dataset also contains a large number of unoccluded normal face images as negative samples in contrastive learning. All images were labeled to indicate the occlusion type and occlusion area, and preprocessed with illumination normalization, noise reduction, etc., to ensure data quality.

[0144] (2) Secondly, the base model includes an occlusion feature encoder and a face image generation model. The function of the perceptual feature encoder is feature extraction. It receives the preprocessed face image, such as a 224x224 pixel RGB matrix, and outputs a fixed-length feature vector, such as a 512-dimensional floating-point array, through a series of convolution, activation, pooling and other operations, as shown in the following formula, where n is 512.

[0145]

[0146] The goal of an occlusion feature encoder is to satisfy occlusion invariance, class discriminativeness, and robustness. Occlusion invariance is the most critical characteristic; the encoder needs to learn to ignore local appearance changes caused by occlusions and focus on the deep structure and semantic information of the unoccluded parts of the face. For example, it should be able to recognize that "a person wearing a blue mask" and "the same person wearing a white mask" are essentially the same person, as their core features are similar. Class discriminativeness, while ensuring occlusion invariance, requires the encoder to be able to clearly distinguish different types of occlusion. For example, the feature vector of "wearing a mask" should be sufficiently far away from the feature vector of "wearing sunglasses" in vector space. Robustness refers to the encoder's ability to withstand non-occlusion factors such as changes in lighting, pose changes (slight head tilt), facial expressions, and image blurring, preventing these factors from interfering with occlusion judgment.

[0147] To meet the stringent requirements of vehicle-mounted systems for computational efficiency and memory usage, the backbone network of the occlusion feature encoder model uses the lightweight convolutional neural network MobileNetV3-Small, optimized for mobile and embedded devices. With a face input size of 224*224*3 and an output feature vector of 512 dimensions, the model's computational cost is approximately 0.021 floating-point operations, making it a very lightweight model with extremely low computational overhead. This makes it highly suitable for deployment on resource-constrained edge computing platforms such as in-vehicle systems. The model structure is as follows: Figure 8 As shown, the last layer of its backbone network is a global average pooling layer, which compresses the feature map of the last convolutional layer into a one-dimensional vector, i.e., the final feature vector.

[0148] Figure 8 In the diagram, Conv2 is a 3x3 ordinary convolution, S2 is a convolution with a stride of 2, BN is feature normalization, hswish is the hard-swish activation function, Bneck is the basic network structure, avg_pool is global average pooling, and FC is fully connected computation. Detailed parameters for each layer are shown in Table 1.

[0149] Table 1. Schematic diagram of model structure parameters

[0150]

[0151] Table 1 shows the bottleneck as the basic structure of the network. SE indicates whether channel attention is used. NL represents the type of activation function, including HS (h-swish) and RE (ReLU). NBN indicates no BN operation. S indicates that the network uses a convolution stride operation for downsampling. The input face region size is 224*224*3, and the output face feature vector is a 512-dimensional feature vector.

[0152] The training method does not employ traditional supervised learning that directly predicts class labels, but instead uses a self-supervised or semi-supervised contrastive learning approach. This involves training the model by constructing "similar" and "dissimilar" sample pairs. To prevent information collapse during contrastive learning and significantly improve the quality of the learned features, a projection head is connected to the output vector of the backbone network for contrastive learning. In this embodiment, the projection head uses a two-layer MLP perceptron with two fully connected layers, such as... Figure 9 As shown. The projection head is used to map the features extracted by the backbone network to a new representation space for the computation of contrastive loss. ReLU activation and BatchNorm are used in between to enhance the non-linearity of the feature mapping. The projection head only exists during the training phase. During the model deployment and inference phases, the output features of the backbone network are already well-trained, and the projection head can be removed during inference to reduce computational cost on the edge devices.

[0153] Contrastive learning is a method that guides the model to learn what is similar to what and what is dissimilar to what. It doesn't require a large number of expensive category labels. In this embodiment, it's used to teach the model to understand "what is occlusion" and "the differences between different occlusion types." Its core support consists of positive and negative sample pairs. Positive sample pairs are samples that semantically belong to the same class or come from the same source, such as two images obtained from a photo of a face wearing a mask after random cropping and color jittering. The model needs to learn to make their feature vectors as close as possible in the embedding space. In this embodiment, this means that the feature vectors of occlusions of the same type should be as close as possible. Negative sample pairs are samples that semantically belong to different classes or come from different sources. The model needs to learn to make their feature vectors as far apart as possible in the embedding space. In this embodiment, this means that the feature vectors between normal and occluded faces, and between different occlusion types, should be as far apart as possible. Contrastive learning uses InfoNCE Loss as the loss function. A detailed description of the categories of face occlusion scenarios that InfoNCE Loss adapts to is as follows:

[0154] The first step is to define the set of occlusion types. ,in Corresponding to unobstructed faces (complete faces), Corresponds to mouth and nose obstruction (the mouth and nose area is blocked). Corresponds to eye occlusion (eye area is blocked). Corresponding to pose occlusion (part of the face is not visible due to pose changes). Corresponding to torso occlusion (the area of ​​the torso below the face occludes the face).

[0155] The second step involves defining anchor samples and features (second sample face images). (where represents the sample of the i-th identity under occlusion type c); the feature extractor outputs normalized features. The training samples are divided into positive and negative samples. Positive samples represent other samples of the same identity and the same occlusion category. Features Negative samples include two types: cross-identity negative samples, which represent samples with different identities but the same occlusion type. Secondly, samples with the same identity but different occlusion types. .

[0156] For anchor point samples Its loss function is defined as follows:

[0157]

[0158] Where P represents the number of positive samples with the same identity and the same occlusion type; N represents the number of negative samples across identities; and sim represents the cosine similarity between two feature values. Indicates the weight of occlusion type (e.g., eye occlusion) More critical for identity verification, it can be configured. ; This indicates type-specific temperature parameters, such as those for complex occlusion or attitude occlusion. configurable To enhance differentiation; For the uniform temperature parameters of the negative samples, Indicates cross-category weights, such as mouth and nose occlusion. and torso cover The differences are significant and can be set. The task of the loss function is to enable the model to accurately learn the association and distinction of facial features under different occlusion scenarios, making the facial features of the same person with the same occlusion type more similar (such as different photos of the same person wearing a mask), and making the facial features of different people or the same person with different occlusion types more distant (such as different people wearing masks, the same person wearing a mask and sunglasses), thereby helping the model to accurately identify the occlusion state in various occlusion scenarios.

[0159] (3) Then, the face image generation model serves as the core technical component for realizing "local few-shot data augmentation" and "real-time knowledge update". This model is designed specifically for the vehicle edge computing environment. Its core objective is to generate a large number of synthetic images (synthetic sample face images) with diverse styles and consistent semantics based on a small number (3-5 frames) of novel occluded face images (original sample face images) provided by user authorization with extremely low computing resource consumption, so as to solve the problem of data scarcity in few-shot learning, while ensuring user privacy and system real-time performance.

[0160] Among various generative models, this embodiment selects the Conditional Variational Autoencoder (CVAE) model. CVAE maps the input image to the distribution parameters (mean and variance) of the latent space through the encoder, and then samples and reconstructs the image from this distribution through the decoder. Introducing "conditional information" (such as occlusion category labels or the original image itself) can control the generated content. The backbone network of the model adopts the U-NET structure. The core of U-Net is a symmetrical encoder-decoder structure, which is highly compatible with the CVAE infrastructure and can perfectly support the process of "encoding input → latent representation → decoding generation". Its structure is as follows: Figure 10 As shown, its input is a 224*224*3 face region image, and the output is the generated 224*224*3 image.

[0161] The original U-Net network takes a 224*224*3 pixel as input and outputs a 224*224*3 pixel. Its computational cost is approximately 0.4 floating-point operations, making it relatively heavy and difficult to deploy on in-vehicle infotainment systems. To adapt to scenarios with limited computing power on in-vehicle systems, U-Net's ordinary convolutions use depthwise separable convolutions, such as... Figure 11As shown, to reduce the number of model parameters, after replacing convolution with depthwise separable convolution, the model computation is reduced by about 75%-80%, approximately 0.09 floating-point operations, making it very suitable for deployment on in-vehicle systems.

[0162] When using U-Net as the decoder part of a CVAE, the loss function typically consists of two parts: reconstruction loss and KL divergence. For image data, the reconstruction loss is generally expressed as mean squared error or cross-entropy loss, while KL divergence is used to measure the difference between the learned latent variable distribution and the prior distribution.

[0163]

[0164] In the formula, To reconstruct the loss, It is a hyperparameter, typically set to 0.5, used to control the balance between reconstruction loss and KL divergence. It is the KL divergence. For a 3-channel image, Using the mean squared error, as shown in the following formula:

[0165]

[0166] In the formula, It is the input image. This is an image generated by U-Net, where N is the number of pixels in the image. and and represent the true value and predicted value of the i-th pixel, respectively. KL divergence is used to ensure that the distribution of latent variables approximates a predefined prior distribution (usually a standard normal distribution), as shown in Equation 5:

[0167]

[0168] in, Denotes KL divergence, and The mean and logarithmic variance of the latent features are represented by J, which represents the number of latent features and is typically taken as 32.

[0169] The training data consists of occluded photos of people's faces (input), along with information about the type of occlusion (e.g., mask, sunglasses) and where it's obscured (using a black-and-white image of the occluded area). The model's learning objective is to reconstruct the occluded photo exactly as it appears. Through extensive learning like this, the model learns how to generate new, realistic facial images under conditions such as "wearing a mask" or "wearing sunglasses."

[0170] (4) Finally, an occlusion feature knowledge base is constructed. The feature vector of each image for each type of occlusion is extracted by the occlusion feature encoder. The feature vector of each type of occlusion is obtained by internal representation and averaging, as shown in Table 2:

[0171] Table 2. Schematic diagram of occlusion feature storage

[0172]

[0173] The occlusion feature encoder and lightweight generative model are trained on the server side, resulting in an occlusion feature knowledge base. The model and knowledge base are then deployed to the vehicle's infotainment system, and occlusion detection is enabled.

[0174] Step 2: Hierarchical Real-Time Occlusion Detection Based on Feature Space: During vehicle operation, the system continuously acquires driver facial images through the onboard camera and locates and aligns the face regions in the images. Subsequently, the aligned face image is input to the occlusion-aware feature encoder deployed on the vehicle's infotainment system to extract the face feature vector of the current frame. Cosine similarity is used to calculate the similarity between the feature vector of the current frame's face and the feature vector in the knowledge base, as defined in the following formula:

[0175]

[0176] In the formula It is a vector sum vector dot product, and They are vectors sum vector The Euclidean norm (i.e., the length of the vector). and They are vectors sum vector The i-th component. The cosine similarity ranges from [-1, 1]. The closer the value is to 1, the more similar the directions of the two vectors are. The closer the value is to -1, the more opposite the directions are. A value of 0 indicates that they are orthogonal (no correlation).

[0177] The system executes the following hierarchical decision-making process, as shown in the flowchart below. Figure 12 As shown:

[0178] (1) First stage: Normal state deviation judgment; The face feature vector (target face feature) of the current frame is compared with the "normal unoccluded face" category prototype (first type face feature) stored in the local occlusion feature knowledge base to calculate the similarity and obtain the first similarity; If the first similarity is lower than the first preset threshold, the threshold is set to 0.45, and the current face state is determined to deviate from the normal unoccluded state, and the second stage judgment is entered; otherwise, it is determined to be the normal unoccluded state and the current system function is maintained.

[0179] (2) Known occlusion category matching determination; based on the determination of "deviation from normal state", the maximum similarity value between the current frame feature vector (target face feature) and the category prototype (second type face feature) of all known face occlusion categories (including but not limited to masks, sunglasses, scarves, helmets) in the local knowledge base is calculated as the second similarity; if the second similarity is higher than or equal to the second preset threshold, the threshold is set to 0.35, then the current occlusion is determined to be one of the known occlusion categories, and the security policy corresponding to the category is triggered (such as adjusting the fatigue detection algorithm when wearing sunglasses).

[0180] Step 3: Novel Occlusion Determination and User Interaction: If the first similarity value is lower than the first preset threshold and the second similarity value is lower than the second preset threshold, the system determines the current face state as "suspected novel occlusion." In response, the system generates a pop-up prompt on the in-vehicle human-machine interface, explaining to the user that an unknown occlusion type has been detected, and requests authorization to collect 3-5 images at the current moment for optimization of local recognition capabilities. This determination result serves as a key signal to trigger subsequent local generative enhancement and knowledge base update processes.

[0181] Step 4: Local Generative Enhancement and Knowledge Base Update: The system automatically saves the most recent 3-5 clear face images after authorization as support samples (original sample face images); when the vehicle is parked and the driver is off the vehicle, and the vehicle is in low-load mode, the lightweight generative model on the vehicle is invoked to generate 100 to 500 synthetic occlusion face images (synthetic sample face images) with diverse styles (such as different poses, lighting, and expressions) using these samples as conditions; subsequently, the feature vectors of all support samples and synthetic images are extracted using the frozen occlusion feature encoder, and their arithmetic mean is calculated to form the "category prototype" of this new occlusion category; this prototype is added to the local feature knowledge base and assigned a temporary label (such as unknown_occlusion_001) to realize the real-time update of the local model and the expansion of recognition capabilities, as shown in Table 3.

[0182] Table 3. Illustration of newly added storage of occlusion features

[0183]

[0184] Step 5: Privacy-preserving data uplink and cloud-based co-evolution: The system performs pixel-level averaging on the generated synthetic image set to obtain an "average image" (average face image) that cannot reconstruct individual identities; this average image is then uploaded to the cloud server; the cloud platform aggregates such average images from multiple vehicles, where a human review team performs semantic annotation (occlusion type labels) and formulates unified risk levels and vehicle response strategies; finally, an update package containing new category standard prototypes, semantic labels, and strategies is pushed to all vehicles via OTA, achieving synchronous upgrades of the global knowledge base and completing the closed-loop evolution from individual discovery to collective intelligence.

[0185] This embodiment employs a feature similarity matching mechanism to lay the foundation for dynamic recognition and expansion. It innovatively adopts an open recognition mechanism of "extracting feature vectors → calculating similarity with various prototypes → determining based on similarity thresholds." Judgment is made by measuring the distance (e.g., cosine similarity) between the current facial features and various "average prototypes" in the knowledge base, eliminating reliance on fixed probability outputs. This not only improves the robustness of recognition for unknowns or edge occlusions, but more importantly, provides a fundamental guarantee for the dynamic expansion of the system—adding new categories only requires adding new prototypes, without modifying or retraining the model, laying a solid technical foundation for subsequent local real-time learning and OTA global upgrades. A powerful base model is constructed by training a feature encoder and facial image generation model with occlusion perception capabilities on the server side. This model, after comparative learning and generative training on a large-scale dataset, can accurately extract occlusion-invariant features and understand the face-occlusion distribution pattern, significantly improving the system's prior cognitive ability, enabling efficient recognition on the vehicle side with only a few samples. High-precision real-time occlusion detection and anomaly discovery are achieved on the vehicle's infotainment system. The system continuously monitors the similarity between feature vectors and known categories. When the similarity is below a threshold for multiple consecutive frames, it accurately identifies a "suspected novel occlusion," demonstrating rapid response and a low false alarm rate, providing a reliable foundation for subsequent learning. A face image generation model is introduced on the vehicle's infotainment system to locally enhance a small number (3-5 frames) of support samples, synthesizing images with diverse styles. This process is completed locally, expanding data diversity, improving prototype robustness, and demonstrating the advantages of edge intelligence. The local knowledge base is updated by calculating the mean of features from the new occluded image set, achieving extremely lightweight adaptation. Features are extracted using a frozen encoder; only new prototypes are calculated and stored, eliminating the need for model fine-tuning, resulting in minimal computational overhead, adapting to the vehicle's computing power limitations, and ensuring immediate effectiveness of recognition capabilities. A complete closed loop from detection and learning to response is achieved. After the knowledge base is updated, the system can immediately identify novel occlusions and, based on the risk level, trigger safety policies (such as disabling fatigue monitoring and issuing warnings), promptly and accurately preventing misjudgments and providing safety assurance for advanced intelligent driving. A privacy-preserving knowledge uplink mechanism based on "image average map" is designed. The vehicle's infotainment system only uploads an average image that cannot reveal the user's identity. The cloud-based system then manually reviews the image, defines its semantics, and formulates strategies before pushing the results to all vehicles via OTA (Over-The-Air). This mechanism transforms individual discovery into collective defense, forming a continuous evolutionary loop that becomes "smarter with use," significantly enhancing the product's intelligence level and competitiveness.

[0186] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0187] Based on the same inventive concept, this application also provides a face occlusion detection device for implementing the face occlusion detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more face occlusion detection device embodiments provided below can be found in the limitations of the face occlusion detection method described above, and will not be repeated here.

[0188] In one embodiment, such as Figure 13 As shown, a face occlusion detection device is provided, including: a face feature extraction module 1301, a first similarity acquisition module 1302, a second similarity acquisition module 1303, and an unknown occlusion recognition module 1304, wherein:

[0189] The face feature extraction module 1301 is used to acquire a target face image, input the target face image into a pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder.

[0190] The first similarity acquisition module 1302 is used to acquire the first similarity between the target face features and the first type of face features stored in the pre-built occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image;

[0191] The second similarity acquisition module 1303 is used to acquire the second similarity between the target face feature and each second type of face feature stored in the occlusion feature knowledge base when the first similarity is less than the first preset threshold; each second type of face feature is a face feature extracted from face images with different known occlusion types.

[0192] The unknown occlusion recognition module 1304 is used to determine that the target face image is a face image with unknown occlusion when the maximum value of each second similarity is less than a second preset threshold.

[0193] In one embodiment, the face occlusion detection device further includes: a knowledge base update module, configured to acquire original sample face images corresponding to unknown occlusion types; the number of original sample face images is a first number; input the original sample face images into a pre-trained face image generation model, and generate synthetic sample face images corresponding to unknown occlusion types through the face image generation model; the number of synthetic sample face images is a second number, the second number being greater than the first number; input each original sample face image and each synthetic sample face image into an occlusion feature encoder, and extract face features corresponding to each original sample face image and each synthetic sample face image through the occlusion feature encoder; construct a third type of face feature based on the face features corresponding to each original sample face image and each synthetic sample face image, and update the occlusion feature knowledge base based on the third type of face feature.

[0194] In one embodiment, the knowledge base update module is further configured to perform pixel averaging on each synthetic sample face image to obtain an average face image corresponding to an unknown occlusion type; upload the average face image to a cloud server; the cloud server is configured to obtain the occlusion type label corresponding to the average face image and return the occlusion type label; use the occlusion type label as the occlusion type corresponding to the third type of face feature, and update the occlusion feature knowledge base using the correspondence between the third type of face feature and the occlusion type.

[0195] In one embodiment, the knowledge base update module is further configured to display a prompt interface; the prompt interface is configured to request the current face image as the original sample face image; in response to the request authorization operation triggered in the prompt interface, a first number of face images are collected, and the first number of face images are used as the original sample face images.

[0196] In one embodiment, the face image generation model includes: an encoder and a decoder; a face occlusion detection device, further comprising: a first model training module, configured to acquire sample occluded face images, and occlusion type information and occlusion region information corresponding to the sample occluded face images; input the sample occluded face images, occlusion type information, and occlusion region information into the encoder of the face image generation model to be trained, extract image features corresponding to the sample occluded face images through the encoder, and fuse the image features with the occlusion type information and occlusion region information to obtain latent features corresponding to the image features; the latent features are image features incorporating occlusion type information and occlusion region information as conditional constraints; input the latent features into the decoder of the face image generation model to be trained, and generate predicted occluded face images through the decoder; construct a first loss value based on the difference between the predicted occluded face image and the sample occluded face image, and construct a second loss value based on the distribution of the latent features, and train the face image generation model using the first loss value and the second loss value.

[0197] In one embodiment, the face occlusion detection device further includes: a second model training module, configured to acquire a first sample face image, and positive and negative sample images corresponding to the first sample face image; the positive sample image is a face image with the same identity as the person in the first sample face image and the same occlusion type; the negative sample images include face images with the same identity as the person in the first sample face image but different occlusion types, and face images with different identities as the person in the first sample face image but the same occlusion type; inputting the first sample face image, the positive sample image, and the negative sample image into an occlusion feature encoder to be trained, extracting a first face feature corresponding to the first sample face image, a second face feature corresponding to the positive sample image, and a third face feature corresponding to the negative sample image through the occlusion feature encoder; obtaining a third similarity between the first face feature and the second face feature, and a fourth similarity between the first face feature and the third face feature, obtaining a loss value based on the third similarity and the fourth similarity, and training the occlusion feature encoder using the loss value.

[0198] In one embodiment, the face occlusion detection device further includes: a knowledge base construction module, configured to acquire a second sample face image and an occlusion type corresponding to the second sample face image; the occlusion type includes: no occlusion and different known occlusion types; input the second sample face image into a pre-trained occlusion feature encoder, and acquire sample face features corresponding to the second sample face image through the occlusion feature encoder; for each occlusion type, construct occlusion type face features corresponding to each occlusion type based on the sample face features of the second sample face image of the same occlusion type; the occlusion type face features include a first type face feature corresponding to the no-occlusion type and a second type face feature corresponding to each known occlusion type; and construct an occlusion feature knowledge base based on the correspondence between each occlusion type and the occlusion type face features.

[0199] In one embodiment, the face occlusion detection device further includes: an occlusion type recognition module, configured to determine that the target face image is an unoccluded face image when the first similarity is greater than or equal to a first preset threshold; and to use the known occlusion type corresponding to the maximum value among the second similarities as the occlusion type corresponding to the target face image when the maximum value among the second similarities is greater than or equal to a second preset threshold.

[0200] Each module in the aforementioned face occlusion detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0201] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a face occlusion detection method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0202] Those skilled in the art will understand that Figure 14 The structure shown is a block diagram of a partial structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0203] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0204] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0205] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0206] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program mentioned can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting face occlusion, characterized in that, The method includes: Acquire a target face image, input the target face image into a pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder; Obtain the first similarity between the target face feature and the first type of face features stored in the pre-constructed occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image; If the first similarity is less than the first preset threshold, the second similarity between the target face feature and each of the second type face features stored in the occlusion feature knowledge base is obtained; each of the second type face features is a face feature extracted from a face image with different known occlusion types. If the maximum value of each of the second similarities is less than a second preset threshold, the target face image is determined to be a face image with an unknown occlusion type. Obtain the original sample face image corresponding to the unknown occlusion type; the number of the original sample face images is a first quantity; The original sample face image is input into a pre-trained face image generation model, and the face image generation model generates a synthetic sample face image corresponding to the unknown occlusion type; the number of the synthetic sample face images is a second number, which is greater than the first number; The original sample face images and the synthetic sample face images are input into the occlusion feature encoder, and the face features corresponding to the original sample face images and the synthetic sample face images are extracted by the occlusion feature encoder. A third type of face feature is constructed based on the face features corresponding to each of the original sample face images and each of the synthesized sample face images; Pixel averaging is performed on each of the synthesized sample face images to obtain the average face image corresponding to the unknown occlusion type; The average face image is uploaded to a cloud server; the cloud server is used to obtain the occlusion type label corresponding to the average face image and return the occlusion type label. The occlusion type label is used as the occlusion type corresponding to the third type of face feature, and the occlusion feature knowledge base is updated using the correspondence between the third type of face feature and the occlusion type.

2. The method according to claim 1, characterized in that, The step of obtaining the original sample face image corresponding to the unknown occlusion type includes: A prompt interface is displayed; the prompt interface is used to request the current face image as the original sample face image; In response to the authorization request triggered in the prompt interface, a first number of face images are acquired, and the first number of face images are used as the original sample face images.

3. The method according to claim 1, characterized in that, The face image generation model includes an encoder and a decoder; the face image generation model is trained through the following steps: Obtain sample occluded face images, as well as occlusion type information and occlusion area information corresponding to the sample occluded face images; The sample occluded face image, the occlusion type information, and the occlusion region information are input into the encoder of the face image generation model to be trained. The encoder extracts the image features corresponding to the sample occluded face image, and the image features are fused with the occlusion type information and the occlusion region information to obtain the latent features corresponding to the image features. The latent features are image features that incorporate the occlusion type information and the occlusion region information as conditional constraints. The latent features are input into the decoder of the face image generation model to be trained, and the decoder generates a predicted occluded face image. A first loss value is constructed based on the difference between the predicted occluded face image and the sample occluded face image, and a second loss value is constructed based on the distribution of the latent features. The face image generation model is trained using the first loss value and the second loss value.

4. The method according to claim 1, characterized in that, The occlusion feature encoder is trained through the following steps: Obtain a first sample face image, and a positive sample image and a negative sample image corresponding to the first sample face image; the positive sample image is a face image with the same identity as the person in the first sample face image and the same occlusion type; the negative sample image includes face images with the same identity as the person in the first sample face image but different occlusion types, and face images with different identities as the person in the first sample face image but the same occlusion type. The first sample face image, the positive sample image, and the negative sample image are input into the occlusion feature encoder to be trained. The occlusion feature encoder extracts the first face feature corresponding to the first sample face image, the second face feature corresponding to the positive sample image, and the third face feature corresponding to the negative sample image. Obtain a third similarity between the first face feature and the second face feature, and a fourth similarity between the first face feature and the third face feature. Obtain a loss value based on the third similarity and the fourth similarity, and use the loss value to train the occlusion feature encoder.

5. The method according to claim 1, characterized in that, The occlusion feature knowledge base is obtained through the following steps: Obtain a second sample face image and the occlusion type corresponding to the second sample face image; the occlusion type includes: no occlusion and different known occlusion types; The second sample face image is input into the pre-trained occlusion feature encoder, and the sample face features corresponding to the second sample face image are obtained through the occlusion feature encoder. For each occlusion type, based on the sample face features of the second sample face image of the same occlusion type, occlusion type face features corresponding to each occlusion type are constructed; the occlusion type face features include the first type face features corresponding to the no-occlusion type, and the second type face features corresponding to each known occlusion type. Based on the correspondence between each occlusion type and the facial features of the occlusion type, the occlusion feature knowledge base is constructed.

6. The method according to any one of claims 1 to 5, characterized in that, After obtaining the first similarity between the target facial features and the first type of facial features stored in the pre-built occlusion feature knowledge base, the method further includes: If the first similarity is greater than or equal to the first preset threshold, the target face image is determined to be an unobstructed face image; After obtaining the second similarity between the target facial features and each of the second-type facial features stored in the occlusion feature knowledge base, the method further includes: If the maximum value among the second similarities is greater than or equal to the second preset threshold, the known occlusion type corresponding to the maximum value is taken as the occlusion type corresponding to the target face image.

7. A face occlusion detection device, characterized in that, The device includes: The face feature extraction module is used to acquire a target face image, input the target face image into a pre-trained occlusion feature encoder, and extract the target face features corresponding to the target face image through the occlusion feature encoder. The first similarity acquisition module is used to acquire the first similarity between the target face features and the first type of face features stored in the pre-built occlusion feature knowledge base; the first type of face features are face features extracted from an unoccluded face image; The second similarity acquisition module is used to acquire the second similarity between the target face feature and each second type of face feature stored in the occlusion feature knowledge base when the first similarity is less than the first preset threshold; each second type of face feature is a face feature extracted from face images of different known occlusion types. An unknown occlusion recognition module is used to determine that the target face image is a face image with unknown occlusion when the maximum value of each of the second similarities is less than a second preset threshold. The knowledge base update module is used to acquire original sample face images corresponding to the unknown occlusion type; the number of original sample face images is a first number; the original sample face images are input into a pre-trained face image generation model, and the face image generation model generates synthetic sample face images corresponding to the unknown occlusion type; the number of synthetic sample face images is a second number, which is greater than the first number; each of the original sample face images and each of the synthetic sample face images are input into the occlusion feature encoder, and the occlusion feature encoder extracts each of the original sample face images and each of the synthetic sample face images. The process involves: constructing a third type of face feature based on the face features corresponding to each original sample face image and each synthesized sample face image; performing pixel averaging on each synthesized sample face image to obtain an average face image map corresponding to the unknown occlusion type; uploading the average face image map to a cloud server; the cloud server obtaining the occlusion type label corresponding to the average face image map and returning the occlusion type label; using the occlusion type label as the occlusion type corresponding to the third type of face feature, and updating the occlusion feature knowledge base using the correspondence between the third type of face feature and the occlusion type.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images