Face recognition detection method and apparatus, computer device, and program product

Abstract

The application is suitable for the technical field of computer vision and three-dimensional data processing, and provides a face recognition detection method, device, computer equipment and program product. The method comprises the following steps: obtaining a shielding metric corresponding to each first data point in a three-dimensional face point cloud; performing a shielding adaptive graph convolution operation on the three-dimensional face point cloud based on the shielding metric corresponding to each first data point, to generate an initial feature map; wherein the initial feature map comprises shielding points and non-shielding points; for the shielding points in the initial feature map, combining a preset face geometry prior model and a feature manifold composed of non-shielding points, performing feature completion on the shielding points, to generate a complete feature map; performing face recognition detection based on the complete feature map, to determine a face identity, thereby improving the face recognition accuracy in a shielding scene.

Description

Technical Field

[0001] This application belongs to the field of computer vision and 3D data processing technology, and in particular relates to a face recognition detection method, device, computer equipment and program product. Background Technology

[0002] 3D face recognition technology is an important research direction in the field of computer vision. In practical applications, the acquired 3D face point cloud data is often incomplete due to occlusion by objects such as masks, hats, hair, and glasses, resulting in the loss of key geometric and texture features of the face, which poses a significant challenge to existing 3D face recognition methods.

[0003] On the one hand, traditional feature extraction methods, such as fixed graph convolutional networks, struggle to extract effective information from regions with missing features when processing incomplete data, resulting in low-quality generated feature representations.

[0004] On the other hand, some existing technologies attempt to fill in missing data before recognition. For example, they utilize the natural symmetry of the face to repair occluded areas, that is, using facial data from the unoccluded side and filling in the occluded side through a mirroring operation. However, this method is only effective for ideal symmetrical occlusion. It is ineffective for asymmetrical occlusion, such as wearing only one pair of glasses, head tilt, or side profile, and may even introduce more noise due to incorrect filling, interfering with the recognition results. Summary of the Invention

[0005] This application provides a face recognition detection method, device, computer equipment, and program product, which can solve the technical problem in the prior art that when processing occluded 3D face point clouds, the feature extraction process lacks the ability to perceive occlusion, and the feature completion result is prone to deviating from the real physiological structure, resulting in low face recognition accuracy.

[0006] In a first aspect, embodiments of this application provide a face recognition detection method, including:

[0007] Obtaining 3D face point clouds First data points in each of the middle Corresponding occlusion metric ;

[0008] Based on each first data point Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ; wherein, the initial feature map Including occluded points and unoccluded points;

[0009] For the initial feature map The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ;

[0010] Based on the complete feature map Facial recognition detection is performed to determine the identity of the person.

[0011] Optionally, the acquisition of three-dimensional face point clouds First data points in each of the middle Corresponding occlusion metric ,include:

[0012] The three-dimensional face point cloud Perform normalization processing;

[0013] Normalized 3D face point cloud The input is fed into a preset occlusion prediction network, and the output is a normalized 3D face point cloud. First data points in each of the middle The corresponding occlusion probability, and the first data points The occlusion probability is used as the first data point Occlusion metric .

[0014] Optionally, the first data point Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ,include:

[0015] For 3D face point cloud Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ;in, For dynamic neighborhood The second data point in;

[0016] For each first data point Based on this first data point Occlusion metric The first data point Its dynamic neighborhood The second data point within The correlation metric parameters are used to calculate the first data point. With the second data point Information transfer weight between ;

[0017] Based on each first data point dynamic neighborhood And information transmission weight For each first data point Perform graph convolution operations to aggregate their dynamic neighborhoods. Second data point The features are used to obtain each first data point. Update features;

[0018] All first data points The updated features are integrated to generate an initial feature map. .

[0019] Optionally, the above refers to 3D face point clouds. Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ,include:

[0020] For each first data point Based on the first data point Occlusion metric The first data point is determined by a preset neighborhood radius calculation formula. Neighborhood radius during graph convolution operation The formula for calculating the neighborhood radius is as follows: , The preset basic neighborhood radius, Used to limit occlusion measurement for neighborhood radius Maximum contribution, avoiding neighborhood radius Measurement due to occlusion Too large and excessively enlarged;

[0021] With the first data point Center, neighborhood radius With radius, in the 3D face point cloud Determine the initial spatial neighborhood;

[0022] Based on the first data point and each second data point in the initial spatial neighborhood The geometric features are used to perform geometric consistency screening on the initial spatial neighborhood, including:

[0023] Curvature filtering: Calculate the first data point curvature Second data point curvature , retain satisfaction The second data point ;

[0024] Normal filtering: Calculate the first data point unit normal Second data point unit normal , retain satisfaction The second data point with an angle greater than the preset value ;

[0025] If the first data point Occlusion metric If the occlusion metric is greater than the first preset occlusion metric, then in the initial spatial neighborhood after geometric consistency filtering, the occlusion metric will be... The second data point is less than or equal to the second preset occlusion metric. As the first data point dynamic neighborhood ;

[0026] If the first data point Occlusion metric If the occlusion metric is less than or equal to the first preset occlusion metric, then the initial spatial neighborhood after geometric consistency filtering is directly used as the first data point. dynamic neighborhood .

[0027] Optionally, the association metric parameter includes geometric distance. Feature similarity;

[0028] For each first data point Based on this first data point Occlusion metric The first data point Its dynamic neighborhood The second data point within The correlation metric parameters are used to calculate the first data point. With the second data point Information transfer weight between ,include:

[0029] Based on the geometric distance and the feature similarity, combined with the first data point Second data point Occlusion relation factors determined by the occlusion relation The first data point was calculated. With the second data point Information transfer weight between .

[0030] Optionally, the training process of the face geometric prior model includes:

[0031] Collect a number of unobstructed 3D face data to form a training sample set;

[0032] Individual facial difference features and expression change features are obtained from the training sample set;

[0033] Statistical analysis was performed on the training sample set to calculate the average face model. ;

[0034] Principal component analysis was used to perform dimensionality reduction extraction of the individual facial difference features to obtain the identity basis vector. ;

[0035] Principal component analysis was used to extract the dimensionality-reducing features of the facial expressions, resulting in expression basis vectors. ;

[0036] Based on the above average face model Identity basis vectors and expression basis vectors Construct a function to represent the shape of a human face: ;in, The shape of a human face, For identity parameters, For facial expression parameters;

[0037] For facial expression parameters Apply L2 norm constraints;

[0038] In the shape of a human face The coordinate error between the actual face shape and the coordinates in the training sample set is used as the optimization objective to construct a loss function;

[0039] Minimize the loss function to obtain the face geometric prior model.

[0040] Optionally, the initial feature map The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ,include:

[0041] Obtain the initial feature map The features of the unoccluded points in the dataset are used to form a set of unoccluded point features; wherein, the unoccluded point is a first data point that satisfies a preset unoccluded condition. ;

[0042] Based on the set of unoccluded point features, a low-dimensional feature manifold is constructed using the Isomap algorithm;

[0043] Based on the real face coordinates of the feature manifold and unoccluded points, a multilayer perceptron is trained as a mapping function.

[0044] Based on each first data point Corresponding occlusion metric For each first data point Mark whether a point is an occlusion point and generate an initial occlusion mask;

[0045] The initial occlusion mask is optimized by removing occlusion points at the edge of the initial occlusion mask that have similar features to the non-occlusion points in the local spatial neighborhood of the occlusion point at that edge, thus obtaining the optimized occlusion mask.

[0046] For each occlusion point in the optimized occlusion mask, select the non-occlusion points in the local spatial neighborhood of the occlusion point. Through the trained mapping function, map the features of the non-occlusion points in the local spatial neighborhood to the parameters of the face geometry prior model. Then, take the average value of the parameters based on the number of non-occlusion points in the local spatial neighborhood to obtain the parameter estimate of the face geometry prior model.

[0047] The parameter estimates are input into the face geometry prior model to generate physical constraint features for each occluded point in the optimized occlusion mask. Simultaneously, based on the constructed feature manifold, the geodesic distance between the occluded point and the non-occluded points in the local spatial neighborhood of the occluded point is calculated on the manifold. The attention weight is calculated based on the geodesic distance. The features of the non-occluded points in the local spatial neighborhood of the occluded point are weighted and aggregated through the attention weight to generate the context features of the occluded point.

[0048] Based on the physical constraint features and context features of each occlusion point, the completion features of the occlusion point are obtained through weighted fusion;

[0049] The completed features of all occluded points are merged with the features of unoccluded points to generate a complete feature map. .

[0050] Secondly, embodiments of this application provide a face recognition detection device, comprising:

[0051] The acquisition module is used to acquire 3D face point clouds. First data points in each of the middle Corresponding occlusion metric ;

[0052] The graph convolution module is used to perform graph convolution based on each first data point. Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ; wherein, the initial feature map Including occluded points and unoccluded points;

[0053] The feature completion module is used to complete the initial feature map. The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ;

[0054] The identification and detection module is used to identify and detect based on the complete feature map. Facial recognition detection is performed to determine the identity of the person.

[0055] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face recognition detection method described in any one of the first aspects above.

[0056] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the face recognition detection method described in any one of the first aspects.

[0057] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to execute the face recognition detection method described in any one of the first aspects.

[0058] The beneficial effects of the embodiments in this application compared with the prior art are:

[0059] By combining a pre-defined facial geometric prior model and a feature manifold composed of unoccluded points, feature completion is performed on the occluded points. Compared with existing technologies that rely solely on simple symmetry, the completed features generated in this application not only conform to the physiological laws of the face but also have a high degree of consistency with the features of the individual's visible area, thereby generating high-quality complete features and significantly improving the accuracy of face recognition in occluded scenarios.

[0060] Furthermore, an occlusion-aware processing mechanism is introduced in the feature extraction stage. By dynamically adjusting the neighborhood and information transmission weights, the feature extraction process can actively adapt to occlusion, aggregating more information from reliable visible areas for highly occluded points, improving the robustness of the initial features, and laying a solid foundation for subsequent high-quality completion.

[0061] Furthermore, the completion mechanism of this invention does not rely on the symmetry assumption, and therefore can effectively handle various symmetrical and asymmetrical occlusions, such as masks, unilateral hair occlusion, large-angle poses, etc., and has stronger robustness and wider applicability to real-world scenarios.

[0062] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart of a face recognition detection method provided in an embodiment of this application;

[0065] Figure 2 This is a schematic diagram of the structure of a face recognition detection system provided in an embodiment of this application;

[0066] Figure 3 This is a schematic diagram of the occlusion adaptive graph convolution operation provided in an embodiment of this application;

[0067] Figure 4 A schematic diagram illustrating the working principle of feature completion provided in the embodiments of this application;

[0068] Figure 5 A schematic diagram illustrating data interaction between modules in the system provided in this application embodiment;

[0069] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0070] Figure labeling: 10 - Acquisition module; 20 - Graph convolution module; 30 - Feature completion module; 40 - Recognition module; S100 - Acquisition of occlusion metric step; S200 - Feature extraction step for occlusion perception; S300 - Feature completion step; S400 - Face recognition detection step; 301 - Center point; 303 - Dynamic neighborhood; 304 - Occlusion point; 305 - Unoccluded point; 306 - Information transfer weight; 401 - Occlusion point; 402 - Unoccluded point; 403 - Face geometric prior model; 404 - Feature manifold; 405 - Physical constraint feature; 406 - Contextual feature; 407 - Complete feature; 408 - Fusion; 501 - 3D face point cloud; 502 - Occlusion metric; 503 - Initial feature map; 504 - Complete feature map; 505 - Face identity. Detailed Implementation

[0071] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0072] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0073] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0074] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0075] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0076] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0077] Example 1

[0078] This embodiment details the complete process and system implementation of a face recognition detection method. (Refer to...) Figure 1 The method mainly includes an occlusion measurement step S100, an occlusion perception feature extraction step S200, a feature completion step S300, and a face recognition detection step S400. Accordingly, refer to... Figure 2 and Figure 5 A system for implementing this method may include an acquisition module 10, a graph convolution module 20, a feature completion module 30, and a recognition module 40. These modules work together to process the entire process from the input 3D face point cloud 501 to the output final face identity 505.

[0079] In the step S100 of acquiring occlusion measurement, the system receives a set of first data points with three-dimensional coordinates, namely a three-dimensional face point cloud 501, which represents the geometric shape of the face. To eliminate scale differences caused by factors such as acquisition equipment and distance, the acquisition module 10 first normalizes the input three-dimensional face point cloud 501. As a specific implementation, the three-dimensional coordinates of each point can be scaled based on the maximum coordinate value of the point cloud in the vertical direction (usually the Y-axis). For example, for any first data point in the point cloud... Its normalized coordinates can be calculated as follows: ,in, It represents the maximum value of the Y-coordinate of all points in the entire point cloud. Understandably, this process ensures scale consistency in subsequent network processing.

[0080] After normalization, the acquisition module 10 feeds the processed 3D face point cloud into a pre-trained occlusion prediction network. This network can be a lightweight convolutional neural network, for example, composed of several stacked 1x1 convolutional layers and max-pooling layers, designed to efficiently predict the probability of occlusion for each first data point in the point cloud. The network then predicts the occlusion probability for each first data point. Output a scalar value between 0 and 1, which represents the occlusion probability of that point. In one embodiment of this application, this occlusion probability is directly used as the first data point. Occlusion metric 502 (also known as occlusion metric) A higher occlusion metric (e.g., close to 1) indicates that the point is highly likely to be located in an area obscured by objects such as masks or hair, while a lower occlusion metric (e.g., close to 0) indicates that the point is located in a clearly visible facial area. By calculating a continuous and fine-grained occlusion metric 502 for each point, the system can accurately quantify the reliability of information at each location in the point cloud, thus providing crucial guiding information for subsequent differential processing.

[0081] The specific structure of the occlusion prediction network can be as follows: Input layer (receives normalized 3D point cloud coordinates, dimension N×3, where N is the total number of the first data points) → 1×1 convolutional layer 1 (output dimension N×64, activation function ReLU) → max pooling layer (pooling kernel size 2) → 1×1 convolutional layer 2 (output dimension N×12, activation function ReLU) → batch normalization layer → 1×1 convolutional layer 3 (output dimension N×1, activation function Sigmoid) → output layer (outputs the occlusion probability of each point). The network training uses the cross-entropy loss function.

[0082] Among them, occlusion measurement It is the first data point of the quantized 3D face point cloud. A scalar value representing the degree to which something is obscured by external objects (such as masks, hair, glasses, etc.), ranging from [0, 1]. This indicates that the point is completely unobstructed and the facial information is 100% reliable; This indicates that the point is completely obscured and the facial information is completely unreliable; This indicates that the point is in a partially occluded state; the larger the value, the lower the reliability of the information. This metric provides a core basis for subsequent dynamic neighborhood adjustment, information transmission weight calculation, and occlusion mask generation.

[0083] Subsequently, the process proceeds to the occlusion-aware feature extraction step S200, executed by the graph convolution module 20. This step aims to generate an initial feature map 503 based on the original 3D face point cloud 501 and the occlusion metric 502 obtained in the previous step. Unlike traditional graph convolutional networks that use a fixed neighborhood and the same weight calculation logic for all points, the graph convolution module 20 in this embodiment performs an occlusion-adaptive graph convolution operation. Its core idea is to dynamically adjust the information aggregation method according to the occlusion metric of each point. Specifically, this operation includes two aspects: dynamically adjusting the neighborhood range and dynamically calculating the information transfer weights.

[0084] Please see Figure 3This figure illustrates the principle of occlusion-adaptive graph convolution operation. For each first data point (i.e., center point 301) in the 3D face point cloud, which serves as the information aggregation center, the graph convolution module 20 first determines a dynamic neighborhood 303 (also referred to as a dynamic neighborhood). Specifically, for center point 301, its neighborhood search radius... It is not a fixed value, but is dynamically calculated using a preset neighborhood radius calculation formula, which is: In this formula, It is the dynamic neighborhood radius of the center point 301. It is a preset base neighborhood radius (e.g., it can be set to 0.05 meters), which defines the base receptive field size of the unoccluded point. This is the occlusion metric at center point 301. It should be noted that... Part of its function is to limit the maximum contribution of occlusion metric to the neighborhood radius, thus preventing the neighborhood radius from being excessively increased due to abnormally large occlusion metrics at individual points, thereby ensuring the stability of the calculation. The intuitive effect of this formula is that when center point 301 is an occluded point 304 (i.e.... When the value is large, its neighborhood radius It will increase accordingly, enabling it to cross neighboring, equally obscured areas to connect and aggregate more distant, more reliable unobscured points 305.

[0085] In After determining the initial spatial neighborhood for the radius, to ensure that all points within the neighborhood belong to the same local surface and to avoid aggregating irrelevant information from different facial regions (such as the nose and cheeks), the graph convolution module 20 also performs geometric consistency filtering on the initial spatial neighborhood. This filtering includes curvature filtering and normal filtering. Curvature filtering requires that the difference between the curvature of the second data point in the neighborhood and the curvature of the center point 301 is less than a preset threshold; normal filtering requires that the dot product of the unit normal vector of the second data point in the neighborhood and the unit normal vector of the center point 301 is greater than a preset angle value.

[0086] The threshold value for the dot product of the unit normal vectors corresponding to the preset angle value is preferably 0.85. According to the local surface characteristics of the three-dimensional face, when the dot product of the unit normal vectors of two points is ≥0.85, the angle between their normals is ≤31.79°, indicating that the two points belong to the same continuous surface (e.g., the same cheek or the same area of ​​the forehead).

[0087] Among them, the threshold for curvature filtering In the middle, curvature Calculated using the average curvature (formula is) , , (Main curvature), this threshold ensures that the curvature of neighboring points is consistent with that of the center point, avoiding including the bridge of the nose (high curvature) in the neighborhood of the cheek point (low curvature).

[0088] After geometric consistency screening, the system performs a final neighbor selection step based on the occlusion metric of center point 301. If the occlusion metric of center point 301 is greater than a first preset occlusion metric (e.g., 0.6), indicating that it is a high-occlusion point, the system selects only the second data points in its geometrically consistent neighborhood whose occlusion metric is less than or equal to a second preset occlusion metric (e.g., 0.3) as its final dynamic neighborhood 303. This forces high-occlusion points to aggregate information only from reliable non-occluded points 305. Conversely, if the occlusion metric of center point 301 is less than or equal to the first preset occlusion metric, indicating that it is a reliable non-occluded point, all neighbor points that have passed geometric consistency screening are directly used as its dynamic neighborhood 303.

[0089] The first preset occlusion metric is set to 0.6, and the second preset occlusion metric is set to 0.3. These values ​​are obtained through statistical analysis of multiple sets (e.g., 1000 sets) of 3D face samples containing occlusions caused by masks, hair, and glasses. When the probability of a data point being truly occluded exceeds 92%, it is judged as a highly occluded point; when When the probability of a data point being unobstructed exceeds 95%, it is determined to be a reliable unobstructed point. This threshold setting can balance the accuracy and robustness of occlusion judgment.

[0090] Among them, dynamic neighborhood 303 is for the first data point. A dynamically generated set of local points used for graph convolution information aggregation, distinct from the fixed-radius neighborhood of traditional graph convolution—its neighborhood range (neighborhood radius) (From the first data point) Occlusion metric The decision, and the second data point in the neighborhood. It needs to pass geometric consistency screening (curvature and normal matching) and occlusion reliability screening (non-occlusion or low occlusion). The core function of dynamic neighborhood is to allow high-occlusion points to aggregate more information from distant reliable non-occlusion points, and to allow non-occlusion points to aggregate information from locally geometrically consistent points, thus avoiding interference from invalid occlusion information.

[0091] This application employs a two-stage filtering method for dynamic neighborhood construction (first geometric consistency filtering, then occlusion reliability filtering), which offers significant advantages over existing technologies (which filter neighborhoods solely based on distance). Existing technologies tend to include occluded points on different surfaces (e.g., a forehead point obscured by hair and an unoccluded cheek point) in the same neighborhood, leading to information aggregation noise. In contrast, this application first excludes points on different surfaces through geometric consistency filtering, and then excludes highly occluded points through occlusion reliability filtering, ensuring that only reliable points on the same surface with low occlusion are retained within the neighborhood. This improves the robustness of the initial features extracted by graph convolution.

[0092] After determining the dynamic neighborhood 303, the graph convolution module 20 calculates the information transfer weight 306 for each second data point within the center point 301 and its dynamic neighborhood 303. This information transfer weight... Also modulated by the occlusion metric, its calculation is based on association metric parameters (including geometric distance and feature similarity) and the occlusion relationship between two points. The specific calculation process is as follows:

[0093] Step a1, for each first data point Calculate the first data point Its dynamic neighborhood Each second data point within geometric distance ;

[0094] Step a2, based on the geometric distance and preset distance attenuation coefficient The geometric distance weight is calculated using the distance weight formula. The distance weight formula is: ;

[0095] Step a3, for each first data point Calculate the first data point Its dynamic neighborhood Each second data point within Feature similarity ;in, The first data point eigenvectors, For the second data point eigenvectors;

[0096] Step a4: The feature similarity is calculated using a similarity weighting formula. Mapping to the [0, 1] interval yields the feature similarity weights. The similarity weight formula is as follows: ;

[0097] Step a5, based on the first data point Occlusion metric Second data point Occlusion metric Determine the occlusion relationship between the two;

[0098] Step a6: Obtain the occlusion relationship factor based on the preset factor rule. ;

[0099] Step a7, based on the geometric distance weights obtained above Feature similarity weights and occlusion relation factor The first data point is obtained by calculating using the weighted product formula. With the second data point Information transfer weight between The formula for the weighted product is: .

[0100] In this embodiment, firstly, the geometric distance weight is calculated. This is applied to the center point 301 and the second data points within its neighborhood. Calculate the Euclidean geometric distance between the two. And through the distance weight formula The geometric distance weights are calculated, where, This is a preset distance decay coefficient, which gives higher weights to points that are closer together. Next, the feature similarity weights are calculated. The feature vector of center point 301 is then calculated. With the second data point eigenvectors (The feature vectors here come from the output or initial input of the previous layer's graph convolution) cosine similarity between And through the similarity weight formula The similarity values ​​are mapped to the [0,1] interval to obtain the feature similarity weight, which ensures that points with more similar features contribute more to information transmission. Then, occlusion measurement is performed based on center point 301. Second data point Occlusion metric The occlusion relationship factor is determined according to the preset factor rules. For example, when an unobstructed point transmits information to an obstructed point ( , ), It can be set to a value greater than 1 (such as 1.5) to enhance the reliable transmission of information, corresponding to Figure 3 Weight enhancement in the middle; when information is transmitted between two occluded points ( , ), It can be set to a value less than 1 (such as 0.5) to suppress the propagation of noise, corresponding to Figure 3 Weight suppression in; in other cases, It can be set to 1. Finally, using the weighted product formula... Multiplying the three weight components above, we get the final information transmission weight 306.

[0101] Based on the dynamic neighborhood 303 calculated for each point and the corresponding information transfer weights 306, the graph convolution module 20 performs a graph convolution operation on each first data point, updating its own features by weighted aggregation of the features of the second data points within its dynamic neighborhood. This process can be represented as follows: ,in, and These are the learnable weight matrices and bias terms in a graph convolutional layer. It is the first data point The dynamic neighborhood, ReLU is a non-linear activation function, plus This represents a residual connection. By stacking multiple layers of such occlusion-adaptive graph convolutional layers, information can be effectively propagated in the point cloud. Ultimately, the updated features of all the first data points are integrated to form an initial feature map 503. In this initial feature map 503, the features of non-occluded points are relatively accurate, while the features of occluded points, although robustly aggregated, may still have incomplete information.

[0102] Next, the process enters the core feature completion step S300, which is executed by the feature completion module 30. Its working principle is as follows: Figure 4 As shown. The goal of this step is to complete the features of the occluded points 401 in the initial feature map 503 using a dual constraint mechanism, ultimately generating a high-quality complete feature map 504. The two constraints are the global physical constraints from the face geometry prior model 403, and the local context constraints from the feature manifold 404 composed of unoccluded points.

[0103] First, the feature completion module 30 requires a pre-defined face geometry prior model 403. The training process of this pre-defined face geometry prior model 403 includes:

[0104] Step b1: Collect several unobstructed 3D face data to form a training sample set;

[0105] Step b2: Obtain individual facial difference features and expression change features from the training sample set;

[0106] Step b3: Perform statistical analysis on the training sample set to calculate the average face model. ;

[0107] Step b4: Principal component analysis is used to perform dimensionality reduction extraction on the individual facial difference features to obtain the identity basis vector. ;

[0108] Step b5: Principal component analysis is used to extract the dimensionality reduction features of the expression to obtain the expression basis vectors. ;

[0109] Step b6, based on the above average face model Identity basis vectors and expression basis vectors Construct a function to represent the shape of a human face: ;in, The shape of a human face, For identity parameters, For facial expression parameters;

[0110] Step b7, adjust expression parameters Apply L2 norm constraints;

[0111] Step b8, in the shape of a human face The loss function is constructed with the coordinate error of the face shape corresponding to the actual face shape in the training sample set as the optimization objective: ;in, It is the face shape generated for the m-th training sample. It is the real face shape of the m-th training sample. It is the identity parameter corresponding to the m-th training sample. These are the facial expression parameters corresponding to the m-th training sample. It is the sum of squared Euclidean distances between corresponding points of the generated face shape and the real face shape;

[0112] Step b9: Minimize the loss function to obtain the face geometric prior model.

[0113] In this embodiment, the model is a parameterized 3D face model capable of generating 3D face shapes that conform to human facial anatomy and expression patterns using a set of low-dimensional parameters. The model itself can be characterized as a function: .in, It is the generated face shape; It is a pre-calculated average face model; and These are the identity basis vector and the expression basis vector, which are usually extracted by dimensionality reduction methods such as principal component analysis on a large-scale unobstructed 3D face database. They encode the identity differences between individuals (such as face shape and facial feature proportions) and the expression changes within individuals (such as joy, anger, sorrow, and happiness), respectively. For identity parameters, The expression parameters are low-dimensional parameters that control facial identity and expression. This model provides global and structurally reasonable constraints for feature completion.

[0114] The specific process of feature completion is as follows: First, the feature completion module 30 extracts features from all non-occluded points 402 (i.e., points with an occlusion metric less than or equal to a preset threshold, such as 0.3) from the initial feature map 503, forming a non-occluded point feature set. Since these features all correspond to real and valid facial information, their distribution in the high-dimensional feature space is not random, but rather contains an inherent low-dimensional structure. The system can use manifold learning algorithms such as Isomap to construct a low-dimensional feature manifold 404 based on this non-occluded point feature set. This manifold can be regarded as a map of the individual's valid facial features, reflecting the nonlinear structural relationships between features.

[0115] Among them, the feature manifold is a nonlinear structural mapping of high-dimensional unoccluded point features in a low-dimensional space. The high-dimensional features of unoccluded points (such as geometric coordinates and texture features) follow the local structural correlations of the face (e.g., the strong correlation between the features of the corner of the eye, the eyelid, and the brow bone), and their distribution forms a continuous low-dimensional surface, i.e., the feature manifold. This manifold can accurately reflect the intrinsic correlations between the features of unoccluded points (e.g., points that are close together have small geodesic distances on the manifold, indicating high feature similarity), providing local structural constraints for the generation of contextual features for occluded points.

[0116] The specific parameters of the Isomap algorithm are set as follows: number of nearest neighbors. (A neighborhood graph is constructed by selecting the 10 nearest neighbors of each unoccluded point, which conforms to the correlation of local facial features.) The dimension of the low-dimensional space after dimensionality reduction is 20. This is determined through reconstruction error analysis: when the dimension is ≥20, the reconstruction error of the manifold for the features of high-dimensional unoccluded points is ≤5%, which can both preserve the core structure of the features and reduce the subsequent computational complexity. The geodesic distance calculation adopts Dijkstra's algorithm (for connected components in the neighborhood graph) to ensure that the distance calculation conforms to the nonlinear structure of the manifold.

[0117] Secondly, the system utilizes a pre-defined mapping function (e.g., a pre-trained multilayer perceptron) that maps the feature vector of an unoccluded point to the parameters of a face geometric prior model 403 that best reconstructs the geometry of that point and its surrounding region. .

[0118] The structure of the multilayer perceptron is as follows: the input layer receives high-dimensional features of unoccluded points, with dimension D, and D is compared with the initial feature map. The network training process involves several layers: Hidden Layer 1 (256 neurons, ReLU activation function) → Hidden Layer 2 (128 neurons, ReLU activation function) → Batch Normalization Layer → Output Layer (output dimension is 51, corresponding to the parameters of the face geometry prior model: identity parameters (50-dimensional) + expression parameters (1-dimensional)). The network training uses a mean squared error loss function (the loss is the coordinate error between the reconstructed face shape and the real face shape generated by the mapped model parameters). The training dataset consists of multiple sets of unoccluded 3D face non-occluded point features and corresponding real model parameters.

[0119] Furthermore, to make the boundaries of the completed region smoother and more natural, the system also optimizes the initial binary occlusion mask generated based on the occlusion metrics of each point. Specifically, the system checks the occlusion points located at the edges of the mask. If the initial features of an edge occlusion point are highly similar to the features of non-occluded points in its local spatial neighborhood, it is considered that the point is likely to be incorrectly marked as an occlusion point and is removed from the set of occlusion points, thus obtaining an optimized occlusion mask.

[0120] Existing occlusion masks generate binary results (0 / 1) based solely on occlusion metrics, which can easily lead to edge misjudgments (e.g., non-occluded points at the junction of the cheek edge and hairline are misjudged as occluded points). This application, however, optimizes the feature similarity by comparing the feature similarity of occluded points at the mask edges with their neighboring non-occluded points (e.g., a cosine similarity ≥ 0.85 is considered a misjudgment), thus eliminating misjudged points. This optimization reduces the misjudgment rate of the occlusion mask, avoids incorrect completion of non-occluded points during subsequent feature completion, and reduces unnecessary computation.

[0121] Then, for each occlusion point 401 in the optimized occlusion mask, double-constraint completion is performed. On one hand, physical constraint features 405 are generated. The system selects the non-occluded points in the local spatial neighborhood of the occlusion point 401, and uses the aforementioned preset mapping function to map the features of these neighborhood non-occluded points to the parameters of the face geometric prior model. Then, the average of all the obtained parameters is taken to obtain a robust parameter estimate. This estimate This represents the identity and expression of the entire face inferred from the visible portion. Inputting this parameter estimate into the face geometry prior model 403 generates a complete 3D face that conforms to biological norms. Features corresponding to the current occlusion point 401 are extracted from this generated face; these are the physical constraint features 405. This feature ensures that the completed result is a natural face globally. On the other hand, contextual features 406 are generated. Based on the constructed feature manifold 404, the system calculates the geodesic distance on the manifold between the current occlusion point 401 (using its features in the initial feature map) and the unoccluded points in its local spatial neighborhood. It is understood that geodesic distance reflects the intrinsic similarity between features better than Euclidean distance. Attention weights are calculated based on this geodesic distance (e.g., higher weights for closer distances), and then these attention weights are used to weighted aggregate the features of the unoccluded points in the neighborhood. The aggregated result is the contextual feature 406. This feature ensures that the completed result maintains a high degree of consistency with the local feature style and details of the currently visible portion of the face.

[0122] Among them, physical constraint feature 405 is an occlusion point feature that conforms to the physiological structure of the human face, generated based on a prior model of facial geometry. Its core constraint lies in global reasonableness—model parameters (identity parameters) inferred from neighboring non-occluded points. Expression parameters This ensures that the generated occlusion point features are consistent with the overall geometry of the face (e.g., the ratio of nose height to nose bridge width, and the relationship between lip thickness and chin contour), avoiding unnatural results such as a disconnect between nose features and cheek features.

[0123] Contextual features are occluded point features generated based on feature manifolds and that fit the style of the local neighborhood features of the occluded point. The core constraint is local consistency—attention weights are calculated using geodesic distance (rather than Euclidean distance) on the manifold to ensure that the generated occluded point features are highly matched with the features of the surrounding unoccluded points (e.g., the texture style and curvature features of an occluded point at the corner of the eye and the surrounding unoccluded eyelid points), avoiding the problem of abrupt changes in local features (e.g., a sudden appearance of rough texture in a smooth cheek).

[0124] Finally, through fusion step 408, the physical constraint feature 405 and the context feature 406 are weighted and fused to obtain the final completion feature 407 of the occlusion point. The fusion formula can be... ,in, It is a hyperparameter used to balance the importance of global and local constraints. Physical constraint characteristics, For contextual features, the initial features of all occluded points are replaced with their corresponding complete features 407, and then merged with the original features of all non-occluded points 402 to generate the final, high-quality complete feature map 504.

[0125] The final step of this application's method is the face recognition detection step S400, executed by the recognition module 40. The recognition module 40 receives the complete feature map 504 output by the feature completion module 30. Since the feature map is now complete and high-fidelity, the recognition module 40 can operate on near-ideal data. Specifically, it first applies a global feature aggregation operation (such as average pooling or max pooling) to the complete feature map 504, compressing it into a fixed-dimensional global face feature vector. Then, it compares this feature vector with the feature vectors of known identities pre-stored in the database, calculating the distance or similarity between them (such as Euclidean distance or cosine similarity). Finally, the identity with the smallest distance and less than a preset judgment threshold is taken as the recognition result, i.e., the face identity 505, and output to the user or subsequent applications.

[0126] Example 2

[0127] As an optional implementation, the occlusion-aware feature extraction step S200 in Embodiment 1 can also be implemented using other technical solutions. In Embodiment 1, the graph convolution module 20 uses a specially designed occlusion-adaptive graph convolutional network. In this embodiment, the occlusion-aware feature extraction function can be implemented through other advanced point cloud processing network architectures, such as attention-based networks.

[0128] Specifically, the graph convolution module 20 can be replaced with a point cloud Transformer-based architecture. In a standard point cloud Transformer, the calculation of attention scores mainly depends on the relationship between queries, keys, and values. To incorporate occlusion awareness into the variant of this embodiment, when calculating the attention score between any two points (the first data point and the second data point), in addition to the relationship between their features, a bias term determined by their occlusion metric is introduced.

[0129] For example, when calculating the center point For neighboring points When focusing on a given amount of attention, the attention score can be adjusted to... ,in This refers to the occlusion bias term. From the center point The query vector is obtained by linear transformation of the features. For neighborhood points The transpose of the key vector obtained by linear transformation of the features. Key vector The dimension, for and Scale the dot product (divide by) ), The attention score is calculated by scaling the dot product.

[0130] Among them, the bias term The calculation rules are the same as the occlusion relationship factor in Example 1. Similar ideas: If It is an obstruction point. If it is a reliable unobstructed point, then To be a positive value, thus improving right Attention weights are assigned to encourage occluded points to acquire information from reliable points; if and If all are occlusion points, then The value is set to negative to reduce the attention between them, thereby suppressing noise propagation. In this way, the attention mechanism can also dynamically and adaptively adjust the weight of information transmission according to the occlusion of points, realizing the same technical idea as in Embodiment 1, that is, enabling the feature extraction process to actively adapt to occlusion, thereby providing a more robust initial feature map 503 for subsequent feature completion steps. This embodiment shows that the core idea of ​​realizing occlusion-aware feature extraction is universal and not limited to a specific network structure.

[0131] Example 3

[0132] This embodiment provides a variant of the face geometric prior model 403, a core component used in the feature completion step S300 of Embodiment 1. Embodiment 1 describes a general parametric face model constructed based on statistical analysis. However, the technical solution of this application is not limited to this specific model.

[0133] As a preferred implementation, the feature completion module 30 can employ a more advanced and refined parametric face model, such as the FLAME model. The FLAME model can not only represent identity and expression, but also explicitly parametrically model head pose, jaw joint movement, etc., which can generate more realistic and dynamic facial geometry.

[0134] In this embodiment, the overall process of feature completion step S300 remains unchanged, but all operations related to the face geometric prior model 403 will be based on the FLAME model. Specifically, a preset mapping function will be trained to map the features of non-occluded points to the parameters of the FLAME model (including shape, expression, pose, etc.). When generating physical constraint features 405 for occluded points 401, the system will estimate the parameters of the FLAME model through the non-occluded points in its neighborhood and input these parameters into the FLAME model to generate a complete, correctly posed, and naturally expressive 3D face. Then, the features at the corresponding positions are extracted from this generated face as physical constraint features 405. The subsequent fusion steps with context features 406 are also the same as in Embodiment 1.

[0135] This embodiment demonstrates the flexibility and scalability of the dual-constraint feature completion framework proposed in this application. Its core idea does not rely on any specific face prior model. Any prior model that can provide parameterized representation and generate reasonable face geometry can be integrated into this framework to provide global physical constraints.

[0136] Example 4

[0137] This embodiment provides a modified implementation of two specific technical points in the feature completion step S300 of Embodiment 1, which involve the construction algorithm of feature manifold 404 and the method of feature fusion 408, respectively.

[0138] On the one hand, regarding the construction of the feature manifold 404, Example 1 uses the Isomap algorithm to construct a low-dimensional manifold for the unoccluded point features. As an optional implementation, other manifold learning algorithms, such as the locally linear embedding algorithm, can also be used. The core idea of ​​the locally linear embedding algorithm is to assume that data points can be linearly reconstructed from their neighborhood points and to strive to maintain this locally linear reconstruction relationship in the low-dimensional space. In the feature completion module 30, the locally linear embedding algorithm can be used to process the set of unoccluded point features, which can also yield an effective low-dimensional feature manifold 404. The subsequent steps of calculating the geodesic distance and context feature 406 based on this manifold remain unchanged. This demonstrates that there are diverse technical choices for constructing feature manifolds.

[0139] On the other hand, regarding the fusion method of physical constraint feature 405 and context feature 406, Example 1 adopts a fixed weighted fusion method, namely... The weight This is a hyperparameter that needs to be manually adjusted. As a more advanced and adaptive implementation, this embodiment employs a learnable gating fusion mechanism.

[0140] Specifically, a small neural network can be designed as a gating unit. The input to this gating unit is the physical constraint feature 405 and the context feature 406 to be fused. Through several fully connected layers and a sigmoid activation function, the gating unit outputs a dynamic fusion weight. Its value is between 0 and 1. The final completion feature 407 is calculated using the following formula: This gating unit can be trained end-to-end with the entire recognition network. During training, the network automatically learns, based on different input samples and occlusion conditions, when to trust physical constraint features representing the global structure more and when to prioritize contextual features that maintain local consistency. For example, when the visible area information is very rich, the network may learn to give higher weights to contextual features; while when the visible area information is sparse, it may rely more on the physical constraints provided by the geometric prior model. This adaptive fusion mechanism makes the feature completion process more intelligent, further improving completion quality and final recognition performance.

[0141] The gating fusion mechanism in this application can dynamically adjust the weights according to the degree of occlusion: when the number of non-occluded neighboring points of an occluded point is ≥15 (rich local information), the dynamic weights output by the gating unit will be adjusted accordingly. The value ranges from 0.3 to 0.4, emphasizing contextual features (ensuring local consistency); when the number of non-occluded neighboring points is less than 5 (sparse local information), The value is set to 0.6~0.7, placing greater emphasis on physical constraint features (ensuring global rationality). This adaptive adjustment improves the matching accuracy of the completed features compared to fixed weights. The accuracy improved by 8.7%, especially in scenarios where a large area of ​​hair on one side was obscured, where the Euclidean distance between the completed features and the real features was significantly reduced.

[0142] Example 5

[0143] The method in this application is compatible with cross-modal inputs of geometric and texture features: if the 3D face point cloud also contains texture information (such as RGB color values), the texture features and geometric features can be concatenated and input into the initial feature extraction process. During dynamic neighborhood adjustment, texture similarity (calculating the cosine similarity of RGB values) is added to the association metric parameter, and the texture feature dimension is incorporated when constructing the feature manifold, with the input dimension of the mapping function being expanded synchronously. After adding texture features, the face recognition accuracy is further improved in scenarios with double occlusion such as masks and glasses.

[0144] In this embodiment, the input is a textured 3D face point cloud, which includes each first data point. geometric coordinates ( In addition to geometric features, it also includes each Corresponding texture information (i.e., RGB color values) The value range is [0, 255], which represents the texture features. Texture information is acquired synchronously through a depth camera: while the depth camera outputs the three-dimensional geometric coordinates, the RGB values ​​of the corresponding pixels are obtained through the color imaging module, and then each point cloud data point is matched with the pixel-point cloud coordinates. Binds to RGB values.

[0145] After the 3D face point cloud normalization processing in the occlusion measurement step S100 of Example 1, a texture feature normalization step is added to ensure that the numerical scales of geometric features and texture features are consistent: for each first data point Normalize the RGB values: , , Normalized texture features The value range is [0,1]; geometric features and texture features are spliced ​​together to form cross-modal initial features: ,in Normalized geometric coordinates in Example 1 , The dimensions are 6 (3D geometry + 3D texture).

[0146] In Example 1, the information transmission weight was calculated. Based on the association metrics (geometric distance, feature similarity), texture similarity is added. The specific calculation and fusion logic is as follows:

[0147] 1) Texture similarity calculation

[0148] For the first data point The second data point in its dynamic neighborhood The cosine similarity of the normalized texture features between the two is calculated using the following formula:

[0149]

[0150] like ,express and The texture is completely uniform (like a single skin tone area); if , indicating that the textures are completely unrelated (such as the textures of human face skin and hair).

[0151] 2) Information transmission weight fusion of texture similarity

[0152] Transform texture similarity into texture weights The original weighting formula is incorporated, and the modified information transmission weighting formula is as follows: .

[0153] in, Geometric distance weights: Same as in Example 1, the formula is: , (Based on 6-dimensional cross-modal feature statistical optimization).

[0154] in, Cross-modal feature similarity weights: Replace the original geometric feature similarity with the initial cross-modal features. The cosine similarity is calculated using the same formula as in Example 1, and the mapped value is [0,1].

[0155] in, For texture weights): Perform sigmoid mapping, the formula is as follows Ensure that the texture similarity is ≥0.5. ≥0.5 (contribution to enhancing texture consistency points).

[0156] in, Occlusion relationship factor: Same as in Example 1 (non-occlusion → occlusion propagation is 1.5, occlusion → occlusion propagation is 0.5).

[0157] Subsequently, the input to the convolutional layer in Example 1 was changed from 3D geometric features to 6D cross-modal initial features. The corresponding learnable parameters of the convolutional layer are adjusted as follows: the original weight matrix W is changed from 3×C (C is the number of output feature channels) to 6×C; the bias term b remains unchanged at 1×C; and the number of output feature channels C is still set to 64.

[0158] In Example 1, when constructing a low-dimensional feature manifold based on the set of unoccluded point features, this example replaces the set of unoccluded point features with cross-modal features. The specific adjustments are as follows:

[0159] 1) Cross-modal feature screening for non-occluded points

[0160] Filtering satisfies For each unoccluded point, extract its 64-dimensional cross-modal features (instead of the original geometric features) from the graph convolution output to form the cross-modal feature set of unoccluded points. .

[0161] 2) Adjusting Isomap algorithm parameters

[0162] Because the feature dimension has been expanded from 3D geometric features to 64D cross-modal features, the Isomap algorithm parameters have been adjusted to ensure manifold reconstruction accuracy: nearest neighbor count The dimensionality of the reduced low-dimensional space was adjusted from 10 to 15 (more nearest neighbors ensure preservation of the local structure of high-dimensional features); the dimensionality of the reduced low-dimensional space was adjusted from 20 to 30; the geodesic distance calculation still uses the Dijkstra algorithm, but the neighborhood graph is constructed based on the Euclidean distance of cross-modal features (instead of the original geometric distance).

[0163] Furthermore, the mapping function input in Example 1, which was originally the geometric features of the unoccluded points, is now changed to the cross-modal features of the unoccluded points. Specifically, the following modifications are made:

[0164]

[0165] Training dataset: Based on the original unoccluded 3D face geometry data, corresponding texture data is added to form training samples with geometry + texture pairing (e.g., 1200 sets, which is 200 more sets of textured samples than the original dataset).

[0166] Loss function: still uses the coordinate error between the model-generated face shape and the real face shape (because texture features are only used for auxiliary parameter mapping, and the final physical constraint features are generated by the geometric prior model and are unrelated to texture);

[0167] Training iteration count: such as increasing from 500 rounds to 600 rounds, to ensure network convergence under high-dimensional input.

[0168] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0169] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0170] Corresponding to the face recognition detection method described in the above embodiments, the face recognition detection device provided in this application embodiment will not be described again.

[0171] It should be noted that the information interaction and execution process between modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0174] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0175] This application provides a computer program product that, when run on a computer device, enables the computer device to perform the steps described in the above-described method embodiments.

[0176] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0177] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0182] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 6 As shown, the computer device of this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown in the diagram), memory 61, and computer program 62 stored in said memory 61 and executable on said at least one processor 60, wherein said processor 60 executes said computer program 62 to implement the steps in any of the above-described embodiments of the face recognition detection method.

[0183] The computer device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0184] The processor 60 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0185] In some embodiments, the memory 61 may be an internal storage unit of the computer device, such as a hard drive or memory. In other embodiments, the memory 61 may be an external storage device of the computer device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 61 may include both internal and external storage units of the computer device. The memory 61 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0186] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0187] The personal information of users processed by the applicant will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0188] The applicant attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

Claims

1. A face recognition detection method, characterized in that, include: Obtaining 3D face point clouds First data points in each of the middle Corresponding occlusion metric ; Based on each first data point Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ; wherein, the initial feature map Including occluded points and unoccluded points; For the initial feature map The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ; Based on the complete feature map Facial recognition detection is performed to determine facial identity; Among them, the first data point Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ,include: For 3D face point cloud Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ;in, For dynamic neighborhood The second data point in; For each first data point Based on this first data point Occlusion metric The first data point Its dynamic neighborhood The second data point within The correlation metric parameters are used to calculate the first data point. With the second data point Information transfer weight between ; Based on each first data point dynamic neighborhood And information transmission weight For each first data point Perform graph convolution operations to aggregate their dynamic neighborhoods. Second data point The features are used to obtain each first data point. Update features; All first data points The updated features are integrated to generate an initial feature map. ; Among them, the three-dimensional face point cloud Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ,include: For each first data point Based on the first data point Occlusion metric The first data point is determined by a preset neighborhood radius calculation formula. Neighborhood radius during graph convolution operation ; With the first data point Center, neighborhood radius With radius, in the 3D face point cloud Determine the initial spatial neighborhood; Based on the first data point and each second data point in the initial spatial neighborhood Based on the geometric features, the initial spatial neighborhood is geometrically consistent and selected; If the first data point Occlusion metric If the occlusion metric is greater than the first preset occlusion metric, then in the initial spatial neighborhood after geometric consistency filtering, the occlusion metric will be... The second data point is less than or equal to the second preset occlusion metric. As the first data point dynamic neighborhood ; If the first data point Occlusion metric If the occlusion metric is less than or equal to the first preset occlusion metric, then the initial spatial neighborhood after geometric consistency filtering is directly used as the first data point. dynamic neighborhood .

2. The face recognition detection method as described in claim 1, characterized in that, The acquisition of 3D face point clouds First data points in each of the middle Corresponding occlusion metric ,include: The three-dimensional face point cloud Perform normalization processing; Normalized 3D face point cloud The input is fed into a preset occlusion prediction network, and the output is a normalized 3D face point cloud. First data points in each of the middle The corresponding occlusion probability, and the first data points The occlusion probability is used as the first data point Occlusion metric .

3. The face recognition detection method as described in claim 1, characterized in that, The formula for calculating the neighborhood radius is as follows: , The preset basic neighborhood radius, Used to limit occlusion measurement for neighborhood radius Maximum contribution, avoiding neighborhood radius Measurement due to occlusion Too large and excessively enlarged.

4. The face recognition detection method as described in claim 3, characterized in that, Based on the first data point and each second data point in the initial spatial neighborhood The geometric features are used to perform geometric consistency screening on the initial spatial neighborhood, including: Curvature filtering: Calculate the first data point curvature Second data point curvature , retain satisfaction The second data point ; Normal filtering: Calculate the first data point unit normal Second data point unit normal , retain satisfaction The second data point with an angle greater than the preset value .

5. The face recognition detection method as described in claim 4, characterized in that, The correlation metric parameters include geometric distance. Feature similarity; For each first data point Based on this first data point Occlusion metric The first data point Its dynamic neighborhood The second data point within The correlation metric parameters are used to calculate the first data point. With the second data point Information transfer weight between ,include: Based on the geometric distance and the feature similarity, combined with the first data point Second data point The occlusion relationship is determined by the occlusion relationship factor. The first data point was calculated. With the second data point Information transfer weight between .

6. The face recognition detection method as described in claim 1, characterized in that, The training process of the facial geometric prior model includes: Collect a number of unobstructed 3D face data to form a training sample set; Individual facial difference features and expression change features are obtained from the training sample set; Statistical analysis was performed on the training sample set to calculate the average face model. ; Principal component analysis was used to perform dimensionality reduction extraction of the individual facial difference features to obtain the identity basis vector. ; Principal component analysis was used to extract the dimensionality-reducing features of the facial expressions, resulting in expression basis vectors. ; Based on the above average face model Identity basis vectors and expression basis vectors Construct a function to represent the shape of a human face: ;in, The shape of a human face, For identity parameters, For facial expression parameters; For facial expression parameters Apply L2 norm constraints; In the shape of a human face The coordinate error between the actual face shape and the coordinates in the training sample set is used as the optimization objective to construct a loss function; Minimize the loss function to obtain the face geometric prior model.

7. The face recognition detection method as described in claim 6, characterized in that, The initial feature map The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ,include: Obtain the initial feature map The features of the unoccluded points in the dataset are used to form a set of unoccluded point features; wherein, the unoccluded point is a first data point that satisfies a preset unoccluded condition. ; Based on the set of unoccluded point features, a low-dimensional feature manifold is constructed using the Isomap algorithm; Based on the real face coordinates of the feature manifold and unoccluded points, a multilayer perceptron is trained as a mapping function. Based on each first data point Corresponding occlusion metric For each first data point Mark whether a point is an occlusion point and generate an initial occlusion mask; The initial occlusion mask is optimized by removing occlusion points at the edge of the initial occlusion mask that have similar features to the non-occlusion points in the local spatial neighborhood of the occlusion point at that edge, thus obtaining the optimized occlusion mask. For each occlusion point in the optimized occlusion mask, select the non-occlusion points in the local spatial neighborhood of the occlusion point. Through the trained mapping function, map the features of the non-occlusion points in the local spatial neighborhood to the parameters of the face geometry prior model. Then, take the average value of the parameters based on the number of non-occlusion points in the local spatial neighborhood to obtain the parameter estimate of the face geometry prior model. The parameter estimates are input into the face geometry prior model to generate physical constraint features for each occluded point in the optimized occlusion mask. Simultaneously, based on the constructed feature manifold, the geodesic distance between the occluded point and the non-occluded points in the local spatial neighborhood of the occluded point is calculated on the manifold. The attention weight is calculated based on the geodesic distance. The features of the non-occluded points in the local spatial neighborhood of the occluded point are weighted and aggregated through the attention weight to generate the context features of the occluded point. Based on the physical constraint features and context features of each occlusion point, the completion features of the occlusion point are obtained through weighted fusion; The completed features of all occluded points are merged with the features of unoccluded points to generate a complete feature map. .

8. A face recognition detection device, characterized in that, include: The acquisition module is used to acquire 3D face point clouds. First data points in each of the middle Corresponding occlusion metric ; The graph convolution module is used to perform graph convolution based on each first data point. Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ; wherein, the initial feature map Including occluded points and unoccluded points; The feature completion module is used to complete the initial feature map. The occluded points in the image are used to perform feature completion based on a preset face geometry prior model and a feature manifold composed of unoccluded points, generating a complete feature map. ; The identification and detection module is used to identify and detect based on the complete feature map. Facial recognition detection is performed to determine facial identity; Among them, the first data point Corresponding occlusion metric The three-dimensional face point cloud Perform occlusion-adaptive graph convolution operations to generate initial feature maps. ,include: For 3D face point cloud Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ;in, For dynamic neighborhood The second data point in; For each first data point Based on this first data point Occlusion metric The first data point Its dynamic neighborhood The second data point within The correlation metric parameters are used to calculate the first data point. With the second data point Information transfer weight between ; Based on each first data point dynamic neighborhood And information transmission weight For each first data point Perform graph convolution operations to aggregate their dynamic neighborhoods. Second data point The features are used to obtain each first data point. Update features; All first data points The updated features are integrated to generate an initial feature map. ; Among them, the three-dimensional face point cloud Each first data point in Based on this first data point Occlusion metric Dynamically adjust the first data point The neighborhood range used for information aggregation during graph convolution operations is used to obtain the first data point. dynamic neighborhood ,include: For each first data point Based on the first data point Occlusion metric The first data point is determined by a preset neighborhood radius calculation formula. Neighborhood radius during graph convolution operation ; With the first data point Center, neighborhood radius With radius, in the 3D face point cloud Determine the initial spatial neighborhood; Based on the first data point and each second data point in the initial spatial neighborhood Based on the geometric features, the initial spatial neighborhood is geometrically consistent and selected; If the first data point Occlusion metric If the occlusion metric is greater than the first preset occlusion metric, then in the initial spatial neighborhood after geometric consistency filtering, the occlusion metric will be... The second data point is less than or equal to the second preset occlusion metric. As the first data point dynamic neighborhood ; If the first data point Occlusion metric If the occlusion metric is less than or equal to the first preset occlusion metric, then the initial spatial neighborhood after geometric consistency filtering is directly used as the first data point. dynamic neighborhood .

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, it causes the computer device to perform the method as described in any one of claims 1 to 7.

Images