A face image recognition method and system, an electronic device, and a storage medium

Abstract

The present application provides a kind of face image recognition method, system, electronic equipment and storage medium, method includes: the pre-set face image to be identified is input into pre-set inverse graph capsule network, image coding is carried out, global shape feature and global texture feature are acquired;Based on pre-set image decomposition rule, global shape feature and global texture feature are respectively decomposed into feature, and component level capsule feature is acquired;Capsule feature is graphically decoded, and at least one component capsule is acquired;Based on the depth information in description parameter, information fusion is carried out to each component capsule respectively, and at least one target capsule is acquired;According to the illumination value of each face component in the pre-acquired face image to be identified, target capsule is rendered, and final face image is acquired, and final face image includes the three-dimensional description information of face.Effectively improve the explainability and application range of inverse graph capsule network, and larger attitude face can be preferably identified or handled.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a face image recognition method, system, electronic device, and storage medium. Background Technology

[0002] In recent years, with the development of deep learning, many visual perception tasks have achieved significant breakthroughs in performance. However, despite achieving good performance on specific datasets, deep learning has significant limitations in the interpretability of decision-making processes, thus restricting its further application in many fields.

[0003] To overcome the current bottlenecks in deep learning and advance from existing artificial intelligence to autonomous machine intelligence, a key aspect lies in enabling machines to possess human-like perception and learning abilities. By observing a large number of images, humans can infer the three-dimensional structure of objects and decompose them into a series of component-level parts, achieving object recognition through the hierarchical relationships between these parts. This unsupervised decomposition reflects how humans understand objects. To enable neural networks to unsupervisedly recognize facial components from a large number of images, most existing methods are based on local aggregation constraints, using a classifier to learn semantically consistent facial components and then reconstructing the original image for neural network training. However, these methods lack interpretability, and due to their reliance on aggregation constraints, the segmented facial components often maintain similar circular regions, resulting in semantic ambiguity and an inability to effectively recognize or process faces with larger poses. Summary of the Invention

[0004] This invention provides a face image recognition method, system, electronic device, and storage medium to solve the problems of poor interpretability and inability to recognize faces with large poses in the prior art.

[0005] This invention provides a face image recognition method, comprising:

[0006] The preset face image to be recognized is input into the preset inverse graph capsule network for image encoding to obtain global shape features and global texture features;

[0007] Based on preset image decomposition rules, feature decomposition is performed on the global shape features and global texture features respectively to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features.

[0008] The capsule features are graphically decoded to obtain at least one component capsule, the component capsule including description parameters of any face component;

[0009] Based on the depth information in the description parameters, information fusion is performed on each of the component capsules to obtain at least one target capsule;

[0010] Based on the illumination values ​​of each facial component in the pre-acquired face image to be identified, the target capsule is rendered to obtain the final face image, which includes three-dimensional descriptive information of the face.

[0011] Optionally, the step of performing feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features includes:

[0012] Cross-focusing is performed on the global shape features and the preset shape basis to obtain the deconstructed face component shape features, the dimension of which is a preset dimension; according to the preset one-hot encoding acquisition rules, shape one-hot encodings corresponding to the number of face components are generated in each dimension; the face component shape features are multiplied by the shape one-hot encodings to obtain the implicit shape features;

[0013] By cross-focusing on the shape features and the preset shape basis, the mathematical expression of the deconstructed face component shape features is obtained as follows:

[0014]

[0015] in, For the shape features of the disassembled facial components, f s For global shape features, The shape base refers to M preset implicit component shape templates, where M represents the number of face components.

[0016] Optionally, the step of performing feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features further includes:

[0017] Cross-focusing is performed between the global texture features and the preset texture base to obtain the deconstructed face component texture features, wherein the dimension of the face component texture features is a preset dimension;

[0018] According to the one-hot encoding acquisition rules, texture one-hot encodings corresponding to the number of face components are generated in each dimension, and the shape one-hot encodings correspond to the texture one-hot encodings.

[0019] The implicit albedo features are obtained by multiplying the texture features of the face component with the one-hot encoding of the texture.

[0020] Optionally, the step of obtaining the inverse graph capsule network includes:

[0021] Obtain a training set, which includes multiple samples and the corresponding real recognition results of the samples;

[0022] The training set is input into a preset original neural network, and the target loss is obtained based on a preset set of loss functions. The target loss includes at least one of the following: image reconstruction loss, semantic consistency loss, sparsity constraint loss, and background constraint loss.

[0023] Based on the target loss and the preset weight parameters, the total loss of the neural network is obtained;

[0024] Based on the total loss of the neural network, the original neural network is iteratively trained to obtain the inverse graph capsule network.

[0025] Optionally, the mathematical expression for the image reconstruction loss is obtained as follows:

[0026]

[0027] in, Ω represents the distance between the image of the face to be identified and the final image of the face, σ is the normalization parameter, and σ is the preset confidence level. σ is used to represent the probability that there is symmetry at any position. I represents the final face image, and I represents the face image to be identified. This represents the image reconstructed from the horizontally flipped albedo map and depth map of the face image to be identified, where ln represents the natural logarithm and exp represents the exponential function.

[0028] The mathematical expression for obtaining the semantic consistency loss is:

[0029]

[0030] in, Let B represent the semantic consistency loss, B be the batch size (where B samples are drawn at a time during network training to calculate the loss function), and M be the number of component capsules or face components. Let τ represent the shape features of the j-th face component in the i-th sample, where τ is a preset hyperparameter, and the operator . represents matrix multiplication.

[0031] Optionally, the mathematical expression for the sparse constraint loss is obtained as follows:

[0032]

[0033] in, This represents the sparse constraint loss, and std represents the standard deviation operation. This represents the visibility of the m-th component capsule at position (x, y);

[0034] The mathematical expression for obtaining the background constraint loss is:

[0035]

[0036] in, V represents the background constraint loss. bg This indicates the visibility of the component capsules used to reconstruct the background in the inverse graph capsule network. D represents the background area. o This represents the depth map of the face component, where γ is a hyperparameter used to estimate the background region.

[0037] Optionally, the step of performing information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule includes:

[0038] Obtain the visibility of each component capsule at any location;

[0039] Based on the visibility and the description parameters of the face component, information fusion is performed to obtain at least one target capsule; wherein the description parameters include: depth information, albedo, and 3D pose parameters.

[0040] The present invention also provides a face recognition system, comprising:

[0041] The encoding module is used to input the preset face image to be recognized into the preset inverse graph capsule network, perform image encoding, and obtain global shape features and global texture features;

[0042] The decomposition module is used to perform feature decomposition on the global shape features and global texture features respectively based on preset image decomposition rules to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features.

[0043] A decoding module is used to perform graphical decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component;

[0044] The fusion module is used to perform information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule;

[0045] The rendering module is used to render the target capsule according to the illumination values ​​of each facial component in the pre-acquired face image to obtain the final face image, which includes three-dimensional description information of the face.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the face image recognition method as described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the face image recognition method as described above.

[0048] This invention provides a face image recognition method, system, electronic device, and storage medium. The method involves inputting a preset face image to be recognized into a preset inverse graph capsule network for image encoding to obtain global shape features and global texture features. Based on preset image decomposition rules, feature decomposition is performed on the global shape features and global texture features to obtain component-level capsule features. These capsule features include implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features. Graphical decoding is performed on the capsule features to obtain at least one component capsule, which includes descriptive parameters for any face component. Based on the depth information in the descriptive parameters, information fusion is performed on each component capsule to obtain at least one target capsule. The target capsules are rendered according to the illumination values ​​of each face component in the pre-acquired face image to obtain the final face image, which includes three-dimensional descriptive information of the face. This method effectively improves the interpretability and application scope of the inverse graph capsule network, enabling better recognition or processing of faces with large poses, high accuracy, and strong adaptability. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the face image recognition method provided by the present invention;

[0051] Figure 2 This is a schematic diagram of the shape feature decomposition process in the face image recognition method provided by the present invention;

[0052] Figure 3 This is a schematic diagram of the process of texture feature decomposition in the face image recognition method provided by the present invention;

[0053] Figure 4 This is a schematic diagram of the process for obtaining the inverse graphic capsule network in the face image recognition method provided by the present invention;

[0054] Figure 5 This is a schematic diagram of the structure of the face image recognition system provided by the present invention;

[0055] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] Compared to traditional machine learning algorithms, deep neural networks possess stronger learning capabilities, primarily due to the vast amounts of training data and their powerful fitting ability. However, despite achieving impressive performance on specific datasets, deep learning faces significant limitations in the interpretability of its decision-making processes, thus restricting its further application in many fields. To overcome these bottlenecks and advance from current artificial intelligence to autonomous machine intelligence, a key aspect is enabling machines to possess human-like perception and learning abilities. By observing numerous images, humans can infer the three-dimensional structure of objects and decompose them into a series of component-level parts, achieving object recognition through the hierarchical relationships between these parts. This unsupervised decomposition reflects human understanding of objects. Taking faces as an example, if deep networks could also possess similar human capabilities, learning hierarchical three-dimensional descriptions of faces from a large number of two-dimensional images, thus providing interpretability to the network's output, it would demonstrate that the network possesses visual perception and understanding capabilities for objects with similar structures, such as faces. The interpretable hierarchical descriptions output by deep networks can then be used to improve their performance in various downstream visual perception tasks, thereby further expanding the application scenarios and scope of deep networks.

[0058] To enable networks to discover or recognize facial features in unsupervised manner from a large number of images, most existing methods are based on local aggregation constraints. They use classifiers to learn semantically consistent facial features and then reconstruct the original image to train the neural network. However, these methods lack interpretability, and because they rely on aggregation constraints, the segmented facial features often maintain similar circular regions, resulting in semantic ambiguity. Furthermore, they cannot effectively recognize or process faces with larger poses. Therefore, this invention proposes a face image recognition method, system, electronic device, and storage medium. The method involves inputting a preset face image to be recognized into a preset inverse graph capsule network for image encoding to obtain shape and texture features. Based on preset image decomposition rules, the shape and texture features are decomposed to obtain component-level capsule features. These capsule features include implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features. The capsule features are then graphically decoded to obtain at least one component capsule, which includes description parameters for any face component. Based on the depth information in the description parameters, information fusion is performed on each component capsule to obtain at least one target capsule. Finally, based on the illumination values ​​of each face component in the pre-acquired face image to be recognized, the target capsule is rendered to obtain the final face image, which includes three-dimensional description information of the face. This method effectively improves the interpretability and application scope of the inverse graph capsule network and can better recognize or process faces with large poses.

[0059] The following examples illustrate this approach. Figures 1-6 This invention describes the face image recognition method, system, electronic device, and storage medium provided by the present invention.

[0060] Please refer to Figure 1 The face image recognition method provided in this embodiment includes:

[0061] S101: Input the preset face image to be recognized into the preset inverse graph capsule network, perform image encoding, and obtain global shape features and global texture features.

[0062] Specifically, the face image to be identified is an unlabeled two-dimensional face image. The inverse graph capsule network consists of multiple graph capsules, each including a series of interpretable graph parameters. By inputting the preset face image to be identified into the preset inverse graph capsule network, the three-dimensional hierarchical representation of the face is encoded into the graph capsules, facilitating unsupervised discovery of face components during subsequent recognition. This expands the inverse graph capsule network's ability to model large-pose faces, making it closer to the real human visual perception process. Ultimately, this enhances network performance while improving the network's interpretability and application scope.

[0063] In some embodiments, a preset image of a face to be identified is input into a preset inverse graph capsule network for image encoding, and the mathematical expressions of global shape features and global texture features are obtained as follows:

[0064] f s f a =ImageEncoder(I)

[0065] Among them, f s For global shape features, f a For global texture features, I represents the face image to be identified, and ImageEncoder represents the encoder in the inverse graph capsule network performing the encoding operation.

[0066] S102: Based on preset image decomposition rules, perform feature decomposition on the global shape features and global texture features respectively to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features. The mathematical expression of this step is:

[0067]

[0068] in, This represents the implicit shape features of the m-th person's face component. Let represent the implicit albedo feature of the m-th face component. M is the number of face components, 1 ≤ m ≤ M. GDM represents the feature decomposition operation performed by the graph decomposition subnetwork in the inverse graph capsule network.

[0069] It should be noted that in step S102 above, based on preset image decomposition rules, the global shape features are decomposed to obtain implicit shape features at the component level. Based on preset image decomposition rules, the global texture features are also decomposed to obtain implicit albedo features at the component level. Here, "component level" refers to the face component level. Implicit shape features refer to shape features before decoding; the human eye cannot intuitively perceive the specific shape described by these features. After decoding, the implicit becomes explicit, and the shape of the component can be intuitively seen. Similarly, implicit albedo features refer to texture features before decoding. Albedo can be simply understood as a texture map without illumination.

[0070] S103: Perform graphical decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component. Specifically, the description parameters include: depth information, albedo, and 3D pose parameters.

[0071] The mathematical expression for step S103 above is:

[0072]

[0073] in, Represents depth information, Represents albedo. This represents the 3D pose parameters, and `GraphicsDecoder` represents the graphics decoding operation performed by the graphics decoder in the inverse graphics capsule network. By performing graphics decoding, the description parameters in each part capsule can be determined more effectively.

[0074] S104: Based on the depth information in the description parameters, perform information fusion on each of the component capsules to obtain at least one target capsule.

[0075] In some embodiments, the step of performing information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule includes:

[0076] The visibility of each component capsule at any location is obtained; based on the visibility and the description parameters of the face component, information fusion is performed to obtain at least one target capsule.

[0077] Specifically, the mathematical expression for obtaining at least one of the target capsules is:

[0078]

[0079]

[0080]

[0081]

[0082] in, Indicates the target capsule, D o A represents the depth information within the target capsule. o p represents the albedo in the target capsule. o represents the 3D pose parameters in the target capsule, and ⊙ represents the dot product operation. Let represent the visibility of the m-th component capsule at position (x, y). This indicates that the result is 1 if the condition of the subscript is met, and 0 otherwise.

[0083] During the information fusion of facial components, a target capsule is visible at (x, y) only when its depth is greater than that of other target capsules. The depth information and albedo of each facial component are multiplied by their corresponding visibility, and then fused to obtain a final overall representation. The 3D pose parameters in the target capsule are the average poses of each facial component. Within the target capsule, both the depth parameters and albedo are defined in a predefined standard pose space. The 3D pose parameters are used to project 3D objects or 3D faces onto the image plane.

[0084] By performing information fusion on each component capsule in S104 to obtain at least one target capsule, the descriptive ability of the target capsule can be significantly improved, thereby enhancing the interpretability of the inverse graph capsule network.

[0085] S105: Based on the illumination values ​​of each facial component in the pre-acquired face image to be identified, the target capsule is rendered to obtain a final face image, which includes three-dimensional description information of the face.

[0086] It should be noted that the illumination values ​​of each facial component can be obtained using a pre-defined additional sub-network, such as a deep convolutional neural sub-network. In some embodiments, the target capsule is rendered using a differentiable renderer in an inverse graph capsule network to complete image reconstruction and obtain the final facial image, which includes three-dimensional descriptive information of the face. This effectively extends the two-dimensional description of the image to be recognized to a three-dimensional description, allowing for effective modeling of facial images under various poses and achieving high-accuracy face recognition. While enhancing the network performance of the inverse graph capsule network, it also significantly improves its interpretability and application scope. In some embodiments, the mathematical expression of the final facial image is:

[0087]

[0088] in, This represents the final face image, ∧ represents the rendering operation, and l represents the lighting value.

[0089] Please refer to Figure 2 In some embodiments, the step of performing shape feature decomposition includes:

[0090] S201: Cross-focus on the global shape features and the preset shape basis to obtain the deconstructed face component shape features, wherein the dimension of the face component shape features is a preset dimension.

[0091] In some embodiments, cross-focusing is applied to the shape features and a preset shape basis to obtain the mathematical expression of the deconstructed facial component shape features as follows:

[0092]

[0093] in, For the shape features of the disassembled facial components, f s For global shape features, The shape base refers to M preset implicit component shape templates, where M represents the number of face components.

[0094] S202: Based on the preset one-hot encoding acquisition rules, generate shape one-hot encodings corresponding to the number of face components in each dimension.

[0095] It is worth mentioning that since the global shape feature we obtain is M*H (M is the number of parts, and H is the feature dimension), a one-hot shape encoding of the same size, also M*H, will be generated for this global shape feature. For each specific dimension, only one face part is 1, and the other face parts are 0.

[0096] Specifically, the mathematical expression for generating shape one-hot codes corresponding to the number of facial components in each dimension is as follows:

[0097]

[0098]

[0099] Where K is the one-hot encoding of the shape, or the attention matrix corresponding to the shape features of the facial components, and hardsoftmax represents the attention operation. The d-th dimension vector represents the shape features of the first person's face component. This represents the d-th dimension vector representing the shape features of the M-th person's facial components. Let d represent the d-th dimension vector of the shape features of the m-th face component, 1≤m≤M, where d represents the dimension and onehot represents the one-hot encoding operation.

[0100] S203: Perform a dot product between the shape features of the facial component and the one-hot shape encoding to obtain the implicit shape features. Its mathematical expression is:

[0101]

[0102] in, It is an implicit shape feature.

[0103] Please refer to Figure 3Texture feature decomposition corresponds to the shape feature decomposition described above, and the steps are basically the same. The difference is that "K" in the texture feature decomposition process is copied from the shape feature decomposition process. That is, the one-hot encoding of the texture is obtained by copying the one-hot encoding of the shape. This ensures that the deconstruction of shape and albedo is synchronous, meaning that the shape and albedo correspond to the same facial component. In some embodiments, the steps for performing texture feature decomposition include:

[0104] S301: Cross-focus on the global texture features and the preset texture base to obtain the deconstructed face component texture features, wherein the dimension of the face component texture features is a preset dimension.

[0105] S302: According to the one-hot encoding acquisition rules, generate texture one-hot encodings corresponding to the number of face components in each dimension, and the shape one-hot encodings correspond to the texture one-hot encodings.

[0106] S303: Multiply the texture features of the face component with the one-hot encoding of the texture to obtain the implicit albedo features.

[0107] Please refer to Figure 4 In some embodiments, the step of obtaining the inverse graph capsule network includes:

[0108] S401: Obtain a training set, which includes multiple samples and the corresponding real recognition results. Obtaining the training set facilitates subsequent training of the preset original neural network.

[0109] S402: Input the training set into a preset original neural network, and obtain the target loss based on a preset set of loss functions. The target loss includes at least one of the following: image reconstruction loss, semantic consistency loss, sparse constraint loss, and background constraint loss.

[0110] S403: Based on the target loss and the preset weight parameters, obtain the total loss of the neural network.

[0111] S404: Based on the total loss of the neural network, iteratively train the original neural network to obtain the inverse graph capsule network. By using the above loss function set to iteratively train the original neural network, the accuracy of the inverse graph capsule network can be significantly improved.

[0112] In some embodiments, the mathematical expression for the image reconstruction loss is obtained as follows:

[0113]

[0114] in, Ω represents the distance between the image of the face to be identified and the final image of the face, σ is the normalization parameter, and σ is the preset confidence level. σ is used to represent the probability that there is symmetry at any position. I represents the final face image, and I represents the face image to be identified. This represents an image reconstructed from a horizontally flipped albedo map and a depth map of the face image to be identified. `ln` represents the natural logarithm, and `exp` represents an exponential function with the natural constant e as its base. The confidence map can be obtained through a pre-defined additional network estimation, and the confidence map represents the probability of symmetry at any location. It is understood that the albedo map contains albedo information, and the depth map contains depth information.

[0115] In some embodiments, the mathematical expression for obtaining the semantic consistency loss is:

[0116]

[0117] in, Let B represent the semantic consistency loss, B be the batch size (where B samples are drawn at a time during network training to calculate the loss function), and M be the number of component capsules or face components. This represents the shape features of the j-th face component in the i-th sample. Let represent the shape features of the m-th face component in the b-th sample, τ be a preset hyperparameter, which is the hyperparameter used in the loss function to control the discrimination power against negative samples, and the operator . represents matrix multiplication, i.e., the inner product operation between vectors. This ensures that features belonging to the same facial component are as similar as possible among different samples, while features of different components in different samples are as far apart as possible.

[0118] In some embodiments, the mathematical expression for obtaining the sparse constraint loss is:

[0119]

[0120] in, This represents the sparse constraint loss, and std represents the standard deviation operation. This can prevent the network from collapsing into using only a single capsule to describe the entire object (target face). Let represent the visibility of the m-th component capsule at position (x, y).

[0121] In some embodiments, the mathematical expression for obtaining the background constraint loss is:

[0122]

[0123] in, V represents the background constraint loss. bgThis indicates the visibility of the component capsules used to reconstruct the background in the inverse graph capsule network. This represents the background region, which is estimated using a preset depth threshold. D o This represents the depth map of the face component (depth information in the target capsule), and γ is a hyperparameter used to estimate the background region. This indicates that the result is 1 if the condition of the subscript is met, and 0 otherwise.

[0124] In some embodiments, based on the target loss and preset weight parameters, the mathematical expression for the total loss of the neural network is obtained as follows:

[0125]

[0126] in, Let λ be the total loss of the neural network. contra , λ sparse , λ bg To balance the preset weight parameters of each loss function, the original neural network can learn effective facial components. Through iterative training, a superior inverse graph capsule network with higher accuracy is obtained.

[0127] The face image recognition system provided by the present invention will be described below. The face image recognition system described below can be referred to in correspondence with the face image recognition method described above.

[0128] Please refer to Figure 5 The face image recognition system provided in this embodiment includes:

[0129] The encoding module 501 is used to input a preset face image to be recognized into a preset inverse graph capsule network, perform image encoding, and obtain global shape features and global texture features.

[0130] The decomposition module 502 is used to perform feature decomposition on the global shape features and global texture features respectively based on preset image decomposition rules to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features.

[0131] The decoding module 503 is used to perform graphic decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component;

[0132] The fusion module 504 is used to perform information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule;

[0133] The rendering module 505 is used to render the target capsule based on the illumination values ​​of each facial component in the pre-acquired face image to obtain a final face image, which includes three-dimensional descriptive information of the face. The encoding module 501, decomposition module 502, decoding module 503, fusion module 504, and rendering module 505 are connected. The face image recognition system in this embodiment can effectively improve the interpretability and application range of inverse graph capsule networks, and can better recognize or process faces with large poses, with high accuracy and strong adaptability.

[0134] In some embodiments, the decomposition module 502 performs feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features, including:

[0135] Cross-focusing is performed on the global shape features and the preset shape basis to obtain the deconstructed face component shape features, the dimension of which is a preset dimension; according to the preset one-hot encoding acquisition rules, shape one-hot encodings corresponding to the number of face components are generated in each dimension; the face component shape features are multiplied by the shape one-hot encodings to obtain the implicit shape features;

[0136] By cross-focusing on the shape features and the preset shape basis, the mathematical expression of the deconstructed face component shape features is obtained as follows:

[0137]

[0138] in, For the shape features of the disassembled facial components, f s For global shape features, The shape base refers to M preset implicit component shape templates, where M represents the number of face components.

[0139] In some embodiments, the step of decomposition module 502 performing feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features further includes:

[0140] Cross-focusing is performed between the global texture features and the preset texture base to obtain the deconstructed face component texture features, wherein the dimension of the face component texture features is a preset dimension;

[0141] According to the one-hot encoding acquisition rules, texture one-hot encodings corresponding to the number of face components are generated in each dimension, and the shape one-hot encodings correspond to the texture one-hot encodings.

[0142] The implicit albedo features are obtained by multiplying the texture features of the face component with the one-hot encoding of the texture.

[0143] In some embodiments, the step of obtaining the inverse graph capsule network includes:

[0144] Obtain a training set, which includes multiple samples and the corresponding real recognition results of the samples;

[0145] The training set is input into a preset original neural network, and the target loss is obtained based on a preset set of loss functions. The target loss includes at least one of the following: image reconstruction loss, semantic consistency loss, sparsity constraint loss, and background constraint loss.

[0146] Based on the target loss and the preset weight parameters, the total loss of the neural network is obtained;

[0147] Based on the total loss of the neural network, the original neural network is iteratively trained to obtain the inverse graph capsule network.

[0148] In some embodiments, the mathematical expression for the image reconstruction loss is obtained as follows:

[0149]

[0150] in, Ω represents the distance between the image of the face to be identified and the final image of the face, σ is the normalization parameter, and σ is the preset confidence level. σ is used to represent the probability that there is symmetry at any position. I represents the final face image, and I represents the face image to be identified. This represents the image reconstructed from the horizontally flipped albedo map and depth map of the face image to be identified, where ln represents the natural logarithm and exp represents the exponential function.

[0151] The mathematical expression for obtaining the semantic consistency loss is:

[0152]

[0153] in, Let B represent the semantic consistency loss, B be the batch size (where B samples are drawn at a time during network training to calculate the loss function), and M be the number of component capsules or face components. Let τ represent the shape features of the j-th face component in the i-th sample, where τ is a preset hyperparameter, and the operator . represents matrix multiplication.

[0154] In some embodiments, the mathematical expression for obtaining the sparse constraint loss is:

[0155]

[0156] in, This represents the sparse constraint loss, and std represents the standard deviation operation. This represents the visibility of the m-th component capsule at position (x, y);

[0157] The mathematical expression for obtaining the background constraint loss is:

[0158]

[0159] in, V represents the background constraint loss. bg This indicates the visibility of the component capsules used to reconstruct the background in the inverse graph capsule network. D represents the background area. o This represents the depth map of the face component, where γ is a hyperparameter used to estimate the background region.

[0160] In some embodiments, the step of the fusion module 504 performing information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule includes:

[0161] Obtain the visibility of each component capsule at any location;

[0162] Based on the visibility and the description parameters of the face component, information fusion is performed to obtain at least one target capsule; wherein the description parameters include: depth information, albedo, and 3D pose parameters.

[0163] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a face image recognition method. This method includes: inputting a preset face image to be recognized into a preset inverse graph capsule network for image encoding to obtain global shape features and global texture features; performing feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features, wherein the capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features; performing graph decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters for any face component; performing information fusion on each component capsule based on depth information in the description parameters to obtain at least one target capsule; and rendering the target capsule according to the illumination values ​​of each face component in the pre-acquired face image to obtain a final face image, wherein the final face image includes three-dimensional description information of the face.

[0164] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the face image recognition method provided by the above methods. The method includes: inputting a preset face image to be recognized into a preset inverse graph capsule network, performing image encoding, and obtaining global shape features and global texture features; performing feature decomposition on the global shape features and global texture features respectively based on preset image decomposition rules to obtain component-level capsule features, wherein the capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features; performing graph decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component; performing information fusion on each component capsule based on the depth information in the description parameters to obtain at least one target capsule; and rendering the target capsule according to the illumination values ​​of each face component in the preset face image to be recognized to obtain a final face image, wherein the final face image includes three-dimensional description information of the face.

[0166] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the face image recognition method provided by the above methods. The method includes: inputting a preset face image to be recognized into a preset inverse graph capsule network, performing image encoding to obtain global shape features and global texture features; performing feature decomposition on the global shape features and global texture features respectively based on preset image decomposition rules to obtain component-level capsule features, wherein the capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features; performing graph decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component; performing information fusion on each component capsule based on the depth information in the description parameters to obtain at least one target capsule; and rendering the target capsule according to the illumination values ​​of each face component in the preset face image to be recognized to obtain a final face image, wherein the final face image includes three-dimensional description information of the face.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A face image recognition method, characterized in that, include: The preset face image to be recognized is input into the preset inverse graph capsule network for image encoding to obtain global shape features and global texture features; Based on preset image decomposition rules, feature decomposition is performed on the global shape features and global texture features respectively to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features. The capsule features are graphically decoded to obtain at least one component capsule, the component capsule including description parameters of any face component; Based on the depth information in the description parameters, information fusion is performed on each of the component capsules to obtain at least one target capsule; Based on the illumination values ​​of each facial component in the pre-acquired face image to be identified, the target capsule is rendered to obtain the final face image, which includes three-dimensional descriptive information of the face. The steps for obtaining the inverse graph capsule network include: Obtain a training set, which includes multiple samples and the corresponding real recognition results of the samples; The training set is input into a preset original neural network, and the target loss is obtained based on a preset set of loss functions. The target loss includes at least one of the following: image reconstruction loss, semantic consistency loss, sparsity constraint loss, and background constraint loss. Based on the target loss and the preset weight parameters, the total loss of the neural network is obtained; Based on the total loss of the neural network, the original neural network is iteratively trained to obtain the inverse graph capsule network; The mathematical expression for the image reconstruction loss is as follows: ； in, This represents the distance between the image of the face to be identified and the final image of the face. For normalization parameters, This is a preset confidence level graph. Used to represent the probability that any position has symmetry This represents the final face image. This represents the image of the face to be identified. This represents an image reconstructed from a horizontally flipped albedo map and depth map of the face image to be identified. Represents the natural logarithm operation. Represents an exponential function; The mathematical expression for obtaining the semantic consistency loss is: ； in, This represents the semantic consistency loss. The batch size refers to the number of samples (B) that are drawn each time during network training to calculate the loss function. The number of component capsules or face components. Indicates the first In the nth sample The shape characteristics of individual facial components For preset hyperparameters, operators This represents matrix multiplication.

2. The face image recognition method according to claim 1, characterized in that, Based on preset image decomposition rules, the steps of performing feature decomposition on the global shape features and global texture features respectively to obtain component-level capsule features include: Cross-focusing is performed on the global shape features and the preset shape basis to obtain the deconstructed face component shape features, the dimension of which is a preset dimension; according to the preset one-hot encoding acquisition rules, shape one-hot encodings corresponding to the number of face components are generated in each dimension; the face component shape features are multiplied by the shape one-hot encodings to obtain the implicit shape features; By cross-focusing on the shape features and the preset shape basis, the mathematical expression of the deconstructed face component shape features is obtained as follows: ； in, The shape features of the disassembled facial components. For global shape features, The shape base refers to M preset implicit component shape templates, where M represents the number of face components.

3. The face image recognition method according to claim 2, characterized in that, The step of performing feature decomposition on the global shape features and global texture features based on preset image decomposition rules to obtain component-level capsule features further includes: Cross-focusing is performed between the global texture features and the preset texture base to obtain the deconstructed face component texture features, wherein the dimension of the face component texture features is a preset dimension; According to the one-hot encoding acquisition rules, texture one-hot encodings corresponding to the number of face components are generated in each dimension, and the shape one-hot encodings correspond to the texture one-hot encodings. The implicit albedo features are obtained by multiplying the texture features of the face component with the one-hot encoding of the texture.

4. The face image recognition method according to claim 1, characterized in that, The mathematical expression for obtaining the sparse constraint loss is: ； in, Represents the sparse constraint loss. This represents the standard deviation calculation. This indicates that the m-th component capsule is in the... Location visibility; The mathematical expression for obtaining the background constraint loss is: ； in, This represents the background constraint loss. This indicates the visibility of the component capsules used to reconstruct the background in the inverse graph capsule network. Indicates the background area. A depth map representing the facial features. These are the hyperparameters used to estimate the background region.

5. The face image recognition method according to claim 1, characterized in that, The step of performing information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule includes: Obtain the visibility of each component capsule at any location; Based on the visibility and the description parameters of the face component, information fusion is performed to obtain at least one target capsule; wherein the description parameters include: depth information, albedo, and 3D pose parameters.

6. A face image recognition system, characterized in that, include: The encoding module is used to input the preset face image to be recognized into the preset inverse graph capsule network, perform image encoding, and obtain global shape features and global texture features; The decomposition module is used to perform feature decomposition on the global shape features and global texture features respectively based on preset image decomposition rules to obtain component-level capsule features. The capsule features include: implicit shape features corresponding to the global shape features and implicit albedo features corresponding to the global texture features. A decoding module is used to perform graphical decoding on the capsule features to obtain at least one component capsule, wherein the component capsule includes description parameters of any face component; The fusion module is used to perform information fusion on each of the component capsules based on the depth information in the description parameters to obtain at least one target capsule; The rendering module is used to render the target capsule according to the illumination values ​​of each facial component in the pre-acquired face image to obtain the final face image, which includes three-dimensional description information of the face. The steps for obtaining the inverse graph capsule network include: Obtain a training set, which includes multiple samples and the corresponding real recognition results of the samples; The training set is input into a preset original neural network, and the target loss is obtained based on a preset set of loss functions. The target loss includes at least one of the following: image reconstruction loss, semantic consistency loss, sparsity constraint loss, and background constraint loss. Based on the target loss and the preset weight parameters, the total loss of the neural network is obtained; Based on the total loss of the neural network, the original neural network is iteratively trained to obtain the inverse graph capsule network; The mathematical expression for the image reconstruction loss is as follows: ； in, This represents the distance between the image of the face to be identified and the final image of the face. For normalization parameters, This is a preset confidence level graph. Used to represent the probability that any position has symmetry This represents the final face image. This represents the image of the face to be identified. This represents an image reconstructed from a horizontally flipped albedo map and depth map of the face image to be identified. Represents the natural logarithm operation. Represents an exponential function; The mathematical expression for obtaining the semantic consistency loss is: ； in, This represents the semantic consistency loss. The batch size refers to the number of samples (B) that are drawn each time during network training to calculate the loss function. The number of component capsules or face components. Indicates the first In the nth sample The shape characteristics of individual facial components For preset hyperparameters, operators This represents matrix multiplication.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the face image recognition method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the face image recognition method as described in any one of claims 1 to 5.

Images