A method for generating three-dimensional face recognition and related device

By acquiring RGB and three-dimensional depth images, calculating the rotation translation matrix and combining the deep neural network model to generate the target face three-dimensional image, solving the problem of low face recognition rate in the existing technology and achieving efficient face recognition effect.

CN113408344BActive Publication Date: 2024-11-26SHENZHEN JIESHUN SCI & TECH IND
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110521828.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-13
Publication Date
2024-11-26
Estimated Expiration
2041-05-13

AI Technical Summary

Technical Problem

In the prior art, there are fewer methods for combining RGB cameras and three-dimensional depth cameras for three-dimensional face recognition, resulting in a low face recognition rate and is difficult to meet the actual application needs.

Method used

By obtaining the RGB image and three-dimensional depth image of the target face, calculating the rotation translation matrix, decomposing the color channel and coordinate channel, combining the deep neural network model for feature extraction and loss value calculation, generating the three-dimensional image of the target face, and improving the face recognition rate .

Benefits of technology

The face recognition rate has been greatly improved, the problem of low face recognition rate in the prior art has been solved, and effective fusion and recognition of RGB-D images have been realized.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113408344B_ABST
    Figure CN113408344B_ABST
Patent Text Reader

Abstract

The present application discloses a method and related device for generating three-dimensional face recognition, which greatly improves the face recognition rate. The present application includes: obtaining an RGB image and a three-dimensional depth image of a target face; calculating a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image; obtaining the positional relationship between the RGB image and the three-dimensional depth image by rotation and / or translation according to the rotation and translation matrix; decomposing the color channel of the RGB image into three independent channels of R, G, and B; decomposing the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z; using the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of the target model; inputting the face image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels; generating a three-dimensional image of the target face through the target model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent security monitoring, and in particular to a method for generating three-dimensional face recognition and related devices. Background Art

[0002] With the rapid advancement of image processing and pattern recognition technology and the convenience of computer vision, face recognition technology is increasingly used in modern life. With the increasing requirements for face recognition liveness detection technology, the application of three-dimensional cameras in actual face recognition products is becoming more and more extensive.

[0003] However, at present, there are very few methods that combine RGB cameras and 3D depth cameras for 3D face recognition, and the low face recognition rate has become a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a method and a related device for generating three-dimensional face recognition, which greatly improves the face recognition rate.

[0005] A first aspect of an embodiment of the present application provides a method for generating three-dimensional face recognition, comprising:

[0006] Obtain the RGB image and 3D depth image of the target face;

[0007] Calculating a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image;

[0008] Obtaining a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix;

[0009] Decomposing the color channels of the RGB image into three independent channels: R, G, and B;

[0010] Decomposing the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z;

[0011] The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0012] Inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0013] A three-dimensional image of the target face is generated by using the target model.

[0014] Optionally, before using the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of the target model, the method further includes:

[0015] Obtain a set of training samples, wherein the training samples include RGB images of at least two human faces;

[0016] Inputting the training sample into an initial model, wherein the initial model is a model established based on a deep neural network;

[0017] Calculating the loss value of the training sample by using the initial model;

[0018] Determine whether the loss value is less than a preset value, and if so, determine the initial model as the target model;

[0019] If not, the parameters of the initial model are adjusted according to the loss value, and the training samples are re-input into the initial model with updated parameters.

[0020] Optionally, calculating the loss value of the training sample by using the initial model includes:

[0021] Extracting a feature vector of the training sample through the initial model;

[0022] The loss value is calculated according to the feature vector.

[0023] Optionally, after obtaining the positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix, the method further includes:

[0024] The rotated and / or translated RGB image and the 3D depth image are aligned to the same size.

[0025] Optionally, before inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels, the method further includes:

[0026] The input sizes of the six data input channels are adjusted to be the same as the sizes of the RGB image and the three-dimensional depth image.

[0027] Optionally, the using the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of the target model includes:

[0028] The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of the target model through the concat function, and the concat function is used to connect multiple channels as the input of the next network layer.

[0029] Optionally, generating a target face three-dimensional image by using the target model includes:

[0030] Extracting a feature vector of the facial image information through the target model;

[0031] Calculate the total loss value of the target model according to the feature vector;

[0032] Determine whether the total loss value is less than a preset value, and if so, generate a three-dimensional image of the target face through the target model;

[0033] If not, the parameters of the target model are adjusted according to the total loss value, and the facial image information is re-input into the updated target model.

[0034] Optionally, obtaining an RGB image and a three-dimensional depth image of the target face includes:

[0035] Obtain an RGB image of the target face through an RGB camera;

[0036] A three-dimensional depth image of the target face is acquired through a three-dimensional depth camera.

[0037] A second aspect of an embodiment of the present application provides a three-dimensional face recognition generation device, including:

[0038] A first acquisition unit is used to acquire an RGB image and a three-dimensional depth image of a target face;

[0039] A first calculation unit, configured to calculate a rotation and translation matrix according to internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect a mapping relationship between the RGB image and the three-dimensional depth image;

[0040] A rotation and translation unit, configured to obtain a positional relationship between the RGB image and the three-dimensional depth image by rotation and / or translation according to the rotation and translation matrix;

[0041] A first decomposition unit is used to decompose the color channel of the RGB image into three independent channels of R, G, and B;

[0042] A second decomposition unit is used to decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y and z;

[0043] A synthesis unit, used to use the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0044] A first input unit, used for inputting the face image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0045] A generating unit is used to generate a three-dimensional image of a target face using the target model.

[0046] Optionally, before the synthesis unit, the device further includes:

[0047] A second acquisition unit, configured to acquire a set of training samples, wherein the training samples include RGB images of at least two human faces;

[0048] A second input unit, used to input the training sample into an initial model, wherein the initial model is a model established based on a deep neural network;

[0049] A second calculation unit, used to calculate the loss value of the training sample through the initial model;

[0050] A judging unit, used to judge whether the loss value is less than a preset value;

[0051] a determination unit, configured to determine the initial model as a target model after the determination unit determines that the loss value is less than a preset value;

[0052] An adjustment unit is used to adjust the parameters of the initial model according to the loss value after the judgment unit determines that the loss value is greater than or equal to a preset value, and re-input the training sample into the initial model with updated parameters.

[0053] A third aspect of an embodiment of the present application provides a three-dimensional face recognition generation device, including:

[0054] Processor, memory, input-output unit, and bus;

[0055] The processor is connected to the memory, the input and output unit and the bus;

[0056] The processor performs the following operations:

[0057] Obtain the RGB image and 3D depth image of the target face;

[0058] Calculating a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image;

[0059] Obtaining a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix;

[0060] Decomposing the color channels of the RGB image into three independent channels: R, G, and B;

[0061] Decomposing the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z;

[0062] The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0063] Inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0064] A three-dimensional image of the target face is generated by using the target model.

[0065] An embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, any one of the three-dimensional face recognition generation methods of the first aspect is performed.

[0066] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0067] In this application, a method for generating three-dimensional face recognition is designed. First, the RGB image and the three-dimensional depth image of the target face are obtained, and the two are input into the target model trained based on the deep neural network model according to multiple RGB image training samples. The target model is used to extract features and calculate loss values, and finally a three-dimensional image of the target face is generated. This method greatly improves the face recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for generating three-dimensional face recognition in an embodiment of the present application;

[0069] Figure 2-1 This is a flow chart of another embodiment of the method for generating three-dimensional face recognition in the embodiment of the present application;

[0070] Figure 2-2 This is a flow chart of another embodiment of the method for generating three-dimensional face recognition in the embodiment of the present application;

[0071] Figure 3 This is a schematic diagram of an embodiment of a flow chart of a device for generating three-dimensional face recognition in an embodiment of the present application;

[0072] Figure 4 This is a flow chart of another embodiment of a device for generating three-dimensional face recognition in an embodiment of the present application;

[0073] Figure 5 This is a flow chart of another embodiment of a device for generating three-dimensional face recognition in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0075] The embodiments of the present application provide a method and a related device for generating three-dimensional face recognition, which greatly improves the face recognition rate.

[0076] See also Figure 1 In one embodiment of the present application, a method for generating three-dimensional face recognition includes:

[0077] 101. Obtain an RGB image and a three-dimensional depth image of the target face;

[0078] It should be noted that in the embodiment of the present application, the RGB image is an MxNx3 color pixel array, in which each color pixel is a three-value group, and the three values ​​correspond to the red, green and blue components of the RGB image at a specific spatial position. RGB can also be regarded as a "stack" formed by three grayscale images. When they are sent to the red, green and blue input terminals of a color display, a color image will be generated on the screen. By convention, the three images that form an RGB color image are called red, green and blue component images. If a picture has (or appears to have) height, width and depth, then it can be called a three-dimensional (or 3D) image, or a three-dimensional depth image.

[0079] In the embodiment of the present application, in order to improve the face recognition rate, it is necessary to simultaneously obtain the RGB image and the three-dimensional depth image of the target face to perform relevant calculations to obtain the final RGB-D image, that is, the three-dimensional image of the target face.

[0080] 102. Calculate a rotation and translation matrix according to internal and external parameters of the RGB image and the three-dimensional depth image, where the rotation and translation matrix can reflect a mapping relationship between the RGB image and the three-dimensional depth image;

[0081] It should be noted that in an embodiment of the present application, after obtaining the RGB image and the three-dimensional depth image, it is also necessary to obtain the mapping relationship from the RGB image to the three-dimensional depth image, or the mapping relationship from the three-dimensional depth image to the RGB image. Specifically, the rotation and translation matrix is ​​calculated through the internal and external parameters of the RGB image and the three-dimensional depth image, and the mapping relationship is obtained through the rotation and translation matrix. The rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image.

[0082] 103. Obtain a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix;

[0083] It should be noted that in the embodiment of the present application, the mapping relationship mapped by the rotation and translation matrix is ​​rotated and / or translated accordingly to adjust the positional relationship between the RGB image and the three-dimensional depth image. This step is to better eliminate the positional deviation between the RGB image and the three-dimensional depth image.

[0084] 104. Decompose the color channel of the RGB image into three independent channels of R, G, and B;

[0085] 105. Decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z;

[0086] It should be noted that, in the embodiment of the present application, in order to input the image information of the acquired RGB image and the three-dimensional depth image into the target model for related training, it is necessary to decompose the RGB color channels of the acquired RGB image into three independent channels of R, G, and B, and at the same time decompose the facial image information of the acquired three-dimensional depth image into three independent channels of x, y, and z. It should be noted that x, y, and z represent three-dimensional coordinates in the image. In practice, these three channels represent information such as size, that is, they correspond to the three dimensions of length, width, and height respectively.

[0087] 106. Use the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0088] It should be noted that in the embodiment of the present application, the three color channels R, G, and B and the x, y, and z depth map coordinate channels are connected by the Concat function, and these channels are used as the input of the next network layer, that is, as the data input channel of the target model that has been trained in advance.

[0089] 107. Input the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0090] In the embodiment of the present application, six-dimensional information is input through the data input channel and training is performed in the target model.

[0091] It should be noted that, usually in the field of numerical analysis, 1-5 hidden layers can solve most problems. When processing image or speech data, the network structure will be more complicated and require hundreds of neural layers. At the same time, it also takes a lot of computing power to train. Therefore, you can use some pre-trained models, such as YOLO, ResNet, VGG, etc., extract the main parts of these network layers, put them into your own network, and train the model on their basis. In this case, the model still only needs to train the few layers added later.

[0092] In the embodiments of the present application, a neural network and data are designed, and the network layer may use ResNet5 or a VGG convolutional neural network, or may use other network structures instead, which is not limited in the present application.

[0093] 108. Generate a three-dimensional image of the target face using the target model.

[0094] It should be noted that in the embodiment of the present application, training is performed through the target model, and the loss value is calculated, the model is adjusted according to the loss value, and finally a three-dimensional image of the target face is generated.

[0095] In the embodiment of the present application, a method for generating three-dimensional face recognition is designed. First, an RGB image and a three-dimensional depth image of the target face are obtained, and both are input into a target model trained based on a deep neural network model according to multiple RGB image training samples. Feature extraction and loss value calculation are performed through the target model, and finally a three-dimensional image of the target face is generated. This method greatly improves the face recognition rate.

[0096] The above is a general description of the generation method of three-dimensional face recognition. The following is a detailed introduction to the generation method of three-dimensional face recognition.

[0097] Please refer to FIG. 2 , another embodiment of the method for generating three-dimensional face recognition in the embodiment of the present application includes:

[0098] 201. Obtain an RGB image of the target face through an RGB camera;

[0099] 202. Acquire a three-dimensional depth image of the target face through a three-dimensional depth camera;

[0100] It should be noted that, in the embodiment of the present application, an RGB image and a three-dimensional image of the face of the same person are acquired by an RGB camera and a three-dimensional depth camera respectively.

[0101] 203. Calculate a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, where the rotation and translation matrix can reflect a mapping relationship between the RGB image and the three-dimensional depth image;

[0102] 204. Obtain a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix;

[0103] In the embodiment of the present application, steps 203 to 204 are similar to the aforementioned steps 102 to 103 and are not repeated here.

[0104] 205. Align the rotated and / or translated RGB image and the 3D depth image to the same size;

[0105] It should be noted that in the embodiment of the present application, in order to better fuse the RGB image and the three-dimensional depth image, it is also necessary to align the acquired RGB image and the three-dimensional depth image to the same size, for example, a size of 112*112 can be used. There is no restriction on the specific size as long as the size is consistent.

[0106] 206. Decompose the color channel of the RGB image into three independent channels of R, G, and B;

[0107] 207. Decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z;

[0108] In the embodiment of the present application, steps 206 to 207 are similar to the aforementioned steps 104 to 105 and are not repeated here.

[0109] 208. Obtain a set of training samples, where the training samples include at least two RGB images of human faces;

[0110] It should be noted that it is generally difficult to obtain RGB images and depth images of so many people. Generally, an initial model is trained with RGB information first, and then fine-tuned with RGB-D information to obtain a better recognition model. Therefore, this application first uses the open source ms1m face recognition data to train a visible light face recognition model with RGB input, and then repeats the training in the manner described in this application to obtain the final target model.

[0111] In the embodiment of the present application, a group of RGB images containing multiple faces are first obtained as training samples for training the initial model.

[0112] 209. Input the training sample into an initial model, where the initial model is a model established based on a deep neural network;

[0113] It should be noted that in the embodiment of the present application, after obtaining the training samples, each training object (RGB image) is successively input into the initial model, which is a model established based on a deep neural network (such as ResNet).

[0114] 210. Extracting a feature vector of the training sample by using the initial model;

[0115] 211. Calculate a loss value according to the feature vector;

[0116] In an embodiment of the present application, the feature vector of each training object is extracted through the convolutional neural network of the initial model, and then flattened by the fully connected layer, and then sent to the loss function to implement classification to obtain a loss value. It should be noted that the loss function can be a softmax loss function, a cosface loss function, an arcface loss function, or other loss functions, and the specific embodiments of the present application are not limited.

[0117] In addition, it should be noted that the feature norm extracted from high-quality images is large, and the feature norm extracted from low-quality images is small. During backward propagation, low-quality image features will produce larger gradients and will also receive more attention from the network.

[0118] 212. Determine whether the loss value is less than a preset value. If so, execute step 213; if not, execute step 214.

[0119] It should be noted that in the embodiment of the present application, a preset value is set, and whether the network has converged is confirmed by judging whether the loss value is less than the preset value. If the loss value is less than the preset value, it means that the network has converged, and step 213 is executed; if the loss value is greater than or equal to the preset value, it means that the network has not converged, and step 214 is executed.

[0120] 213. Determine the initial model as a target model;

[0121] It should be noted that, in the embodiment of the present application, if the loss value is less than the preset value, it means that the network has converged, the initial model is determined as the target model, and step 215 is executed.

[0122] 214. Adjust the parameters of the initial model according to the loss value, and re-input the training sample into the initial model with updated parameters;

[0123] It should be noted that in the embodiment of the present application, if the loss value is greater than or equal to the preset value, it means that the network has not converged, and the parameters of the initial model are adjusted according to the loss value, and the training sample is re-input into the initial model after the updated parameters, the loss value is recalculated, and it is determined whether the loss value is less than the preset value. This step is terminated until the loss value is less than the preset value, and step 213 is entered.

[0124] 215. Using the concat function, the three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of the target model, wherein the concat function is used to connect multiple channels as inputs of the next network layer, and the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0125] It should be noted that in the embodiment of the present application, the three color channels R, G, and B and the x, y, and z depth map coordinate channels are connected by the Concat function, and these channels are used as the input of the next network layer, that is, as the data input channel of the target model that has been trained in advance.

[0126] 216. Adjust the input sizes of the six data input channels to be the same as the sizes of the RGB image and the three-dimensional depth image;

[0127] It should be noted that the network data input size is consistent with the size of the RGB image and the 3D depth image after alignment in step 205, that is, 112*112, and the data input is 6 channels (RGBxyz). During training, the Batchsize is set according to the actual situation. The Batchsize size is negatively correlated with the complexity of the network, that is, different networks will occupy different amounts of video memory under the same Batchsize. For example, if 8 Tesla V100 32G cards are used, the Batchsize is actually set to 512, that is, the network input size is 512*6*112*112.

[0128] 217. Input the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0129] In the embodiment of the present application, step 217 is similar to the aforementioned step 107 and will not be repeated here.

[0130] 218. Extracting a feature vector of the facial image information through the target model;

[0131] 219. Calculate the total loss value of the target model according to the feature vector;

[0132] In an embodiment of the present application, the feature vector of each training object is extracted through the convolutional neural network of the target model, and then flattened by the fully connected layer, and then sent to the loss function for classification to obtain the loss value.

[0133] It should be noted that the feature norm extracted from high-quality images is large, and the feature norm extracted from low-quality images is small. During backward propagation, low-quality image features will produce larger gradients and will also receive more attention from the network.

[0134] 220. Determine whether the total loss value is less than a preset value;

[0135] It should be noted that in the embodiment of the present application, a preset value is set, and whether the network has converged is confirmed by judging whether the total loss value is less than the preset value. If the loss value is less than the preset value, it means that the network has converged, and step 221 is executed; if the loss value is greater than or equal to the preset value, it means that the network has not converged, and step 222 is executed.

[0136] It should be noted that the embodiment of the present application designs a loss function to supervise the input data and labels until the network converges. The loss function can be softmax, cosface, arcface, asofmax and other loss functions.

[0137] Additionally, choosing the learning rate is very important and needs to be readjusted whenever you adjust other parameters of the network.

[0138] In order to find the best learning rate, you can start from a very low value and then slowly multiply it by a constant until it reaches a very high value. For example, in the embodiment of the present application, the Arcface loss used in training, the initial learning rate can be set to 0.1, and the learning rate is multiplied by 0.1 every 150,000 training times until 500,000 training times.

[0139] 221. Generate a three-dimensional image of a target face using the target model.

[0140] It should be noted that in the embodiment of the present application, if the total loss value is less than the preset value, it means that the network has converged and the three-dimensional image of the target face is generated directly through the target model.

[0141] 222. Adjust the parameters of the target model according to the total loss value, and re-input the facial image information into the updated target model.

[0142] It should be noted that in the embodiment of the present application, if the total loss value is greater than or equal to the preset value, it means that the network has not converged, and the parameters of the target model are adjusted according to the total loss value, and the facial image information is re-input into the updated target model, the loss value is recalculated, and it is determined whether the loss value is less than the preset value. This step is terminated until the loss value is less than the preset value, and step 221 is entered.

[0143] In an embodiment of the present application, a method for generating three-dimensional face recognition based on an RGBD camera is designed. An RGB image and a three-dimensional depth image of the same person's face are obtained through an RGB camera and a three-dimensional depth camera respectively. The two together constitute three-dimensional face information, which is then put into a pre-trained target model for training to generate a three-dimensional image of the target face. The entire training framework is completed based on a deep network, which is simple and fast, and greatly improves the face recognition rate.

[0144] See also Figure 3 In one embodiment of the present application, a device for generating three-dimensional face recognition includes:

[0145] The first acquisition unit 301 is used to acquire an RGB image and a 3D depth image of a target face;

[0146] A first calculation unit 302 is used to calculate a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, where the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image;

[0147] A rotation and translation unit 303, configured to obtain a positional relationship between the RGB image and the three-dimensional depth image by rotation and / or translation according to the rotation and translation matrix;

[0148] A first decomposition unit 304 is used to decompose the color channel of the RGB image into three independent channels of R, G, and B;

[0149] A second decomposition unit 305 is used to decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y and z;

[0150] A synthesis unit 306 is used to use the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0151] A first input unit 307, used to input the face image information of the RGB image and the 3D depth image into the target model through the six data input channels;

[0152] The generating unit 308 is configured to generate a three-dimensional image of the target face using the target model.

[0153] In the embodiment of the present application, a generation method for three-dimensional face recognition is designed. First, the first acquisition unit 301 acquires the RGB image and the three-dimensional depth image of the target face, and then the first input unit 307 inputs the two into the target model trained based on the deep neural network model according to multiple RGB image training samples, and the target model is used to extract features and calculate loss values, and finally the generation unit 308 generates the three-dimensional image of the target face. This method greatly improves the face recognition rate.

[0154] The above is a general description of the functions of each unit of the generation device for three-dimensional face recognition. The following is a detailed description of the functions of each unit of the generation device for three-dimensional face recognition.

[0155] See also Figure 4 In the embodiment of the present application, another embodiment of the generation device of three-dimensional face recognition includes:

[0156] The first acquisition unit 401 is used to acquire an RGB image and a 3D depth image of a target face;

[0157] A first calculation unit 402 is used to calculate a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, where the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image;

[0158] A rotation and translation unit 403, configured to obtain a positional relationship between the RGB image and the three-dimensional depth image by rotation and / or translation according to the rotation and translation matrix;

[0159] A first decomposition unit 404 is used to decompose the color channel of the RGB image into three independent channels of R, G, and B;

[0160] A second decomposition unit 405 is used to decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y and z;

[0161] A second acquisition unit 406 is used to acquire a set of training samples, where the training samples include at least two RGB images of human faces;

[0162] A second input unit 407, used to input the training sample into an initial model, where the initial model is a model established based on a deep neural network;

[0163] A second calculation unit 408, configured to calculate the loss value of the training sample using the initial model;

[0164] A judging unit 409 is used to judge whether the loss value is less than a preset value;

[0165] A determination unit 410, configured to determine the initial model as a target model after the determination unit 409 determines that the loss value is less than a preset value;

[0166] The adjustment unit 411 is used to adjust the parameters of the initial model according to the loss value after the judgment unit 409 determines that the loss value is greater than or equal to the preset value, and re-input the training sample into the initial model with updated parameters.

[0167] A synthesis unit 412 is used to use the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0168] A first input unit 413, used to input the face image information of the RGB image and the 3D depth image into the target model through the six data input channels;

[0169] The generating unit 414 is configured to generate a three-dimensional image of the target face using the target model.

[0170] In the embodiment of the present application, the functions of each unit module are the same as those described above. Figure 1 The steps correspond to those in the embodiment shown in FIG. 2 and are not described again here.

[0171] See also Figure 5 Another embodiment of the generation device of three-dimensional face recognition in the embodiment of the present application includes:

[0172] Processor 501, memory 502, input and output unit 503 and bus 504;

[0173] The processor 501 is connected to the memory 502, the input and output unit 503 and the bus 504;

[0174] The processor 501 performs the following operations:

[0175] Obtain the RGB image and 3D depth image of the target face;

[0176] Calculating a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image;

[0177] Obtaining a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix;

[0178] Decomposing the color channels of the RGB image into three independent channels: R, G, and B;

[0179] Decomposing the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z;

[0180] The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples;

[0181] Inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels;

[0182] A three-dimensional image of the target face is generated by using the target model.

[0183] In this embodiment, the function of the processor 501 is the same as that of the aforementioned Figure 1 The steps in the embodiment shown in FIG. 2 correspond to each other and will not be described in detail here.

[0184] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0186] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for generating three-dimensional face recognition, characterized in that: include: Obtain the RGB image and 3D depth image of the target face; A rotation and translation matrix is ​​calculated according to the internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image, and the mapping relationship mapped by the rotation and translation matrix is ​​rotated and / or translated accordingly to adjust the positional relationship between the RGB image and the three-dimensional depth image to eliminate the positional deviation between the RGB image and the three-dimensional depth image; Obtaining a positional relationship between the RGB image and the three-dimensional depth image by rotating and / or translating according to the rotation and translation matrix; Aligning the rotated and / or translated RGB image and the three-dimensional depth image to the same size; Decomposing the color channels of the RGB image into three independent channels: R, G, and B; Decomposing the coordinate channel of the three-dimensional depth image into three independent channels of x, y, and z; The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples; Inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels; A three-dimensional image of the target face is generated by using the target model.

2. The generation method according to claim 1, characterized in that: Before using the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of the target model, the method further includes: Obtain a set of training samples, wherein the training samples include RGB images of at least two human faces; Inputting the training sample into an initial model, wherein the initial model is a model established based on a deep neural network; Calculating the loss value of the training sample by using the initial model; Determine whether the loss value is less than a preset value, and if so, determine the initial model as the target model; If not, the parameters of the initial model are adjusted according to the loss value, and the training samples are re-input into the initial model with updated parameters.

3. The generation method according to claim 2, characterized in that: The calculating the loss value of the training sample by using the initial model includes: Extracting a feature vector of the training sample through the initial model; The loss value is calculated according to the feature vector.

4. The generation method according to claim 1, characterized in that: Before inputting the facial image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels, the method further includes: The input sizes of the six data input channels are adjusted to be the same as the sizes of the RGB image and the three-dimensional depth image.

5. The generation method according to any one of claims 1 to 4, characterized in that: The method uses the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of the target model, including: The three channels of R, G, and B and the three channels of x, y, and z are used as six data input channels of the target model through the concat function, and the concat function is used to connect multiple channels as the input of the next network layer.

6. The generation method according to any one of claims 1 to 4, characterized in that: The step of generating a target face three-dimensional image by using the target model comprises: Extracting a feature vector of the facial image information through the target model; Calculate the total loss value of the target model according to the feature vector; Determine whether the total loss value is less than a preset value, and if so, generate a three-dimensional image of the target face through the target model; If not, the parameters of the target model are adjusted according to the total loss value, and the facial image information is re-input into the updated target model.

7. The generation method according to any one of claims 1 to 4, characterized in that: The step of obtaining an RGB image and a three-dimensional depth image of a target face includes: Obtain an RGB image of the target face through an RGB camera; A three-dimensional depth image of the target face is acquired through a three-dimensional depth camera.

8. A three-dimensional face recognition generation device, characterized in that: include: A first acquisition unit is used to acquire an RGB image and a three-dimensional depth image of a target face; A first calculation unit is used to calculate a rotation and translation matrix according to the internal and external parameters of the RGB image and the three-dimensional depth image, wherein the rotation and translation matrix can reflect the mapping relationship between the RGB image and the three-dimensional depth image, and perform corresponding rotation and / or translation according to the mapping relationship mapped by the rotation and translation matrix to adjust the positional relationship between the RGB image and the three-dimensional depth image to eliminate the positional deviation between the RGB image and the three-dimensional depth image; A rotation and translation unit, configured to obtain a positional relationship between the RGB image and the three-dimensional depth image by rotation and / or translation according to the rotation and translation matrix; and for adjusting the input sizes of the six data input channels to be the same as the sizes of the RGB image and the three-dimensional depth image; A first decomposition unit is used to decompose the color channel of the RGB image into three independent channels of R, G, and B; A second decomposition unit is used to decompose the coordinate channel of the three-dimensional depth image into three independent channels of x, y and z; A synthesis unit, used to use the three channels of R, G, and B and the three channels of x, y, and z as six data input channels of a target model, wherein the target model is a model trained based on a deep neural network model according to multiple RGB image training samples; A first input unit, used for inputting the face image information of the RGB image and the three-dimensional depth image into the target model through the six data input channels; A generating unit is used to generate a three-dimensional image of a target face using the target model.

9. The generating device according to claim 8, characterized in that Before the synthesis unit, the device further comprises: A second acquisition unit, configured to acquire a set of training samples, wherein the training samples include RGB images of at least two human faces; A second input unit, used to input the training sample into an initial model, wherein the initial model is a model established based on a deep neural network; A second calculation unit, used to calculate the loss value of the training sample through the initial model; A judging unit, used to judge whether the loss value is less than a preset value; a determination unit, configured to determine the initial model as a target model after the determination unit determines that the loss value is less than a preset value; An adjustment unit is used to adjust the parameters of the initial model according to the loss value after the judgment unit determines that the loss value is greater than or equal to a preset value, and re-input the training sample into the initial model with updated parameters.