Face reconstruction method, electronic device and storage medium based on generative adversarial network

Abstract

The present invention relates to the field of image processing technology, and in particular to a face reconstruction method, electronic device and storage medium based on a generative adversarial network. The present invention uses face images with occlusion and face images without occlusion to simultaneously train a joint network, and through the mutual influence and restriction of the two face images, the accuracy of face reconstruction can be effectively improved; at the same time, the target generative adversarial network obtained based on the training of the two face images can simultaneously reshape the two types of face images, thereby increasing the scope of application.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a face reconstruction method, electronic device and storage medium based on a generative adversarial network. Background Art

[0002] With the development and widespread application of face recognition technology, the security of applications that rely on face recognition technology has gradually received attention. Traditional face attribute editing methods, such as image filtering and shape models, can find key areas of the face, but because these technologies often focus on processing high-level semantic features and directly operate on raw pixels, while ignoring the processing of local texture and color information, the generated images have poor accuracy and realism. In addition, these methods usually require a lot of manual parameter adjustments, which are complicated to operate and cannot meet the requirements of real-time or efficient modification, which further affects the accuracy of the generated images. Therefore, it is urgent to solve this problem. Summary of the invention

[0003] In order to avoid and overcome the technical problems existing in the prior art, the present invention provides a face reconstruction method, an electronic device and a storage medium based on a generative adversarial network. The present invention can effectively improve the accuracy of face reconstruction.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A face reconstruction method based on a generative adversarial network comprises the following steps:

[0006] S1, obtaining original face images, and performing face correction on the original face images so that each original face image is in a unified coordinate space, and obtaining a corrected face image after the original face image is corrected;

[0007] S2, performing partial or complete occlusion processing on the face in the corrected face image, and recording the image obtained after the occlusion processing as the occluded face image;

[0008] S3, encoding the corrected face image into a first latent vector through an encoding operation; and encoding the occluded face image into a second latent vector through an encoding operation;

[0009] S4. Based on the first latent vector and the second latent vector, training a target generative adversarial network;

[0010] S5. Use target generative adversarial network for face reshaping.

[0011] As a further solution of the present invention: the specific process of training the target generative adversarial network is as follows:

[0012] S41, inputting the first latent vector into the first generative adversarial network to generate a first generated image; then inputting the first generated image into the first neural network to classify the first generated image and assign a classification label corresponding to the first generated image;

[0013] S42, constructing a first linear regression model with each pixel of the first latent vector as a feature and with the classification label of the first generated image corresponding to the first latent vector as an attribute; inputting each first latent vector and the corresponding classification label into the first linear regression model to train the first linear regression model, and stopping the training after reaching a set training accuracy, and obtaining each first feature weight of the first linear regression model at this time; and using each first feature weight to weight each corresponding feature in the first latent vector, so as to convert each first latent vector into a corresponding first weighted latent vector;

[0014] S43, inputting the second latent vector into the second generative adversarial network and processing it according to the processing process from step S41 to step S42, so as to obtain a corresponding second linear regression model, and converting each second latent vector into a corresponding second weighted latent vector;

[0015] S431, inputting the second latent vector into the second generative adversarial network to generate a second generated image; then inputting the second generated image into the second neural network to classify the second generated image and assign a classification label corresponding to the second generated image;

[0016] S432, constructing a second linear regression model with each pixel of the second latent vector as a feature and with the classification label of the second generated image corresponding to the second latent vector as an attribute; inputting each second latent vector and the corresponding classification label into the second linear regression model to train the second linear regression model, and stopping the training after reaching a set training accuracy, and obtaining each second feature weight of the second linear regression model at this time; and using each second feature weight to weight each corresponding feature in the second latent vector, so as to convert each second latent vector into a corresponding second weighted latent vector;

[0017] S44, performing vector splicing on the first weighted latent vector and the second weighted latent vector corresponding to the same original face image to form a spliced ​​vector;

[0018] S45. Combine the first generative adversarial network and the second generative adversarial network in parallel to form a joint network, and then input the concatenated vectors into the first generative adversarial network and the second generative adversarial network in the joint network respectively, so as to train the first generative adversarial network and the second generative adversarial network in the joint network simultaneously, and stop the training after the joint network reaches the set training accuracy; the joint network that has completed the training is called the target generative adversarial network.

[0019] As a further solution of the present invention: the face correction process is as follows:

[0020] S11, inputting the original face image into a face detector for face detection to determine the position of the face in the original face image;

[0021] S12, using a pre-trained facial key point prediction model to extract key points from the face detected in step S11;

[0022] S13, selecting a number of key points from the extracted key points as standard key points;

[0023] S14, calculating the affine transformation matrix between the original facial image and the standard key points, and using the affine transformation matrix, performing translation, rotation and scaling operations on the original facial image to adjust the original facial image to the position corresponding to the standard key points; through the operations of steps S11 to S14, the adjustment of each original facial image can be completed, thereby making each original facial image in a unified coordinate space.

[0024] As a further solution of the present invention: the first linear regression model and the second linear regression model both adopt ridge regression, and the objective function of ridge regression is as follows:

[0025] ;

[0026] In the formula, L represents the objective function of ridge regression; X The feature matrix corresponding to the first latent vector or the feature matrix corresponding to the second latent vector is collectively referred to as a feature matrix; W represents a first weight matrix composed of each first feature weight or a second weight matrix composed of each second feature weight, collectively referred to as a weight matrix; Y represents the true value of the classification label of the current first generated image or the true value of the classification label of the current second generated image, collectively referred to as the true value of the classification label; ||·||2 represents the second-order norm; β Represents a trade-off parameter.

[0027] As a further solution of the present invention: when obtaining the weight matrix W Then, define the weight matrix W Attribute Strength α , and the attribute strength α With the weight matrix W Multiply each other to obtain the corresponding enhanced weight matrix Z=α × W ; Then the weight matrix will be strengthened Z The corresponding latent vector Z Add, and obtain the corresponding weighted potential vector Z'=Z+ Z .

[0028] As a further solution of the present invention: the objective function of the joint network is as follows:

[0029] ;

[0030] In the formula, M represents the objective function of the joint network; G 1 means correcting the face image, represents the target image generated by the concatenated vector in the joint network in the first adversarial generation network; G 2 means blocking the face image, represents the target image generated by the concatenated vector in the joint network in the second adversarial generation network; λ Indicates an adjustable parameter.

[0031] As a further solution of the present invention: using image editing software, partially or completely occlude the face in the corrected face image according to set requirements.

[0032] As a further solution of the present invention: the process of face reshaping using a target generative adversarial network is as follows: a new original face image to be reshaped is obtained, and according to the occlusion condition of the original face image, the image after face correction of the original face image is input into the corresponding generative adversarial network in the target generative adversarial network for face reshaping, that is: if the original face image is not occluded, the face corrected image corresponding to the original face image is input into the first generative adversarial network to generate a target image, and the target image is the reshaped image; if the original face image is occluded, the face corrected image corresponding to the original face image is input into the second generative adversarial network to generate a target image, and the target image is the reshaped image.

[0033] As a further solution of the present invention: the neural network adopted is a convolutional neural network.

[0034] An electronic device comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected in sequence, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the above-mentioned reshaping method.

[0035] A storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned reshaping method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention uses face images with occlusion and face images without occlusion to simultaneously train a joint network. Through the mutual influence and restriction of the two face images, the accuracy of face reconstruction can be effectively improved. At the same time, the target generative adversarial network obtained based on the two face images training can simultaneously reshape these two types of face images, thereby improving the scope of application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the present invention.

[0039] Figure 2 It is a schematic diagram of the structure of the electronic device in the present invention.

[0040] In the figure: 10, electronic device; 11, processor; 12, memory; 13, input device; 14, output device. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1-2 The remodeling method of the present invention mainly includes the following contents.

[0043] 1. Face correction.

[0044] The original face images are obtained, and face correction is performed on the original face images so that each original face image is in a unified coordinate space, and a corrected face image is obtained after the original face images are corrected.

[0045] Through face correction, the face can be adjusted to a standard position so that the model is not affected by differences in face angles and expressions, thereby achieving better editing results. This process usually includes two steps: face key point detection and face correction. The specific process is as follows:

[0046] S11, inputting the original face image into a face detector for face detection to determine the position of the face in the original face image.

[0047] S12. Use a pre-trained facial key point prediction model to extract key points from the face detected in step S11.

[0048] S13. Select several key points from the extracted key points as standard key points.

[0049] S14, calculate the affine transformation matrix between the original face image and the standard key points, and use the affine transformation matrix to translate, rotate and scale the original face image to adjust the original face image to the position corresponding to the standard key points. Through the operations of steps S11 to S14, the adjustment of each original face image can be completed, so that each original face image is in a unified coordinate space.

[0050] Face correction example:

[0051] Face key point detection: Use the dlib library in machine learning to detect face key points. The dlib library provides a 68-point face feature point model that can find the key areas of the face, such as the eyes, nose, mouth, and face contour. Specifically, first use the face detector of the dlib library to detect the original face image. Then, use the pre-trained 68-point face feature point model to extract key points from the detected face. In this way, the key points of each face image are obtained.

[0052] Face correction: After obtaining the key points of the face, face correction is performed through affine transformation. Specifically, the centers of the two eyes and the center of the mouth of the face are usually selected as standard key points, and then the affine transformation matrix between the input original face image and the standard key points is calculated. By applying this affine transformation matrix, the input face image can be adjusted to the position corresponding to the standard key points, thereby completing the face correction. This also means that all faces will be converted to a unified coordinate space, which is conducive to model learning and attribute editing.

[0053] 2. Get the occluded face image.

[0054] Partial or complete occlusion of the face in the corrected face image usually depends on the image editing software used, such as Photoshop. The following is a basic step for occlusion processing based on Photoshop:

[0055] 1) Open the image:

[0056] Start Photoshop and open the corrected face image that needs to be occluded.

[0057] 2) Select the tool:

[0058] Choose the appropriate tool based on the specific needs of the mask. Common tools include the "Lasso Tool", "Polygonal Lasso Tool" or "Magic Wand Tool", etc., which are used to select the area that needs to be masked.

[0059] 3) Create a selection:

[0060] Use the selection tool to draw a closed selection on the corrected face image to select part or all of the face that needs to be masked.

[0061] 4) Apply occlusion:

[0062] Right-click on the selection and choose the Fill option.

[0063] In the pop-up "Fill" dialog box, select "Content" as "Color" or "Pattern" and set the desired color or pattern. If you select color, you can customize the color or use the preset color; if you select pattern, you can select a suitable pattern to fill.

[0064] Click the "OK" button and the software will fill the selected area with the selected color or pattern to achieve the masking effect.

[0065] 5) Save the image:

[0066] After the occlusion process is completed, the modified corrected face image is saved to the specified location. The types of occluded face images presented in the end include faces wearing sunglasses, faces with painted faces, faces wearing masks, etc.

[0067] 3. Get the weighted latent vector.

[0068] The corrected face image is encoded into a first latent vector through an encoding operation; and the occluded face image is encoded into a second latent vector through an encoding operation.

[0069] The first latent vector is input into the first generative adversarial network to generate a first generated image; the first generated image is then input into the first neural network to classify the first generated image and assign a classification label corresponding to the first generated image. The second latent vector is input into the second generative adversarial network to generate a second generated image; the second generated image is then input into the second neural network to classify the second generated image and assign a classification label corresponding to the second generated image. Classification labels include gender, age, etc.

[0070] A first linear regression model is constructed with each pixel of the first latent vector as a feature and the classification label of the first generated image corresponding to the first latent vector as an attribute. Each first latent vector and the corresponding classification label are input into the first linear regression model to train the first linear regression model, and the training is stopped after reaching the set training accuracy, and each first feature weight of the first linear regression model is obtained at this time. Each first feature weight is used to weight each corresponding feature in the first latent vector to convert each first latent vector into a corresponding first weighted latent vector.

[0071] A second linear regression model is constructed with each pixel of the second latent vector as a feature and the classification label of the second generated image corresponding to the second latent vector as an attribute. Each second latent vector and the corresponding classification label are input into the second linear regression model to train the second linear regression model, and the training is stopped after reaching the set training accuracy, and each second feature weight of the second linear regression model is obtained at this time. Each second feature weight is used to weight each corresponding feature in the second latent vector to convert each second latent vector into a corresponding second weighted latent vector.

[0072] To find the direction of change of the corresponding specific attribute in the latent space, all latent vectors are combined with the attribute values ​​of the corresponding face image and used as input data for linear regression. In this step, the accuracy of classification needs to be ensured. In order to prevent overfitting, ridge regression is used in this process, and L1 or L2 regularization is added.

[0073] The first linear regression model and the second linear regression model both use ridge regression, and the objective function of ridge regression is as follows:

[0074] ;

[0075] The key to this objective function is to appropriately select β If the value is too large, the weight matrix W The constraint can reduce the training error, but the resulting model may be too simple, resulting in underfitting. If the value is too small, the weight matrix W The constraints are reduced, and the model will adapt to the training set data as much as possible, which may lead to overfitting.

[0076] In getting the weight matrix W Then, define the weight matrix W Attribute Strength α , and the attribute strength α With the weight matrix W Multiply each other to obtain the corresponding enhanced weight matrix Z=α ×W ; Then the weight matrix will be strengthened Z The corresponding latent vector Z Add, and obtain the corresponding weighted potential vector Z'=Z+ Z . This can be done by changing α The size of the attribute is used to control the degree of attribute change. α It can be positive or negative. A positive number indicates an increase in the attribute, while a negative number indicates a decrease in the attribute. α Values ​​can also produce varying degrees of property change, providing a great deal of flexibility.

[0077] 4. Generate target generative adversarial network.

[0078] The first weighted latent vector and the second weighted latent vector corresponding to the same original face image are vector-concatenated to form a concatenated vector. The first generative adversarial network and the second generative adversarial network are combined in parallel to form a joint network, and then the concatenated vector is respectively input into the first generative adversarial network and the second generative adversarial network in the joint network to train the first generative adversarial network and the second generative adversarial network in the joint network simultaneously, and the training is stopped after the joint network reaches the set training accuracy; the joint network that has completed the training is called the target generative adversarial network.

[0079] The first generative adversarial network and the second generative adversarial network are parallel and independent of each other and are used separately according to the type of the original face image.

[0080] A new original face image to be reshaped is obtained, and according to the occlusion condition of the original face image, the face-corrected image of the original face image is input into the corresponding generative adversarial network in the target generative adversarial network for face reshaping, that is: if the original face image is not occluded, the face-corrected image corresponding to the original face image is input into the first generative adversarial network to generate a target image, and the target image is the reshaped image; if the original face image is occluded, the face-corrected image corresponding to the original face image is input into the second generative adversarial network to generate a target image, and the target image is the reshaped image.

[0081] The present invention combines the first generative adversarial network GAN1 and the second generative adversarial network GAN2 by means of feature fusion. Specifically, the first generative adversarial network GAN1 and the second generative adversarial network GAN2 respectively include a generator (G1, G2) and a discriminator (D1, D2).

[0082] Generator structure: The structures of generators G1 and G2 are basically the same, both using encoder-decoder structures, but with different parameters. The encoder is used to map the input image to a low-dimensional latent space, and the decoder decodes the weighted latent vector into the target image.

[0083] Discriminator structure: The structures of discriminator D1 and discriminator D2 are basically the same, which are used to distinguish between real images and target images.

[0084] Feature fusion: During the training process, the feature map output by encoder G1 (i.e., the first weighted latent vector) and the feature map output by encoder G2 (i.e., the second weighted latent vector) are concatenated. The concatenated feature map (concatenated vector) is input into the decoder of each generative adversarial network to generate the final target image.

[0085] The objective function of the joint network is as follows:

[0086] ;

[0087] By optimizing the above objective function M , so that the joint network learns to pay attention to its occluded regions when encoding and generating face images.

[0088] Attribute Editing: When performing attribute editing, use the occlusion recognizer to perform occlusion analysis on the input image. Then send the image to the occlusion encoder for encoding to obtain a latent vector containing occlusion information. Perform attribute editing according to the previously designed method, but when applying the attribute vector, use the occlusion information obtained by the occlusion recognizer to adjust the influence of the attribute vector. For example, if the recognizer determines that there is severe occlusion in the image, it is necessary to reduce the influence of the attribute vector on the latent vector to prevent the occlusion from having too much influence on the editing result.

[0089] 5. Introduce the attention mechanism.

[0090] The attention mechanism is an important and effective tool in deep learning, especially for image processing tasks. By introducing the attention mechanism, the joint network can focus on important parts of the image (such as eyes, nose, mouth, etc. for face editing), so that more attention is paid to changes in these key areas when doing attribute editing. This can not only improve the accuracy of editing, but also help deal with occlusion problems to a certain extent.

[0091] Self-Attention Layer: This type of attention mechanism can capture global dependencies in the input. For an input feature map, this layer calculates the attention scores between all positions and uses these scores to update each position in the feature map.

[0092] In the structure, a self-attention layer can be added at an appropriate position in the encoder or decoder. The specific operation is as follows:

[0093] For the input feature map X First, we need to calculate the query Q, key K and value V, which is usually achieved by applying different one-dimensional convolution operations: Q=Conv1D(Q)X, K=Conv1D(K)X, V=Conv1D(V)X; where Conv1D(Q), Conv1D(K) and Conv1D(V) are one-dimensional convolutional layers used to obtain different embeddings.

[0094] Then, the attention scores of each position with respect to other positions are calculated: scores = softmax(QK'). Here K' is the transpose of the key K, and softmax is used to normalize the attention scores to between 0 and 1.

[0095] Finally, use this score to perform a weighted sum on V to obtain the updated feature map: Z=scoresV.

[0096] 6. Post-processing and quality control.

[0097] The images generated by GAN often have some undesirable features, such as over-smoothing or some undesirable artifacts. This system uses the super-resolution method SRCNN to improve the resolution and clarity of face images, and uses the denoising method DnCNN to remove artifacts.

[0098] SRCNN: Create a three-layer convolutional neural network with 9-5-5 convolution kernel and relu as the activation function.

[0099] Training network: Use low-resolution images as input, high-resolution images as targets, and MSE as loss function.

[0100] Application network: The image generated by GAN is enlarged to the target size and then input into the trained SRCNN network to obtain a super-resolution image.

[0101] DnCNN: Design a multi-layer deep convolutional neural network, in which the first and last layers use convolution kernels of corresponding sizes, the middle layer uses a convolution kernel of size 3x3, and the activation function is set to relu.

[0102] Training network: Use noisy images as input, original images as targets, and MSE as loss function.

[0103] Application network: Input the image generated by GAN into the trained DnCNN network to obtain the denoised image.

[0104] In summary, this method can efficiently and naturally edit the attributes of the face according to the needs of the user, maintaining the high quality and realism of the generated pictures. At the same time, it greatly facilitates the user's operation, provides a better user experience, and has wide application value.

[0105] like Figure 2 As shown, the electronic device 10 includes one or more processors 11 and corresponding memory 12 .

[0106] The processor 11 may be a central processing unit or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions. The memory 12 may include one or more computer program products, which may include various forms of computer storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer storage medium, and the processor 11 may run the program instructions to implement the decision behavior decision method and / or other desired functions of the various embodiments of the present application described above.

[0107] In the example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). For example, the input device 13 may also include a keyboard, a mouse, etc. The output device 14 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0108] Of course, to simplify, Figure 2 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0109] In addition to the above-mentioned methods and devices, the embodiments of the present application may also extend to a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the decision-making behavior decision method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0110] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0111] In addition, an embodiment of the present application may also be a readable computer storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the decision-making behavior decision-making method according to various embodiments of the present application described in the above specific implementation process section of this specification.

[0112] The computer storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a storage medium. The storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0113] Of course, it will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0114] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A face reconstruction method based on generative adversarial network, characterized in that: The following steps are involved: S1, obtaining original face images, and performing face correction on the original face images so that each original face image is in a unified coordinate space, and obtaining a corrected face image after the original face image is corrected; S2, performing partial or complete occlusion processing on the face in the corrected face image, and recording the image obtained after the occlusion processing as the occluded face image; S3, encoding the corrected face image into a first latent vector through an encoding operation; and encoding the occluded face image into a second latent vector through an encoding operation; S4. Based on the first latent vector and the second latent vector, training a target generative adversarial network; S5, face reshaping using target generative adversarial network; The specific process of training the target generative adversarial network is as follows: S41, inputting the first latent vector into the first generative adversarial network to generate a first generated image; then inputting the first generated image into the first neural network to classify the first generated image and assign a classification label corresponding to the first generated image; S42, constructing a first linear regression model with each pixel of the first latent vector as a feature and with the classification label of the first generated image corresponding to the first latent vector as an attribute; inputting each first latent vector and the corresponding classification label into the first linear regression model to train the first linear regression model, and stopping the training after reaching a set training accuracy, and obtaining each first feature weight of the first linear regression model at this time; and using each first feature weight to weight each corresponding feature in the first latent vector, so as to convert each first latent vector into a corresponding first weighted latent vector; S43, inputting the second latent vector into the second generative adversarial network and processing it according to the processing process from step S41 to step S42, so as to obtain a corresponding second linear regression model, and converting each second latent vector into a corresponding second weighted latent vector; S44, performing vector splicing on the first weighted latent vector and the second weighted latent vector corresponding to the same original face image to form a spliced ​​vector; S45, combining the first generative adversarial network and the second generative adversarial network in parallel to form a joint network, and then inputting the concatenated vectors into the first generative adversarial network and the second generative adversarial network in the joint network respectively, so as to train the first generative adversarial network and the second generative adversarial network in the joint network simultaneously, and stopping the training after the joint network reaches a set training accuracy; the joint network that has completed the training is called a target generative adversarial network; The first linear regression model and the second linear regression model both use ridge regression, and the objective function of ridge regression is as follows: ； In the formula, L represents the objective function of ridge regression; X The feature matrix corresponding to the first latent vector or the feature matrix corresponding to the second latent vector is collectively referred to as a feature matrix; W represents a first weight matrix composed of each first feature weight or a second weight matrix composed of each second feature weight, collectively referred to as a weight matrix; Y represents the true value of the classification label of the current first generated image or the true value of the classification label of the current second generated image, collectively referred to as the true value of the classification label; ||·||2 represents the second-order norm; β represents the trade-off parameter; In getting the weight matrix W Then, define the weight matrix W Attribute Strength α , and the attribute strength α With the weight matrix W Multiply each other to obtain the corresponding enhanced weight matrix Z=α × W ; Then the weight matrix will be strengthened Z The corresponding latent vector Z Add, and obtain the corresponding weighted potential vector Z'=Z+ Z ; The process of face reshaping using a target generative adversarial network is as follows: a new original face image to be reshaped is obtained, and according to the occlusion of the original face image, the image after face correction of the original face image is input into the corresponding generative adversarial network in the target generative adversarial network for face reshaping, that is: if the original face image is not occluded, the face corrected image corresponding to the original face image is input into the first generative adversarial network to generate a target image, and the target image is the reshaped image; if the original face image is occluded, the face corrected image corresponding to the original face image is input into the second generative adversarial network to generate a target image, and the target image is the reshaped image.

2. The face reconstruction method based on generative adversarial network according to claim 1, characterized in that: The objective function of the joint network is as follows: ； In the formula, M represents the objective function of the joint network; G 1 means correcting the face image, represents the target image generated by the concatenated vector in the joint network in the first adversarial generation network; G 2 means blocking the face image, represents the target image generated by the concatenated vector in the joint network in the second adversarial generation network; λ Indicates an adjustable parameter.

3. The face reconstruction method based on generative adversarial network according to claim 2, characterized in that: Use image editing software to partially or completely mask the face in the corrected face image according to the set requirements.

4. An electronic device, characterized in that: The invention comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected in sequence, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute a face reconstruction method based on a generative adversarial network as described in any one of claims 1 to 3.

5. A storage medium, characterized in that: The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes a face reconstruction method based on a generative adversarial network as described in any one of claims 1 to 3.

Images