A face image rotation method and device and a storage medium

Abstract

A face image rotation method, device and storage medium, the method comprises the steps of: acquiring three-dimensional face data; constructing a three-dimensional face model database according to the three-dimensional face data; training a deep convolutional neural network using the three-dimensional face data; generating a rotated two-dimensional face grayscale image according to the deep convolutional neural network; and generating a rotated two-dimensional face color texture image using style transfer rendering according to the two-dimensional face grayscale image. The face image rotation method, device and storage medium provided by the application rely on a neural network model focusing on three-dimensional face model shape parameter regression, can improve the accuracy of rotating face shape structure, and retain face deep features; the style transfer adversarial neural network is used to extract face decoupling features, which can well render the high-frequency details of the rotated face; while ensuring the accuracy of the rotated face contour positioning feature, the face texture color and high-frequency details can be accurately rendered, and the face deep features are retained.

Description

Technical Field

[0001] This invention belongs to the field of facial recognition technology, specifically relating to a facial image rotation method, device, and storage medium. Background Technology

[0002] In the field of face recognition, only roll-axis face rotation can be handled relatively well. For face images along the yaw and pitch axes, face alignment techniques for non-frontal faces are still not well addressed. Directly applying similarity transformations to rotated faces along the yaw and pitch axes results in significant deformation of the aligned face, losing some features of the original image. Existing techniques also struggle to recover occlusion information and lack high precision in depicting the detailed shape and texture of rotated faces.

[0003] Using ordinary transformation methods will significantly alter the shape of the face in the image, and because there are no constraints or references, the reliability of the transformed face image is not high.

[0004] Maintaining a good shape and texture structure after face rotation is essential for the usability of rotated images. Ordinary transformation methods significantly alter the shape of the face in an image, and due to the lack of constraints and references, the reliability of rotated face images obtained through traditional methods is low. Two-dimensional face images reconstructed from 3D faces have certain constraints on shape and structure; generative adversarial networks (GANs) have shown unexpected effectiveness in rendering high-frequency details. Directly reconstructing a 3D face from a 2D face image, rotating it, and projecting it onto a 2D plane to obtain a rotated face image guarantees a certain level of shape and texture structure. However, current 3D face reconstruction algorithms cannot achieve the required texture accuracy for high-resolution detail rendering. The texture vectors reconstructed using 3DMM models combined with neural networks generally have fewer than 100 dimensions, resulting in low texture accuracy. Consequently, the shape and texture structure of the rotated face obtained from this method also fails to meet accuracy requirements.

[0005] Current face rotation algorithms mostly use multi-angle images of the same face to rotate the face, which leads to various problems. On the other hand, multi-angle face images are not easy to obtain in practical applications. The need for face image rotation mostly exists in scenarios where a single frontal face is obtained from different angles of its profile or where the frontal face is recovered from its profile.

[0006] Although using Generative Adversarial Networks (GANs) to rotate face images can capture high-frequency details, their ability to rotate faces is limited by their dependence on multi-view paired training data and their overfitting to specific data. Summary of the Invention

[0007] In view of the above problems, the present invention provides a method, apparatus and storage medium for rotating a face image to overcome the above problems or at least partially solve the above problems.

[0008] To address the aforementioned technical problems, this invention provides a method for rotating a face image, the method comprising the following steps:

[0009] Acquire 3D face data;

[0010] A 3D face model database is constructed based on the aforementioned 3D face data;

[0011] The three-dimensional face data was used to train a deep convolutional neural network;

[0012] A rotated two-dimensional grayscale image of a face is generated based on the deep convolutional neural network;

[0013] A rotated 2D face color texture image is generated using style transfer rendering based on the 2D face grayscale image.

[0014] Preferably, the step of constructing a 3D face model database based on the 3D face data includes the following steps:

[0015] Mark facial key points on the three-dimensional face data to obtain a three-dimensional face point cloud dataset;

[0016] Perform sphere segmentation on the aforementioned 3D face point cloud dataset;

[0017] The 3D face point cloud dataset is processed using face models to obtain a 3D face model database.

[0018] Preferably, training a deep convolutional neural network using the 3D face data includes the following steps:

[0019] Obtain the corresponding frontal face image, two-dimensional face image, and rotated face image from the three-dimensional face data;

[0020] Add rotation angle labels to the two-dimensional face image;

[0021] Filter out face images whose rotation angle label exceeds a preset value;

[0022] Calculate the facial similarity between the frontal face image and the remaining two-dimensional face images;

[0023] Obtain a deep convolutional neural network;

[0024] The deep convolutional neural network is trained using the rotated face image and the corresponding frontal face image.

[0025] Preferably, generating a rotated 2D grayscale image of a face based on the deep convolutional neural network includes the following steps:

[0026] The 3D face data is input into a deep convolutional neural network to obtain a 3D face shape model;

[0027] Determine the two-dimensional plane based on the rotation requirements;

[0028] The three-dimensional face data is projected onto the two-dimensional plane to obtain an initial two-dimensional face image;

[0029] The pixel values ​​that fill in the missing pixel regions of the initial two-dimensional face image;

[0030] Optimize the pixel values ​​in the pixel-deficient region.

[0031] Preferably, the step of generating a rotated two-dimensional face color texture image using style transfer rendering based on the two-dimensional face grayscale image includes the following steps:

[0032] Obtain a style transfer adversarial generative network;

[0033] The style transfer adversarial generative network described above is used to extract facial style features;

[0034] Obtain the two-dimensional grayscale image of the human face;

[0035] The facial style features are rendered onto the two-dimensional grayscale image of the face to obtain a rotated two-dimensional facial color texture image.

[0036] Preferably, the spherical segmentation of the 3D face point cloud dataset includes the following steps:

[0037] Locate the tip of the nose in the three-dimensional face point cloud dataset;

[0038] Traverse all the facial key points and obtain the maximum Euclidean distance in three-dimensional space between the other facial key points and the tip of the nose of the face;

[0039] The face portion of the three-dimensional face point cloud dataset within the sphere is extracted using a preset multiple of the maximum Euclidean distance in the three-dimensional space as the sphere radius;

[0040] Remove unnecessary data from below the neck and the back of the head.

[0041] Preferably, the step of processing the 3D face point cloud dataset using face models to obtain a 3D face model database includes the following steps:

[0042] Set up the LSFM model building environment;

[0043] Input the 3D face point cloud dataset;

[0044] Dense mapping is performed on the aforementioned 3D face point cloud dataset;

[0045] PGA principal component analysis was performed on the aforementioned 3D face point cloud dataset;

[0046] A three-dimensional face model database is constructed based on the dense correspondence and the PGA principal component analysis.

[0047] This application also provides a face image rotation device, the device comprising:

[0048] A 3D face data acquisition module is used to acquire 3D face data;

[0049] A 3D face model database construction module is used to construct a 3D face model database based on the 3D face data;

[0050] A deep convolutional neural network training module is used to train a deep convolutional neural network using the three-dimensional face data.

[0051] A two-dimensional face grayscale image generation module is used to generate a rotated two-dimensional face grayscale image based on the deep convolutional neural network;

[0052] A rotating 2D face color texture image generation module is used to generate a rotating 2D face color texture image based on the 2D face grayscale image using style transfer rendering.

[0053] This application also provides an electronic device, the electronic device comprising:

[0054] At least one processor; and,

[0055] A memory communicatively connected to the at least one processor; wherein,

[0056] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the aforementioned face image rotation methods.

[0057] This application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform any of the aforementioned face image rotation methods.

[0058] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: The face image rotation method, device and storage medium provided in this application rely on a neural network model focused on the regression of shape parameters of three-dimensional face models, which can improve the accuracy of rotating face shape structure while preserving deep face features; it uses style transfer adversarial neural network to extract face decoupling features, and renders the high-frequency details of the rotated face well; while ensuring the accuracy of the rotated face contour positioning features, it can accurately render face texture color and high-frequency details, and preserve deep face features, so that the rotated face has high credibility. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating a face image rotation method provided in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the structure of a face image rotation device provided in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of the structure of a non-transitory computer-readable storage medium provided in an embodiment of the present invention. Detailed Implementation

[0064] The present invention will be described in detail below with reference to specific embodiments and examples, thereby making the advantages and various effects of the present invention more clearly apparent. Those skilled in the art should understand that these specific embodiments and examples are for illustrative purposes only and are not intended to limit the present invention.

[0065] Throughout this specification, unless otherwise specified, the terminology used herein should be understood as having the meaning commonly used in the art. Therefore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. In the event of any conflict, this specification shall prevail.

[0066] Unless otherwise specified, all raw materials, reagents, instruments and equipment used in this invention can be purchased from the market or prepared by existing methods.

[0067] like Figure 1 In this embodiment of the application, the present invention provides a method for rotating a face image, the method comprising the steps of:

[0068] S1: Acquire 3D face data;

[0069] In this embodiment of the application, three-dimensional face data can be acquired through an image acquisition device, and the three-dimensional face data is stored in obj format.

[0070] S2: Construct a three-dimensional face model database based on the three-dimensional face data;

[0071] In this embodiment of the application, the step of constructing a three-dimensional face model database based on the three-dimensional face data includes the following steps:

[0072] Mark facial key points on the three-dimensional face data to obtain a three-dimensional face point cloud dataset;

[0073] Perform sphere segmentation on the aforementioned 3D face point cloud dataset;

[0074] The 3D face point cloud dataset is processed using face models to obtain a 3D face model database.

[0075] In this embodiment of the application, when constructing a three-dimensional face model database based on three-dimensional face data, 68 facial key points are first marked on the three-dimensional face data to obtain a three-dimensional face point cloud dataset. Then, the three-dimensional face point cloud dataset is spherically segmented, and the three-dimensional face point cloud dataset is processed using a face model to obtain a three-dimensional face model database.

[0076] In this embodiment of the application, the step of spherically segmenting the 3D face point cloud dataset includes the following steps:

[0077] Locate the tip of the nose in the three-dimensional face point cloud dataset;

[0078] Traverse all the facial key points and obtain the maximum Euclidean distance in three-dimensional space between the other facial key points and the tip of the nose of the face;

[0079] The face portion of the three-dimensional face point cloud dataset within the sphere is extracted using a preset multiple of the maximum Euclidean distance in the three-dimensional space as the sphere radius;

[0080] Remove unnecessary data from below the neck and the back of the head.

[0081] In this embodiment of the application, when performing sphere segmentation on the three-dimensional face point cloud dataset, 68 facial key points have been marked on the three-dimensional face data. At this time, the 34th facial key point among the 68 facial key points is used to locate the tip of the nose. Then, the 68 facial key points are traversed to obtain the maximum Euclidean distance between the other facial key points and the tip of the nose in three-dimensional space. Then, the face part of the three-dimensional face point cloud within the sphere is cropped with a radius of 1.2 times the maximum Euclidean distance to remove unnecessary data below the neck and the back of the head.

[0082] In this embodiment of the application, the step of processing the 3D face point cloud dataset using face models to obtain a 3D face model database includes the following steps:

[0083] Set up the LSFM model building environment;

[0084] Input the 3D face point cloud dataset;

[0085] Dense mapping is performed on the aforementioned 3D face point cloud dataset;

[0086] PCA principal component analysis was performed on the aforementioned 3D face point cloud dataset;

[0087] A three-dimensional face model database is constructed based on the dense correspondence and the PCA principal component analysis.

[0088] In this embodiment of the application, when the three-dimensional face point cloud dataset is processed using a face model to obtain a three-dimensional face model database, the construction environment of the LSFM model is first set up, the three-dimensional face point cloud data in obj format is input, and the three-dimensional face LSFM model can be automatically constructed after dense correspondence and PCA principal component analysis.

[0089] In this embodiment, the 3DMM model is a three-dimensional face statistical model that decomposes the face into a shape model and a texture model. The shape model and texture model are obtained by superimposing shape vectors and texture vectors, respectively, as shown in the following formula:

[0090] M = (S, T)

[0091]

[0092]

[0093] Where M represents the 3D face model, S represents the shape vector, T represents the texture vector, and Si and Ti represent the shape vector basis and texture vector basis, respectively. The 3DMM model simplifies the 3D face reconstruction problem into a parameter solving problem, while LSFM is a more accurate and larger-scale 3DMM model.

[0094] S3: Train a deep convolutional neural network using the aforementioned 3D face data;

[0095] In this embodiment of the application, training a deep convolutional neural network using the 3D face data includes the following steps:

[0096] Obtain the corresponding frontal face image, two-dimensional face image, and rotated face image from the three-dimensional face data;

[0097] Add rotation angle labels to the two-dimensional face image;

[0098] Filter out face images whose rotation angle label exceeds a preset value;

[0099] Calculate the facial similarity between the frontal face image and the remaining two-dimensional face images;

[0100] Obtain a deep convolutional neural network;

[0101] The deep convolutional neural network is trained using the rotated face image and the corresponding frontal face image.

[0102] In this embodiment, when training a deep convolutional neural network using the 3D face data, the following steps are first taken: First, the corresponding frontal face image, 2D face image, and rotated face image from the 3D face data are acquired. Rotation angle labels are added to the 2D face image, specifically yaw axis rotation angle labels and pitch axis rotation angle labels. Then, face images whose rotation angle labels exceed preset values ​​(yaw axis rotation angle exceeding 80°, pitch axis exceeding 75°) are filtered out. The ArcFace algorithm is then used to calculate the face similarity between the frontal face image and the remaining 2D face images. Finally, a deep convolutional neural network is acquired, and the rotated face image and the corresponding frontal face image are used to train the deep convolutional neural network.

[0103] In this embodiment, the deep convolutional neural network is a ResNet50 network; its fully connected layers are modified to have 99 neurons, predicting the 99-dimensional shape parameters of the LSFM model; the loss function uses Landmark loss, pixel-level loss, and perceptual layer loss; the Landmark loss employs a loss function strategy, expressed as:

[0104]

[0105] Where, ω n This represents the weight of keypoint regions. Keypoints around the nose, eyes, and mouth are weighted at 10, while other areas are weighted at 1. The original face image is stripped of RGB colors, and the pixel-level loss uses an L2 loss strategy, expressed as:

[0106] Lphoto (x)=∥P i -P i (x)∥2

[0107] Where I and I′ represent the original image and the reconstructed two-dimensional image, respectively, and ||·|| represents the L2 loss; the perceptual layer loss uses a face recognition network ArcFace to extract features and calculate the distance between face feature values.

[0108] S4: Generate a rotated two-dimensional grayscale image of a face based on the deep convolutional neural network;

[0109] In this embodiment of the application, the step of generating a rotated two-dimensional grayscale image of a face based on the deep convolutional neural network includes the following steps:

[0110] The 3D face data is input into a deep convolutional neural network to obtain a 3D face shape model;

[0111] Determine the two-dimensional plane based on the rotation requirements;

[0112] The three-dimensional face data is projected onto the two-dimensional plane to obtain an initial two-dimensional face image;

[0113] The pixel values ​​that fill in the missing pixel regions of the initial two-dimensional face image;

[0114] Optimize the pixel values ​​in the pixel-deficient region.

[0115] In this embodiment of the application, when generating a rotated two-dimensional face grayscale image based on the deep convolutional neural network, the three-dimensional face data is first input into the deep convolutional neural network to obtain a three-dimensional face shape model. Then, a two-dimensional plane is determined according to the rotation requirements, and the three-dimensional face data is projected onto the two-dimensional plane to obtain an initial two-dimensional face image. Then, the pixel values ​​of the pixel missing areas of the initial two-dimensional face image are filled in, and the pixel values ​​of the pixel missing areas are optimized.

[0116] In this embodiment of the application, the three-dimensional face shape model is obtained through the following steps: inputting the three-dimensional face data to be rotated into a deep convolutional neural network, processing it according to the rotation angle, outputting the 99-dimensional shape parameters predicted by the deep convolutional neural network, and then combining the shape vector baseline of the LSFM model to obtain the three-dimensional face shape model.

[0117] In this embodiment, the initial two-dimensional face image is obtained through the following steps: First, the two-dimensional plane is determined according to the rotation requirements. For the three-dimensional model obtained by three-dimensional reconstruction of the face image, the two-dimensional face projected under zero rotation matches the original face image, that is, the projection plane is the yz plane; the normal vector perpendicular to this plane is... Rotation around the yaw axis is the same as rotation around the z-axis, and rotation around the pitch axis is the same as rotation around the y-axis. The normal vector... Rotate by the negative angle of the required rotation angle along the yaw and pitch axes to obtain... (-sinαpitch, cosαpitch·cosβyaw, cosαpitch·sin βyaw); with It is the normal vector. The coordinates are used to construct a projection plane at the intersection point; the Z-Buffer hidden surface removal algorithm is used, and then the 3D face is projected onto the 2D plane.

[0118] In this embodiment, based on the symmetry structure of the face, when a face pixel or pixel region is missing, the pixel value is first filled using symmetry, and then all pixel values ​​of the missing point or region are optimized based on the four-neighbor method. The optimization strategy proceeds from the outside in. For a single pixel, the four-neighbor method is used directly; for a pixel region, optimization is performed from the outside in using a region-shrinking approach.

[0119] S5: Generate a rotated two-dimensional face color texture image using style transfer rendering based on the two-dimensional face grayscale image.

[0120] In this embodiment of the application, the step of generating a rotated two-dimensional face color texture image using style transfer rendering based on the two-dimensional face grayscale image includes the following steps:

[0121] Obtain a style transfer adversarial generative network;

[0122] The style transfer adversarial generative network described above is used to extract facial style features;

[0123] Obtain the two-dimensional grayscale image of the human face;

[0124] The facial style features are rendered onto the two-dimensional grayscale image of the face to obtain a rotated two-dimensional facial color texture image.

[0125] In this embodiment, the style transfer adversarial generative network is StyleGan2 Encoder. Then, dlib is used to align the original face image to 1024×1024. The style transfer adversarial generative network StyleGan2 Encoder is used to extract the style features of the face. Then, the face style features (skin color, hair, wrinkle details and eye details, etc.) are rendered onto a two-dimensional face grayscale image to obtain the face rotation map of the original face image.

[0126] like Figure 2 In this embodiment of the application, a face image rotation device is also provided, the device comprising:

[0127] The 3D face data acquisition module 10 is used to acquire 3D face data;

[0128] The 3D face model database construction module 20 is used to construct a 3D face model database based on the 3D face data;

[0129] Deep convolutional neural network training module 30, used to train a deep convolutional neural network using the three-dimensional face data;

[0130] A two-dimensional face grayscale image generation module 40 is used to generate a rotated two-dimensional face grayscale image based on the deep convolutional neural network;

[0131] The rotating two-dimensional face color texture image generation module 50 is used to generate a rotating two-dimensional face color texture image based on the two-dimensional face grayscale image using style transfer rendering.

[0132] The face image rotation device provided in this application can perform the face image rotation method provided in the above steps.

[0133] The following is for reference. Figure 3 The diagram illustrates a structural schematic of an electronic device 100 suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0134] like Figure 3 As shown, the electronic device 100 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device 100. The processing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0135] Typically, the following devices can be connected to I / O interface 105: input devices 106 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 109. Communication device 109 allows electronic device 100 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 100 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0136] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 109, or installed from storage device 108, or installed from ROM 102. When the computer program is executed by processing device 101, it performs the functions defined in the methods of embodiments of this disclosure.

[0137] The following is for reference. Figure 4 The diagram illustrates a structure of a computer-readable storage medium suitable for implementing embodiments of the present disclosure, the computer-readable storage medium storing a computer program that, when executed by a processor, can implement the face image rotation method as described above.

[0138] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0139] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0140] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0141] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0142] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0144] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0145] This application provides a face image rotation method, apparatus, and storage medium that relies on a neural network model focused on the regression of shape parameters of a 3D face model. This model can improve the accuracy of rotating the face shape structure while preserving deep face features. It utilizes a style transfer adversarial neural network to extract face decoupling features, effectively rendering high-frequency details of the rotated face. While ensuring accurate positioning of the rotated face contour features, it can accurately render face texture, color, and high-frequency details, and preserve deep face features, resulting in a highly reliable rotated face.

[0146] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A face image rotation method, characterized in that, The method comprises the steps of: acquiring three-dimensional face data; constructing a three-dimensional face model database according to the three-dimensional face data; training a deep convolutional neural network using the three-dimensional face data; generating a rotated two-dimensional face grayscale image according to the deep convolutional neural network; generating a rotated two-dimensional face color texture image using style transfer rendering according to the two-dimensional face grayscale image; The step of constructing a three-dimensional face model database according to the three-dimensional face data comprises the steps of: labeling face key points on the three-dimensional face data and obtaining a three-dimensional face point cloud data set; ball cutting the three-dimensional face point cloud data set; processing the three-dimensional face point cloud data set using a face model and obtaining a three-dimensional face model database; The step of training a deep convolutional neural network using the three-dimensional face data comprises the steps of: acquiring corresponding front face images, two-dimensional face images and rotated face images in the three-dimensional face data; adding a rotation angle label to the two-dimensional face images; filtering out face images with rotation angle labels exceeding a preset value; calculating face similarity between the front face images and the remaining two-dimensional face images; acquiring a deep convolutional neural network; training the deep convolutional neural network using the rotated face images and the corresponding front face images; The step of generating a rotated two-dimensional face grayscale image according to the deep convolutional neural network comprises the steps of: inputting the three-dimensional face data into a deep convolutional neural network and obtaining a three-dimensional face shape model; determining a two-dimensional plane according to rotation requirements; projecting the three-dimensional face data onto the two-dimensional plane and obtaining an initial two-dimensional face image; filling in pixel values of pixel missing areas of the initial two-dimensional face image; optimizing the pixel values of the pixel missing areas; The step of generating a rotated two-dimensional face color texture image using style transfer rendering according to the two-dimensional face grayscale image comprises the steps of: acquiring a style transfer generative adversarial network; extracting face style features using the style transfer generative adversarial network; acquiring the two-dimensional face grayscale image; rendering the face style features onto the two-dimensional face grayscale image and obtaining a rotated two-dimensional face color texture image; The step of ball cutting the three-dimensional face point cloud data set comprises the steps of: locating a face tip position in the three-dimensional face point cloud data set; iterating through all the face key points and obtaining a three-dimensional space maximum Euclidean distance between other face key points and the face tip position; cutting a face part in a ball using a preset multiple of the three-dimensional space maximum Euclidean distance as a ball radius; removing unnecessary data below the neck and at the back of the head; The step of processing the three-dimensional face point cloud data set using a face model and obtaining a three-dimensional face model database comprises the steps of: building an LSFM model construction environment; inputting the three-dimensional face point cloud data set; performing dense correspondence on the three-dimensional face point cloud data set; performing PCA principal component analysis on the three-dimensional face point cloud data set; constructing a three-dimensional face model database according to the dense correspondence and the PCA principal component analysis.

2. A face image rotating apparatus adapted to the method of claim 1, characterized in that, The device comprises: a three-dimensional face data acquisition module for acquiring three-dimensional face data; The three-dimensional face model database construction module is configured to construct a three-dimensional face model database according to the three-dimensional face data. The deep convolutional neural network training module is configured to train a deep convolutional neural network using the three-dimensional face data. The two-dimensional face grayscale image generation module is configured to generate a rotated two-dimensional face grayscale image according to the deep convolutional neural network. The rotated two-dimensional face color texture image generation module is configured to generate a rotated two-dimensional face color texture image using style transfer rendering according to the two-dimensional face grayscale image.

3. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the face image rotation method of claim 1.

4. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the face image rotation method of claim 1.

Images