64 results about "Face synthesis" patented technology

The invention relates to a rotary face expression learning method based on a generative adversarial network. The method comprises the following main contents: a non-coupling expression learning framework (DR-GAN) based on a generative adversarial network, improvement on face images with any attitude through a conversion model, and improvement on face synthesis of target attitudes through expression of interpolations. The method includes the following processes: providing an attitude code to a decoder, increasing attitude estimation constraint in a discriminator, separating attitude change features by feature expression learnt by DR-GAN learning in an explicit manner, taking one or more face images of a person as input, generating a unified identity feature expression, and generating any number of synthesized images of the person with different attitudes. The invention brings forward a non-coupling expression learning framework based on a generative adversarial network, which is used for rotary faces and face recognition and makes further contribution to new designs in the modeling field and to innovation solutions in the lie detection field.

The invention provides a fully-automatic face seamless synthesis-based video synthesis method. With the fully-automatic face seamless synthesis-based video synthesis method adopted, insufficient real-time property in high-definition video processing of a face detection algorithm in the prior art can be solved. The fully-automatic face seamless synthesis-based video synthesis method of the invention comprises the following steps that: a video communication application provided by an intelligent television terminal is utilized to perform video connection; an image or video file which is locally arranged or arranged in a cloud server is adopted as a background (BG) to be synthesized; face detection is respectively performed on data foreground (FG) and background (BG) data which are acquired by a camera through using the face detection algorithm, and geometric transformation coefficients are calculated through face internal key point positioning and facial contour lines or a face minimum bonding rectangle frame; and accurate registration from the foreground (FG) to the background (BG), and face region data synthesis can be accomplished. With the fully-automatic face seamless synthesis-based video synthesis method of the invention adopted, the face image of a user can be conveniently synthesized into any existing images or videos in the process of video communication, and therefore, a sense of science and technology and interestingness can be added in the video communication, and fully-automatic seamless face synthesis of non-specific people can be realized.

The invention discloses a real-time face optimal selection method based on a video sequence. The method comprises the following steps: a face image is acquired; face clarity, face size and opening degree of human eyes act as three indexes for face quality evaluation so that a face integrated evaluation score ImageScore is calculated; the ImageScore of the first frame of the face image is assigned to an initial face optimal image MaxScore and the face image is stored; the MaxScore after assignment is compared with the ImageScore of other frames of the face image, and the face image corresponding to the higher ImageScore is stored and the ImageScore is assigned to the MaxScore again; and the MaxScore value is cyclically updated and the stored optimal face image is updated until the image sequence with the face cannot be acquired, and the cycle is ended and the image corresponding to the stored MaxScore is the optimal face image. The face image with higher quality can be screened for identification via comparison of the integrated evaluation indexes of face quality in the video sequence under the situation of not knowing the face sequence image in advance; besides, the calculation method is simple with real-time performance.

On a synthesis task of a human face, a multilevel sparse expression three-time conversion virtual generation neural network TTGAN is constructed based on an adversarial generation network CycleGAN architecture. The TTGAN proposes and joins a multi-level sparse representation model and a three-time conversion consistency constraint, and the TTGAN is a result under the synergistic effect of a plurality of generative adversarial networks for the target face synthesis of a face image pair. Wherein the multi-level sparse representation model is used for constraining features extracted by differentfeature extraction layers of a generated network in an input picture, including identity information related to a target image; The three times of conversion consistency constraint utilizes three different samples which contain network state information and are generated by one time of circulation of the model, so that the two generative adversarial networks of the whole model are guided to cooperate with each other. The multi-level sparse representation and the three-time conversion consistency constraint provided by the TTGAN further increase the image generation capability of the CycleGAN,so that the synthesized face image can obtain a better result in the aspects of keeping face identity information and showing more reality.

The invention discloses a semantic-based audio-driven digital human generation method and system. The generation method comprises the following steps of obtaining a target audio and a first human faceimage sequence; performing feature extraction on the target audio to obtain corresponding audio features; inputting the audio features into a pre-trained semantic conversion network, and performing semantic conversion on the audio features by the semantic conversion network to obtain a corresponding semantic motion sequence which comprises a plurality of mouth semantic graphs; and acquiring to-be-rendered face images with the same number as the mouth semantic graphs based on a first face image sequence, shielding mouth areas of the to-be-rendered face images, performing face synthesis based on the mouth semantic graphs and the to-be-rendered face images, and generating a synthesized face sequence. According to the invention, conversion between audio and facial semantics is realized through the semantic conversion network, and accurate expression of mouth shapes is realized by utilizing the facial semantics.

The invention discloses a multi-pose facial expression recognition method based on a generative adversarial network. The multi-pose facial expression recognition method based on a generative adversarial network comprises: adding a front face synthesis module to an expression recognition system under the multi-face posture in the expression recognition process, inputting a face detected by the system and a synthesized front face into a recognition network at the same time, to improve the recognition performance under large-posture deflection of the face, and therefore expression recognition under various face deflection postures is achieved. The beneficial effects of the multi-face posture expression recognition system of front face synthesis module constructed based on proposed generativeadversarial network mainly are that: 1, the front face of the original image can be synthesized by the aid of the front face synthesis module based on the generative adversarial network through the input human face at any angle, front face information is provided for an expression recognition system, and correct recognition of expression information during large-posture deflection of the human face is guaranteed;

An image processing system is disclosed that allows a user to participate in a given content selection or to substitute any of the actors or characters in the content selection. A user can modify an image by replacing an image of an actor with an image of the corresponding user (or a selected third party). Various parameters associated with the actor to be replaced are estimated for each frame. A static model is obtained of the user (or the selected third party). A face synthesis technique modifies the user model according to the estimated parameters associated with the selected actor. A video integration stage superimposes the modified user model over the actor in the original image sequence to produce an output video sequence containing the user (or selected third party) in the position of the original actor.

The invention provides an image conversion model training method and device, a heterogeneous face recognition method and device and equipment, and the method comprises the steps: obtaining a to-be-recognized face sketch, and cutting the to-be-recognized face sketch to obtain a face region sketch; inputting the face region sketch into a pre-trained image conversion model for processing to generatea second face synthesis image; performing feature extraction on the second face synthesis image to obtain a feature vector of the second face synthesis image; and matching the feature vector of the second face synthesis image with feature vectors of a plurality of real face images stored in a database to obtain a face recognition result. Therefore, the face area is cut from the to-be-recognized face sketch to reduce the interference of the background area on face recognition, and meanwhile, the preset image conversion model is adopted to perform image conversion to convert heterogeneous face recognition into homogeneous face recognition, so that the accuracy of heterogeneous face recognition is improved.

The invention discloses a multi-view face synthesis method based on tensor resolution and Delaunay triangulation, and mainly solves the problem of difficulty in face image synthesis under continuous changing views in the prior art. The method comprises the following steps of: extracting outline information of a face image by a characteristic point labeled method; separating out a view coefficientmatrix of human face data in a training set by a tensor resolution method; fitting sample bands of the view coefficient matrix; building characteristic points of a new view by adopting a tensor resolution formula; synthesizing the face image of the new view by adopting the Delaunay triangulation and linear affine transformation according to the known face image. The method has the advantages of authentic face synthesis result, wide synthesis view ranges and low operation complexity and can be applied to the field of computer vision or face synthesis under different views in a multimedia technology.

The invention provides a 3D instant messaging system comprising clients and a server. A face synthesis unit and a voice synthesis unit are disposed inside each client. The face synthesis unit comprises a face feature extraction device, a model nesting device and a texture mapping device. The face feature extraction device is used for extracting face features from 2D face photos. The model nesting device is used for projecting 3D face mesh models onto the 2D face photos according to the extracted face features, so that texture coordinates of the 3D face mesh models can be obtained. The texture mapping device is used for mapping the 2D face photos back to 3D face meshes so as to form 3D faces. The voice synthesis unit is used for generating voice flows and 3D face animation according to the 3D faces and input text, and outputting the voice flows and the 3D face animation to the server. The server is used for achieving information interaction between the clients. According to the invention, 3D technology is introduced into the messaging system, so that users can design customized 3D face animation and chat on the internet with the customized 3D face animation, and the practical operation can be more interesting and vivid.

The invention discloses a face transfer method capable of maintaining expression information based on CNN. The method achieves the face conversion effect of saving the feature information by combiningthe face recognition network and the expression recognition network, and may transfer the face of a picture A to the face of another picture B, and the expression information of the picture B and therest of the non-face information are still maintained during the conversion process. The invention solves two core technical problems that: the problem that the synthesized part is not in harmony with the original part in the process of face synthesis, and the problem of missing face information after the synthesis part and the original part are synthesized, including the lost of identification information and the expression information. By implementing the embodiments of the invention, the demand for image processing in people's lives may be increased, so that the face processing may be applied in various ways. At the same time, the "photographing unsuitable people" may generate more own images by means of image synthesis.

The invention relates to a face recognition model construction method for face recognition, and the method comprises the steps: obtaining a plurality of pieces of sample image data, carrying out the feature extraction of the sample image data through a feature extraction model, and obtaining race face features corresponding to a plurality of race identifications; determining race face feature setscorresponding to the race identifiers according to the race face features, and training by using the race face feature sets to obtain an initial face generation model; verifying the initial face generation model, and obtaining a required face generation model after the verification is passed; synthesizing race face synthesis images corresponding to the race identifiers by utilizing a face generation model; extracting face features of the race face synthesis image and adding the face features into a race face feature set; and training and verifying the initial face recognition model by using the race face feature set to obtain a required face recognition model. By adopting the method, the face recognition model with relatively high race face recognition accuracy can be effectively generated, so that the face recognition accuracy is effectively improved.

The invention provides a cross-age face recognition method. According to the method, a cross-age face recognition system composed of two modules (a maximum entropy feature description module and an aging perception de-noising automatic coding module) is obtained by training a large amount of face images including four age groups, thereby realizing recognition on any two different age face images. The maximum entropy feature description module allocates codes including maximum information amount by using maximum entropy splitting of a decision tree, the aging perception de-noising automatic coding module reconstructs a feature descriptor of any age group into feature descriptors of four different age groups, the descriptors are integrated to obtain a face integrated feature vector for eliminating aging influencing, and face recognition is finally realized by calculating the cosine distance of integrated feature vectors of different faces. The method can well reduce the information loss problem of some traditional descriptors, eliminates the influence of the aging factor in cross-age face recognition, and has good performance in the cross-age face recognition problem.

The present invention discloses techniques for producing a synthesized facial model synchronized with voice. According to one embodiment, synchronizing colorful human or human-like facial images with voice is carried out as follows: determining feature points in a plurality of image templates about a face, wherein the feature points are largely concentrated below eyelids of the face, providing a colorful reference image reflecting a partial face image, dividing the reference image into a mesh including small areas according to the feature points on the image templates, storing chromaticity data of respective pixels on selected positions on the small areas in the reference image, coloring each of the templates with reference to the chromaticity data, and processing the image templates to obtain a synthesized image.

The invention provides a front face synthesis method and system based on a generative adversarial network, and the method comprises the steps: detecting and segmenting a face part from an input image,and carrying out the face alignment, so as to obtain a to-be-synthesized face image; estimating the head posture according to the face key points, and dividing the face data set into a front face setand a non-front face set according to the degree of freedom of head rotation; utilizing a pre-training model of a face recognition deep neural network to extract identity features of an input face image to perform training of a supervision network; and synthesizing a corresponding front face image based on the generative adversarial network according to the input side face image. Through face symmetry constraint and identity feature constraint, the synthesized front face is more natural, and the identity features of the front face are better maintained.

According to the implementation of the invention, a scheme for face synthesis is provided. In the scheme, a first image about the face of a first user and a second image about the face of a second user are acquired, a first feature characterizing the identity of the first user is extracted from the first image, a second feature characterizing a plurality of attributes of the second image is extracted from the second image, wherein the plurality of attributes do not include the identity of the second user, then, a third image is generated with respect to the face of the first user based on thefirst feature and the second feature and represents a plurality of attributes of the identity of the first user and the second image. According to the face synthesis scheme disclosed by the invention,the face image of any identity can be subjected to identity preservation-based image synthesis regardless of whether the face image of the person with the identity exists in the training data set ornot. In addition, when the model used for face synthesis is trained, the scheme does not need to label any other attributes except the identity of the person.

The invention discloses a robust automatic face fusion method. The method relates to the technical field of image synthesis, and comprises the following steps: carrying out occlusion processing on a face image A and a face image B to obtain a four-channel image A and a four-channel image B, with the four-channel image A comprising identity features in a synthetic image, and the four-channel imageB comprising attribute features in the synthetic image; encoding the four-channel image A and the four-channel image B to obtain an encoding feature A and an encoding feature B; and combining the coding feature A and the coding feature B through a generative adversarial network, and outputting a face synthesis image. According to the method, a characteristic channel of a shielding mask is added, so that the synthesized characteristic has more effective information, and the method is more robust to a complex scene in practice; occlusion information is enhanced through feature reconstruction, amore complex face fusion scene can be processed, and the applicability is wider; the image segmentation is used for generating a feature mask and fusing the feature mask into original information, andthe boundary of image segmentation is expanded.

The invention discloses a virtual face recognition method based on a synthetic image and deep learning. An image library is generated by using the image synthesis technology, then the generated imagelibrary acts as a training data set to establish a deep learning model, and the successfully trained model is applied to recognize the inputted face image. According to the method, intelligent synthesis of the characteristic face image and intelligent recognition of the actual face image can be realized so that face synthesis and recognition can be effectively and accurately completed.

The invention provides a diversified face image synthesis method and system. The method comprises the following steps: acquiring a source face picture, a target face picture and attribute tag information; according to the source face picture, the target face picture and the face synthesis network model, obtaining a realistic face picture with a source face expression, a target face identity feature and a specified attribute, wherein the face synthesis network model comprises a face feature point generator and a geometry-attribute perception generator; the face feature point generator is used for extracting feature points of a source face and a target face as face geometric feature information, extracting expression information from the face geometric feature information, and migrating the expression information of any source face to the target face in a potential space; and the geometry-attribute perception generator is used for correspondingly extracting identity features and specified attribute information from the target face and the label respectively, and generating a realistic face picture with a source face expression, the target face identity features and specified attributes in combination with the expression information.

The invention discloses a face synthesis image detection method and device, and the method comprises the steps: inputting a to-be-detected image into a trained network model, enabling a face detectionnetwork in the network model to obtain an image containing a face frame based on the to-be-detected image, and outputting the image to an authenticity discrimination network in the network model; andenabling the authenticity discrimination network to discriminate whether the to-be-detected image is a face synthesis image based on the image containing the face frame. For to-be-detected images obtained by tampering by using different face changing technologies, accurate detection can be realized through the network model comprising the face detection network and the authenticity discriminationnetwork. The method is good in universality, the human face detection network can accurately detect the human face in the to-be-detected image, the authenticity discrimination network can discriminate the authenticity of the to-be-detected image only based on the human face features based on the image containing the human face frame, interference of the background of the to-be-detected image is avoided, and therefore the discrimination result obtained through the method is high in accuracy.

The invention discloses a dual-channel depression angle face fusion correction GAN network and a face fusion correction method, and the GAN network reconstructs a clear front face through employing the global structure of a low-resolution front face and the local texture of a high-resolution depression angle face, and improves the precision of a face recognition system. The established GAN networkcomprises a super-resolution reconstruction network, an attitude correction network, a head attitude estimation module, a face registration module, a face integration module and other main function modules. The method comprises the following steps: firstly, improving a low-resolution front face to the same resolution as a high-resolution depression angle face through a super-resolution reconstruction network; estimating a head overlooking angle; completing overlooking attitude correction of a high-resolution face through an attitude correction network; realizing pixel-level alignment of the high-resolution face and the high-resolution face by using an optical flow registration method; and finally converting the estimated head overlooking angle into a fusion weight to perform angle-adaptive face synthesis. According to the method, the clear front face can be accurately reconstructed, and a new thought is provided for monitoring video face recognition.

The embodiment of the invention discloses a face image generation method and device, a server and a storage medium. The method comprises the steps of obtaining a face image of a target person and an expression label corresponding to the face image; performing face detection on the face image to obtain a standard face image of the face image; performing expression synthesis according to the standard face image and the expression label by utilizing an expression generation model to obtain a first synthesized face image; performing face synthesis according to the standard face image and the firstsynthetic face image to obtain a second synthetic face image; and generating a face image including the second synthetic face image. By adopting the method and the device, the generated expression can be more stable, and the editing requirements of the user on the expression, particularly micro-expression, can be met. The invention further relates to the blockchain technology, the index information of the face image including the second synthetic face image can be written into the block chain, and meanwhile, the invention further relates to an image processing technology in the field of artificial intelligence.

The invention relates to a face age synthesis method and system, and relates to the technical field of face synthesis, and the method comprises the steps of carrying out the preprocessing of a test image, inputting the test image into a face age synthesis model, and obtaining face images of different age groups, wherein the training of the face age synthesis model comprises the step of optimizingparameters of a generator network by utilizing a loss function of the generator network according to an attention mask and a synthesized face image, taking the synthesized face image and the corresponding original input as negative samples; taking randomly selected face images in the randomly generated target age labels during synthesis as positive samples and inputting same into the discriminatornetwork to obtain the authenticity probabilities of the corresponding face images, and optimizing parameters of the discriminator network through a loss function of the discriminator network according to the probabilities; and when the number of iterations reaches the maximum number of iterations, obtaining a trained face age synthesis model. According to the invention, the visual quality of faceage synthesis can be improved.

The invention relates to the field of software image synthesis, and provides a video generation method and device, equipment and a storage medium. The method comprises the following steps: decomposing each piece of audio data into phoneme data through an automatic speech recognition system, and calculating a posterior probability of each piece of phoneme data to obtain a phoneme posterior probability; extracting a face expression parameter of each frame of image data in the corresponding video data through a 3D face reconstruction technology to obtain an expression feature vector; generating a target expression model from the expression feature vectors and the phoneme posterior probabilities of the video data corresponding to the multiple pieces of audio data through a recurrent neural network; obtaining a to-be-replaced target video; extracting a face three-dimensional reconstruction model of each frame of image data in the corresponding video data through a 3D face reconstruction technology to obtain virtual image data; and inputting the target video to be replaced and the virtual image data into the generative adversarial neural network to obtain a target generation model. And the face synthesis speed is improved.

The invention discloses a virtual face synthesis method, a system, a device and a storage medium, and the method comprises the following steps: obtaining a video sequence and each frame of picture according to video data after the video data is obtained; recognizing faces in the pictures in sequence, and obtaining position information and contour information of the faces; combining the position information, the contour information and a preset face 3D model to carry out face synthesis; and combining the pictures subjected to face synthesis according to a video sequence to generate video data subjected to face synthesis. According to the method, the human face in the video is recognized, and the recognized human face is replaced by combining the preset human face 3D model, so that the humanface in the video is replaced by the own human face, the high requirement of a user for changing the human face in the video is met, more fun is indirectly brought to the user, and the method and thedevice can be widely applied to a video data processing technology.

The present invention provides face synthesis using generative adversarial networks. The training a generative adversarial network (GAN) for use in facial recognition comprises providing an input image of a particular face into a facial recognition system to obtain a faceprint; obtaining, based on the input faceprint and a noise value, a set of output images from a GAN generator; obtaining feedback from a GAN discriminator, wherein obtaining feedback comprises inputting each output image into the GAN discriminator and determining a set of likelihood values indicative of whether each output image comprises a facial image; determining, based on each output image, a modified noise value; inputting each output image into a second facial recognition network to determine a set of modified faceprints; defining, based on each modified noise value and modified faceprint, feedback for the GAN generator, wherein the feedback comprises a first value and a second value; and modifying control parameters of the GAN generator.

The invention discloses a large-scale three-dimensional face synthesis system for sample similarity suppression. The system comprises three-dimensional scanning equipment which is used for obtaining athree-dimensional face model and point cloud data thereof; the first processing module is used for preprocessing the acquired three-dimensional face model and the point cloud data thereof; the secondprocessing module is used for synthesizing the human face three-dimensional model to obtain a synthesized three-dimensional human face model; and the third processing module is used for carrying outsimilarity detection on the synthesized three-dimensional face model and the sampled three-dimensional face model set so as to output the synthesized three-dimensional face model meeting the similarity requirement. According to the system, on the basis of carrying out three-dimensional face model synthesis; the problem of data privacy required to be considered by a three-dimensional face model synthesis technology is fully considered, and the similarity between a synthesis model and a real acquired face model is suppressed by designing a similarity test model, so that an output three-dimensional face synthesis result can meet the special requirement of face data privacy.

The invention discloses a face image synthesis method and device, an electronic device and a storage medium. The method comprises the steps of obtaining a face portrait image and a face photo image; obtaining a face portrait training image block set and a face photo training image block set according to the face portrait image and the face photo image; training a deep neural network model according to the face portrait training image block set and the face photo training image block set, and obtaining a deep feature representation model according to the deep neural network model; and obtaininga first combination weight set according to the face photo test image, the probability graph model and the depth feature representation model, and obtaining a face synthesis image of the face photo test image according to the first combination weight set. According to the method, the depth feature representation and the probability graph model are adopted for modeling, and compared with an existing method, uncontrollable situations such as complex backgrounds, illumination and shielding can be better coped with, so that the image quality and the detail structure quality of the synthesized image are improved.

The invention relates to a smiling face synthesis method based on a segment-type sparse component analysis model. The method comprises steps: firstly, the segment-type sparse component analysis model for face representation is derived; then, a rule for reconstruction and projection is given based on the model; a projection coefficient is obtained by using the projection rule, and the reconstruction rule is used for reconstructing an inputted face; the above projection and reconstruction process is repeated on the face after reconstruction; and finally, multiple face images after reconstruction are outputted as a smile synthesis process for the inputted face. The method has the significant effects that the synthesized face is basically reasonable and smooth, and the synthesized face has a sense of reality.