Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Virtual object image synthesis method and device, electronic equipment and storage medium

A virtual object and image technology, applied in electrical components, character and pattern recognition, image data processing, etc., can solve problems such as loss, failure to realize image fusion, etc.

Pending Publication Date: 2020-12-29
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF0 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the above-mentioned mouth shape simulation fusion method only affines the coordinate relationship of the key points of the mouth image on the virtual object image. On the one hand, the same coordinate relationship, the mouth of the virtual object image and the original mouth shape image The embodied mouth shape may not correspond to the same expression voice. On the other hand, only the coordinate relationship of the key points of the affine mouth image of the virtual object completely loses the information of the original mouth image, and the fusion of the image is not realized.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Virtual object image synthesis method and device, electronic equipment and storage medium
  • Virtual object image synthesis method and device, electronic equipment and storage medium
  • Virtual object image synthesis method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

preparation example Construction

[0036] in particular, figure 2 is a flowchart of a virtual object image synthesis method according to an embodiment of the present application, such as figure 2 As shown, the method includes:

[0037] Step 101, acquiring the image of the virtual object including the face of the virtual object, and multiple frames of original mouth and face images corresponding to the voice segment.

[0038] It can be understood that the image of the virtual object including the face of the virtual object in the embodiment of the present application may be the base plate image in the fusion scene, and the image of the virtual object may include not only the face of the virtual object but also the face of the virtual object. Body parts, etc., wherein, the image of the virtual object can be an image of a person, an animal, or even a landscape, which is not limited here.

[0039]In addition, the multi-frame original mouth and face images corresponding to the voice segment can be obtained by co...

example 1

[0047] In this example, as image 3 As shown, the step of processing the first facial key points according to the preset algorithm to generate the first 3D facial position and posture information corresponding to the face of the virtual object includes:

[0048] Step 301: Perform coordinate transformation processing on the first facial key points according to preset transformation parameters to generate a first 3D facial mesh model.

[0049] In the actual execution process, the conversion of the facial information in the world coordinate system to the image of the virtual object is related to the internal and external parameters of the camera. Therefore, the transformation parameters in this embodiment can be understood as the internal and external parameters of the camera. , or the algorithm parameters that reflect the internal and external parameters of the camera, where the acquisition of the internal and external parameters of the camera can be obtained by Zhang Zhengyou's...

example 2

[0054] In this example, the structured light is projected on the face of the virtual object to obtain the modulated structured light image of the virtual object's face, and then, the structured light image is demodulated to obtain the first facial key point corresponding to the virtual object's face Depth information, based on the depth information and the coordinate information of the first facial key point can be constructed to obtain the first 3D facial mesh model corresponding to the first facial key point, thereby, through the model analysis of the first 3D facial mesh model Corresponding posture information can be obtained, for example, a 3D coordinate system (including three directions of x, y, and z) is pre-constructed, and the first 3D facial mesh model is placed in the coordinate system to collect posture information (read in The three displacements in the three directions of x, y, and z are used as attitude information).

[0055] Step 104: Process each second facial...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a virtual object image synthesis method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence, in particular to a computervision and deep learning technology. The implementation scheme is as follows: extracting a first face key point of the face of a virtual object and a second face key point of each frame of original mouth-shaped face image; processing the virtual first face key point to generate a first 3D face position and posture information corresponding to the face of the virtual object; processing each secondface key point of each frame of original mouth-shaped face image to generate second 3D face shape vertex information corresponding to each frame of original mouth-shaped face image; generating a target mouth shape face image corresponding to each frame of original mouth shape face image; and synthesizing a virtual object speaking image segment corresponding to the voice segment according to the virtual object image and each frame of target mouth-shaped face image. Therefore, the original mouth-shaped face image is adjusted to be merged under the posture and the position of the virtual object image, and the naturalness of the merged image is improved.

Description

technical field [0001] The present application relates to the field of artificial intelligence, specifically to the field of computer vision and deep learning, and in particular to a virtual object image synthesis method, device, electronic equipment and storage medium. Background technique [0002] Virtual objects are combined with digital images and artificial intelligence to generate humanoid video images. One of the more widely used fields is the voice broadcast of virtual objects, even if the virtual objects reflect the mouth shape of speaking in the form of normal people. [0003] In the related technology, the coordinate relationship of the key points of the mouth shape in the mouth shape image is extracted, and the coordinates of the key points of the mouth area in the virtual object image corresponding to the virtual object are adjusted according to the coordinate relationship, so as to realize the mouth shape simulation speech reflect. [0004] However, the above-...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T19/20G06T15/04
CPCG06T19/20G06T15/04G06T2200/04G06T13/40G06T17/00G06T2219/2021A63F13/655G06T7/70G10L15/04G06T5/50H04N21/4307G06T2207/30201G06T7/73G06T17/20G06T2207/20221G06V40/171
Inventor 郭汉奇胡天舒马明明洪智滨
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com