A video audio-video matching method, related device and storage medium

A matching method, audio-visual technology, applied in the direction of carrier indicating device, speech analysis, speech synthesis, etc., can solve the problems of image sequence flaws, poor authenticity of video content, etc.

Active Publication Date: 2020-08-21
TENCENT TECH (SHENZHEN) CO LTD
View PDF23 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the video content generated by GAN only includes image sequences and no speech content, and is limited by the lack of training data and the instability of training methods, the generated image sequences often have obvious defects, which leads to the generation of video content is less authentic

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A video audio-video matching method, related device and storage medium
  • A video audio-video matching method, related device and storage medium
  • A video audio-video matching method, related device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0124] The embodiment of the present application provides a video audio-picture matching method, a related device, and a storage medium, which are used to locate the position of the moving segment in the image sequence by using the start and end marker positions in the process of synthesizing the video, and from the The activity segment is matched with the voice segment, so that the synthesized video segment is more in line with the natural law of the character when speaking, and has better authenticity. In addition, the movement direction of the start and stop signs can be used to match the voice segment and the activity segment in an orderly manner, which can improve Consistency and continuity of fragment synthesis.

[0125] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The application discloses a video audio-video matching method, a related device and a storage medium, which are used in the field of artificial intelligence. The method of this application includes: receiving the voice sequence sent by the client; obtaining the voice segment to be matched from the voice sequence; obtaining the initial position of the start-stop mark and the moving direction of the start-stop mark from the image sequence; The movement direction and the voice segment to be matched are obtained to obtain the activity segment to be matched; the voice segment to be matched and the activity segment to be matched are synthesized to obtain a video segment. In the process of synthesizing video, this application uses the position of the start and stop marks to locate the position of the moving segment in the image sequence, and matches the moving segment with action with the voice segment, so that the synthesized video segment is more in line with the natural law of the character when speaking , with better authenticity.

Description

technical field [0001] The present application relates to the field of artificial intelligence, and in particular to a video audio-video matching method, a related device and a storage medium. Background technique [0002] With the continuous development of science and technology, computer vision technology has been in great demand in many fields such as digital entertainment, medical health and security monitoring. Synthesizing photorealistic visual content is not only of great commercial value, but also has been desired by the industry. [0003] At present, a method of generating video through Generative Adversarial Networks (GAN) is proposed, that is, using a neural network to map known image textures to an unseen scene, and perform mapping on the mapped image. Repair and completion to generate the desired video content. [0004] However, the video content generated by GAN only includes image sequences and no speech content, and is limited by the lack of training data a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04N21/234H04N21/233H04N21/439H04N21/44G10L25/57
CPCH04N21/234H04N21/233H04N21/439H04N21/44G10L25/57G10L21/055G11B27/031G11B27/10G10L13/02G11B27/34
Inventor 凌永根黄浩智沈力
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap