Video audio and picture matching method, related device and storage medium

A matching method, audio-visual technology, applied in carrier indicating devices, speech analysis, speech synthesis, etc., can solve problems such as poor authenticity of video content, image sequence defects, etc.

Active Publication Date: 2020-06-02
TENCENT TECH (SHENZHEN) CO LTD
View PDF23 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the video content generated by GAN only includes image sequences and no speech content, and is limited by the lack of training data and the

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video audio and picture matching method, related device and storage medium
  • Video audio and picture matching method, related device and storage medium
  • Video audio and picture matching method, related device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0124] The embodiment of the present application provides a video audio-picture matching method, a related device, and a storage medium, which are used to locate the position of the moving segment in the image sequence by using the start and end marker positions in the process of synthesizing the video, and from the The activity segment is matched with the voice segment, so that the synthesized video segment is more in line with the natural law of the character when speaking, and has better authenticity. In addition, the movement direction of the start and stop signs can be used to match the voice segment and the activity segment in an orderly manner, which can improve Consistency and continuity of fragment synthesis.

[0125] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a video audio and picture matching method, a related device and a storage medium, and is applied to the field of artificial intelligence. The method comprises the following steps: receiving a voice sequence sent by a client; obtaining a to-be-matched voice clip from the voice sequence; obtaining the initial position of a start-stop identifier and the moving direction of thestart-stop identifier from an image sequence; obtaining a to-be-matched movable clip according to the initial position of the start-stop identifier, the moving direction of the start-stop identifierand the to-be-matched voice clip; and performing synthesis processing on the to-be-matched voice clip and the to-be-matched movable clip to obtain a video clip. In the process of synthesizing the video, the positions of the movable clips in the image sequence can be positioned by utilizing the start-stop identification positions, and the movable clips with actions are matched with the voice clips,so that the synthesized video clips more accord with the natural law of a person during speaking and have better authenticity.

Description

technical field [0001] The present application relates to the field of artificial intelligence, and in particular to a video audio-video matching method, a related device and a storage medium. Background technique [0002] With the continuous development of science and technology, computer vision technology has been in great demand in many fields such as digital entertainment, medical health and security monitoring. Synthesizing photorealistic visual content is not only of great commercial value, but also has been desired by the industry. [0003] At present, a method of generating video through Generative Adversarial Networks (GAN) is proposed, that is, using a neural network to map known image textures to an unseen scene, and perform mapping on the mapped image. Repair and completion to generate the desired video content. [0004] However, the video content generated by GAN only includes image sequences and no speech content, and is limited by the lack of training data a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N21/234H04N21/233H04N21/439H04N21/44G10L25/57
CPCH04N21/234H04N21/233H04N21/439H04N21/44G10L25/57G10L21/055G11B27/031G11B27/10G10L13/02G11B27/34
Inventor 凌永根黄浩智沈力
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap