Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Portrait audio video synchronous calibration device and method

A technology of voice and video, calibration device, applied in image communication, instrument, character and pattern recognition, etc., can solve problems such as inability to recognize, out of synchronization of voice information and video information, inability to judge motion without sound, etc., to improve computing power. performance, the effect of reducing the amount of information storage

Active Publication Date: 2016-11-02
JIANGSU UNIV
View PDF7 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

During the recording process, due to hardware or network problems, the voice information and video information will be out of sync.
Traditional audio and video synchronization calibration generally uses manual playback of audio and video files frame by frame. When an error is found, the method of manual calibration requires a lot of work; some synchronization methods that add time stamps can only recognize voice information with time stamps and Video information cannot identify voice information and video information that have not been added with a time stamp; there are also some methods that match the characteristics of the motion amplitude in the video frame with the characteristics of the voice information, which requires movement to produce changes in the sound information, and cannot be judged. movement that produces sound

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Portrait audio video synchronous calibration device and method
  • Portrait audio video synchronous calibration device and method
  • Portrait audio video synchronous calibration device and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0065] Embodiment 1: to the synchronous voice and video detection process

[0066] Step S1, read the audio and video header file information, obtain the total time length of the audio and video 72, the unit is second, a certain moment of the audio and video is t, 1≤t≤72;

[0067] Step S2, set the dynamic lips array P[k], 1≤k≤72, set the initial value of all elements in the array P to 0, set the vocal array S[f], 1≤f≤72, set the array S The initial value of all elements in is set to 0;

[0068] Step S3, sequentially extracting the picture frames at time t of the video file, image 3 It is the binary image of the image frame extracted at the 32nd second of the video file, Figure 6 It is the binary image of the picture frame extracted in the 31st second of the video file, using face recognition technology to recognize the i human face area M in the picture frame at a certain moment t,i , 1≤i≤I, I=1, Figure 4 From image 3 A face region M extracted from 32,1 , Figure 7 F...

Embodiment 2

[0077] Embodiment 2: To asynchronous voice and video detection and calibration process

[0078] Step S1, read the audio and video header file information, obtain the total length of the audio and video time 58, the unit is second, a certain moment of the audio and video is t, 1≤t≤58;

[0079] Step S2, set the dynamic lips array P[k], 1≤k≤58, set the initial value of all elements in the array P to 0, set the vocal array S[f], 1≤f≤58, set the array S The initial value of all elements in is set to 0;

[0080] Step S3, sequentially extracting the picture frames at time t of the video file, Figure 11 is the binary image of the image frame extracted from the 19th video file, Figure 14 It is the binary image of the picture frame extracted from the 18th second of the video file, and the i face area M in the picture frame at a certain moment is recognized by face recognition technology t,i , 1≤i≤I, I=3, Figure 12 From Figure 11 The three face regions M extracted from 19,1 ,M ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a portrait audio video synchronous calibration device and a portrait audio video synchronous calibration method. According to the portrait audio video synchronous calibration device and the portrait audio video synchronous calibration method, existing proven face recognition technology, dynamic lip recognition technology, voice extraction technology and the like are used, and informatization means and hardware implementation are designed, and therefore, and a portrait audio video synchronous calibration function is realized; low-time complexity left shift, right shift and XOR computation are adopted, so that computational performance can be improved; and timestamp information is not required to be added to audio and video files, and the volume of information storage can be reduced. The portrait audio video synchronous calibration device and method of the invention are applied to the synchronous detection of portrait audio videos and calibration of asynchronous audio videos.

Description

technical field [0001] The invention belongs to the technical field of multimedia information processing, and in particular relates to a voice and video synchronous calibration device and method for portraits. Background technique [0002] With the popularity and development of multimedia and the Internet, portrait voice and video applications are used in various fields, such as talk entertainment programs, network anchor programs, and large-scale open online courses. The voice information and video information used in portrait audio and video are generally recorded separately by different hardware, and then comprehensively processed by a computer to synthesize a voice and video file that can be played directly. During the recording process, due to hardware or network problems, the voice information and video information will be out of sync. Traditional audio and video synchronization calibration generally uses manual playback of audio and video files frame by frame. When a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04N21/43H04N21/8547G06K9/00
CPCH04N21/4307H04N21/8547G06V40/171
Inventor 陈潇君苟建平詹天明成科扬陈小波詹永照毛启容柯佳汪满容
Owner JIANGSU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com