Unlock instant, AI-driven research and patent intelligence for your innovation.

Audio and video synchronization optimization method and device

A technology for audio and video synchronization and video data, applied in image communication, selective content distribution, electrical components, etc.

Pending Publication Date: 2021-04-13
TENCENT TECH (SHENZHEN) CO LTD
View PDF15 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The present invention provides a method and device for audio and video synchronization optimization, which is used to solve the problem that the existing audio and video synchronization scheme cannot solve the problem of audio and video out of synchronization caused by PTS numerical error in the source file

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Audio and video synchronization optimization method and device
  • Audio and video synchronization optimization method and device
  • Audio and video synchronization optimization method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0101] In view of the fact that when the PTS value of the audio / video in the current source file is wrong due to compression, media, etc., the synchronization scheme will inevitably go wrong. The embodiment of this application provides a method for audio and video synchronization optimization. In video synchronization optimization, the above-mentioned computer vision technology and NLP (Natural Language Processing, natural language processing) technology are applied.

[0102] Such as figure 2 As shown, the audio and video synchronization optimization method provided by the embodiment of the present application is as follows:

[0103] Step S201, performing preliminary synchronization on the audio data and video data according to the time stamps respectively corresponding to the decoded audio data and video data;

[0104] As mentioned above, a device with audio and video playback function receives audio and video source data in a compressed form, and first performs audio and v...

Embodiment 2

[0155] Such as image 3 As shown, a flow chart of a method for audio and video synchronization optimization provided by an embodiment of the present invention specifically includes the following steps:

[0156] Step S301, acquiring audio and video source data;

[0157] The audio and video source data is acquired according to the above method, which will not be described in detail here.

[0158] Step S302, separating audio and video source data to obtain audio data and video data, and storing them in corresponding audio data buffer and video data buffer respectively;

[0159] The audio and video source data is data in a compressed encapsulation format. Use the audio and video separator Demuxer to separate the audio source data and video source data in the audio and video source data in the encapsulated format, and parse the compressed audio frames from the audio source data and store them in the audio data buffer, from the video source data The compressed video frame is pars...

Embodiment 3

[0198] A method for optimizing audio-video synchronization in the present invention is described above, and a device for performing the above-mentioned audio-video synchronization optimization is described below.

[0199] see Figure 4 , an embodiment of the present invention provides an audio and video synchronization optimization device, including:

[0200] A preliminary synchronization module 401, configured to perform preliminary synchronization on the audio data and video data according to the time stamps respectively corresponding to the decoded audio data and video data;

[0201] The content recognition module 402 is used to utilize computer vision analysis technology to carry out content recognition to the video segment where the video data of the current video output point is located, and judge whether there is a sound source object;

[0202] The sound recognition module 403 is used for utilizing the PCM data verification method to carry out sound recognition to the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an audio and video synchronization optimization method and device, and the device comprises the steps: carrying out the preliminary synchronization of audio data and video data according to the timestamps corresponding to the decoded audio data and video data; performing content identification on a video clip where the video data of the current video output point is located by utilizing the computer vision analysis technology, and judging whether a sound source object exists or not; performing voice recognition on audio clips within preset time before and after the current audio data output point by using a PCM data verification method, and judging whether the audio clips are voice clips or not; matching the result of content identification with the result of voice identification, and when it is determined that the video clip and the audio clip are not synchronous, synchronizing the audio data and the video data again and then outputting the synchronized audio data and video data. According to the audio and video synchronization optimization method and device provided by the invention, the problem of audio and video asynchronization caused by PTS numerical errors in a source file, which cannot be solved by an existing audio and video synchronization scheme, is solved.

Description

technical field [0001] The invention relates to the technical field of audio and video synchronization, in particular to a method and device for optimizing audio and video synchronization. Background technique [0002] For videos with sound and images, it is necessary to deal with the synchronization of audio and video during playback. Audio and video synchronization refers to the coordination between the sound and the picture in the film, that is, when the user hears the sound corresponding to the picture when watching the video, he will not feel that the picture is ahead or delayed. Audio and video synchronization is an important content in the research of quality of service (QoS) in multimedia systems, and out of synchronization will seriously affect the viewing experience of users. [0003] The current solution to the problem of audio and video synchronization is to choose a reference clock, and when generating audio and video data streams, mark each audio / video data fr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04N21/43H04N21/44H04N21/439
CPCH04N21/4307H04N21/44008H04N21/4394
Inventor 赵登
Owner TENCENT TECH (SHENZHEN) CO LTD