Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A method for removing compression noise in video call video based on voice cues

A technology for video compression and video calling, which is used in TV, image communication, color TV, etc., and can solve the problem of ignoring the role of voice.

Active Publication Date: 2020-12-08
SHANDONG UNIV
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

With the development of video calls and webcasting, people have higher and higher requirements for video quality, while the existing deep neural network-based video enhancement restoration technology ignores the role of voice

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method for removing compression noise in video call video based on voice cues
  • A method for removing compression noise in video call video based on voice cues
  • A method for removing compression noise in video call video based on voice cues

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0081] A method for removing compression noise from video calls based on voice cues, such as figure 1 shown, including the following steps:

[0082] A. Building datasets and data preprocessing

[0083] 1) Collect speech videos containing people's heads, and construct a video call video data set;

[0084] 2) the speech video of the people's head that step 1) collects is original video compresses, subframes successively, carries out feature extraction to the voice signal in described original video, constructs training set and test set;

[0085] B. Establish a video compression noise removal model based on voice cues

[0086] Such as Image 6 As shown, the video compression noise removal model based on speech cues includes a speech feature encoder model, an image feature encoder, a generator network model, an image authenticity discriminator, and a video continuity discriminator; the speech feature encoder model is used to encode speech features; the image feature encoder is...

Embodiment 2

[0098] According to a method for removing compression noise in a video call based on voice clues described in Embodiment 1, the difference is that:

[0099] Step A, building dataset and data preprocessing, video call video dataset is the original video Including selecting and downloading a large number of speech videos containing human heads from the Internet, setting a total of N segments, namely V i Represents the i-th video, including the following steps:

[0100] a. Read N segments of video, extract the voice signal, and standardize the voice signal into a monophonic voice file of the same frequency;

[0101] B, carry out MFCC feature extraction to the monophonic speech file after the processing that step a obtains, each sampling interval of each monophonic speech file is extracted to m dimension MFCC feature, and each monophonic speech file is correspondingly extracted to a MFCC feature matrix A with n columns and m rows, n refers to the number of sampling interval...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method for removing compression noise of video call videos based on voice clues. The method comprises the following steps: A, constructing a data set and preprocessing data;B, establishing a video compression noise removal model based on a voice clue: constructing a voice feature encoder model, a generative video compression noise removal model with the voice clue, an image authenticity discriminator and a video continuity discriminator; constructing an overall loss function to perform subsequent model optimization; C, training a video compression noise removal model based on a voice clue; and D, testing the denoising effect of the video compression noise removal model based on the voice clue, inputting the low-bit-rate and low-quality video call video and the corresponding voice signal into the model according to the trained denoising model, and outputting the high-quality video without the compression noise. According to the invention, the voice signal isused as an important clue for video call video de-compression noise, and a better video recovery effect is obtained.

Description

technical field [0001] The invention relates to a method for removing compression noise in video calls based on voice clues, and belongs to the technical fields of video recovery and video enhancement. Background technique [0002] Video compression noise refers to the blurring effect, ringing effect, color block effect and other noises that affect the user's perception experience due to the lossy compression of the original video by data compression technology. Currently common data compression methods include JPEG, WebP, and HEVC-MSP, etc. These methods use imprecise approximate representations to encode data to achieve the purpose of saving transmission bandwidth and space storage. In order to improve video quality and ensure user experience when compression technology is used, researchers have conducted a lot of research work on the removal of compression noise. However, there is no work on the removal of special video compression noise such as video calls, and the rest...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04N7/14H04N21/4788H04N5/21H04N21/44
CPCH04N5/21H04N7/141H04N21/44H04N21/4788
Inventor 贲晛烨翟鑫亮李玉军魏文辉王丹凤任家畅
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com