A method for removing compression noise in video call video based on voice cues
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A technology for video compression and video calling, which is used in TV, image communication, color TV, etc., and can solve the problem of ignoring the role of voice.
Active Publication Date: 2020-12-08
SHANDONG UNIV
View PDF10 Cites 0 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
With the development of video calls and webcasting, people have higher and higher requirements for video quality, while the existing deep neural network-based video enhancement restoration technology ignores the role of voice
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0081] A method for removing compression noise from video calls based on voice cues, such as figure 1 shown, including the following steps:
[0082] A. Building datasets and data preprocessing
[0083] 1) Collect speech videos containing people's heads, and construct a video call video data set;
[0084] 2) the speech video of the people's head that step 1) collects is original video compresses, subframes successively, carries out feature extraction to the voice signal in described original video, constructs training set and test set;
[0085] B. Establish a video compression noise removal model based on voice cues
[0086] Such as Image 6 As shown, the video compression noise removal model based on speech cues includes a speech feature encoder model, an image feature encoder, a generator network model, an image authenticity discriminator, and a video continuity discriminator; the speech feature encoder model is used to encode speech features; the image feature encoder is...
Embodiment 2
[0098] According to a method for removing compression noise in a video call based on voice clues described in Embodiment 1, the difference is that:
[0099] Step A, building dataset and data preprocessing, video call video dataset is the original video Including selecting and downloading a large number of speech videos containing human heads from the Internet, setting a total of N segments, namely V i Represents the i-th video, including the following steps:
[0100] a. Read N segments of video, extract the voice signal, and standardize the voice signal into a monophonic voice file of the same frequency;
[0101] B, carry out MFCC feature extraction to the monophonic speech file after the processing that step a obtains, each sampling interval of each monophonic speech file is extracted to m dimension MFCC feature, and each monophonic speech file is correspondingly extracted to a MFCC feature matrix A with n columns and m rows, n refers to the number of sampling interval...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention relates to a method for removing compression noise of video call videos based on voice clues. The method comprises the following steps: A, constructing a data set and preprocessing data;B, establishing a video compression noise removal model based on a voice clue: constructing a voice feature encoder model, a generative video compression noise removal model with the voice clue, an image authenticity discriminator and a video continuity discriminator; constructing an overall loss function to perform subsequent model optimization; C, training a video compression noise removal model based on a voice clue; and D, testing the denoising effect of the video compression noise removal model based on the voice clue, inputting the low-bit-rate and low-quality video call video and the corresponding voice signal into the model according to the trained denoising model, and outputting the high-quality video without the compression noise. According to the invention, the voice signal isused as an important clue for video call video de-compression noise, and a better video recovery effect is obtained.
Description
technical field [0001] The invention relates to a method for removing compression noise in video calls based on voice clues, and belongs to the technical fields of video recovery and video enhancement. Background technique [0002] Video compression noise refers to the blurring effect, ringing effect, color block effect and other noises that affect the user's perception experience due to the lossy compression of the original video by data compression technology. Currently common data compression methods include JPEG, WebP, and HEVC-MSP, etc. These methods use imprecise approximate representations to encode data to achieve the purpose of saving transmission bandwidth and space storage. In order to improve video quality and ensure user experience when compression technology is used, researchers have conducted a lot of research work on the removal of compression noise. However, there is no work on the removal of special video compression noise such as video calls, and the rest...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more