Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A Video Rain and Snow Removal Method Based on Multi-scale Convolutional Sparse Coding

A convolutional sparse coding, multi-scale technology, applied in the field of multi-scale convolutional sparse coding of video rain and snow removal, can solve the problems of inability to obtain the effect of rain and snow removal, the problem of training data bias, and difficulty in obtaining

Active Publication Date: 2020-10-27
XI AN JIAOTONG UNIV
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, these labeling information are often difficult to obtain for rainy and snowy videos with a specific structure in practice, or require a lot of manpower and material resources to obtain. The types of rain and snow that appear in the data are used to remove rain and snow, but for the rain and snow videos that are not reflected in the training data, effective video removal of rain and snow effects cannot be obtained

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Video Rain and Snow Removal Method Based on Multi-scale Convolutional Sparse Coding
  • A Video Rain and Snow Removal Method Based on Multi-scale Convolutional Sparse Coding
  • A Video Rain and Snow Removal Method Based on Multi-scale Convolutional Sparse Coding

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0107] The real raining video data shown in Figure 2(a) is used as the experimental object of the present invention, and the video is a real raining video taken in a static scene without moving objects. The size of the video data is 720×480×120, the number of scales is 3, the corresponding size is 11×11, 9×9, 5×5, the maximum number of iteration steps is 5, and the background rank is 2.

[0108] see figure 1 , the process is as follows:

[0109] Step S1: Obtain rainy video X∈R h×w×T , where h, w represent the length and width of the video, T represents the number of video frames, initialize model variables and parameters; where X can be decomposed into:

[0110] X=B+F+R

[0111] where B,F,R∈R h×w×T represent the background, foreground and rain layers of the video, respectively.

[0112] Step S2: Construct a multi-scale convolutional sparse coding rain stripe detection model according to the structural characteristics of rain in the video;

[0113]

[0114]

[0115]...

Embodiment 2

[0176] The snow video data shown in Figure 4 (a)) is used as the experimental object of the present invention, and the video is a real snow video shot in a static scene. The video data size is 360×270×100, the number of scales is 3, the corresponding size is 11×11, 9×9, 5×5, the maximum number of iteration steps is 5, and the background rank is 2.

[0177] see figure 1 , the process is as follows:

[0178] Step S1: Obtain snowy video X∈R h×w×T , where h, w represent the length and width of the video, T represents the number of video frames, initialize model variables and parameters; where X can be decomposed into:

[0179] X=B+F+R

[0180] where B,F,R∈R h×w×T represent the background, foreground and snow layers of the video, respectively.

[0181] Step S2: Construct a multi-scale convolutional sparse coding snow block detection model according to the structural characteristics of the snow in the video;

[0182]

[0183]

[0184] Step S3: Construct a moving object d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a video rain removing and snow removing method based on multi-scale convolution sparse coding. Under the assumption of a low-rank background, the rain and snow components and the moving prospect in the video are estimated at the same time. Firstly, video data containing rain and snow noise is acquired, and a model is initialized; a generation model of the rain and snow graph is built according to the characteristics of the rain and snow and the video prospect; according to the structural characteristic that the rain and snow are imaged in the video, the moving rain andsnow are repeatable and multi-scale rain strip local blocks on the image, a multi-scale convolution sparse coding model is established related to the rain and snow; a moving object detection model isestablished according to the characteristics of the video foreground sparsity; the model is integrated into rain and snow under the maximum likelihood estimation framework model; a rain-snow video anda rain-removing snow model are applied, so that rain and snow videos and other statistical variables are obtained, and rain and snow videos are output. The invention aims to establish a high-qualityvideo rain-removing snow model based on the rain and snow generation principle and the rain and snow noise structure characteristics, so that the snow removing and snow removing technology can be widely applied in more complicated practical scenes.

Description

technical field [0001] The invention relates to a video image processing technology for outdoor shooting images, in particular to a multi-scale convolution sparse coding video rain and snow removal method. Background technique [0002] With the deepening of the national "Sky Eye" project, outdoor surveillance video is playing an increasingly powerful security function. However, because the shooting of the outdoor shooting system is often affected by bad weather (such as rain, snow, fog), the details of the captured video or image are destroyed, and the background part is blocked by highlighted raindrops, rain strips, and snow blocks, making it impossible to use The captured images are further processed, such as pedestrian re-identification, object detection, image segmentation and recognition, etc. Therefore, removing rain and snow from video images has become a technology that has emerged in the field of computer vision in recent years. On the premise of preserving the de...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06T5/00
CPCG06T5/002G06T2207/10016G06T2207/20081
Inventor 孟德宇李明晗赵谦谢琦
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com