Image reflection elimination method based on panoramic camera

A panoramic camera and image technology, applied in the field of image processing, can solve problems such as failure to eliminate reflections, difficulty in data acquisition, and increase in relative motion amplitude, and achieve the effects of reducing ill-posedness, expanding the scope of application, and reducing difficulty.

Active Publication Date: 2022-01-11
PEKING UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, both methods have their own disadvantages: 1) It is an ill-posed problem to use a single color image for reflection elimination. Confuse the background layer with the reflection layer, resulting in the failure of reflection elimination; 2) Using multiple images for reflection elimination can achieve better reflection elimination effect than the single-image method, but it needs to take multiple images, and the data acquisition process is cumbersome. It has high requirements for shooting equipment and photographers
The shooting process is complicated, data acquisition is difficult, and the requirements for photographers and shooting equipment are very high
[0030] 3. Existing technology 3 uses the image sequence with reflection acquired by an ordinary camera to perform reflection elimination, and the camera needs to be in a stable moving state when acquiring the image sequence. When the camera is not stable enough or moves too fast, the relative motion of the background layer and the reflection layer The amplitude will increase, resulting in a significant decrease in the effect of reflection cancellation
Therefore, this technology has high requirements for photographers and shooting equipment, and cannot be a stable and reliable reflection cancellation solution

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image reflection elimination method based on panoramic camera
  • Image reflection elimination method based on panoramic camera
  • Image reflection elimination method based on panoramic camera

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach 1

[0061] Specific implementation mode 1. Combination Figure 4 to Figure 7 Describe this embodiment, an image reflection elimination method based on panoramic photography, this embodiment mainly consists of four steps: data preprocessing, rough reflection alignment, fine reflection alignment, and background image restoration, in which fine reflection alignment and background image restoration are performed by deep neural network The correspondingly designed modules in the network to achieve:

[0062] 1. Data preprocessing: For the input panoramic image, a mask (mask) of the area affected by reflection interference and a mask of the corresponding reflection scene area are given by the user, and the equirectangular projection and the conventional perspective projection are used. Coordinate transformation relationship, reproject the selected reflection area and reflection scene area into images under conventional perspective projection, and define it as a mixed image M and a reflec...

specific Embodiment approach 2

[0094] Specific embodiment two, combine Figure 6 This embodiment will be described. In this embodiment, a panoramic camera is used to capture images, and the image reflection elimination method based on panoramic photography described in the first specific embodiment is used to obtain a background image after reflection elimination.

[0095] First, build a shooting system and shoot: use a panoramic camera (such as Ricoh Theta series and Insta360ONE series) and a tripod to form a shooting system. After the construction is completed, you can shoot panoramic images with glass reflections in front of the glass (the ratio of width to height is 2: 1), the resolution varies with the camera;

[0096] Then data processing and reflection elimination: input the captured panoramic image with glass reflection into the system proposed in this patent to obtain the reflection elimination result. First, the user needs to mark the mask of the mixed image and the reflection scene, and the syst...

specific Embodiment approach 3

[0097] Specific embodiment three, combine Figure 7 This embodiment will be described. In this embodiment, an ordinary camera is used to capture images, and the image reflection elimination method based on panoramic photography described in the first specific embodiment is used to obtain a background image after reflection elimination.

[0098] First, set up the shooting system and shoot: use a common camera (SLR camera, mirrorless camera or mobile phone), shoot in front of the glass facing the background scene and the reflection scene respectively, and get a mixed image and a reflection scene image, the resolution varies with the Varies with different cameras;

[0099] Then, reflection elimination: input the captured mixed image and reflection scene image into the system proposed in this patent to obtain the reflection elimination result (compared with the panoramic image, the step of marking the mixed image and reflection scene image area by the user is omitted). A roughly ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The image reflection elimination method based on panoramic photography involves the field of image processing technology, and solves the corresponding problems existing in the reflection elimination method used in the prior art. It consists of four steps: data preprocessing, rough reflection alignment, fine reflection alignment, and background image recovery. , the method proposed in the present invention uses panoramic photography technology to introduce the content information of the reflection scene through the panoramic image, which greatly reduces the ill-posedness of reflection elimination, and makes the effect of reflection elimination greatly surpass the method of only using a single mixed image, avoiding the confusion due to Issues with degraded reflection removal results caused by background scenes and reflective scene content. A reflection alignment method is proposed to convert the reflection scene in the panoramic image into the corresponding glass reflection image, and the designed background restoration network restores a high-quality background image guided by the glass reflection image. Using a single panoramic image with glass reflection interference areas greatly reduces the difficulty of data acquisition.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to an image reflection elimination method based on panoramic photography. Background technique [0002] With the development of computer technology, computer computing power is gradually strengthened, machine learning and deep learning technology are advancing rapidly, and computer vision-related technologies are gradually applied to various scenarios, such as face detection of mobile phone cameras, retouching and beautifying pictures, and taking pictures at night. Pedestrian detection and road recognition in human driving, face recognition for mobile payment and station identity detection, or simultaneous positioning and mapping tasks for robots, etc. With the advent of the era of big data and intelligence, more and more application scenarios need the support of computational vision technology. Massive video and image data need to be processed urgently, which highlights th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06T5/50G06T3/40G06N3/04G06N3/08
CPCG06T5/50G06T3/4038G06N3/084G06T2207/10016G06T2207/20221G06T2200/32G06N3/045
Inventor 施柏鑫洪雨辰郑乾赵泠然蒋旭东葛治中
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap