Polarization Imaging System with Super-Resolution Fusion

Abstract

The present invention discloses a polarization imaging system for generating super-resolution images. The polarization imaging system is composed of a plurality of functional elements for generating super-resolution images, including a camera device, a polarization analysis module, a weight module, a fusion module, and an output module. The lens of the imaging system captures light from an object detected and acted on by a polarization sensor. The polarization sensor generates a plurality of low-resolution images, calculates the degree of linear polarization and the polarization angle of the light rays, and generates a guiding mask based on the degree of linear polarization and the polarization angle. The system uses an algorithm to combine one or more low-resolution images to produce a final super-resolution image as output.

Description

Technical Field

[0001] The present invention relates to an imaging system. More specifically, the present invention relates to a polarization imaging system for generating super-resolution images. Background Art

[0002] In digital capture, traditional image sensors and the human visual system can only be used to detect the intensity and narrow-band wavelength of light waves and express them as signals for generating images. However, these signals completely lose polarization information and limit the perception ability of the imaging system. Light waves can be characterized by their amplitude, wavelength, phase, and polarization. On the contrary, the goal of polarization sensing is to simultaneously measure the amplitude and polarization of the incident light field. Various visual systems in the animal kingdom, including insects, fish, crustaceans, and mantis shrimp, are sensitive to polarization information. The combination of polarization and intensity information enables various tasks, such as navigation, communication, and positioning, thereby enhancing the visual contrast of an object against its background.

[0003] Polarization imaging technology has a wide range of applications in fields such as earth remote sensing, astronomical observation, target recognition, medical diagnosis, and three-dimensional reconstruction. Natural light does not exhibit polarization characteristics. Natural light is uniformly distributed in all directions and has the same amplitude. For partially polarized light, the amplitudes of light waves in different polarization directions are different, and the amplitudes in two mutually perpendicular directions have the maximum amplitude and the minimum amplitude. Therefore, some substances can be distinguished by the degree of polarization (such as the degree of polarization of natural substances and camouflage substances).

[0004] The principle of polarization imaging is: when natural light (unpolarized light) interacts with a substance (such as reflection, refraction, scattering, and absorption), the emitted light usually becomes partially polarized light or linearly polarized light. According to Kirchhoff's law and Fresnel's formula, the degree of polarization of the emitted light is directly related to the inherent properties of the material interface and the reflection angle (or refraction angle). The inherent characteristics of the material interface include composition, structure, roughness, and inclusions.

[0005] U.S. Patent 9,426,362 B2 assigned to MEMS Drive Inc. describes an image resolution enhancement technique. Applying the techniques disclosed herein significantly improves the resolution of images captured by a camera having a limited image sensor size and a limited pixel density. However, the described techniques do not utilize a polarization sensor within the camera that helps measure physical characteristics that are undetectable using conventional imaging.

[0006] Another Chinese patent 105551009A transferred to Harbin Institute of Technology overcomes the limitations of the above-mentioned prior art to some extent. The prior art relates to an image fusion method based on continuous terahertz confocal scanning polarization imaging. The invention enhances the image quality by adopting image fusion involving image interpolation. The invention overcomes the limitations of the previously described US patent by introducing polarization imaging. The described technology provides many benefits: not only detecting geometric shapes and surfaces, but also measuring physical properties that cannot be detected using conventional imaging. In addition, the described technology can be used to enhance the contrast of objects that are difficult to distinguish otherwise. However, a major drawback of the described invention is that for the process of interpolation and obtaining good image quality, a large number of points need to be acquired, which increases the imaging time. When considering a smaller number of points to reduce the imaging time, the quality of the obtained images is poor.

[0007] To overcome the drawbacks of the above-mentioned prior art, another Chinese patent 105139339A transferred to the Chinese People's Liberation Army Academy of Officers discloses a polarization image super-resolution reconstruction method based on multi-level filtering and sample matching. The invention uses polarization imaging and generates images with clear contour structures, indistinct edges, and rich detail information. However, the described invention lacks image fusion technology, resulting in poor retention of the spectral content of the images.

[0008] Therefore, to overcome the drawbacks of the above-mentioned prior art, the present invention discloses a polarization imaging system for generating super-resolution images. The polarization imaging system of the present invention adopts the technologies of polarization imaging and image fusion to generate super-resolution images.

[0009] It is now clear that a variety of methods and systems suitable for various purposes have been developed in the prior art. In addition, even though these inventions are suitable for the specific purposes they are directed to, these inventions are not suitable for the purposes described so far in the present invention. Summary of the Invention

[0010] Light waves can be characterized by their amplitude, wavelength, phase, and polarization. The polarization of light is caused by the scattering of light in the air reflected or refracted from an object. This information is orthogonal to the other two fundamental properties of light: intensity and color captured by a conventional image sensor.

[0011] Polarization is a fundamental property of light and describes the direction of oscillation of the electric field of light. Most light sources (such as the sun) emit unpolarized light. Unpolarized light has vibrations in randomly oriented directions perpendicular to the propagation direction. For light to be polarized, the randomly oriented vibrations are eliminated or converted into linear, circular, or elliptical electromagnetic waves.

[0012] Polarization imaging can provide rich information about the world around us that cannot be seen by conventional image sensors and the human visual system. Measuring both the amplitude and polarization of the incident light field simultaneously is the goal of polarization sensing. Polarization image sensors can extend the polarization perception ability of imaging systems. In the present invention, a polarization image fusion method is proposed to generate enhanced super-resolution images using four low-resolution intensity images with different polarization directions.

[0013] Due to the high-reflection imaging conditions in many industrial applications, visual inspection can be challenging. Polarization cameras can help reveal hidden material properties and provide visual clarity beyond that of standard cameras. Polarization cameras can be used to filter out unwanted reflections or glare, and to enhance contrast by coloring light at the polarization angle. Normal color and monochrome sensors detect the intensity and wavelength of incident light, while special polarization sensors used inside polarization cameras can detect and filter the polarization angle from reflective, refractive, or scattering surfaces.

[0014] The main objective of the present invention is to provide a polarization super-resolution system, which includes a camera device, a polarization analysis module, a weight module, a fusion module, and an output module that generates a super-resolution image. The camera device includes a main lens system and a polarization image sensor. The polarization sensor used is a CMOS sensor characterized by a four-way polarizer.

[0015] The light beam from the object focused by the main lens system is recorded by the polarization image sensor. The amount of light entering the camera is limited by the aperture of the camera's main lens. The focusing lens in the system can be formed by a flexible transparent elastic member or a liquid lens, and the refractive power of the focusing lens can be changed by changing the interface shape of the focusing lens to adjust the focal length to the object. The imaging system can include a single-focus lens, a zoom lens, a shift lens, etc., or can be interchangeable with other capture optical systems having various characteristics (f-number (aperture value), focal length, etc.).

[0016] Another objective of the present invention is to provide a polarization image sensor similar to a color filter array composed of array pixels with different polarization directions. A close-up image of the polarization sensor shows a sensor with polarization pixels A, B, C, and D having four polarization directions. The array pixels provide light modulation per pixel. The disclosed imaging system receives the raw data and extracts four low-resolution images with different polarization directions.

[0017] Another objective of the imaging system is to calculate the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP) based on the sub-images of the polarization analysis module. The degree of polarization (DOP) is a quantity used to describe the portion of an electromagnetic wave that is polarized. Light is reflected from the surface of an object as polarized light and unpolarized light. The degree of linear polarization of the reflected light depends on the surface conditions (material, color, roughness, etc.) and the angle of reflection. The polarization angle represents the angle between the polarization plane and a reference plane.

[0018] A completely polarized wave has a DOP of 100%, while an unpolarized wave has a DOP of 0%. Using the information on the polarization direction, the distortion of a plane and the direction of the distortion can be identified, and reflections can be eliminated. The weight module creates a weight mask based on a guided map calculated from the DOLP / AOLP.

[0019] Another objective of the present invention is to provide an enhanced super-resolution image with detailed information by means of a fusion module. The fusion module combines one or more weighted sub-images obtained to generate a super-resolution image. Then, the fusion module uses an algorithm to generate a super-resolution image by fusing the imaging of one or more weighted sub-images, where the fused imaging is based on surface reflection reduction or surface normal mapping.

[0020] In conjunction with the accompanying drawings, other objectives and aspects of the present invention will become more apparent from the following detailed description, which illustrates the features according to embodiments of the present invention by way of example.

[0021] To achieve the above and related objectives, the present invention may be implemented in the form shown in the accompanying drawings. However, it should be noted that the drawings are merely illustrative, and changes may be made to the specific structures shown and described within the scope of the appended claims.

[0022] Although the present invention has been described above according to various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functions described in one or more individual embodiments are not limited to being applied to the specific embodiments described, but may be applied individually or in various combinations to one or more other embodiments of the present invention, whether or not these embodiments are described, and whether or not these features are presented as part of the described embodiments. Therefore, the breadth and scope of the present invention should not be limited by any of the above exemplary embodiments.

[0023] Broad words and phrases such as "one or more", "at least", "but not limited to", or other similar phrases in some cases should not be understood to mean that a narrower situation is expected or required in cases where such broad phrases may not exist. Description of the Drawings

[0024] In conjunction with the accompanying drawings, the objects and features of the present invention will become more apparent from the following description and the appended claims. It should be understood that these drawings only depict typical embodiments of the present invention and should not be considered as limiting the scope of the present invention. The present invention will be described and explained with additional features and details by using the drawings, wherein:

[0025] Figure 1 Shows the architecture of a polarization imaging system;

[0026] Figure 2 Shows the block diagram of a display polarization super-resolution image system;

[0027] Figure 3 Shows the polarization analysis module within the polarization super-resolution image system;

[0028] Figure 4 Shows four sub-images generated by the polarization analysis module;

[0029] Figure 5 Shows the weight module within the polarization super-resolution image system;

[0030] Figure 6 Shows the weight module according to the present invention;

[0031] Figure 7 Shows the fusion and output module within the polarization super-resolution image system;

[0032] Figure 8 Shows the fusion and output module according to the present invention;

[0033] Figure 9 Shows a method for generating a super-resolution image; and

[0034] Figure 10 Shows a method for generating a super-resolution image by polarization. Detailed Description of the Invention

[0035] Polarization cameras can be used to filter out unwanted reflections or glares and enhance contrast by coloring light of the polarization angle. Normal color and monochrome sensors detect the intensity and wavelength of incident light, while special polarization sensors used within polarization cameras can detect and filter out the polarization angle from reflective, refractive, or scattering surfaces. Polarization imaging can provide rich information about the world around us that is not visible to conventional image sensors and the human visual system. Simultaneously measuring the amplitude and polarization of the incident light field is the goal of polarization detection.

[0036] Figure 1Shows the architecture of a polarization imaging system. Architecture 100 includes a camera 104, a lens 106 within the camera 104, and a polarization sensor 108 within the camera 104. A light beam from an object 102 focused by the main lens 106 system is recorded by the polarization image sensor 108. The focusing lens 106 in the system can be formed by a flexible transparent elastic member or a liquid lens, and the refractive power of the focusing lens 106 can be changed by changing the interface shape of the focusing lens 106 so as to adjust the focal length to the object. The imaging system 100 can include a single focal length lens, a zoom lens, a shift lens, etc., or can be interchangeable with other capture optical systems having various characteristics (f-number (aperture value), focal length, etc.).

[0037] The polarization image sensor 108 built in the polarization camera 104 simultaneously captures an intensity image and a partially polarized image of the object. For this purpose, in the polarization image sensor 108, a finely patterned polarizer array having a plurality of different polarization principal axes is arranged on the image sensor 108 array. The semiconductor-based image sensor 108 of the present invention includes a patterned polarization converter, a linear polarizer, and a sensor array. The sensor array can be a CCD (charge-coupled image sensor) or a CMOS (complementary metal oxide semiconductor (CMOS)) sensor array. The sensor pixels are arranged in an orthogonal pixel matrix on the sensor array. Advantageously, the sensor pixels form a rectangular array, especially a square array.

[0038] The present invention provides a polarization image sensor 108 capable of simultaneously obtaining an intensity image and polarization information. The amount of light entering the camera 104 is limited by the aperture of the main lens 106. The polarization image sensor 108 is similar to a color filter array composed of arrayed pixels having different polarization directions. Figure 1 Shows a close-up image of polarization pixels. A, B, C, and D represent four pixels providing light modulation per pixel with four polarization directions. The polarizer array is composed of four polarizers (90°, 45°, 135°, and 0°) placed at different angles on each pixel. Each four-pixel block constitutes a computing unit.

[0039] The relationship between polarizers in different directions allows the calculation of the degree and direction (angle) of polarization. As Figure 1 shown, the polarization image is remapped onto the photodetector 108. For the polarization sensor, adjacent pixels represent signals with different polarization directions. The imaging system 100 receives the raw data and extracts four low-resolution images with different polarization directions.

[0040] Figure 2Shows the architecture of a polarization imaging system. The figure shows an exemplary polarization super-resolution system 200, which includes a camera device 202, a polarization analysis module 204, a weight module 206, a fusion module 208, and an output module 210. The camera device 202 includes a main lens system and a polarization image sensor. The system 200 receives raw data and extracts four low-resolution images with different polarization directions from the raw data. In the preprocessing of the sub-images, a smoothing filter is used to remove bad pixels and reduce the noise level.

[0041] The architecture also includes a polarization analysis module 204, which includes an image extractor that receives one or more of the above sub-images and extracts one or more processed sub-images from one or more of the above sub-images. The polarization analysis module 204 also includes a processor that calculates the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP) for each of the one or more processed sub-images.

[0042] The degree of polarization (DOP) is a quantity used to describe the portion of an electromagnetic wave that is polarized. Light is reflected from the surface of an object as polarized light and unpolarized light. The DOP of the reflected light depends on the surface conditions (material, color, roughness, etc.) and the angle of reflection. The polarization angle represents the angle between the polarization plane and a reference plane. A completely polarized wave has a DOP of 100%, while an unpolarized wave has a DOP of 0%. Using the information on the polarization direction, the degree and direction of distortion of a plane can be identified, and reflections can be eliminated.

[0043] The weight module 206 creates a weight mask based on a guidance map calculated from the DOLP / AOLP. The weight module includes a processor that generates a guidance map for each of the one or more processed sub-images based on the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP). The weight module 206 also includes a weight mask module that calculates a weight mask for each of the one or more processed sub-images based on the guidance map. In addition, the weight mask module assigns a weight mask to the one or more processed sub-images to generate one or more weighted sub-images.

[0044] The fusion module 208 combines one or more weighted sub-images via an algorithm. The obtained super-resolution image is generated by an output module 210 connected to the fusion module 208. The fusion module 208 uses an algorithm to combine one or more weighted sub-images through fusion imaging, where the fusion imaging is based on surface reflection reduction or surface normal mapping.

[0045] Figure 3 Shows a polarization analysis module within a polarization super-resolution image system. The polarization analysis module 300 includes an image extractor 302 that receives one or more low-resolution sub-images (obtained from a polarization sensor) and extracts one or more processed sub-images from the one or more low-resolution sub-images. In addition, the polarization analysis module includes a processor 304 that calculates the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP) for each of the one or more processed sub-images. The processor 304 may be present in a computing device such as a personal computer, an embedded computer, a single-board computer (such as a Raspberry Pi, etc.), a portable computing device (such as a tablet), a controller, or any other computing device or device system capable of performing the functions described herein.

[0046] The polarization analysis module 300 generates a DOLP / AOLP map based on the sub-images. The degree of polarization (DOP) is a quantity used to describe the portion of an electromagnetic wave that is polarized, and the polarization angle represents the angle between the polarization plane and a reference plane. Based on the polarization map, a guidance mask is generated.

[0047] Figure 4 Shows four sub-images generated by the polarization analysis module. The four extracted sub-images 400 are shown as 400a (sub-image A), 400b (sub-image B), 400c (sub-image C), 400d (sub-image D). The polarization state of a light beam is represented by a Stokes vector. These vectors can be characterized by the light intensities at 0°, 45°, 90°, and 135°. Two parameters (the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP)) are estimated based on the Stokes vector. These parameters are helpful for various fusion imaging, such as surface reflection reduction, surface normal mapping. As shown in the equations given below, the polarization state of a light beam is represented by a Stokes vector. These vectors can be characterized by the light intensities at 0°, 45°, 90°, and 135°. Two parameters (the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP)) are estimated based on the Stokes vector. These parameters are helpful for various fusion imaging, such as surface reflection reduction, surface normal mapping.

[0048]

[0049]

[0050] The polarization analysis module generates a DOLP / AOLP map based on the sub-images. Based on the polarization map, a guidance mask is generated. The guidance mask represents an image segmentation with different degrees of polarization.

[0051] Figure 5Shows a weight module within a polarization super-resolution image system. The weight module 500 creates a weight mask based on a guidance map calculated from DOLP / AOLP. The weight module 500 includes a processor 502 that generates a guidance map for each of one or more processed sub-images based on the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP).

[0052] The weight module further includes a weight mask module 504 that calculates a weight mask for each of one or more processed sub-images based on the guidance map. Additionally, the weight mask module 504 assigns the weight mask to one or more processed sub-images to generate one or more weighted sub-images.

[0053] Figure 6 Shows a weight module according to the present invention. The architecture describes polarization raw data 602 from which four sub-images 604 are generated. As Figure 6 shown, the four sub-images 604a, 604b, 604c, and 604d are combined to generate a degree of polarization image 606. For each of the four sub-images 604a, 604b, 604c, 604d, the weight module generates weight maps 608a, 608b, 608c, and 608d respectively. Weight map A 608a, weight map B 608b, weight map C 608c, and weight map D 608d are generated based on a guidance map 610 calculated from DOLP / AOLP. Additionally, a combined weight map 612 is formed based on the guidance map 610.

[0054] When the difference in adjacent pixel intensities is small (DOLP≈Lthre), high spatial resolution image reconstruction is achieved. With a high degree of polarization (DOLP≈Hthre), reflections can be eliminated by selecting different weights for image fusion. When Lthre < DOLP < Hthre, the information from the four sub-images (604a - 604d) can be used to estimate the super-resolution image. As described above, according to the polarization raw image, a group of low-resolution sub-images with different polarization states {sub-images: A, B, C, D} is given. To obtain a high-resolution image, we can adopt the patch redundancy of natural images at the same scale for the same sub-images and adopt cross-patches of different sub-images with the help of a guidance mask. This method can be converted into the following algorithm. For each block in sub-image i (where i can be A, B, C, and D), its self-similarity patch can be found in the same image i, and their similarity can be calculated using the following equation.

[0055]

[0056]

[0057] We can find the similar image patches {SP1, SP2, SP3, …} by calculating the square of the L2 distance between the template image T n×n and the same image patch I in the search window m×m where m > n. Then, groups of similar image patches {{PA, SPA1, SPA2, SPA3, …}{PB, SPB1, SPB2, SPB3, …}{PC, SPC1, SPC2, SPC3, …}{PD, SPD1, SPD2, SPD3, …},} are obtained at the same positions in the sub-images A, B, C, D. To obtain the precise transformation of sub-pixels, a distance map for each image patch can also be obtained by minimizing the L2 distance of the sub-window between the template and the query position (Equation 4).

[0058] Figure 7 Shows the fusion and output modules within the polarization super-resolution image system. The fusion and output module 700 of the polarization imaging system includes a fusion module 702 and an output module 706. The fusion module combines one or more weighted sub-images to produce a super-resolution image. The fusion module 702 combines one or more weighted sub-images by performing fusion imaging using an algorithm 704, where the fusion imaging is based on surface reflection reduction or surface normal mapping.

[0059] As Figure 7 shown, the fusion module 702 is connected to the output module 706 that produces the final super-resolution image 708. An enhanced super-resolution image with detailed information can be obtained by the fusion module 702. As Figure 7 shown, an image 712 with arbitrary polarization can also be achieved through a data cube 710.

[0060] The purpose of super-resolution fusion 800 is to provide a method for estimating an enhanced super-resolution image 808 based on single-polarization raw data. Generally speaking, sub-images are combined to form a polarization degree image 802, as described above. A guidance mask 804 based on the polarization degree image 802 is assigned a weight map 806 to produce a super-resolution image 808 as Figure 8 shown.

[0061] Assuming that sufficient image patches and sub-pixel transformations (described earlier) are obtained, they are fused with selected weights. For each block in the reference sub-image, its similar image patch is found in the remaining sub-images. A new set of weight maps w n,i can be generated based on brightness under the guidance map, where n is the image patch number and i represents the i-th block image. As shown below, the weight map is a function of pixel intensity and self-similarity. When calculating the fused image, pixels with good exposure and high similarity should be weighted more.

[0062] w n,i = f(I n , D n ) (5)

[0063] The weights between various polarization images can be achieved by the distance between the image patch intensity In and the target intensity Itar. The weights between similar image patches are determined by the self-similarity distance Dn of the similar image patches. Given the sub-pixel transformation, an enhanced super-resolution image HR will be obtained after fusion using Equation (6).

[0064]

[0065] To retain the detail information and good exposure areas, high resolution is the goal of the fusion algorithm. The enhanced super-resolution image is obtained by merging the polarization sub-images under the weight map.

[0066] Figure 9 A method for generating a super-resolution image as shown in flowchart 900 is illustrated. The method includes: capturing an image 902 of an object disposed in front of a camera. The amount of light entering the camera can be limited by the aperture of the main lens of the camera. The next step includes: sensing the polarization perception of the image of the object by means of a polarization sensor to create one or more low-resolution sub-images 904. The polarization image sensor is a CMOS sensor, which is similar to a color filter array composed of an array of pixels with different polarization directions. Close-up images of polarization pixels A, B, C, and D representing four pixels with four polarization directions have been previously shown and described. Next, one or more processed sub-images are extracted from the one or more sub-images by means of an image extractor 906. In the preprocessing of the sub-images, a smoothing filter is used to remove bad pixels and reduce the noise level.

[0067] The next step includes: calculating the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP) 908 for each of the one or more processed sub-images. Then, in step 910, a guidance map is generated for each of the one or more processed sub-images based on the calculated degree of linear polarization (DOLP) and the angle of linear polarization (AOLP). In addition, a weight mask is calculated for each of the one or more processed sub-images based on a guidance mask 912. Next, the weight mask is assigned to the one or more processed sub-images to generate one or more weighted sub-images 914. Finally, in step 916, the one or more weighted sub-images are merged to generate a super-resolution image.

[0068] Figure 10A method of generating a super-resolution image by polarization as shown in the flowchart 1000 is shown. The flowchart method includes: capturing an image 1002 of an object disposed in front of a camera. The amount of light entering the camera can be limited by the aperture of the main lens of the camera. The next step includes: sensing the polarization perception of the object image by means of a polarization sensor to create one or more low-resolution sub-images 1004. The polarization image sensor is a CMOS sensor, which is similar to a color filter array composed of an array of pixels with different polarization directions. Close-up images of polarization pixels A, B, C, and D representing four pixels with four polarization directions have been shown and described previously. The next step includes: extracting one or more processed sub-images 1006 from the one or more sub-images by means of an image extractor. In the preprocessing of the sub-images, a smoothing filter is used to remove bad pixels and reduce the noise level.

[0069] The next step includes: calculating the degree of linear polarization (DOLP) and the angle of linear polarization (AOLP) 1008 for each of the one or more processed sub-images. Then, in step 1010, a guidance map is generated for each of the one or more processed sub-images based on the calculated degree of linear polarization (DOLP) and the angle of linear polarization (AOLP). In addition, a weight mask is calculated for each of the one or more processed sub-images based on a guidance mask 1012. Next, the weight mask is assigned to the one or more processed sub-images to generate one or more weighted sub-images 1014. Finally, in step 1016, the one or more weighted sub-images are combined to generate a super-resolution image. In the final step 1018, an output module is responsible for outputting the final super-resolution image.

[0070] Although various embodiments of the present invention have been described above, it should be understood that they are presented by way of example rather than limitation. Similarly, the drawings may depict an exemplary architecture or other configuration for the present invention, which helps to understand the features and functions that may be included in the present invention. The present invention is not limited to the exemplary architecture or configuration shown, but various alternative architectures and configurations can be used to achieve the desired features.

[0071] Although the present invention has been described above according to various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functions described in one or more of the individual embodiments are not limited to their application to the specific embodiments described, but can be applied alone or in various combinations to one or more other embodiments of the present invention, whether or not these embodiments are described, and whether or not these features are presented as part of the described embodiments. Therefore, the breadth and scope of the present invention should not be limited by any of the above exemplary embodiments.

[0072] Broad terms and phrases such as "one or more", "at least", "but not limited to", or other similar phrases in some cases should not be construed to mean that the narrower case is intended or required in instances where such broad phrases may not be present.

Claims

1. A polarization imaging system for generating super-resolution images, wherein, The polarization imaging system includes: A camera for capturing images, wherein the camera includes: A lens; and A polarization image sensor, wherein the polarization image sensor senses the polarization perception of the image to create one or more sub-images; A polarization analysis module, wherein the polarization analysis module includes: An image extractor, wherein the image extractor receives the one or more sub-images and extracts one or more processed sub-images from the one or more sub-images; and A first processor, wherein the first processor calculates the degree of linear polarization and the angle of linear polarization for each of the one or more processed sub-images; A weight module, wherein the weight module includes: A second processor, wherein the second processor generates a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the angle of linear polarization; A weight mask module, wherein the weight mask module calculates a weight mask for each of the one or more processed sub-images based on the guidance map, and wherein the weight mask module assigns the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; and A fusion module, wherein the fusion module combines the one or more weighted sub-images to generate the super-resolution image.

2. The polarization imaging system according to claim 1, wherein The degree of linear polarization and the polarization angle are based on the Stokes vector.

3. The polarization imaging system according to claim 1, wherein The fusion module combines the one or more weighted sub-images by fusion imaging.

4. The polarization imaging system according to claim 3, wherein The fusion imaging is based on surface reflection reduction or surface normal mapping.

5. The polarization imaging system according to claim 1, wherein The one or more sub-images are low-resolution intensity images with different polarization directions.

6. The polarization imaging system according to claim 1, wherein The fusion module is an algorithm for combining the one or more weighted sub-images.

7. The polarization imaging system according to claim 6, characterized in that The algorithm retains the detail information and good exposure area of the super-resolution image.

8. An imaging processing system for generating a super-resolution image by polarization, wherein, The imaging processing system includes: An electronic device for capturing images, wherein the electronic device includes: A camera with a lens; and At least one polarization image sensor with a pixel array having different polarization directions, wherein the at least one polarization image sensor senses the polarization perception of the image to create one or more sub-images; A polarization analysis module, wherein the polarization analysis module includes: An image extractor, wherein the image extractor receives the one or more sub-images and extracts one or more processed sub-images from the one or more sub-images; and A first processor, wherein the first processor calculates the degree of linear polarization and the angle of linear polarization based on Stokes vector analysis for each of the one or more processed sub-images; A weight module, wherein the weight module includes: A second processor, wherein the second processor generates a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the angle of linear polarization, and wherein the second processor calculates the pixel intensity and similarity of the one or more processed sub-images based on the guidance map; A weight mask module, wherein the weight mask module calculates a weight mask for each of the one or more processed sub-images based on the guidance map, and wherein the weight mask module assigns the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; and A fusion module, wherein the fusion module combines the one or more weighted sub-images to form a super-resolution image; and An output module, wherein the output module provides the super-resolution image by polarization.

9. The imaging processing system according to claim 8, wherein The at least one polarization image sensor includes a plurality of polarization direction channels.

10. The imaging processing system according to claim 9, wherein, The number of the polarization direction channels is four.

11. The imaging processing system according to claim 8, wherein The one or more sub-images are low-resolution intensity images with different polarization directions.

12. The imaging processing system according to claim 8, wherein The Stokes vector represents the polarization level of a light beam.

13. The imaging processing system according to claim 12, wherein The Stokes vector is characterized by the light intensity of the light beam at different angles.

14. The imaging processing system according to claim 13, wherein The different angles are 0°, 45°, 90° or 135°.

15. The imaging processing system according to claim 8, wherein The degree of linear polarization and the angle of linear polarization form a data cube.

16. The imaging processing system according to claim 15, wherein The data cube performs arbitrary polarization on the one or more sub-images.

17. A method for generating a super-resolution image, wherein, The method includes: Capturing an image; Sensing the polarization perception of the image to create one or more sub-images; Extracting one or more processed sub-images from the one or more sub-images; Generating a degree of linear polarization and an angle of linear polarization for each of the one or more processed sub-images; Calculating a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the angle of linear polarization; Calculating a weight mask for each of the one or more processed sub-images based on the guidance map; Assigning the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; and Combining the one or more weighted sub-images to generate the super-resolution image.

18. A method for generating a super-resolution image by polarization, wherein, The method includes: Capturing an image; Sensing the polarization perception of the image to create one or more sub-images; Extracting one or more processed sub-images from the one or more sub-images based on bad pixels and noise levels; Generating a degree of linear polarization and an angle of linear polarization for each of the one or more processed sub-images based on the Stokes vector; Generating a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the angle of linear polarization; Calculating a weight mask for each of the one or more processed sub-images based on the guidance map; Assigning the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; Combining the one or more weighted sub-images to generate a super-resolution image; and Outputting the super-resolution image.

19. A computer-usable medium having computer program logic for causing at least one processor in a computer system to generate a super-resolution image via a software platform, the computer program logic including: Capturing an image; Sensing the polarization perception of the image to create one or more sub-images; Extract one or more processed sub-images from the one or more sub-images; Generate a degree of linear polarization and a linear polarization angle for each of the one or more processed sub-images; Calculate a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the linear polarization angle; Calculate a weight mask for each of the one or more processed sub-images based on the guidance map; Assign the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; And Merge the one or more weighted sub-images to generate the super-resolution image.

20. A computer-usable medium having computer program logic for causing at least one processor in a computer system to generate a super-resolution image via a software platform, the computer program logic comprising: Capture an image; Sense the polarization perception of the image to create one or more sub-images; Extract one or more processed sub-images from the one or more sub-images based on bad pixels and noise levels; Generate a degree of linear polarization and a linear polarization angle for each of the one or more processed sub-images based on the Stokes vector; Generate a guidance map for each of the one or more processed sub-images based on the degree of linear polarization and the linear polarization angle; Calculate a weight mask for each of the one or more processed sub-images based on the guidance map; Assign the weight mask to the one or more processed sub-images to generate one or more weighted sub-images; Merge the one or more weighted sub-images to generate a super-resolution image; And Output the super-resolution image.

Images