Polarization imaging system for forward street view and vertical sky light reuse and working method thereof

Abstract

The polarization imaging system and method for forward street view and vertical skylight multiplexing can synchronously capture forward street view visual image and vertical skylight polarization characteristics, has high integration, combines visual navigation based on forward street view and bionic polarization navigation of vertical skylight without using multiple sensors, and improves the precision and environmental adaptability of navigation. The system comprises a first band-pass filter (1), a second band-pass filter (2), a polarization beam splitter prism (3), an imaging lens (4), a color polarization camera (5), a POE (6), an upper computer (7) and a power module (8).

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of navigation and positioning of unmanned motion platforms, and in particular to a polarization imaging system for forward street view and vertical sky light reuse, and a working method of the polarization imaging system for forward street view and vertical sky light reuse. BACKGROUND

[0002] With the rapid development of unmanned motion platforms, autonomous navigation and positioning has become a key technology, and plays an increasingly important role in many fields such as transportation, resource exploration, mobile communication, etc. The common navigation technologies at present mainly include inertial navigation, satellite navigation, celestial navigation, geomagnetic navigation, etc. The inertial navigation system has the advantages of good autonomy and high short-time accuracy, but the error will accumulate over time in long-time work. The satellite navigation system has the characteristics of all-weather and global, and has no cumulative error, but the satellite signal is easy to be disturbed by electromagnetic interference, and cannot work normally in the city with high buildings and the jungle with high and dense trees. The celestial navigation system has a high cost and a low degree of integration, and the geomagnetic navigation system is easy to be disturbed by local ferromagnetic materials. The above mainstream navigation systems are difficult to completely meet the needs of practical applications. Therefore, seeking new auxiliary navigation methods through new research ideas and methods has become a hot research topic.

[0003] In recent years, visual navigation technology has been deeply researched and widely applied in unmanned vehicles, unmanned aerial vehicles and intelligent robots due to its high sensor integration and convenient use. For ground unmanned motion platforms, the forward street view contains rich information such as roads, buildings, obstacles and traffic signal markers, and is often used as the most important information source in visual navigation technology. However, the visual navigation based on single forward street view lacks absolute direction reference and scale information, and the accuracy is difficult to guarantee, and there are problems of easy generation of pure rotation error and poor environmental adaptability.

[0004] Bionic polarization navigation is a new type of auxiliary navigation method that uses the polarization characteristics of sky scattered light for navigation. Natural light emitted by the sun is scattered by atmospheric particles to produce scattered light, and the polarization characteristics of scattered light form a regular distribution pattern in the entire sky dome, which contains rich navigation information. Since the large-scale sky light polarization pattern is difficult to be disturbed by human beings, and its error does not accumulate over time, it can help solve the problem of autonomous navigation and positioning of unmanned motion platforms in complex environments with strong interference for a long time.

[0005] The single forward-looking street-based visual navigation capability is limited, and the reasonable fusion of the bionic polarization navigation and the visual navigation can effectively make up for the shortcomings of the visual navigation, such as lacking of direction reference, being prone to pure rotation error and poor environmental adaptability. The polarization navigation device developed by combining the forward-looking street and the vertical sky light can realize the reasonable combination of the two navigation methods by using one sensor, improve the navigation accuracy and simplify the device structure, and has a broad development prospect. SUMMARY

[0006] In order to overcome the defects of the prior art, the technical problem to be solved by the present application is to provide a polarization imaging system combining forward-looking street and vertical sky light, which can simultaneously capture forward-looking street visual images and vertical sky light polarization characteristics, has high integration, combines forward-looking street-based visual navigation and vertical sky light bionic polarization navigation without using multiple sensors, and improves the accuracy and environmental adaptability of navigation.

[0007] The technical scheme of the present application is that the polarization imaging system combining forward-looking street and vertical sky light comprises a first band-pass filter (1), a second band-pass filter (2), a polarization light splitting prism (3), an imaging lens (4), a color polarization camera (5), a POE (6), an upper computer (7) and a power module (8).

[0008] The system is installed on an unmanned motion platform, and the imaging optical axis thereof is perpendicular to the horizontal plane and points to the sky.

[0009] The unpolarized natural light from the forward-looking street is incident into the imaging system through the first band-pass filter, and reaches the imaging lens after being reflected by the polarization light splitting prism. The scattered sky light from the sky is partially polarized light, is incident into the system through the second band-pass filter, and the polarization component thereof is transmitted to the imaging lens through the polarization light splitting prism. The mixed forward-looking street unpolarized light and the vertical polarization sky light are imaged by the color polarization camera through the imaging lens.

[0010] The color polarization camera internally integrates a nano-grating polarization array, a Bayer filter array and a photoelectric detector array, and realizes the collection of the mixed image. The upper computer processes the collected mixed image, first separates the forward-looking street image and the polarization sky light image from the collected mixed image according to the arrangement rule of the Bayer filter array and performs preprocessing, and then calculates the atmospheric polarization mode from the polarization sky light image according to the arrangement rule of the nano-grating polarization array. The POE performs data transmission between the upper computer and the color polarization camera, and provides direct current power supply for the color polarization camera. The power module provides alternating current power supply for the upper computer and the POE.

[0011] The polarization imaging system for forward street view and vertical sky multiplexing disclosed in this invention achieves synchronous and rapid imaging of light with two polarization characteristics in two directions based on a single sensor through dual modulation of wavelength and polarization. It boasts advantages such as small size, high system integration, and good synchronization. Based on this polarization imaging system, biomimetic polarization navigation can be rationally combined with visual navigation, effectively overcoming the shortcomings of visual navigation such as lack of directional reference, susceptibility to pure rotational errors, and poor environmental adaptability, thereby improving navigation accuracy and environmental adaptability.

[0012] A method for operating this polarization imaging system that multiplexes forward street view and vertical sky light is also provided, comprising the following steps:

[0013] (1) The system is installed on an unmanned motion platform, and its imaging optical axis is perpendicular to the horizontal plane and points to the sky;

[0014] (2) Unpolarized natural light from the forward street scene enters the imaging system through the first bandpass filter, and after being reflected by the polarizing beam splitter, it reaches the imaging lens; the scattered sky light from the sky is partially polarized light, which enters the system through the second bandpass filter, and the polarized component in it is transmitted through the polarizing beam splitter and reaches the imaging lens.

[0015] (3) The aliased forward street scene unpolarized light and vertically polarized sky light are imaged by a color polarization camera through the imaging lens; the color polarization camera integrates a nano grating polarization array, a Bayer filter array and a photodetector array to realize the acquisition of aliased images.

[0016] (4) The host computer processes the acquired aliased images. First, according to the arrangement of the Bayer filter array, the forward street view image and the polarized sky light image are separated from the acquired aliased images and preprocessed. Then, according to the arrangement of the nano-grating polarization array, the atmospheric polarization mode is calculated from the polarized sky light image. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a polarization imaging system that multiplexes forward street view and vertical skylight according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the working method of the polarization imaging system for multiplexing forward street view and vertical skylight according to the present invention.

[0019] Figure 3 This is a schematic diagram illustrating the principle and effect of the host computer processing the acquired aliased images. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the term "comprising" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

[0022] like Figure 1 As shown, this polarization imaging system that combines forward street view and vertical sky light includes: a first bandpass filter 1, a second bandpass filter 2, a polarization beam splitter 3, an imaging lens 4, a color polarization camera 5, a POE 6, a host computer 7, and a power module 8.

[0023] The system is installed on an unmanned moving platform, with its imaging optical axis perpendicular to the horizontal plane and pointing towards the sky. Unpolarized natural light from the forward street scene enters the imaging system through the first bandpass filter, and after being reflected by a polarizing beam splitter, it reaches the imaging lens. Scattered sky light from the sky is partially polarized light, which enters the system through the second bandpass filter. Its polarized components are transmitted through a polarizing beam splitter and reach the imaging lens. The superimposed unpolarized light from the forward street scene and the vertically polarized sky light are imaged by a color polarization camera through the imaging lens.

[0024] The color polarization camera integrates a nano-grating polarization array, a Bayer filter array, and a photodetector array to acquire aliased images. The host computer processes the acquired aliased images. First, according to the arrangement of the Bayer filter array, it separates the forward street scene image and the polarized sky light image from the acquired aliased image and performs preprocessing. Then, according to the arrangement of the nano-grating polarization array, it calculates the atmospheric polarization mode from the polarized sky light image. The PoE performs data transmission between the host computer and the color polarization camera and provides DC power to the color polarization camera. The power module provides AC power to the host computer and the PoE.

[0025] The polarization imaging system for forward street view and vertical sky multiplexing disclosed in this invention achieves synchronous and rapid imaging of light with two polarization characteristics in two directions based on a single sensor through dual modulation of wavelength and polarization. It boasts advantages such as small size, high system integration, and good synchronization. Based on this polarization imaging system, biomimetic polarization navigation can be rationally combined with visual navigation, effectively overcoming the shortcomings of visual navigation such as lack of directional reference, susceptibility to pure rotational errors, and poor environmental adaptability, thereby improving navigation accuracy and environmental adaptability.

[0026] Preferably, the center wavelength of the passband of the first bandpass filter is in the band where the red light transmittance of the Bayer filter array in the color polarization camera is high.

[0027] Preferably, the center wavelength of the passband of the first bandpass filter is 630 nm.

[0028] Preferably, the center wavelength of the passband of the second bandpass filter is in the band where the blue light transmittance of the Bayer filter array in the color polarization camera is high.

[0029] Preferably, the center wavelength of the passband of the second bandpass filter is 450 nm.

[0030] Preferably, the arrangement of the nano-grating micro-polarization array used in the color polarization camera is as follows: four pixels are evenly arranged as a calculation unit, and each calculation unit contains at least three different nano-gratings with different linear polarization angles. The uniform arrangement of the same calculation units constitutes the nano-grating micro-polarization array.

[0031] Preferably, the image detector array used in the color polarization camera is a Sony IMX250MYR; the arrangement of its Bayer filter array is as follows: every four adjacent pixels form a computing unit, each unit is covered with a microfilter of the same color, and the red, blue, and green microfilters are arranged alternately; in the filtering characteristic curve, the center wavelengths of red light and blue light are 450nm to 470nm and 610nm to 630nm, respectively; the arrangement of its nanograting micropolarization array is as follows: four pixels form a computing unit, which is evenly arranged, and each computing unit contains nanogratings with four different polarization angles of 0°, 45°, 90°, and 135°.

[0032] Preferably, based on the arrangement of the Bayer filter array and the nanograting micro-polarization array of the color polarization camera, the host computer processes the acquired aliased images: first, it extracts all pixel combinations covered by the red filter in the Bayer filter array to form a forward street view polarization image, and extracts all pixel combinations covered by the blue filter in the Bayer filter array to form a vertical skylight polarization image; for the forward street view polarization image, it eliminates polarization characteristics by calculating the sum of pixel values ​​corresponding to 0° and 90° in the micro-polarization array to obtain an unbiased forward street view image; for the vertical skylight polarization image, it uses Stokes vectors to calculate the polarization angle image and polarization degree image, thereby obtaining the atmospheric polarization mode.

[0033] like Figure 2 As shown, the working method of this polarization imaging system that multiplexes forward street view and vertical sky light includes the following steps:

[0034] (1) The system is installed on an unmanned motion platform, and its imaging optical axis is perpendicular to the horizontal plane and points to the sky;

[0035] (2) Unpolarized natural light from the forward street scene enters the imaging system through the first bandpass filter, and after being reflected by the polarizing beam splitter, it reaches the imaging lens; the scattered sky light from the sky is partially polarized light, which enters the system through the second bandpass filter, and the polarized component in it is transmitted through the polarizing beam splitter and reaches the imaging lens.

[0036] (3) The aliased forward street scene unpolarized light and vertically polarized sky light are imaged by a color polarization camera through the imaging lens; the color polarization camera integrates a nano grating polarization array, a Bayer filter array and a photodetector array to realize the acquisition of aliased images.

[0037] (4) The host computer processes the acquired aliased images. First, according to the arrangement of the Bayer filter array, the forward street view image and the polarized sky light image are separated from the acquired aliased images and preprocessed. Then, according to the arrangement of the nano-grating polarization array, the atmospheric polarization mode is calculated from the polarized sky light image.

[0038] Preferably, step (4) includes: firstly, extracting all pixel combinations covered by the red filter in the Bayer filter array to form a forward street view polarized image, and extracting all pixel combinations covered by the blue filter in the Bayer filter array to form a vertical skylight polarized image; for the forward street view polarized image, by calculating the sum of the pixel values ​​corresponding to 0° and 90° in the micro-polarization array, the polarization characteristics are eliminated to obtain an unbiased forward street view image; for the vertical skylight polarized image, the polarization angle image and polarization degree image are calculated using the Stokes vector to obtain the atmospheric polarization mode.

[0039] Compared with the prior art, the present invention has the following advantages and features:

[0040] (1) The system described in this invention is the first to propose a polarization imaging system that combines forward street view and vertical sky light. This device achieves synchronous and rapid imaging of light with two polarization characteristics in two directions based on a single sensor by dual modulation of wavelength and polarization, and has the advantages of small size, high system integration and good synchronization.

[0041] (2) The system described in this invention can reasonably combine biomimetic polarization navigation and visual navigation, effectively overcoming the shortcomings of single visual navigation, such as lack of directional reference, easy generation of pure rotation error and poor environmental adaptability, and overcoming the problem of low information content of single biomimetic polarization navigation, thereby improving the accuracy and environmental adaptability of navigation.

[0042] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A polarization imaging system that multiplexes forward street view and vertical skylight, characterized in that: It includes: First bandpass filter (1), second bandpass filter (2), polarizing beam splitter (3), imaging lens (4), color polarizing camera (5), POE (6), host computer (7) and power module (8); The system is installed on an unmanned moving platform, with its imaging optical axis perpendicular to the horizontal plane and pointing towards the sky. Unpolarized natural light from the forward street scene enters the imaging system through the first bandpass filter, and after being reflected by a polarizing beam splitter, it reaches the imaging lens. Scattered sky light from the sky is partially polarized light, which enters the system through the second bandpass filter. Its polarized components are transmitted through a polarizing beam splitter and reach the imaging lens. The superimposed unpolarized light from the forward street scene and the vertically polarized sky light are imaged by a color polarization camera through the imaging lens. The color polarization camera integrates a nano-grating polarization array, a Bayer filter array, and a photodetector array to acquire aliased images. The host computer processes the acquired aliased images. First, according to the arrangement of the Bayer filter array, it separates the forward street scene image and the polarized sky light image from the acquired aliased image and performs preprocessing. Then, according to the arrangement of the nano-grating polarization array, it calculates the atmospheric polarization mode from the polarized sky light image. The PoE performs data transmission between the host computer and the color polarization camera and provides DC power to the color polarization camera. The power module provides AC power to the host computer and the PoE.

2. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 1, characterized in that: The center wavelength of the passband of the first bandpass filter is located in the band where the red light transmittance of the Bayer filter array in the color polarization camera is high.

3. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 2, characterized in that: The center wavelength of the passband of the first bandpass filter is 630 nm.

4. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 3, characterized in that: The center wavelength of the passband of the second bandpass filter is located in the band where the blue light transmittance of the Bayer filter array in the color polarization camera is high.

5. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 4, characterized in that: The center wavelength of the passband of the second bandpass filter is 450 nm.

6. The polarization imaging system for multiplexing forward street view and vertical skylight according to any one of claims 1-5, characterized in that: The arrangement of the nano-grating micro-polarization array used in the color polarization camera is as follows: four pixels are evenly arranged as a calculation unit, and each calculation unit contains at least three different nano-gratings with different linear polarization angles. The uniform arrangement of the same calculation units constitutes the nano-grating micro-polarization array.

7. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 6, characterized in that: The color polarization camera uses a Sony IMX250MYR image detector array; its Bayer filter array is arranged as follows: every four adjacent pixels form a computing unit, each unit is covered with a microfilter of the same color, and the red, blue and green microfilters are arranged alternately; in the filtering characteristic curve, the center wavelengths of red light and blue light are 450nm to 470nm and 610nm to 630nm, respectively; its nanograting micropolarization array is arranged as follows: four pixels form a computing unit, which is evenly arranged, and each computing unit contains nanogratings with four different polarization angles of 0°, 45°, 90° and 135°.

8. The polarization imaging system for multiplexing forward street view and vertical skylight according to claim 7, characterized in that: Based on the arrangement of the Bayer filter array and the nanograting micro-polarization array of the color polarization camera, the host computer processes the acquired aliased images: First, it extracts all pixel combinations covered by the red filter in the Bayer filter array to form a forward street view polarization image, and extracts all pixel combinations covered by the blue filter in the Bayer filter array to form a vertical skylight polarization image; for the forward street view polarization image, it eliminates polarization characteristics by calculating the sum of pixel values ​​corresponding to 0° and 90° in the micro-polarization array to obtain an unbiased forward street view image; for the vertical skylight polarization image, it uses Stokes vectors to calculate the polarization angle image and polarization degree image, thereby obtaining the atmospheric polarization mode.

9. The operating method of the polarization imaging system for multiplexing forward street view and vertical skylight according to any one of claims 1-8, characterized in that: It includes the following steps: (1) The system is installed on an unmanned motion platform, and its imaging optical axis is perpendicular to the horizontal plane and points to the sky; (2) Unpolarized natural light from the forward street scene enters the imaging system through the first bandpass filter, and after being reflected by the polarizing beam splitter, it reaches the imaging lens; the scattered sky light from the sky is partially polarized light, which enters the system through the second bandpass filter, and the polarized component in it is transmitted through the polarizing beam splitter and reaches the imaging lens. (3) The aliased forward street scene unpolarized light and vertically polarized sky light are imaged by a color polarization camera through the imaging lens; the color polarization camera integrates a nano grating polarization array, a Bayer filter array and a photodetector array to realize the acquisition of aliased images. (4) The host computer processes the acquired aliased images. First, according to the arrangement of the Bayer filter array, the forward street view image and the polarized sky light image are separated from the acquired aliased images and preprocessed. Then, according to the arrangement of the nano-grating polarization array, the atmospheric polarization mode is calculated from the polarized sky light image.

10. The working method of the polarization imaging system for multiplexing forward street view and vertical skylight according to claim 9, characterized in that: Step (4) includes: firstly, extracting all pixel combinations covered by the red filter in the Bayer filter array to form a forward street view polarization image, and extracting all pixel combinations covered by the blue filter in the Bayer filter array to form a vertical skylight polarization image; for the forward street view polarization image, by calculating the sum of the pixel values ​​corresponding to 0° and 90° in the micro-polarization array, the polarization characteristics are eliminated to obtain an unbiased forward street view image; for the vertical skylight polarization image, the polarization angle image and polarization degree image are calculated using the Stokes vector to obtain the atmospheric polarization mode.

Images