An optical image processing system and method based on spatial multi-dimension polarization information
By using an optical image processing system based on spatial multi-dimensional polarization information, the problems of long image processing time and difficulty in acquiring polarization information in existing technologies are solved. This system enables real-time transformation and multi-dimensional acquisition of image information, improving the real-time performance and efficiency of imaging.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING RES INST OF SPATIAL MECHANICAL & ELECTRICAL TECH
- Filing Date
- 2022-11-28
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies suffer from drawbacks such as long post-processing time for computer images, weak image processing capabilities of optical systems, and inability to acquire polarization information simultaneously. Traditional polarization spectroscopy techniques also suffer from problems such as noise sensitivity, channel crosstalk, and low spectral resolution, resulting in long image processing times and low accuracy.
An optical image processing system based on spatial multi-dimensional polarization information is adopted. Through the image processing requirement module, the optical system spectrum modulation module, the spectrum control module programming module, the imaging optical system forward calculation module, and the polarization requirement information detector module, the real-time transformation and multi-dimensional acquisition of image information are realized. Combined with polarization elements, selective noise reduction is performed to obtain clear images.
It enables real-time transformation and multi-dimensional acquisition of image information, reduces the computation time of subsequent image processing systems, improves the real-time performance and efficiency of imaging, and possesses the high efficiency of optical system imaging.
Smart Images
Figure CN116029889B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an optical image processing system and method based on spatial multi-dimensional polarization information, belonging to the field of computational optical imaging. Background Technology
[0002] Computational imaging technology integrates knowledge from multiple disciplines such as optics, graphics, and information processing, and is currently a hot research topic in interdisciplinary fields. Through innovation in imaging modes and improvement in algorithm processing, computational imaging technology simplifies traditional hardware imaging systems and enhances imaging capabilities, especially providing a feasible solution for the simultaneous acquisition of multidimensional information. It has broad application prospects in fields such as aerospace remote sensing, security monitoring, and imaging under special conditions.
[0003] Light field information possesses seven dimensions, while current detectors operate in a two-dimensional imaging mode, often discarding other dimensions to acquire limited dimensional information. Multi-dimensional information encompasses more feature information about the target; for example, polarization information can reflect relevant information about the target surface, and can be used to detect particle size, morphology, and roughness of material surfaces. It can also detect and identify faint targets, and its applications include polarization dehazing, atmospheric sounding, and deep space exploration. Image processing, on the other hand, can often extract and enhance key information as needed.
[0004] Currently, acquired and captured image information requires subsequent processing on a backend computer based on actual application needs. The time required from acquisition to image processing is considerable, and reducing the computation time of the subsequent image processing system is crucial. However, traditional polarization spectroscopy suffers from issues such as noise sensitivity, channel crosstalk, and low spectral resolution, affecting the accuracy of the acquired polarization spectra. Due to the high-dimensional nature of polarization spectral information, dimensionality reduction is often achieved through dimensionality segmentation, resulting in sacrifices in temporal and spatial resolution. Existing image processing technologies often employ subsequent computer algorithms for restoration, which is time-consuming. Systems using optical processing methods can currently only handle simple needs such as noise reduction and depth of field, lacking broader image processing capabilities and the ability to detect polarization information. Summary of the Invention
[0005] The technical problem solved by this invention is to address the shortcomings of existing technologies, such as the time required for post-processing images in computers, the weak image processing capabilities of optical systems, and the inability to acquire polarization information simultaneously. This invention proposes an optical image processing system and method based on spatial multi-dimensional polarization information.
[0006] The present invention solves the above-mentioned technical problem through the following technical solution:
[0007] An optical image processing system based on spatial multi-dimensional polarization information includes an image processing requirement module, an optical system spectrum modulation module, a spectrum control module programming module, an imaging optical system forward calculation module, and a polarization requirement information detector module, wherein:
[0008] The image processing requirement module determines the corresponding image processing convolution factor based on the image processing task requirements and constructs the image processing requirement model required for the image processing task.
[0009] The optical system's spectrum modulation module determines the corresponding Fourier transform parallel light region based on the image processing convolution factor determined by the image processing requirement module, designs the spectrum modulation model, determines the spectrum adjustment model, and outputs the desired modulation surface information to the spectrum control module's programming module.
[0010] The spectrum control module programming module receives modulation surface information, and according to the image processing method required by the image processing task, writes a program to transform the modulation pattern and phase modulation plate shape, and creates a spectrum modulation surface pattern or phase modulation plate shape. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectrum modulation surface pattern or phase modulation plate shape.
[0011] The forward calculation module of the imaging optical system constructs a forward model of the imaging optical system based on the image processing requirement model and the spectrum adjustment model;
[0012] The polarization demand information detector module uses the forward model of the imaging optical system to selectively split or process polarization information in the output image. It also uses polarization elements within the polarization demand information detector module to selectively denoise polarized light information, thereby acquiring and outputting a clear image of the target scene.
[0013] The image processing requirement model implements image processing functions including edge extraction, Gaussian blur, noise reduction, dehazing, and deblurring based on the image processing convolution factor.
[0014] In the optical system's spectrum modulation module, during the spectrum modulation model design process, the high-frequency and low-frequency information and regions required by the image processing task are characterized, the required information is extracted, and after designing the modulation surface mask and phase information distribution, the spectrum adjustment model is constructed.
[0015] The spectrum control module programming module uses a built-in spectrum modulation algorithm to program and decompose the spectrum adjustment model. According to different image processing requirements, different pre-written programs are selected to control the corresponding mask to present different coded patterns or to control the phase modulation plate surface shape through code. After external light passes through the encoding plate or modulation plate, edge extraction, contour enhancement, Gaussian blur, noise reduction, dehazing, and deblurring are completed, and the modulated image is output.
[0016] Based on the image processing requirement model, the spectrum modulation model, and the modulated light output by the spectrum control module programming module, the final processed image result is determined, and the forward model of the imaging optical system is constructed. This model is then used to selectively process the polarization information of the output image from the spectrum control module programming module.
[0017] The forward model of the imaging optical system selects and collects polarization information based on preset input requirements. When processing the output image, it works with the polarization requirement information detector module to selectively split or process polarization information, and uses polarization elements to selectively denoise polarized light information.
[0018] The polarization requirement information detector module uses a pixel polarization detector to acquire the polarization of the modulated image output from the forward model of the optical system. Based on the polarization application requirements of the final image, the polarization element in the polarization requirement information detector module is used to selectively denoise the polarized light information, determine whether to split or process the polarization information, and acquire and output the polarization information image.
[0019] The forward model of the imaging optical system is specifically as follows:
[0020]
[0021] In the formula, f represents the obtained clear image data, O represents the input data of the output image, h1 represents spatial spectrum modulation, h2 represents polarization information extraction, and η represents the noise error introduced during sampling.
[0022] An optical image processing method based on spatial multi-dimensional polarization information includes:
[0023] Based on the requirements of the image processing task, determine the corresponding image processing convolution factor and construct the image processing requirement model required for the image processing task.
[0024] Based on the image processing convolution factor determined by the image processing requirement module, the corresponding Fourier transform parallel light region is determined, and a spectrum modulation model is designed to determine the spectrum adjustment model.
[0025] Determine the desired modulation surface information for image processing requirements, and create a spectral modulation surface pattern or phase modulation plate shape according to the image processing method required by the image processing task. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectral modulation surface pattern or phase modulation plate shape.
[0026] A forward model of the imaging optical system is constructed, and the polarization information is selectively split or processed in the output image. The polarization information is selectively denoised by polarization elements, and a clear image of the target scene is acquired and output.
[0027] During image processing in the forward model of the imaging optical system, polarization information is split or processed using a pixel polarization detector. Polarization elements are used to selectively denoise and process polarized light information. The pixel polarization detector is embedded with polarizers of different polarization angles to achieve polarization information modulation at different angles.
[0028] The advantages of this invention compared to the prior art are:
[0029] This invention provides an optical image processing system and method based on spatial multi-dimensional polarization information. The system adopts a system design that combines optical system spectrum modulation with polarization acquisition to realize real-time transformation and multi-dimensional acquisition of image information. Based on the instantaneous processing performance of the optical system for incident light, an image with special requirements can be obtained in a single exposure. This can significantly reduce the calculation time of the subsequent image processing system and transfer data processing from the traditional back-end calculation operation to the front-end optical system. It has the characteristics of real-time imaging and high efficiency of optical system. Attached Figure Description
[0030] Figure 1 A flowchart of the image acquisition process of a computational optical system for image processing provided for the invention;
[0031] Figure 2 A schematic diagram of the layout of the spatial multi-dimensional polarization acquisition device provided for the invention;
[0032] Figure 3 Example diagram of the design of the spectrum modulation model provided for the invention;
[0033] Figure 4 A flowchart of a spatial multi-dimensional polarization acquisition method provided for the invention;
[0034] Figure 5 A schematic diagram of an example of a spatial multi-dimensional polarization acquisition device provided for the invention; Detailed Implementation
[0035] An optical image processing system and method based on spatial multi-dimensional polarization information is disclosed. The processing system, composed of optical image processing systems based on spatial multi-dimensional polarization information, adopts a design process that combines optical system spectrum modulation with polarization acquisition to realize real-time transformation and multi-dimensional acquisition of image information. This can significantly reduce the computation time of subsequent image processing systems, and transfer data processing from traditional back-end computing operations to the front-end optical system. It has the characteristics of real-time imaging and high efficiency of optical systems.
[0036] The specific functions of each module in the processing system are as follows:
[0037] The image processing requirements module determines the corresponding image processing convolution factor based on the image processing task requirements and constructs the image processing requirements model required for the image processing task.
[0038] The optical system spectrum modulation module determines the corresponding Fourier transform parallel light region based on the image processing convolution factor determined by the image processing requirement module, designs the spectrum modulation model, determines the spectrum adjustment model, and outputs the desired modulation surface information of the image processing requirements to the spectrum control module programming module.
[0039] The spectrum modulation module programming module receives modulation surface information, and according to the image processing method required by the image processing task, creates a spectrum modulation surface pattern or phase modulation plate shape. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectrum modulation surface pattern or phase modulation plate shape.
[0040] The forward calculation module of the imaging optical system constructs a forward model of the imaging optical system based on the image processing requirement model and the spectrum adjustment model;
[0041] The polarization demand information detector module uses the forward model of the imaging optics system to selectively split or process polarization information in the output image. It also uses polarization elements within the polarization demand information detector module to selectively denoise polarized light information, thereby acquiring and outputting a clear image of the target scene.
[0042] The image processing requirement model implements image processing functions including edge extraction, Gaussian blur, noise reduction, dehazing, and deblurring based on the image processing convolution factor.
[0043] In the optical system spectrum modulation module, during the spectrum modulation model design process, the high-frequency and low-frequency information and regions required by the image processing task are characterized, the required information is extracted, and after designing the modulation surface mask and phase information distribution, the spectrum adjustment model is completed.
[0044] The spectrum control module programming module uses a built-in spectrum modulation algorithm to programmatically decompose the spectrum adjustment model. According to different image processing requirements, different pre-written programs are selected to control the corresponding mask to present different coded patterns or to control the phase modulation plate surface shape through code. After external light passes through the encoding plate or modulation plate, edge extraction, contour enhancement, Gaussian blur, noise reduction, dehazing, and deblurring are completed, and the modulated image is output.
[0045] Based on the image processing requirement model, modulation model, and modulated light output by the spectrum control module programming module, the final processed image result is determined, and the forward model of the imaging optical system is constructed. This model is then used to selectively process polarization information in the output image of the spectrum control module programming module.
[0046] The forward model of the imaging optical system selects and collects polarization information based on preset input requirements. When processing the output image, it works with the polarization requirement information detector module to selectively split or process polarization information, and uses polarization elements to selectively denoise polarized light information.
[0047] The polarization demand information detector module uses a pixel polarization detector to acquire polarization of the modulated image output from the forward model of the optical system. Based on the polarization application requirements of the final image, the polarization element in the polarization demand information detector module is used to select and denoise the polarized light information, determine whether to split or process the polarization information, and acquire and output the polarization information image.
[0048] The forward model of an imaging optical system is as follows:
[0049]
[0050] In the formula, f represents the obtained clear image data, O represents the input data of the output image, h1 represents spatial spectrum modulation, h2 represents polarization information extraction, and η represents the noise error introduced during sampling.
[0051] The optical image processing method based on spatial multi-dimensional polarization information has the following specific steps:
[0052] Based on the requirements of the image processing task, determine the corresponding image processing convolution factor and construct the image processing requirement model required for the image processing task.
[0053] Based on the image processing convolution factor determined by the image processing requirement module, the corresponding Fourier transform parallel light region is determined, and a spectrum modulation model is designed to determine the spectrum adjustment model.
[0054] Determine the desired modulation surface information for image processing requirements, and create a spectral modulation surface pattern or phase modulation plate shape according to the image processing method required by the image processing task. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectral modulation surface pattern or phase modulation plate shape.
[0055] A forward model of the imaging optical system is constructed, and the polarization information is selectively split or processed in the output image. The polarization information is selectively denoised by polarization elements, and a clear image of the target scene is acquired and output.
[0056] During image processing in the forward model of the imaging optical system, polarization information is split or processed using a pixel polarization detector. Polarization elements are used to selectively denoise polarized light information. The pixel polarization detector is embedded with polarizers of different polarization angles to achieve polarization information modulation at different angles.
[0057] The following description, in conjunction with the accompanying drawings and preferred embodiments, provides further details:
[0058] In the current embodiment, such as Figure 1 As shown, the computational optical system for image processing according to an embodiment of the present invention includes: S1, an image processing requirement module constructed according to actual needs; S2, a spectrum modulation module for the optical system corresponding to image processing; S3, a spectrum control module programming module; S4, a computational imaging optical system forward module; and S5, a polarization requirement information detector module. The image processing requirement model constructed according to actual needs involves constructing corresponding image processing convolution factors based on image processing requirements, thus completing the parameterized construction of the requirement model for the processing scheme. The spectrum modulation module for the optical system corresponding to image processing in S2 designs a corresponding spectrum modulation model for the Fourier transform parallel light region corresponding to the optical system. The spectrum control module programming module in S3 receives modulation surface information from the output of S2 and, according to the required image processing method, creates a spectrum modulation surface pattern or phase modulation plate shape. In S3, the spectrum control module programming module is located after the optical system spectrum modulation module in step S2. In S4, the computational imaging optical system forward module is a model of a fast computational optical system and acquisition method, used for information acquisition by the optical image processing system. The computational imaging optical system forward module realizes the modulation of the optical two-dimensional Fourier spectrum image. In S5, polarizers with different polarization angles are embedded in front of the pixel polarization detector to achieve polarization information modulation at different angles. In S5, the polarization demand information detector module reprocesses the image calculated and processed by the optical system, selectively splitting or passing polarization information, and selectively denoising polarized light information through polarization elements to obtain and output a clear image of the target scene.
[0059] like Figure 2 The diagram shows the layout of the spatial multi-dimensional polarization acquisition device provided by the invention. The upper black rectangular detector acquires unprocessed imaging information, and the lower black rectangular detector acquires imaging information from the computational optical system. The spatial multi-dimensional polarization acquisition method is as follows: Figure 4 As shown in the diagram, a schematic diagram of a spatial multi-dimensional polarization acquisition device is presented. Figure 5 As shown.
[0060] When designing the spectral modulation module S2 of the optical system corresponding to image processing, a transmittance of better than 50% is used to ensure the light flux provided by the acquisition system. The size of the spatial coding random modulation module S3 needs to be matched with the focal length and the camera pixel size.
[0061] After receiving the requirement model, the optical system spectrum modulation module S2 designs the corresponding spectrum modulation module for the Fourier transform parallel light region of the optical system. The spectrum modulation model design is as follows: Figure 3As shown, the desired modulation surface information is output through the output terminal of S2 and imaged on the polarization requirement information detector module S5 at the back end.
[0062] The S5 pixel polarization detector module is equipped with polarizers of different polarization angles, and can acquire 0°, 45°, 90° and 135° polarized light through initial settings.
[0063] In one embodiment of the present invention, spectral surface coding is used to implement the edge extraction function.
[0064] In one embodiment of the present invention, the forward model of the computational imaging optical system described in S4 can be represented as follows:
[0065]
[0066] f represents the data obtained in the embodiment, O represents the original input data, h1 represents spatial spectrum modulation, h2 represents the subsequent extraction of polarization information, and η represents the noise error introduced during system sampling.
[0067] In one embodiment of the present invention, the data collected by the pixel polarization detector module S3, combined with the polarization spectrum compression sampling forward module S4, is input into the high-precision polarization spectrum data reconstruction module S5 to realize the calculation and reconstruction of polarization spectrum data. The polarization spectrum images obtained from the calculation and reconstruction are then sequentially connected to obtain a polarization spectrum video.
[0068] The computational optical system for image processing according to embodiments of the present invention can obtain images with specific processing requirements in a single exposure. This invention significantly reduces the computation time of subsequent image processing systems, transferring data processing from traditional back-end computational operations to the front-end optical system, and features real-time and high-efficiency optical system imaging.
[0069] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.
[0070] The contents not described in detail in this specification are common knowledge to those skilled in the art.
Claims
1. An optical image processing system based on spatial multi-dimensional polarization information, characterized in that: It includes an image processing requirements module, an optical system spectrum modulation module, a spectrum control module programming module, an imaging optical system forward calculation module, and a polarization requirements information detector module, among which: The image processing requirement module determines the corresponding image processing convolution factor based on the image processing task requirements and constructs the image processing requirement model required for the image processing task. The optical system's spectrum modulation module determines the corresponding Fourier transform parallel light region based on the image processing convolution factor determined by the image processing requirement module, designs the spectrum modulation model, determines the spectrum adjustment model, and outputs the desired modulation surface information to the spectrum control module's programming module. The spectrum control module programming module receives modulation surface information, and according to the image processing method required by the image processing task, writes a program to transform the modulation pattern and phase modulation plate shape, and creates a spectrum modulation surface pattern or phase modulation plate shape. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectrum modulation surface pattern or phase modulation plate shape. The forward calculation module of the imaging optical system constructs a forward model of the imaging optical system based on the image processing requirement model and the spectrum adjustment model; The polarization demand information detector module uses the forward model of the imaging optical system to selectively split or process polarization information in the output image. The polarization element in the polarization demand information detector module achieves selective noise reduction of polarized light information, and acquires and outputs a clear image of the target scene. The image processing requirement model implements image processing functions including edge extraction, Gaussian blur, noise reduction, dehazing, and deblurring based on the image processing convolution factor. In the optical system's spectrum modulation module, during the spectrum modulation model design process, the high-frequency and low-frequency information and regions required by the image processing task are analyzed for features. The required information is extracted into components. After designing the modulation surface mask and phase information distribution, the spectrum adjustment model is constructed. The spectrum control module programming module uses a built-in spectrum modulation algorithm to program and decompose the spectrum adjustment model. According to different image processing requirements, different pre-written programs are selected to control the corresponding mask to present different coded patterns or to control the phase modulation plate surface shape through code. After external light passes through the encoding plate or modulation plate, edge extraction, contour enhancement, Gaussian blur, noise reduction, dehazing, and deblurring are completed, and the modulated image is output. Based on the image processing requirement model, the spectrum modulation model, and the modulated light output by the spectrum control module programming module, the final processed image result is determined, and the forward model of the imaging optical system is constructed. This model is then used to selectively process the polarization information of the output image from the spectrum control module programming module.
2. The optical image processing system based on spatial multi-dimensional polarization information according to claim 1, characterized in that: The forward model of the imaging optical system selects and collects polarization information based on preset input requirements. When processing the output image, it works with the polarization requirement information detector module to selectively split or process polarization information, and uses polarization elements to selectively denoise polarized light information. The polarization requirement information detector module uses a pixel polarization detector to acquire the polarization of the modulated image output from the forward model of the optical system. Based on the polarization application requirements of the final image, the polarization element in the polarization requirement information detector module is used to selectively denoise the polarized light information, determine whether to split or process the polarization information, and acquire and output the polarization information image.
3. An optical image processing method implemented by the optical image processing system based on spatial multi-dimensional polarization information according to claim 2, characterized in that... include: Based on the requirements of the image processing task, determine the corresponding image processing convolution factor and construct the image processing requirement model required for the image processing task. Based on the image processing convolution factor determined by the image processing requirement module, the corresponding Fourier transform parallel light region is determined, and a spectrum modulation model is designed to determine the spectrum adjustment model. Determine the desired modulation surface information for image processing requirements, and create a spectral modulation surface pattern or phase modulation plate shape according to the image processing method required by the image processing task. After receiving external light, it is converted into an output image after Fourier transform processing after passing through the spectral modulation surface pattern or phase modulation plate shape. Construct a forward model of the imaging optical system, selectively split or process the polarization information of the output image, use polarization elements to selectively denoise the polarized light information, and acquire and output a clear image of the target scene. During image processing in the forward model of the imaging optical system, polarization information is split or processed using a pixel polarization detector. Polarization elements are used to selectively denoise and process polarized light information. The pixel polarization detector is embedded with polarizers of different polarization angles to achieve polarization information modulation at different angles.
4. The optical image processing method according to claim 3, characterized in that: The forward model of the imaging optical system is specifically as follows: In the formula, f This represents the obtained clear image data. The input data represents the output image. h 1 indicates spatial spectrum modulation. h 2 indicates polarization information extraction. This represents the noise error introduced during sampling.