Image sensor, imaging device, and imaging method

Abstract

This application relates to the field of optoelectronic imaging technology, specifically to an image sensor, imaging device, and imaging method. The image sensor includes a pixel array, which comprises multiple functional units arranged in an array. Each functional unit includes at least two pixel sub-units. Each pixel sub-unit includes at least two polarization pixels and at least one spectral pixel. The spectral pixel includes a first photosensitive pixel layer and a narrowband filter layer. In each functional unit, the narrowband filter layer of the spectral pixel corresponding to the at least two pixel sub-units has a different center wavelength. This application enables efficient acquisition of polarization and spectral information through a single exposure and effectively reduces device size and manufacturing costs.

Description

Technical Field

[0001] This application relates to the field of optoelectronic imaging technology, and more specifically, to an image sensor, imaging device, and imaging method. Background Technology

[0002] With the development of smart terminals, machine vision, and computational imaging technologies, image sensors not only need to acquire two-dimensional intensity information of targets, but also need to have the ability to sense multi-dimensional optical characteristics such as the polarization state of light. Polarization imaging can reflect the microscopic geometric structure and material properties of an object's surface and is widely used in fields such as 3D reconstruction, anti-reflection, and material identification; while spectral imaging acquires light intensity information at different wavelengths to achieve accurate color reproduction, color temperature detection, and material composition analysis.

[0003] However, traditional imaging devices typically employ a discrete design, acquiring polarization and spectral images separately through multiple independent sensors or complex optical switching devices. This approach suffers from drawbacks such as large size, high cost, and difficulty in miniaturization, making it unsuitable for mobile devices. Furthermore, current technologies cannot simultaneously acquire polarization and spectral information in a single exposure, limiting the imaging device's ability to operate in complex environments. Summary of the Invention

[0004] The purpose of this application is to provide an image sensor, imaging device and imaging method that can achieve efficient acquisition of polarization and spectral information through a single exposure, and can effectively reduce the size of the device and manufacturing cost.

[0005] This application is implemented as follows: In a first aspect, this application provides an image sensor, including a pixel array, the pixel array including a plurality of functional units arranged in an array; each functional unit including at least two pixel sub-units; each pixel sub-unit including at least two polarized pixels and at least one spectral pixel, the spectral pixel including a first photosensitive pixel layer and a narrowband filter layer; wherein, in each functional unit, the narrowband filter layer of the spectral pixel corresponding to the at least two pixel sub-units has a different center wavelength.

[0006] As an optional implementation, the polarized pixel includes a second photosensitive pixel layer and a micro-polarizer, the micro-polarizer having a preset linear polarization transmission direction; wherein, in each pixel sub-unit, the polarization transmission directions of the micro-polarizers of at least two polarized pixels are at an angle to each other.

[0007] As an optional implementation, in each pixel sub-unit, the polarization transmission direction angle of the micro-polarizers on at least two polarized pixels is greater than 60°.

[0008] As an optional implementation, the pixel subunit includes three polarization pixels and one spectral pixel arranged in a 2*2 array.

[0009] As an optional implementation, in the three polarized pixels included in the pixel sub-unit, the polarization transmission direction of the micro-polarizer of the first polarized pixel makes an angle of 45° with the polarization transmission direction of the micro-polarizer of the second polarized pixel; the polarization transmission direction of the micro-polarizer of the third polarized pixel is orthogonal to the polarization transmission direction of the first polarized pixel.

[0010] As an optional implementation, the polarization pixel further includes a first infrared filter layer.

[0011] As an optional implementation, each of the functional units further includes at least one infrared pixel, the infrared pixel comprising a third photosensitive pixel layer and a second infrared filter layer.

[0012] Secondly, this application provides an imaging method applied to the aforementioned image sensor, the method comprising: A polarization image corresponding to at least two polarization directions is obtained based on the first light signal received from the target scene by the polarization pixel; a multi-channel spectral image is obtained based on the second light signal received from the target scene by the spectral pixel.

[0013] As an optional implementation, the method further includes: calculating the color temperature information of the target scene by means of the light intensity distribution of different bands in the multi-channel spectral image.

[0014] As an optional implementation, the method further includes: correcting the original image based on the color temperature information of the target scene to obtain a corrected image; wherein the original image is an image displayed on the screen of an electronic device or an image captured by the camera of an electronic device.

[0015] Thirdly, this application provides an imaging device, including an infrared light source, a lens, and the aforementioned image sensor; the infrared light source is used to emit infrared polarized light toward a target scene; the light signal reflected from the target scene passes through the lens and enters the image sensor.

[0016] The beneficial effects of the embodiments of this application include: The image sensor, imaging device, and imaging method provided in this application embodiment, by coordinating polarization pixels and multi-band spectral pixels to construct a hierarchical structure of pixel sub-units and functional units, achieves synchronous acquisition of polarization and spectral information in a single exposure, significantly improving imaging efficiency and avoiding motion artifacts caused by multi-frame acquisition. While ensuring the directional contrast required for polarization imaging, this application embodiment supports parallel acquisition of multi-spectral bands within a local area, providing a high-quality data foundation for color temperature detection, color restoration, and material identification. Furthermore, the image sensor in this application embodiment has high integration and a compact structure, making it suitable for applications with stringent requirements for size and real-time performance, such as mobile terminals and autonomous driving. It effectively solves the technical problems of large size and high cost of traditional discrete systems and the limited functionality and incomplete spectral information of existing integrated sensors. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is one of the schematic diagrams of the image sensor pixel array in the embodiments of this application; Figure 2 This is a second schematic diagram of the structure of the image sensor pixel array according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the polarization pixel in an embodiment of this application.

[0019] Icons: 100 - Pixel array; 101 - Functional unit; 102 - Pixel subunit; 103 - Polarized pixel; 104 - Spectral pixel; 105 - Micro polarizer; 106 - First infrared filter layer; 107 - Second photosensitive pixel layer; 108 - Pixel circuit layer; 109 - Microlens. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] Traditional imaging devices typically employ a discrete design, acquiring polarization and spectral images separately through multiple independent sensors or complex optical switching devices. This approach suffers from drawbacks such as large size, high cost, and difficulty in miniaturization, making it unsuitable for mobile devices. Furthermore, current technologies cannot simultaneously acquire polarization and spectral information in a single exposure, limiting the imaging device's ability to operate in complex environments.

[0023] To address the aforementioned technical problems, embodiments of this application provide an image sensor, an imaging device, and an imaging method.

[0024] Reference Figure 1 , Figure 2 As shown, the image sensor provided in this application embodiment includes a pixel array 100, which includes a plurality of functional units 101 arranged in an array; each functional unit 101 includes at least two pixel sub-units 102; each pixel sub-unit 102 includes at least two polarized pixels 103 and at least one spectral pixel 104, the spectral pixel 104 including a first photosensitive pixel layer and a narrowband filter layer; wherein, in each functional unit 101, the narrowband filter layer of the spectral pixel 104 corresponding to the at least two pixel sub-units 102 has a different center wavelength.

[0025] It should be noted that, in this embodiment of the application, the hierarchical structure design of the pixel array 100 enables the synchronous acquisition of polarization and spectral information in a single exposure.

[0026] Specifically, the pixel array 100 of the image sensor is composed of multiple pixel sub-units 102 arranged in an array. Each pixel sub-unit 102 integrates at least two polarized pixels 103 with different polarization directions and at least one spectral pixel 104 with a narrowband filter layer, which is used to simultaneously acquire multi-directional polarized light intensity and spectral information of a specific band in a local area.

[0027] It should be noted that the narrowband filter layer in this application embodiment includes a color filter and a metasurface structure; high-precision wavelength selection can be achieved through the color filter or the metasurface structure.

[0028] Structurally, multiple adjacent pixel sub-units 102 further form functional units 101. In each functional unit 101, the narrowband filter layer of the spectral pixel 104 corresponding to at least two pixel sub-units 102 has a different center wavelength, thereby covering multiple spectral bands at the functional unit 101 level and realizing parallel capture of multispectral light intensity of the target scene. During operation, the sensor can obtain raw image data containing spatial, polarization, and spectral dimensions through a single exposure, and reconstruct polarization and spectral images respectively, thereby supporting multimodal applications such as color temperature detection and polarization imaging. This effectively solves the problems of large size and inability to simultaneously image in traditional discrete systems, improving imaging efficiency and environmental adaptability.

[0029] In terms of performance, the image sensor provided in this application embodiment achieves synchronous acquisition of polarization and spectral information in a single exposure by coordinating the layout of polarization pixels 103 and multi-band spectral pixels 104 to construct a hierarchical structure of pixel sub-units 102 and functional units 101. This significantly improves imaging efficiency and avoids motion artifacts caused by multi-frame acquisition. While ensuring the directional contrast required for polarization imaging, this application embodiment supports parallel acquisition of multi-spectral bands in local areas, providing a high-quality data foundation for color temperature detection, color restoration, and material recognition. In addition, the image sensor in this application embodiment has high integration and a compact structure, making it suitable for applications with stringent requirements for size and real-time performance, such as mobile terminals and intelligent driving. It effectively solves the technical problems of large size and high cost of traditional discrete systems and the limited functionality and incomplete spectral information of existing integrated sensors.

[0030] As an optional implementation, each polarized pixel 103 includes a second photosensitive pixel layer 107 and a micro-polarizer 105, the micro-polarizer 105 having a preset linear polarization transmission direction; wherein, in each pixel sub-unit 102, the polarization transmission directions of the micro-polarizers 105 of at least two polarized pixels 103 are at an angle to achieve multi-directional sampling of the polarization state of the incident light.

[0031] It should be noted that when three or more polarized pixels 103 are set inside a pixel subunit 102, the polarization transmission direction of the micro-polarizers 105 of some polarized pixels 103 can be the same, or the polarization transmission direction of the micro-polarizers 105 of each polarized pixel 103 can be different.

[0032] It should be noted that the demodulation accuracy of polarization information is closely related to the angular difference between the sampled polarization directions. When the angle between the two polarization directions is too small (e.g., close to 0°), the light intensity difference between different polarization channels is weak and easily affected by sensor noise, dark current, and quantization error, resulting in large deviations in the calculated degree of polarization and polarization angle, thus reducing imaging accuracy.

[0033] Therefore, in each pixel sub-unit 102, the polarization transmission directions of the micro-polarizers 105 of at least two polarized pixels 103 need to have a certain angle. Increasing the angle helps to improve the polarization degree and the calculation accuracy of the polarization angle, thereby improving the accuracy of imaging.

[0034] For example, in the three polarized pixels included in the pixel subunit 102, the polarization transmission direction of the micro-polarizer 105 of the first polarized pixel 103 is 0°; the polarization transmission direction of the micro-polarizer 105 of the second polarized pixel 103 is 45°; and the polarization transmission direction of the micro-polarizer 105 of the third polarized pixel 103 is 90°.

[0035] For example, in each pixel subunit 102, the polarization transmission direction angle of the micro-polarizers 105 on at least two polarized pixels 103 is greater than 60°.

[0036] It should be noted that in this embodiment, the included angle is set to be greater than 60°, especially when it is close to orthogonal. This maximizes the signal contrast between different polarization channels within a common incident angle range, thereby significantly improving the signal-to-noise ratio and robustness of polarization parameter back-calculation. In other words, a near-orthogonal angle between the polarization transmission directions increases the difference in light intensity in different directions, making the calculation process less sensitive to interference signals. This reduces the errors in the calculated polarization angle and degree of polarization, effectively improving imaging accuracy and reliability. Furthermore, a larger included angle helps simplify the complexity of subsequent image processing algorithms and improves real-time performance.

[0037] Therefore, the embodiments of this application limit the polarization transmission direction angle to be greater than 60° to ensure that the sensor can still stably and accurately acquire high-quality polarization images under complex lighting conditions, so as to meet the application requirements of three-dimensional imaging, anti-reflection, material identification and other applications.

[0038] As an optional implementation, each functional unit 101 includes a plurality of pixel sub-units 102 distributed in an N*N array; wherein N is greater than or equal to 2.

[0039] For example, each functional unit 101 may include nine pixel subunits 102 arranged in a 3x3 array. Alternatively, each functional unit may include sixteen pixel subunits 102 arranged in a 4x4 array, as needed. No particular limitation is made, and those skilled in the art can configure it as required.

[0040] For example, refer to Figure 2 As shown, each pixel subunit 102 includes three polarization pixels 103 and one spectral pixel 104. The three polarization pixels and one spectral pixel 104 are close to each other and arranged in a 2*2 array.

[0041] For example, each pixel subunit 102 includes two polarization pixels 103 and two spectral pixels 104, arranged in a 2*2 array. When there are two or more spectral pixels 104 in the pixel subunit 102, the corresponding narrowband filter layers have the same center wavelength or different center wavelengths.

[0042] It should be noted that the pixel subunit 102 may also include multiple spectral pixels 104 and multiple polarization pixels 103, arranged in a 3*3 array or a 2*3 array.

[0043] In other words, the pixel subunit 102 in this embodiment of the application includes a plurality of spectral pixels 104 and a plurality of polarization pixels 103 arranged in an m*n array, where m is greater than or equal to 2 and n is greater than or equal to 2. The values ​​of m and n can be equal or unequal.

[0044] This application does not impose a specific limitation on the number of spectral pixels 104 and polarization pixels 103 in the pixel sub-unit 102, and those skilled in the art can set them as needed. The center wavelengths of the spectral pixels 104 in the same pixel sub-unit 102 can be the same or different.

[0045] For example, the pixel subunit 102 includes five polarization pixels 103 and one spectral pixel 104, with the six pixels arranged in a 2*3 array.

[0046] It should be noted that when there are many spectral pixels 104 in pixel subunit 102 (many is defined as more than one spectral pixel 104 in pixel subunit 102), the following two cases apply when each pixel subunit 102 contains two or more spectral pixels 104: Firstly, if at least two spectral pixels 104 acquire different center wavelengths, it means that spectral information of two bands can be acquired within the same area, such as corresponding to green light and red light respectively. Therefore, dual-band synchronous spectral sensing can be achieved in a local area, which is beneficial for rapid color temperature estimation or color correction.

[0047] Secondly, if the center wavelengths acquired by at least two spectral pixels 104 are the same, it is equivalent to double sampling of the same band. By averaging the output of the two pixels, the signal-to-noise ratio of the band can be improved, achieving redundant design and enhancing system robustness.

[0048] It should be noted that when only one spectral pixel 104 is set in the pixel sub-unit 102, the pixel sub-unit 102 has a stronger polarization information acquisition capability. Multiple polarization directions are distributed in the same small area, reducing the inconsistency between directions caused by scene movement or lighting changes. Within a limited pixel area, the integrity of polarization sampling is prioritized, while the spectral dimension is introduced, which is a lightweight multimodal fusion design.

[0049] Although there is only one spectral pixel 104 in the pixel subunit 102, it provides key wavelength prior information, which can be used for color temperature detection, white balance guidance, polarization image denoising and other technical directions.

[0050] In addition, compared to the design of integrating multiple different narrowband filter layers in pixel sub-units 102, the single-spectrum pixel 104 solution has lower requirements for the alignment accuracy of narrowband filter layer micromachining and higher yield.

[0051] In other words, when a specific application scenario focuses more on color and spectral information, aims to achieve high-quality color temperature detection or multispectral analysis, and can tolerate slightly weaker polarization imaging capabilities, one or more spectral pixels 104 can be set in the pixel sub-unit 102. When a specific application scenario requires high-precision polarization imaging as its core objective, such as 3D reconstruction, 3D recognition, or material recognition, and spectral information is only used for auxiliary correction or environmental perception, then one spectral pixel 104 can be arranged in the pixel sub-unit 102.

[0052] Reference Figure 3 As shown, as an optional implementation, a first infrared filter layer 106 is provided on the polarization pixel 103.

[0053] It should be noted that visible light signals are weak under external conditions such as low light, smoke, rain, and snow, while infrared light, such as 850nm or 940nm, has stronger penetrating power and can be used in conjunction with active infrared illumination (such as infrared fill lights or lasers) to achieve all-weather imaging.

[0054] This embodiment integrates a first infrared filter layer 106 on the polarization pixel 103, allowing only infrared light of specific wavelengths to pass through the sensor, suppressing interference from visible light and other wavelength noise, and significantly improving the signal-to-noise ratio and imaging contrast. Simultaneously, infrared polarization information is more sensitive to the reflectivity of object surface materials (such as metal, plastic, and glass), helping to improve the accuracy of applications such as liveness detection, de-reflection, and 3D reconstruction. Therefore, this embodiment expands the applicability of image sensors in low-light and complex environments, enhancing the practicality of polarization imaging.

[0055] When the image sensor of this embodiment is working, incident light passes through the lens and illuminates the pixel array 100. Each polarized pixel 103 in the smallest repeating unit, due to its surface-integrated micro-polarizer 105, only allows light with a specific polarization direction to pass through, thus sampling the polarization state of the light field. The polarized pixels 103 equipped with a first infrared filter layer 106 further filter out infrared light of a specific wavelength, ensuring high-quality polarization signals can still be acquired in low-visible-light environments. Simultaneously, the spectral pixels 104 collect light intensity information of a specific wavelength through their narrowband filter layer. The image sensor simultaneously acquires the original mosaic image containing polarization and spectral information in a single exposure. Subsequent de-mosaic algorithms and image reconstruction processes reconstruct multi-directional polarized images and multi-channel spectral images, respectively. The scene color temperature can be calculated based on the spectral image, and polarization degree and polarization angle analysis and three-dimensional imaging can be performed based on the multi-polarized image.

[0056] Reference Figure 3 As shown, the micro-polarizer 105 of the polarization pixel 103 and the first infrared filter layer 106 are stacked on the second photosensitive pixel layer 107.

[0057] It should be noted that the first infrared filter layer 106 can selectively transmit near-infrared light and block visible light, so that the polarization pixel 103 is only sensitive to infrared light in a specific band, thereby avoiding visible light interference and improving the signal-to-noise ratio of infrared imaging. At the same time, combined with the micro polarizer 105, this structure can achieve accurate perception of the polarization state of near-infrared light, support infrared polarization imaging function, and enhance the image sensor's ability to identify targets and acquire multi-dimensional optical information synchronously in complex lighting environments.

[0058] Furthermore, the polarization pixel 103 also includes a pixel circuit layer 108; the pixel circuit layer 108 can be disposed on the backlight side of the second photosensitive pixel layer 107, or it can be disposed between the second photosensitive pixel layer 107 and the micro polarizer 105. In addition, a microlens 109 can be disposed on the first infrared filter layer 106 as needed.

[0059] It should be noted that the position of the first infrared filter layer 106 is not limited. The first infrared filter layer 106 can be disposed above the micro polarizer 105; the first infrared filter layer 106 can also be disposed between the micro polarizer 105 and the second photosensitive pixel layer 107; or the first infrared filter layer 106 can be disposed above the microlens 109; or the first infrared filter layer 106 can be disposed between the second photosensitive pixel layer 107 and the pixel circuit layer 108.

[0060] As an optional implementation, each functional unit also includes at least one infrared pixel, which includes a third photosensitive pixel layer and a second infrared filter layer.

[0061] It should be noted that the setting of infrared pixels expands the image sensor's ability to perceive near-infrared light in this embodiment, enabling it to achieve clear imaging not only in low-light or nighttime environments, but also for material composition analysis, biometric identification (such as liveness detection), and ambient light source judgment, thereby improving the accuracy of automatic white balance and color reproduction. At the same time, combined with polarization pixels 103 and spectral pixels 104, infrared pixels support the simultaneous acquisition of multi-dimensional optical information in a single exposure, enhancing the system's environmental perception capability in complex scenes.

[0062] As an optional implementation, in functional unit 101, the passbands of the narrowband filter layers of multiple spectral pixels 104 are adjacent and partially overlap in the visible light band, so as to achieve continuous coverage of the visible light band.

[0063] The embodiments of this application can achieve continuous and complete coverage of the visible spectrum, thereby improving the integrity of spectral information and the accuracy of color reproduction.

[0064] It should be noted that the visible light band is approximately 380nm to 780nm, and different wavelengths within this band correspond to different color perceptions. If there are large gaps between the spectral channels formed by the spectral pixels 104, it will lead to the loss of key spectral features, affecting the accuracy of color temperature calculation, white balance, and material recognition. By making the passbands of each narrowband filter layer adjacent and partially overlapping, spectral blind spots can be effectively avoided, ensuring that the spectral energy of the target scene is fully sampled across the entire visible light range.

[0065] It should be noted that a spectral scanning-like sampling mechanism is constructed by utilizing narrow-band filter layers with adjacent and partially overlapping passbands on multiple spectral pixels 104 within functional unit 101. When light is incident, different wavelength components are selectively transmitted by filter layers with different center wavelengths. Since each passband is continuously distributed in the visible light range and has overlapping regions, a smooth transition response coverage is formed between adjacent spectral channels, allowing the energy distribution of the entire visible spectrum to be captured segmentally and continuously. By performing inversion processing on these overlapping sampled light intensity data (such as least-squares fitting or spectral reconstruction algorithms based on prior libraries), the complete spectral reflectance of the target scene can be reconstructed with high accuracy, thereby achieving accurate color temperature detection, color correction, and spectral image generation.

[0066] The embodiments of this application achieve efficient perception of continuous spectrum without increasing the number of exposures, and integrate functions such as color temperature detection on the basis of polarization imaging.

[0067] Unlike the above implementation, in functional unit 101, the center wavelengths of the narrowband filter layers of multiple spectral pixels 104 are arranged at intervals in the visible light band to achieve segmented sampling of the visible spectrum.

[0068] It should be noted that, unlike the continuous coverage scheme, this embodiment of the application adopts a narrowband filter layer design with spacing in functional unit 101. That is, the center wavelengths of the narrowband filter layers on multiple spectral pixels 104 are distributed at certain intervals in the visible light band, rather than being continuously adjacent. This structure realizes segmented sampling of the visible spectrum. Its core principle is: by selecting several representative bands as "spectral sampling points", the key features of the target spectral energy are captured, thereby efficiently representing the overall spectral information with a smaller number of channels.

[0069] Since human vision and most imaging applications have a certain tolerance for color perception, it is not necessary to obtain a completely continuous spectral curve. Sampling only in key bands (such as blue, green, red, and yellow) is sufficient to reconstruct sufficiently accurate color information through algorithms. This approach significantly reduces system complexity and the amount of data processed while ensuring color reproduction accuracy. Furthermore, the spaced arrangement avoids excessive crosstalk between spectral channels, improving spectral resolution and making it suitable for applications with high requirements for cost, power consumption, and real-time performance.

[0070] For example, each functional unit 101 includes four spectral pixels 104, and the center wavelengths of the narrowband filter layers corresponding to the four spectral pixels 104 are 450nm, 530nm, 600nm, and 650nm, respectively. Among them, 450nm is the blue band, 530nm is the green band, 600nm is the orange band, and 650nm is the red band.

[0071] It should be noted that these four bands are arranged alternately within the visible light range, covering the main areas where the human eye is most sensitive. When light from the target scene enters the image sensor, each spectral pixel (104) records the light intensity value of the corresponding band. The subsequent image processing module uses the sampled discrete data, employing interpolation or color matrix lookup methods, to calculate the standard tristimulus values ​​(XYZ or RGB), thereby reconstructing the true color, calculating the correlated color temperature, or determining the material properties of the object.

[0072] The imaging method provided in this application is applied to the aforementioned image sensor, and the method includes: A polarization image corresponding to at least two polarization directions is obtained based on the first light signal received from the target scene by the polarization pixel; a multi-channel spectral image is obtained based on the second light signal received from the target scene by the spectral pixel.

[0073] It should be noted that traditional methods require multiple exposures or time-division switching to acquire polarization and spectral data, making them susceptible to dynamic changes in the scene. The method provided in this application can simultaneously capture spatial, polarization, and spectral information in a single exposure, avoiding motion artifacts and ensuring the temporal and spatial consistency of multimodal data, greatly improving the accuracy and reliability of imaging. Regardless of whether under strong light, weak light, or complex lighting conditions, this application embodiment can simultaneously acquire polarization and spectral information, enhancing the system's stability and applicability in various practical scenarios.

[0074] It should be noted that, compared to discrete multi-sensor systems, the embodiments of this application only require one image sensor to complete multimodal imaging, which simplifies the optical structure, reduces size and power consumption, and makes it easier to integrate into portable or embedded devices.

[0075] The embodiments of this application do not require mechanical scanning or repeated acquisition, which significantly shortens the data acquisition cycle and meets the needs of autonomous driving, robot vision, mobile terminals and other applications for high-speed, real-time imaging.

[0076] As an optional implementation, the method further includes: calculating the color temperature information of the target scene by means of the light intensity distribution of different bands in the multi-channel spectral image.

[0077] It's important to note that color temperature is an indicator of the color of a light source, usually expressed in Kelvin (K). Light sources with different color temperatures have different spectral power distributions. For example, low color temperature light sources (such as 2700K) are biased towards red, while high color temperature light sources (such as 6500K) are biased towards blue. By analyzing the light intensity distribution of a target scene across multiple spectral bands, the spectral power distribution of the light source can be deduced, and its color temperature can then be calculated.

[0078] The spectral pixels 104 in the image sensor acquire light intensity information in specific wavelength bands through a narrowband filter layer, forming a multi-channel spectral image. These wavelength bands can cover key wavelengths within the visible light spectrum, thereby providing sampling points for the spectral power distribution of reflected light from the target scene. It should be noted that common color temperature calculations include those based on tristimulus values ​​and lookup tables; specific calculation methods can be performed by those skilled in the art based on existing technologies.

[0079] As an optional implementation, the method further includes: correcting the original image based on the color temperature information of the target scene to obtain a corrected image; wherein the original image is an image displayed on the screen of an electronic device or an image captured by the camera of an electronic device.

[0080] It should be noted that white balance correction can be performed on the image displayed on the screen or the image captured by the camera based on the color temperature information of the target scene. Specifically, the color temperature of the target scene, either overall or locally, is calculated based on a multi-channel spectral image. Then, the corresponding white balance gain coefficient is determined by looking up a table or through a mapping function. This gain is applied to the image displayed on the screen or the image captured by the camera, and the total light intensity component is modified accordingly to compensate for the gain of pixels in different areas.

[0081] It should be noted that during the correction process, the weight parameters in the fusion algorithm can be adaptively adjusted based on the color temperature information. For example, in high color temperature (cold light, >6000K) environments, atmospheric scattering is strong, and polarization signals are easily affected by haze. In this case, the fusion weight of high-frequency polarization details should be reduced, and the smoothing term should be enhanced to suppress noise. On the other hand, in low color temperature (warm light, <4000K) environments, indoor lighting may cause strong specular reflections on glass or plastic surfaces. In this case, the differential weight of the orthogonal polarization direction image should be increased to enhance the de-reflection effect.

[0082] It should be noted that the type of light source under the current lighting conditions is identified based on the previously calculated color temperature (e.g., 3000K represents warm white light, and 6500K represents cool white light). Different light sources have different reflection characteristics in different polarization directions. For example, under warm white light, metal or glass may produce strong specular reflection, resulting in artifacts or noise in the polarized image.

[0083] It's important to note that if a low color temperature (warm light) is detected, it indicates more specular reflection and strong contrast changes. In this case, the de-reflection processing intensity should be increased to reduce noise caused by specular reflection. If a high color temperature (cool light) is detected, atmospheric scattering is stronger, which may lead to overall image blurring or increased noise. In this case, the smoothing filter should be enhanced to reduce high-frequency noise. During image processing, these noises caused by ambient light are reduced by adjusting the filter intensity or selecting appropriate filtering methods, such as mean filtering or Gaussian filtering.

[0084] For example, a calculated color temperature of 3000K indicates warm white light illumination. A suitable guided filter for removing specular reflections is selected; this filter effectively preserves edge details while removing noise in smooth areas. The guided filter is applied to the polarized image, and the filter parameters are adjusted to adapt to the current lighting conditions, thereby reducing artifacts caused by specular reflections and improving image quality.

[0085] The imaging device provided in this application includes: an infrared light source, a lens, and the aforementioned image sensor. The infrared light source is used to emit infrared polarized light toward the target scene; the light signal reflected from the target scene passes through the lens and then enters the image sensor.

[0086] When the imaging device of this embodiment is working, the infrared light source emits infrared light with a specific polarization direction towards the target scene. After being reflected by the object surface, the infrared polarized light carries the target's shape, material, and polarization characteristics information. It is then converged by the lens and incident on the pixel array 100 of the image sensor. The polarization pixels 103 and spectral pixels 104 in the image sensor simultaneously capture the reflected light intensity of different polarization directions and multiple spectral bands in a single exposure. Through system processing, multi-channel spectral images and multi-polarization images can be obtained. Based on the spectral images, the scene color temperature can be calculated. Combined with the polarization images, white balance correction, noise suppression, and other optimization processing can be performed. At least two polarization images can be used for polarization three-dimensional imaging processing to retrieve the depth map of the target scene, thereby achieving efficient fusion perception of spectral, polarization, and three-dimensional information in a single exposure.

[0087] This application embodiment integrates an infrared light source with an image sensor capable of polarization and spectral fusion sensing, enabling simultaneous acquisition of multimodal information of a target scene in a single exposure. It can efficiently acquire spectral, polarization, and three-dimensional depth information. The imaging device has a compact structure, does not require multi-frame synthesis, avoids motion artifacts, supports all-weather operation, and significantly improves imaging efficiency and environmental adaptability. It is especially suitable for intelligent vision applications such as liveness detection, dereflection, and high-precision three-dimensional reconstruction.

[0088] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image sensor, characterized in that, The system includes a pixel array (100), which comprises a plurality of array-arranged functional units (101); each functional unit (101) includes at least two pixel subunits (102); each pixel subunit (102) includes at least two polarized pixels (103) and at least one spectral pixel (104), the spectral pixel (104) including a first photosensitive pixel layer and a narrowband filter layer; wherein, in each functional unit (101), the narrowband filter layer of the spectral pixel (104) corresponding to the at least two pixel subunits (102) has a different center wavelength.

2. The image sensor according to claim 1, characterized in that, The polarized pixel (103) includes a second photosensitive pixel layer (107) and a micro-polarizer (105), wherein the micro-polarizer (105) has a preset linear polarization transmission direction; wherein, in each pixel sub-unit (102), the polarization transmission directions of the micro-polarizers (105) of at least two polarized pixels (103) are at an angle to each other.

3. The image sensor according to claim 2, characterized in that, In each pixel subunit (102), the polarization transmission direction angle of the micro-polarizers (105) on at least two polarized pixels (103) is greater than 60°.

4. The image sensor according to claim 2, characterized in that, The pixel subunit (102) includes three polarization pixels (103) arranged in a 2*2 array and one spectral pixel (104).

5. The image sensor according to claim 4, characterized in that, In the three polarized pixels (103) included in the pixel subunit (102), the polarization transmission direction of the micro-polarizer (105) of the first polarized pixel (103) is at an angle of 45° with the polarization transmission direction of the micro-polarizer (105) of the second polarized pixel (103); the polarization transmission direction of the micro-polarizer (105) of the third polarized pixel (103) is orthogonal to the polarization transmission direction of the first polarized pixel (103).

6. The image sensor according to any one of claims 1-5, characterized in that, The polarization pixel (103) also includes a first infrared filter layer (106).

7. The image sensor according to any one of claims 1-5, characterized in that, Each of the functional units (101) further includes at least one infrared pixel, the infrared pixel comprising a third photosensitive pixel layer and a second infrared filter layer.

8. An imaging method, characterized in that, Applied to the image sensor according to any one of claims 1-7, the method comprises: Based on the first light signal received from the target scene by the polarization pixel, a polarization image corresponding to at least two polarization directions is obtained; A multi-channel spectral image is obtained based on the second light signal received from the target scene by the spectral pixels.

9. The imaging method according to claim 8, characterized in that, The method further includes: calculating the color temperature information of the target scene by means of the light intensity distribution of different bands in the multi-channel spectral image.

10. The imaging method according to claim 9, characterized in that, The method further includes: correcting the original image based on the color temperature information of the target scene to obtain a corrected image; wherein the original image is an image displayed on the screen of an electronic device or an image captured by the camera of an electronic device.

11. An imaging device, characterized in that, It includes an infrared light source, a lens, and an image sensor as described in any one of claims 1-7; the infrared light source is used to emit infrared polarized light toward the target scene; the light signal reflected from the target scene passes through the lens and enters the image sensor.

Images