Optical filter array, imaging device and vehicle

Abstract

The invention provides an optical filter array, an imaging device and a vehicle, and relates to the technical field of optical device preparation, the optical filter array comprises K unit optical filters, the K unit optical filters comprise an R optical filter, a G optical filter and a B optical filter, the R optical filter is used for filtering a red color component, the G optical filter is used for filtering a green color component, and the B optical filter is used for filtering a blue color component; the optical filter B is used for filtering blue color components; the K unit optical filters further comprise a first unit optical filter, the first unit optical filter is provided with a polarizer and / or the first unit optical filter is provided with an optical filter, and K is an integer greater than or equal to 4. On the basis that the processing cost and difficulty of the optical filter array are low, in the processing process, only the polarizer and / or the optical filter need to be arranged on the first unit optical filter, it can be guaranteed that the resolution ratios of RGB images obtained before and after the optical filter array is processed are the same, and multi-dimensional optical information with high precision can be obtained on the basis of the optical filter array.

Description

Technical Field

[0001] The present application relates to the field of optical technology, and more specifically, to a filter array, an imaging device and a vehicle. Background Art

[0002] With the development of technologies such as industrial vision and autonomous driving, the information obtained by conventional RGB cameras is difficult to meet the needs of the system. In some edge scenarios, the perception system needs to obtain multi-dimensional optical information to make up for the defects. From the perspective of perception information acquisition, the perception system can obtain multi-dimensional optical information of the current scene through specific filters. However, filters processed based on the existing imaging chip coating method are usually expensive and will greatly reduce the single-pixel spatial resolution of the RGB sensor carried by the filter, resulting in insufficient accuracy of the obtained multi-dimensional optical information.

[0003] Therefore, how to process the filter at a lower cost so that high-precision multi-dimensional optical information can be obtained through the filter is an issue that needs to be solved urgently. Summary of the invention

[0004] The present application provides a filter array, an imaging device, a filter processing method and a vehicle. The filter array has a low processing cost, and multi-dimensional optical information with high accuracy can be obtained through the filter.

[0005] In a first aspect, a filter array is provided, comprising: K unit filters, the K unit filters comprising an R filter, a G filter and a B filter, wherein the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component; the K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

[0006] It should be understood that the above-mentioned filter array includes an RGB color array, and conventional filters are usually arranged based on the RGB color array, so the above-mentioned filter array can be obtained by processing based on conventional filters, so the processing difficulty and cost of obtaining the above-mentioned filter array are relatively low.

[0007] For example, the array form of the above-mentioned filter array can be simply referred to as RGB form. Based on the above-mentioned filter array, a filter array of RGGB form can also be constructed. Of course, the K unit filters can also include a Y filter, or the above-mentioned G filter can be replaced with a Y filter, for example, to construct a filter array of RYB form.

[0008] For example, when the array form of the above-mentioned filter array is RGB form or RYB form, the above-mentioned K should be greater than or equal to 4; when the array form of the above-mentioned filter array is RGGB form or RYYB form, the above-mentioned K should be greater than or equal to 5.

[0009] For example, the array form of the above filter array can also be other forms, such as RGGGB, RRGGB, etc., which is not limited in the embodiments of the present application.

[0010] For example, the filter array is applied to an imaging device. In the imaging device, the polarizer and the filter may be directly disposed on the first unit filter or between the first unit filter and a photosensitive element of the imaging device.

[0011] Based on the above technical solution, the processing cost of the above filter array is relatively low. During the processing, it is only necessary to set a polarizer and / or a filter on the first unit filter. This not only ensures that the RGB image resolution obtained by the filter array before and after processing is the same, but also based on the filter array, it is possible to obtain multi-dimensional optical information with higher accuracy.

[0012] In combination with the first aspect, in some implementations of the first aspect, the first unit filter is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize the light corresponding to the full color component.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the first unit filter is provided with a polarizer, and the first unit filter is used to filter a single color component, and the polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component or a blue color component.

[0014] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer corresponding to the first unit filter of the above filter array, a full-color polarization map or a monochrome polarization map can be obtained, so that the filter including the filter array can be applied to scenes such as glare / reflection elimination, stress detection and contrast enhancement.

[0015] In combination with the first aspect, in some implementations of the first aspect, the first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

[0016] By way of example, the first partial band may be a narrowband band, and correspondingly, a narrowband spectral image may be acquired through the filter array of the filter setting; the first partial band may also be a broadband band, and correspondingly, a broadband spectral image may be acquired through the filter array of the filter setting.

[0017] By way of example, the first unit filter may also be provided with the polarizer.

[0018] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, a corresponding filter is set corresponding to the first unit filter in the filter array, so a spectral image can also be obtained based on the filter array. Since the first unit filter can also be set with a polarizer, a polarization image can also be obtained based on the filter array.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the above-mentioned K unit filters also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second partial band.

[0020] By way of example, the K unit filters mentioned above may also include a third unit filter, which is also used to filter the full color component. The third unit filter is provided with a filter, and the filter provided in the third unit filter can be used to filter the color component of the third partial band, and of course, can be used to filter the color component of the first partial band or the second partial band mentioned above.

[0021] For example, the K unit filters may include more than three unit filters for filtering all color components, and these unit filters are respectively provided with filters, and the color components of partial bands filtered by these filters may be the same, different from each other, or partially the same.

[0022] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, the above filter array has strong scalability, and multiple filters can be set in one filter array, so multi-spectral images or hyperspectral images can also be obtained based on the filter array.

[0023] In combination with the first aspect, in certain implementations of the first aspect, when the first unit filter is provided with a polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with a filter, the filter array is used in a second scenario; or, when the first unit filter is provided with a polarizer and a filter, the filter array is used in a third scenario.

[0024] For example, the first scene may be a scene that is prone to glare, in which the polarization direction of the polarizer disposed on the first unit filter is associated with the polarization angle of the light that causes image overexposure.

[0025] For example, the second scene may be a material recognition scene, a target recognition scene, etc., in which the color component of the partial band that can be filtered by the filter set in the first unit filter is associated with the color information in the scene. In addition, the color component of the partial band that can be filtered by the filter set in the first unit filter can also be associated with the demand information of the scene, for example, night imaging demand, thermal imaging demand, and the corresponding partial band that can be filtered by the filter can be a band other than visible light.

[0026] For example, the third scene may be a mixed scene, for example, a scene that is prone to glare and requires material identification. The polarization direction of the polarizer and the partial band that the filter can filter are described in detail in the corresponding description of the above example.

[0027] Based on the above technical solution, the above filter array can have a variety of polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0028] In a second aspect, an imaging device is provided, which includes a filter array and a photosensitive element in any possible implementation of the first aspect.

[0029] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two, and when polarizers are provided in at least two filter arrays, the polarization directions of the polarizers in at least two filter arrays are the same or different.

[0030] For example, the at least two filter arrays may be arranged in the same filter.

[0031] For example, the polarization directions of the polarizers in at least two filter arrays may be all the same, or all different, or may be partially the same and partially different.

[0032] Based on the above technical solution, the imaging device can obtain polarization images corresponding to multiple polarization directions, and can also obtain RGB images.

[0033] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two, and when filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different.

[0034] Based on the above technical solution, the imaging device can acquire multispectral images or hyperspectral images, and can also acquire RGB images.

[0035] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two, and when polarizers and filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different.

[0036] Based on the above technical solution, not only RGB images can be obtained, but also polarization images and multispectral images or hyperspectral images can be obtained. Based on these image information, the full optical function for digital image processing can be constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0037] In a third aspect, a vehicle is provided, comprising an imaging device as in any possible implementation of the second aspect.

[0038] In a fourth aspect, a method for processing a filter is provided, the method comprising: obtaining a filter to be processed, the filter to be processed comprising a filter array, the filter array comprising K unit filters, the K unit filters comprising an R filter, a G filter and a B filter, the R filter being used to filter a red color component, the G filter being used to filter a green color component, the B filter being used to filter a blue color component, and the K unit filters also comprising a first unit filter; processing the filter array to obtain a target filter, comprising: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, where K is an integer greater than or equal to 4.

[0039] For example, a filter may include at least one filter array, so processing the filter means processing each filter array in the filter.

[0040] Based on the above technical solution, the filter can be processed at a relatively low cost, and while ensuring that the resolution of the RGB image obtained through the processed filter remains unchanged, the corresponding polarization image information and / or spectral image information can also be obtained.

[0041] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter the full color component, and the polarizer provided for the first unit filter is used to polarize light corresponding to the full color component.

[0042] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter a single color component, and the polarizer set for the first unit filter is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component, or a blue color component.

[0043] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer corresponding to the first unit filter of the above filter array, a full-color polarization map or a monochrome polarization map can be obtained, so that the filter including the filter array can be applied to scenes such as glare / reflection elimination, stress detection and contrast enhancement.

[0044] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter the full color component, and the filter set for the first unit filter is used to filter the color component of the first partial band.

[0045] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, the corresponding filter is set corresponding to the first unit filter in the filter array, so a spectral image can also be obtained based on the filter array.

[0046] In combination with the fourth aspect, in certain implementations of the fourth aspect, the K unit filters also include a second unit filter, a filter is set for the second unit filter, and the filter set in the second unit filter is used to filter the color component of the second partial band.

[0047] Based on the above technical solution, the color arrangement form (RGB form) of the filter array has not changed compared to before processing, so the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, the above filter array has strong scalability, and multiple filters can be set in one filter array, so multi-spectral images or hyperspectral images can also be obtained based on the filter array.

[0048] In combination with the fourth aspect, in certain implementations of the fourth aspect, first scene information is obtained, where the first scene information includes light information of the first scene, and the light information is used to indicate the polarization angle of light that causes image overexposure; based on the light information, a polarizer with a polarization direction of the first polarization direction is set for the first unit filter.

[0049] In combination with the fourth aspect, in certain implementations of the fourth aspect, second scene information is obtained, the second scene information including demand information and color element information of the second scene; based on the demand information and the color element information, a filter for filtering the color components of the target band is set for the first unit filter, and the color components of the target band are associated with the demand information and the color element information.

[0050] For example, the second scene information may also include the light information, so the polarization direction of the polarizer and the color component of the partial band filtered by the filter may be determined based on the light information, the requirement information and the color element information. Then, the polarizer and filter determined above are set for each first unit filter in the filter array. Of course, in the case where the filter array includes a plurality of unit filters for filtering all color components, the polarization directions of the plurality of polarizers and the color components of the partial band filtered by the plurality of filters may also be determined based on the light information, the requirement information and the color element information.

[0051] Based on the above technical solution, the above filter array can have a variety of polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0052] In a fifth aspect, a filter processing device is provided, the device comprising: an acquisition unit, used to acquire a filter to be processed, the filter to be processed comprising a filter array, the filter array comprising K unit filters, the K unit filters comprising an R filter, a G filter and a B filter, the R filter being used to filter a red color component, the G filter being used to filter a green color component, the B filter being used to filter a blue color component, and the K unit filters also comprising a first unit filter; an operation unit, used to process the filter array to obtain a target filter, comprising: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, K being an integer greater than or equal to 4.

[0053] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first unit filter is used to filter the full color component, and the polarizer provided for the first unit filter is used to polarize the light corresponding to the full color component.

[0054] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned first unit filter is used to filter a single color component, and the polarizer set for the first unit filter is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component, or a blue color component.

[0055] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first unit filter is used to filter all color components, and the filter set for the first unit filter is used to filter the color components of the first partial band.

[0056] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned K unit filters also include a second unit filter, a filter is set for the second unit filter, and the filter set in the second unit filter is used to filter the color component of the second part of the band.

[0057] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned operating unit is specifically used to: obtain first scene information, the first scene information including light information of the first scene, the light information being used to indicate the polarization angle of light that causes image overexposure; and according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter.

[0058] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned operation unit is specifically used to: obtain second scene information, the second scene information including demand information and color element information of the second scene; according to the demand information and the color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and the color element information.

[0059] In a sixth aspect, a filter processing device is provided, the device comprising: a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute a method in any possible implementation mode of the method design of the fourth aspect above.

[0060] In the seventh aspect, a chip system is provided, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to echo a signal from a memory of the electronic device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes a method in any possible implementation of the method design of the fourth aspect above.

[0061] In an eighth aspect, a computer-readable storage medium is provided, storing a computer program or instruction, wherein the computer program or instruction is used to implement the method in any possible implementation manner in the method design of the fourth aspect.

[0062] In a ninth aspect, a computer program product is provided. When the computer program code or instructions are executed on a computer, the computer executes a method in any possible implementation of the method design of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a schematic diagram of a filter for obtaining polarization image information;

[0064] Figure 2 is a schematic diagram of a filter array proposed in an embodiment of the present application;

[0065] Figure 3 It is a schematic flow chart of a method for processing an optical filter proposed in an embodiment of the present application;

[0066] Figure 4 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application;

[0067] Figure 5 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application;

[0068] Figure 6 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application;

[0069] Figure 7 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application;

[0070] Figure 8is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0071] Fig. 9 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0072] Fig.10 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application;

[0073] Fig.11 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application;

[0074] Fig.12 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application;

[0075] Fig.13 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application;

[0076] Fig.14 It is a schematic diagram of a filter processing device 1400 proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0078] The technical solution of the embodiment of the present application can be applied to an imaging system. In an existing imaging system, the corresponding optical information in the current scene is usually obtained through a filter. Taking a filter that can obtain polarized image information as an example, a polarizer can be set on each sub-sheet of the filter.

[0079] Figure 1 A schematic diagram of a filter used to obtain polarization image information.

[0080] Figure 1 Two groups of filter arrays based on the Bayer filter are shown, and in these two groups of filter arrays, every four sub-chips correspond to one color, that is, every four sub-chips correspond to one pixel of the RGB sensor, and the four sub-chips correspond to polarizers of four polarization directions respectively.

[0081] Taking sub-chip 1 as an example, according to the principle of digital image processing, before performing digital image processing on the target image obtained through sub-chip 1, it is necessary to obtain all target images with the same color and positive polarization direction as sub-chip 1. Then, among all the sub-chips for obtaining these target images, the more sub-chips there are between adjacent sub-chips, the lower the RGB image resolution of the corresponding RGB sensor. Taking sub-chip 2 adjacent to sub-chip 1 as an example, there are 3 sub-chips between sub-chip 1 and sub-chip 2.

[0082] However, if a polarizer is not provided above the Bayer filter and the RGB image is directly obtained through the Bayer filter, it is not necessary to divide the filter for each color. Therefore, the filter for each color corresponds to one pixel of the RGB sensor.

[0083] Taking red filter 1 as an example, according to the principle of digital image processing, before performing digital image processing on the target image obtained through red filter 1 (geometric parameters are equivalent to the above-mentioned sub-chip 1), it is necessary to obtain the target images of all red filters. Then, among all the red filters for obtaining these target images, the more filters are spaced between adjacent red filters, the lower the RGB image resolution of the corresponding RGB sensor. Taking red filter 2 adjacent to red filter 1 as an example, there is a green filter between red filter 1 and red filter 1.

[0084] It can be seen that although the above method can obtain polarization image information, it greatly reduces the RGB image resolution of the RGB sensor. It also changes the pixel color arrangement of the Bayer filter, that is, one sub-sheet corresponds to one pixel color in the Bayer filter before the change, and four sub-sheets correspond to one pixel color in the Bayer filter after the change, thereby reducing the number of RGB pixels of the RGB sensor. In addition, based on the above Figure 1 The filters shown also fail to obtain optical information such as spectral images.

[0085] In view of this, the embodiments of the present application propose a filter array, an imaging device, a filter processing method and a vehicle, which can process the filter at a relatively low cost so that the image acquired through the filter can be processed to extract relatively accurate multi-dimensional optical information, which includes RGB image information, polarization image information and spectral image information.

[0086] The present application embodiment proposes a filter array, Figure 2 It is a schematic diagram of a filter array proposed in an embodiment of the present application.

[0087] The filter array includes: K unit filters, the K unit filters include an R filter, a G filter and a B filter, wherein the R filter is used to filter the red color component, the G filter is used to filter the green color component, and the B filter is used to filter the blue color component; the K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

[0088] refer to Figure 2As shown, all filter arrays in this example include 4 unit filters, that is, the above K is equal to 4. The drawings of this application mark the unit filter used to filter the full color component as A, so Figure 2 The first unit filter is marked as A.

[0089] The first unit filter in the filter array 1 is provided with a polarizer, and the embodiment of the present application does not limit the polarization direction of the polarizer, so the polarization direction of the polarizer provided in the first unit filter may also be other directions, for example, the filter array 2. Based on the filters including the filter array 1 and the filter array 2, not only RGB images but also polarized images can be obtained.

[0090] The first unit filter in the filter array 3 is provided with a filter. Since the filter is used to filter the color components of a part of the wavelength band, the first unit filter and the filter can be combined to filter the color components of a part of the wavelength band. Figure 2 The first unit filter after the filter is set is marked as 1, and the number "1" is used to indicate that the filter is used to filter the color components of a specific partial band. The embodiment of the present application does not limit the color components of the partial band corresponding to the filter, so the filter set in the first unit filter can also be used to filter the color components of other partial bands. Based on the filter including the above-mentioned filter array 3, not only RGB images can be obtained, but also spectral images can be obtained.

[0091] The first unit filter in the filter array 4 is provided with not only a polarizer but also a filter. Based on the filters including the filter array 4, not only an RGB image but also a polarization image and a spectral image can be obtained.

[0092] In addition, before and after the polarizer and / or filter are set on the first unit filter, the color arrangement of the filter RGB of the filter array proposed in the embodiment of the present application will not change, so the resolution of the RGB image obtained based on the filter will not be reduced due to the polarizer and / or filter being set on the first unit filter.

[0093] Based on the above technical solution, the processing cost of the above filter array is relatively low. During the processing, it is only necessary to set a polarizer and / or a filter on the first unit filter. This can not only ensure that the RGB image resolution obtained by the filter array before and after processing is the same, but also based on the filter array, it is possible to obtain multi-dimensional optical information with high accuracy.

[0094] Correspondingly, the embodiment of the present application also proposes a method for processing a filter. Figure 3 It is a schematic flow chart of a method for processing an optical filter proposed in an embodiment of the present application.

[0095] S310: Obtain a filter to be processed, where the filter to be processed includes a filter array, where the filter array includes K unit filters, where the K unit filters include an R filter, a G filter, and a B filter, where the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component, and the K unit filters also include a first unit filter.

[0096] In some possible embodiments, the filter to be processed may be a Bayer filter, or the arrangement of the color filters of the K unit filters may correspond to the color arrangement of a conventional Bayer filter, and the embodiment of the present application does not limit the specific color arrangement of the Bayer filter.

[0097] S320: Processing the filter array to obtain a target filter, including:

[0098] Solution a: Provide a polarizer for the first unit filter; or,

[0099] Solution b: Setting a filter for the first unit filter; or,

[0100] Solution c: a polarizer is provided for the first unit filter, and a filter is provided for the first unit filter.

[0101] Since the optical filter is an important component of the imaging device, the above processing method may specifically be a processing method for the imaging device, and may also be understood as a part of the process for processing the imaging device.

[0102] It should be understood that the embodiment of the present application does not limit the order of setting the polarizer and the filter for the first unit filter in the above scheme c.

[0103] In some possible embodiments, the polarization direction of the polarizer may be any direction, and the optical filter may be a broadband filter or a narrowband filter.

[0104] Based on the above technical solution, the filter can be processed at a relatively low cost, and while ensuring that the resolution of the RGB image obtained through the processed filter remains unchanged, the corresponding polarization image information and / or spectral image information can also be obtained.

[0105] It should be understood that the above filter array includes a filter array processed based on the above solution a, solution b or solution c. The specific forms of the filter arrays processed based on these three solutions will be described below.

[0106] In some possible embodiments, taking the color arrangement of the Bayer filter as an example, four unit filters are usually used as a group of filter arrays, including filters of three colors: R filter, G filter and B filter. Based on these three colors of filters, a variety of arrangement forms can be combined in the filter array, for example, RGB form, RGGB form, etc. Of course, the above-mentioned G filter can also be replaced with a Y filter, which is used to filter the yellow color component, and then based on these three colors of filters, a variety of arrangement forms can be combined in the filter array, for example, RYB form, RYYB form. The above-mentioned three colors of filters can also be replaced with filters of other colors, which will not be repeated here.

[0107] In some possible embodiments, the filter array includes filters of the above three colors and also includes a first unit filter. Taking a filter array consisting of four unit filters as an example, when the arrangement form of the filter array is RGB, the first unit filter is used to filter all color components. When the arrangement form of the filter array is RGGB, the first unit filter is used for a single color component. In this example, the single color component is a green color component. Of course, when the filter array includes other color arrangements, the color component filtered by the first unit filter is the same as in the above example, that is, when there is no repeated color filter, the first unit filter is used for the full color component. When there is a repeated color filter, the first unit filter is used for a single color component. The single color component is the color component corresponding to the repeated color.

[0108] In some possible embodiments, the array form of the above-mentioned filter array may also be other forms, for example, RGGGB, RRGGB, etc., which is not limited in the embodiments of the present application.

[0109] Based on the above three solutions, the filter array and the corresponding filter described in the following embodiments can be obtained.

[0110] In some possible embodiments, the first unit filter in the filter array is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize the light corresponding to the full color component.

[0111] Figure 4 It is a schematic diagram of the structure of a filter proposed in an embodiment of the present application. Figure 4 The filter to be processed on the left refers to the filter without polarizer and / or filter, and has not been further processed. Figure 4The target filter on the right is the filter proposed in the embodiment of the present application, which includes multiple groups of filter arrays, and each group of filter arrays includes an R filter, a G filter, a B filter and a first unit filter.

[0112] refer to Figure 4 As shown, the drawings of this application mark the unit filter used to filter the full color component as A, so Figure 4 The first unit filter is marked as A.

[0113] In some possible embodiments, one filter may include two or more of the above filter arrays, and the polarization directions of the polarizers arranged in the first unit filters in each filter array may be different. Figure 4 As shown, the filter includes a first filter array, a second filter array, a third filter array, and a fourth filter array, wherein the first unit filter of the first filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter of the second filter array is provided with a polarizer with a polarization direction of the second polarization direction, the first unit filter of the third filter array is provided with a polarizer with a polarization direction of the third polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer with a polarization direction of the fourth polarization direction. Therefore, based on the filter, a full-color multi-directional polarization map can be obtained.

[0114] In some possible embodiments, the first polarization direction to the fourth polarization direction differ by 45°, for example, the first polarization direction is 0°, the second polarization direction is 45°, the third polarization direction is 90°, and the fourth polarization direction is 135°. Of course, the first polarization direction to the fourth polarization direction may also be polarization directions of other reasonable angles.

[0115] It should be understood that the RGB color arrangement of the filter array will not change due to the setting of the polarizer on the first unit filter, so before and after the polarizer is set, the resolution of the RGB image obtained by the filter including the filter array will not be reduced, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer, a full-color polarization image with a lower resolution can also be obtained. Although the resolution of the full-color polarization image is lower than the resolution of the RGB image, the full-color polarization image is sufficient to cope with most scenes such as glare / reflection elimination, stress detection, and contrast enhancement. Moreover, the resolution of the full-color polarization image obtained in this scheme is the same as the resolution of the monochrome polarization image obtained by the above-mentioned prior art scheme 2. Therefore, the resolution of the RGB image obtained based on the filter including the filter array will not be reduced after adding a polarizer to the filter array, that is, it can ensure that the number of RGB pixels of the RGB sensor is not reduced, and the polarization image can be obtained, and the processing cost can be effectively controlled.

[0116] Figure 5 It is a schematic diagram of the structure of another filter proposed in an embodiment of the present application. Figure 5 The filter to be processed on the left refers to the filter without polarizer and / or filter, and has not been further processed. Figure 5 The target filter on the right is the filter proposed in the embodiment of the present application.

[0117] Figure 5 The color format of each filter array shown is similar to Figure 4 Each set of filter arrays shown has the same color format.

[0118] In some possible embodiments, reference Figure 5 As shown, more than two filter arrays can be included, and the polarization directions of the polarizers set in the first unit filters in each filter array can be the same: the first unit filter of the first filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter of the second filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter of the third filter array is provided with a polarizer with a polarization direction of the first polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer with a polarization direction of the first polarization direction. Therefore, based on the filter, a full-color unidirectional polarization map can be obtained.

[0119] It should be understood that the polarization direction of the polarizer of the first unit filter of each group of filter arrays may also be other directions, and the present application does not limit this.

[0120] Based on the above Figure 5It can be seen from the filter shown that the RGB color arrangement of the filter array will not change due to the setting of the polarizer on the first unit filter, so before and after the polarizer is set, the resolution of the RGB image obtained by the filter including the filter array will not be reduced, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer, a high-resolution full-color unidirectional polarization map can be obtained, and the resolution of the full-color unidirectional polarization map is equal to the resolution of the RGB image, and even if the polarization map directly obtained through the target filter is a unidirectional polarization map, two full-color unidirectional polarization maps with the same angle (for example, the polarization angle is 135°) are subjected to full-color channel subtraction processing to obtain a full-color unidirectional polarization map of another polarization direction (that is, the polarization angle is 45°). Although the polarization map obtained based on the target filter can correspond to two polarization directions, the full-color unidirectional polarization map based on these two polarization directions is sufficient to cope with those specific scenes such as glare / reflection elimination, stress detection and contrast enhancement. Therefore, the resolution of the RGB image obtained based on the filter including the above-mentioned filter array will not be reduced after adding a polarizer to the filter array, that is, it can ensure that the number of RGB pixels of the RGB sensor is not reduced, and a polarized image with the same resolution as the RGB image can be obtained, and the processing cost can be effectively controlled.

[0121] In some possible embodiments, the first unit filter is provided with a polarizer, and the first unit filter is used to filter a single color component, and the polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component or a blue color component.

[0122] Figure 6 It is a schematic diagram of the structure of another filter proposed in an embodiment of the present application. Figure 6 The filter to be processed on the left refers to the filter without polarizer and / or filter, and has not been further processed. Figure 6 The target filter on the right is the filter proposed in the embodiment of the present application. The filter includes multiple groups of filter arrays, and each group of filter arrays includes 1 R filter, 2 G filters, and 1 B filter, wherein one of the 2 G filters can be used as the first unit filter, so the single color component filtered by the first unit filter is the green color component.

[0123] In some possible embodiments, Figure 6 The polarization directions of the polarizers of the first unit filters in each filter array may also be different. Figure 6As shown, the filter includes a first filter array, a second filter array, a third filter array, and a fourth filter array, wherein the first unit filter of the first filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter of the second filter array is provided with a polarizer with a polarization direction of the second polarization direction, the first unit filter of the third filter array is provided with a polarizer with a polarization direction of the third polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer with a polarization direction of the fourth polarization direction. Therefore, based on the filter, a monochrome (green) multi-directional polarization diagram can be obtained.

[0124] refer to Figure 6 As shown, the RGGB color arrangement of the filter array will not change due to the setting of the polarizer in the first unit filter, so before and after the polarizer is set, the resolution of the RGB image obtained by the filter including the filter array will not be reduced, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer corresponding to one G filter (i.e., the first unit filter) of each group of filter arrays, a monochrome multi-directional polarization image with a lower resolution can be obtained. Although the resolution of the green polarization image is lower than that of the RGB image, the monochrome multi-directional polarization image is sufficient to cope with specific scenes such as glare / reflection elimination, stress detection, and contrast enhancement. Moreover, the resolution of the monochrome multi-directional polarization image obtained in this scheme is the same as the resolution of the monochrome polarization image obtained by the above-mentioned prior art scheme 2. Therefore, the resolution of the RGB image obtained based on the filter including the filter array will not be reduced after adding a polarizer to the filter array, that is, it can ensure that the number of RGB pixels of the RGB sensor is not reduced, and the polarization image can be obtained, and the processing cost can be effectively controlled.

[0125] Figure 7 It is a schematic diagram of the structure of another filter proposed in an embodiment of the present application. Figure 7 The filter to be processed on the left refers to the filter without polarizer and / or filter, and has not been further processed. Figure 7 The target filter on the right is the filter proposed in the embodiment of the present application.

[0126] Figure 7 The color format of each filter array shown is similar to Figure 6 The color form of each group of filter arrays shown is the same. Among them, one of the two G filters included in each group of filter arrays can be used as the first unit filter, so the single color component filtered by the first unit filter is also the green color component.

[0127] In some possible embodiments, reference Figure 7As shown, the polarization directions of the polarizers provided in the first unit filters in each filter array can be the same: the first unit filter in the first filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter in the second filter array is provided with a polarizer with a polarization direction of the first polarization direction, the first unit filter in the third filter array is provided with a polarizer with a polarization direction of the first polarization direction, and the first unit filter in the fourth filter array is provided with a polarizer with a polarization direction of the first polarization direction. Therefore, based on this filter, a monochrome (green) unidirectional polarization diagram can be obtained.

[0128] refer to Figure 7 As shown, the RGGB color arrangement of the filter array will not change due to the polarizer being set in the first unit filter, so before and after the polarizer is set, the resolution of the RGB image obtained by the filter including the filter array will not be reduced, thereby ensuring the accuracy of the RGB image information. In addition, through the polarizer corresponding to one G filter (i.e., the first unit filter) of each group of filter arrays, a high-resolution monochromatic unidirectional polarization image can be obtained, and the resolution of the monochromatic unidirectional polarization image is equal to the resolution of the RGB image, and even if the polarization image directly obtained through the target filter is a unidirectional polarization image, two monochromatic unidirectional polarization images with the same angle (for example, the polarization angle is 135°) are subtracted by the above G channel (the G channel corresponding to the polarizer), and a monochromatic unidirectional polarization image of another polarization direction (i.e., the polarization angle is 45°) can be obtained. Although the polarization image obtained based on the target filter can correspond to two polarization directions, the monochromatic unidirectional polarization image based on these two polarization directions is sufficient to cope with those specific scenes such as glare / reflection elimination, stress detection, and contrast enhancement. Therefore, the resolution of the RGB image obtained based on the filter including the above-mentioned filter array will not be reduced after adding a polarizer to the filter array, that is, it can ensure that the number of RGB pixels of the RGB sensor is not reduced, and a polarized image with the same resolution as the RGB image can be obtained, and the processing cost can be effectively controlled.

[0129] In some possible embodiments, in any filter array described in the above embodiments, the first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

[0130] It should be understood that, since the first unit filter array of the above filter array is provided with a filter, the filter including the above filter array can be applied to some special application scenarios, such as night vision scenarios or temperature measurement scenarios.

[0131] Figure 8It is a schematic diagram of the structure of another filter proposed in an embodiment of the present application.

[0132] In some possible embodiments, the first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial wavelength band.

[0133] In some possible embodiments, reference Figure 8 As can be seen from (a), the filter includes multiple groups of filter arrays, and each group of filter arrays includes an R filter, a G filter, a B filter and a first unit filter, and the first unit filter is provided with a filter for filtering the color component of the infrared band (marked as IR in the figure), wherein the infrared band can be the above-mentioned first partial band.

[0134] Therefore, based on the above filter, not only the above RGB image can be obtained, but also the infrared spectrum image can be obtained.

[0135] In some possible embodiments, the K unit filters in the above-mentioned filter array also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second partial band.

[0136] In some possible embodiments, reference Figure 8 It can be seen from (b) that the filter includes multiple groups of filter arrays, and each group of filter arrays includes an R filter, a G filter, a B filter, a first unit filter and the above-mentioned second unit filter, and the first unit filter is provided with a filter for filtering the color component of the first partial band (marked as 1 in the accompanying drawing), and the second unit filter is provided with a filter for filtering the color component of the second partial band (marked as 2 in the accompanying drawing).

[0137] Based on the above filter, not only the above RGB image can be obtained, but also multiple spectral images can be obtained.

[0138] In some possible embodiments, the first partial band or the second partial band may be a narrowband band, and correspondingly, a narrowband spectral image may be acquired through the filter array of the above-mentioned filter setting; the first partial band or the second partial band may also be a broadband band, and correspondingly, a broadband spectral image may be acquired through the filter array of the above-mentioned filter setting.

[0139] refer to Figure 8As shown, the RGB color arrangement of the filter array will not change due to the polarizer being set on the first unit filter, so the resolution of the RGB image obtained by the filter including the filter array will not be reduced before and after the polarizer is set, thereby ensuring the accuracy of the RGB image information. The first unit filter of each filter array is provided with a filter, and based on the filter, not only the RGB image can be collected, but also the spectral image can be collected at the same time, and the number of RGB pixels of the RGB sensor is guaranteed not to be reduced, and the resolution of the polarization image is guaranteed to be the same as that of the RGB image, and the processing cost is effectively controlled.

[0140] In some possible embodiments, the color component filtered by the first unit filter is not necessarily the color component corresponding to the repeated color filter in the filter array, and can also be used to filter the color component corresponding to the non-repeated color filter, for example, the blue color component in the RGGB filter array.

[0141] In some possible embodiments, the filter includes at least two filter arrays. When filters are provided in at least two filter arrays, color components of partial bands filtered by the filters in at least two filter arrays are the same or different.

[0142] Fig. 9 It is a schematic diagram of the structure of another target filter proposed in an embodiment of the present application.

[0143] It should be understood that Fig. 9 The filter provided in the embodiment of the present application is shown in FIG. Each filter array group in the filter includes an R filter, a G filter, a B filter and a first unit filter.

[0144] refer to Fig. 9 As shown in (a), the filter includes a 1st filter array, a 2nd filter array, a 3rd filter array, a 4th filter array, a 5th filter array, a 6th filter array, a 7th filter array, an 8th filter array, and a 9th filter array, wherein the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with filters for filtering color components of different partial bands.

[0145] refer to Fig. 9 As shown in (b), the filter includes a 1st filter array, a 2nd filter array, a 3rd filter array, a 4th filter array, a 5th filter array, a 6th filter array, a 7th filter array, an 8th filter array, and a 9th filter array, wherein the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with filters for filtering color components of the same partial band.

[0146] Based on the above filter, since the color arrangement form (RGB form) of the filter array has not changed compared to before processing, the resolution of the RGB image obtained based on the target filter has not changed, thereby ensuring the accuracy of the RGB image information. In addition, by setting a corresponding filter corresponding to the first unit filter in the filter array, multispectral images or hyperspectral images can also be obtained based on the filter array.

[0147] In some possible embodiments, in any one of the filter arrays described in the above embodiments, the first unit filter is provided with a polarizer and a filter, the first unit filter is used to filter the full color component, the polarizer provided in the first unit filter is used to polarize the light corresponding to the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

[0148] The filter comprises at least two filter arrays. When filters are arranged in at least two filter arrays, the color components of the partial wavebands filtered by the filters in the at least two filter arrays are the same or different.

[0149] Fig.10 It is a schematic diagram of the structure of another filter proposed in an embodiment of the present application.

[0150] It should be understood that Fig.10 The target filter shown is the filter proposed in the embodiment of the present application. The filter includes the 1st filter array, the 2nd filter array, the 3rd filter array, the 4th filter array, the 5th filter array, the 6th filter array, the 7th filter array, the 8th filter array, and the 9th filter array. Each group of filter arrays in the filter includes an R filter, a G filter, a B filter, and a first unit filter. Among them, the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with filters for filtering color components of different partial bands. In addition, the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with polarizers with the same polarization direction.

[0151] In some possible embodiments, the filters respectively arranged in the first unit filters corresponding to the above-mentioned 1st filter array to the 9th filter array can also be used to filter the color components of the same partial band; the polarization directions of the polarizers respectively arranged in the first unit filters corresponding to the above-mentioned 1st filter array to the 9th filter array can also be different, or partially the same.

[0152] Of course, when each filter array group also includes the second unit filter, the second unit filter of each filter array group can be respectively provided with a filter and / or a polarizer. When each filter array group includes a plurality of unit filters for filtering all color components, these unit filters can be provided with a filter and / or a polarizer.

[0153] Based on the above technical solution, not only RGB images can be obtained, but also polarization images and multispectral images or hyperspectral images can be obtained. Based on these image information, the full optical function for digital image processing can be constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0154] In some possible embodiments, when the first unit filter is provided with a polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with a filter, the filter array is used in a second scenario; or, when the first unit filter is provided with a polarizer and a filter, the filter array is used in a third scenario.

[0155] In some possible embodiments, the first scene may be a scene that is prone to glare, in which the polarization direction of the polarizer disposed in the first unit filter is associated with the polarization angle of the light that causes image overexposure.

[0156] In some possible embodiments, the second scene may be a material recognition scene, a target recognition scene, etc., in which the color components of the partial bands that can be filtered by the filter set in the first unit filter are associated with the color information in the scene. In addition, the color components of the partial bands that can be filtered by the filter set in the first unit filter can also be associated with the demand information of the scene, for example, night imaging requirements, thermal imaging requirements, and correspondingly, the partial bands that can be filtered by the filter can be bands other than visible light.

[0157] In some possible embodiments, the third scene may be a mixed scene, for example, a scene that is prone to glare and requires material identification. The polarization direction of the polarizer and the partial band that the filter can filter are described in detail in the above example.

[0158] Based on the above technical solution, the above filter array can have a variety of polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0159] Since the different forms of filter arrays or filters proposed in the embodiments of the present application can be applied to different application scenarios, it is also necessary to obtain corresponding scene information during the filter processing process to obtain the desired filter array or filter.

[0160] For the above Figures 4 to 7 The filters shown and other extended forms can be processed by the following methods:

[0161] Fig.11 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application.

[0162] S1110: Acquire first scene information, where the first scene information includes light information of the first scene, where the light information is used to indicate a polarization angle of light that causes image overexposure.

[0163] S1120: According to the light information, a polarizer having a polarization direction of the first polarization direction is set for the first unit filter.

[0164] In some possible embodiments, the first polarization direction is associated with light information, and the polarization angle of the light that causes the image overexposure can be inferred by the location, light source type, and time of the application scene. For example, if the location is outdoors, the light source type is the sun, and the time is 3:00 p.m., then based on the principle of optical imaging, the polarization angle of the light that may cause the image overexposure can be inferred, and then the polarization direction of the polarizer can be determined based on the polarization angle.

[0165] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information can be represented by multiple data sets, which can be used as training sets for training neural network models, and the trained neural network model can be used in the process of processing filters, that is, light information is input into the neural network model, and the neural network model outputs the polarization direction of the polarizer, and then the processing equipment can set the polarizer of the target polarization direction for the filter to be processed.

[0166] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information can also be determined through preliminary experiments combined with an optical imaging principle model. Different application scenarios are constructed in multiple preliminary experiments, and the polarization direction of the polarizer corresponding to different light information is solved in combination with the optical imaging principle model. Different application scenarios correspond to the above-mentioned different light information.

[0167] For the above Figure 8 , Fig. 9 The filters shown and other extended forms can be processed by the following methods:

[0168] Fig.12 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application.

[0169] S1210: Acquire second scene information, where the second scene information includes requirement information and color element information of the second scene.

[0170] S1220: According to the requirement information and the color element information, a filter for filtering a color component of a target wavelength band is set for the first unit filter, where the color component of the target wavelength band is associated with the requirement information and the color element information.

[0171] In some possible embodiments, the demand information and color element information of the second scene are known information before processing the filter, for example, traffic environment detection, agricultural environment detection, factory equipment detection, etc. In addition, different application scenarios correspond to different needs and different color elements. Combining the demand information and color element information, the main color gamut and the band range of light in the application scenario can be determined. For example, in the agricultural environment detection scene, in addition to the spectrum corresponding to red, green, and blue with a wide spectral band, the main color gamut also needs to obtain specific narrow-band spectra in the green color gamut and the yellow color gamut to obtain a multi-spectral image or a hyperspectral image. In the traffic environment detection scene, in addition to the spectrum corresponding to red, green, and blue with a wide spectral band, the main color gamut also needs to obtain specific narrow-band spectra in various color gamuts due to the large number of color elements in the traffic environment. And in some complex application scenarios, such as the traffic environment at night, in order to enable the image processing device to obtain accurate environmental information, through the target filter, it is necessary not only to obtain specific narrow-band spectra in various color gamuts, but also to obtain broadband spectra of near-infrared light. It can be seen that the above-mentioned requirement information, the above-mentioned color element information and the color components of the above-mentioned target band are related.

[0172] Since the target semantic information corresponding to each spectral curve distribution can be determined according to the spectral curve distribution corresponding to the multispectral image or the hyperspectral image, object recognition at the material level can be achieved, which has a wider range of applications.

[0173] In some possible embodiments, Fig. 9 The multiple filters in the shown filter can be narrowband filters and / or broadband filters, so based on the filter obtained by the above method, both broadband spectral images and narrowband spectral images can be obtained, and broadband spectral images and narrowband spectral images can be obtained at the same time, which has a wide range of applications and strong flexibility.

[0174] In some possible embodiments, the correlation between the color components of the target band of the filter, the required information and the color element information can be represented by multiple data sets, and the multiple data sets can be used as training sets for training the neural network model, and the trained neural network model can be used in the process of processing the filter, that is, the application scenario information and color element information are input into the neural network model, and the neural network model outputs the color components of the target band, and then the filter corresponding to the color components of this part of the band is set for the filter to be processed.

[0175] In some possible embodiments, the correlation between the color components of the target band of the filter, the required information and the color element information can also be determined through preliminary experiments combined with the optical imaging principle model. Different application scenarios are constructed in multiple preliminary experiments, and the color components of the target band corresponding to different required information and color element information are solved in combination with the optical imaging principle model. Different application scenarios correspond to the above-mentioned different required information and color element information.

[0176] Based on the above technical solution, the RGB color arrangement of the filter to be processed is not changed, and the filter can be processed at a relatively low cost, so that not only RGB images but also multispectral / hyperspectral images can be obtained based on the processed target filter, and the single-pixel spatial resolution and the number of RGB pixels of the RGB sensor will not be reduced, so that multi-dimensional optical information with higher accuracy can be obtained through the filter.

[0177] For the above Fig.10 The filters shown and other extended forms can be processed by the following methods:

[0178] Fig.13 It is a schematic flow chart of another method for processing a filter proposed in an embodiment of the present application.

[0179] S1310: Acquire first scene information, where the first scene information includes light information of the first scene, where the light information is used to indicate a polarization angle of light that causes image overexposure.

[0180] S1320: According to the light information, a polarizer having a polarization direction of the first polarization direction is set for the first unit filter.

[0181] S1330: Acquire second scene information, where the second scene information includes requirement information and color element information of the second scene.

[0182] S1340: According to the requirement information and the color element information, a filter for filtering a color component of a target wavelength band is set for the first unit filter, where the color component of the target wavelength band is associated with the requirement information and the color element information.

[0183] Among them, the embodiment of the present application does not limit the sequence between S1310 to S1320 and S1330 to S1340.

[0184] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information can be represented by multiple first data sets, and the correlation between the color components of the target band of the above-mentioned filter, the required information and the color element information can be represented by multiple second data sets. The first data set and the second data set can be used as training sets for training the neural network model, and the trained neural network model can be used in the process of processing the filter, that is, the required information, color element information and light information are input into the neural network model, and the neural network model outputs the color components of the target band and / or the polarization direction of the polarizer, and then the processing equipment can set the filter corresponding to the color components of the target band and / or the polarizer corresponding to the polarization direction for the filter to be processed.

[0185] In some possible embodiments, the correlation between the first polarization direction and the light information and the correlation between the color components, required information and color element information of the target band of the filter can also be determined through preliminary experiments combined with optical imaging principle models. Different application scenarios are constructed in multiple preliminary experiments, and the specific forms of polarizers and filters corresponding to different application scenarios are solved in combination with the optical imaging principle model, and different application scenarios correspond to the above-mentioned different application scenario information and color element information.

[0186] Based on the above technical scheme, the RGB color arrangement of the filter to be processed is not changed, and the filter can be processed at a relatively low cost, so that RGB images, polarization images, and narrow-band multispectral / narrow-band hyperspectral images can be simultaneously acquired based on the processed target filter, and the single-pixel spatial resolution and the number of RGB pixels of the RGB sensor will not be reduced, so that high-precision multidimensional optical information can be acquired through the filter, and then a full-optical function for digital image processing is constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0187] Based on the above embodiments, it can be seen that the specific structure and application scenario of the filter are interrelated. Conversely, when the imaging device selects a filter for imaging operation, it can also first select a corresponding filter from the filter pool based on the current application scenario, wherein the filter pool is a pre-processed target filter described in the various above embodiments. Of course, the image processing method will be different based on filters of different structures. Finally, the processed image is sent to the downstream application module, and the downstream application module uses the processed image for functional scenarios such as autonomous driving perception, industrial vision, security monitoring or remote sensing.

[0188] In addition, an embodiment of the present application also provides a device for implementing any of the above methods. For example, a filter processing device is provided, which includes a unit (or means) for implementing any of the above filter processing methods.

[0189] Fig.14 It is a schematic diagram of a filter processing device 1400 proposed in an embodiment of the present application.

[0190] In some possible embodiments, the apparatus 1400 includes:

[0191] The acquisition unit 1410 is used to acquire a filter to be processed, where the filter to be processed includes a filter array, where the filter array includes K unit filters, where the K unit filters include an R filter, a G filter, and a B filter, where the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component, and the K unit filters also include a first unit filter;

[0192] The operation unit 1420 is used to process the filter array to obtain a target filter, including: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, where K is an integer greater than or equal to 4.

[0193] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain first scene information, the first scene information includes light information of the first scene, and the light information is used to indicate the polarization angle of the light that causes image overexposure; according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter.

[0194] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain second scene information, the second scene information including demand information and color element information of the second scene; according to the demand information and color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and color element information.

[0195] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain first scene information, the first scene information includes light information of the first scene, and the light information is used to indicate the polarization angle of the light that causes image overexposure; according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter; and, obtain second scene information, the second scene information includes demand information and color element information of the second scene; according to the demand information and the color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and the color element information.

[0196] An embodiment of the present application further proposes an imaging device, which includes a filter array and a photosensitive element proposed in any of the above embodiments.

[0197] In some possible embodiments, the number of the filter arrays is at least two. When polarizers are provided in at least two filter arrays, polarization directions of the polarizers in the at least two filter arrays are the same or different.

[0198] In some possible embodiments, the number of the filter arrays is at least two. When polarizers are provided in at least two filter arrays, polarization directions of the polarizers in the at least two filter arrays are the same or different.

[0199] In some possible embodiments, the number of the above-mentioned filter arrays is at least two, and when polarizers and filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different.

[0200] In some possible embodiments, the at least two filter arrays may be arranged in the same filter, which may be the above-mentioned Figures 4 to 7 , Fig. 9 , Fig.10 and other extended forms of filters.

[0201] An embodiment of the present application also proposes a vehicle, which includes the imaging device proposed in the above embodiment.

[0202] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0207] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0208] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A filter array, characterized in that: include: K unit filters, the K unit filters include an R filter, a G filter and a B filter, the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component; The K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

2. The filter array according to claim 1, characterized in that: The first unit filter is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize the light corresponding to the full color component.

3. The filter array according to claim 1, characterized in that: The first unit filter is provided with a polarizer, which is used to filter a single color component. The polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of the red color component, the green color component or the blue color component.

4. The filter array according to claim 1 or 2, characterized in that: The first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

5. The filter array according to claim 4, characterized in that: The K unit filters also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second partial band.

6. The filter array according to any one of claims 2 to 5, characterized in that: When the first unit filter is provided with the polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with the filter, the filter array is used in a second scenario; or, when the first unit filter is provided with the polarizer and the filter, the filter array is used in a third scenario.

7. An imaging device, characterized in that: The method comprises the filter array and the photosensitive element as claimed in any one of claims 1 to 6.

8. The imaging device according to claim 7, characterized in that: The number of the filter arrays is at least two. When polarizers are provided in at least two of the filter arrays, the polarization directions of the polarizers in at least two of the filter arrays are the same or different.

9. The imaging device according to claim 7, characterized in that: The number of the filter arrays is at least two. When filters are arranged in at least two of the filter arrays, the color components of the partial wavelength bands filtered by the filters in at least two of the filter arrays are the same or different.

10. The imaging device according to claim 7, characterized in that The number of the filter arrays is at least two. When polarizers and filters are provided in at least two of the filter arrays, the color components of the partial bands filtered by the filters in at least two of the filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two of the filter arrays are the same or different.

11. A vehicle, characterized in that: Comprising the imaging device as claimed in any one of claims 7 to 10.

Images