Filter sheet array, imaging apparatus and vehicle

Abstract

The present application relates to the technical field of preparation of optical devices. Provided are a filter sheet array, an imaging apparatus and a vehicle. The filter sheet array comprises K unit filter sheets, wherein the K unit filter sheets comprise an R filter sheet, a G filter sheet and a B filter sheet, the R filter sheet being used for filtering a red color component, the G filter sheet being used for filtering a green color component, and the B filter sheet being used for filtering a blue color component; and the K unit filter sheets further comprise a first unit filter sheet, and the first unit filter sheet is provided with a polarizer and / or the first unit filter sheet is provided with a filter device, K being an integer greater than or equal to 4. Thus, the processing cost and difficulty of a filter sheet array are relatively low; during processing, it is only necessary to provide a polarizer and / or a filter device on the first unit filter sheet; it can be ensured that RGB images acquired by means of the filter sheet array before and after processing have the same resolution, and multi-dimensional optical information with relatively high accuracy can also be acquired on the basis of the filter sheet array.

Description

Filter array, imaging device, and vehicle

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 18, 2023, with application number 202311056465.4 and invention name “Filter Array, Imaging Device and Vehicle”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of optical technology, and more particularly, to a filter array, an imaging device, and a vehicle. Background Art

[0003] With the development of technologies such as industrial vision and autonomous driving, the information obtained by conventional RGB cameras is no longer sufficient to meet the needs of the system. In some edge scenarios, the perception system needs to obtain multi-dimensional optical information to compensate for the shortcomings. From the perspective of perceptual information acquisition, the perception system can use specific filters to obtain multi-dimensional optical information of the current scene. However, filters processed based on existing imaging chip coating methods are generally expensive and will significantly reduce the single-pixel spatial resolution of the RGB sensor equipped with the filter, resulting in insufficient accuracy of the obtained multi-dimensional optical information.

[0004] Therefore, how to process the filter at a lower cost so that high-precision multi-dimensional optical information can be obtained through the filter is an urgent problem to be solved.

[0005] Summary of the Invention

[0006] The present application provides a filter array, an imaging device, a filter processing method and a vehicle. The filter array has a low processing cost and can obtain multi-dimensional optical information with high accuracy through the filter.

[0007] In a first aspect, a filter array is provided, comprising: K unit filters, the K unit filters including an R filter, a G filter, and a B filter, wherein the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component; the K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

[0008] It should be understood that the above-mentioned filter array includes an RGB color array, and conventional filters are usually arranged based on the RGB color array. Therefore, the above-mentioned filter array can be obtained by processing based on conventional filters, so the processing difficulty and cost of obtaining the above-mentioned filter array are relatively low.

[0009] For example, the array form of the above-mentioned filter array can be simply referred to as RGB form. Based on the above-mentioned filter array, a filter array in the form of RGGB can also be constructed. Of course, the K unit filters can also include a Y filter, or the above-mentioned G filter can be replaced with a Y filter, for example, to construct a filter array in the form of RYB.

[0010] For example, when the array form of the above-mentioned filter array is RGB form or RYB form, the above-mentioned K should be greater than or equal to 4; when the array form of the above-mentioned filter array is RGGB form or RYYB form, the above-mentioned K should be greater than or equal to 5.

[0011] For example, the array form of the above filter array can also be other forms, such as RGGGB, RRGGB, etc., which is not limited in the embodiments of the present application.

[0012] For example, the above filter array is applied to an imaging device. Then, in the imaging device, the above polarizer and filter can be directly arranged on the first unit filter, or can be arranged between the first unit filter and the photosensitive element of the imaging device.

[0013] Based on the above technical solution, the processing cost of the above filter array is relatively low. During the processing, it is only necessary to set a polarizer and / or filter on the first unit filter. This not only ensures that the resolution of the RGB image obtained by the filter array before and after processing is the same, but also allows for the acquisition of highly accurate multi-dimensional optical information based on the filter array.

[0014] In combination with the first aspect, in some implementations of the first aspect, the first unit filter is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize light corresponding to the full color component.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the first unit filter is provided with a polarizer, and the first unit filter is used to filter a single color component, and the polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component, or a blue color component.

[0016] Based on this technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image acquired based on the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, the polarizer corresponding to the first unit filter of the filter array can produce a full-color polarization map or a monochromatic polarization map, making the filter array suitable for applications such as glare / reflection reduction, stress detection, and contrast enhancement.

[0017] In combination with the first aspect, in some implementations of the first aspect, the first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

[0018] For example, the first part of the band can be a narrowband band, and correspondingly, a narrowband spectral image can be obtained through the filter array of the above-mentioned filter setting; the first part of the band can also be a broadband band, and correspondingly, a broadband spectral image can be obtained through the filter array of the above-mentioned filter setting.

[0019] For example, the first unit filter may further be provided with the polarizer.

[0020] Based on the above technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image obtained based on the target filter remains unchanged, thereby ensuring the accuracy of the RGB image information. In addition, by configuring the corresponding filters corresponding to the first unit filters in the filter array, spectral images can also be acquired based on this filter array. Since the first unit filters can also be configured with polarizers, polarization images can also be acquired based on this filter array.

[0021] In combination with the first aspect, in some implementations of the first aspect, the above-mentioned K unit filters also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second part of the band.

[0022] For example, the above-mentioned K unit filters can also include a third unit filter, which is also used to filter the full color component. The third unit filter is provided with a filter. The filter provided in the third unit filter can be used to filter the color component of the third partial band. Of course, it can be used to filter the color component of the above-mentioned first partial band or the second partial band.

[0023] For example, the K unit filters may include three or more unit filters for filtering all color components, and these unit filters are respectively provided with filters, and the color components of the partial bands filtered by these filters may be the same, different from each other, or partially the same.

[0024] Based on this technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image acquired based on the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, the filter array is highly scalable, and multiple filters can be configured within a single filter array, enabling the acquisition of multispectral or hyperspectral images.

[0025] In combination with the first aspect, in certain implementations of the first aspect, when the first unit filter is provided with a polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with a filter, the filter array is used in a second scenario; or, when the first unit filter is provided with a polarizer and a filter, the filter array is used in a third scenario.

[0026] For example, the first scene may be a scene that easily generates glare. In this scene, the polarization direction of the polarizer provided in the first unit filter is associated with the polarization angle of light that causes overexposure of the image.

[0027] For example, the second scenario can be a material recognition scenario, an object recognition scenario, or the like. In such a scenario, the color components of the partial wavelength band that can be filtered by the filter provided in the first unit filter are associated with the color information of the scene. Furthermore, the color components of the partial wavelength band that can be filtered by the filter provided in the first unit filter can also be associated with the requirements of the scene, such as night imaging requirements or thermal imaging requirements. Accordingly, the partial wavelength band that can be filtered by the filter can be a band outside of visible light.

[0028] For example, the third scene may be a mixed scene, for example, a scene that is prone to glare and requires material identification. The polarization direction of the polarizer and the partial bands that the filter can filter are described in detail in the corresponding description of the above example.

[0029] Based on the above technical solution, the above filter array can have multiple polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0030] In a second aspect, an imaging device is provided, which includes a filter array and a photosensitive element as in any possible implementation of the first aspect.

[0031] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two, and when polarizers are provided in at least two filter arrays, the polarization directions of the polarizers in at least two filter arrays are the same or different.

[0032] For example, the at least two filter arrays may be arranged in the same filter.

[0033] For example, the polarization directions of the polarizers in at least two filter arrays may be all the same, all different, or partially the same and partially different.

[0034] Based on the above technical solution, the imaging device can obtain polarization images corresponding to multiple polarization directions, and can also obtain RGB images.

[0035] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two, and when filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in the at least two filter arrays are the same or different.

[0036] Based on the above technical solution, the imaging device can acquire multispectral images or hyperspectral images, and can also acquire RGB images.

[0037] In combination with the second aspect, in certain implementations of the second aspect, the number of the above-mentioned filter arrays is at least two. When polarizers and filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different.

[0038] Based on the above technical solution, not only RGB images can be obtained, but also polarization images and multispectral images or hyperspectral images can be obtained. Based on this image information, a plenoptic function for digital image processing can be constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0039] In a third aspect, a vehicle is provided, comprising an imaging device as in any possible implementation of the second aspect.

[0040] In a fourth aspect, a method for processing a filter is provided, the method comprising: obtaining a filter to be processed, the filter to be processed comprising a filter array, the filter array comprising K unit filters, the K unit filters comprising an R filter, a G filter and a B filter, the R filter being used to filter a red color component, the G filter being used to filter a green color component, the B filter being used to filter a blue color component, and the K unit filters also comprising a first unit filter; processing the filter array to obtain a target filter, comprising: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, where K is an integer greater than or equal to 4.

[0041] For example, a filter may include at least one filter array, so processing the filter means processing each filter array in the filter.

[0042] Based on the above technical solution, the filter can be processed at a relatively low cost, and while ensuring that the resolution of the RGB image obtained by the processed filter remains unchanged, the corresponding polarization image information and / or spectral image information can also be obtained.

[0043] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter all color components, and the polarizer provided for the first unit filter is used to polarize light corresponding to the all color components.

[0044] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter a single color component, and the polarizer set for the first unit filter is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component, or a blue color component.

[0045] Based on this technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image acquired based on the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, the polarizer corresponding to the first unit filter of the filter array can produce a full-color polarization map or a monochromatic polarization map, making the filter array suitable for applications such as glare / reflection reduction, stress detection, and contrast enhancement.

[0046] In combination with the fourth aspect, in certain implementations of the fourth aspect, the first unit filter is used to filter all color components, and the filter set for the first unit filter is used to filter the color components of the first partial band.

[0047] Based on this technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image acquired based on the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, by configuring the corresponding filters for the first unit filters in the filter array, spectral images can also be acquired based on this filter array.

[0048] In combination with the fourth aspect, in some implementations of the fourth aspect, the K unit filters also include a second unit filter, a filter is set for the second unit filter, and the filter set in the second unit filter is used to filter the color component of the second part of the band.

[0049] Based on this technical solution, the color arrangement (RGB format) of the filter array remains unchanged compared to before processing, so the resolution of the RGB image acquired based on the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, the filter array is highly scalable, and multiple filters can be configured within a single filter array, enabling the acquisition of multispectral or hyperspectral images.

[0050] In combination with the fourth aspect, in certain implementations of the fourth aspect, first scene information is obtained, where the first scene information includes light information of the first scene, and the light information is used to indicate the polarization angle of the light that causes image overexposure; based on the light information, a polarizer with a polarization direction of the first polarization direction is set for the first unit filter.

[0051] In combination with the fourth aspect, in certain implementations of the fourth aspect, second scene information is obtained, and the second scene information includes demand information and color element information of the second scene; based on the demand information and color element information, a filter for filtering the color components of the target band is set for the first unit filter, and the color components of the target band are associated with the demand information and color element information.

[0052] For example, the second scene information may also include the light information. Therefore, the polarization direction of the polarizer and the color components of the partial wavelength bands filtered by the filter can be determined based on the light information, the required information, and the color element information. The determined polarizer and filter are then set for each first unit filter in the filter array. Of course, if the filter array includes multiple unit filters for filtering all color components, the polarization directions of the multiple polarizers and the color components of the partial wavelength bands filtered by the multiple filters can also be determined based on the light information, the required information, and the color element information.

[0053] Based on the above technical solution, the above filter array can have multiple polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0054] In a fifth aspect, a filter processing device is provided, which includes: an acquisition unit for acquiring a filter to be processed, the filter to be processed includes a filter array, the filter array includes K unit filters, the K unit filters include an R filter, a G filter and a B filter, the R filter is used to filter the red color component, the G filter is used to filter the green color component, the B filter is used to filter the blue color component, and the K unit filters also include a first unit filter; an operation unit for processing the filter array to obtain a target filter, including: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, where K is an integer greater than or equal to 4.

[0055] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first unit filter is used to filter all color components, and the polarizer provided for the first unit filter is used to polarize light corresponding to the all color components.

[0056] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned first unit filter is used to filter a single color component, and the polarizer set for the first unit filter is used to polarize the light corresponding to the single color component, and the single color component includes any one of the red color component, the green color component, or the blue color component.

[0057] In combination with the fifth aspect, in certain implementations of the fifth aspect, the first unit filter is used to filter all color components, and the filter provided for the first unit filter is used to filter the color components of the first partial band.

[0058] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned K unit filters also include a second unit filter, a filter is set for the second unit filter, and the filter set in the second unit filter is used to filter the color component of the second part of the band.

[0059] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned operating unit is specifically used to: obtain first scene information, the first scene information including light information of the first scene, the light information being used to indicate the polarization angle of the light that causes image overexposure; and according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter.

[0060] In combination with the fifth aspect, in certain implementations of the fifth aspect, the above-mentioned operating unit is specifically used to: obtain second scene information, the second scene information including demand information and color element information of the second scene; according to the demand information and color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and color element information.

[0061] In the sixth aspect, a filter processing device is provided, which includes: a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the method in any possible implementation method of the method design of the fourth aspect above.

[0062] In the seventh aspect, a chip system is provided, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to echo a signal from the memory of the electronic device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method in any possible implementation of the method design of the fourth aspect above.

[0063] In an eighth aspect, a computer-readable storage medium is provided, storing a computer program or instruction, which is used to implement the method in any possible implementation manner in the method design of the fourth aspect.

[0064] In the ninth aspect, a computer program product is provided. When the computer program code or instructions are executed on a computer, the computer executes the method in any possible implementation of the method design of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] FIG1 is a schematic diagram of a filter for obtaining polarization image information;

[0066] FIG2 is a schematic diagram of a filter array proposed in an embodiment of the present application;

[0067] FIG3 is a schematic flow chart of a method for processing an optical filter according to an embodiment of the present application;

[0068] FIG4 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0069] FIG5 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0070] FIG6 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0071] FIG7 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0072] FIG8 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0073] FIG9 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0074] FIG10 is a schematic structural diagram of another filter proposed in an embodiment of the present application;

[0075] FIG11 is a schematic flow chart of another method for processing an optical filter according to an embodiment of the present application;

[0076] FIG12 is a schematic flow chart of another method for processing an optical filter according to an embodiment of the present application;

[0077] FIG13 is a schematic flow chart of another method for processing an optical filter according to an embodiment of the present application;

[0078] FIG14 is a schematic diagram of a filter processing device 1400 proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solution in this application will be described below with reference to the accompanying drawings.

[0080] The technical solution of the embodiments of the present application can be applied to imaging systems. In existing imaging systems, the corresponding optical information in the current scene is usually obtained through filters. Taking filters that can obtain polarized image information as an example, a polarizer can be set on each sub-sheet of the filter.

[0081] FIG1 is a schematic diagram of a filter for obtaining polarization image information.

[0082] Figure 1 shows two sets of filter arrays based on Bayer filters. In these two sets of filter arrays, every four sub-chips correspond to one color, that is, every four sub-chips correspond to one pixel of the RGB sensor, and the four sub-chips correspond to polarizers in four polarization directions respectively.

[0083] Taking sub-chip 1 as an example, according to the principles of digital image processing, before performing digital image processing on the target image acquired through sub-chip 1, it is necessary to acquire all target images with the same color and positive polarization as sub-chip 1. Therefore, among all the sub-chips used to acquire these target images, the greater the number of sub-chips between adjacent sub-chips, the lower the RGB image resolution of the corresponding RGB sensor. For example, taking sub-chip 2, which is adjacent to sub-chip 1, as an example, there are three sub-chips between sub-chips 1 and 2.

[0084] However, if you don't place a polarizer above the Bayer filter and directly use the Bayer filter to obtain an RGB image, there is no need to separate the filters for each color. Therefore, each color filter corresponds to one pixel of the RGB sensor.

[0085] Taking red filter 1 as an example, according to the principles of digital image processing, before performing digital image processing on the target image acquired through red filter 1 (with geometric parameters identical to those of sub-filter 1 described above), it is necessary to acquire target images for all red filters. Therefore, among all the red filters used to acquire these target images, the greater the number of filters between adjacent red filters, the lower the RGB image resolution of the corresponding RGB sensor. For example, taking red filter 2, which is adjacent to red filter 1, as an example, there is a green filter between red filter 1 and red filter 1.

[0086] As can be seen, while the above method can obtain polarization image information, it significantly reduces the RGB image resolution of the RGB sensor. Furthermore, the pixel color arrangement of the Bayer filter is altered. In the original Bayer filter, one sub-slice corresponds to one pixel color, while in the modified Bayer filter, four sub-slices correspond to one pixel color, thereby reducing the number of RGB pixels in the RGB sensor. Furthermore, the filter shown in Figure 1 above cannot obtain optical information such as spectral images.

[0087] In view of this, the embodiments of the present application propose a filter array, an imaging device, a filter processing method and a vehicle, which can process the filter at a relatively low cost so that the image obtained through the filter can be processed to extract more accurate multi-dimensional optical information, which includes RGB image information, polarization image information and spectral image information.

[0088] An embodiment of the present application proposes a filter array. FIG2 is a schematic diagram of a filter array proposed in an embodiment of the present application.

[0089] The filter array includes: K unit filters, the K unit filters include an R filter, a G filter and a B filter, wherein the R filter is used to filter the red color component, the G filter is used to filter the green color component, and the B filter is used to filter the blue color component; the K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

[0090] 2 , all filter arrays in this example include four unit filters, i.e., K is equal to 4. The drawings of this application mark the unit filters for filtering all color components as A, so the first unit filter is marked as A in FIG2 .

[0091] The first unit filter in filter array 1 is provided with a polarizer. The embodiment of the present application does not limit the polarization direction of the polarizer. Therefore, the polarization direction of the polarizer provided in the first unit filter can also be other directions, for example, filter array 2. Based on the filters including filter array 1 and filter array 2 described above, not only RGB images can be obtained, but also polarized images.

[0092] The first unit filter in the filter array 3 is provided with a filter. Since the filter is used to filter the color components of a partial band, the combined functions of the first unit filter and the filter can be used to filter the color components of a partial band. Correspondingly, in FIG2 , the first unit filter after the filter is provided is marked as 1, and the number "1" is used to indicate that the filter is used to filter the color components of a specific partial band. The embodiment of the present application does not limit the color components of the partial band corresponding to the filter, so the filter provided in the first unit filter can also be used to filter the color components of other partial bands. Based on the filter comprising the above-mentioned filter array 3, not only RGB images can be obtained, but also spectral images.

[0093] The first unit filter in the filter array 4 is provided with not only a polarizer but also a filter. Based on the filters comprising the filter array 4, not only an RGB image but also a polarization image and a spectral image can be obtained.

[0094] In addition, before and after the polarizer and / or filter are set on the first unit filter, the color arrangement of the RGB filters of the filter array proposed in the embodiment of the present application does not change, so the resolution of the RGB image obtained based on the filter will not be reduced due to the polarizer and / or filter being set on the first unit filter.

[0095] Based on the above technical solution, the processing cost of the above filter array is relatively low. During the processing, it is only necessary to set a polarizer and / or filter on the first unit filter. This not only ensures that the resolution of the RGB image obtained by the filter array before and after processing is the same, but also allows for the acquisition of highly accurate multi-dimensional optical information based on the filter array.

[0096] Correspondingly, an embodiment of the present application further proposes a method for processing an optical filter. FIG3 is a flow chart of the method for processing an optical filter proposed in an embodiment of the present application.

[0097] S310: Obtain a filter to be processed, where the filter to be processed includes a filter array, which includes K unit filters. The K unit filters include an R filter, a G filter, and a B filter. The R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component. The K unit filters also include a first unit filter.

[0098] In some possible embodiments, the filter to be processed may be a Bayer filter, or the arrangement of the color filters of the K unit filters may correspond to the color arrangement of a conventional Bayer filter, and the embodiments of the present application do not limit the specific color arrangement of the Bayer filter.

[0099] S320: Processing the filter array to obtain a target filter, including:

[0100] Solution a: Provide a polarizer for the first unit filter; or

[0101] Option b: Set a filter for the first unit filter; or,

[0102] Solution c: A polarizer is provided for the first unit filter, and a filter is provided for the first unit filter.

[0103] Since the optical filter is an important component of the imaging device, the above-mentioned processing method may be specifically a processing method for the imaging device, and may also be understood as a part of the process for processing the imaging device.

[0104] It should be understood that the embodiment of the present application does not limit the order of setting the polarizer and the filter for the first unit filter in the above solution c.

[0105] In some possible embodiments, the polarization direction of the polarizer may be any direction, and the optical filter may be a broadband filter or a narrowband filter.

[0106] Based on the above technical solution, the filter can be processed at a relatively low cost, and while ensuring that the resolution of the RGB image obtained by the processed filter remains unchanged, the corresponding polarization image information and / or spectral image information can also be obtained.

[0107] It should be understood that the above filter array includes filter arrays processed based on the above solution a, solution b or solution c. The specific forms of the filter arrays processed based on these three solutions will be described below.

[0108] In some possible embodiments, taking the color arrangement of a Bayer filter as an example, a filter array is typically formed of four unit filters, including three color filters: an R filter, a G filter, and a B filter. These three color filters can be combined in a variety of arrangements within the filter array, such as RGB, RGGB, and the like. Of course, the G filter can also be replaced with a Y filter for filtering the yellow component. These three color filters can then be combined in a variety of arrangements within the filter array, such as RYB and RYYB. The three color filters can also be replaced with filters of other colors, which will not be elaborated upon here.

[0109] In some possible embodiments, the filter array includes filters of the three colors described above and also includes a first unit filter. For example, a filter array comprising four unit filters is configured as a group. When the filter array is arranged in an RGB format, the first unit filter is configured to filter all color components. When the filter array is arranged in an RGGB format, the first unit filter is configured to filter a single color component. In this example, the single color component is the green component. Of course, when the filter array includes other color arrangements, the color components filtered by the first unit filter are similar to those in the above example. That is, when there are no repeated color filters, the first unit filter is configured to filter all color components. When there are repeated color filters, the first unit filter is configured to filter a single color component, which is the color component corresponding to the repeated color.

[0110] In some possible embodiments, the array form of the above-mentioned filter array can also be other forms, for example, RGGGB, RRGGB, etc., which is not limited in the embodiments of the present application.

[0111] Based on the above three solutions, the filter array and the corresponding filter described in the following embodiments can be obtained.

[0112] In some possible embodiments, the first unit filter in the filter array is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize light corresponding to the full color component.

[0113] Figure 4 is a schematic diagram of the structure of a filter proposed in an embodiment of the present application. The filter to be processed on the left side of Figure 4 is a filter without a polarizer and / or filter, and has not been further processed. The target filter on the right side of Figure 4 is the filter proposed in an embodiment of the present application. The filter includes multiple filter arrays, each of which includes an R filter, a G filter, a B filter, and a first unit filter.

[0114] Referring to FIG. 4 , the drawings of this application mark the unit filter for filtering all color components as A, so the first unit filter is marked as A in FIG. 4 .

[0115] In some possible embodiments, a single filter may include two or more of the aforementioned filter arrays, and the polarization directions of the polarizers provided in the first unit filters of each filter array may be different. Referring to FIG4 , the filter includes a first filter array, a second filter array, a third filter array, and a fourth filter array. The first unit filter of the first filter array is provided with a polarizer having a first polarization direction, the first unit filter of the second filter array is provided with a polarizer having a second polarization direction, the first unit filter of the third filter array is provided with a polarizer having a third polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer having a fourth polarization direction. Therefore, based on this filter, a full-color multi-directional polarization map can be obtained.

[0116] In some possible embodiments, the first polarization direction to the fourth polarization direction differ from each other by 45°, for example, the first polarization direction is 0°, the second polarization direction is 45°, the third polarization direction is 90°, and the fourth polarization direction is 135°. Of course, the first polarization direction to the fourth polarization direction may also be polarization directions of other reasonable angles.

[0117] It should be understood that the RGB color arrangement of the filter array does not change due to the addition of a polarizer to the first unit filter. Therefore, the resolution of the RGB image obtained by the filter array does not decrease before or after the polarizer is added, thereby ensuring the accuracy of the RGB image information. Furthermore, the polarizer can also be used to obtain a lower-resolution full-color polarization image. Although the resolution of the full-color polarization image is lower than that of the RGB image, it is sufficient to address most scenarios such as glare / reflection reduction, stress detection, and contrast enhancement. Furthermore, the resolution of the full-color polarization image obtained in this solution is the same as the resolution of the monochromatic polarization image obtained using the aforementioned prior art solution 2. Therefore, the resolution of the RGB image obtained by the filter array does not decrease after the polarizer is added to the filter array. This ensures that the number of RGB pixels in the RGB sensor is not reduced, while still allowing the acquisition of polarization images and effectively controlling processing costs.

[0118] Figure 5 is a schematic diagram of the structure of another optical filter proposed in an embodiment of the present application. The filter to be processed on the left side of Figure 5 is a filter without a polarizer and / or filter, and has not been further processed, while the target filter on the right side of Figure 5 is the filter proposed in an embodiment of the present application.

[0119] The color form of each filter array group shown in FIG5 is the same as the color form of each filter array group shown in FIG4.

[0120] In some possible embodiments, as shown in FIG5 , two or more filter arrays described above may be included, and the polarizers provided in the first unit filters of each filter array may have the same polarization direction: the first unit filter of the first filter array may have a polarizer provided in the first polarization direction, the first unit filter of the second filter array may have a polarizer provided in the first polarization direction, the first unit filter of the third filter array may have a polarizer provided in the first polarization direction, and the first unit filter of the fourth filter array may have a polarizer provided in the first polarization direction. Therefore, based on this filter array, a full-color unidirectional polarization map can be obtained.

[0121] It should be understood that the polarization direction of the polarizer provided in the first unit filter of each group of the filter array may also be other directions, and this application does not limit this.

[0122] Based on the filter shown in Figure 5, the RGB color arrangement of the filter array does not change due to the placement of a polarizer on the first unit filter. Therefore, the resolution of the RGB image obtained through the filter array does not decrease before or after the polarizer is placed, thereby ensuring the accuracy of the RGB image information. Furthermore, the polarizer enables the acquisition of a high-resolution, full-color, unidirectional polarization map with a resolution equal to that of the RGB image. Even if the polarization map obtained directly through the target filter is a unidirectional polarization map, performing a full-color channel subtraction process on two full-color, unidirectional polarization maps with the same angle (e.g., a polarization angle of 135°) can yield a full-color, unidirectional polarization map for a different polarization direction (i.e., a polarization angle of 45°). While the polarization map obtained using the target filter can correspond to two polarization directions, a full-color, unidirectional polarization map based on these two polarization directions is sufficient for specific scenarios such as glare / reflection reduction, stress detection, and contrast enhancement. Therefore, the resolution of the RGB image obtained based on the filter array including the above-mentioned filter array will not be reduced after the polarizer is added to the filter array. In other words, the number of RGB pixels of the RGB sensor is not reduced, and a polarized image with the same resolution as the RGB image can be obtained, while effectively controlling processing costs.

[0123] In some possible embodiments, the first unit filter is provided with a polarizer, which is used to filter a single color component. The polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of a red color component, a green color component, and a blue color component.

[0124] Figure 6 is a schematic diagram of the structure of another filter proposed in an embodiment of the present application. The filter to be processed on the left side of Figure 6 refers to a filter that has not been further processed and is not equipped with a polarizer and / or filter, while the target filter on the right side of Figure 6 is the filter proposed in an embodiment of the present application. This filter includes multiple groups of filter arrays, and each group of filter arrays includes one R filter, two G filters, and one B filter. Among them, one of the two G filters can be used as the first unit filter, so the single color component filtered by this first unit filter is the green color component.

[0125] In some possible embodiments, the polarization directions of the polarizers provided in the first unit filters of each filter array shown in FIG6 may also be different. Referring to FIG6 , the filter array includes a first filter array, a second filter array, a third filter array, and a fourth filter array. The first unit filter of the first filter array is provided with a polarizer having a first polarization direction, the first unit filter of the second filter array is provided with a polarizer having a second polarization direction, the first unit filter of the third filter array is provided with a polarizer having a third polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer having a fourth polarization direction. Therefore, based on this filter array, a monochromatic (green) multi-directional polarization pattern can be obtained.

[0126] As shown in FIG6 , the RGGB color arrangement of the filter array does not change due to the addition of a polarizer to the first unit filter. Therefore, the resolution of the RGB image obtained by the filter array does not decrease before or after the polarizer is added, thereby ensuring the accuracy of the RGB image information. Furthermore, the polarizer corresponding to each G filter (i.e., the first unit filter) in each filter array group can produce a lower-resolution monochromatic multi-directional polarization image. Although the resolution of the green polarization image is lower than that of the RGB image, this monochromatic multi-directional polarization image is sufficient for specific scenarios such as glare / reflection reduction, stress detection, and contrast enhancement. Furthermore, the resolution of the monochromatic multi-directional polarization image obtained in this solution is the same as that obtained using the prior art solution 2. Therefore, the resolution of the RGB image obtained by the filter array does not decrease after adding a polarizer to the filter array. This ensures that the number of RGB pixels in the RGB sensor is not reduced, while still allowing the acquisition of polarized images and effectively controlling processing costs.

[0127] Figure 7 is a schematic diagram of the structure of another optical filter proposed in an embodiment of the present application. The filter to be processed on the left side of Figure 7 is a filter without a polarizer and / or filter, and has not been further processed, while the target filter on the right side of Figure 7 is the filter proposed in an embodiment of the present application.

[0128] The color format of each filter array group shown in FIG7 is the same as the color format of each filter array group shown in FIG6. In particular, one of the two G filters included in each filter array group can serve as the first unit filter, so the single color component filtered by the first unit filter is also the green color component.

[0129] In some possible embodiments, as shown in FIG7 , the polarizers provided in the first unit filters of each filter array may have the same polarization direction: the first unit filter of the first filter array is provided with a polarizer having a polarization direction of the first polarization direction, the first unit filter of the second filter array is provided with a polarizer having a polarization direction of the first polarization direction, the first unit filter of the third filter array is provided with a polarizer having a polarization direction of the first polarization direction, and the first unit filter of the fourth filter array is provided with a polarizer having a polarization direction of the first polarization direction. Therefore, based on this filter array, a monochrome (green) unidirectional polarization pattern can be obtained.

[0130] As shown in Figure 7, the RGGB color arrangement of the filter array does not change due to the placement of a polarizer on the first unit filter. Therefore, the resolution of the RGB image obtained by the filter array does not decrease before or after the polarizer is placed, thereby ensuring the accuracy of the RGB image information. In addition, the polarizer corresponding to each G filter (i.e., the first unit filter) in each filter array group can obtain a high-resolution monochromatic unidirectional polarization map. The resolution of this monochromatic unidirectional polarization map is equal to that of the RGB image. Even if the polarization map obtained directly through the target filter is a unidirectional polarization map, subtracting two monochromatic unidirectional polarization maps with the same angle (e.g., a polarization angle of 135°) through the G channel (the G channel corresponding to the polarizer) can obtain a monochromatic unidirectional polarization map for the other polarization direction (i.e., a polarization angle of 45°). Although the polarization map obtained based on the target filter can correspond to two polarization directions, a monochromatic unidirectional polarization map based on these two polarization directions is sufficient for specific scenarios such as glare / reflection reduction, stress detection, and contrast enhancement. Therefore, the resolution of the RGB image obtained based on the filter array including the above-mentioned filter array will not be reduced after the polarizer is added to the filter array. In other words, the number of RGB pixels of the RGB sensor is not reduced, and a polarized image with the same resolution as the RGB image can be obtained, while effectively controlling processing costs.

[0131] In some possible embodiments, in any filter array described in the above embodiments, the first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

[0132] It should be understood that since the first unit filter array of the above filter array is provided with a filter, the filter including the above filter array can be applied to some special application scenarios, such as night vision scenarios or temperature measurement scenarios.

[0133] FIG8 is a schematic structural diagram of another filter proposed in an embodiment of the present application.

[0134] In some possible embodiments, the first unit filter is provided with a filter, and the first unit filter is used to filter the full color component. The filter provided in the first unit filter is used to filter the color component of the first partial band.

[0135] In some possible embodiments, referring to (a) in FIG8 , it can be seen that the filter includes multiple groups of filter arrays, and each group of filter arrays includes an R filter, a G filter, a B filter and a first unit filter, and the first unit filter is provided with a filter for filtering the color component of the infrared band (marked as IR in the drawing), wherein the infrared band can be the above-mentioned first partial band.

[0136] Therefore, based on the above filter, not only the above RGB image can be obtained, but also the infrared spectrum image can be obtained.

[0137] In some possible embodiments, the K unit filters in the above-mentioned filter array also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second partial band.

[0138] In some possible embodiments, referring to (b) in Figure 8, it can be seen that the filter includes multiple groups of filter arrays, and each group of filter arrays includes an R filter, a G filter, a B filter, a first unit filter and the above-mentioned second unit filter, and the first unit filter is provided with a filter for filtering the color component of the first partial band (marked as 1 in the drawing), and the second unit filter is provided with a filter for filtering the color component of the second partial band (marked as 2 in the drawing).

[0139] Based on the above filters, not only the above RGB images can be obtained, but also multiple spectral images can be obtained.

[0140] In some possible embodiments, the first partial band or the second partial band may be a narrowband band, and correspondingly, a narrowband spectral image may be acquired through the filter array of the above-mentioned filter setting; the first partial band or the second partial band may also be a broadband band, and correspondingly, a broadband spectral image may be acquired through the filter array of the above-mentioned filter setting.

[0141] As shown in Figure 8 , the RGB color arrangement of the filter array is not altered by the addition of a polarizer to the first unit filter. Therefore, the resolution of the RGB image obtained by the filter array is not reduced before or after the polarizer is applied, thereby ensuring the accuracy of the RGB image information. The first unit filter of each filter array is equipped with a filter. This filter enables the simultaneous capture of not only RGB images but also spectral images, while maintaining the number of RGB pixels in the RGB sensor and ensuring the same resolution between the polarization image and the RGB image. This effectively controls processing costs.

[0142] In some possible embodiments, the color component filtered by the first unit filter is not necessarily the color component corresponding to the repeated color filter in the filter array, but can also be used to filter the color component corresponding to the non-repeated color filter, for example, the blue color component in the RGGB filter array.

[0143] In some possible embodiments, the optical filter includes at least two filter arrays. When filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in the at least two filter arrays are the same or different.

[0144] FIG9 is a schematic structural diagram of another target filter proposed in an embodiment of the present application.

[0145] It should be understood that Figure 9 shows the filter proposed in the embodiment of the present application. In the filter, each filter array includes an R filter, a G filter, a B filter and a first unit filter.

[0146] Referring to (a) in Figure 9, the filter includes a 1st filter array, a 2nd filter array, a 3rd filter array, a 4th filter array, a 5th filter array, a 6th filter array, a 7th filter array, an 8th filter array, and a 9th filter array, wherein the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with filters for filtering color components of different partial bands.

[0147] Referring to (b) in Figure 9, the filter includes a 1st filter array, a 2nd filter array, a 3rd filter array, a 4th filter array, a 5th filter array, a 6th filter array, a 7th filter array, an 8th filter array, and a 9th filter array, wherein the first unit filters corresponding to the 1st filter array to the 9th filter array are respectively provided with filters for filtering color components of the same partial band.

[0148] Because the color arrangement (RGB format) of the filter array remains unchanged compared to pre-processing, the resolution of the RGB image acquired using the target filter remains unchanged, thus ensuring the accuracy of the RGB image information. Furthermore, by configuring the corresponding filters for the first unit filters in the filter array, multispectral or hyperspectral images can also be acquired using this filter array.

[0149] In some possible embodiments, in any one of the filter arrays described in the above embodiments, the first unit filter is provided with a polarizer and a filter, the first unit filter is used to filter the full color component, the polarizer provided in the first unit filter is used to polarize the light corresponding to the full color component, and the filter provided in the first unit filter is used to filter the color component of the first part of the band.

[0150] The filter comprises at least two filter arrays. When filters are provided in at least two filter arrays, the color components of the partial wavelength bands filtered by the filters in the at least two filter arrays are the same or different.

[0151] FIG10 is a schematic structural diagram of another filter proposed in an embodiment of the present application.

[0152] It should be understood that the target filter shown in Figure 10 is the filter proposed in the embodiment of the present application. The filter includes a first filter array, a second filter array, a third filter array, a fourth filter array, a fifth filter array, a sixth filter array, a seventh filter array, an eighth filter array, and a ninth filter array. Each filter array group in the filter includes an R filter, a G filter, a B filter, and a first unit filter. Among them, the first unit filters corresponding to the first filter array to the ninth filter array are respectively provided with filters for filtering color components of different partial bands. In addition, the first unit filters corresponding to the first filter array to the ninth filter array are respectively provided with polarizers with the same polarization direction.

[0153] In some possible embodiments, the filters respectively provided in the first unit filters corresponding to the above-mentioned 1st filter array to the 9th filter array can also be used to filter the color components of the same partial band; the polarization directions of the polarizers respectively provided in the first unit filters corresponding to the above-mentioned 1st filter array to the 9th filter array can also be different, or partially the same.

[0154] Of course, when each filter array group further includes the aforementioned second unit filters, the second unit filters of each filter array group may be respectively provided with filters and / or polarizers. When each filter array group includes multiple unit filters for filtering all color components, each of these unit filters may be provided with filters and / or polarizers.

[0155] Based on the above technical solution, not only RGB images can be obtained, but also polarization images and multispectral images or hyperspectral images can be obtained. Based on this image information, a plenoptic function for digital image processing can be constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0156] In some possible embodiments, when the first unit filter is provided with a polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with a filter, the filter array is used in a second scenario; or, when the first unit filter is provided with a polarizer and a filter, the filter array is used in a third scenario.

[0157] In some possible embodiments, the first scene may be a scene that easily generates glare. In this scene, the polarization direction of the polarizer provided in the first unit filter is associated with the polarization angle of light that causes image overexposure.

[0158] In some possible embodiments, the second scenario may be a material recognition scenario, a target recognition scenario, or the like. In such a scenario, the color components of the partial wavelength band that can be filtered by the filter provided in the first unit filter are associated with the color information of the scene. Furthermore, the color components of the partial wavelength band that can be filtered by the filter provided in the first unit filter may also be associated with the requirements of the scene, such as night imaging requirements or thermal imaging requirements. Accordingly, the partial wavelength band that can be filtered by the filter may be a band outside of visible light.

[0159] In some possible embodiments, the third scene may be a mixed scene, for example, a scene that is prone to glare and requires material identification. The polarization direction of the polarizer and the partial wavelength bands that the filter can filter are described in detail in the above examples.

[0160] Based on the above technical solution, the above filter array can have multiple polarization forms and / or spectral combination forms, and different polarization forms and / or different spectral combination forms can be applied to different application scenarios, so the filter has a wide range of applications.

[0161] Since the different forms of filter arrays or filters proposed in the embodiments of the present application can be applied to different application scenarios, it is also necessary to obtain corresponding scene information during the filter processing process to obtain the desired filter array or filter.

[0162] The filters shown in Figures 4 to 7 and other extended forms can be processed by the following methods:

[0163] FIG11 is a flow chart of another method for processing an optical filter proposed in an embodiment of the present application.

[0164] S1110: Acquire first scene information, where the first scene information includes light information of the first scene, where the light information is used to indicate a polarization angle of light that causes image overexposure.

[0165] S1120: Setting a polarizer having a first polarization direction for the first unit filter according to the light information.

[0166] In some possible embodiments, the first polarization direction is associated with light information, and the polarization angle of the light that may cause image overexposure can be inferred based on the location, light source type, and time of the application scene. For example, if the location is outdoors, the light source type is the sun, and the time is 3:00 p.m., then based on the principles of optical imaging, the polarization angle of the light that may cause image overexposure can be inferred, and the polarization direction of the polarizer can be determined based on this polarization angle.

[0167] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information can be represented by multiple data sets, which can be used as training sets for training neural network models. The trained neural network model can be used in the process of processing filters, that is, light information is input into the neural network model, and the neural network model outputs the polarization direction of the polarizer. Then, the processing equipment can set the polarizer with the target polarization direction for the filter to be processed.

[0168] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information can also be determined through preliminary experiments combined with an optical imaging principle model. Different application scenarios are constructed in multiple preliminary experiments, and the polarization direction of the polarizer corresponding to different light information is solved in combination with the optical imaging principle model. Different application scenarios correspond to the above-mentioned different light information.

[0169] The filters shown in Figures 8 and 9 and other extended forms can be processed by the following methods:

[0170] FIG12 is a schematic flow chart of another method for processing an optical filter proposed in an embodiment of the present application.

[0171] S1210: Acquire second scene information, where the second scene information includes requirement information and color element information of the second scene.

[0172] S1220: Setting a filter for filtering a color component of a target wavelength band for the first unit filter according to the requirement information and the color element information, where the color component of the target wavelength band is associated with the requirement information and the color element information.

[0173] In some possible embodiments, the requirements and color element information for the second scenario are known before the filters are processed. Examples include traffic environment monitoring, agricultural environment monitoring, and factory equipment monitoring. Furthermore, different application scenarios correspond to different requirements and color elements. Combining these requirements and color element information can determine the primary color gamut and wavelength range of the light in the application scenario. For example, in agricultural environment monitoring scenarios, in addition to the spectra corresponding to the broad spectral bands of red, green, and blue, specific narrowband spectra within the green and yellow color gamuts are also required to acquire multispectral or hyperspectral images. In traffic environment monitoring scenarios, in addition to the spectra corresponding to the broad spectral bands of red, green, and blue, the primary color gamut also requires acquisition of specific narrowband spectra within each color gamut due to the numerous color elements in the traffic environment. Furthermore, in complex application scenarios, such as nighttime traffic environments, in order for the image processing device to obtain accurate environmental information, the target filter must not only acquire specific narrowband spectra within each color gamut but also a broadband spectrum of near-infrared light. It can be seen from this that the above-mentioned demand information, the above-mentioned color element information and the color components of the above-mentioned target band are associated with each other.

[0174] Since the target semantic information corresponding to each spectral curve distribution can be determined based on the spectral curve distribution corresponding to the multispectral image or hyperspectral image, object recognition at the material level can be achieved, which has a wider range of applications.

[0175] In some possible embodiments, the multiple filters in the filter shown in Figure 9 can be narrowband filters and / or broadband filters. Therefore, based on the filter obtained by the above method, both broadband spectral images and narrowband spectral images can be obtained, and broadband spectral images and narrowband spectral images can be obtained at the same time, which has a wide range of applications and strong flexibility.

[0176] In some possible embodiments, the correlation between the color components of the target band of the filter, the required information, and the color element information can be represented by multiple data sets, which can be used as training sets for training a neural network model. The trained neural network model can be used in the process of processing the filter, that is, the application scenario information and color element information are input into the neural network model, the neural network model outputs the color components of the target band, and then the filter corresponding to the color components of this part of the band is set for the filter to be processed.

[0177] In some possible embodiments, the correlation between the color components of the target band of the filter, the demand information and the color element information can also be determined through preliminary experiments combined with the optical imaging principle model. Different application scenarios are constructed in multiple preliminary experiments, and the color components of the target band corresponding to different demand information and color element information are solved in combination with the optical imaging principle model. Different application scenarios correspond to the above-mentioned different demand information and color element information.

[0178] Based on the above technical solution, the RGB color arrangement of the filter to be processed is not changed, and the filter can be processed at a low cost, so that not only RGB images but also multispectral / hyperspectral images can be obtained based on the processed target filter, and the single-pixel spatial resolution and the number of RGB pixels of the RGB sensor will not be reduced, so that high-precision multidimensional optical information can be obtained through the filter.

[0179] The filters shown in Figure 10 and other extended forms can be processed by the following methods:

[0180] FIG13 is a flow chart of another method for processing an optical filter proposed in an embodiment of the present application.

[0181] S1310: Acquire first scene information, where the first scene information includes light information of the first scene, where the light information is used to indicate a polarization angle of light that causes image overexposure.

[0182] S1320: Setting a polarizer having a first polarization direction for the first unit filter according to the light information.

[0183] S1330: Acquire second scene information, where the second scene information includes requirement information and color element information of the second scene.

[0184] S1340: According to the requirement information and the color element information, set a filter for filtering a color component of a target wavelength band for the first unit filter, where the color component of the target wavelength band is associated with the requirement information and the color element information.

[0185] The embodiment of the present application does not limit the order between S1310 to S1320 and S1330 to S1340.

[0186] In some possible embodiments, the association between the above-mentioned first polarization direction and the light information can be represented by multiple first data sets, and the association between the color components of the target band of the above-mentioned filter, the required information and the color element information can be represented by multiple second data sets. The first data set and the second data set can be used as training sets for training the neural network model, and the trained neural network model can be used in the process of processing the filter, that is, the required information, color element information and light information are input into the neural network model, and the neural network model outputs the color components of the target band and / or the polarization direction of the polarizer, and then the processing equipment can set the filter corresponding to the color components of the target band and / or the polarizer corresponding to the polarization direction for the filter to be processed.

[0187] In some possible embodiments, the correlation between the above-mentioned first polarization direction and the light information and the correlation between the color components, required information and color element information of the target band of the above-mentioned filter can also be determined through preliminary experiments combined with optical imaging principle models. Different application scenarios are constructed in multiple preliminary experiments, and the specific forms of polarizers and filters corresponding to different application scenarios are solved in combination with the optical imaging principle model. Different application scenarios correspond to the above-mentioned different application scenario information and color element information.

[0188] Based on the above technical solution, the RGB color arrangement of the filter to be processed is not changed, and the filter can be processed at a low cost, so that RGB images, polarization images, and narrow-band multispectral / narrow-band hyperspectral images can be simultaneously acquired based on the processed target filter, and the single-pixel spatial resolution and the number of RGB pixels of the RGB sensor will not be reduced, so that high-precision multidimensional optical information can be acquired through the filter, and then a plenoptic function for digital image processing can be constructed, which helps to improve the precision and accuracy of subsequent digital image processing results.

[0189] Based on the above embodiments, it can be seen that the specific structure of the filter and the application scenario are interrelated. Conversely, when an imaging device selects a filter for imaging operations, it can also first select a corresponding filter from a filter pool based on the current application scenario, where the filter pool is a pre-processed target filter described in the various embodiments above. Of course, the image processing method will also vary based on the different filter structures. Ultimately, the processed image is sent to a downstream application module, which uses the processed image in functional scenarios such as autonomous driving perception, industrial vision, security monitoring, or remote sensing.

[0190] In addition, embodiments of the present application also provide a device for implementing any of the above methods. For example, a filter processing device is provided, which includes a unit (or means) for implementing any of the above filter processing methods.

[0191] FIG14 is a schematic diagram of a filter processing device 1400 proposed in an embodiment of the present application.

[0192] In some possible embodiments, the apparatus 1400 includes:

[0193] An acquisition unit 1410 is configured to acquire a filter to be processed, where the filter to be processed includes a filter array, the filter array includes K unit filters, the K unit filters include an R filter, a G filter, and a B filter, the R filter is configured to filter a red color component, the G filter is configured to filter a green color component, and the B filter is configured to filter a blue color component. The K unit filters also include a first unit filter.

[0194] The operating unit 1420 is used to process the filter array to obtain a target filter, including: setting a polarizer for the first unit filter, and / or setting a filter for the first unit filter, where K is an integer greater than or equal to 4.

[0195] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain first scene information, the first scene information including light information of the first scene, the light information is used to indicate the polarization angle of the light that causes image overexposure; according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter.

[0196] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain second scene information, the second scene information including demand information and color element information of the second scene; according to the demand information and color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and color element information.

[0197] In some possible embodiments, the above-mentioned operation unit 1420 is specifically used to: obtain first scene information, which includes light information of the first scene, and the light information is used to indicate the polarization angle of the light that causes image overexposure; according to the light information, set a polarizer with a polarization direction of the first polarization direction for the first unit filter; and, obtain second scene information, the second scene information includes demand information and color element information of the second scene; according to the demand information and the color element information, set a filter for filtering the color component of the target band for the first unit filter, and the color component of the target band is associated with the demand information and the color element information.

[0198] An embodiment of the present application further provides an imaging device, which includes a filter array and a photosensitive element provided in any of the above embodiments.

[0199] In some possible embodiments, the number of the filter arrays is at least two. When polarizers are provided in at least two filter arrays, the polarization directions of the polarizers in the at least two filter arrays are the same or different.

[0200] In some possible embodiments, the number of the filter arrays is at least two. When polarizers are provided in at least two filter arrays, the polarization directions of the polarizers in the at least two filter arrays are the same or different.

[0201] In some possible embodiments, the number of the above-mentioned filter arrays is at least two. When polarizers and filters are provided in at least two filter arrays, the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two filter arrays are the same or different.

[0202] In some possible embodiments, the at least two filter arrays may be disposed in the same filter, which may be the filter of Figures 4 to 7, 9, 10, or other extended forms.

[0203] An embodiment of the present application further provides a vehicle, which includes the imaging device provided in the above embodiment.

[0204] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0205] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0209] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0210] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A filter array, characterized in that: include: K unit filters, the K unit filters include an R filter, a G filter and a B filter, the R filter is used to filter a red color component, the G filter is used to filter a green color component, and the B filter is used to filter a blue color component; The K unit filters also include a first unit filter, the first unit filter is provided with a polarizer, and / or the first unit filter is provided with a filter, and K is an integer greater than or equal to 4.

2. The filter array according to claim 1, characterized in that: The first unit filter is provided with a polarizer, the first unit filter is used to filter the full color component, and the polarizer is used to polarize the light corresponding to the full color component.

3. The filter array according to claim 1, characterized in that: The first unit filter is provided with a polarizer, which is used to filter a single color component. The polarizer is used to polarize light corresponding to the single color component, and the single color component includes any one of the red color component, the green color component or the blue color component.

4. The filter array according to claim 1 or 2, characterized in that: The first unit filter is provided with a filter, the first unit filter is used to filter the full color component, and the filter provided in the first unit filter is used to filter the color component of the first partial band.

5. The filter array according to claim 4, characterized in that: The K unit filters also include a second unit filter, which is used to filter the full color component. The second unit filter is provided with a filter, and the filter provided in the second unit filter is used to filter the color component of the second partial band.

6. The filter array according to any one of claims 2 to 5, characterized in that: When the first unit filter is provided with the polarizer, the filter array is used in a first scenario; or, when the first unit filter is provided with the filter, the filter array is used in a second scenario; or, when the first unit filter is provided with the polarizer and the filter, the filter array is used in a third scenario.

7. An imaging device, characterized in that: The invention comprises the filter array and the photosensitive element as claimed in any one of claims 1 to 6.

8. The imaging device according to claim 7, characterized in that: The number of the filter arrays is at least two. When polarizers are provided in at least two of the filter arrays, the polarization directions of the polarizers in at least two of the filter arrays are the same or different.

9. The imaging device according to claim 7, characterized in that: The number of the filter arrays is at least two. When filters are arranged in at least two of the filter arrays, the color components of the partial wavelength bands filtered by the filters in at least two of the filter arrays are the same or different.

10. The imaging device according to claim 7, characterized in that The number of the filter arrays is at least two. When polarizers and filters are provided in at least two of the filter arrays, the color components of the partial bands filtered by the filters in at least two of the filter arrays are the same or different, and the color components of the partial bands filtered by the filters in at least two of the filter arrays are the same or different.

11. A vehicle, characterized in that: Comprising the imaging device as claimed in any one of claims 7 to 10.

Images