Compressed image sensor module obtained by pixel rearrangement

Abstract

In some examples, a two-dimensional filter array arrangement for use in conjunction with an imaging device is configured to generate multi-band image data corresponding to an object or scene to be imaged. At least a portion of the filter array arrangement includes: an M x N array of spectral filter elements defining a set of unit cells, each unit cell including a P x Q sub-array of spectral filter elements, where P < M and Q < N, the spectral filter elements of the unit cells restricting imaging to a plurality of selected regions of the electromagnetic spectrum; wherein a first unit cell includes a set A of the spectral filter elements of the filter array arrangement, a second unit cell connected to the first unit cell includes a set B of spectral filter elements, the first unit cell and the second unit cell are arranged relative to each other so as to include a first set X of spectral filter elements and a second set Y of spectral filter elements, where X = {x|x ∈ (A ∩ B)} and Y = {y|y ∈ (A Δ B)}; and wherein the first set X of spectral filter elements forms a repeating pattern of the filter array arrangement.

Description

Technical Field

[0001] Aspects of the present invention generally relate to sensor modules, and more specifically but not limited to, filter arrays used in sensors of sensor modules. Background Art

[0002] Image sensors used in imaging devices (e.g., image sensors implemented in user devices) typically utilize multiple color channels to form a color filter array (CFA). The CFA can include a set of color filters on a grid of photodetectors to form an image sensor for the imaging device. A common pattern of color filters in a CFA for visible light imaging is the Bayer mosaic pattern, which includes green, red, and blue filter elements in a ratio of 2:1:1.

[0003] In addition to the above channels, spectral sensors can also use many other channels to image bands of the electromagnetic spectrum that may not be visible to the naked eye. Within a limited range, a spectral sensor can use as many filter channels as there are sensor pixels, covering corresponding different bands, but typically some sensor pixels use the same "color" filter as other pixels, such as the filters in the Bayer pattern.

[0004] In higher resolution spectral sensors, multiple sets of filter arrangements can be repeated within the area of the spectral image sensor. For example, a 4x4 filter arrangement forming a mosaic pattern covering 16 different bands can be repeated to provide a high resolution spectral sensor. Such image sensors can be used in various applications. For example, due to the ability to measure multiple smaller wavelength ranges, spectral sensors can produce more information than ordinary image sensors (e.g., image sensors with a Bayer CFA pattern), and this additional information can be used to improve the color of images / videos, improve image segmentation, detect different materials or material properties / conditions, etc.

[0005] Although using such higher resolution spectral image sensors is attractive, the size of the image sensor also increases due to the increased number of bands of information that the sensor can capture. Summary of the Invention

[0006] According to a first aspect, there is provided a two-dimensional filter array arrangement for use in conjunction with an imaging device, the filter array arrangement being for generating multi-band image data corresponding to an object or scene to be imaged, at least a portion of the filter array arrangement comprising: an M x N array of spectral filter elements for defining a set of unit cells, each unit cell comprising a P x Q sub-array of spectral filter elements, where P < M and Q < N, the spectral filter elements of the unit cells restricting imaging to a plurality of selected regions of the electromagnetic spectrum; wherein a first unit cell comprises a set A of the spectral filter elements of the filter array arrangement, and a second unit cell connected to the first unit cell comprises a set B of spectral filter elements, the first unit cell and the second unit cell being arranged relative to each other so as to comprise a first set X of spectral filter elements and a second set Y of spectral filter elements, where X = {x|x ∈ (A ∩ B)} and Y = {y|y ∈ (A Δ B)}; and wherein the first set X of spectral filter elements forms a repeating pattern of the filter array arrangement.

[0007] The filter array arrangement is particularly useful for spectral imaging purposes because it enables multiple bands of the electromagnetic spectrum to be imaged using an image sensor arrangement that is smaller than would otherwise be the case. For example, a relatively small arrangement results in a suitably advantageous reduction in the size of the sensor module, which is achieved by arranging the filter elements so as to provide unit cells that can share certain filter elements effectively, as opposed to more conventional arrangements where a mosaic pattern of filter elements is repeated over the image sensor area. Such arrangements typically result in some channels being over-represented, or at least more prevalent in the array. In the arrangement of the present invention, this over-representation is exploited to reduce the size of the unit cells, which include multiple different filter elements but include a subset of filter elements that can be shared with neighbouring unit cells.

[0008] In the two-dimensional filter array arrangement, the first set of spectral filter elements may form a first repeating pattern in a first direction of the filter array arrangement. The first set of spectral filter elements may form a second repeating pattern in a second direction of the filter array arrangement, the second direction being orthogonal to the first direction. That is, the unit cells may overlap in one or more dimensions of the arrangement.

[0009] In one example, the second set of spectral filter elements may form a third repeating pattern in the first direction of the filter array arrangement. The second set of spectral filter elements may form a fourth repeating pattern in the first direction of the filter array arrangement. Thus, there may be some unit cells in which there are mirror images of certain spectral filter elements. For example, certain shared or non-shared filter elements may be mirrored in adjacent, neighbouring or connected unit cells, etc.

[0010] In one implementation of the first aspect, the relative complement of A in B That is to say, for a pair of adjacent or connected unit cells, there may be no spectral filter in set A that is not in set B.

[0011] In one implementation of the first aspect, That is to say, the number n(.) of spectral filters generated due to the difference between the relative complements can be empty.

[0012] In some implementations of the first aspect, the relative complement B\A of A in B may not be equal to the relative complement A\B of B in A.

[0013] The first unit cell and the second unit cell share a vertical boundary. The first unit cell and the second unit cell share a horizontal boundary. Thus, for example, a pair of unit cells can have an overlapping vertical boundary because there are multiple shared spectral filters that define the vertical boundary of the unit cells. The same is true for the horizontal boundary.

[0014] The vertical boundary and / or the horizontal boundary between the unit cells may include at least one spectral filter in the first set of spectral filters.

[0015] In one implementation of the first aspect, the arrangement of the spectral filters in the second set of spectral filters within the first unit cell and the second unit cell may be symmetric with respect to the symmetry line defined by the shared boundary between the first unit cell and the second unit cell. For example, the shared boundary may be a horizontal boundary or a vertical boundary.

[0016] The arrangement of the spectral filters in the second set of spectral filters within the first unit cell and the second unit cell may be the same. In another example, the arrangement of the spectral filters in the second set of spectral filters within the first unit cell and the second unit cell may be different. For example, there may or may not be a mirror image of one or more spectral filters, or some other positioning that causes a change in the position of certain spectral filters in the connected unit cells.

[0017] One or more spectral filters in the second set of spectral filters may be replaced with alternative spectral filters for restricting imaging to regions of the electromagnetic spectrum not covered by other spectral filters in the second set of spectral filters. The alternative spectral filters may be disposed in the first unit cell and / or the second unit cell.

[0018] That is, the spectral filter element associated with a given channel can be replaced with another filter element sensitive to a different channel. In one example, thus, a certain number of filter elements for a channel can be replaced, where the channel can include a relatively larger number of filter elements compared to other arrangements (see the green channel in a typical Bayer mosaic pattern), enabling "imaging" of a larger number of channels. Broadly speaking, the number of replaced filter elements may be low compared to the number of filter elements of other channels in the arrangement, especially those where no replacement occurs, but in some cases, this may be perfectly acceptable.

[0019] According to a second aspect, there is provided an imaging device for generating multi-band image data representing a spectral image cube of an object or a scene, the imaging device comprising: an imaging sensor including a two-dimensional sensor array; a two-dimensional filter array arrangement according to any one of the preceding claims, wherein the two-dimensional filter array arrangement is positioned in front of or embedded in the sensor array with respect to the optical path of the imaging device. The two-dimensional sensor array may include a grid of photodetectors defining a pixel array of the imaging device, wherein each of the spectral filter elements of the two-dimensional filter array arrangement is positioned to coincide with a pixel. The imaging device may further comprise: a processor for: generating a plurality of spectral band images of the object or the scene, each spectral band image corresponding to image data generated by pixels restricted to an imaging region of the electromagnetic spectrum defined by the spectral filter element.

[0020] According to a third aspect, there is provided a method for providing a filter array arrangement, the method comprising: placing a first unit cell relative to a second unit cell, the first unit cell and the second unit cell being placed according to the first aspect.

[0021] According to a fourth aspect, there is provided a user device comprising: an imaging device according to the second aspect. In one implementation of the fourth aspect, the user device may be in the form of a smart device for using data generated by the imaging device, such as a smart mobile phone. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more fully understand the present invention, reference is now made, by way of example only, to the following description in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram showing a part of a filter array arrangement;

[0024] Figure 2 A schematic diagram showing a part of a filter array arrangement provided by an example;

[0025] Figures 3a to 3dA schematic diagram showing a part of a provided filtering array arrangement;

[0026] Figure 4 A schematic diagram showing a part of a provided filtering array arrangement;

[0027] Figure 5 A schematic diagram showing a part of a provided filtering array arrangement;

[0028] Figure 6 A schematic diagram showing a part of a provided filtering array arrangement;

[0029] Figure 7 A schematic diagram showing a part of a provided filtering array arrangement;

[0030] Figures 8a to 8e A schematic diagram showing parts of a provided filtering array arrangement;

[0031] Figure 9 A schematic diagram showing a provided imaging device. Detailed Description

[0032] The following describes exemplary embodiments in sufficient detail to enable those of ordinary skill in the art to implement and realize the systems and processes described herein. Importantly, it is understood that the embodiments can be provided in many alternative forms and should not be construed as limited to the examples described herein. Thus, while the embodiments can be modified in various ways and take various alternative forms, their specific embodiments are shown in the drawings and are described in detail below as examples. It is not intended to be limited to the specific forms disclosed. Instead, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. In all the drawings and the appropriate detailed description, the elements of the exemplary embodiments are consistently denoted by the same reference numerals.

[0033] The terms used herein to describe the embodiments are not intended to limit the scope. The articles "a" and "the" are in the singular form because they have only one referent, but the use of the singular form herein should not exclude the existence of multiple referents. In other words, unless the context clearly indicates otherwise, elements referred to in the singular can be one or more in number. It should be further understood that the term "comprising" used herein is used to indicate the presence of the stated features, items, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, items, steps, operations, elements, components, and / or their combinations.

[0034] Unless otherwise defined, all terms (including technical and scientific terms) used herein shall be construed in accordance with the customary usage in the art. It should also be understood that terms in common usage should be construed as having the customary meaning in the relevant art, unless explicitly defined herein, and should not be construed in an idealized or overly formal sense.

[0035] Spectral sensors can be used to generate data covering multiple bands of the electromagnetic spectrum. Each band (or channel) typically covers a narrow band that can have a full width half maximum (FWHM), such as 10 nm, 20 nm, or 50 nm, and the channels can have individual widths. The spectral sensor can cover a wavelength range from 400 nm to 1000 nm, and can even cover a wavelength range of several micrometers.

[0036] Spectral sensors with multiple channels can be used to perform multispectral imaging or hyperspectral imaging. The main difference between multispectral / hyperspectral imaging lies in the number of channels used / supported (the number of channels typically used in hyperspectral imaging is usually one order of magnitude larger than that used for multispectral imaging). Sensors with multiple channels can include channels with peak wavelengths (e.g., 400 nm, 420 nm, …, 600 nm, 620 nm, 630 nm, …, 850 nm, 940 nm, etc.), and these peak wavelengths can be selected during the design phase. For example, channels can be selected such that they are evenly distributed within a given wavelength range; alternatively, channels can be selected to support specific use cases. For example, a spectral sensor for color processing can have many channels that are more densely populated in the visible wavelength region of the electromagnetic spectrum compared to, for example, the near-infrared region.

[0037] For example, in the context of user devices such as smartphones, the benefits that a high-resolution spectral image sensor can provide can be offset by the corresponding unwanted increase in size and weight caused by its implementation. That is, in order to accommodate a larger sensor that can generate image data on a relatively larger number of channels compared to, for example, a Bayer mosaic sensor, the user device may need to be larger in size and heavier in weight.

[0038] According to one example, a multi-channel filter array arrangement is provided that is capable of using a smaller sensor / module size while substantially maintaining sensing quality. The arrangement includes a plurality of spectral filter elements that are used to limit imaging to a selected region of the electromagnetic spectrum. Each spectral filter element can form a pixel. That is, each spectral filter element can be positioned on a detector of a detector grid. In some examples, identical pixels that are adjacent to each other can be removed, which enables a smaller sensor size to be achieved in one or more dimensions. This has a knock-on effect of achieving a smaller optical module height (since the optical module height is proportional to the diagonal length of the effective area of the sensor).

[0039] Figure 1 A schematic diagram showing a part of the filter array arrangement is shown. Figure 1 The arrangement shown includes a 2x3 CFA pattern, where 8 unit cells 101 (or "super pixels", 3 of which are circled) are provided, and each unit cell itself includes 6 sensor pixels 103 (as shown numbered 1 to 6). The term "pixel" as used herein refers to the filter element of the filter array arrangement used in conjunction with an imaging device, and the two can be used interchangeably.

[0040] Therefore, each sensor pixel represents a different spectral filter element. Thus, Figure 1 The arrangement shown is capable of imaging up to 6 different channels, and each channel can be selected to generate data for a specific frequency band of the electromagnetic spectrum. As shown, each unit cell 101 is repeated in order to form a filter array arrangement, where a total of 48 sensor pixels are provided.

[0041] To implement the filter array arrangement provided by one example, Figure 1 The pattern of the sensor pixels shown can be rearranged as described below.

[0042] Figure 2 A schematic diagram showing a part of the filter array arrangement provided by one example is shown. Figure 2 The arrangement shown shows Figure 1 The rearrangement of the pixel order of the sensor pixels 103 shown. In this rearrangement, the channel order of the unit cells is changed such that channels having the same channel response are close to each other. Thus, in the first row 201 of the arrangement, the pixel order is 1, 2, 3, 3, 2, 1 (instead of Figure 1 the 1, 2, 3, 1, 2, 3 shown). As Figure 1 before, three unit cells are circled.

[0043] Figure 3a A schematic diagram showing a part of the filter array arrangement provided by one example is shown. In Figure 3a the arrangement shown, duplicate pixels are removed. That is, instead of just changing the channel order (asFigure 2 Rather than that shown, overlapping unit cells are generated by deleting duplicate pixels / compressing the pattern. This can be performed in one or more directions. For example, referring to Figure 2 , unit cells 203 and 205 are compressed by deleting pixels 207, 209 (or pixels 211, 213) to form a pair of unit cells 301, 303 that share pixels (i.e., Figure 3a the pixels 3 and 6 shown). This has the beneficial effect of reducing the size of the sensor using this arrangement. As with Figure 2 , three unit cells are circled. Figure 3a The same arrangement as on the left is shown on the right, but without circles so that the numbers can be seen more clearly. Figure 4 And Figure 5 The same is true.

[0044] Figure 3b A schematic diagram showing a part of a filtering array arrangement provided by an example is shown. Figure 3b The example shown is based on the example shown in Figure 3a to show how some channels may be overrepresented compared to other channels. In the example shown in Figure 3b , channels 2 and 5 are more frequent than other channels. This will be described in more detail below.

[0045] Figure 4 A schematic diagram showing a part of a filtering array arrangement provided by an example is shown. In the arrangement shown in Figure 4 , other duplicate pixels are deleted. That is, the pattern is compressed in another direction (orthogonal to the direction described in connection with Figure 3a ). In the example shown in Figure 4 , and in connection with Figure 3a , it can be seen that the duplicate pixels in row 305 are deleted to compress these two rows into one row (401). Again, three unit cells are circled to show the overlapping / shared pixels.

[0046] Figure 5 A schematic diagram showing a part of a filtering array arrangement provided by an example is shown. In the arrangement shown in Figure 5 (an implementation derived from the arrangement shown in Figure 3a ), compression is performed in one direction, and additional pixels 501 for one or more other channels are added (i.e., two additional channels are satisfied by adding a pixel for channel 7 and a pixel for channel 8). It should be noted that the number of pixels for each of these channels is less than the number of pixels for channels 1 to 6, because in some cases, presenting these channels at a relatively low resolution may be sufficient.

[0047] In some of the examples described above, some channels will occur more frequently than others. For example, in combination with Figure 2 , Figure 3a and Figure 3b , channels 2 and 5 occur more frequently than other channels. If additional channels are to be included, some of the filter elements that would be channels 2 or 5 can be replaced with alternative filter elements. Additionally, when removing duplicate rows or columns in only one direction, the sensor can support binning of adjacent channels with the same color.

[0048] By replacing some of the filter elements of the channels that occur more frequently compared to other channels (achieved by filter elements for additional channels of interest), the number of channels available to the sensor for generating information can be increased. For example, if Figure 2 or Figure 3b some of the filter elements of channels 2 and 5 as shown are replaced with channels "a" and / or "b", then 7 / 8 channels can be used, although it should be noted that since only some channels are replaced, channels "a" and / or "b" will provide a lower resolution than the remaining channels. In some applications / use cases, this may be acceptable. Additionally, if channels "a" and / or "b" are in different rows and columns of an array arrangement, as Figure 3c and Figure 3d shown, they will provide sampling of these channels over a larger area of the array, which may be beneficial as it will provide more accurate measurements for these channels compared to the case where the corresponding filter elements are clustered in a relatively small area of the array. In an alternative form, "a" and / or "b" can be in the same row, for example row 2 instead of rows 2 and 4, and for example row 6 instead of rows 6 and 8.

[0049] Consider another example, represented by a 4x4 CFA pattern, where each unit cell consists of 16 sensor pixels. That is, one unit cell includes 16 sensor pixels, covering up to 16 channels.

[0050] Figure 6 A schematic diagram showing a part of a filter array arrangement provided by an example is shown. Figure 6 Sensor pixels numbered 1 to 16 are shown, thereby forming unit cell 601, which is repeated to form filter array arrangement 600 (which includes a total of 256 sensor pixels). As an example, four unit cells are circled.

[0051] Similar to the example described above in combination with Figures 1 to 5 , the pixel order can be rearranged. According to one example, the channel order of the filter elements in adjacent unit cells is changed. After rearrangement, channels with the same channel response are close to each other. Thus, for example, in combination with Figure 6, rearrange the channel order such that in the first row 603, the order is 1, 2, 3, 4, 4, 3, 2, 1 (instead of Figure 5 1, 2, 3, 4, 1, 2, 3, 4 as shown). It should be noted that as Figure 6 shown, all rows are mirrored in a similar manner, and pixels 6 and 7 are also mirrored. Alternatively, pixels 6 and 7 are not mirrored (i.e., all channels except channels 6, 7, 10, 11 are mirrored).

[0052] After rearrangement, some duplicate pixels are removed. As Figure 7 shown, which shows a schematic diagram of a part of a filtered array arrangement provided by an example.

[0053] In Figure 7 , the highlighted number of pixels is reduced to 1 / 4 of its original value in columns and rows. In this way, overlapping unit cells can be provided, where some channels are shared between the unit cells, as shown by the overlapping circles. Sharing in this way reduces the height and width required for the sensor. In the example shown in Figure 7 , all pixels except pixels 6, 7, 10, and 11 will overlap. As shown, these pixels can be mirrored or not. Preferably, mirroring the pixels ensures that the surrounding pixels are more similar than not mirroring.

[0054] Therefore, a two-dimensional filtered array arrangement for use in conjunction with an imaging device is provided, where the filtered array arrangement is used to generate multi-band image data corresponding to an object or scene to be imaged. Thus, the imaging device can generate data representing an image cube of the object or scene. However, compared to a typical spectral sensor array, due to rearrangement and sharing, the sensor size can be significantly reduced while maintaining the ability to generate high-resolution data on the required channels, enabling use in user devices with a smaller form factor (e.g., a smartphone).

[0055] According to one example, the filtered array arrangement includes an M x N array of spectral filters. The filters are selected such that their spectral responses allow selected wavelengths of the electromagnetic spectrum to be transmitted to detectors aligned with the filters in order to generate data representing multiple channels. For example, these channels can include red, green, and blue channels, but can also include other channels that can be outside the human visual range, such as channels selected in the infrared and ultraviolet regions of the electromagnetic spectrum.

[0056] The filters are used to define a set of unit cells. For example, as described above in connection with Figures 2 to 7As described, some unit cells are circled. That is, the unit cell includes a mosaic pattern of filter elements, and this mosaic pattern can be repeated to form a filter array arrangement. Thus, for a filter array arrangement including an M x N array of spectral filter elements, each unit cell includes a P x Q sub-array of spectral filter elements, where P < M and Q < N. The spectral filter elements of the unit cell limit imaging to multiple selected regions of the electromagnetic spectrum.

[0057] According to one example, a first unit cell includes a set A of spectral filter elements of a filter array arrangement, and a second unit cell connected to and adjacent to the first unit cell includes a set B of spectral filter elements. The first unit cell and the second unit cell are arranged relative to each other such that there is a first set X of spectral filter elements and a second set Y of spectral filter elements, where X = {x|x ∈ (A ∩ B)}, Y = {y|y ∈ (A Δ B)}; where the first set X of spectral filter elements forms a repeating pattern of the filter array arrangement. In other words, Figure 7 As an example, a first unit cell 701 and a second unit cell 703 are connected. The first set X of filter elements of unit cells 701, 703 includes the filter elements of A that are also in B (or equivalently, all filter elements of B that are also in A). Thus, for Figure 7 the unit cells shown, X = {1 - 5, 8, 9, 12 - 16}, and Y = {6, 7, 10, 11}.

[0058] For edge unit cells 705, 707, X = {1 - 4, 8, 12 - 16}, Y = {5 - 7, 9 - 11}. The difference in the constituent filter elements of X for unit cells 701, 703, 705, 707 only stems from their positions in the array. That is, for example, unit cells 705 and 707 are arranged at the outer edges of the array; thus, due to fewer surrounding unit cells, the degree of filter element sharing is reduced.

[0059] For example, as Figure 7 can be seen, a repeating pattern of unit cells is provided in order to define the entire array arrangement. The unit cells can be repeated in the vertical and / or horizontal directions.

[0060] According to one example, a filter array arrangement including rows and columns of filter elements is thus configured such that at least some image rows have filter elements that have at least one other channel response on their left and right sides (e.g., Figure 3a , filter elements 1 and 2 (and 3 and 2) in row 1 (and rows 4, 5, and 8); 6 and 5 (and 4 and 5) in row 3 (and rows 7, etc.). This particular pixel is shared by multiple unit cells.

[0061] This can be generalized such that at least one column or row of the arrangement has a specific filter element, the specific filter element including at least corresponding left and right or top and bottom filter elements having another channel response; that is, the specific filter element is shared by a plurality of unit cells. If the specific channel is replaced to increase the scope in which the arrangement can be used, it may not be applicable.

[0062] Figures 8a to 8e A schematic diagram showing parts of a filter array arrangement provided by an example is shown. In Figures 8a to 8e the example of, Figures 8a to 8e is a representation based on a 4x4 mosaic pattern of channels (i.e., where the filter array pattern size is greater than 3x3, such as 3x4 or 4x4 or larger), and different variants that can be provided by reordering the remaining filter elements can be generated. For example, different orderings can be generated to determine the priority of different items: for example, allowing a more similar environment (which may be easier for making filters) or allowing a more uniform sampling, i.e., a similar distance between filter elements of the same channel. In one example, when reordering, mirroring is not performed in some pixel groups.

[0063] Combined with Figure 8a and Figure 8b the variants shown, it can be seen that at least some image rows have a specific filter element, the specific filter element including filter elements having another (different) channel response on its left and right (e.g., filter element 4 and filter element 3, such as filter element 1 and filter element 2 in row 1 (and row 7 and row 13); 16 and 15, 13 and 14 in row 4 (and row 10)). The specific filter element is shared by a plurality of unit cells.

[0064] At least some columns have a specific filter element, the specific filter element including filter elements having another (different) channel response above and below it (e.g., filter element 13 and filter element 9, such as filter element 1 and filter element 5 in column 1 (and column 7 and column 13); Figures 8a to 8d 4 and 8, 16 and 12 in column 4 (and column 10) shown. The specific filter element is shared by a plurality of unit cells. In one example, this can be generalized such that at least one column or row has a specific filter element having at least another (different) channel response to its left and right or top and bottom; that is, the specific filter element is shared by a plurality of unit cells. Thus, the "row rule" (as described in combination with Figure 8a and Figure 8b ) applies to Figure 8a and Figure 8b (e.g., as opposed to the rules shown in Figure 8c and Figure 8d ), while the "column rule" applies to all variants shown in Figures 8a to 8d .

[0065] In other words, the image array includes sub-patterns forming unit cells that overlap each other at least in one of the horizontal or vertical directions, where each second unit cell in the horizontal direction is mirrored, and each second unit cell in the (array arrangement's) vertical direction is mirrored. Each mirrored unit cell is mirrored such that at least 4 corner filter elements are mirrored (e.g., Figures 8a to 8e 1, 4, 13, 16 in the above example of Figures 2 to 5 and 1, 3, 4, 6 in the above example of

[0066] Figure 9 Fig. shows a schematic diagram of an imaging device provided by an example. The imaging device 900 is used to generate multi-band image data representing a spectral image cube of an object or scene to be imaged. The imaging device 900 includes an imaging sensor 901, and the imaging sensor 901 includes a two-dimensional sensor array 903. The sensor array 903 includes a grid of detector elements 905. Each detector element can generate an electrical signal in response to exposure to electromagnetic radiation such as light, UV, etc. In one example, some elements 905 may be sensitive only to a specific region of the electromagnetic spectrum.

[0067] The two-dimensional filter array arrangement 907 is positioned in front of the sensor array with respect to the optical path 909 of the imaging device. Alternatively, the filter array arrangement 907 can be embedded in the sensor array 903, for example, by fabricating a filter on top of the detector elements (i.e., pixels) to become part of the sensor array.

[0068] In Figure 9 the example shown, the two-dimensional filter array arrangement 907 is the two-dimensional filter array arrangement described above in connection with any one of Figures 2 to 8a to Figure 8e Each spectral filter element 907 of the two-dimensional filter array arrangement is positioned to coincide with a pixel of the sensor array 903. The imaging device includes a processor 911 for: generating multiple spectral band images of the object or scene, each spectral band image corresponding to image data generated by pixels restricted to an imaging region of the electromagnetic spectrum defined according to the spectral filter element 907. The imaging device 900 can form part of a user device 913.

Claims

1. A two-dimensional filter array arrangement for use in combination with an imaging device, characterized in that, The filter array arrangement is configured to generate multi-band image data corresponding to an object or a scene to be imaged, and at least a part of the filter array arrangement includes: An M x N array of spectral filters for defining a set of unit cells, each unit cell including a P x Q sub-array of spectral filters, where P < M and Q < N, and the spectral filters of the unit cell limit imaging to a plurality of selected regions of the electromagnetic spectrum; Wherein, a first unit cell includes a set A of spectral filters of the filter array arrangement, and a second unit cell connected to the first unit cell includes a set B of spectral filters. The first unit cell and the second unit cell are arranged relative to each other so as to include a first set X of spectral filters and a second set Y of spectral filters, where X = {x|x∈(A∩B)} and Y = {y|y∈(AΔB)}; Wherein, the first set X of the spectral filters forms a repeating pattern of the filter array arrangement, and one or more spectral filters in the second set of the spectral filters are replaced with alternative spectral filters, and the alternative spectral filters are configured to limit imaging to regions of the electromagnetic spectrum not covered by other spectral filters in the second set of the spectral filters.

2. The two-dimensional filter array arrangement according to claim 1, wherein The first set of the spectral filters forms a first repeating pattern in a first direction of the filter array arrangement.

3. The two-dimensional filter array arrangement according to claim 2, wherein The first set of the spectral filters forms a second repeating pattern in a second direction of the filter array arrangement, and the second direction is orthogonal to the first direction.

4. The two-dimensional filter array arrangement according to claim 2, wherein, The second set of the spectral filters forms a third repeating pattern in the first direction of the filter array arrangement.

5. The two-dimensional filter array arrangement according to claim 3, wherein The second set of the spectral filters forms a fourth repeating pattern in the second direction of the filter array arrangement.

6. The two-dimensional filter array arrangement according to any one of claims 1-5 above, characterized in that, The relative complement of A in B 7. The two-dimensional filter array arrangement according to any one of claims 1-5 above, characterized in that, n(.) refers to the number of elements in a set.

8. The two-dimensional filter array arrangement according to any one of claims 1 to 5, characterized in that The relative complement B\A of A in B is not equal to the relative complement A\B of B in A.

9. The two-dimensional filter array arrangement according to claim 8 above, characterized in that, The first unit cell and the second unit cell share a vertical boundary.

10. The two-dimensional filter array arrangement according to claim 8, characterized in that, The first unit cell and the second unit cell share a horizontal boundary.

11. The two-dimensional filter array arrangement according to claim 9 or 10, characterized in that, The shared boundary between the unit cells includes at least one spectral filter in the first set of the spectral filters.

12. The two-dimensional filter array arrangement according to claim 9 or 10 above, characterized in that, The arrangement of the spectral filters in the second set within the first unit cell and the second unit cell is symmetric with respect to a symmetry line defined by the shared boundary between the first unit cell and the second unit cell.

13. The two-dimensional filtering array arrangement according to claim 1, wherein The arrangement of the spectral filters in the second set within the first unit cell and the second unit cell is the same.

14. The two-dimensional filter array arrangement according to claim 1, wherein The arrangement of the spectral filters in the second set within the first unit cell and the second unit cell is different.

15. The two-dimensional filter array arrangement according to claim 1, wherein, The alternative spectral filters are provided in the first unit cell and / or the second unit cell.

16. An imaging device, characterized in that, The imaging device is configured to generate multi-band image data representing a spectral image cube of an object or a scene, and the imaging device includes: An imaging sensor including a two-dimensional sensor array; A two-dimensional filter array arrangement according to any one of the above claims, wherein the two-dimensional filter array arrangement is positioned in front of or embedded in the sensor array with respect to the optical path of the imaging device.

17. The imaging device according to claim 16, wherein, The two-dimensional sensor array includes a grid of photodetectors defining a pixel array of the imaging device, wherein each of the spectral filter elements of the two-dimensional filter array arrangement is positioned to coincide with a pixel.

18. The imaging device according to claim 16 or 17, characterized in that, Further included is: a processor for: generating a plurality of spectral band images of the object or the scene, each spectral band image corresponding to image data generated by pixels restricted to an imaging region of the electromagnetic spectrum defined according to the spectral filter elements.

19. A method for providing a filter array arrangement, characterized in that, The method includes: placing a first unit cell relative to a second unit cell according to the two-dimensional filter array arrangement according to any one of claims 1 to 15.

20. A user equipment, characterized in that, including an imaging device according to any one of claims 16 to 18.

Images