A sensing module for combined imaging and depth sensing, an optical system, a method and computer program

Abstract

A sensing module for combined imaging and depth sensing is provided by the present disclosure, the sensing module comprising: a first pixel array at a first layer of the sensing module, the first pixel array configured to acquire image data of a scene at a first wavelength range; a second pixel array at a second layer of the sensing module, the second pixel array configured acquire depth data of the scene at a second wavelength range; and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range. An optical system, a method and a computer program are also provided by the present disclosure.

Description

[0001] A SENSING MODULE FOR COMBINED IMAGING AND DEPTH SENSING, AN OPTICAL SYSTEM, A METHOD AND COMPUTER PROGRAM

[0002] BACKGROUND:

[0003] Field of the Disclosure

[0004] The present disclosure relates to a sensing module for combined imaging and depth sensing, an optical system, a method and computer program.

[0005] Description of the Related Art

[0006] The “background” description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present invention.

[0007] Image sensors can be used in order to acquire image data. That is, the individual pixels of an image sensor can be used to acquire image data of an environment (or scene). Image data can include still image data (such as a photo) or moving image data (such as a video). Image sensors can also be used to obtain image data related to depth information of a scene.

[0008] Image sensors can be included in sensing modules. Sensing modules are devices which contain at least one image sensor and one or more optical elements for focusing light onto the image sensor. Sensing modules may also include circuitry for performing image data acquisition and / or processing of image data from the image sensor.

[0009] Sensing modules are used in a wide range of devices, such as imaging devices. The sensing module may comprise a number of individual sensors which can be used to acquire data of an environment.

[0010] An example of a sensing module may include a sensing module comprising pixels configured to detect red, green and blue light (a so-called RGB sensor). However, a RGB sensor is one type of sensor which can be used in a sensing module. For example, a sensor may be configured to detect light corresponding to a different colour space, such as CYGM orYUV. Furthermore, another type of sensor which can be used in a sensing module may be a so-called depth sensor (which comprises pixels capable of detecting depth information of the scene). For some applications, including augmented reality or virtual reality applications, it may be advantageous to provide a sensing module with a number of different types of image sensors (i.e. a number of sensors formed of different types of pixels).

[0011] Optimal optical properties of a sensing module may vary depending on the intended usage of the sensing module. The optical properties of the sensing module can be configured through use of different optical elements forming an optical system of the sensing module.

[0012] However, when a sensing module comprises a plurality of different types of sensors, it can be very difficult to provide appropriate optical properties for the different types of sensors of the sensing module. This can impact the performance of the sensing module.

[0013] It is an aim of the present disclosure to address these issues.

[0014] SUMMARY:

[0015] Embodiments of the present disclosure are defined by the independent claims. Further aspects of the disclosure are defined by the dependent claims.

[0016] In accordance with embodiments of the disclosure, an improved sensing module is provided, capable of providing appropriate optical properties for different types of sensors forming part of the sensing module. This improves the performance of the sensing module. Moreover, the improved performance of the sensing module can be obtained without increasing the form factor of the sensing module.

[0017] The present disclosure is not particularly limited to these advantageous technical effects. Further technical effects will become apparent to the skilled person when reading the disclosure.

[0018] BRIEF DESCRIPTION OF THE DRAWINGS:

[0019] Figure 1 illustrates an example of an apparatus in accordance with embodiments of the disclosure;

[0020] Figure 2 illustrates an example of a stacked sensing module in accordance with embodiments of the disclosure;

[0021] Figure 3 illustrates an example pixel configuration of a sensing module in accordance with embodiments of the disclosure;

[0022] Figure 4 illustrates an example sensing module in accordance with embodiments of the disclosure; Figure 5 illustrates an example spectrally variable aperture in accordance with embodiments of the disclosure;

[0023] Figure 6 illustrates an example implementation of a sensing module in accordance with embodiments of the disclosure;

[0024] Figure 7 illustrates an example implementation of a sensing module in accordance with embodiments of the disclosure;

[0025] Figure 8 illustrates an example method in accordance with embodiments of the disclosure.

[0026] DESCRIPTION OF THE EMBODIMENTS:

[0027] The foregoing paragraphs have been provided by way of general introduction, and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings (wherein like reference numerals designate identical or corresponding parts throughout the several views).

[0028] Referring to Figure 1, an apparatus 1000 according to embodiments of the disclosure is shown. Typically, an apparatus 1000 according to embodiments of the disclosure is a computer device such as a personal computer or a terminal connected to a server. Indeed, in embodiments, the apparatus may also be a server. The apparatus 1000 is controlled using a microprocessor or other processing circuitry 1002. In some examples, the apparatus 1000 may be a portable computing device such as a mobile phone, laptop computer or tablet computing device.

[0029] The processing circuitry 1002 may be a microprocessor carrying out computer instructions or may be an Application Specific Integrated Circuit. The computer instructions are stored on storage medium 1004 which maybe a magnetically readable medium, optically readable medium or solid state type circuitry. The storage medium 1004 may be integrated into the apparatus 1000 or may be separate to the apparatus 1000 and connected thereto using either a wired or wireless connection. The computer instructions may be embodied as computer software that contains computer readable code which, when loaded onto the processor circuitry 1002, configures the processor circuitry 1002 to perform a method according to embodiments of the disclosure.

[0030] Additionally, an optional user input device 1006 is shown connected to the processing circuitry 1002. The user input device 1006 may be a touch screen or may be a mouse or stylist type input device. The user input device 1006 may also be a keyboard or any combination of these devices. A network connection 1008 may optionally be coupled to the processor circuitry 1002. The network connection 1008 may be a connection to a Local Area Network or a Wide Area Network such as the Internet or a Virtual Private Network or the like. The network connection 1008 may be connected to a server allowing the processor circuitry 1002 to communicate with another apparatus in order to obtain or provide relevant data. The network connection 1002 may be behind a firewall or some other form of network security.

[0031] Additionally, shown coupled to the processing circuitry 1002, is a display device 1010. The display device 1010, although shown integrated into the apparatus 1000, may additionally be separate to the apparatus 1000 and may be a monitor or some kind of device allowing the user to visualise the operation of the system. In addition, the display device 1010 may be a printer, projector or some other device allowing relevant information generated by the apparatus 1000 to be viewed by the user or by a third party.

[0032] As explained in the Background, sensing modules are used in a wide range of devices, including imaging devices. The types of sensors which are contained within the sensing module and the configuration of the sensing module may change depending on the specific application of the sensing module. For some situations, a sensing module containing a combination of different types of image sensors (formed of different types of pixels) may be required.

[0033] Consider, now, the example of Figure 2 of the present disclosure. Figure 2 of the present disclosure illustrates an example of a stacked sensing module in accordance with embodiments of the disclosure. A stacked sensing module is one example of a type of sensing module in accordance with embodiments of the disclosure. However, while the example of Figure 2 is shown with reference to a sensing module having a specific configuration of layers, it will be appreciated that the present disclosure is not particularly limited in this regard.

[0034] The stacked sensing module 2000 of the example of Figure 2 comprises an optical system 2002, an RGB pixel layer 2004 and a depth sensing pixel layer 2006. The RGB pixel layer 2004 is an example of a first pixel layer and the depth sensing pixel layer 2006 is an example of a second pixel layer in accordance with embodiments of the disclosure.

[0035] The optical system 2002 of the sensing module 2000 is shared by the RGB pixel layer 2004 and the depth sensing pixel layer 2006 of the sensing module 2000. That is, light from a scene reaches the RGB pixel layer 2004 and the depth sensing pixel layer 2006 having passed through the optical system 2002 of the sensing module. The optical system 2002 is configured in order to control the optical properties of the sensing module. For example, the optical system 2002 may be configured to focus light from the scene onto the RGB pixel layer 2004 and the depth sensing pixel layer 2006 of the sensing module.

[0036] Since the optical system 2002 is shared between the RGB pixel layer 2004 and the depth sensing pixel layer 2006, the optical properties of the sensing module 2000 with respect to the RGB pixel layer 2004 are the same as the optical properties of the sensing module 2000 with respect to the depth sensing pixel layer 2006 of the sensing module.

[0037] One advantage of combining image sensors of a first and second type in the same sensor module is to ease the fusion of their output signal. For example, with the sensing module 2000 of the example of Figure 2, there is no parallax error between the RGB image data from the RGB pixel layer 2004 and depth data from the depth sensing pixel layer 2006.

[0038] Furthermore, combining pixels of a first pixel type and a second pixel type in the same pixel module enables a form factor of a device to be reduced as compared to the use of separate sensing modules for the separate types of pixels. This is useful for applications which require a compact device (such as mobile devices or the like).

[0039] However, despite these advantages, it will be appreciated that different types of pixels may require different optical properties in order to achieve a desired or optimum level of performance.

[0040] For example, the RGB pixel layer 2004 and the depth sensing pixel layer 2006 of the sensing module 2000 in the example of Figure 2 may require different optical properties in order to achieve a desired or optimum level of performance.

[0041] Consider, now, the example of Figure 3 of the present disclosure. The example of Figure 3 illustrates an example pixel configuration of a sensing module in accordance with embodiments of the disclosure.

[0042] In this example, the pixels of the RGB pixel layer 2004 are arranged in a grid of pixels (or an array) having a pattern as illustrated in Figure 3. The pixels are the RGB pixel layer 2004 may be any suitable type of pixel for acquiring image data of the scene in a first wavelength range (such as a visible wavelength range of light). For example, the pixels of the RGB pixel layer may be complementary metal oxide semiconductor (CMOS) pixels. However, the present disclosure is not particularly limited in this regard. Alternatively, the pixels may be CCD pixels. In contrast, the depth sensing pixels are pixels which are capable of measuring depth data. For example, the depth sensing pixels may be configured to acquire information concerning the time of flight to an object. In examples, the depth sensing pixels in the depth sensing pixel layer 2006 may be pixels such as single photon avalanche detectors (SPAD) pixels or the like. Depth sensing pixels will be described in more detail later.

[0043] In this example, the pixel pitch of pixels in the RGB pixel layer is P pm. This may be significantly smaller than the pixel pitch of pixels in the depth sensing pixel layer 2006. For example, the depth pixel pitch in the example of Figure 3 is N *P pm (where N is any number greater than 1). In this specific example, N = 8, such that the pixel pitch of the pixels in the depth sensing pixel layer 2006 are 8 times the size of the pixels in the RGB pixel layer. However, this is merely one example and the present disclosure is not particularly limited in this regard. More generally, N can be any number greater than 1 (includes numbers smaller than 8 or much greater than 8).

[0044] While this specific example of Figure 3 shows a comparison between the pixel pitch of an RGB pixel and a depth sensing pixel, it will be appreciated that the present disclosure is not particularly limited in this regard. More generally, it will be appreciated that different types of pixels in different layers of a sensing module may have different pitch sizes. Moreover, even for a specific example of an RGB pixel and a depth sensing pixel (such as described with reference to Figure 3 of the present disclosure) the difference in pixel pitch is not limited to the specific examples shown. The difference in pixel pitch between the RGB and depth sensing pixel may be very different to that shown in this specific example.

[0045] While the example of Figure 2 and 3 has been described with reference to an RGB sensor, it will be appreciated that the present disclosure is not particularly limited in this regard. In examples, a different colour space may be used. For example, the pixel of the first pixel layer may detect light in a colour space such as CYGM. In examples, the pixels may detect light in a YUV colour space. Accordingly, the present disclosure is not particularly limited to the example of RGB pixels in the first pixel layer.

[0046] In some embodiments, the pixel pitch of pixels in the depth sensing pixel layer 2006 corresponds to a multiple of the pixel pitch of pixels in the RGB pixel layer 2004. In some embodiments, the multiple is an integer chosen between 2 and 32. However, the present disclosure is not particularly limited in this regard. In some examples, the pixels of the RGB pixel layer and the pixels of the depth pixel layer may be aligned. For example, in the example of Figure 2 and 3 there are 64 pixels of the RGB pixel layer aligned with one pixel of the depth pixel layer. However, the present disclosure is not particularly limited in this regard and many different example configurations and alignments between the pixels of the RGB pixel layer and the pixels of the depth pixel layer may be provided.

[0047] In examples, the pixel pitch (or form factor) may be different in the x-direction and the y- direction of the pixel. For example, a pixel pitch in the x-direction may be larger or smaller than a pixel pitch in the y-direction (for either the RGB pixels or the depth sensing pixels). A change in the form factor in this way may change the alignment between pixels of the RGB pixel layer and pixels of the depth pixel layer. For example, a depth pixel (of the depth pixel layer) may be aligned with X RGB pixels in the x-direction and Y RGB pixels in the y- direction (where X and Y are different integer values). Accordingly, while X and Y are both equal to 8 in the example of Figure 3, the present disclosure is not particularly limited in this regard and more generally the value of X and the value of Y may be different to each other.

[0048] In some embodiments, the pixel pitch of pixels in the depth sensing pixel layer 2006 is 8 times larger than the pixel pitch of pixels in the RGB pixel layer 2004 as illustrated on Figure 3. However, it will be appreciated that a difference in pixel pitch between the pixels in the different layers of the sensing module may have a significant impact on the optical system which is required. As an example, the difference in pixel pitch between the RGB pixel layer 2004 and the depth pixel layer 2006 may lead to a different ideal f-number (also referred to as f / # or f-stop) for the RGB pixel layer and the depth sensing pixel layer. The f-number is a measure of the ability of an optical system to gather light. That is, an f-number is calculated as the ratio of a lens’s focal length to its aperture diameter. In general, a higher f-number relates to a smaller aperture while a lower f-number relates to a larger aperture. A lens with a lower minimum f-number is sometimes referred to as a fast lens (as it has a larger aperture and can gather light more quickly).

[0049] It will be appreciated that, in general, there is a trade-off between f-number, resolution and optical depth for an optical system. Low f-number is important for radiometric efficiency of the optical system. However, the lower the f-number, the smaller the achievable resolution becomes. Furthermore, the lower the f-number, the smaller the achievable depth of field becomes.

[0050] In addition, there is a trade-off between pixel pitch and resolution. Smaller pixel pitch requires higher resolution, yet a smaller pixel pitch leads to a smaller depth of field.

[0051] In view of these trade-offs, it will be appreciated that a RGB pixel used for imaging has a higher resolution requirement and a lower f-number requirement (meaning that the f-number can be larger (as passive illumination can be used)). On the other hand, depth pixels used for acquiring depth data have a lower resolution requirement but a higher demand on the f- number (meaning that the f-number must be smaller). Accordingly, when combining pixel types in a sensing module - such as that illustrated in Figure 2 of the present disclosure - it is difficult to provide appropriate optical properties for the different types of pixels of the sensing module (as the different types of pixels may individually benefit from different optical characteristics of the optical system). This can impact the performance of the sensor module.

[0052] In view of these issues (and also those issues described in the Background), a sensing module, optical system, method and computer program are provided in accordance with embodiments of the disclosure.

[0053] <Sensing Module and Optical System>

[0054] Consider, now, Figure 4 of the present disclosure. Figure 4 illustrates an example sensing module in accordance with embodiments of the disclosure.

[0055] The sensing module 4000 of Figure 4 comprises a first pixel array 4002 at a first pixel layer of the sensing module 4000, a second pixel array 4004 at a second pixel layer of the sensing module 4000 and an optical system 4006.

[0056] The first pixel array 4002 at the first pixel layer of the sensing module is configured to acquire image data of a scene at a first wavelength range.

[0057] The second pixel array 4004 at the second pixel layer of the sensing module 4000 is configured to acquire depth data of the scene at a second wavelength range.

[0058] Furthermore, the optical system 4006 of the sensing module 4000 comprises a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range.

[0059] In this way, the sensing module 4000 can provide a different aperture for the first and the second pixel layer with a single (shared) optical system 4006, which improves the performance of the sensing module as the sensing module can provide appropriate optical properties for the different types of pixel layers forming part of the sensing module. Moreover, since the sensing module 4000 retains only a single shared optical system 4006, this improvement in performance of the sensing module can be achieved without increasing the form factor of the sensing module 4000. Therefore, a more compact sensing module having improved performance can be provided.

[0060] In examples, the pixels of the first pixel array 4002 may comprise a type of pixel such as light accumulation pixels (including, for example, CMOS type pixels). Each pixel of the first pixel array 4002 may be configured to detect light of the first wavelength range. In examples, the light of the first wavelength range may be visible (RGB) light, having a wavelength range of approximately 430nm - 650nm). In examples, a number of different filters may be provided over the different pixels of the first pixel array in order that the first pixel array can be used to capture image data corresponding to different wavelengths of light in this wavelength range (such as separate red, green and blue image data, for example).

[0061] The number of pixels in the first pixel array 4002 of the sensor module 4000 is not particularly limited in accordance with embodiments of the disclosure. However, it will be appreciated that the number of pixels in the first pixel array may be significantly more than illustrated in the example of Figure 3 of the present disclosure. Indeed, in examples, the first pixel array 4002 may comprise many millions of pixels. However, the present disclosure is not particularly limited to any specific number of pixels forming the first pixel array 4002.

[0062] In examples, the pixels of the second pixel array may be configured to detect light of a second wavelength range, where that second wavelength range is different from the first wavelength range. For example, when used to detect the depth information, the pixels of the second pixel array may be configured to detect light of an infrared wavelength range. In examples, light of an infrared wavelength range may include light of a wavelength range of 650nm to 1400nm. In examples, the second wavelength range may include light of approximately 940nm.

[0063] However, it will be appreciated that the second wavelength range is not specifically limited to these particular examples. More generally any suitable wavelength range may be used for the second wavelength range depending on the situation to which the embodiments of the disclosure are applied.

[0064] In examples, the pixels of the second pixel array are depth sensing pixels. In examples, the depth sensing pixels are configured to acquire depth data based on time-of-flight information.

[0065] In examples, the pixels of the second pixel array 4004 may comprise single photon detectors (i.e. pixels which can detect arrival of a single photon). In examples, the pixels of the second pixel array 4004 may include single photon avalanche diode detectors (SPAD). A SPAD comprises a photodiode with a reverse bias voltage higher than the breakdown voltage of the photodiode. A single photon incident on the SPAD can trigger a self-sustaining avalanche. Therefore, the SPAD is very sensitive with the ability to detect a single photon.

[0066] However, the present disclosure is not particularly limited in this regard. Any other type of pixel may be used within the depth sensing pixel layer of the sensing module as required depending on the situation to which the embodiments of the disclosure are applied. In examples, it may be desirable to use a pixel array comprising pixels capable of detecting arrival of a single photon to detect depth within an environment. That is, pixels capable of detecting arrival of a single photon (such as SPAD pixels) can be used in order to accurately determine time of arrival of a photon. Therefore, a time-of-flight can be used to determine depth, based on a time at which light emitted from a light emitter and reflected by an object in the scene arrives at the SPAD. The depth can be determined in accordance with the time it takes for the light to travel from the light emitter to an object and then back to the SPAD.

[0067] In examples, the light emitter may be configured to emit light of the second wavelength range (such as infrared light). In examples, the light emitter (not shown) may be part of the sensing module. In examples, a light emitter may be a vertical-cavity surface-emitting laser (VCSEL). However, the present disclosure is not particularly limited in this regard. In examples, any suitable type of light emitter can be used. Indeed, the light emitter (if included as part of the sensor module) may be any light source which can emit photons of light which can be used in order to measure depth in a scene (based on time of flight). In examples, the light emitter may emit light outside the visible range; this enables the light emitter to be used alongside a visible light sensor as part of the sensor module.

[0068] In examples, processing circuitry (not shown) may be provided as part of the sensing module.

[0069] In examples, the processing circuitry may be used in order to perform (or control) image data acquisition processing to acquire image data from the first and / or second pixel layer of the sensing module. Alternatively or in addition, in examples, the processing circuitry may be used to perform image processing.

[0070] In examples, the processing circuitry may use the data from the image sensor to generate the depth information. However, the way in which the processing circuitry generates the depth information using the data from the second pixel layer is not particularly limited in accordance with embodiments of the disclosure. For example, any suitable process or algorithm to produce depth information from time of arrival information can be used in accordance with the present disclosure depending upon the situation to which embodiments of the disclosure are applied.

[0071] Nevertheless, it will be appreciated that from time of arrival information (which can be obtained from the pixels of the second pixel layer of the sensing module) the time taken from emission of light (e.g. from a light emitter) to detection of the light at the pixels of the second pixel layer of the sensing module after reflection from an object in the scene can be determined. This information can be used in order to determine the distance to the object and thus depth information of the scene. The number of pixels in the second pixel array 4004 of the sensor module 4000 is not particularly limited in accordance with embodiments of the disclosure. However, it will be appreciated that the number of pixels in the second pixel array may be significantly more than illustrated in the example of Figure 3 of the present disclosure. However, the present disclosure is not particularly limited to any specific number of pixels forming the second pixel array 4004.

[0072] In examples, the optical system may further comprise at least one additional optical element configured to focus light from the scene onto the first pixel array and the second pixel array. In examples, the at least one additional optical element may comprise a reflective optical element (such as a mirror or a mirror system) or a refractive optical element (such as a lens or lens system). In examples, the at least one additional optical element may include an objective lens stack which can be combined with the spectrally variable aperture to create a different optical aperture for the different wavelength ranges of light.

[0073] In examples, the optical system may focus light from the scene onto the first pixel array. That is, when formed in a stacked sensing module, the optical system may be configured to focus light on the first pixel array (at the first layer of the sensing module). This is because the first layer of the stacked sensing module may be used in order to produce image data of the scene. Therefore, it is important that the light from the scene is focused on this first layer of the stacked sensing module. However, the second sensing layer may then be at a position from the in-focus position. Nevertheless, since the second sensing layer is used to acquire depth information, a small offset from the in-focus position will not impact the performance of the sensing module. Therefore, in examples, the optical system may be optimized such that the light from the scene is focused on the first layer of the stacked sensing module.

[0074] In examples, the top and bottom pixel layers (that is, the first pixel layer and the second pixel layer) may share a trench structure for guiding incident light from one layer to the next. Accordingly, in examples, light may be focused only on the first pixel layer in order to provide focused light for the first and second pixel layers (i.e. for the different spectral bandwidths of light).

[0075] In examples, the optical system may exhibit axial chromatic aberration such that different wavelengths of light are brought to a focus in different planes. For example, the optical system may be configured to exhibit axial chromatic aberration to place the focus of light of the first wavelength band at the first pixel layer and light of the second wavelength band at the second pixel layer. Accordingly, the manner by which the optical system focuses light from the scene onto the first and second pixel layer of the sensing module is not particularly limited in accordance with embodiments of the disclosure.

[0076] In examples, the optical system 4006 may be provided independently from the other elements of the sensing module which have been described with reference to Figure 4 of the present disclosure. In examples, an optical system (with a spectrally variable aperture as has been described) may be use with other example sensing modules than that described with reference to Figure 4 of the present disclosure. Accordingly, the present disclosure is not particularly limited to the combination of the sensing module and optical system as described with reference to Figure 4.

[0077] Accordingly, embodiments of the disclosure also provide an optical system comprising: a spectrally variable aperture configured to provide an aperture of a first aperture size for light from a scene of a first wavelength range acquired by a first pixel array of a sensing module and an aperture of a second aperture size for light from a scene of a second wavelength range acquired by a second pixel array of a sensing module.

[0078] As illustrated in Figure 4 of the present disclosure, at least two sensing layers may be present within the sensing module. Different types of pixels (for sensing different types of light) may be present at the different layers of the sensing module. Indeed, in embodiments of the disclosure, the first layer of the sensing module is stacked on the second layer of the sensing module and wherein light incident from the scene reaches the first layer of the sensing module and the second layer of the sensing module in that order. A stacked sensor (comprising these different sensor layers) enables the form factor of the sensing module to be further reduced.

[0079] Further details of the spectrally variable aperture of the optical system 4006 of the sensing module will now be described.

[0080] <Spectrally Variable Aperture>

[0081] As explained with reference to Figure 4 of the present disclosure, the optical system 4006 of the sensing module 4000, according to embodiments of the disclosure, comprises a spectrally variable aperture. The spectrally variable aperture enables the optical system of the sensing module to provide different optical characteristics for the pixels of the first and second sensing layer.

[0082] Consider, now, Figure 5 of the present disclosure. Figure 5 illustrates a spectrally variable aperture in accordance with embodiments of the disclosure (see the top panel of Figure 5 of the present disclosure). The spectrally variable aperture 5000 of the example of Figure 5 can be used in an optical system 4006 as part of a sensing module 4000 as described with reference to Figure 4 of the present disclosure.

[0083] The spectrally variable aperture 5000 may be part of a focusing element of the optical system 4006. In examples, the spectrally variable aperture 5000 may be formed as an aperture stop of the sensing module. As will be appreciated by the skilled person, an aperture stop is an element of an optical system (such as optical system 4006) which controls the amount of light reaching the image plane.

[0084] In examples, one or more optical properties of the optical system 4006 may be included as settings which are configurable. In examples, the one or more settings may be configurable by a user. In examples, the one or more settings may be configurable automatically (e.g. by processing circuitry of the sensor module and / or an imaging device or other computational device). In examples, the one or more settings may be configurable based on a combination of user input and processing circuitry. In examples, the processing circuitry may provide a range of recommended values for a setting from which the user can make a selection. In examples, the processing circuitry may automatically configure a number of the settings while a user may configure one or more other settings of the optical system.

[0085] In examples, the aperture stop may be an optical setting which can be configured (e.g. by a an aperture setting). In examples, a value of an aperture setting may be adapted in order to increase or decrease an aperture size for a specific wavelength range.

[0086] Furthermore, in examples, the spectrally variable aperture may not be provided as a separate independent element of the optical system 4006 of the sensing module. Rather, in examples, the sensing module may be formed as part of one or more optical elements of the optical system. In examples, the spectrally variable aperture may comprise a partial coating on one or more lens elements of optical system. In examples, the partial coating may include an annular coating on one or more lens elements of the optical system.

[0087] In accordance with embodiments of the disclosure, the spectrally variable aperture 5000 is an element that creates a different optical aperture for different wavelengths of light. For example, the spectrally variable aperture may create a different optical aperture for the RGB channel and the depth channel (where, in examples, the depth channel comprises light of an infrared wavelength range).

[0088] Since a difference in optical aperture can be created for light of different wavelengths, it becomes possible to control the optical aperture for the different layers of the sensing module. Accordingly, improved performance of the sensing module can be achieved since desired optical characteristics can be provided by the optical system 4006 for each of the layers of pixels (sensing layers) within the sensing module.

[0089] In this example, the spectrally variable aperture 5000 comprises a peripheral zone 5000B. The peripheral zone 5000B defines a central zone 5000A. Each of the peripheral zone 5000B and the central zone 5000A may have different optical characteristics. However, it will be appreciated that this configuration of the spectrally variable aperture 5000 is only one example of a spectrally variable aperture which can be used in accordance with embodiments of the disclosure. More generally, any spectrally variable aperture which can provide a different optical aperture for different wavelengths of light can be used in accordance with embodiments of the disclosure. The specific configuration of the spectrally variable aperture used in the sensing module may vary depending on the situation to which the embodiments of the disclosure are applied.

[0090] In the example of Figure 5, the spectrally variable aperture 5000 may be configured in order to provide a larger f-number (smaller aperture) for light of a first wavelength range compared to the f-number for light of a second wavelength range. The first wavelength range may be visible light (being light which can be detected by the RGB pixels of the RGB pixel layer of the sensing module). The second wavelength range may be infrared light (which can be detected by the depth sensing pixels of the depth sensing pixel layer of the sensing module).

[0091] For example, the peripheral zone 5000B may be configured as an optical low pass filter. The optical low pass filter may be configured such that light of longer wavelength (such as infrared light) is able to pass through the filter, while light of a shorter wavelength (such as light of the visible wavelength range) is unable to pass through the filter.

[0092] In examples, such as that illustrated in Figure 5 of the present disclosure, the peripheral portion zone may be an annular zone surrounding the central zone 5000A. However, the present disclosure is not particularly limited to this example configuration and other configurations of the spectrally variable aperture 5000 may be used in accordance with embodiments of the disclosure.

[0093] An example of transmission of light through the optical low pass filter 5000B is illustrated in the bottom panel of Figure 5 of the present disclosure. In this example, the optical low pass filter of the peripheral zone 5000B prevents transmission of the visible wavelength light (of the RGB wavelength range) and allows transmission of the infrared wavelengths of light (which can be detected by the depth sensing pixels).

[0094] In contrast, the central zone 5000A defined by the peripheral zone 5000B may allow transmission of light of all wavelength ranges (i.e. at least light of the visible wavelength range and the infrared wavelength range used by the pixels of the RGB pixel layer and the depth sensing pixel layer respectively).

[0095] In this way, the optical system 4006 of the sensing module 4000 can be configured to provide different optical characteristics for the light of the different wavelength ranges. The light of the visible wavelength range (RGB wavelength ranges of light) will experience a larger f-number (smaller aperture) since light of this wavelength range can pass only through the central zone 5000A of the spectrally variable aperture 5000. On the other hand, light of the infrared wavelength range (used for depth detection in this example) will experience a smaller f-number (larger aperture) since light of this wavelength range can pass through both the central zone 5000A and the peripheral zone 5000B of the spectrally variable aperture 5000.

[0096] Returning to the trade-offs mentioned earlier, this means that it is possible to provide a larger f-number (smaller aperture) for the RGB pixels of the RGB pixel layer of the sensing module. These pixels have a smaller form factor (compared to the depth pixels) and use of the larger f-number (smaller aperture) means that a higher resolution is achievable. On the other hand, the depth sensing pixels of the depth sensing pixel layer can experience a smaller f-number (larger aperture). These pixels have a larger form factor (compared to the RGB pixels) and use of the smaller f-number (larger aperture) means that radiometric efficiency can be improved.

[0097] Thus, it becomes possible to optimize the optical properties of the optical system for the different pixel layers of the sensing module. For example, it becomes possible to optimize sharpness for the first pixel layer (used to generate an image of the scene) while optimizing radiometric transfer for the second pixel layer (used to generate depth information of the scene).

[0098] Hence, more generally, a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range is provided in accordance with embodiments of the disclosure. Indeed, in examples, the spectrally variable aperture may be configured to provide a greater aperture (smaller f-number) for light of the second wavelength range and a lower aperture (higher f-number) for light of the first wavelength range.

[0099] As explained, the configuration of the spectrally variable aperture is not particularly limited to that specific configuration illustrated with reference to Figure 5 of the present disclosure. More generally, any suitable configuration of the spectrally variable aperture may be used as required depending on the situation to which the embodiments of the disclosure are applied. In this example, the spectrally variable aperture is configured to provide a different aperture size for two different wavelength ranges of light. However, the present disclosure is not particularly limited in this regard. More generally, the spectrally variable aperture may be configured to provide different aperture sizes for a number of different wavelength ranges of light.

[0100] Furthermore, the relative size of the peripheral zone and the central zone illustrated in Figure 5 of the disclosure is merely one such example. The actual relative size of the zones may vary depending on the situation to which the embodiments of the disclosure are applied. For example, the relative size of the peripheral zone and the central zone of the spectrally variable aperture required may depend on the difference in the pitch size of the different types of pixels contained within the sensing module.

[0101] The way in which the low pass filter of the spectrally variable aperture is created or formed is not particularly limited in accordance with embodiments of the disclosure. However, in some examples, the spectrally variable aperture may comprise an optical element that is coated with an optical filtering element on the peripheral zone. In examples, the coating may include the use of a thin-film. In examples, the thin-film may include use of a multi-layer thin-film technology. However, in examples, the coating may be formed through deposition of one or more materials with a combination of spectral transmission properties. The specific combination of materials used to create the coating in such an example are not particularly limited and may vary in accordance with the specific spectral transmission properties which are required.

[0102] Use of an optical coating on an optical element in order to form the different zones of the spectrally variable aperture enables the different zones to be formed without increasing the size (thickness) of the spectrally variable aperture. Therefore, use of an optical coating to form the different zones of the spectrally variable aperture may be advantageous when certain restrictions exist on the size of the sensing module.

[0103] In examples, the spectrally variable aperture may be formed as a partial coating on one or more optical elements of the optical system. For example, the spectrally variable aperture may be formed by a partial (in some examples, annular) coating on one or more lenses of the optical system.

[0104] In examples, the spectrally variable aperture may be formed using a chemical and / or mechanical processes. For example, the spectrally variable aperture may be formed by a process of chemically or mechanically etching an optical element of the optical system. In examples, an annular spectrally variable aperture may be created through chemical or mechanical removal of a central portion of a filter, for example. In examples, one or more optical properties of the spectrally variable aperture may be variable (and thus not constant) across a surface (such as a diameter) of the spectrally variable aperture. For example, the transmission of the spectrally variable aperture does not need to be uniform across the diameter of the spectrally variable aperture. In examples, the boundary between the different transmission regions (having different transmission properties) of the spectrally variable aperture may be formed such that there is a smooth transition between transmission regions. A non-uniform spectrally variable aperture may provide even greater control over the properties of the optical system.

[0105] <Example lmplementations>

[0106] Turning now to Figure 6 of the present disclosure, an example implementation of a sensing module is illustrated. The sensing module is a sensing module in accordance with embodiments of the disclosure. That is, the sensing module is a sensing module comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range.

[0107] The top panel of Figure 6 illustrates the path of light through the sensing module for a wavelength range of 430nm-650nm (an example of a visible (or RGB) wavelength range). The bottom panel of Figure 6 illustrates the path of light through the same sensing module for light of a wavelength range of approximately 940nm (an example of an infrared wavelength or light which can be used for depth sensing).

[0108] The sensing module comprises a first collection of lenses 6000, a spectrally variable aperture 5000 and a second collection of lenses 6002. A sensing portion 6004 is also provided. An arrangement such as that illustrated in the example of Figure 6 (where the spectrally variable aperture 5000 is provided between the first and second collection of lenses) may be used in a wide angle lens, for example.

[0109] Taken together, the first collection of lenses 6000, the spectrally variable aperture 5000 and the second collection of lenses of the sensing module form an example implementation of an optical system such as optical system 4006 as described with reference to Figure 4 of the present disclosure. These elements of the sensing module are configured to focus light from the scene and control the amount of light from the scene reaching the sensing portion 6004.

[0110] Furthermore, the sensing portion 6004 includes a first pixel layer and a second pixel layer. The first pixel layer comprises pixels for detecting light of the RGB wavelength range and can be used in order to form an image of the scene. The second pixel layer comprises pixels for detecting light of the infrared wavelength range and can be used in order to obtain depth information (i.e. how far an object in the scene (or a part of the scene) is from the sensing module).

[0111] The sensor module of this example is configured such that the sensor module has an f- number of 1.83 for light of the wavelength range 430nm-650nm. In particular, light from the scene of the wavelength range 430nm-650nm can pass only through a central zone of the spectrally variable aperture. This reduces the effective aperture size (i.e. increases the effective f-number) of the aperture for light of this wavelength range.

[0112] In contrast, the same sensor module is configured such that the sensor module has an f- number of 1.26 for light of the wavelength range of approximately 940nm. That is, light of the wavelength range of approximately 940nm can pass through both the central zone and a peripheral zone of the spectrally variable aperture. This increases the effective aperture size (i.e. decreases the effective f-number) of the aperture for light of this wavelength range.

[0113] Indeed, with the increase in aperture size (decrease in f-number) which can be achieved with the spectrally variable aperture of the present disclosure, light of the infrared wavelength range has approximately twice the irradiance of light of the visible (RGB) wavelength range.

[0114] With this configuration, light of the visible (RGB) wavelength range has a Nyquist limit of 111 Ip / mm and an MTF at the Nyquist limit of approximately 70%.

[0115] On the other hand, light of the infrared wavelength range (used for depth sensing) has a Nyquist limit of 12.5 Ip / mm and an MTF at the Nyquist limit of approximately 70%.

[0116] Therefore, a similar level of performance of the sensing module can be achieved for light of the visible wavelength range and light of the infrared wavelength range. Moreover, the level of performance which can be achieved is improved compared to legacy sensing modules (since the optical system can be optimized for both the first and the second pixel layer individually).

[0117] Consider, now, Figure 7 of the present disclosure. Figure 7 of the present disclosure illustrates an example implementation of a sensing module in accordance with embodiments of the disclosure.

[0118] The sensing module illustrated in Figure 7 is a sensing module in accordance with embodiments of the disclosure. That is, the sensing module is a sensing module comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of a second wavelength range. The implementation of the sensing module illustrated in Figure 7 of the present disclosure is an example implementation which may be used for use in a mobile device (or other type of device which requires a more compact form factor). An example of a mobile device includes a portable computing device (such as a mobile phone) as described with reference to Figure 1 of the present disclosure.

[0119] In this example, the sensing module comprises an objective lens stack 7000. Furthermore, a spectrally variable (i.e. wavelength dependent) aperture 5000 is formed as the aperture stop of the sensing module. In this example, the aperture stop is formed in front of the objective lens stack 7000. In this example, a glass cover 7004 is provided at the front of the objective lens stack 7000 of the sensing module. The glass cover protects the objective lens stack 7000 from foreign objects (such as dust or the like).

[0120] As illustrated in Figure 7, in examples a series of 5 extended aspherical lenses may be provided as part of the objective lens stack 7000. The 5 extended aspherical lenses are used in order to focus light from the scene onto the image plane. However, the present disclosure is not particularly limited in this regard. Any suitable arrangement of lenses may be provided within the objective lens stack 7000 as required depending on the situation to which the embodiments of the disclosure are applied. In examples, the objective lens stack 7000 may comprise a plurality of molded plastic elements which are used to focus the light onto the sensor.

[0121] In this example, the first and second pixel layers of the sensing module are not shown. However, the first and second pixel layers of the sensing module may be located at the image plane of the sensing module.

[0122] Since the spectrally variable aperture is provided as part of the sensing module of the example implementation of Figure 7, the sensing module can provide a different aperture for the first and second pixel layer with a single (shared) optical system 4006, which improves the performance of the sensing module as the sensing module can provided appropriate optical properties for the different types of pixels forming part of the sensing module. Moreover, since the objective lens stack 7000 comprises a plurality of aspheric lenses, the form factor of the sensing module can be further reduced. In examples, the spectrally variable aperture may be formed as a coating on one or more of the lenses of the objective lens stack 7000 (and not as a separate element of the optical system). This may further reduce the form factor of the sensing module.

[0123] While a number of example implementations of the sensing module in accordance with embodiments of the disclosure have been illustrated with reference to Figure 6 and Figure 7 of the present disclosure, it will be appreciated that the present disclosure is not particularly limited to these specific example implementations. More generally, embodiments of the disclosure provide a sensing module for combined imaging and depth sensing, the sensing module comprising: a first pixel array at a first layer of the sensing module, the first pixel array configured to acquire image data of a scene at the first wavelength range; a second pixel array at a second layer of the sensing module, the second pixel array configured acquire depth data of the scene at the second wavelength range; and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range.

[0124] <Method>

[0125] Hence, more generally, a method of combining imaging and depth sensing is provided in accordance with embodiments of the disclosure.

[0126] Figure 8 illustrates an example method in accordance with embodiments of the disclosure. The example method of Figure 8 may be performed by an apparatus such as that described with reference to Figure 1 of the present disclosure with a sensing module as described with reference to Figure 4 of the present disclosure.

[0127] In particular, in examples, the method of Figure 8 may be a method of combined imaging and sensing module with a sensing module having a first pixel array at a first layer of the sensing module, a second pixel array at a second layer of the sensing module and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of a first wavelength range and an aperture of a second aperture size for light from the scene of a second wavelength range.

[0128] However, the embodiments of the present disclosure are not particularly limited in this respect.

[0129] The example method of Figure 8 starts at step S8000 and proceeds to step S8002.

[0130] In step S8002, the method comprises acquiring image data of the scene at the first wavelength range with the first pixel array.

[0131] In step S8004, the method comprises acquiring depth data of the scene at the second wavelength range with the second pixel array.

[0132] The method then proceeds to and ends with step S8006.

[0133] The present disclosure is not particularly limited to the example illustration of the method as described with reference to Figure 8. For example, one or more additional steps may be included within a method in accordance with embodiments of the disclosure. Moreover, the present disclosure is not limited to the order of the steps as described with reference to Figure 8 of the present disclosure.

[0134] <Computer Program>

[0135] Furthermore, it will be appreciated that the methods of the present disclosure may be carried out on conventional hardware (such as that described previously herein) suitably adapted as applicable by software instruction or by the inclusion or substitution of dedicated hardware.

[0136] Thus, the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, PROM, RAM, flash memory or any combination of these or other storage media, or realized in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device.

[0137] Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks.

[0138] <Clauses>

[0139] In addition, embodiments of the present disclosure can be arranged in accordance with the following numbered clauses:

[0140] 1. A sensing module (4000) for combined imaging and depth sensing, the sensing module comprising: a first pixel array (4002) at a first layer of the sensing module, the first pixel array configured to acquire image data of a scene at a first wavelength range; a second pixel array (4004) at a second layer of the sensing module, the second pixel array configured acquire depth data of the scene at a second wavelength range; and an optical system (4006) comprising a spectrally variable aperture (5000) configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range.

[0141] 2. The sensing module according to clause 1, wherein the pixels of the second pixel array are depth sensing pixels. 3. The sensing module according to clause 2, wherein the depth sensing pixels are configured to acquire depth data based on time-of-flight.

[0142] 4. The sensing module according to any preceding clause, wherein a pitch of the pixels of the second pixel array is larger than a pitch of the pixels of the first pixel array.

[0143] 5. The sensing module according to any preceding clause, wherein the spectrally variable aperture is formed as an aperture stop of the optical system.

[0144] 6. The sensing module according to any preceding clause, wherein the optical system further comprises at least one optical element configured to focus light from the scene onto the first pixel array and the second pixel array.

[0145] 7. The sensing module according to any preceding clause, wherein the first layer of the sensing module is stacked on the second layer of the sensing module and wherein light incident from the scene reaches the first layer of the sensing module and the second layer of the sensing module in that order.

[0146] 8. The sensing module according to any preceding clause, wherein the spectrally variable aperture comprises a peripheral zone defining a central zone and wherein the peripheral zone is configured as a low-pass filter.

[0147] 9. The sensing module according to any preceding clause, wherein the spectrally variable aperture is configured to provide a greater aperture for light of the second wavelength range and a lower aperture for light of the first wavelength range.

[0148] 10. The sensing module according to any preceding clause, wherein the spectrally variable aperture is coated with an optical filtering element on the peripheral zone.

[0149] 11. The sensing module according to any preceding clause, wherein the optical system is configured to focus light from the scene on the first layer of the sensing module.

[0150] 12. The sensing module according to any preceding clause, wherein the first wavelength range includes visible light and the second wavelength range includes infrared light.

[0151] 13. The sensing module according to any preceding clause, wherein the spectral variable aperture comprises a partial coating on one or more lens elements of optical system.

[0152] 14. A method of combined imaging and depth sensing, the method comprising, with a sensing module having a first pixel array at a first layer of the sensing module, a second pixel array at a second layer of the sensing module and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from a scene of a first wavelength range and an aperture of a second aperture size for light from the scene of a second wavelength range, acquiring image data (S8002) of the scene at the first wavelength range with the first pixel array; and acquiring depth data (S8004) of the scene at the second wavelength range with the second pixel array.

[0153] 15. A computer program comprising instructions which, when implemented by a computer, cause the computer to perform the method of clause 14.

[0154] 16. An optical system comprising: a spectrally variable aperture configured to provide an aperture of a first aperture size for light from a scene of a first wavelength range acquired by a first pixel array of a sensing module and an aperture of a second aperture size for light from a scene of a second wavelength range acquired by a second pixel array of a sensing module.

[0155] It will be appreciated that numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure may be practiced otherwise than as specifically described herein.

[0156] In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing apparatus, it will be appreciated that a non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.

[0157] It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and / or processors may be used without detracting from the embodiments.

[0158] Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more data processors and / or digital signal processors. The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and / or processors.

[0159] Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the technique.

Claims

CLAIMS:1) A sensing module for combined imaging and depth sensing, the sensing module comprising: a first pixel array at a first layer of the sensing module, the first pixel array configured to acquire image data of a scene at a first wavelength range; a second pixel array at a second layer of the sensing module, the second pixel array configured acquire depth data of the scene at a second wavelength range; and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from the scene of the first wavelength range and an aperture of a second aperture size for light from the scene of the second wavelength range.2) The sensing module according to claim 1 , wherein the pixels of the second pixel array are depth sensing pixels.3) The sensing module according to claim 2, wherein the depth sensing pixels are configured to acquire depth data based on time-of-flight.4) The sensing module according to claim 2, wherein a pitch of the pixels of the second pixel array is larger than a pitch of the pixels of the first pixel array.5) The sensing module according to claim 1 , wherein the spectrally variable aperture is formed as an aperture stop of the optical system.6) The sensing module according to claim 1 , wherein the optical system further comprises at least one optical element configured to focus light from the scene onto the first pixel array and the second pixel array.7) The sensing module according to claim 1 , wherein the first layer of the sensing module is stacked on the second layer of the sensing module and wherein light incident from the scene reaches the first layer of the sensing module and the second layer of the sensing module in that order.8) The sensing module according to claim 1 , wherein the spectrally variable aperture comprises a peripheral zone defining a central zone and wherein the peripheral zone is configured as a low-pass filter.9) The sensing module according to claim 1 , wherein the spectrally variable aperture is configured to provide a greater aperture for light of the second wavelength range and a lower aperture for light of the first wavelength range.2510) The sensing module according to claim 1 , wherein the spectrally variable aperture is coated with an optical filtering element on the peripheral zone.11) The sensing module according to claim 1 , wherein the optical system is configured to focus light from the scene on the first layer of the sensing module.12) The sensing module according to claim 1 , wherein the first wavelength range includes visible light and the second wavelength range includes infrared light.13) The sensing module according to claim 1 , wherein the spectral variable aperture comprises a partial coating on one or more lens elements of optical system.14) A method of combined imaging and depth sensing, the method comprising, with a sensing module having a first pixel array at a first layer of the sensing module, a second pixel array at a second layer of the sensing module and an optical system comprising a spectrally variable aperture configured to provide an aperture of a first aperture size for light from a scene of a first wavelength range and an aperture of a second aperture size for light from the scene of a second wavelength range, acquiring image data of the scene at the first wavelength range with the first pixel array; and acquiring depth data of the scene at the second wavelength range with the second pixel array.15) A computer program comprising instructions which, when implemented by a computer, cause the computer to perform the method of claim 14.16) An optical system comprising: a spectrally variable aperture configured to provide an aperture of a first aperture size for light from a scene of a first wavelength range acquired by a first pixel array of a sensing module and an aperture of a second aperture size for light from a scene of a second wavelength range acquired by a second pixel array of a sensing module.

Images