Projection device

Abstract

A device for 3D imaging using structured light, including an array of individually addressable light emitting modules (308), where each light emitting module projects light with a different pattern (224). One or more cameras (314a, 314b) detect the projected light from each light emitting module. The array of light emitting modules and the cameras are integrated in a common housing (336).

Description

[0001] Projection Device

[0002] This invention relates to the imaging of 3D objects. Structured light 3D cameras typically project structured light patterns, for example vertical stripes and dots, onto a scene. A camera (or pair of cameras) detects perturbations in the projected patterns in order to determine distance, depth and surface information of the objects in the scene. Such 3D imaging systems have a variety of applications including robotics vision, industrial inspection, gaming, and mapping. There are also small-scale 3D imaging systems, e.g. dental scanners and optical profilometers.

[0003] In order to image an entire scene, many patterns need to be projected. For example, a typical structured light projector illuminates a spatial light modulator, and projects the resulting image. This type of system is typically large and complex, has a fixed focal distance, and suffers losses for sinusoidal and especially sparse patterns. Laser triangulation systems sweep a single light pattern (e.g. a single line) across a scene, where many images are acquired to form a 3D map. Other systems, for example those using micromirror interferometer projectors, are more efficient, but are limited to projecting sinusoidal patterns.

[0004] From a first aspect, the invention provides a system for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules, wherein each light emitting module is arranged to project light with a different pattern; one or more cameras arranged to detect the projected light from each light emitting module; and a processing portion arranged to form a depth map based on the projected light detected by the one or more cameras.

[0005] From a second aspect, the invention provides a device for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules arranged on a substrate, wherein each light emitting module is arranged to project light with a different pattern. Thus it will be seen that, in accordance with the invention, an array of individually addressable light emitting modules is provided, where each light emitting module is able to project a different light pattern. For example, the different light patterns may be different patterns of dots, stripes, or any other desired pattern. As will be appreciated by the skilled person, projecting different light patterns from the same device may allow for a scene to be imaged in 3D without the need for scanning back and forth to cover the whole scene. Each light emitting module is individually addressable, and thus the system is able to quickly change the pattern of light being projected by changing which light emitting module is on or off. Therefore, the present invention may allow for projection of structured light onto a scene faster and more efficiently than currently available devices, as the device is not limited by the speed of scanning of the projected pattern.

[0006] Furthermore, different light emitting modules may be able to simultaneously project different light patterns, thus allowing for the projection of a composite pattern which is complex with the same speed and efficiency as a simple pattern. The array of light emitting modules may also allow for a compact projector size compared to other projectors such as a spatial light modulator or laser raster scanner, as the projected patterns are encoded into the hardware of the array of light emitting modules. Sparse patterns may also be projected more efficiently than with existing devices, e.g. spatial light modulators. The patterns projected from different light emitting modules may be spatially separated or partially or fully superimposed on one another. This may allow for the system to be used in a variety of circumstances, whether the required projected pattern to accurately image a scene is dense or sparse, or requires different sinusoidal intensity profiles.

[0007] The camera(s) is arranged to detect the projected light which will be reflected from objects present in the scene being imaged. The processing portion is arranged to form a depth map using the detected projected light, for example by comparing the light patterns detected by the one or more cameras to how the light patterns would appear if there were no objects present, e.g. as if the light were being projected normally onto a flat surface. The light patterns will be perturbed by interaction with any objects in the scene being imaged, and the perturbations may be used by the processing portion to determine the shape and depth of objects in the scene. By projecting different light patterns, different features can be picked out from the scene and a depth map formed.

[0008] Although a single camera may be used to image the projected pattern, preferably at least two cameras are arranged to detect the projected pattern. This may facilitate imaging - e.g. by allowing ambiguities to be resolved using different views of the projected pattern. For example, the projector may not need any calibration once set up, as the projected pattern and hence depth map may be deduced from the correlation between the sets of images detected by each camera, e.g. the projector may be able to self-calibrate.

[0009] The array of light emitting modules may be arranged to emit different patterns (e.g. from different light emitting modules) in a sequence. This may enable any ambiguities in a scene to be resolved.

[0010] In a set of embodiments, each light emitting module is arranged to emit patterns with different spatial frequencies. For example, each light emitting module may be arranged to project a series of lines or dots, wherein the lines or dots have a different spacing. For example, patterns with between 10-100 vertical stripes may be used, with different patterns being projected and detected sequentially. Preferably, the system is arranged such that every part of a scene is illuminated by a plurality of the different patterns of light. For example, to cover a scene with said vertical lines and resolve any ambiguities, four patterns with different spatial frequencies may be projected.

[0011] The array of light emitting modules may be a 2D array, wherein each light emitting module is arranged on a plane. This arrangement may allow for the light emitting modules to be fabricated easily with a precise, known placement, for example using lithography.

[0012] In a set of embodiments, each light emitting module comprises at least one light source and an optical element. Each light source may be arranged to direct light at the respective optical element, where the optical element generates the light pattern from the light. In a set of such embodiments, the array of light emitting modules comprises an array of light sources and a corresponding array of optical elements. There may be a respective optical element for each light source or set of light sources. For example, the array of light sources and array of optical elements may map onto one another.

[0013] The light source may comprise an LED, laser (e.g. an edge-emitting laser), or any other suitable light source for projecting onto a scene as will be appreciated by the skilled person. Preferably, the light source comprises a vertical-cavity surface-emitting laser (VCSEL). VCSELs are efficient, and emit light with a narrow emission angle and a narrow spectral width, which may allow for sharp, collimated patterns to be projected. Furthermore, VCSELs may be modulated at high speeds, which may be useful if the light emitting modules are being turned on and off quickly in sequence, in order to quickly project a range of patterns e.g. with different spatial frequencies.

[0014] Additionally, the cost of integrating an array of VCSELs, e.g. arranging a plurality of VCSELs onto the same substrate, e.g. in an integrated circuit, may be lower than assembling individual light sources arranged on different integrated circuits. Furthermore, an array of VCSELs may be smaller than other light source arrangements, e.g. when compared to an array of LEDs.

[0015] The optical element in each light emitting module may be arranged above the light source, e.g. in the direction of projection. For example, the light sources may be arranged in a 2D array, and the optical elements may be arranged as a corresponding 2D array arranged on top of the light sources, in a plane parallel to and spaced from the plane of the light sources. The optical elements may be stacked on top of the respective light sources during fabrication of the array of light emitting modules, such that there may be precise alignment between each light source and its respective optical element.

[0016] The optical elements may comprise a lens, wherein the lens is arranged to focus the light from the respective light source(s) of each light emitting module.

[0017] In a set of embodiments, at least one (preferably every one) of the optical elements is a diffractive optical element. The diffractive optical elements of different light emitting modules may be arranged to project different patterns. The respective diffractive optical elements may be selected in order to project particular patterns, e.g. by using a micro-structured surface relief profile to shape the light transmitted from the light source into a desired distribution by diffraction. For example, the diffractive optical elements may be fabricated to project particular patterns suitable for a given application, e.g. sparse arrays of dots for facial recognition. A variety of different patterns, e.g. with different spatial frequencies, may be used to image a complex scene, whilst only a small number of patterns may be used to image simpler scenes, or for lower resolution imaging.

[0018] In a set of embodiments the diffractive optical element(s) comprise a plurality of phasedelaying levels. For example, a single pattern projected from an individual diffractive optical element may be made up of a plurality of differently phase-shifted sub-patterns. This may give the depth map formed by the processing portion increased resolution and / or may allow for further tailoring of the projected pattern.

[0019] The diffractive optical element(s) may act as a beam-splitter encoded as a phase mask. For example, in a simple diffractive optical element, the light source is split into two virtual source points. The interference between the virtual source points creates a diffraction pattern which has a sinusoidal intensity variation, and a spatial frequency proportional to the separation between the virtual source points. Multiple virtual source points may be created with a single diffractive optical element, or multiple adjacent (e.g. stacked) diffractive optical sub-elements, where the resulting projected pattern may be determined by the interference of all of the combined light paths from the virtual source points to the object upon which the light is being projected.

[0020] In a set of embodiments, the projected light from each light emitting module is collimated in a first direction, and divergent in a second direction, wherein the first direction and second direction are perpendicular. For example, each light emitting module may project a pattern of lines. A line pattern may be advantageous as fewer different patterns, e.g. with different spatial frequencies, may be required to illuminate the entire scene.

[0021] The light source(s) of each light emitting module may be aligned with the optical element, in order for optimal transmission of light through the optical element. For example, the light source(s), may be arranged on a substrate to form the array of light emitting modules using lithography. Fabrication of the light emitting modules using lithography may allow for precise positioning of the light source(s) relative to the respective optical element. For example, where the light source(s) comprise one or more VCSELs, the VCSELs may be positioned with between 50-100 pm separation. This may additionally allow for more VCSELs to be positioned in a single device, thus increasing the intensity of emitted light, and / or the variety of patterns that may be projected.

[0022] The optical element, for example the diffractive optical element, may be integrated into each light emitting module. For example, the light sources may be integrated into a light source array in a first layer, and the respective optical elements may be integrated into an optical element array in a second layer, where the second layer is arranged above the first layer to form a stack providing the array of light emitting modules. This arrangement may allow for the light transmitted through each respective optical element to be maximised.

[0023] The optical elements may comprise any combination of lenses and / or mirrors, and / or diffractive optical elements. For example, the optical elements may each comprise several diffractive optical elements in series. This may allow for the light pattern emitted from the optical element to be more precisely selected and / or refined.

[0024] It will be appreciated by the skilled person that while the light emitting modules may be spaced from one another - e.g. on a common substrate, at the working distance from the device for 3D imaging, such a spacing will typically have a negligible difference on the detection of the projected pattern.

[0025] In a set of embodiments, each light emitting module comprises a plurality of primary emitters, wherein each primary emitter is arranged to project a light pattern at a different lateral position to another primary emitter of the plurality of primary emitters, e.g. an adjacent primary emitter. Each primary emitter may comprise at least one light source, e.g. at least one VCSEL. However, there may only be one optical element for each light emitting module, i.e. the primary emitters of a light emitting module could share an optical element. Therefore, whilst each light emitting module may comprise a plurality of primary emitters that may vary the lateral position of the projected pattern, the structure of the projected pattern from each light emitting module may be fixed, e.g. the spatial frequency of the projected pattern could be fixed. This may enable an entire scene to be covered with light projected from the primary emitters in the array of light emitting modules over time, such that every object in a given scene may be imaged, e.g. to produce a high-resolution 3D map.

[0026] The plurality of primary emitters may be arranged in a 2D planar array, such that the plurality of primary emitters are arranged in the same plane. Each primary emitter within a light emitting module may be spaced from the adjacent primary emitter, e.g. the primary emitters may be equally spaced. Each primary emitter may be spaced a different distance from the other primary emitters, and / or may be arranged in a grid, or other formation. Therefore, each primary emitter may illuminate the respective optical element for the light emitting module from a different, known angle 5. Where a light source is a distance f from the respective optical element, the pattern is laterally shifted (e.g. shifted perpendicular to the direction of projection of light) by a distance A, according to the relationship 8 = . As a result, the position of the pattern projected from each primary emitter within the light emitting module is different. Making each primary emitter within the light emitting module individually addressable can thus allow the projection of light patterns with different lateral positions.

[0027] In one example, each light emitting module comprises three primary emitters, wherein the primary emitters are spaced such that three lateral positions of light may be projected, e.g. 0, 120, and 240 degrees. By illuminating different primary emitters, the light patterns may be shifted laterally and thus cover the entire scene. Furthermore, the camera(s) may detect the lateral shift of the detected light, such that the processing portion can better estimate the relative distances travelled by the light to various parts of the image in order to allowing greater accuracy when forming the depth map of the scene being imaged. For example, three different lateral positions of a single pattern may be used to image a scene, in order to fully illuminate the scene, and / or because the reflectance of objects in the scene may be unknown. These three different patterns may be used to calculate the depth corresponding to each camera pixel.

[0028] As outlined above, in accordance with some embodiments, three different lateral positions of light may be projected for each light pattern projected. Therefore, where there are a total of four light emitting modules (e.g. where each light emitting module emits a pattern with a different spatial frequency) twelve different patterns may be used to image a single scene. Arrangements in accordance with the invention advantageously facilitate quickly changing the pattern being projected for efficient imaging.

[0029] Whilst imaging using two projected patterns, each with a different spatial frequency, may be sufficient for 3D imaging under ideal conditions with low noise, this may not be sufficient in noisy environments. Furthermore, laterally shifting each pattern fewer than three times (e.g. fewer than three primary emitters per light emitting module) may not cover the entire scene and thus could make it difficult to obtain a reliable 3D image. However, in some circumstances more than three lateral shifts (i.e. denser sampling) may reduce performance of the device.

[0030] Therefore, in some embodiments, the array of individually addressable light emitting modules is arranged to emit patterns with at least 3, preferably 4 or 5 spatial frequencies, e.g. with three lateral positions per spatial frequency. For example, the array of light emitting modules may comprise three light emitting modules, wherein each light emitting module comprises three individually addressable primary emitters and a diffractive optical element, wherein each diffractive optical element projects light with a different spatial frequency.

[0031] Furthermore, projecting patterns with different lateral positions simultaneously may allow for the projection of more complex, non-periodic patterns, with the same efficiency as a simpler, e.g. sinusoidal, pattern. The processor may be programmed with the known patterns that may be projected for the different combinations of primary emitters that are turned on.

[0032] In a set of embodiments, each primary emitter comprises an elongate array of secondary emitters. The elongate array of secondary emitters of each primary emitter may be driven by the same control signal. Each secondary emitter may comprise a light source, e.g. a VCSEL. The plurality of secondary emitters may increase the intensity of light emitted from each primary emitter, thus allowing for a sharper pattern to be projected, e.g. when projecting patterns at far-field objects, or in a bright environment, e.g. outdoors. Furthermore, the elongate arrangement of secondary emitters may project a sharp array of vertical lines, for example such that it may be easier to illuminate an entire scene with projected light as the projected pattern may only be shifted laterally in one direction to illuminate the entire scene. In a set of embodiments, the secondary emitters in each elongate array are mutually incoherent. This may reduce speckle contrast in the projected light. For example, for a primary emitter comprising n secondary emitters, the speckle noise may be reduced by a factor of Vn. This may improve the resolution of the projected pattern, allowing for smaller or fast-moving objects in the scene to be imaged. A moving diffuser could be provided, which may additionally or alternatively reduce speckle noise in the projected pattern.

[0033] In accordance with the present invention it may not be necessary to use scanning in order to image a scene. This may be useful in small-scale devices or environments where scanning would be difficult to achieve, e.g. endoscopes, microrobots, or microdrones.

[0034] Furthermore, embodiments of the present invention may be able to achieve a high resolution image at very close distances to the object being imaged, due to the range of patterns that can be quickly projected. For example, the present invention may be applicable in a fingerprint scanner at working distances of approximately 10 - 50mm, or even an endoscope where tissue is imaged very close to the imaging apparatus.

[0035] The Applicant has appreciated that this is novel and inventive in its own right, and thus when viewed from another aspect, the invention comprises an endoscope for 3D imaging of internal tissue of a human or animal subject using structured light, comprising: an endoscope lumen; an array of individually addressable light emitting modules arranged in the lumen to project light onto the tissue; and one or more cameras arranged in the lumen to detect the projected light from each light emitting module; wherein each light emitting module is arranged to project light with a different pattern; and wherein the cameras are arranged to detect the light reflected from the tissue. Typically, when 3D imaging using structured light, the only light that is projected onto a scene is the structured light itself. However, the Applicant has appreciated that shining flat-field light of a particular spectrum or at a particular wavelength may also be useful in some circumstances. For example, this may be useful where shining a particular wavelength of light causes features to become visible where they otherwise would not be seen. One such situation is where the device is used to image blood vessels; using a particular wavelength of light for the flat-field light source allows the bloodoxygenation to be imaged as well as the tissue structure. This may be useful therefore where the array of individually addressable light emitting modules form part of an endoscope.

[0036] Thus, in a set of embodiments, the system or device comprises a flat-field light source arranged to project light with a different spectrum to the light projected by the array of individually addressable light emitting modules. The flat-field light source may be arranged on the same substrate as the array of individually addressable light emitting modules.

[0037] In some embodiment, the flat-field light source is arranged to project light with multiple different spectra at different times, e.g. the flat-field light source may comprise at least two light sources arranged to project different spectra, or the flat-field light source may comprise an adjustable filter. For example, the flat-field light source may comprise one or more LED emitters, e.g. to project red, green blue, and / or near-infrared light.

[0038] In order to obtain a 3D image of a scene, the array of individually addressable light emitting modules will typically project light over a series of exposures, each at a different spatial frequency and / or lateral position. The flat-field light source may be controlled to project light before, after, and / or in-between the exposures by the array of individually addressable light emitting modules, i.e. the flat-field light source may be controlled to project light at a different time to the array of individually addressable light emitting modules. The flat-field light source may project different light spectra during each projection. For example, to accurately obtain a 3D image, 12-15 exposures of structured light from the array of individually addressable light emitting modules may be required. Thus, by projecting different spectra with the flat-field light source (preferably before, after, or between the different exposures), the camera may obtain a multi-spectral image that can be mapped onto a 3D point cloud by the processor, where the 3D point cloud is produced by imaging the structured light. Therefore, this arrangement allows for the production of multispectral 3D images without significantly increasing the size or cost of the device (e.g. due to the simplicity of the light source required for the flat-field light source), and without requiring any significant additional time to perform the multi-spectral measurements.

[0039] Alternatively the flat-field light source could project flat-field light at the same time as the array of individually addressable light emitting modules project structured light. For example, the camera may comprise a filter, and / or the flat-field light source and array of individually addressable light emitting modules may comprise narrow band sources, such that the structured light may be distinguished from the flat-field light by the camera and processor. This may further decrease the time required to generate a multi-spectral image.

[0040] The Applicant has appreciated that such arrangements are novel and inventive in their own right, and thus when viewed from another aspect, the invention provides a device for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules, wherein each light emitting module is arranged to project light with a different pattern; a second light source arranged to project a different spectrum of light to the array of light emitting modules; one or more cameras arranged to detect the projected light from each light emitting module.

[0041] The second light source could be part of the array of individually addressable light emitting modules. However, in a set of embodiments the second light source is separate. This is particularly appropriate where, accordance with a sub-set of embodiments, the second light source arranged to project flat-field light. For example, the array of light emitting modules may project structured light at a particular wavelength, whilst the second light source (e.g. a one or more emitters e.g. VCSELS or LEDs) may be arranged to project flat-field light at a different wavelength. For example, there may not be a diffractive optical element array arranged over the second light source. In a set of embodiments, the array of individually addressable light emitting modules comprises a diced wafer stack, i.e. the array of individually addressable light emitting modules are fabricated by etching multiple wafers, aligning and stacking the wafers, and later dicing the wafer stack into multiple individual devices (i.e. multiple individual arrays of light emitting modules). This means that only one set of wafers needs to be aligned, rather than aligning many individual components, greatly reducing the time and cost of assembling multiple light emitting modules.

[0042] The array of light emitting modules may comprise an emitter wafer comprising at least one light source (e.g. a plurality of primary emitters), and an optical wafer comprising at least one optical element (e.g. a diffractive optical element). The optical wafer may comprise a diffractive optical element etched on the top side, and spacer etched on the bottom side to separate the diffractive optical element form the array of emitters.

[0043] In another example, the array of light emitting modules comprises three stacked wafers, e.g. an emitter wafer comprising at least one light source, a spacer wafer, and an optical wafer comprising at least one optical element. In this arrangement, the optical wafer does not need to be etched with a spacer on the bottom side; instead both sides of the optical wafer may be etched with an optical element.

[0044] In an exemplary set of embodiments, a large emitter wafer is etched with many light sources and a large optical wafer is etched with many optical elements. The larger emitter wafer is stacked underneath and aligned with the optical wafer, such that each light source is aligned with at least one optical element. The large stacked wafer is then diced into individual diced wafer stacks, where each diced wafer stack comprises an array of light emitting modules.

[0045] The Applicant has further appreciated that it may be useful to integrate the components used to image the structured light with the array of light emitting modules for projecting the structured light, e.g. to avoid the need to align the camera with the light emitting modules; and to allow for imaging in environments where space is very limited, e.g. in endoscopy applications. Thus, when viewed from another aspect, the invention comprises a device for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules arranged on a common substrate, wherein each light emitting module is arranged to project light with a different pattern; and one or more cameras arranged to detect the projected light from each light emitting module; wherein the array of light emitting modules and the cameras are integrated in a common housing.

[0046] In a set of embodiments, the camera comprises one or more image sensors arranged on a common substrate with the array of individually addressable light emitting modules, e.g. the image sensor(s) may be arranged on the same substrate as the array of primary emitters. The image sensor(s) may be arranged adjacent to the array of emitters, e.g. two image sensors arranged on either side of the array of emitters.

[0047] In another set of embodiments, the camera comprises one or more image sensors on a first substrate and the array of individually addressable light emitting modules is arranged on a second substrate on top of the one or more image sensors. For example, the device may comprise one image sensor, with the array of individually addressable light emitting modules covering a portion of the image sensor. This may divide the image sensor into two virtual image sensors which can be useful as it is simpler to read from one image sensor rather than two, simplifying synchronizing and interfacing between the image sensor(s), light emitting modules and a processor. This may further reduce the size and cost of the device, as fewer wires and a smaller area for connectors are needed. Such arrangements beneficially allow the device to be manufactured as a diced wafer stack, which is placed on top of the image sensor.

[0048] The device may further comprise one or more imaging lenses arranged on a lens substrate above the one or more image sensors, wherein each imaging lens is arranged to focus light onto a respective image sensor. Thin imaging lenses (e.g. refractive, diffractive, or meta-surface) can be fabricated on a single substrate. These can provide a good image resolution, especially for monochromatic light. For example, in embodiments where each light emitting module comprises an optical element(s), the one or more imaging lenses may be arranged on the same lens substrate as the array of optical elements e.g. two imaging lenses may be arranged on either side of the array of optical elements. In an exemplary set of embodiments, the imaging lens(es) comprise refractive surfaces on both sides of the lens substrate, while the optical elements (e.g. diffractive optical elements) are etched or stamped on one side of the lens substrate. Thus, in a set of embodiments wherein each light emitting module comprises at least one light source (e.g. an array of primary emitters) and an optical element, the one or more image sensors and the light source are arranged on the common substrate, the array of optical elements and the imaging lenses are arranged on the lens substrate, and the lens substrate is arranged on top of the common substrate.

[0049] In some embodiments, a plurality of devices are manufactured using fan-out waferlevel packaging. This may allow for multiple devices to be manufactured at once on the same substrate, and later diced into individual devices. For example, the image sensors, array of emitters, and / or additional light sources (e.g. flat-field light sources), to be used in multiple devices, are arranged on an interposer wafer. A large optical wafer may be etched with many optical elements and stacked and bonded on top of the interposer wafer. Then, the interposer wafer and the optical wafer can be diced into individual devices, in order to efficiently produce many devices at once.

[0050] In some embodiments, the camera comprises at least two image sensors on a first substrate, comprising a space in between the image sensors, and the array of individually addressable light emitting modules is arranged on a second substrate in the space in-between the image sensors. For example, a chip including the array of individually addressable light emitting modules may be positioned in between the image sensors using fan-out wafer-level packaging techniques. A lens substrate may further be arranged on top of the first and second substrates. Such arrangements may allow the device to be manufactured as a diced wafer stack, without the array of individually addressable light emitting modules covering the image sensor(s).

[0051] In a set of embodiments, the system further comprises at least one micromechanical actuator arranged to move at least one of the optical elements relative to the respective light source(s).

[0052] For example, the actuator may be arranged to move the optical element, and / or the actuator may be arranged to move the light source(s). The actuator may be attached to the optical elements and / or light sources. Different light emitting modules could have their own actuators or arrays of optical elements / light sources could be moved by a common actuator. This actuator could introduce a small lateral shift (e.g. between approximately 1 pm and 10 pm) between the light source(s) and the optical element.

[0053] The lateral shift introduced by the actuator may shift the position of the projected pattern. For example, by moving each light source relative to its respective optical element, the angle between the light source and optical element is varied, and thus the lateral position of the projected pattern is also varied. Where each light emitting module comprises a plurality of primary emitters, whereby turning on different primary emitters changes the lateral position of light projected, this arrangement may be used to introduce an additional lateral shift. For example, the actuator may allow for shifting of the lateral position of light by a different amount than the possible shifts allowed by the arrangement of primary emitters. This may be useful where many different lateral positions are used to fully cover a scene, and thus it may not be practicable to position an array of primary emitters for every desired lateral position of light to be projected.

[0054] The actuator could be arranged to have a continuous range of movement but in a set of embodiments, the actuator is arranged to move in discrete steps. For example, the actuator may be arranged to move between certain predetermined positions corresponding to a position of each light source relative to each diffractive optical element corresponding to certain lateral positions, e.g. 0, 120 and 240 degrees.

[0055] The actuator may comprise a piezoelectric actuator, or an electrostatic actuator. The actuator may be controlled by the processing portion.

[0056] In an exemplary embodiment, an array of light sources (e.g. VCSELs) is integrated onto a common substrate connected to a piezoelectric actuator, e.g. a stack of piezoelectric material, by a spring. When a voltage of 0-100 V is applied to the piezoelectric actuator, the entire array of light sources will shift by between approximately 1 - 10 pm relative to the respective array of optical elements (e.g. the array of diffractive optical elements). The voltage applied to the piezoelectric actuator may depend on the desired lateral shift. For example, a voltage of 100 V may shift the array of light sources by approximately 10 pm. Where there is a 1 mm distance between the array of light sources and the array of optical elements, the pattern is shifted laterally by a distance of 10 mrad, which may be sufficient for a 240 degree phase shift of a periodic line pattern with a spatial frequency of approximately 1 line / degree.

[0057] In a set of embodiments, the actuator is arranged to move each light source relative to its respective optical element in one dimension. For example, each light source may be moved along a straight line (e.g. with a linear actuator). However this is not essential - the actuator (or another actuator) may be arranged to move each light source in two dimensions, for example in a direction parallel to the plane of the array of light emitting modules.

[0058] The Applicant has recognised that as well as employing an actuator to provide lateral shifts as set out above, an actuator could have application in providing another mechanism to generate different patterns, and therefore when viewed from another aspect, the invention provides a system for 3D imaging using structured light, comprising: an array of optical elements; at least one light source arranged to direct light onto at least one optical element in the array of optical elements; an actuator arranged to provide movement of the light source and the array of optical elements relative to each other such that light is projected from the array of optical elements with different patterns at different relative positions of the light source and array of optical elements; one or more cameras arranged to detect the projected light from the array of optical elements; and a processing portion arranged to form a depth map based on the projected light detected by the one or more cameras.

[0059] From another aspect, the invention provides a device for 3D imaging using structured light, comprising: an array of optical elements; at least one light source arranged to direct light onto at least one optical element in the array of optical elements; and an actuator arranged to provide movement of the light source and the array of optical elements relative to each other such that light is projected from the array of optical elements with different patterns at different relative positions of the light source and array of optical elements.

[0060] Thus it will be seen that, in accordance with these aspects of the invention, the actuator is arranged to move the light source relative to the array of optical elements, to project different light patterns. For example, the actuator may be connected to the light source, or the array of optical elements. The skilled person will appreciate that as the light source may be moved relative to the array of optical elements, a plurality of different patterns may be projected by moving the light source to different relative positions, e.g. by illuminating different optical elements within the array of optical elements which generate different patterns. Therefore, a single light source may be used to project a plurality of different patterns.

[0061] Embodiments of these aspects of the invention may be able to quickly and efficiently project a sequence of different light patterns for 3D imaging. For example, the different light patterns may be different patterns of dots, stripes, or any other desired pattern. Projecting different light patterns from the same device may allow for a scene to be imaged in 3D without the need for scanning back and forth to cover the whole scene. Therefore, when compared to currently available projectors based on scanning mirrors or laser triangulation, embodiments of the present invention may simpler, smaller, faster, and more power efficient.

[0062] As previously described the camera(s) can detect the projected light reflected from objects present in the scene being imaged and which can be analysed to determine perturbations in the different patterns to form a depth map.

[0063] In a set of embodiments, the array of optical elements comprises an array of diffractive optical elements. Each diffractive optical element in the array of diffractive optical elements may be arranged to project a different pattern. Each diffractive optical element may be selected in order to project a particular pattern, e.g. by using a microstructured surface relief profile to shape the light transmitted from the light source into a desired distribution by diffraction. For example, each diffractive optical element may be fabricated to project a particular pattern suitable for a given application, e.g. a sparse array of dots for facial recognition, and / or to project a pattern with a particular spatial frequency. In a set of embodiments, the actuator is arranged to shift the light source and the array of optical elements relative to each other in order to project a different pattern by illuminating different optical elements at different times. Additionally or alternatively, the actuator is arranged to shift the light source and the array of optical elements such that the light source illuminates the same optical element but from different angles at different times, e.g. by moving the light source by between approximately 0.5 urn - 3 urn. By illuminating the optical element from a different angle, the lateral position of the pattern projected by the optical element may be shifted. The advantages of this are outlined above in respect of the previous aspects of the invention.

[0064] In a set of embodiments the actuator comprises a micromechanical actuator, e.g. a piezoelectric actuator. The actuator may be controlled in response to feedback from the processing portion. Of course more than one actuator may be provided, e.g. to provide a push-pull arrangement and / or to provide movement in more than one direction.

[0065] In a set of embodiments, the actuator operates in discrete steps, e.g. by positioning the light source and array in one of a plurality of pre-determined mutual positions. For example such positions could correspond to projecting a desired pattern from a particular optical element. The positions may additionally or alternatively vary the lateral position of the projected pattern, e.g. 0, 120, or 240 degrees.

[0066] The light source may comprise an LED, laser (e.g. an edge-emitting laser), or any other suitable light source for projecting onto a scene as will be appreciated by the skilled person. In a set of embodiments, the light source comprises a vertical-cavity surface-emitting laser (VCSEL). In a set of embodiments, the light source comprises an array of emitters, e.g. an array of VCSELs.

[0067] Features of any aspect or embodiment described herein may, wherever appropriate, be applied to any other aspect or embodiment described herein. Where reference is made to different embodiments or sets of embodiments, it should be understood that these are not necessarily distinct but may overlap. Certain preferred embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0068] Figure 1 is a perspective view of a first device for 3D imaging using structured light in accordance with the invention;

[0069] Figure 2a is a schematic diagram of a first pattern of light projected using the device shown in Figure 1 ;

[0070] Figure 2b is a schematic diagram of a second pattern of light projected using the device shown in Figure 1 ;

[0071] Figure 3a is a schematic diagram of a first lateral position of light projected using the device shown in Figure 1 ;

[0072] Figure 3b is a schematic diagram of a second lateral position of light projected using the device shown in Figure 1 ;

[0073] Figure 3c is a schematic diagram of a third lateral position of light projected using the device shown in Figure 1 ;

[0074] Figure 4 is a schematic diagram of an alternative layout of the light sources of the device shown in Figures 1 and 2;

[0075] Figure 5 is a block diagram of a system for operating the device for 3D imaging;

[0076] Figure 6a is a perspective cross-sectional view of a second embodiment of a device for 3D imaging using structured light;

[0077] Figure 6b is a perspective view of the device in Figure 6a;

[0078] Figure 7 is a block diagram of a system for operating the device of Figs. 6a and 6b;

[0079] Figure 8 is a schematic diagram of a system for 3D imaging which can be used with either the device shown in Figure 1 or the device shown in Figures 6a and 6b;

[0080] Figure 9 is a perspective view of a device for 3D imaging using structured light in accordance with a third embodiment of the invention;

[0081] Figure 10 is a schematic diagram of a device for 3D imaging using structured light in accordance with a fourth embodiment of the invention; and

[0082] Figure 11a is a perspective view of a device for 3D imaging using structured light in accordance with a fifth embodiment of the invention;

[0083] Figure 11b is another perspective view of the device shown in Figure 11a;

[0084] Figure 12 is a perspective view of an endoscope including the device shown in Figure 9, 10, or 11a. Figure 1 shows a device 100 for 3D imaging which includes an array of emitter groups 106a-d positioned on the surface of a substrate 102. An array of diffractive optical elements 104a-d is arranged in a layer which in use sits on top of the substrate layer on which the emitter groups 106a-d are mounted so that he light emitted from each emitter group passes through a respective diffractive optical element within the array of diffractive optical elements 104a-d.

[0085] Each emitter group 106a-d is independently addressable, i.e. an individual emitter group may be turned on or off independently of the other emitter groups. Each diffractive optical element diffracts the light from the respective emitter group 106a-d such that a different light pattern is projected from each diffractive optical element, more specifically each diffractive optical element projects light with a different spatial frequency, i.e. a different periodic pattern of light.

[0086] Each emitter group 106a, 106b, 106c, 106d is made up of a sub-array of three elongate primary emitters 108a, 108b, 108c spaced at a distance x from each other. The primary emitters within each emitter group 106a-d may have different spacings.

[0087] Each primary emitter 108a, 108b, 108c is made up from a column of seven secondary emitters 110. The secondary emitters 110 making up each primary emitter 108a, 108b, 108c are controlled using the same control signal so that they effectively act as a linear light source. As shown in Figure 1 , a primary emitter 108b within an emitter group 106b is turned on such that light is projected along a path 103 to the respective diffractive optical element 104b.

[0088] Each secondary emitter 110 is an individual vertical-cavity surface-emitting laser (VSCEL). VCSELs are considered to be a good choice as they are efficient, and smaller than other coherent light sources available, and emit light with a narrow emission angle and a narrow spectral width, which allows for sharp, collimated patterns to be projected. Furthermore, VCSELs may be modulated at high speeds, which is useful if the light emitting modules are being turned on and off quickly in sequence, in order to quickly project a range of patterns e.g. with different spatial frequencies. The device 100 shown in Figure 1 includes four emitter groups 106a-d, with four different respective diffractive optical elements 104a-d. Therefore, the device 100 is able to project light with four different spatial frequencies. Two different patterns 109, 111 , projected by two different emitter groups 106b, 106c and respective diffractive optical elements 104b, 104c, are shown in Figures 2a and 2b. For example, the pattern 109 shown in Figure 2a has a different spatial frequency to the pattern 111 shown in Figure 2b.

[0089] The secondary emitters 110 making up each primary emitter 108a, 108b, 108c are mutually incoherent. The incoherence between the secondary emitters helps to reduce the speckle contrast of the projected pattern. Within a primary emitter, the secondary emitters are spaced by approximately 10 - 100 pm.

[0090] Each primary emitter 108a, 108b, 108c within each emitter group 106a-d is individually addressable and typically during operation of the device 100, when an emitter group 106a-d is turned on, only one of its primary emitters108a-c is turned on at a time. This allows the position of the light pattern emitted from each emitter group 106a-d to be shifted by turning on a different primary emitter 108a, 108b, 108c.

[0091] Because each primary emitter 108a, 108b, 108c is spaced a distance x from the adjacent primary emitter, each primary emitter illuminates the respective diffractive optical element 104a-d from a different angle. The gives rise to a lateral position shift of the projected light pattern depending on which of the primary emitters 108a, b, c is illuminated, as shown with reference to Figures 3a, 3b, and 3c. The distance x determines the relative lateral positions of light emitted from the primary emitters in each emitter group. For example, the distance x may be between approximately 1 and 10 pm, which allows for a meaningful phase shift of 0, 120, and 240 degrees in the projected light pattern.

[0092] Figure 3a shows a schematic diagram of a first light pattern 113a projected from a first primary emitter 108a within an emitter group 106b, using the device 100. The light passes through the respective diffractive optical element 104b for the emitter group 106b. For reference, a stationary reference point 115 is shown. As shown in Figure 3a, the stationary reference point 115 is on the left hand side of a dark fringe of the first light pattern 113a. Figure 3b shows a schematic diagram of a second light pattern 113b projected using a second primary emitter 108b within the emitter group 106b. The second primary emitter 108b is spaced a distance x from the first primary emitter 108a, and thus the second light pattern 113b is shifted relative to the first light pattern 113a shown in Figure 3a. As both the first and second primary emitters 108a, 108b are within the same emitter group 106b, they both project light through the same diffractive optical element 104b and thus project patterns with the same spatial frequency, but laterally shifted relative to one another. This is seen by the stationary reference point 115 being positioned in the centre of the dark fringe of the second light pattern 113b, and thus the second light pattern 113b is shifted to the right of the first light pattern 113a.

[0093] Figure 3c shows a schematic diagram of a third light pattern 113c projected from a third primary emitter 108c within the emitter group 106b. The third primary emitter 108c is spaced a distance x from the second primary emitter 108b, and thus the third light pattern is shifted relative to the second light pattern 113b shown in Figure 3b. Therefore, the projected light patterns are shifted slightly in space depending on which primary emitter 108a-c the light was originally emitted from.

[0094] By addressing different primary emitters 108a-c, the projected pattern can be thus shifted to three different lateral positions, such that the entire scene is covered by the projected patterns.

[0095] This is repeated for each emitter group 108b-d, where each light emitting module projects a different pattern. By having the scene illuminated four times, each with a different pattern, ambiguities arising from imaging with a single pattern can be resolved and thus the entire scene can be accurately imaged.

[0096] As described with reference to Figure 1, each emitter group 106a, 106b, 106c, 106d is aligned with a respective associated diffractive optical element 104a, 104b, 104c, 104d. As described with reference to Figure 2, each diffractive optical element is manufactured to project an array of vertical lines with a particular spatial frequency. However any other desired pattern could be chosen - which need not be a regular pattern. The pattern chosen will typically be selected based on the scene being imaged by the device 100. For example, it may be preferable to project a sparse array of dots for forming a depth map involving facial recognition, whereas a more complex pattern may be preferable for imaging a complex scene where many objects are at different depths.

[0097] As will be appreciated therefore, the device 100 is able to emit twelve different patterns of light, as there are four emitter groups 106a-d with respective diffractive optical elements 104a-d projecting four different light patterns (with four different spatial frequencies), with each primary emitter 108a-c of a given group able to give that pattern one of three different phases.

[0098] Each VCSEL emits light with the same wavelength, for example approximately 940 nm. The typical extent of each diffractive optical element is 200-500pm, while the resolution of the diffractive optical element pattern should be on the order of the wavelength or less. The spacing between the diffractive optical elements should be as small as possible but without light from one VCSEL entering the neighbour's diffractive optical element.

[0099] It will be appreciated by the skilled person that the device 100 may include more or fewer than the four emitter groups 106a-d shown, e.g. only two emitter groups with two corresponding diffractive optical elements where only two different patterns are to be projected. Each primary emitter may also be made up from more or fewer secondary emitters, for example only a single secondary emitter may make up each primary emitter e.g. where it is desirable to project many different light patterns whilst maintaining a small area of emitter groups.

[0100] Figure 4 shows an alternative arrangement of secondary emitters 112 to the arrangement shown in Figures 1 and 2, where the secondary emitters 112 making up each primary emitter 108a, 108b, 108c, are offset from each other. This may be useful if the separation between primary emitters needed is smaller than the minimum distance between secondary emitters allowed by the hardware, e.g. the minimum distance between VCSELs.

[0101] The primary emitters can be turned on and off quickly, such that the scene can be imaged quickly, for example by a camera or pair of cameras. A given scene may be imaged using the device 100 by projecting a series of patterns from each emitter group in sequence. Different primary emitters may be turned on at different times through a measurement cycle, in order to project patterns with different lateral positions to fully image a scene.

[0102] Figure 5 shows schematically a system for operating a system for 3D imaging using structured light. A processor 118 is connected to two cameras (shown schematically as one block) 114 and emitter groups 106 of the device 100 shown in Figure 1. The processor 118 is able to control which primary emitters 108 are turned on within each emitter group 106 and when, in order to emit a desired sequence of different patterns. The processor 118 is programmed to cause the device 100 to project a particular pattern with a particular spatial frequency at a given time. Similarly, the processor 118 can control the lateral position of the projected pattern, as it is able to control each primary emitter 108a-c independently. The processor 118 is thus programmed to project a particular sequence of light patterns over a given area.

[0103] The cameras 114 feed-back the detected pattern reflected from the object being imaged to the processor 118. The processor 118 then processes the images detected by the cameras 114 in order to form a depth map, i.e. to image the scene in 3D. It would also be possible for the processor 118 to detect if there are any features which have not been sufficiently resolved, e.g. any areas missed out by the projected patterns, and so extend the imaging sequence such as by projecting more patterns which allows for the missing features and / or areas to be imaged and resolved.

[0104] Figure 6a shows a schematic diagram of a second device for 3D imaging using structured light in accordance with the invention. The second device 200 includes a substrate 222, where an array of light sources 221 such as VCSELS are integrated into the surface of the substrate 222. Each light source in the array of light sources 221 is individually addressable, i.e. each light source may be turned on and off independently to the other light sources in the array.

[0105] An array of diffractive optical elements 224 is arranged in a layer parallel to and spaced from the substrate 222. There may not be a one to one correspondence between diffractive optical elements 224 and light sources 221 ; for example there may be many more diffractive optical elements 224 compared to the number of light sources 221. Each diffractive optical element 224 is designed to project a different pattern of light. For example, each diffractive optical element 224 may project a vertical line pattern with a different spatial frequency.

[0106] The substrate 222 is connected to an actuator 220, e.g. a piezoelectric actuator. The actuator 220 moves the substrate 222 laterally along a first direction 223, in a plane parallel to and spaced from the array of diffractive optical elements 224. The substrate 222 may also be moved in a second orthogonal direction, for example using a second actuator .

[0107] The actuator 220 is able to move the substrate 222 to align different light sources 221 with different diffractive optical elements 224, in order to project different patterns. Furthermore, the actuator 220 is able to move each light source by a smaller amount relative to a specific diffractive optical element in order to change the lateral position of the projected pattern. The skilled person will appreciate that in theory only a single light source 221 would be required on the substrate 222 in order to project a full range of different patterns and lateral positions in sequence, by moving the light source to a different diffractive optical element in the array, and position relative to a given diffractive optical element. However, it is preferable to have multiple light sources 221, e.g. as shown in Figure 6a, as it may be faster to project a sequence of different light patterns by combining both movement of the light sources 221 by moving the substrate 222 with the actuator 220, with turning the light sources 221 on and off.

[0108] Figure 6b is a perspective cross-sectional view of the second device for 3D imaging shown in Figure 6a. The array of light sources 221 is shown as a linear array of five light sources, mounted on a substrate. The substrate is mounted on the actuator 220, which in this case is a stack of piezoelectric material, which is connected to a voltage source (not shown). When a voltage of between 0 and 100 V is applied to the stack of piezoelectric material, the substrate will shift between 0 and 10 pm relative to the array of diffractive optical elements 225 along the first direction 223. The shift of between 0 and 10 pm is used to change the lateral position of the projected light from each light source 221. Each light source 221 may be turned on independently from the other light sources 221 , and thus the second device 200 is able to project five different patterns at any time, or a superposition of patterns, where each pattern can be shifted to many different lateral positions. Furthermore, the actuator 220 may additionally induce a high-frequency, low- amplitude vibration of the array of light sources 222 (e.g. at a rate much faster than the camera frame rate), in order to reduce the speckle contrast of the detected pattern, whilst not blurring the resulting 3D image.

[0109] Figure 7 shows schematically a second system for operating the device of Figures 6a and 6b. The processor 218 is connected to the cameras 214, the actuator 220, and the array of light sources 221. The processor 218 is able to control the actuator 220, to move the substrate 222 to different positions relative to the array of diffractive optical elements. The processor 218 is also able to control which light source in the array of light sources 221 is turned on.

[0110] For example, the processor 218 is programmed with which actuator position 220 corresponds to which respective diffractive optical element being aligned with a light source, and what pattern that diffractive optical element projects (e.g. the spatial frequency of lines). The processor 218 is programmed to cause the device 200 to project a particular pattern, e.g. with a particular spatial frequency. Furthermore, the processor 218 can control the lateral position of the projected pattern, as it is able to control the actuator 220 to move a particular light source 221 to a different position relative to a given diffractive optical element. The processor 218 is thus also programmed to control the lateral position too, in the same way as for the first embodiment.

[0111] The analysis of the images fed back from the cameras 214 can also be carried out by the processor 218 using the same approach as previously described to form a depth map, i.e. to image the scene in 3D.

[0112] Figure 8 is a schematic diagram of a system for 3D imaging using either the device 100 shown in Figure 1, or the device 200 shown in Figure 6a. The device 100, 200 projects a periodic line pattern 224 with a given spatial frequency. The periodic pattern 224 is projected onto an object, e.g. a sphere 226, with the projected pattern being perturbed by interaction with the surface of the sphere 226. The perturbed pattern 224’ is detected by the cameras 114, 214, and processed by the processing portion to form a depth map. Figure 9 is a perspective view of a third device 300 in accordance with the invention for 3D imaging using structured light, which includes an array of emitter groups 308 between a first image sensor 314a and a second image sensor 314b, all positioned on a common substrate 334. This embodiment is provided in a very compact package which makes it suitable for applications where small size is important such as placement in an endoscope.

[0113] A lens substrate 332 is positioned above the common substrate 334 and includes: a first lens 328a above the first image sensor 314a; a second lens 328b above the second image sensor 314b; and a diffractive optical element array 304 arranged above the array of emitter groups 308. The first and second lenses 328a, b are moulded into the lens substrate 332 and the diffractive optical element array 304 is etched (or stamped / imprinted) into the lens substrate. This means the diffractive optical element array 304 and first and second lenses 328, 328b can be fabricated as a single lens substrate 332. The array of emitter groups 308 is mounted so that the light emitted from each emitter group passes through a respective diffractive optical element (not shown individually) within the diffractive optical element array 304.

[0114] The array of emitter groups 308 shown includes five emitter groups and the diffractive optical element array 304 includes five diffractive optical elements (not individually shown), where each emitter group is individually addressable, and each diffractive optical element has a different spatial frequency. Each emitter group projects light through a respective diffractive optical element, and thus the third embodiment device 300 can project light with different spatial frequencies as described with reference to Figure 1.

[0115] Each group of emitters within the array of emitter groups 308 is similar to the emitter group described with reference to Figure 4, i.e. each emitter group includes three offset lines of VSCEL emitters. Each offset line is individually addressable and so can project light independently of the other emitters. Thus, each emitter group can project three phases of light, to shift the light pattern projected by the device 300 into three different positions, as described with reference to Figure 1. Therefore, the device 300 can project light with five different spatial frequencies, at three different positions for each spatial frequency. The skilled person will appreciate that the number of emitter groups in the array of emitter groups 308 and the number of offset lines within each emitter group can be selected depending on the requirements of a given application.

[0116] The device 300 also includes a housing 336 and electrical connectors 330 connecting the first and second image sensors 314a, 314b and the array of emitter groups 308 to an external processor and control (e.g. as described with reference to Figure 7).

[0117] The light from the array of emitter groups 308 is projected onto a scene, where the projected light is then detected by the first and second image sensors 314a, 314b. The first and second lenses 328a, 328b focus the light incident thereon onto the respective first and second image sensors 314a, 314b. Thus, the device 300 provides both the projection and detection of structured light in one package. This is useful in applications where it would be difficult to accommodate and align a separate camera, e.g. in endoscopy. The two image sensors 314a, 314b are connected to a processor, e.g. to operate as described with reference to Figure 5. This arrangement means that the emitter group 308 can be easily aligned with its respective diffractive optical element in the array of diffractive optical elements 304, by aligning the common substrate 334 and the lens substrate 332 (i.e. each emitter group does not need to be individually aligned with each diffractive optical element).

[0118] Figure 10 is a schematic diagram of a fourth device 400 for 3D imaging using structured light in accordance with the invention. The device 400 includes a single large image sensor 414, and an array of emitter groups 408. A diffractive optical element array 404 is positioned on top of the array of emitter groups 408, such that each emitter group 408 has a corresponding diffractive optical element 404. In this embodiment, the image sensor 414 is positioned directly underneath the array of emitter groups 408, on a different substrate. Thus, the array of emitter groups 408 block the centre of the image sensor 414 from detecting light, which divides the image sensor 414 into two portions, on either side of the emitter group 408. The two sensor portions act as two virtual cameras, e.g. to operate as described with reference to Figure 9. The image sensor 414 and array of emitter groups 408 are connected to a processor, to control operation of the array of emitter groups 408 and to determine which portion of the image sensor 414 is detecting light. This arrangement means that the device 400 only requires one image sensor, which may further reduce the size and cost of fabrication compared to the third device 300. The array of emitter groups 408 are arranged in a vertical line on top of the image sensor, in order to avoid blocking a significant portion of the image sensor. It is also particularly suitable for employing wafer processing techniques to produce the emitters and diffractive optical elements by dicing a stacked wafer sandwich to produce individual modules which are placed over corresponding image sensors.

[0119] Figures 11a and 11b are perspective cutaway views of a fifth device 500 in accordance with the invention for 3D imaging using structured light, which includes an array of emitter groups 508 between a first image sensor 514a and a second image sensor 514b, all positioned on a common substrate 534. Two flat-field light sources 538a, 538b are arranged on either side of the array of emitter groups 508, diagonally across from the first and second image sensors 514a, 514b.

[0120] The device 500 operates similarly to the third embodiment described with respect to Figure 9. However, in addition to projecting structured light patterns with the array of emitter groups 508, the device 500 also projects flat-field light using one or both of the first and second flat-field light sources 538a, 538b. The flat-field light sources 538a, 538b are operated by a processor to project light before, after, and in-between the exposures of the array of emitter groups 508, when imaging an object. This allows the image sensors 514a, 514b to obtain a multi-spectral image of an object. For example, the first flat-field light source 538a may project light with a different wavelength to the second flat-field light source 538b, such that the device 500 can be quickly switched between projecting different wavelengths of light.

[0121] This may be useful to obtain 3D imaging objects where some features are only visible under a certain light, e.g. blood vessels. This may be used for example when the device 500 is mounted onto the end of an endoscope, e.g. as described with reference to Figure 12. By combining the flat-field light sources 538a, b, the image sensors 514a, b, and emitter groups 508 in a device 500, a single device may be used for multi- spectral 3D imaging in applications where it would be impractical to introduce too many devices, e.g. in endoscopy. Figure 12 shows a perspective view of an endoscope 600 incorporating the device 500 shown in Figure 11a. It will be appreciated by the skilled person that the device 500 could also be the device 300 shown in Figure 9, or the device 400 shown in Figure 10 . The endoscope 600 includes a flexible housing 640, which is inserted into the human or animal tissue to be imaged. A tip portion 642 extends from the flexible housing 640, and the device 500 is mounted onto the end of the tip portion 642. The associated wiring, e.g. for powering and transmitting data from the device 500, extends from the device 500 through the flexible housing 640.

[0122] Thus, the endoscope 600 allows for the device 500 to 3D image inside a human or animal body, by projecting and detecting structured and optionally flat-field light using the device 500. It will be appreciated by those skilled in the art that the invention has been illustrated by describing one or more specific embodiments thereof, but is not limited to these embodiments; many variations and modifications are possible, within the scope of the accompanying claims.

Claims

CLAIMS1. A device for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules arranged on a common substrate, wherein each light emitting module is arranged to project light with a different pattern; and one or more cameras arranged to detect the projected light from each light emitting module; wherein the array of light emitting modules and the cameras are integrated in a common housing.

2. The device as claimed in claim 1 , wherein the one or more cameras and are arranged on the common substrate.

3. The device as claimed in claim 1, wherein the one or more cameras are arranged on a different, camera substrate and the array of individually addressable light emitting modules is disposed above the camera substrate.

4. The device as claimed in any preceding claim, comprising one or more imaging lenses arranged on a lens substrate above the one or more cameras, wherein each imaging lens is arranged to focus light onto a respective camera.

5. The device as claimed in any preceding claim, comprising a flat-field light source arranged to project light with a different spectrum to the light projected by the array of individually addressable light emitting modules.

6. The device as claimed in claim 5, wherein the flat-field light source is arranged to project light at a different time to the array of individually addressable light emitting modules.

7. The device as claimed in claim 5 or 6, wherein the flat-field light source is arranged to project light with at least two different spectra.

8. The device as claimed in any preceding claim, wherein each light emitting module comprises at least one light source and an optical element.

9. The device as claimed in claim 8, further comprising at least one micromechanical actuator arranged to move at least one of the optical elements relative to the respective light source(s).

10. The device as claimed in claim 8 or 9, wherein at least one of the optical elements is a diffractive optical element.

11. The device as claimed in any preceding claim, wherein each light emitting module is arranged to emit patterns with different spatial frequencies.

12. The device as claimed in any preceding claim, wherein the projected light from each light emitting module is collimated in a first direction, and divergent in a second direction, wherein the first direction and second direction are perpendicular.

13. The device as claimed in any preceding claim, wherein each light emitting module comprises a plurality of primary emitters, wherein each primary emitter is arranged to project a light pattern at a different lateral position to another primary emitter of the plurality of primary emitters.

14. The device as claimed in claim 13, wherein the plurality of primary emitters are arranged in the same plane.

15. The device as claimed in claim 13 or 14, wherein each primary emitter comprises an elongate array of secondary emitters.

16. The device as claimed in claim 15, wherein the secondary emitters in each elongate array are mutually incoherent.

17. A system for 3D imaging using structured light, comprising the device as claimed in any preceding claim, and a processing portion arranged to form a depth map based on the projected light detected by the one or more cameras.

18. An endoscope for 3D imaging of internal tissue of a human or animal subject using structured light, comprising: an endoscope lumen; an array of individually addressable light emitting modules arranged in the lumen to project light onto the tissue; and one or more cameras arranged in the lumen to detect the projected light from each light emitting module; wherein each light emitting module is arranged to project light with a different pattern; and wherein the cameras are arranged to detect the light reflected from the tissue.

19. A device for 3D imaging using structured light, comprising: an array of individually addressable light emitting modules, wherein each light emitting module is arranged to project light with a different pattern; a second light source arranged to project a different spectrum of light to the array of light emitting modules; one or more cameras arranged to detect the projected light from each light emitting module.

20. The system as claimed in claim 19, wherein the second light source is arranged to project flat-field light.

21. A system for 3D imaging using structured light, comprising: an array of optical elements; at least one light source arranged to direct light onto at least one optical element in the array of optical elements; an actuator arranged to provide movement of the light source and the array of optical elements relative to each other such that light is projected from the array of optical elements with different patterns at different relative positions of the light source and array of optical elements; one or more cameras arranged to detect the projected light from the array of optical elements; and a processing portion arranged to form a depth map based on the projected light detected by the one or more cameras.

22. The system as claimed in claim 21 , wherein the array of optical elements comprises an array of diffractive optical elements.

23. The system as claimed in claim 21 or 22, wherein the actuator is arranged to shift the light source and the array of optical elements relative to each other in order to project a different pattern by illuminating different optical elements at different times.

24. The system as claimed in any of claims 21-23, wherein the actuator comprises a micromechanical actuator.

25. A device for 3D imaging using structured light, comprising: an array of optical elements; at least one light source arranged to direct light onto at least one optical element in the array of optical elements; and an actuator arranged to provide movement of the light source and the array of optical elements relative to each other such that light is projected from the array of optical elements with different patterns at different relative positions of the light source and array of optical elements.

Images