Imaging system for imaging a scene comprising one or more objects

Abstract

An imaging system for imaging a scene comprising an object O, the system comprises a light source P arranged to illuminate the scene by projecting light rays through an array 1101 of illumination poin

Description

FIELD Embodiments described herein relate to an imaging system for imaging a scene comprising one or more objects. BACKGROUND Three-dimensional surface imaging (3D surface imaging) is a fast growing field of technology. The term “3D surface imaging” as used herein can be understood to refer to the process of generating a 3D representation of the surface(s) of an object by capturing spatial information in all three dimensions - in other words, by capturing depth information in addition to the two-dimensional spatial information present in a conventional image or photograph. This 3D representation can be visually displayed as a “3D image” on a screen, for example. A number of different techniques can be used to obtain the data required to generate a 3D image of an object’s surface. These techniques include, but are not limited to, structured light illumination, time of flight imaging, holographic techniques, stereo systems (both active and passive) and laser line triangulation. In each case, the data may be captured in the form of a “point cloud”, in which intensity values are recorded for different points in three-dimensional space, with each point in the cloud having its own set of {x,y,z} coordinates and an associated intensity value I. In some cases, 3D information can be obtained by illuminating the object with a projector and detecting, on a camera, the light reflected at each point on the object as the illumination pattern changes over time. The patterns generated by the projector might include phase shifted sinusoidal patterns and / or gray code patterns, as discussed in WO2017125507A1. A sequence of 2D images can be captured with a different illumination pattern being used for each image. Once captured, the data contained in the sequence of images can be used to determine the depth of each point on the object surface. In more detail, for each point on an object in the field of view, two corresponding positions can be defined: (i) the position in the projector array from which the light that is incident on that point on the object is emanating, and (ii) the camera pixel coordinate i.e. the pixel position in the camera at which the light reflected by that point on the object is captured. Using a suitable algorithm, and taking into account the relative positions of the camera and projector (these relative positions being determined straightforwardly using a standard calibration measurement), the images captured at the camera can be processed in order to determine, for each camera pixel, the corresponding coordinate in the array of illumination points. Having established, for a given camera pixel p, that the pixel p is receiving light from a point on the projector g, for example, a position estimate E of a point on the object having coordinates {x,y,z} can be derived by using known triangulation methods, akin to those used for stereo vision, taking into consideration the lens parameters, distance between the camera and projector etc. Similar methods can be used where the light source is a point source. These techniques are discussed further in “3D Imaging, Analysis and Applications, Chapter 3.4” (Liu, Y. etal., Springer Cham, 11 / 12 September 2020, ISBN 978-3-030-44070-1) as well as “Active Triangulation 3D Imaging Systems for Industrial Inspection” (Drouin, M. and Beraldin, J., in “3D Imaging, Analysis and Applications, Second Edition”, 2020). The above techniques work well provided that each camera pixel is only illuminated by rays from a single point on the projector throughout the sequence of images used to reconstruct the 3D image. This requirement will be met for many real-world scenes in which the object(s) being imaged are composed of diffuse, mostly Lambertian surfaces. In many industrial applications however, additional reflections (referred to herein as “inter-reflections”) may be present and will violate these assumptions). For example, metallic or shiny plastic components will mean that light incident on the object will be reflected multiple times before it reaches the camera. The same assumptions will also be violated if the object comprises one or more transparent surfaces, in which case a camera pixel may receive light rays reflected from both the transparent surface and another reflective surface, with those rays emanating from different points on the projector. Conventional methods that use a combination of gray codes and / or phase shifted line patterns for 3D imaging will fail to provide accurate 3D data once camera pixels start to receive signals from more than one point on the projector. Such conventional methods usually employ a single frequency sine wave as the illumination pattern and measure the phase shift of that sine wave as the illumination pattern is shifted in space. If a camera pixel receives multiple signals from different points on the projector, the sine wave will add with itself and result in only one, erroneous phase shift measurement. The error in the phase will in turn give rise to further errors when reconstructing the 3D image of the object. The ability to successfully account for such inter-reflections when building 3D images of objects would be advantageous for numerous applications. One particular application lies in the field of automated warehouses, where robotic agents are used to identify and collect goods from shelves and other storage facilities. These agents may use cameras to map the contours of the objects they seek to pick up and transport through the warehouse. If the agent is unable to distinguish light rays that have been reflected once from those that have undergone multiple reflections, the agent’s 3D reconstruction of the object(s) will be comprised, meaning that the agent might either fail to retrieve the item or else damage it during collection. SUMMARY According to a first aspect of the present invention, there is provided an imaging system for imaging a scene comprising an object, the system comprising: a light source arranged to illuminate the scene by projecting light rays through an array of illumination points in space, wherein for each one of a sequence of time steps, the light source is configured to illuminate the scene with a different illumination pattern by projecting light rays through a different group of points in the array, wherein for each illumination pattern, a plurality of light rays are projected through different points in the array; a detector comprising an array of pixel elements, the detector being arranged to capture an image at each time step by detecting, on the array of pixel elements, light projected towards the scene and reflected from one or more surfaces of the object towards the detector; and an image processor configured to: identify, for a first pixel element of the detector, and based on knowledge of the illumination pattern used in each time step, one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass during the sequence of time steps; and determine, for each of the identified regions, whether a path travelled by the light ray that passes through the respective region and is incident on the pixel element satisfies one or more geometric constraints, the one or more geometric constraints being satisfied by light rays that have only been reflected once from the object before reaching the detector. The illumination patterns may comprise a series of line patterns in which one or more lines are projected onto the scene. Each line pattern may comprise a plurality of parallel lines. Each of the one or more regions of the array of illumination points may comprise a column of points in the array, and each line is formed by rays of light that pass through a respective one of the columns. The line patterns may be chosen such that each line is projected onto the scene once over the sequence of time steps. The image processor may be configured to identify the one or more regions of the array of illumination points through which respective light rays(s) incident on the first pixel element pass by determining the timing of peaks detected in the light signal at the first pixel element during the sequence of time steps. The illumination patterns may further comprise a series of coded patterns from which can be determined a global location in the array of illumination points through which light rays incident on each pixel of the detector pass. The coded patterns may comprise gray code patterns. The image processor may be configured to identify the region(s) of the array through which light ray(s) incident on the first pixel element pass based on (i) the timing of peaks detected in the light signal at the first pixel element during the course of illuminating the scene with the plurality of lines patterns and (ii) the signal seen in the detector pixel over the course of illuminating the scene with the coded patterns. The light source may be configured to generate the illumination patterns by defining an illumination sequence for respective regions of points in the array of illumination points, the illumination sequence specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region. Each region of points may comprises a respective line of points in the array of illumination points. For each pixel element, the image processor may be configured to model the light signal incident on the pixel element over the sequence of time steps as a function of contributions from the regions in the array of illumination points. The image processor may be configured to identify the one or more regions of the array through which respective light rays(s) incident on the first pixel element pass by determining the one or more regions whose illumination sequences, when combined, result in the light signal seen on the first pixel element. Each illumination sequence may be encoded using one or more bits, the one or more bits defining a relative amplitude of light to be projected through the region of points towards the scene in each time step. The one or more bits may specify a frequency modulation to be applied to the light projected through the region of points over the course of the sequence of time steps. Each illumination sequence may be encoded as a sequence of bits, wherein each bit is associated with a respective time step, the value of each bit defining a relative amplitude of light to be projected through the region of points towards the scene in the respective time step. For each region, the illumination sequence may define one or more time steps at which light is to be projected from the region with a first amplitude, and one or more time steps at which the amplitude of light projected from the region is either reduced compared to the first amplitude or is zero. The illumination sequences may be defined such that for each one of the region of points, the number of time steps in the sequence for which light will be projected with the first amplitude is the same. Each region of points may comprise a respective line of points in the array of illumination points. The number of bit changes between the illumination sequences for each consecutive pair of lines may be the same. The regions may comprise columns of points in the array of illumination points and the sequence of bits for each region may be defined such that for any one region, a correlation between the sequence of bits for that region and the sequence of bit for other regions within a disparity window of that region is reduced compared to the correlation between the sequence of bits for that region and the sequence of bits for other regions outside of the disparity window, the disparity window defining the maximum number of consecutive regions in the array of illumination points, the light from which is capable of being detected on a single pixel of the detector according to the geometry of the system. The image processor may be configured to determine if a path travelled by a light ray incident on the first pixel element satisfies the one or more geometric constraints by: resolving the light detected at the first pixel element into one or more separate detected signals, each signal being associated with light having passed through a different respective region of the array; selecting one of the detected signal(s) at the first pixel element, the selected signal being associated with light having passed through a first region of the array; identifying a second pixel element located in the same epipolar plane as the first pixel element, or in a same plane perpendicular to the epipolar plane, and within a threshold number of pixel elements from the first pixel element; resolving the light detected at the second pixel element into one or more separate detected signals, each signal being associated with light having passed through a different respective region of the array; selecting one of the detected signal(s) at the second pixel element, the selected signal being associated with light having passed through a second region of the array that lies within a threshold distance of the first region; determining an order for the first and second regions of the array based on the order in which the first pixel element and second pixel element appear on the detector; determining, based on knowledge of the illumination patterns, if the order of the first and second regions of the array matches an order that would be expected if light rays from the source and incident on the first and second pixel elements were only reflected once from the object before reaching the detector. In the event the image processor determines that the order of the first and second regions of the array is the reverse of the expected order, the image processor may be configured to discount the signal selected for the first pixel element when determining a distance of the object from the detector. The image processor may be configured to determine if the order of the first and second regions of the array matches the expected order by: assigning a number to each region of the array based on its spatial location within the array; assigning a number to each pixel element based on its spatial location in the detector; generating a graph whose axes show the numbering of each region of the array against the numbering of each pixel element; plotting, on the graph, a first point whose coordinates reflect the number of the first region and the number of the first pixel element, and a second point whose coordinates reflect the number of the second region and the number of the second pixel; fitting a line to the first and second points on the graph; and determining whether the fitted line has a positive or negative gradient. Identifying one or more regions of the array through which respective light rays(s) incident on the pixel element pass may comprise identifying a row and column in the array of illumination points through which each respective light ray passes. Determining whether the path travelled by a light ray that passes through a respective region of the array and is incident on the first pixel element satisfies the one or more geometric constraints may comprise determining whether the one or more regions lies in the same epipolar plane as the pixel element. The illumination patterns may comprise a first set of illumination patterns and a second set of illumination patterns; wherein the first set of illumination patterns comprise a first series of line patterns in which one or more lines are projected onto the scene, the one or more lines being oriented at a first angle with respect to the baseline of the imaging system; and the second set of illumination patterns comprise a second series of line patterns in which one or more lines are projected onto the scene, the one or more lines being oriented at a second angle with respect to the baseline, the second angle being different from the first angle. The illumination patterns may comprise a first set of illumination patterns and a second set of illumination patterns; wherein the light source is configured to generate the first set of illumination patterns by defining an illumination sequence for each one of a first group of regions of points in the array of illumination points, the illumination sequence for each region in the first group of regions specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region. . wherein the light source is configured to generate the second set of illumination patterns by defining an illumination sequence for each one of a second group of regions of points in the array of illumination points, the illumination sequence for each region in the second group of regions specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region. Each region of points in the first group of regions may define a respective line of points in the array of illumination points, each line being inclined at a first angle to the baseline of the imaging system. Each region of points in the second group of regions may define a respective line of points in the array of illumination points, each line being inclined at a second angle to the baseline of the imaging system, wherein the first angle is different from the second angle. For each pixel element, the image processor may be configured to: model the light signal incident on the pixel element over the course of the first set of illumination patterns as a function of contributions from the first group of regions; model the light signal incident on the pixel element over the course of the second set of illumination patterns as a function of contributions from the second group of regions; and identify the one or more regions of the array through which respective light rays(s) incident on the first pixel element pass by: determining one or more regions in the first group of regions whose illumination sequences, when combined, result in the light signal seen on the first pixel element when illuminating the object with the first set of illumination patterns; and determining one or more regions in the second group of regions whose illumination sequences, when combined, result in the light signal seen on the first pixel element when illuminating the object with the second set of illumination patterns. The first set of illumination patterns and the second patterns may be generated such that if the first set of illumination patterns and the second set of illumination patterns were projected onto the scene at the same time, any one line in the first set of illumination patterns will cross any one line of the second illumination patterns a maximum of once. The first set of illumination patterns and the second patterns may be generated such that, for each pixel element, the lines in the first set of illumination patterns and the second set of illumination patterns cross the epipolar plane for the pixel element a maximum of once. The illumination patterns may be generated by multiplying a first subset of illumination patterns by a second subset of illumination patterns. The image processor may be configured to identify the one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass by separately decoding, from the captured images, the information contained in the first subset of illumination patterns and the information contained in the second subset of illumination patterns. A majority of points within each pattern in the first subset of illumination patterns may have zero intensity. The first subset of illumination patterns may comprise a series of lines extending in a first direction. The second subset of illumination patterns may comprise a series of patterns in which intensity is modulated in the direction different from the first direction. Each pattern in the second subset of illumination patterns may be phase shifted to a different extent. The intensity modulation in the second subset of illumination patterns may be a sinusoidal modulation. The light source may be configured to generate respective groups of illumination patterns by multiplying each respective illumination pattern in the first subset of illumination patterns by each pattern in the second subset of illumination patterns. Identifying, for a first pixel element of the detector one or more regions of the array through which respective light rays(s) incident on the pixel element pass during the sequence of time steps may comprises: determining, for each group of illumination patterns, a first component of the signal seen in the pixel element when illuminated by that group of illumination patterns, the first component of the signal comprising information encoded in the first subset of illumination patterns; determining for each group of illumination patterns, a second component of the signal seen in the pixel element when illuminated by that group of illumination patterns, the second component of the signal comprising information encoded in the second subset of illumination patterns; identifying, based on the first component of the signal, a group of regions in the illumination array through which a respective one of the light rays passed; and identifying, based on the second component of the signal, one of the regions in the group of regions through which the light ray passed. The imaging system may be configured to generate a 3D image of the object using knowledge of the one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass. When generating the 3D image of the object, the system may be configured to discount signals incident on the pixel element and identified as coming from light rays that do not satisfy the one or more geometric constraints. The light source may comprise a projector having an array of projector elements, each illumination pattern being generated by activating one or more of the projector elements. According to a second aspect of the present invention, there is provided a robotic device configured to manipulate a physical object, the robotic device being configured to map a 3D surface of the object by using a system according to the first aspect of the present invention. According to a third aspect of the present invention, the imaging system according to the first aspect may be used for imaging one or more objects. BRIEF DESCRIPTION OF DRAWINGS Embodiments of the invention will now be described by way of example with reference to the accompanying drawings in which: Figure 1 shows an example of an imaging system according to an embodiment; Figures 2A and 2B show examples of light sources for use in a system according to an embodiment; Figure 3A shows an example of an imaging system according to an embodiment; Figure 3B shows a detector array as used in the imaging system of Figure 3A; Figure 3C shows a projector array as used in the imaging system of Figure 3A; Figure 4 shows an example of an imaging system according to an embodiment; Figure 5 shows an example of an imaging system according to an embodiment; Figure 6 shows an example of an imaging system according to an embodiment; Figures 7A to 7D show an example of how a temporal sequence of illumination patterns may be constructed in one embodiment; Figure 8 shows a table of intensity values measured at a camera pixel in each one of a sequence of time steps, according to an embodiment; Figure 9 shows the intensity values of Figure 8 plotted on a graph; Figures 10A and 10B show plots of signals incident on different camera pixels and emanating from different columns of the projector, in the system of Figure 4; Figure 11 shows an example of an epipolar plane in the imaging system of Figure 1; Figure 12 shows an example of inter-reflections that occur in the epipolar plane of Figure 11; Figures 13A and 13B show plots of signals incident on different camera pixels and emanating from different columns of the projector, in the system of Figure 5; Figures 14A and 14B show plots of signals incident on different camera pixels and emanating from different columns of the projector, in the system of Figure 6; Figure 15 shows an example of how a plurality of illumination sequences may be defined by using a code matrix, according to an embodiment; Figures 16a and 16B show examples of how illumination sequences associated with different regions in an array of illumination points may contribute to a signal detected at a camera pixel, according to an embodiment; Figure 17 shows a cross-correlation matrix for the codes in the matrix of Figure 17; Figure 18 shows a magnified section of the cross-correlation matrix of Figure 17; Figure 19 shows another example of how a plurality of illumination sequences may be defined by using a code matrix, according to an embodiment; Figure 20 shows a cross-correlation matrix for the codes in the matrix of Figure 19; Figure 21 shows plots of signals incident on different camera pixels and emanating from different columns of the projector, in an embodiment; Figure 22 shows examples of illumination patterns that may be used in embodiments; Figures 23A - 23C show the location of points in a projector array from which different light rays incident on a particular camera pixel may emanate; Figures 24A - 24C show the location of points in a projector array from which different light rays incident on a particular camera pixel may emanate; Figures 25A - 25C show the location of points in a projector array from which different light rays incident on a particular camera pixel may emanate; Figures 26A - 26C show the location of points in a projector array from which different light rays incident on a particular camera pixel may emanate; Figure 27A shows an example of two illumination patterns, as may be used in an embodiment; Figure 27B shows the illumination patterns of Figure 27A having undergone a lens distortion; Figure 28 shows an example of two subsets of illumination patterns that can be combined to form a sequence of illumination patterns according to an embodiment; Figure 29 shows the result of multiplying the patterns in the two subsets of illumination patterns shown in Figure 28; Figures 30A - 30B show the location of points in a projector array from which different light rays incident on a particular camera pixel may emanate; Figures 31A - 31D illustrate how the rays Ri - Re shown in Figure 4 will reflect in different directions, depending on the orientation of the partially reflective surface; Figure 32 shows an example of how the angle of incline of a surface from which interreflections occur may be used to define different regimes for detecting such interreflections; and Figure 33 shows an example in which an imaging system according to an embodiment is employed by a robotic device in a factory or warehouse. DETAILED DESCRIPTION Figure 1 shows an example of an imaging system 100 according to embodiments described herein. The system comprises a light source 101 and one or more detectors 103, which are used to capture images of a scene comprising at least one object 105 placed on a surface 107. The detector is mounted at an offset to the light source. An image processor 109 is used to process the images captured on the detector. The detector 103 may comprise a camera such as a CCD sensor, a CMOS sensor, or event based vision sensor (EVS), for example. In embodiments described herein, the light source 101 comprises a projector having an array of individually addressable pixel elements, which can be activated to form different patterns of illumination. The projector array 101 is shown in more detail in Figure 2A, where the layout of the pixel elements in the projector defines an array of illumination points. Different groups of the pixel elements may be activated at different times, to generate the illumination patterns. For example, individual regions of the projector elements, such as columns or rows may be activated so as to project one or more shapes (e.g. lines) of light 109a, 109b, .... 109n onto the object(s) in the field of view. Although the light source 101 of Figure 1 comprises a projector, it will be understood that this is just one example. The light source used in embodiments may be one of a number of different types of light source capable of projecting light rays through an array of illumination points in space, and able to generate different illumination patterns by projecting light through different groups of points within the array at different times. As an example, Figure 2B shows an alternative light source comprising a point source such as a laser 205, which is arranged to project light onto a mirror 207. The mirror 207 can be rotated quickly, causing the laser beam to sweep through a plane 209 as it is directed towards the scene. An array of illumination points is formed by points within the plane 209 that the different rays of light from the laser pass through as the mirror rotates, each ray having a specific direction. The laser may be time synchronized with the mirror and modulated to generate the desired patterns. Figure 3 shows a schematic of the projector 101 and the camera 103 of the imaging system in more detail. Here we will assume, without loss of generality, that the projector array is oriented such that the horizontal axis of the projector 101 aligns with the baseline between the camera 203 and the projector. The projector may be configured to illuminate a single (vertical) column of light emitting elements at a time, with each column being illuminated sequentially. The reference numerals 101a, 101b, 101c, 101d, 101eand 101f indicate individual columns in the projector array, whilst the reference numerals 103a, 103b, 103c, 103d, 103e and 103f indicate individual pixels in the camera array. As a further illustration, Figure 3B shows the camera array with a single camera pixel element 103a highlighted, whilst Figure 3C shows the projector array with a single column of elements 101a activated. It will be appreciated that Figures 3A - 3C are for purpose of illustration, and the projector array and camera array may each comprise many thousands of elements, rather than the small number shown here. The columns of the projector array 101 may be illuminated sequentially and observed synchronously by camera pixels that capture the light reflected from the surface of the object. Figure 3A shows three rays Ri, R2 and R3, which emanate from the respective projector columns 101a, 101b, 101c, and which are reflected directly from the points Pi, P2 and P3 towards the respective camera pixels 103a, 103b, 103c. In this case, the three pixels 203a, 203b, 203c will each receive light from a single one of the projector columns during the course of the illumination sequence. In many cases, however, a single camera pixel may receive light signals from multiple columns of the projector over the course of the illumination sequence. This can be further understood with reference to Figures 4 to 6. Figure 4 shows the same projector 101 of Figure 1 with projector columns 101a, 101b, 101c, 101d, 101e and 101f. As in Figure 3A, the projector illuminates the object with a sequence of illumination patterns, whereby each column is illuminated sequentially and observed synchronously by camera pixels that capture the light reflected from points Pi - P3. As before, the rays Ri, R2 and R3, which emanate from the projector columns 101a, 101b, 101c, travel directly from the projector to the points Pi - P3 on the diffuse surface of the object, from where they are reflected directly to the respective camera pixels 103a, 103b, 103c. The rays R4, Rs and Re meanwhile, which emanate from the respective projector columns 101d, 101e, 101f, are incident at points P4, Ps and Pe on a partially reflective surface of the object, from where they are reflected to the points P3, P2 and Pi. The rays R4, Rs and Rs are in turn reflected from the diffuse surface towards the camera 103, where they are detected in the same three pixel elements 101a, 101b, 101c as rays Ri, R2 and R3. Since the rays Ri, R2 and R3 form a direct path between the projector and the camera i.e. the rays Ri, R2 and R3 are reflected once from the object and directly towards the camera, these rays will provide the correct estimate of object depth, as marked by the circles at Pi, P2 and P3 once triangulation is carried out. In contrast, the rays R4, Rs and Rs, having undergone an addition (inter)-reflection will provide an incorrect estimate of depth (marked with crosses) when used fortriangulation. Figure 5 shows an example similar to that shown in Figure 4, but in this case, the rays Ri, R2 and R3 are each blocked from reaching the diffuse surface by an obstruction 501. Consequently, the only signals to reach the camera pixels 103a, 103b, 103c will be those from the light rays R4, Rs and Rs, which undergo inter-reflection from the partially reflective surface before they reach the camera. A further scenario is also shown in Figure 6, where instead of a partially reflective surface, the object includes a semi-transparent surface. A portion of each light ray incident on the semi-transparent surface may pass through that surface before being reflected back towards the camera from the diffuse surface beneath. The remaining portion of the light incident on the semi-transparent surface may be immediately reflected back to the camera, meaning that the same camera pixel will again receive two light signals emanating from different columns on the projector. For example, camera pixel 103a may receive a portion of light from ray Ri that passes through the semi-transparent surface and is reflected back to the camera from point Pi on the surface underneath. The same camera pixel 103a will also receive light from the portion of ray R4 that is reflected off the semi-transparent surface. Embodiments described herein provide a means for discriminating between a first type of signal that arises from light rays having undergone direct reflection (either from an opaque surface or a partially transparent surface), and a second type of signal that arises from light rays that have been reflected multiple times from the object before reaching the detector. The first type of signal may be used to derive accurate depth information for the object(s) in the scene. The second type of signal, meanwhile, will result in erroneous measurements of depth if used for triangulation. Embodiments described herein can enable these different types of signal to be distinguished from one another by considering one or more geometric constraints that must apply to light rays that have been directly reflected from the object. This is achieved by illuminating the object(s) with a sequence of illumination patterns, knowledge of which makes it possible to resolve, for a particular pixel element in the detector, the individual region(s) or point(s) in the projector array from which light rays incident on that pixel originate. By considering the location in the projector array from which each light signal originates, it is then possible to determine if the light rays from those locations will satisfy the geometric constraints, and where they do not, to discard the signals in question before computing any depth information for the object(s). A method for identifying where the received light signals have undergone one or more inter-reflections according to a first embodiment will now be described with reference to Figures 7 to 21. In the present embodiment, the object(s) in a scene are illuminated using a temporal sequence of illumination patterns. Figure 7 shows an example of such how the temporal sequence of illumination patterns may be constructed in one embodiment. Here, the illumination patters include a series of line patterns, which are generated by activating different columns of the projector light elements. The columns of the projector elements can be thought of as being divided into a series of m groups, each group containing a number K of columns. For example, the group m = 1 may include columns 1 to K, the column m = 2 may include columns K+1 to 2K, the group m = 3 may include columns 2 / ^+1 to 3K and so on. If the projector has a total number of columns N, and the number of columns per group is K, then the number of groups m will be N / K. In the example shown in Figure 7, the value K is chosen as 16. For the purpose of explanation, Figure 7 only shows the first four groups of columns; it will be appreciated, however, that the projector may comprise a much larger number of columns across its width. For example, the total number of columns N may be of the order of 1968. Assuming that the number of columns K in each group is 16, this will equate to a total number of groups m = 123. At each time step, a single column of projector elements will be activated in each group of columns. Figure 7A shows the columns illuminated at the first time step in the sequence. Here, the first column in each group of columns is illuminated. Figure 7B shows the columns illuminated at the second time step in the sequence t2. Here, the second column in each group of columns is illuminated. Figure 7C shows the columns illuminated at the third time step in the sequence t3. Here, the third column in each group of columns is illuminated. Figure 7D shows the columns illuminated at the fourth time step in the sequence t4. Here, the fourth column in each group of columns is illuminated. Since there are 16 columns in each group, the illuminated lines remain spaced apart by 16 columns at each time step. It also follows that the total number of patterns in the sequence will be equal to the pitch (column spacing) K\ once the 16th column in each group has been activated, each one of the columns of projector elements will have been activated once across the sequence of time steps. (It will be appreciated that in some embodiments, the lines of illumination may be widened so that two or more adjacent columns of projector pixels are illuminated at once; in each case, the column(s) of elements to be activated in the subsequent time step will begin with the first column that was not illuminated in the previous time step. So, for example, columns 1 and 2 may be activated at time tr, columns 3 and 4 may be activated at time t2 and columns 5 and 6 may be activated at time t3, etc. For each camera pixel in the detector array, it is possible to measure the intensity of light received in that camera pixel over the course of the sequence of images. Figure 8 shows an example plot of measured intensity in a camera pixel at each one of the 16 time steps in the sequence of line patterns, as normalized by the time step having the highest intensity value across the sequence of images. Figure 9 shows the normalized measurements of intensity plotted on a graph. Here, it is possible to discern clearly two peaks in intensity at time points t6 and t14; these peaks indicate frames in the temporal sequence of images in which a signal reflected from one of the object surfaces was reflected back to the camera pixel. A threshold may be applied to separate genuine peaks in the signal from background noise. Since each column of the projector is illuminated once only in the sequence of line patterns, and the camera pixel of Figure 8 sees two peaks over the course of that signal, it follows that the camera pixel is detecting light projected from two separate columns in the projector array. By identifying the position in the temporal sequence of the two peaks, and knowing which of the projector columns were active when those images were captured, it is possible to determine which of the projector columns the rays incident on the camera pixel originate from. Note that at this stage, it is not possible to determine the global position of those columns as the camera does not know which group of columns m the rays originate from; that is, referring to Figure 7, it is not known whether the columns are those of the first set of 16 columns, or the second set of 16 columns, or the third set of columns etc. Instead, what can be determined at this stage is which column number 1 to 16 (within the as yet unspecified set of columns) was illuminating the scene when a reflection was detected at the camera pixel 103 - in this case, column numbers 6 and 14. In order to obtain the absolute column number, an additional set of coded patterns may be projected onto the field of view so as to enable the global localization of the column position. The coded patterns may comprise, for example, gray codes i.e. a set of binary codes, with the property that only a single bit changes between subsequent codes (this makes the codes more robust against e.g. lens blurring). The gray codes may be projected, for example, such that projector columns 0 and 1 project a gray code index 0, projector columns 2 and 3 project a gray code index 1, and in more general projector columns y, y +1 project a gray code index y / Z, where Z is the number of repetitions of each code (in the example above Z = 2). For a projector that is 1020 pixels wide, and with the same code index used for every two subsequent projector columns, one would need to generate 510 different gray codes. Minimally, this will require a gray code consisting of T = 9 bits, yielding 29 = 512 different codes. The first image will project the first bit of all the codes, the second image will project the second bit of all the codes, and so on. The number of columns Z over which each code repeats is a variable that can be tuned (more repetitions means fewer gray code patterns), although in general it is preferable for the number of gray codes to be rather high, such that the number of columns having the same gray code is smaller than the pitch K of the lines used in each pattern, and with sufficient extra margin to cover thick transparent objects. If we let gc(r, c, t) be the value of the camera pixel at position (r, c) at time t (which is equivalent to image number t) within the set of images comprising the gray code images, the gray codes can be used to identify the projector column through the following steps: 1. For each camera pixel, find the minimum and maximum pixel intensity across the images, respectively a(r, c) and b(r,c). 2. Calculate a per camera-pixel threshold t(r, c) = a^r’c^b<^r’c^ 3. Classify each camera pixel observation as: x(r,ct)= >f(r'C) ( 0, otherwise 4. Convert the pixel observation into its Gray Code number by calculating n(r, c) = St=o x(.r> c> t) ’ 5. Use a lookup table that returns code index i when the gray code number n is input. 6. Convert the code to projector column by multiplying it with Z. The above algorithm assumes that the gray codes are designed such the code consisting of only zero-bits and only one-bits are excluded from the projected signal. It will be appreciated that other methods are available to decode the gray codes and retrieve the underlying column numbers, as described in “3D Imaging, Analysis and Applications” (Liu, Y. et al., Springer Cham, 11 / 12 September 2020, ISBN 978-3-030-44070-1), which contains a recent overview of state-of-the-art methods. Having processed both the images captured when illuminating the scene with the line patterns shown in Figure 4 and the images captured when illuminating the scene with the gray codes, two sets of results will be obtained for the projector columns. We can use gp(r, c) to denote the projector column returned for a camera pixel from the gray code as per the above algorithm, and mp(r, c, / cO to denote the projector column(s) returned when illuminating the scene with the line patterns. Here, kt indicates the position (i.e. frame number) of the jth peak seen at the camera pixel when illuminating the scene with the line patterns (in the example seen in Figure 8, = 6 and k2 = 14). As before, we denote the pitch of the lines in the line patterns as K. The data signals from the line patterns and the gray code patterns can be combined to obtain the final projector column value fp(r, c, k^ for each peak kt seen at a particular camera pixel by calculating: fp(r,c,kt) = mod(mp(r,c,ki) — gp(r,c) + o,K) + gp(r,c) Here, o is an offset used to ensure the alignment of the data from the line patterns and the gray code patterns and mod is the modulo function. It will be recognized that the use of gray codes here is optional, and stems from the decision to use multiple illumination lines in each one of the line patterns. For improved performance, in case of no gray codes being used, the multiple illumination lines should also only be used alongside knowledge of the disparity window. In some embodiments, a single column of the projector may be illuminated at each time step in the sequence of line patterns, thereby allowing the absolute column number to be retrieved immediately without the need to also illuminate the scene with the gray code patterns. Limiting the number of illumination lines to a single line in each pattern does, however, carry the cost of a much higher acquisition time. In many applications, therefore, it will be preferable to proceed with a combined multi-line and gray code approach. By carrying out the steps above, it is possible to determine, for each camera pixel, the column(s) in the projector array from which the camera pixel receives a light signal. This information can then be used to generate a graph, showing the columns in the projector array that each camera pixel receives a signal from. As an example of this, Figure 10 shows a plot of the projector columns that are illuminated when each camera pixel detects a light signal Sj in the arrangement shown in 4. Here, the camera pixels 103a, 103b, 103c... are arranged along the x axis in the order in which the pixels would expect to receive light in the event that the projector columns were illuminated in the order shown on the y axis and with no inter-reflections being present; that is, in the event that the projector columns were illuminated in the order 101a, 101b, 101c... etc., the geometry of the system means that one would expect to see light appear in camera pixels in the order 103a, 103b, 103c ....etc. This ordering can be determined at the outset when the imaging system is first calibrated. Figure 10B shows the same information as in Figure 10A, but with the x and y axes switched, such that time now runs on the x axis; that is, the x axis shows the sequence of projector columns being illuminated over time, whilst the y axis shows the camera pixels that detect light when each respective projector column is illuminated. If one considers the camera pixel 103b, for example, it can be seen that this receives two light signals S3 and S4 during the course of the sequence. The first of these two signals, S3, is received at t2, when the projector column 101b is illuminated. The second of the received signals S4 is received at ts when the projector column 101e is illuminated. Next, for each one of the signals Sj, an inspection can be made of neighboring camera pixels for signals that are within a threshold distance from the pixel under consideration, where that threshold is defined in projector column space. Referring to the signal S3 detected in camera pixel 103b in Figure 10A, for example, this means finding the signals Si and S5 as they are close both in the camera domain and projector domain. Signal S4 can be ignored as it is the same camera pixel 103b, and signals S2 and Se can be ignored as they are too far away from the pixel in question; the exact size of the neighborhood to consider will vary with the total system resolution but in general, a neighborhood of approximately 5x5 camera pixels (and a threshold of 3-4 projector pixels) may be a suitable implementation. The same process can then be applied in respect of the second signal S4 in camera pixel 103b; here, signals S2 and Se are identified as lying within the necessary threshold. The next step is to estimate the gradient in the camera pixel domain from camera to projector pixels based on the neighbouring points. In Figure 10A, the gradient can be determined by fitting line 1001 to points Si, S3 and S5, and fitting line 1003 to points S2, S4 and Se (other robust methods for gradient estimation for subsequent filtering can of course also be considered e.g. using RANSAC based methods to fit the gradient). In this example, with the camera pixels and projector columns arranged on their respective axes in the manner discussed above, there is a positive correlation 1001 between the camera pixels and the projector columns for the direct reflections (rays Ri, R2 and R3). For the indirect, inter-reflections (rays R4, Rs and Re), the correlation 1003 between the camera pixels and projector columns is negative. This is due to the fact that the partially specular surface in Figure 4 acts partially as a mirror, meaning that the projected pattern will be mirrored along the mirror axis. The reverse correlation is thus due to the mirror symmetry imposed onto the projected pattern sequence. The above principle can be further understood with reference to Figures 11 and 12. Figure 11 shows a ray diagram for the imaging system of Figure 1. Here, the two planes 1101, 1103 represent the projector array and the camera sensor array, respectively, with the projector having its optical centre at P and the camera having its optical centre at C. The letter O represents the object being imaged. The line 1105 joining P and C defines the baseline of the camera / projector system. Together, the points C, P and O define the epipolar plane for the object O. The face of the projector array 1107 and the face of the camera sensor array 1109 are depicted below. With the projector plane 1101 and camera plane 1103 oriented as they are, the row of projector elements 1111 and the row of camera pixel elements 1113 are seen to lie in the same epipolar plane. Figure 12A shows the same arrangement as in Figure 11, with light rays being emitted from three projector elements that are spatially arranged in the order x, x + 7, ... x + n in the row of projector elements 1113. The rays from the projector elements x and x + 7 are reflected from a first surface 1201 before then being reflected from the second surface 1203 towards the camera. The light ray from projector element x + n does not undergo any such inter-reflections but passes directly to the surface 1201 before being reflected towards the camera sensor. Figure 12B shows the locations in the row of camera pixels 1113 at which the rays x, x + 7, ... x + n are incident on the camera for the case shown in Figure 12A. Figure 12C shows how the locations would change in the event that there were no inter-reflections from the second surface 1203. It can be seen from Figure 12B that the inter-reflections from the second surface 1203 result in the signals x and x + 7 arriving at pixels in the row of camera pixel elements 1113 in the reverse order to that which would be the case had they not been first reflected from the surface 1203. Accordingly, it is possible to define a geometric constraint that must be satisfied by light rays having been emitted from neighbouring regions in the projector array and which have been reflected only once from the object; namely, such light rays, when incident on the detector array, should follow a spatial ordering consistent with their spatial ordering in the projector array. As further examples of this, Figures 13A and 13B show plots corresponding to Figures 10A and 10B for the scenario in Figure 5. Here, there is no positive correlation between the signals detected in the camera pixels, but instead only a negative correlation 1301, 1303. The signals incident on the camera pixels 103a, 103b, 103c do not, therefore, satisfy the necessary geometric constraint and should be discarded for the purpose of determining depth measurements. Figures 14A and 14B show plots corresponding to Figures 10A and 10B for the scenario in Figure 6; in this case, it is possible to discern two lines for the groups of signals Si, S3, S5 and S2, S4, Se, where both lines have positive gradients 1401, 1403. The positive gradients indicate that the signals received at the camera satisfy the above first geometric constraint, whilst the presence of multiple signals in each camera pixel may be accounted for by considering that some of the signals observed are ones arising from reflection off a (single) transparent surface, rather than multiple reflections off different surfaces. The line patterns described above are one example of a set of illumination patterns that may be used to identify which region(s) of the projector a particular camera pixel receives light from. An alternative embodiment will now be described in which the problem of identifying the region in the array of illumination points from which a particular camera pixel detects light can be considered as a compressive sensing problem. In this embodiment, the scene is again illuminated with a temporal sequence of illumination patterns, with each pattern being different from the others. Here, the illumination patterns are generated by defining respective illumination sequences for different regions in the array of illumination points. For each region, the illumination sequence defines how the amplitude of light projected from the respective region towards the scene varies at each time step. Figure 15 shows an example of how the illumination sequences may be defined by using a code matrix M. In this example, each region of points in the array of illumination points is taken to be a column in that array; as discussed above, however, this is by no means essential and in other embodiments, the regions may be formed as rows in the array of illumination points, or indeed any one of a number of different shapes formed from points in that array. Each region (in this example, column) is assigned a binary code, which in Figure 15 can be seen to extend in the vertical direction of the code matrix. In the present embodiment, the code comprises a sequence of bits that define, for each time step, whether or not that particular projector column is to be illuminated. For each column of the matrix, the white elements indicate time steps in the sequence for which the respective projector column should be switched on, whilst the black element indicate time steps in the sequence for which the respective column will be switched off. Thus, each column of the projector may be illuminated more than once throughout the sequence of time steps. Using these binary codes, each single camera pixel will receive one measurement of intensity Sj per time step. These measurements can be combined into a signal vector S = {s1,s2, ...,5 / } where I is the total number of images captured, equal to the number of time steps in the sequence. The signal vector S will be a linear mixture of the mixture of the codes projected, plus any ambient light X, which can be assumed to be constant during exposure. We can let the linear mixture be represented by a vector R = {r1;r2, -, rP), containing the relative response of each individual code projected. Subtracting the response from ambient light, the vector / ? will be very sparse, meaning that most elements will be zero. A reasonable estimate for ambient light can easily be achieved by A' = minS;. This means that the signal vector received by the camera i pixel can be modelled as: S = MxR+A After subtracting the estimated ambient light S' = S - A' the observation model can be formed as: S' = M x R (Here, each non-zero element of / ? will be positive, since it is not possible to “subtract” light from a scene). The number of non-zero elements in R will depend on the object being imaged. If the camera pixel only receives light from a single column of the projector, the vector / ? will contain a single non-zero element representing the code associated with that column of the projector. If the camera pixel is receiving light from multiple columns, because of inter-reflections and / or the presence of transparent surfaces, there will be a return signal for multiple ones of the codes being projected. This means that / ? will contain several non-zero elements. The above principle can be further understood with reference to Figures 16A and 16B. Figure 16A shows an example in which the scene is illuminated with light from three columns on the projector. Each projector column has an illumination sequence encoded as series of bits, which define the time steps at which the projector column is on. In the case of column 1, the illumination sequence may be written as 100011001, indicating that column 1 is to be switched on at time steps 1,5,6 and 9. The projector column 2 has the illumination sequence 010100110, meaning it is switched on at time steps 2, 4, 7 and 8. Projector column 3 has the illumination sequence 10100101, meaning it is switched on at time steps 1,3,6 and 8. It can be seen from this that each column is switched on a total of four times during the course of the acquisition. The upper line in Figure 16A shows the signal detected at a camera pixel over the course of the time sequence. Here, the signal detected at the camera maps directly on to the illumination sequence of projector column 1, with no contribution from projector columns 2 or 3. Following this, the vector R can be determined as R = {1,0,0}. Turning to Figure 16B, the codes for each one of the projector columns 1,2 and 3 are the same as in Figure 16A. Here, however, the signal detected at the camera pixel is a linear combination of the codes from columns 1 and 3, thus R = {1,0,1}. In order to determine which column(s) the particular camera pixel is receiving light from, therefore, it is necessary to solve for / ? in the equations above. As I « P, the linear system is grossly underdetermined, but since R is sparse, it is possible to use a non-negative least squares method to solve the problem and obtain an estimate of R (this estimate being denoted R' in what follows). More precisely, it is possible to solve: min |\M x R' - S'| |2, subject to S >0 In practice, owing to noise and signal blurring, R will not be fully sparse, but it is still sufficient to obtain an estimate R' where any very small are considered to be zero. One could envision other optimization targets to achieve the same sparsity, for example: where A is a suitable parameter balancing fit and sparsity. In practice, however, the method set out above will work well with appropriate algorithms. There are a number of suitable such algorithms known in the art. One example is the Lawson-Hanson algorithm, which performs an iterative optimization of the function, as described in “Solving Least-Squares Problems”, (Lawson, C. L. and R. J. Hanson., Upper Saddle River, NJ: Prentice Hall. 1974. Chapter 23, p. 161). Another method is Orthogonal Matching Pursuit, which performs the same process (see “Matching pursuits with timefrequency dictionaries”, (Mallat S.G., and Zhang, Z. , IEEE Transactions on Signal Processing, Vol. 41(12), 1993, p3397-3415). In summary, these algorithms work by first finding the code that matches the received signal S best, and determining which subsequent code can be used to explain the residual D. The process is repeated by adding additional codes through continued iterations until the residual is sufficiently small. It is of course possible to optimize the probing of by e.g. using the indices of the maximas of S; to speed up the process. After the optimization loop has converged (e.g. the residual D is sufficiently small), one can further remove noise by disregarding elements in R' which are below a chosen threshold. The threshold can be either set absolute or relative to the maximal element of / ?', where a relative threshold is the preferred option. Furthermore, in the case of camera pixels having a single return signal, the neighbouring codes are easily included as the secondary candidates. These can be filtered out by post-filtering R' to remove subsequent non-zero indices, preserving only the maximal of the subsequent codes. By itself, the Lawson-Hanson method (and similar methods) will only determine the closest integer code indices that can explain the signal S’. For enhanced 3D precision, it is necessary to recover the code index with sub-integer precision. In practice, this means refining a detected integer code index (and thus projector column) into a projector column / code index r- with more than integer position. This can be performed by calculation the correlation score CS(j) = £-=1 M(i, + j) ■ ,-] <j <J with the J neighbouring codes (typically 1-2), fitting a parabola to CS(j) and using the maxima point of the fitted parabola as the sub-integer code index. The optimization problem can be further constrained by considering the relative disparity window between the camera and the projector. The disparity window defines the number of consecutive columns in the projector array whose light is capable of being detected on the same camera pixel. Owing to the geometry of the light source and the detector, a limit will be placed on that number. Accordingly, codes that are allocated to projector columns that lie outside of that disparity window (i.e. are located more than a certain number of columns away from the column in question) can be excluded from consideration. In most relevant setups, this may exclude up to 90% of the codes, with the contribution from those codes being set to zero for the camera pixel. There are also possibilities to employ methods based on deep learning to determine the contribution of each code to the signal detected at the camera pixel. An overview of applicable methods is provided in the journal publication “Deep learning for compressive sensing: a ubiquitous systems perspective,” (Machidon, A. L. and Pejovic, V., Artificial Intelligence Review, vol. 56, no. 4, pp. 3619-3658, Apr. 2023). These methods can be trained on a dataset generated by performing linear combinations of the codes of M, where a limited number of codes (2-4 typically) are mixed together with different amplitudes and used as input, and with their corresponding code indices used as groundtruth for training the network. The camera / projector system will blur out the projected codes. This can be compensated by pre-blurring the matrix M with the expected point spread function prior to R' estimation, and / or performing a local search for neighbouring codes once a primary code is identified in the iterative optimization loop. It will be appreciated that the results obtained using the code-based approach shown in Figures 15 and 16 can be enhanced by control of certain parameters. In the embodiment shown here, each code has a constant amplitude (i.e. the number of “on” bits is the same for all codes, meaning that each column of the projector will be illuminated for the same overall amount of time over the course of the acquisition). The number of bit changes in codes for adjacent columns may also be kept constant, somewhat akin to a gray code. These constraints are by no means essential, but can improve the convergence speed of the algorithm. Moreover, the performance may be improved by reducing the correlation of codes allocated to columns within the disparity window of the system. As is well-known in the art, the disparity window references the code range that can be observed by a single camera pixel due to the geometry of the imaging setup. For one particular camera pixel, an object at zero distance from the camera will observe code index c0, whilst an object at infinite distance will observe code index . c0 and will vary with whichever pixel is selected. If prior knowledge on e.g. working range is available (e.g. that all objects will be within a distance d1 and d2, the minimum and maximum code indices per camera pixel can be further reduced, thus limiting the effective disparity window W further. If W is the disparity window for a projector column p', then it is desirable for the correlation X(p") = • M(i,p"), Ip" - p'| <W a \p" - p’\ >C)} to be as low as possible for all p", with the exception of the codes that are less than C code indices away from pr, where C is the width of the main lobe of the cross-correlation matrix. The precise value of C will vary according to the code design strategy but for a code design similar to the one exemplified in Figure 15, where one has four bits switched on per code, and one bit is switched on and one off per transition, the main lobe has a width of C = 3. This effective disparity window can also be used during runtime to optimize the performance of the decoding algorithm, as mentioned earlier. The improvement afforded by controlling the above parameters can be understood with reference to Figures 17 to 20. In more detail, Figure 17 shows the cross-correlation matrix for the codes shown in Figure 15. It can be seen here that the intensity along the diagonal is constant, this being a result of the fact that each code has the same number of “on” bits. It can also be seen in Figure 17 that the correlation between codes is particularly reduced for columns that are located close to the column in question and which lie within the disparity window; by reducing this correlation, it is possible to minimize interference between those columns whose light is capable of being detected in the same camera pixel. Figure 18 shows a magnified section of the matrix of Figure 17, in which the width C of the main lobe can be seen to equal that of three columns. C should preferably also be kept as low as possible, but in practice a trade-off between W, C,I and X must be found to strike a balance between performance and time. In the exemplified case, W = 100, C = 3,1 = 37, X <1.0 where I is the number of images in the sequence. As a comparative example, Figure 19 shows an alternative code matrix in which the only constraint that is applied is that a single bit changes between consecutive codes (note that although the codes for pairs of consecutive columns are the same in this example, this does not affect the results). Figure 20 shows the cross-correlation matrix for the code matrix shown in Figure 19. In contrast to the cross-correlation matrix shown in Figure 17, the intensity along the diagonal varies quite strongly; this is due to the fact that the number of “on” bits per code varies, meaning that signal normalization is required. There is also no clear band around the main diagonal where interference is minimized; this is due to that no effort has been made to design the code such that the correlation between codes within the disparity window is minimized. It will be appreciated that whilst the embodiments described above with reference to Figures 15 to 20 rely on the regions (columns) being switched “on” and “off” at different time steps, in practice, the intensity of the light source may still have a finite amplitude at each time step; that is, the system may be configured such that for time steps in the sequence where the column is “off”, light will still emanate from that region of the array of illumination points, but with a reduced amplitude compared to time steps where the column is “on”. In the extreme case, this reduced amplitude may be zero, but in general “off” need not signify the absence of signal altogether. Moreover, in some embodiments, the illumination sequence for each region (e.g. column) may be defined by applying a particular frequency modulation to the amplitude of light emanating from that region. That is, rather than each region having a binary sequence of “on” and “off” periods throughout the course of the acquisition, the amplitude of the intensity may be made to vary more continuously over the course of the acquisition. For example, the intensity of light emanating from each region may vary sinusoidally over the course of the image frames, with the frequency of that modulation being different for each region. In this case, each illumination sequence may still be encoded using one or more bits, but here the bits will encode the frequency modulation to be applied for that region. The signal then detected at each camera pixel over the course of the image acquisition will be a sum of the frequencies associated with regions in the array of illumination points from which light incident on the camera pixel is emanating. In a similar way to that described above, the signal detected on the camera can be unmixed to recover the contributing frequencies, and in turn the regions (columns) from which light detecting in that pixel is emanating. In the same way that the illumination patterns shown in Figure 7 may be used to generate a plot indicating the projector column(s) that different camera pixels receive light from, so the compressive sensing method discussed above with reference to Figures 15 to 20 may also be used to generate such a plot. An example is shown in Figure 21. As before, the camera pixels 103a, 103b, 103c ..., are arranged along the x axis in the order in which the pixels would expect to receive light in the event that the projector columns were illuminated in the order shown on the y axis and with no interreflections being present. In this example, a total of 12 signals Si to S12 are detected in camera pixels 103a to 103f. Camera pixel 103a, for example, can be seen to receive a light signal Si when column 101a is illuminated. Camera pixel 103b receives a light signal S2 when column 101c of the projector is illuminated and also receives a light signal Se when column 101k is illuminated. Camera pixel 103c registers light signals from three projector columns: a signal S3 when column 101e is illuminated, a signal s? when column 1011 is illuminated, and a signal S10 when column 1011 is illuminated. Using the plot shown in Figure 21, it is possible to deduce that signals Sw, Sn, and S12 originate from rays of light that have undergone one or more inter-reflections, as those signals together lie on a negative gradient 2101. Following this, signals Sw, Sn, and S12 can be discounted when performing triangulation measurements to ascertain depth information about the object. The remaining signals, Si to S9 each lie on a slope of positive gradient 2103, 2105. It will be appreciated that where the signals do lie on a positive slope, this does not necessarily rule out those signals from being the result of inter-reflections. It simply means that those signals cannot be immediately discounted as being the result of interreflections. In this case, further analysis of these signals may be required before it can be determined whether or not to discount them when computing depth information about the object. As discussed above, in embodiments described herein, the sequence of illumination patterns is designed such that it is possible to resolve, for any one camera pixel, the region(s) of points in the array of illumination points from which that pixel receives light. In the embodiments described above, the region(s) can be understood to correspond to columns of points in the array of illumination points. By considering the spatial ordering of the pixel elements in a particular epipolar plane, and the spatial ordering of the columns in the array of illumination points from which those pixel elements receive light, it is possible to determine whether mirror symmetry applies to the light rays associated with those columns, and in turn whether there are inter-reflections occurring within or close to that epipolar plane. It will be appreciated that the same principle can also be used to identify inter-reflections that occur in a direction perpendicular to the epipolar plane; in this case, rather than the sequence of illumination patterns being designed such that it is possible to resolve, for any one camera pixel, the columns of points in the array of illumination points from which that pixel receives light, the sequence of illumination patterns may be designed such that it is possible to resolve the individual rows of points in the array of illumination points from which that pixel receives light. Then, by considering the spatial ordering of the pixel elements in a plane perpendicular to the epipolar plane, and the spatial ordering of the rows in the array of illumination points from which those pixel elements receive light, it is possible to determine whether mirror symmetry applies to the light rays associated with those rows. In this way, the mirror symmetry can be used to identify where light rays have undergone inter-reflections from a surface inclined at any angle between 0 and 90° to the epipolar plane. A second embodiment will now be described that can be used to further discriminate between signals arising from direct reflections and signals arising from inter-reflections. The second embodiment relies on the use of an epipolar constraint to identify signals that arise as a result of inter-reflections. In this embodiment, it is necessary to determine simultaneously, for each signal detected at a given pixel on the camera sensor, both the horizontal and vertical coordinates of the projector element from which that signal emanates (in contrast to the first embodiment, therefore, each “region” may be understood here to correspond to an individual point in the array of illumination points, at a given column and row position). This in turn requires that the scene be illuminated using patterns that can allow for those horizontal and vertical coordinates to be accurately resolved. One possibility of achieving this goal is to project light from a single point on the projector and scan that point throughout the scene, using a zigzag pattern, for example. However, such an approach is highly inefficient, as a million or so different positions may need to be individually addressed, resulting in the data capture time being too long to be of practical use. To save time, a plurality of one dimensional (1D) patterns can be used. A 1D pattern can be understood to mean a pattern in which the illumination is uniform along a single axis, referred to herein as the “pattern axis”. As an example, a “vertical pattern” may be understood as being one that has a vertical pattern axis, as in the case where the pattern is a series of vertical lines, whilst a “horizontal pattern” will have a horizontal pattern axis, as in the case where the pattern is a series of horizontal lines. A vertical pattern will be able to provide information on the horizontal coordinates of the projector elements from which the light signals emanate, whilst a horizontal pattern will be able to provide information on the vertical coordinates of the projector elements from which the light signals emanate. Figure 22 shows further examples of different 1D patterns, which may include: • Single line patterns, where a single line is swept across the scene, one image being captured for each line position; • Multiline patterns, where multiple equidistant lines are projected, with the lines being swept across the scene until each position in the scene has been illuminated at least once; • Binary codes that rely on correlation and / or compressive sensing methods to decipher the individual code contributions; and • Spatial frequency sweeps employing (possibly phase shifted) sine waves. As discussed above with reference to Figure 7, some patterns may utilise multiple parallel lines that repeat spatially; in such cases, it will be necessary to avoid confusion between lines, either by having sufficiently few lines that geometrical constraints can be used to avoid such confusion, or by combining the codes with more global codes (e.g. Gray Codes) to achieve disambiguation. The direction and pattern type can be chosen at will according to the imaging problem at hand. It will be appreciated that the patterns in Figure 22 are illustrative only, and are scaled down dramatically with respect to resolution; in practice, most patterns will require higher resolution and / or pattern count to be used successfully. Figure 22 shows only the first nine patterns in each group i.e. the illumination patterns as they appear at each one of the first nine time points h - tg. The arrows at the base of each column indicate the direction of pattern axis for each of the four patterns. Taking the case of the single line patterns shown in Figure 22, the projected signal Pat time t can be defined as: P(rn> C„,t) = I ’ P I p p J (o; otherwise Where rp, cp indicates row / column of projector and t is pattern index (in time). The patterns will be projected one at a time, cycling through t, and the camera will capture images / (t) that are synchronized with the pattern projection. For a given pixel having coordinates rc, cc in the camera pixel array, the sequence of intensities / t(t) = I(r, c, t), can be analysed to find any global and local maxima of amplitude in time. There are several methods available for detecting such maxima in the time sequence. One example can be to first select the points where both previous and subsequent measurements are lower than the point being examined, followed by disregarding points not meeting an amplitude threshold. The patterns discussed above will each allow multiple reflections per camera pixel to be individually identified and localized (to within a wraparound window). It is noted that binary codes, frequency sweeps and similar patterns may need preprocessing steps such as Fourier transforms or cross-correlations before performing the identification and localization of the projector elements from which the different light signals originate. A single 1D pattern will not, by itself, be sufficient to distinguish the positions along the pattern axis of two projector elements whose light signals are both detected in the same camera pixel. This is exemplified in Figures 23A, 23B and 23C. Figure 23A shows the locations of two light emitting elements 2301,2303 on the projector, each of which emits a light signal that is reflected by the object to the same pixel in the camera sensor. The light signal from one of the two projector elements 2301 is a direct reflection (i.e. one that is reflected once from the object towards the camera sensor), whilst the light signal from the other projector element undergoes one or more interreflections before it arrives at the camera sensor. The first projector element 2301 lies within the epipolar plane 2305, whilst the second projector element 2303 lies outside the epipolar plane. If the projector illuminates the object with a sequence of vertical lines, meaning that the same signal is produced for all pixels in a particular column of the projector, then any vertical position information will be lost in the signal received at the camera. This is the case shown in Figure 23B, where the repetitive nature of the vertical pattern means that it is only possible to determine that the two projector elements 2301, 2303 lie in columns V1 and V2 of the projector array, with no information obtained regarding their position in the vertical direction (i.e. the row numbers of those projector elements). Figure 23C shows how the row numbers of the two projector elements might be inferred from epipolar considerations (snapping to the epipolar plane 2305), but this will result in an erroneous determination of the row number for the projector element 2303, which is (incorrectly) identified as being located in the same row as the projector element 2301, in the epipolar plane. To address the above problem, in the method described herein, a combination of two different 1D patterns with different axes of orientation may be used to sufficiently resolve the vertical position. The complexity in distinguishing between signals from direct reflections and those from inter-reflections can be reduced through recognition of the fact that the vertical position of the projector elements from which the direct reflections originate should not violate the epipolar constraint. This observation can provide a dramatic reduction in the number of patterns with which the object must be illuminated to identify the inter-reflections. To help explain this principle, it is useful to consider an example of two axes orientations that do not achieve the desired goal, as shown in Figures 24A, 24B and 24C. While the combination of these axes does not allow reflections to be resolved, this exemplifies the problem and also explains why particular orientations of axis are beneficial in terms of resolving reflections. Figure 24A shows the locations in the projector array of two projector elements whose light signals are both detected in the same camera pixel. The first of these projector elements 2401 emits a light signal that is reflected directly from the object towards the camera, whilst the second projector element 2403 emits a light signal that undergoes one or more inter-reflections before reaching the camera. If it can be determined that the signal from the second projector element 2403 does not follow the epipolar constraint (i.e. it lies outside the epipolar plane 2405), then the projector element 2403 can be discarded as a candidate origin of the light signal that is reflected directly off the object to the camera pixel. Figure 24B shows the results of projecting horizontal and vertical 1D patterns on the object. In effect, these pattern project the signal down to the axis of the patterns (points V1, V2, H1 and H2); that is, it is possible to determine that the origin of the light signal that is reflected directly off the object lies at one of the four coordinate points [V1, H1 ], [V2, H2], [V1, H2], and [V2, H1], Figure 24C shows the four projector elements 2401, 2403, 2407, 2409 that lie at these coordinates. It can be seen that there is an ambiguity in matching here; it is not possible to determine whether the two light signals located in the camera pixel originate from the two projector elements 2401, 2403 as shown in Figure 24A, or whether those light signals emanate from the other two projector elements 2407, 2409. While one could use e.g. amplitude or other information to try to resolve this ambiguity problem, in practice it is difficult to do so reliably. Since the two pairs of candidate elements 2401 / 2403 and 2407 / 2409 both include one point that is located within the epipolar plane 2405 and one point that is located outside of the epipolar plane, the epipolar constraint cannot be used to resolve the ambiguity and in turn identify the projector element 2401 as the true source of the light that is reflected directly from the object to the camera. Figures 25A - 25C show an example embodiment in which a different combination of 1D patterns to that shown in Figures 25A - 25C is used to determine the locations in the projector array of two projector elements whose light signals are both detected in the same camera pixel. As in Figures 24A - 24C, the horizontal axis indicates the baseline direction with the vertical axis perpendicular to the baseline. Referring to Figure 25A, the signals detected in the camera pixel originate from the projector elements 2501 and 2503. The first of these projector elements 2501 emits a light signal that is reflected directly from the object towards the camera, whilst the second projector element 2503 emits a light signal that undergoes one or more inter-reflections before reaching the camera. In this case, a mixture of vertically-oriented lines and diagonal lines are used as the illumination patterns. Here, signals are observed in the camera pixel when columns H1 and H2 are illuminated, and when the diagonal lines d1 and d2 are illuminated, as shown in Figure 25B. Figure 25C shows the locations of the projector elements 2501, 2503, 2507, 2509 that lie at the intersections of the columns H1, H2 and the diagonal lines d1, d2. As in the case of Figures 24A - 24C, there remains an ambiguity as to whether the signals emanate from the projector elements 2501,2503 or from the other pair of projector elements 2507, 2509. In this case, however, each one of the points lies outside the epipolar plane 2505, with the exception of projector element 2501. Accordingly, it is possible to discount the remaining three projector elements 2503, 2507, 2509 as being the source of the light that is reflected directly from the object to the camera pixel. Thus, the possible errors stemming from inter-reflections or from the ambiguity in association of lines H1 and H2 with lines d1 and d2 can be removed simultaneously. It will be appreciated that the method described above with reference to Figures 25A -25C will also achieve the desired outcome in the event that the two projector elements from which the light signals are received align with the direction of the projected lines. Figures 26A - 26C show such an example. As in Figures 25A - 25C, the horizontal axis indicates the baseline direction with the vertical axis perpendicular to the baseline. Referring to Figure 26A, the signals detected in the camera pixel originate from the projector elements 2601 and 2603. The first of these projector elements 2601 emits a light signal that is reflected directly from the object towards the camera, whilst the second projector element 2603 emits a light signal that undergoes one or more interreflections before reaching the camera. As in Figures 25A - 25C, a mixture of vertically-oriented lines and diagonal lines are used as the illumination patterns. In this case, however, the locations of the two projector elements 2601,2603 are aligned with the slant of the diagonal lines; that is, both projector elements 2601,2603 will be activated at the same time when the diagonal line d1 is illuminated. In contrast to Figures 24A - 24C and 25A - 25C, the problem of determining which projector element is the origin of the signal reflected directly towards the camera is simplified as there is no ambiguity and the projector element 2601 can easily be selected as the origin of that signal using the epipolar constraint. It will be appreciated that the two 1D patterns need not be of the same type; they can, for example, be mixed freely from any of the types of patterns shown in Figure 22, including patterns that provide either a complete or partial 1D mapping of the scene. The embodiments as described above rely on the assumption that the number of reflected points observed per camera pixel is rather low (typically 1 or 2 inter-reflections in addition to the signal from direct reflection). This is in correspondence with how most real-world scenes behave. The performance of these embodiments can be optimized based on the following principles (it will be appreciated that in cases where speed of acquisition is a crucial factor, one or more of these principles may be violated to more or lesser extent in the interests of obtaining a faster acquisition): • There must be two different orientations of the 1D patterns employed. This is to be able to calculate the intersection points between the lines generated by analyzing the reflections of the points. The two different orientations must differ sufficiently to ensure that the intersection points are found with sufficient precision to allow for filtering using the epipolar constraint. • If the two 1D patterns were to be projected onto the scene at the same time, any one line in the first set of illumination patterns should cross any one line in the second illumination patterns a maximum of once. • The lines in the first set of illumination patterns, and the lines in the second set of illumination patterns should cross any one epipolar plane a maximum of once. • Neither of the two orientations should be parallel with the baseline of the cameraprojector system. If one of the orientations is parallel with the baseline, the ambiguity problem illustrated in Figures 24A - 24C will occur, meaning that the epipolar constraint cannot be used to resolve the ambiguity in which one of the candidate locations is the true source of the light that undergoes direct reflection towards the camera. • The baseline of the camera-projector system should be known such that the epipolar constraint can be employed to filter out points that do not fall within the epipolar plane. These requirements still allow significant freedom in the patterns being projected, including those that are affected by lens distortion or other effects that mean the projected lines are no longer entirely straight. As an example, Figure 27A shows two 1D patterns, comprising of a series of vertical parallel lines, and a second series of parallel diagonal lines. Figure 27B shows the same two 1D patterns having undergone a lens distortion; owing to this distortion, the “pattern axes” in Figure 27B will not necessarily be straight, but there will exist a transform from the linear space to the actual projected pattern. The present embodiment utilizes two different subsets of illumination patterns (with different orientations) with those patterns being projected sequentially. Such an embodiment offers efficiency in terms of the small number of patterns employed (i.e. the sum of the number of patterns in each orientation). It is, however, still dependent on matching two different observations, which may result in a less robust method in the event that the signals are very weak and approaching the detection limit. To address this problem, a further embodiment will now be described in which a combination of patterns is projected simultaneously. For the purposes of illustration we will use the following two subsets of illumination patterns: 1. Vertical lines with equidistant spacing, with the spacing sufficient that the likelihood of two lines being confounded is sufficiently low. This typically means that the line spacing is far larger than the expected disparity range for the system (with an additional margin to allow for reflections). 2. Horizontal sine waves that are phase shifted. Figure 28 shows an example of two subsets of illumination patterns that can be combined for the mixed encoding. The first pattern is a vertical “multi-line” pattern, whilst the second pattern is a horizontal sine-wave pattern. Figure 28 shows the first subset often spatially shifted variations of the multi-line pattern mH, ml2,... ml10, and the second subset of three phase shifted variations of the sine-wave pattern sin1, sin2 and sin3. Were these patterns to be projected sequentially, they would provide no additional information and the same ambiguity problem would occur as described above in relation to Figures 23A - 23C. By multiplying the elements in the two subsets Ic (r, c, ni) = I(r, c, m, p) of patterns with each other, it is possible to obtain the illumination patterns shown in Figure 29. Multiplication is performed by multiplying each pattern element-wise with each other. The patterns so obtained can then be used as follows: • Project the images onto the scene in sequence, and capture them with a camera. Let the set of received images be I(r, c, m, p) where r, c indicates row and column, m indicates multiline index and p indicates phase index, with both indices starting at 0. • For each group of three images containing the phase shifts superimposed onto a single multiline image, calculate the per camera pixel phase and amplitude across the three images. It can be useful to do this through complex numbers, and thus calculate Ic (r, c, m) by 2 ~j2np ■ e 3 p = 0 The amplitude of Ic will now contain the response of the multiline signal (e.g. the horizontal position), whereas the phase of / c will contain the vertical position (modulo the sine wave repetition interval). • The value of Ic can now be used to obtain both the horizontal and vertical coordinates of the projector pixel observed by a camera pixel as follows: • First, detect the k strongest peaks per camera pixel in the time domain (e.g. across m). This will yield, for each camera pixel (r, c), a set M containing the position in time (and thus the projector column) of the strongest peaks for each camera pixel. • For each of the detected peaks m' e M, compare the angle of the complex number arg / c(r, c,m') (which indicates the projector row emitting the observed peak) with the projector row expected from epipolar geometry (the epipolar plane). If the discrepancy is too large, the peak will removed from the set m' as it is likely an inter-reflection. Figures 30A and 30B show the use of the embedded encoding approach for identifying signals arising from inter-reflections. Figure 30A shows the location of two projector elements 3001, 3003 whose emitted light signals are both detected in the same camera pixel. The first of these projector elements 3001 emits a light signal that is reflected directly from the object towards the camera, whilst the second projector element 3003 emits a light signal that undergoes one or more inter-reflections before reaching the camera. Figure 30B shows the results obtained by projecting and processing the embedded encoding patterns shown in Figure 29. The amplitude information of the patterns indicates that the projector elements 3001, 3003 are located in columns V1 and V2, whilst the embedded phase information indicates that the vertical position of the projector element in column V1 is at H1 (indicated by short line), and the vertical position of the projector element in column V2 is at H2 (also indicated by a short line). Since only one of these two projector elements is located in the epipolar plane 3005, the projector element 3001 can be identified as the one whose light signal is reflected directly from the object towards the camera. It should be noted that the approach described above allows sufficient separation of reflections to ensure the embedded phase information remains largely unaffected. The multi-line part of the code ensures that the time of the direct signal and the indirect reflection appears at separate times. The zero-regions between the multi-line reduces any interference between different signals that could affect the embedded phase information. The pattern decoding itself can be performed efficiently by decoding the phase information first and independently of any decoding of the multiline. The decoding of the vertical multi-lines can be done directly on the amplitude information, without regard for the embedded phase information. Only in the final step is the link made between the two encoded patterns. Unlike fully 2D patterns, in which each projector pixel has its unique temporal signature that needs to matched against an enormous database of per-pixel signatures, this ensures that decoding step can be performed very easily using modern massively parallel architectures. It will further be appreciated that many alternative pairs of patterns can be selected to achieve this embedded encoding. It is however beneficial that the first subset of illumination patterns is sparse e.g. multiline or similar patterns that have large regions of zero intensity (and preferably, where a majority of points in each pattern have zero intensity) to avoid interference problems with the second pattern set. By way of example, the first subset of illumination patterns may be based on any one of the single line (vertical), multi-line (diagonal) and binary code (vertical) patterns shown in Figure 22. For the second subset of illumination patterns, there are fewer considerations as the reflection separation ability is largely ensured by the first subset of illumination patterns. In the embodiment described above, phase shifted sine waves have been used as an example for the second subset, but this is by no means essential and many other alternate patterns can be envisioned, including: Hamiltonian codes • Gray codes • Multi-frequency sine waves As an example, rather than using three phase-shifted sine wave patterns as shown in Figure 28, four patterns may be used encoding different codes in each row: Code 1:1 10 0 (shown in row 1) Code 2:0110 (shown in row 2) Code 3: 0 0 1 1 (shown in row 3) Code 4:1010 (shown in row 4) Code 5: 1 0 0 1 (shown in row 5) Code 6:0101 (shown in row 6) In this example, it will be possible to decode six different codes, with each code corresponding to a different row of the projector array, in the same way that a particular phase of the sine wave would correspond to a particular row. The amplitude may be obtained by summing across the four images captured when multiplying each one of the first subset of illumination patterns with the four coded patterns. The code index may then be obtained by correlating the four images with the code bank shown above, and identifying the closest one of those six codes (in practice, one would might interpolate between the six codes to achieve sub-code precision in resolving the row). Whilst by no means an essential requirement, the system may have an improved performance if the second subset of illumination patterns are able to provide a constant amplitude for all codes projected. This is the case with three-phase sine waves but can also be achieved using other codes if suitably designed. It will be noted that the use of embedded encoding as described above with reference to Figures 28 and 30 allows both the column position and row position of a point in the illumination array to be identified as the source of a light ray incident on a camera pixel. This means that one can use the mirror symmetry constraint described in earlier embodiments to identify any inter-reflections in both the horizontal and vertical directions. Accordingly, using this approach, one can identify and filter out interreflections even if the epipolar plane is not known but only the overall ordering of the projected array and camera array is known. This can be beneficial to e.g. avoid costly calibration steps required to establish the epipolar plane for each individual camera pixel. The embodiments as described herein allow one to identify, for a given camera pixel, light signals that are incident on that pixel and which have undergone one or more inter-reflections between leaving the projector and arriving at the camera. It will be appreciated that precisely which of the methods above works best for identifying and discarding signals arising from inter-reflections will depend largely on the orientation(s) of the reflecting surfaces. Referring back to Figure 4, Figures 31A - 31D illustrate how the rays Ri - Re shown in Figure 4 will reflect in different directions, depending on the orientation of the partially reflective surface. Beginning with Figure 31A, this shows the case in which the partially reflective surface 3101 is oriented perpendicularly (0 = 90°) to the epipolar plane 3103. In this case, the mirror symmetry and order inversion is present as described above in reference to Figures 10 to 14. Figure 31B shows the case in which the partially reflective surface 3101 is oriented at 0 = 60° with respect to the epipolar plane 3103. Here, the mirror symmetry is still somewhat preserved. At the same time, the reflected points are reflected off the original epipolar plane. Figure 31C shows the case in which the partially reflective surface 3101 is oriented at 0 = 45° with respect to the epipolar plane 3103. In this case, the mirror symmetry is zeroed out. The points are however reflected outside the original epipolar plane. Figure 31D shows the case in which the partially reflective surface 3101 is oriented at 0 = 30° with respect to the epipolar plane 3103. In this case, within the epipolar plane, the points will appear in the same order as regular points and the mirror symmetry can no longer be used as a basis for distinguishing the direct light paths from the inter-reflections. The points are, however, reflected outside the original epipolar plane; hence it may still be possible to use the mirror symmetry constraint to identify inter-reflections perpendicular to the epipolar plane. Following the above, inter-reflections can be assigned to one or more of three regimes, depending on the angle of inclination 6 between the surface(s) from which those rays are reflected and the epipolar plane. Figure 32 shows the range of angles covered by these three regimes. The first regime R1 accounts for cases in which the reflective surface is oriented at an angle in the approximate range 6 = 0° - 45°; in this regime, the mirror symmetry I order inversion in the plane perpendicular to the epipolar axis (or near perpendicular to the epipolar axis) may provide a suitable geometric constraint for determining whether or not light incident on the camera has undergone one or more inter-reflections en-route to the camera. The second regime R2 will account for cases in which the reflective surface is oriented at an angle in the approximate range 6 = 45° - 90°°; in this case, the mirror symmetry I order inversion in the epipolar axis (or close to the epipolar axis) as described above in relation to Figures 11 and 12 may provide a suitable geometric constraint for determining whether or not light incident on the camera has undergone one or more inter-reflections en-route to the camera. The third regime R3 will account for cases in which the reflective surface is oriented at an angle in the approximate range 6 = 0° - 90°; in this case, the epipolar constraint as described in reference to Figures 23 to 30 may be used for determining whether or not light incident on the camera has undergone one or more inter-reflections en-route to the camera. It can be seen from Figure 32 that most angles 6 will be covered by two of the three regimes, meaning that inter-reflections may be identified using both the epipolar constraint and the mirror symmetry constraint, with the latter being applied in the epipolar plane for the angles 0° - 45° and in the perpendicular plane for the angles 45° - 90°. However, the methods using these different constraints will be subject to greater noise at the boundaries 3201a, 3201b, 3201c; therefore, the use of the mirror symmetry constraint will provide more optimal results for 0 = 0° - 5° and 0 = 85 - 90°, whilst the epipolar constraint will provide more optimal results for 6 = 40° - 50°. Following the above, and assuming one has some prior knowledge of the shape and orientation of the reflecting surfaces, it will be able to use such information to achieve a compromise between performance and acquisition time. The exact selection of method(s) may be adapted, for example, as follows: • If one knows that the inter-reflections are more likely to fall in the first regime, a single 1D pattern can be applied, followed by the gradient-based filtering approach, as shown in Figures 10, 13, 14 and 21. This will be the fastest acquisition method that still allows reflection suppression. • If there are limited amounts of inter-reflections present, and the signal-to-noise level is good, the use of two 1D patterns with different orientations may be sufficient to achieve desired performance whilst keeping overall acquisition time reasonable. • If the signal-to-noise level is low and the number of inter-reflections in the second regime is limited, the embedded encoding approach may be used. This will take more acquisition time, but may be necessary to perform filtering also in more challenging scenes. • If the number of inter-reflections in the second regime is large, the combined approach may be utilized to enable the best possible removal of inter-reflections. This will take more time, but may provide the finest sieve for such unwanted reflections. It will further be appreciated that, in view of the “overlap” between Regime 1 and Regime 2 (see Figure 32), there will be scenarios in which more than one of the described methods will yield a highly satisfactory result in terms of discriminating such inter-reflections. In total, the methods described herein can facilitate management of the full set of reflections observed in most relevant industrial scenes, independently of the orientation of those reflections. The methods described herein can remove the incorrect, indirect paths, leaving the correct, direct paths for 3D reconstruction. It will further be appreciated that the embodiments described above with reference to Figures 14 - 22 may be combined to provide maximum protection towards interreflections occurring in the second regime of Figure 32. As an example, one may use the following combination of patterns: • Vertical multilines with embedded phase encoding • Diagonal multilines without embedded phase encoding Both the embedded phase encoding and the diagonal multilines will be subject to wraparound problems, as the pattern repeats vertically along the projector column. To circumvent this, it may be beneficial to let the vertical wraparound be different for the embedded phase encoding and the diagonal multiline. For example, one may choose the embedded phase encoding to have a short vertical wraparound distance, and the diagonal multilines to have a longer wraparound distance. To decode this, one may proceed as follows: • First process the vertical multilines with embedded phase encoding according to the procedure described above with reference to Figures 28 to 30. Doing so will leave a set of peaks position that have been pre-qualified as unlikely to be influenced by interreflections outside of the epipolar plane. However, as the embedded phase shifting wraps around, some “long distance” reflections may remain. • Combine the pre-qualified points with the diagonal multilines according to the procedure described above in relation to Figures 25 and 26 to further filter out also long distance reflections. Ideally, the wraparound distance for the embedded phase encoding and the diagonal multiline would differ, such that the two procedures would be able to detect and filter out reflections of different sizes mostly regardless of vertical transport distance, it will be appreciated that global codes (e.g. the compressive sensing described above with reference to Figures 15 to 20) may effectively be used for long range reflections as well. Here, a combination of a set of vertical binary codes and diagonal binary codes can effectively be used together with a set of vertical and diagonal multi-lines to simultaneously handle both short and long reflections whilst simultaneously achieving high 3D resolution. Embodiments as described herein may be used in numerous industrial settings. Figure 33 shows an example in which an imaging system according to an embodiment is employed by a robotic device in a factory or warehouse. The robotic device 3300 may be one of a number of such devices operating in the factory or warehouse. The robotic device 3300 includes one or more arms 3301 configured to manipulate an object 2603. The robotic arm 2601 may be used to lift the object 3303 off a shelf 3305 on which the object is placed, for example. In order to do so, the robotic device will need to determine the object’s position in space in order to properly engage with the object. The robotic device’s ability to do so may be compromised, however, if the object 3303 has multiple reflective surfaces. To mitigate against this, the robotic device may utilize an imaging system as described herein to distinguish between direct reflections and inter-reflections, and so enable it to more accurately profile the shape and size of the objects being manipulated. In the example shown in Figure 33, the robotic device itself includes an imaging system 3309 according to the embodiments described herein, which it uses to map the surface(s) of the object prior to engaging with it. It will be appreciated, however, that the imaging system need not be part of the robotic device itself, but could be housed separately from the robotic device, and used to feed information about the object to the robotic device prior to its engaging with the object 3303. The imaging system may also be used for inspecting the surface of objects. In this case, the 3D data captured of the objects would be compared with e.g. CAD models to look for defects, or one could use local surface characterization methods (e.g. edge detection filters) to detect surface imperfections. In the case of a defect is detected, either a robotic device similar to Figure 33 may be used to remove the object, or one may log the error in a suitable database for later processing and handling of the part. It will be appreciated that implementations of the subject matter and the operations described in this specification can be realized in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be realized using one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). While certain embodiments have been described, these embodiments have been presented by way of example only and are not intended to limit the scope of the invention. Indeed, the novel methods, devices and systems described herein may be embodied in a variety of forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the invention. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the invention.

Claims

1. An imaging system for imaging a scene comprising an object, the system comprising:a light source arranged to illuminate the scene by projecting light rays through an array of illumination points in space, wherein for each one of a sequence of time steps, the light source is configured to illuminate the scene with a different illumination pattern by projecting light rays through a different group of points in the array, wherein for each illumination pattern, a plurality of light rays are projected through different points in the array;a detector comprising an array of pixel elements, the detector being arranged to capture an image at each time step by detecting, on the array of pixel elements, light projected towards the scene and reflected from one or more surfaces of the object towards the detector; andan image processor configured to:identify, for a first pixel element of the detector, and based on knowledge of the illumination pattern used in each time step, one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass during the sequence of time steps; anddetermine, for each of the identified regions, whether a path travelled by the light ray that passes through the respective region and is incident on the pixel element satisfies one or more geometric constraints, the one or more geometric constraints being satisfied by light rays that have only been reflected once from the object before reaching the detector.

2. An imaging system according to claim 1, wherein the illumination patterns comprise a series of line patterns in which one or more lines are projected onto the scene.

3. An imaging system according to claim 2, wherein each line pattern comprises a plurality of parallel lines.

4. An imaging system according to claim 2 or 3, wherein each of the one or more regions of the array comprises a column of points in the array, and each line is formed by rays of light that pass through a respective one of the columns.

5. An imaging system according to any one of claims 2 to 4, wherein the line patterns are chosen such that each line is projected onto the scene once over the sequence of time steps.

6. An imaging system according to any one of the preceding claims, wherein the image processor is configured to identify the one or more regions of the array through which respective light rays(s) incident on the first pixel element pass by determining the timing of peaks detected in the light signal at the first pixel element during the sequence of time steps.

7. An imaging system according to any one of claims 2 to 6, wherein the illumination patterns further comprise a series of coded patterns from which can be determined a global location in the array of illumination points through which light rays incident on each pixel of the detector pass.

8. An imaging system according to claim 7, wherein the coded patterns comprise gray code patterns.

9. An imaging system according to claim 7 or 8, wherein the image processor is configured to identify the region(s) of the array through which light ray(s) incident on the first pixel element pass based on (i) the timing of peaks detected in the light signal at the first pixel element during the course of illuminating the scene with the plurality of lines patterns and (ii) the signal seen in the detector pixel over the course of illuminating the scene with the coded patterns.

10. An imaging system according to claim 1, wherein the light source is configured to generate the illumination patterns by defining an illumination sequence for respective regions of points in the array of illumination points, the illumination sequence specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region.

11. An imaging system according to claim 10, wherein each region of points comprises a respective line of points in the array of illumination points.

12. An imaging system according to claim 10 or 11, wherein for each pixel element, the image processor is configured to model the light signal incident on the pixel elementover the sequence of time steps as a function of contributions from the regions in the array of illumination points;the image processor being configured to identify the one or more regions of the array through which respective light rays(s) incident on the first pixel element pass by determining the one or more regions whose illumination sequences, when combined, result in the light signal seen on the first pixel element.

13. An imaging system according to any one of claims 10 to 12, wherein each illumination sequence is encoded using one or more bits, the one or more bits defining a relative amplitude of light to be projected through the region of points towards the scene in each time step.

14. An imaging system according to claim 13, wherein the one or more bits specify a frequency modulation to be applied to the light projected through the region of points over the course of the sequence of time steps.

15. An imaging system according to claim 14, wherein each illumination sequence is encoded as a sequence of bits, wherein each bit is associated with a respective time step, the value of each bit defining a relative amplitude of light to be projected through the region of points towards the scene in the respective time step.

16. An imaging system according to claim 15, wherein for each region, the illumination sequence defines one or more time steps at which light is to be projected from the region with a first amplitude, and one or more time steps at which the amplitude of light projected from the region is either reduced compared to the first amplitude or is zero.

17. An imaging system according to claim 16, wherein the illumination sequences are defined such that for each one of the region of points, the number of time steps in the sequence for which light will be projected with the first amplitude is the same.

18. An imaging system according to claim 16 or 17, wherein each region of points comprises a respective line of points in the array of illumination points and wherein the number of bit changes between the illumination sequences for each consecutive pair of lines is the same.

19. An imaging system according to any one of claims 17 to 18, wherein the regions comprise columns of points in the array of illumination points and the sequence of bitsfor each region is defined such that for any one region, a correlation between the sequence of bits for that region and the sequence of bit for other regions within a disparity window of that region is reduced compared to the correlation between the sequence of bits for that region and the sequence of bits for other regions outside of the disparity window, the disparity window defining the maximum number of consecutive regions in the array of illumination points, the light from which is capable of being detected on a single pixel of the detector according to the geometry of the system.

20. An imaging system according to any one of the preceding claims, wherein: the image processor is configured to determine if a path travelled by a light ray incident on the first pixel element satisfies the one or more geometric constraints by: resolving the light detected at the first pixel element into one or more separate detected signals, each signal being associated with light having passed through a different respective region of the array;selecting one of the detected signal(s) at the first pixel element, the selected signal being associated with light having passed through a first region of the array;identifying a second pixel element located in the same epipolar plane as the first pixel element, or in a same plane perpendicular to the epipolar plane, and within a threshold number of pixel elements from the first pixel element;resolving the light detected at the second pixel element into one or more separate detected signals, each signal being associated with light having passed through a different respective region of the array;selecting one of the detected signal(s) at the second pixel element, the selected signal being associated with light having passed through a second region of the array that lies within a threshold distance of the first region;determining an order for the first and second regions of the array based on the order in which the first pixel element and second pixel element appear on the detector;determining, based on knowledge of the illumination patterns, if the order of the first and second regions of the array matches an order that would be expected if light rays from the source and incident on the first and second pixel elements were only reflected once from the object before reaching the detector.

21. An imaging system according to claim 20, wherein:in the event the image processor determines that the order of the first and second regions of the array is the reverse of the expected order, the image processor is configured to discount the signal selected for the first pixel element whendetermining a distance of the object from the detector.

22. An imaging system according to claim 20 or 21, wherein the image processor is configured to determine if the order of the first and second regions of the array matches the expected order by:assigning a number to each region of the array based on its spatial location within the array;assigning a number to each pixel element based on its spatial location in the detector;generating a graph whose axes show the numbering of each region of the array against the numbering of each pixel element;plotting, on the graph, a first point whose coordinates reflect the number of the first region and the number of the first pixel element, and a second point whose coordinates reflect the number of the second region and the number of the second pixel;fitting a line to the first and second points on the graph; and determining whether the fitted line has a positive or negative gradient.

23. An imaging system according to any one of the preceding claims, wherein identifying one or more regions of the array through which respective light rays(s) incident on the pixel element pass comprises identifying a row and column in the array of illumination points through which each respective light ray passes.

24. An imaging system according to claim 23, wherein determining whether the path travelled by a light ray that passes through a respective region of the array and is incident on the first pixel element satisfies the one or more geometric constraints comprises determining whether the one or more regions lies in the same epipolar plane as the pixel element.

25. An imaging system according to claim 24, wherein the illumination patterns comprise a first set of illumination patterns and a second set of illumination patterns;wherein the first set of illumination patterns comprise a first series of line patterns in which one or more lines are projected onto the scene, the one or more lines being oriented at a first angle with respect to the baseline of the imaging system; andthe second set of illumination patterns comprise a second series of line patterns in which one or more lines are projected onto the scene, the one or more lines being oriented at a second angle with respect to the baseline, the second angle beingdifferent from the first angle.

26. An imaging system according to claim 24, wherein the illumination patterns comprise a first set of illumination patterns and a second set of illumination patterns;wherein the light source is configured to generate the first set of illumination patterns by defining an illumination sequence for each one of a first group of regions of points in the array of illumination points, the illumination sequence for each region in the first group of regions specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region.. wherein the light source is configured to generate the second set of illumination patterns by defining an illumination sequence for each one of a second group of regions of points in the array of illumination points, the illumination sequence for each region in the second group of regions specifying a variation in the amplitude of light projected through the respective region over the course of the sequence of time steps, the illumination sequence being different for each region.

27. An imaging system according to claim 26, wherein:each region of points in the first group of regions defines a respective line of points in the array of illumination points, each line being inclined at a first angle to the baseline of the imaging system;each region of points in the second group of regions defines a respective line of points in the array of illumination points, each line being inclined at a second angle to the baseline of the imaging system, wherein the first angle is different from the second angle.

28. An imaging system according to claim 26 or 27, wherein for each pixel element, the image processor is configured to:model the light signal incident on the pixel element over the course of the first set of illumination patterns as a function of contributions from the first group of regions;model the light signal incident on the pixel element over the course of the second set of illumination patterns as a function of contributions from the second group of regions; andidentify the one or more regions of the array through which respective light rays(s) incident on the first pixel element pass by:determining one or more regions in the first group of regions whose illumination sequences, when combined, result in the light signal seen on the first pixel elementwhen illuminating the object with the first set of illumination patterns; and determining one or more regions in the second group of regions whose illumination sequences, when combined, result in the light signal seen on the first pixel element when illuminating the object with the second set of illumination patterns.

29. An imaging system according to any one of claims 25 and 27 to 28, wherein the first set of illumination patterns and the second patterns are generated such that if the first set of illumination patterns and the second set of illumination patterns were projected onto the scene at the same time, any one line in the first set of illumination patterns will cross any one line of the second illumination patterns a maximum of once.

30. An imaging system according to claim 29, wherein the first set of illumination patterns and the second patterns are generated such that, for each pixel element, the lines in the first set of illumination patterns and the second set of illumination patterns cross the epipolar plane for the pixel element a maximum of once.

31. An imaging system according to any one of the preceding claims, wherein the illumination patterns are generated by multiplying a first subset of illumination patterns by a second subset of illumination patterns; wherein:the image processor is configured to identify the one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass by separately decoding, from the captured images, the information contained in the first subset of illumination patterns and the information contained in the second subset of illumination patterns.

32. An imaging system according to claim 31, wherein a majority of points within each pattern in the first subset of illumination patterns have zero intensity.

33. An imaging system according to claim 31 or 32, wherein the first subset of illumination patterns comprises a series of lines extending in a first direction.

34. An imaging system according to claim 33, wherein the second subset of illumination patterns comprises a series of patterns in which intensity is modulated in the direction different from the first direction.

35. An imaging system according to claim 34, wherein each pattern in the second subset of illumination patterns is phase shifted to a different extent.

36. An imaging system according to claim 34 or 35, wherein the intensity modulation in the second subset of illumination patterns is a sinusoidal modulation.

37. An imaging system according to any one of claims 31 to 36, wherein the light source is configured to generate respective groups of illumination patterns by multiplying each respective illumination pattern in the first subset of illumination patterns by each pattern in the second subset of illumination patterns.

38. An imaging system according to claim 37, wherein identifying, for a first pixel element of the detector one or more regions of the array through which respective light rays(s) incident on the pixel element pass during the sequence of time steps comprises:determining, for each group of illumination patterns, a first component of the signal seen in the pixel element when illuminated by that group of illumination patterns, the first component of the signal comprising information encoded in the first subset of illumination patterns;determining for each group of illumination patterns, a second component of the signal seen in the pixel element when illuminated by that group of illumination patterns, the second component of the signal comprising information encoded in the second subset of illumination patterns;identifying, based on the first component of the signal, a group of regions in the illumination array through which a respective one of the light rays passed; andidentifying, based on the second component of the signal, one of the regions in the group of regions through which the light ray passed.

39. An imaging system according to any one of the preceding claims, wherein the system is configured to generate a 3D image of the object, using knowledge of the one or more regions of the array of illumination points through which respective light rays(s) incident on the pixel element pass;wherein when generating the 3D image of the object, the system is configured to discount signals incident on the pixel element and identified as coming from light rays that do not satisfy the one or more geometric constraints.

40. An imaging system according to any one of the preceding claims, wherein the light source comprises a projector having an array of projector elements, each illuminationpattern being generated by activating one or more of the projector elements.

41. A robotic device configured to manipulate a physical object, the robotic device being configured to map a 3D surface of the object by using a system according to any 5 one of the preceding claims to image the object.

42. Use of an imaging system according to any one of claims 1 to 40 for imaging one or more objects.55

Images