Time-of-flight sensor system

Abstract

A time-of-flight sensor system (100) comprising an illumination source (113) configured to provide illumination (103) to illuminate a subject (101) for which a time-of-flight is to be measured, an optical system having an actuator (111) comprising a support structure and a movable element (109) movable relative to the support structure, and a sensor (107) having a sensor surface. The sensor is configured to sense light from the illumination source (113) scattered by the subject (101) and provide depth data from the sensed light. The actuator (111) is configured to cause the illumination (103) to move over at least a portion of the subject (101) to generate an output frame by moving the movable element. The time-of-flight sensor system (100) further comprises a position sensor configured to determine position data of the movable element (109). The time-of-flight sensor system (100) further comprises a processor (115) configured to associate the position data with the depth data to provide a depth map.

Description

[0001] field The present invention relates to a time-of-flight sensor system and a method for sensing light scattered by a subject in a time-of-flight sensor system.

[0002] background Time-of-flight sensor systems use time of flight to resolve the distance between the sensor and the subject for each point in an image. In a direct time-of-flight system, time of flight is measured, for example, by measuring the round-trip time of an artificial light signal or pulse traveling to and from the subject and subsequently reflecting off the subject. Therefore, the distance to the subject is the speed of light (3 x 10⁻⁶). 8 ms -1 The time of flight is half the product of the measured round-trip flight time to the subject. Alternatively, in an indirect time-of-flight system, the flight time can be based on measuring the phase difference between the transmitted and received signals.

[0003] Invisible light wavelengths can be used in time-of-flight camera systems to avoid interfering with the subject being imaged (which can also be captured using a visible light camera). Near-infrared (NIR) bands (wavelengths from 750 nm to 1.4 μm) are typically chosen due to the availability of small (portable) lasers with good resolution potential.

[0004] Time-of-flight three-dimensional (3D) sensors can use light provided by artificial light sources. As is well known, in some 3D sensing systems, illumination is scanned across the subject to generate output frames.

[0005] Overview The inventors of this invention have realized that it is advantageous to know the position of a movable element of an actuator with high accuracy, wherein the movement of the movable element is caused by illumination provided on the subject by a light source. As explained in more detail below, knowing the position of said movable element with high accuracy allows for more accurate processing of data captured by the time-of-flight sensor system, and thus advantageously provides a better depth map output.

[0006] Furthermore, the inventors of this invention have recognized the benefits of reducing the potential latency introduced in arrangements that require signal transmission to provide information about the position of movable elements. These arrangements are described in more detail below.

[0007] This invention is defined by the independent claims, which should now be referenced. Optional features are set forth in the dependent claims.

[0008] According to one aspect of the invention, a time-of-flight sensor system is provided, comprising: an illumination source configured to provide illumination for illuminating a subject to which the time of flight is to be measured; an optical system having an actuator including a support structure and a movable element movable relative to the support structure; and a sensor having a sensor surface and configured to sense light scattered by the subject from the illumination source and to provide depth data based on the sensed light, wherein the actuator is configured to move the illumination on at least a portion of the subject by moving the movable element to generate an output frame, and the time-of-flight sensor system further includes a position sensor configured to determine position data of the movable element. The position data may include the position of the movable element at a given time. The combination of the position sensors enables the time-of-flight sensor system to have accurate position information of the movable element. The depth data may include one or more depth data points generated from light scattered by different points of the subject as the illumination moves on at least a portion of the subject. The depth data points may correspond to instances of illumination emission and may be associated with position data points. In different instances, location data can be associated with depth data at two or more locations on the subject.

[0009] In doing so, accurate location data can be correlated with depth data provided by the sensor. This advantageously provides for the generation of an accurate depth map.

[0010] The time-of-flight sensor system can be any compatible device or equipment or can be installed in any compatible device or equipment, including smartphones, mobile computing devices, laptops, tablet computing devices, security systems, gaming systems, augmented reality systems, augmented reality devices, wearable devices, drones, aircraft, spacecraft, vehicles, autonomous vehicles, robotic devices, consumer electronics, home appliances, and home automation devices.

[0011] Illumination can be provided or emitted by any suitable light source. For example, the illumination source can be a non-visible light source or a near-infrared light source. The light source may include at least one laser, a laser array (e.g., a vertical-cavity surface-emitting laser (VCSEL) array), or may include at least one light-emitting diode (LED).

[0012] The actuator can be configured to move the movable element at a relatively high speed. For example, at a speed of 0.08 m / s, or 1 μm in 12.5 μs. At these scales, the latency and transmission time of signal transmission are significant. For example, the latency and transmission time associated with data transfer from the position sensor to, for example, a processor can be considered. This is especially true if at least 9 bits are required to encode the position data to the desired precision. For example, in a typical application requiring 1 µm precision over a length of 400 µm, there are at least 400 unique values. Therefore, a 9-bit system (2 9 =512) to encode with sufficient precision.

[0013] The latency in such signals can originate from various sources. Position sensors may include signal converters (e.g., analog-to-digital converters), which can have a certain amount of latency. For example, a signal latency of 10µs is typical for such converters.

[0014] Signal transmission itself can include inherent latency. Communication interfaces such as the Serial Peripheral Interface (SPI) can have relatively low overhead and are also capable of operating at relatively high clock rates. However, clock rates above 10 MHz cause some problems. Even at this speed, there is at least a latency of 1 µs. In practice, the latency is typically 10 µs. Internal integrated circuits (I2C) may have higher overhead but are generally not supported above 3.4 MHz. This means a latency of at least 8 µs.

[0015] Position sensors can transmit determined position data to a processor. This can introduce latency from the aforementioned sources. Therefore, it is desirable to reduce the impact of this latency in order to provide accurate position information for movable elements. Various methods for reducing these impacts are provided.

[0016] For example, the hardware components of a time-of-flight sensor system can be synchronized. The position sensor can be synchronized with the illumination source so that position data is determined when the illumination source provides illumination. This means that, as long as the synchronization is consistent, position data can be determined at the start of illumination launch, at any time during illumination launch, or at the end of illumination launch. The position sensor may include a signal transducer, and the signal transducer can be synchronized with the illumination source so that position data is determined when the illumination source provides illumination. By synchronizing the position sensor and the illumination source, the impact of latency in signal transmission or processing is greatly reduced because the system (e.g., the processor) knows the position data of the movable element at a specific instance of illumination launch. Therefore, a direct correlation can be made between depth data received from a specific instance of illumination launch and the position data of the movable element at that time of illumination launch. The depth data and position data for each instance can thus be accurately correlated to produce an accurate depth map, and the latency effect is significantly reduced.

[0017] Location data may not be directly obtained from location sensors. In other words, location data may not be correlated with depth. Data is acquired in real time. Conversely, location data can be obtained by: 1) obtaining stored data from the memory module. Location data, 2) Obtain Location data corresponding to one or more move commands. 3) Obtain location data through prediction, or 4) Position data is obtained through interpolation, as explained in further detail below. Therefore, when When a processor correlates location data with depth data to provide a depth map, it can perform additional processing steps beyond mapping location data to depth data.

[0018] The time-of-flight system may also include a memory module. Position data can be stored in the memory module. The processor can be configured to associate depth data with the position data stored in the memory to provide a depth map. In addition to the position data, each position data sample may be accompanied by a timestamp / corresponding time data (e.g., in the form of metadata). This data can be used in interpolation / prediction methods as described.

[0019] Position data can be one-dimensional position data indicating the linear displacement of a movable element from a reference point (e.g., zero position or default position). Position data can preferably refer to two-dimensional coordinates (e.g., coordinates on the XY plane) representing the position of the movable element within the region of interest. Position data can also refer to three-dimensional coordinates representing the position of the movable element within the volume of interest.

[0020] Another way to reduce the impact of latency is to predict the position data of the movable element. For example, the optical system may also include a controller configured to control the movement of an actuator by providing one or more movement commands. In other words, the controller can instruct the actuator to move the movable element. These one or more commands can control the actuator to move the movable element in a first direction or multiple subsequent directions, where the directions can be the same or different. A position sensor can be configured to determine position data corresponding to each of the one or more movement commands. When the controller provides a movement command to the actuator, the position sensor can be configured to determine the position data of the movable element after movement according to the controller's instructions. The position sensor can determine position data for each of the one or more movement commands. The determined position data can be stored in memory. The correspondence between the one or more movement commands can also be stored in memory. Therefore, one or more movement commands and their corresponding position data of the movable element can be stored in memory. By doing so, if a movement command provided by the controller has been previously issued, the memory will contain the position data of the movement command and its corresponding movable element. A time-of-flight system can, for example, use a processor to predict the position data of a movable element after a movement command is issued, without requiring the position sensor to determine the position data of the movable element again. The processor can be configured to correlate depth data with stored position data corresponding to one or more movement commands to provide a depth map. This avoids the impact of latency in transmitting position data from the position sensor to the processor.

[0021] One or more movement commands provided by the controller can be synchronized with illumination provided by a lighting source, such that position data corresponds to each of the movement commands. Therefore, the position data of the movable element can be known from memory, and its correspondence with depth data from sensors can be obtained through synchronization. Thus, accurate correlation and depth maps can be provided.

[0022] The processor can be configured to calculate the moving speed of a movable element based on position data stored in memory. For example, two or more position data points can be stored in memory. The processor can use two or more position data points, based on first and subsequent position data points or multiple position data points, to calculate the moving speed of the movable element. The processor can also be configured to extrapolate future position data based on the stored position data. The processor can also calculate the moving direction of the movable element based on the stored position data points. Based on one or more of these, the processor can be configured to extrapolate or calculate the future position of the movable element. The depth data and the predicted position data can then be correlated to produce an accurate depth map. Most advantageously, the position data calculated or predicted by the processor will correspond to an instance of illumination emission, so that the depth data can be appropriately correlated with the position data. The corresponding point of illumination emission can be at the beginning, during, or end of the emission, as described above.

[0023] A position sensor can be configured to periodically determine the position data of a movable element. The period can be a regular interval. The interval can be set manually by the user or calibrated at the factory. The periodically determined position data can be stored in memory. A processor can be configured to interpolate intermediate position data based on the periodically determined position data. That is, the processor can predict the position data at a point between one or more periodically determined position data points.

[0024] The position sensor may include an actuator configured to drive movement of a movable element. The actuator may be configured to move illumination over at least a portion of the subject in a scanning mode. The scanning mode may include moving illumination over at least a portion of the subject along a first direction. For example, the scanning mode may include moving illumination over at least a portion of the subject along a single direction (e.g., from one side of the subject to the other) to substantially cover or completely cover the subject. The scanning mode may also include moving illumination over at least a portion of the subject along a second direction. In a plane, the first direction may be perpendicular to the second direction or at an angle to the second direction. That is, the first direction may be at a non-zero angle to the second direction. The scanning mode may be a raster scan mode. The scanning mode may be boustrophedonic. Increasing the number of points in the scanning mode can result in more uniform illumination of the subject or field of view, which can allow for increased resolution of the output frame. However, the more points in the scanning mode, the more frames need to be captured and combined to generate the output frame. The more frames there are, the more time is required to combine them. Therefore, a scanning mode suitable for the application can be selected.

[0025] The lighting can have any suitable form or shape. For example, lighting can include: a beam with a beam projection configured to be tessellate; a beam with a circular or polygonal beam projection; or light stripes. It will be understood that these are merely example types of lighting and are not limiting. By tessellate, it means that the beam shape is configured to substantially cover the subject without beam shape overlap when the lighting is moved over at least a portion of the subject. This may be without gaps between projections, or it may be with gaps between projections. The lighting source can be configured to provide illumination in discrete flashes.

[0026] The actuator may include a position sensor. The actuator may also include a shape memory alloy (SMA) component configured to drive movement of a movable element during contraction. The SMA component may be an SMA actuator wire.

[0027] SMA (Stone Marbled Wire) is a material that changes shape with temperature as it transforms from a martensitic to an austenitic phase. When an SMA wire is heated, it shortens, and when it cools, it becomes more elastic and can be stretched by applying force. In actuators, SMA wires are used because they allow for a rapid response when heated (typically by applying current and dissipating power through the wire's resistance). When cooled, SMA returns to its longer state by utilizing its elasticity and applying a counterforce (which can be provided by an elastic device or another SMA wire).

[0028] The actuator may include at least two SMA components. For example, the actuator may include four or eight SMA components. The SMA components may be arranged in such a way that they allow the movable element to move relative to the support structure in two orthogonal directions (e.g., along the x and y axes) perpendicular to a nominal principal axis (e.g., the z-axis) extending through the movable element. At least two SMA components may be connected between the movable element and the support structure and arranged to move the movable element during retraction. In an arrangement including four SMA components, the four SMA components may be arranged in a ring around a nominal principal axis (referred to herein as the optical axis). The four SMA components may consist of a first pair of SMA components arranged on opposite sides of the optical axis and a second pair of SMA components arranged on opposite sides of the optical axis. The first pair of SMA components may be selectively actuated to move the movable element relative to the support structure in a first direction, while the second pair of SMA components may be selectively actuated to move the movable element relative to the support structure in a second direction transverse to the first direction. Movement in directions other than parallel to the SMA components can be driven by a combination of actuations of these pairs of SMA components to provide a linear combination of movements of the movable element in the lateral direction. Another way to observe this movement is that the simultaneous contraction of any pair of adjacent SMA components in the ring will drive the movable element to move in the direction that bisects the two SMA components (i.e., produce diagonal movement).

[0029] Therefore, the SMA component can be selectively driven to move the movable element relative to the support structure to any position within its range of motion in two orthogonal directions perpendicular to the optical axis. The size of the range of motion depends on the geometry and contraction range of the SMA component within its normal operating parameter range.

[0030] To facilitate movement of the movable element relative to the supporting structure, a bearing can be provided between the supporting structure and the movable element. Alternatively, a sliding bearing can be provided on the supporting structure to enable movement of the movable element.

[0031] The movable element may include a lens element. A support structure may support the image sensor. The lens element may be arranged to focus illumination onto the subject, whereby light scattered by the subject can be captured by the image sensor. The image sensor may capture an image and may be of any suitable type, such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) device.

[0032] The SMA component may include a position sensor. In this embodiment, the position sensor may be configured to determine position data by measuring the resistance of the SMA component. A sensing resistor may be connected in series with the SMA component. Measurement circuitry may be provided to perform a measurement indicating at least the potential difference across the SMA component. The time-of-flight sensor system may also include a measurement switch configured to be connected between the SMA component and the sensing resistor. The measurement switch may be configured to be connected to measurement circuitry to enable the measurement circuitry to perform the measurement, or connected to circuitry bypassing the sensing resistor.

[0033] Such a device improves the efficiency of measuring the resistance of SMA components while maintaining sensitivity and accuracy. This thus provides accurate measurement of the position of movable elements. The measurement switch makes it possible to bypass the sensing resistor when the SMA component is actuated (i.e., heated) and switch the sensing resistor back into the circuit when measurement is required. It may only be necessary to measure resistance intermittently. Between measurements, the sensing resistor can be bypassed. This reduces the power required to heat the SMA component, thereby improving efficiency.

[0034] Alternatively, the measurement circuit can be configured to perform a measurement of the potential difference across at least the SMA component, indicating a reference potential. The reference potential can be connected to a connection potential located on the opposite side of the sensing resistor and the SMA component, such that the reference potential is equal to the connection potential.

[0035] Furthermore, it is possible to use only very short measurement pulses (i.e., inputting only a small amount of power to the SMA component) when performing measurements. For example, the measurement pulse could be shorter than the pulse used to heat the SMA component to control its length. By using short measurement pulses, measurements can be performed without undesirable heating of the SMA component.

[0036] The contraction of the SMA component causes a movement of the illumination through the movement of the movable element. The resistance of the SMA component at a given time thus provides an indication of the position data of the movable element at that time, and therefore the position data can be used in conjunction with depth data.

[0037] Position sensors may include magnetic sensors, such as Hall effect sensors or magnetic tunnel junctions.

[0038] The actuator can instead be a voice coil motor. In this example, the voice coil motor can be separate from the position sensor. That is, the voice coil motor can be a different component from the position sensor.

[0039] The lighting source can be supported on a movable component.

[0040] According to another aspect of the invention, a method for sensing light scattered from a subject in a time-of-flight sensor system is also provided, the method comprising: illuminating the subject to which the time of flight is to be measured by an illumination source; sensing the light scattered from the illumination source by the subject by a sensor having a sensor surface to provide depth data based on the sensed light; moving a movable element relative to a support structure by an actuator to move the illumination over at least a portion of the subject to generate an output frame; and determining position data of the movable element by a position sensor.

[0041] According to another aspect of the invention, a non-transitory computer-readable medium is provided, comprising instructions for performing the methods described above. The non-transitory computer-readable medium may be, for example, a solid-state memory, a microprocessor, a CD or DVD-ROM, a programmable memory such as non-volatile memory (such as flash memory) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Brief description of the attached diagram Some claimed embodiments of the invention will now be described by way of example only with reference to the accompanying drawings, in which: Figure 1 This is a schematic diagram of a time-of-flight sensor system embodying one aspect of the present invention; Figure 2 This is a schematic diagram of the illumination of a subject using a time-of-flight sensor system embodying one aspect of the present invention; Figure 3 This is a diagram illustrating the operation of a time-of-flight sensor system that embodies one aspect of the present invention; Figure 4 This is a diagram illustrating the operation of a time-of-flight sensor system that embodies one aspect of the present invention; Figure 5 This is a diagram illustrating the operation of a time-of-flight sensor system embodying one aspect of the invention; and Figure 6 This is a schematic diagram of an actuator for a time-of-flight sensor system that embodies one aspect of the present invention.

[0043] Similar features are represented by similar reference numbers.

[0044] Detailed description Now refer to Figures 1 to 6 To describe an example time-of-flight sensor system.

[0045] Figure 1A time-of-flight sensor system 100 is shown. System 100 includes an illumination source, in this example, a vertical-cavity surface-emitting laser (VCSEL) 113 for illuminating a subject 101 to which the time of flight is to be measured. System 100 also includes an optical system comprising an actuator 111. In this example, actuator 111 comprises four SMA components. This actuator is described in more detail below and by way of... Figure 6 As shown. Actuator 111 is configured to move illumination 103 provided by VCSEL 113 by moving movable element 109 relative to a support structure (not shown). More specifically, movable element 109 can move in a direction orthogonal to the illumination optical axis. Movable element 109 includes a lens element. System 100 also includes sensor 107 having a sensor surface and configured to sense light scattered from VCSEL 113 by subject 101 and to provide depth data based on the sensed light. Notably, system 100 also includes a position sensor. In this example, actuator 111 includes a position sensor. However, it will be understood that the position sensor may be a component separate from the actuator. In this example, the position sensor determines the position data of the movable element by measuring the resistance of at least one SMA component. The sensing resistor is connected in series with the SMA component and provides measurement circuitry to perform a measurement indicating the potential difference across at least one SMA component. System 100 also includes processor 115 connected to VCSEL 113, sensor 107 and position sensor.

[0046] Processor 115 also includes a controller. During use, VCSEL 113 provides illumination 103 in stripe form and in discrete flashes. The controller then provides a movement command to actuator 111 to move movable element 109, causing illumination 103 to move over at least a portion of subject 101. When at least one SMA component is heated, the movable element is moved by the actuator through the contraction of at least one SMA component. While movable element 109 moves illumination 103 over subject 101, VCSEL continues to provide flashing stripe illumination 103. Notably, in this example, a position sensor is synchronized with the VCSEL. In particular, the position sensor includes an analog-to-digital converter, and this converter is synchronized with the VCSEL. The position sensor provides position data of the movable element by measuring the resistance of at least one SMA component. Due to synchronization, this measurement is synchronized to be performed when the VCSEL emits illumination 103. In particular, the position sensor is synchronized to perform the measurement at the start of illumination. However, it will be understood that the measurement can be performed at any time during illumination or at the end of illumination, provided consistent synchronization exists. Light scattered by the subject 101 is received by sensor 107, which provides depth data to processor 115 based on the received light. A position sensor provides position data to the processor. As illumination 103 scans the subject 101, multiple data points of both position and depth data are acquired, and all of these data points are provided to the processor. This data transmission can typically introduce latency from the various sources mentioned above. However, this latency effect is mitigated due to the synchronization of the position sensor with the VCSEL, as the processor knows which position data points correspond to which depth data values. Therefore, the position and depth data are correlated by the processor, and an accurate depth map 117 is generated by avoiding the effects of latency.

[0047] Will understand, although Figure 1 The arrangement shown provides a system configured to reduce latency for data transmission in the system, but other arrangements can also reduce latency in other ways as described above and below.

[0048] Figure 1 The example time-of-flight system 100 shown is implemented in a smartphone device, but it will be understood that a time-of-flight system can be implemented in any suitable system.

[0049] Figure 2The scanning pattern of illumination on subject 207 is illustrated. In this example, illumination is provided by a VCSEL (not shown) in the form of flashing stripes. The illumination moves along a single axis from right to left on subject 207 in a first direction. Illumination 201 with a first flash is provided first. The illumination scans on the subject as the movable element is moved by an actuator. Subsequently, a second flash 203 and a third flash 205 are provided. With each flash, depth data is acquired by a receiving sensor. In addition, position data corresponding to the position of the movable element is acquired to correspond to each flash of the scanning pattern. Thus, multiple data points for each depth and position data can be acquired to provide correlation and generate a depth map.

[0050] It will be understood that although the flashing stripes are provided and scanned in a single direction in this example, the shape of the illumination can be any suitable shape as described above, and it can also scan in two or more directions.

[0051] Figure 3 It shows the use of, for example Figure 1 The diagram shown is of a time-of-flight sensor system. Figure 3 The synchronization of the position sensor and VCSEL is emphasized. Illumination intensities of 201, 203, and 205 correspond to... Figure 2 Flash illumination is scanned onto subject 207. A synchronization signal is provided at the point where illumination is emitted from the VCSEL. The synchronization signal enables the position sensor to determine the positions 301, 303, and 305 of the movable element at each instance of illumination emission. By providing this synchronization, a correspondence between the position data obtained from the sensor and the obtained depth data can be obtained for each flash illumination. This thus mitigates the impact of signal transmission latency in the system. The position of the movable element can therefore be known with high accuracy, which allows for the generation of accurate depth maps.

[0052] Figure 4 It shows relief Figure 1 A graph showing alternative ways to wait time in a time-of-flight sensor system. Figure 4The time-of-flight sensor system shown is used to predict the position data of a movable element using extrapolation. The position sensor periodically determines the position data 401 of the movable element. In this example, the intervals between each determination are constant, but it will be understood that they need not be constant. The position data 401 is stored in a memory module connected to a processor. Based on the position data 401, the processor extrapolates to predict future position data points 403 of the movable element. In this example, the processor also calculates the moving speed of the movable element based on the position data points 401, but it will be understood that this is not necessary for extrapolating future position data points. Therefore, the position data of the movable element can be predicted, making it possible to determine the position data point corresponding to each emission instance of light 201, 203, 205. By predicting the position data points 403, the amount of measurement required from the position sensor is reduced. Furthermore, since the position data points can be obtained through prediction / calculation, it is not necessary to determine the position data points at specific times. Therefore, this reduces the amount of waiting time introduced into the system and thus improves the accuracy of the position data, and ultimately improves the accuracy of the depth map generated by associating depth data with the position data.

[0053] Figure 5 Another use of the time-of-flight system is illustrated, configured to reduce the impact of latency in another way. In this example, position data of a movable element can be interpolated. In this example, a position sensor periodically determines the position data of the movable element. Again, although position data 503 is determined at regular intervals in this example, it will be understood that it is not necessary to determine position data 503 at regular intervals. Position data 503 is stored in a memory module connected to a processor. Based on position data point 503, the processor interpolates to determine position data 501 of the movable element at a given time between one or more position data points 503. The system is thus able to determine the position data of the movable element 501 at any point in time without requiring further measurements from the position sensor. Once the system knows when light is emitted, the processor can interpolate to find the position of the movable element at the illumination emission instance, and this position data point can be determined to correspond to each instance of emission of light 201, 203, 205. This again mitigates the impact of latency transmitted from the position sensor and thus improves the accuracy of the position data.

[0054] Figure 6 As shown in Figure 1 The SMA actuator arrangement implemented in the time-of-flight sensor system shown. The actuator arrangement 10 includes a total of four SMA actuator wires 11, 12, 13, and 14, which are connected between the support block 16, which forms part of the support structure and is mounted on the base, and the movable element 15.

[0055] Each of the SMA actuator wires 11 to 14 is held taut, thereby applying a force between the movable element 15 and the support block 16 in a direction perpendicular to the nominal main axis (here referred to as the optical axis). In operation, the SMA actuator wires 11 to 14 move the movable element 15 relative to the support block 16 in two orthogonal directions perpendicular to the optical axis.

[0056] SMA actuator wires 11 to 14 are connected at one end to the movable element 15 via corresponding crimping members 17, and at the other end to the support block via crimping members 18. The crimping members 17 and 18 crimp the wires to mechanically retain them (optionally reinforced by the use of adhesive). The crimping members 17 and 18 also provide electrical connections to the SMA actuator wires 11 to 14. However, it will be understood that any suitable means for connecting the SMA actuator wires 11 to 14 can be used alternatively.

[0057] Four SMA conductors 11 to 14 are arranged in a loop around the optical axis. The four SMA conductors consist of a first pair of SMA conductors 11 and 13 arranged on opposite sides of the optical axis and a second pair of SMA conductors 12 and 14 arranged on opposite sides of the optical axis. The first pair of SMA conductors 11 and 13 can be selectively driven to move the movable element 15 relative to the support structure in a first direction, while the second pair of SMA conductors 12 and 14 can be selectively driven to move the movable element 15 relative to the support structure in a second direction transverse to the first direction. Movement in directions other than parallel to the SMA conductors 11 to 14 is driven by combined actuation of these pairs of SMA conductors to provide a linear combination of lateral movements of the movable element. Alternatively, this movement can be observed when the simultaneous contraction of any pair of adjacent SMA conductors in the loop will drive the movable element to move in the direction that bisects the two SMA conductors (i.e., producing diagonal movement).

[0058] Therefore, SMA conductors 11 to 14 can be selectively driven to move the movable element 15 to any position within its range of motion relative to the support structure in two orthogonal directions perpendicular to the optical axis. The size of the range of motion depends on the geometry and contraction range of the SMA conductors within their normal operating parameter range.

[0059] Embodiments of the present invention have been described. It will be understood that variations and modifications can be made to the described embodiments within the scope of the present invention. For example, Figures 3 to 5 The graphs shown are all linear. It will be understood that movable elements can move at different speeds and / or in more than one direction. For example, with... Figures 3 to 5 The charts shown may correspond to sine curves.

[0060] Furthermore, a time-of-flight sensor system can reduce latency in another way for the aforementioned arrangement. The controller can provide movement commands to the actuator to move a movable element, causing the illumination to move over the subject. When the controller has issued a movement command to the actuator, the position sensor determines the position of the movable element. That is, the position sensor determines the position of the movable element corresponding to the movement command issued by the controller. Each time a movement command is issued, the position sensor determines the resulting position of the movable element. The obtained position data is stored in a memory module. If the controller repeats the movement command, the memory module will contain the position data corresponding to the movement command. Therefore, the processor can predict position data by accessing the position data stored in the memory module without requiring the position sensor to re-determine the position data of the movable element. When position data can be predicted, this eliminates the need for a position sensor, thereby reducing the latency introduced by transmitting position data from the position sensor and thus improving the accuracy of the position data. This ultimately improves the accuracy of the depth map.

Claims

1. A time-of-flight sensor system, comprising: A light source configured to provide illumination to illuminate the subject, the time of flight of which will be measured; An optical system having an actuator, the actuator comprising a support structure and a movable element movable relative to the support structure; A sensor having a sensor surface and configured to sense light scattered by the subject from the illumination source, and to provide depth data based on the sensed light, wherein, The actuator is configured to move the illumination over at least a portion of the subject by moving the movable element to generate an output frame, and the time-of-flight sensor system further includes: a position sensor configured to determine position data of the movable element; and a processor configured to correlate the position data with the depth data to provide a depth map. The actuator includes a shape memory alloy (SMA) component, wherein the SMA component is configured to drive movement of the movable element during contraction; and The position sensor and the lighting source are synchronized so that the position data is determined when the lighting source provides illumination.

2. The time-of-flight sensor system according to claim 1, wherein, The processor is configured to associate the location data with the depth data at two or more locations on the subject in different instances.

3. The time-of-flight sensor system according to claim 1 or claim 2, wherein, The position sensor includes a signal converter that is synchronized with the lighting source, such that the position data is determined when the lighting source provides illumination.

4. The time-of-flight sensor system according to claim 1, wherein, The location data cannot be obtained directly from the location sensor.

5. The time-of-flight sensor system according to claim 1 or claim 4 further includes a memory module, wherein, The location data is stored in a memory module.

6. The time-of-flight sensor system according to claim 5, wherein, The processor is configured to associate the depth data with location data stored in the memory module to provide the depth map.

7. The time-of-flight sensor system according to claim 6, wherein, The optical system also includes a controller configured to control the movement of the actuator by providing one or more movement commands.

8. The time-of-flight sensor system according to claim 7, wherein, The position sensor is configured to determine position data corresponding to each of the one or more movement commands.

9. The time-of-flight sensor system according to claim 8, wherein, The one or more movement commands provided by the controller are synchronized with the lighting provided by the lighting source, such that the position data corresponds to each of the one or more movement commands.

10. The time-of-flight sensor system according to claim 8 or claim 9, wherein, The processor is configured to associate depth data with location data corresponding to the one or more movement commands to provide the depth map.

11. The time-of-flight sensor system according to claim 5, wherein, The processor is configured to calculate the moving speed of the actuator based on the position data stored in the memory module.

12. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, and 11, wherein, The processor is also configured to extrapolate future location data based on the location data.

13. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, and 11, wherein, The position sensor is configured to periodically determine the position data of the movable element.

14. The time-of-flight sensor system according to claim 13, wherein, The processor is configured to interpolate intermediate position data based on periodically determined position data.

15. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, and 14, wherein, The location data includes the position of the actuator at a given time.

16. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, and 14, wherein, The position sensor includes a driver configured to drive the movement of the actuator.

17. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, and 14, wherein, The actuator is configured to move the illumination over at least a portion of the subject in a scanning mode.

18. The time-of-flight sensor system according to claim 17, wherein, The scanning mode includes moving the illumination along a first direction over at least a portion of the subject.

19. The time-of-flight sensor system according to claim 18, wherein, The scanning mode also includes moving the illumination along a second direction over at least a portion of the subject.

20. The time-of-flight sensor system according to claim 19, wherein, The first direction is perpendicular to the second direction or at an angle to the second direction in a plane.

21. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, 14, and 18-20, wherein, The illumination includes: a beam with a beam projection configured as a grid; a beam with a circular or polygonal beam projection; or light stripes.

22. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, 14, and 18-20, wherein, The light source is configured to provide illumination in discrete flashes.

23. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, 14, and 18-20, wherein, The actuator includes the position sensor.

24. The time-of-flight sensor system according to claim 1, wherein, The SMA component includes the position sensor.

25. The time-of-flight sensor system according to claim 24, wherein, The position sensor is configured to determine the position data by measuring the resistance of the SMA component.

26. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, 14, 18-20, and 24-25, wherein, The position sensor includes a magnetic sensor.

27. The time-of-flight sensor system according to any one of claims 1-2, 4, 6-9, 11, 14 and 18-20, wherein, The actuator includes a voice coil motor.

28. The time-of-flight sensor system according to claim 27, wherein, The voice coil motor is separate from the position sensor.

29. A method for sensing light scattered from a subject in a time-of-flight sensor system, the method comprising: The subject is illuminated by a light source, and the flight time will be measured. A sensor with a sensor surface senses light scattered by the subject from the illumination source to provide depth data based on the sensed light; The actuator moves a movable element relative to the support structure to move the illumination over at least a portion of the subject to generate an output frame; and The position sensor determines the position data of the movable element. The processor correlates the location data with the depth data to provide a depth map. The actuator includes a shape memory alloy (SMA) component, wherein the SMA component is configured to drive movement of the movable element during contraction; and The position sensor and the lighting source are synchronized so that the position data is determined when the lighting source provides illumination.

30. A computer program product for instructing a computer to perform the method according to claim 29.

Images