Coordinate measurement device with an indirect time of flight sensor

Abstract

A method includes: projecting, while a device is at a first pose, a pattern at a first angular position; acquiring, using a iToF device, a first frame of the pattern; projecting, while the device is at the first pose, the pattern at a second angular position; acquiring, using the iToF device, a second frame of the pattern; and computing first measurements to points in the environment. The method further includes projecting, while the device is at a second pose, the pattern at a third angular position; acquiring, using the iToF device, a third frame of the pattern; projecting, while the device is at the second pose, the pattern at a fourth angular position; acquiring, using the iToF device, a fourth frame of the pattern; and computing second measurements to points in the environment based at least in part on at least one of the third frame and the fourth frame.

Description

COORDINATE MEASUREMENT DEVICE WITH AN INDIRECT TIME OFFLIGHT SENSORCROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application is related to the Provisional Patent Application Serial No. 63 / 519103 entitled, “COORDINATE MEASUREMENT DEVICE WITH AN INDIRECT TIME OF FLIGHT SENSOR filed August 11, 2023, the contents of which are incorporated by reference herein.BACKGROUND

[0002] The subject matter disclosed herein relates to a system for measuring three-dimensional (3D) coordinates in an environment, and in particular to a system and method for measuring a pattern of light using a time of flight sensor.

[0003] A traditional time-of-flight (ToF) scanner is a scanner in which the distance to a target point is determined based on the speed of light in air of a beam of light traveling between the scanner and a target point. Traditional ToF scanners are typically used for scanning closed or open spaces such as interior areas of buildings, industrial installations and tunnels. They are used, for example, in industrial applications and accident reconstruction applications. A laser scanner optically scans and measures objects in a volume around the scanner through the acquisition of data points representing object surfaces within the volume. Such data points are obtained by transmitting a beam of light onto the objects and collecting the reflected or scattered light to determine the distance, two-angles (i.e., an azimuth and a zenith angle), and optionally a gray-scale value. This raw scan data is collected, stored and sent to a processor or processors to generate a 3D image representing the scanned area or object. For the case in which the light source within a scanner is a laser, such a scanner is often referred to as a laser scanner. The term laser scanner is often also used for scanners that use light sources that are not lasers, such as light sources using superluminescent diodes for example.

[0004] ToF measuring systems such as those used in laser scanners are typically one of two types: a phased-based ToF scanner or a pulsed ToF scanner. In a typical phase-based ToF scanner, a beam of light is modulated at a plurality of frequencies before being launched to a target. After the modulated beam of light has completed a round trip to and from the target, it is demodulated to determine the returning phase of the each of the plurality of frequencies. A processor within the ToF scanner uses the demodulated frequencies and the speed of light in air to determine a distance from the scanner to the target. In contrast, a pulsed ToF scanner typically emits a short pulse of light and measures the elapsed time between launch of the pulse and return of the pulse after having completed a round trip to the target. A processor within the pulsed ToF scanner determines the distance from the scanner to the target based at based at least in part on the measured elapsed time and the speed of light in air. The ToF scanners used in laser scanners today typically include a single optical detector that measures the signal returned from the target. Such optical detectors typically measure up to frequencies of several hundred MHz or down to pulse widths of a few picoseconds to nanoseconds.

[0005] More recently, ToF methods are being employed in camera sensors having a collection or array of photosensitive elements. Each of the photosensors in the array serves the same function as the single optical detector in a traditional ToF laser scanner, but the photosensors typically are more limited in the speed of their response and their optical bandwidths. On the other hand, arrays of photosensors are relatively inexpensive, thereby offering advantages where the range and accuracy requirements are not as stringent as for traditional laser scanners.

[0006] A device that uses an array of sensors to measure a stream of modulated light is said to be an indirect ToF (or iToF) device, while a device that uses an array of sensors to measure pulsed light is said to be a direct ToF (or dToF) device. If an array of pixels using a ToF is included within a camera having a camera lens, then both distances to the target points are determined based on the signals received by the array of pixels. One difficulty faced by an iToF -based 3D measuring system is determining the number of complete phase cycles that have elapsed before a projected beam of lightreflects off an object and travels to the iToF camera. Another difficulty faced by an iToF-based 3D measuring system is in sorting out the different projected pattern elements received by pixels of the iToF camera.

[0007] While existing systems for measuring distance and angles to an object are suitable for their intended purposes, the need for improvement remains, particularly in providing 3D measurement system having the features described herein.BRIEF DESCRIPTION OF THE DISCLOSURE

[0008] In one embodiment, a method for computing depth measurements with a measuring device using an indirect time of flight (iToF) device is proved. The method includes causing the measuring device to be arranged at a first position and a first orientation of an environment, the measuring device configured to project a pattern in the environment. The method further includes projecting, while the measuring device is at the first position and the first orientation, the pattern at a first angular position and acquiring, using the iToF device, a first frame of the pattern. The method further includes projecting, while the measuring device is at the first position and the first orientation, the pattern at a second angular position and acquiring, using the iToF device, a second frame of the pattern. The method further includes computing first measurements to points in the environment based at least in part on at least one of the first frame and the second frame. The method further includes causing the first orientation of the measuring device to change to a second orientation. The method further includes projecting, while the measuring device is at the first position and the second orientation, the pattern at a third angular position and acquiring, using the iToF device, a third frame of the pattern. The method further includes projecting, while the measuring device is at the first position and the second orientation, the pattern at a fourth angular position and acquiring, using the iToF device, a fourth frame of the pattern. The method further includes computing second measurements to points in the environment based at least in part on at least one of the third frame and the fourth frame.

[0009] According to an embodiment, a measuring device is provided. The measuring device includes a housing and a projector operably coupled to the housing,the projector configured to project a pattern into an environment. The measuring device further includes an indirect time-of-fhght (ToF) camera coupled to the housing, the indirect ToF camera having a lens and a two-dimensional array of pixels, each pixel configured to convert reflected light into an electrical signal. The measuring device further includes one or more processors operably coupled to the housing, the projector and the indirect ToF camera, the one or more processors configured to compute depth measurements of points of the environment by performing operations. The operations include projecting, while the measuring device is at a first position and a first orientation of the environment, the pattern at a first angular position and acquiring, using the iToF camera, a first frame of the pattern. The operations further include projecting, while the measuring device is at the first position and the first orientation, the pattern at a second angular position and acquiring, using the iToF camera, a second frame of the pattern. The operations further include computing first measurements to the points in the environment based at least in part on at least one of the first frame and the second frame. The operations further include projecting, while the measuring device is at the first position and a second orientation, the pattern at a third angular position and acquiring, using the iToF camera, a third frame of the pattern. The operations further include projecting, while the measuring device is at the first position and the second orientation, the pattern at a fourth angular position and acquiring, using the iToF camera, a fourth frame of the pattern. The operations further include computing second measurements to the points in the environment based at least in part on at least one of the third frame and the fourth frame.

[0010] These and other advantages and features will become more apparent from the following description taken in conjunction with the drawings.BRIEF DESCRIPTION OF DRAWINGS

[0011] The subject matter, which is regarded as the disclosure, is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features, and advantages of the disclosure are apparent fromthe following detailed description taken in conjunction with the accompanying drawings in which:

[0012] FIG. l is a block diagram showing elements of a system for measuring three-dimensional coordinates in accordance with an embodiment;

[0013] FIG. 2A is a schematic illustration of one type of projector for use in an indirect time-of-flight (iToF) measurement system according to an embodiment;

[0014] FIG. 2B is a schematic illustration of a collection of spots projected onto a surface according to an embodiment;

[0015] FIG. 3 A is a schematic representation of a system having a projector that projects onto a surface a pattern captured by an iToF distance measuring device according to an embodiment;

[0016] FIG. 3B illustrates a method for using triangulation to eliminate range ambiguity in distance measurements of the system of FIG. 3 A;

[0017] FIG. 3C illustrates a method for using triangulation to reduce correspondence ambiguity in measurement by the system of FIG. 3 A;

[0018] FIG. 3D is a schematic illustration showing a projected spot of light displayed in relation to multiple pixels of an iToF sensor;

[0019] FIG. 4 A is a schematic representation of a system having a projector that projects onto a surface a pattern captured by two iToF distance measuring devices according to an embodiment;

[0020] FIG. 4B illustrates a method for using triangulation to eliminate range ambiguity in distance measurements of the system of FIG. 4 A;

[0021] FIG. 4C illustrates a method for using triangulation to reduce correspondence ambiguity in a measurement by the system of FIG. 4 A;

[0022] FIG. 5A illustrate a triangulation method for determining 3D coordinates of an object surface placed relatively near a 3D measuring device having a single projector and a single TOF camera according to an embodiment;

[0023] FIG. 5B illustrates a triangulation method for determining 3D coordinates of an object surface placed near a 3D measuring device having a single projector and two TOF cameras according to an embodiment;

[0024] FIGs. 6A and 6B are graphical illustrations of a waveform that might be emitted by a projector and received by an iToF sensor, respectively, according to an embodiment oe;

[0025] FIG. 6C is a graphical representation of signals produced by returning light for a first phase condition of zero degrees, and FIG. 6D is a graphical representation of signals produced by returning light for a second phase condition of 90 degrees, according to an embodiment;

[0026] FIG. 7A is a graphical representation of the FOV of a single color camera added to a 3D measuring instrument having two TOF devices according to an embodiment;

[0027] FIGs. 7B, 7C are illustrations of arrangements for two color cameras for placement in an TOF measuring device according to an embodiment;

[0028] FIGs. 8A, 8B, 8C are graphical representations showing the overlap in coverage of two two-dimensional (2D) image sensors when rotating around a vertical axis and stopping four, five, and six times, respectively, in covering 360 degrees according to an embodiment;

[0029] FIG. 9A shows an image captured by a camera without correction for lens aberrations and other camera parameter errors according to an embodiment;

[0030] FIG. 9B shows an image captured by a camera with correction to remove lens aberrations and other camera parameter errors according to an embodiment;

[0031] FIG. 10 is an isometric view of a 3D measuring system according to an embodiment;

[0032] FIG. 11 is a view of a 3D measuring system according to another embodiment;

[0033] FIG. 12 is a block diagram of elements within a system processing unit according to another embodiment;

[0034] FIG. 13 depicts a flow diagram of a method for computing depth measurements with a measuring device using an iToF device and a moving pattern according to one or more embodiments described herein;

[0035] FIGS. 14A-14D depict an environment having a measuring device for computing depth measurements using an iToF device and a moving pattern; and

[0036] FIGS. 15A-15C depict architectures for projecting a moving pattern according to one or more embodiments described herein.

[0037] The detailed description explains embodiments of the disclosure, together with advantages and features, by way of example with reference to the drawings.DETAILED DESCRIPTION OF THE DISCLOSURE

[0038] Embodiments of the present disclosure provide for a low cost system for measuring three-dimensional (3D) coordinates and generating a dense 3D point cloud. Embodiments of the present disclosure provide for a sensor array that allows for measurement of 3D coordinates over an area based at least in part on the speed of light as the sensor array is rotated about an axis. Still further embodiments of the present disclosure provide for emitting a pattern of light onto one or more surfaces in the environment and measuring 3D coordinates of elements of the pattern using an iToF or dToF sensor.

[0039] Embodiments of the present disclosure further provide for a low-cost system that uses triangulation to reduce ambiguities in measured distances and also to reduce ambiguities among imaged points and measured distances. Other embodiments provide for fast and cost-effective 3D scanning measurements.

[0040] Embodiments described herein further provide for accurate depth measurements with a measuring device using an iToF device by implementing a moving projection pattern.

[0041] Referring now to FIG. 1, an embodiment of a system 100 is shown for measuring 3D coordinates on objects, also referred to as measuring 3D coordinates of surfaces in an environment. The system 100 includes a housing 102 configured to rotate about one or more axes. The housing 102 can be a structural member, such as a structural frame, configured to house and / or mount one or more components. One or more rotation mechanisms rotate the housing 102 about a corresponding axis. An example is the rotation mechanism 104 that rotates the system 100 about a vertical axis (e.g. an axis perpendicular to a plane of the floor in the environment), for example. Each rotation mechanism 104 optionally includes an angular transducer 106 configured to read the angle of rotation of the housing 102 about a particular axis. An example of an angular transducer 106 is an angular encoder. In an embodiment, the rotation mechanism 104 is optionally coupled to a support structure such as a tripod (not shown), for example. In an embodiment, the rotation mechanism includes a motor (not shown) that allows for selective rotation of the housing 102. In an embodiment, the housing 102 is selectively rotated in incremental steps (e.g., a predetermined angular rotation) and paused for predetermined amounts of time. In still another embodiment, the housing 102 is continuously rotated at a predetermined speed.

[0042] Within or coupled to the housing 102 is a measurement device 110. In an embodiment, the measurement device 110 includes a projector 112 that emits a patterned light beam onto an object surface. The patterned light beam reflects off the object surface and is received by the ToF camera 114, which measures distances from the measurement device 110 to illuminated points on the object surface, the measureddistance values being partly based on timing signals shared by the ToF camera 114, the clock / timer 116, the projector 112, and the processor 118, the rotation mechanism 104, and the optional angular transducer 106.

[0043] The ToF camera 114 further includes a lens 118 and a ToF sensor 120. It should be appreciated that while a single lens element is illustrated, this is for example purposes and the claims should not be so limited. In other embodiments, the lens 118 is a set of optical components. In some cases, the ToF sensor 120 is used in either an iToF or dToF mode. For example, FRAMOS offers an FSM-IMX570 development kit for Jetson AGX Orin / Xavier. This development kit demonstrates the iToF mode using the Sony IMX570PLR-C ToF sensor. The demonstration kit includes drives and a demonstration application for NVIDIA® Jetson AGX Orin™ and AGX Xavier™. Sony Semiconductor Solutions Corporation is located in the Kanagawa Prefecture of Japan. FRAMOS has its headquarters in Taufkirchen, Germany.

[0044] Referring now to FIG. 2 A, an embodiment is shown of projector 200, schematically represented in a cross-sectional view. In an embodiment, the projector 200 is a projector comprising a diffractive optical element (DOE) assembly. The projector 200 includes a vertical cavity surface-emitting laser (VCSEL) array 202. Light from laser emitter elements 204 of the VCSEL array 202 pass through a lens 208, which in an embodiment is an aspherical lens. The light continues on to a mask element 212 having a mask pattern 214 and passes onto a detector plane 220. In an embodiment, a front view of the detector plane 220 is shown in FIG. 2B. The resulting pattern 230 produced by the action of the DOE 212 comprises a collection of spots 232 that form a larger pattern 236. In other embodiments, other patterns are produced at the detector plane 220 according to the mask pattern 214 without deviating from the teachings herein. The light source is chosen to be a VCSEL array 202 because of the relatively high spatial coherence of every single VSCEL and relatively low temporal coherence of the VCSEL array. Further, the use of a VCSEL array provides further advantages in providing for high optical power at relatively low cost. These properties of the VCSEL array enable the bundles of light emitted by the VCSEL arrays to be relatively narrow while still reducing or minimizing the effects of laser speckle.

[0045] In other embodiments, other DOE-type projectors are used. For example, Digigram (headquarters in Saint Martin, France), Himax (headquarters in Tainan City, Taiwan) and AMS-Osram (headquarters in Premstatten, Pennsylvania) offer VCSEL based projectors that are commercially available. In other embodiments, projectors are made using metalenses, which are flat lenses that use metasurfaces to focus light. Metalenses comprise collections of artificial antennae that manipulate the amplitude phase and polarization of the incident light. Examples of companies that make metalenses are Coherent (headquarters in Saxonburg, Pennsylvania) and Metalenz (headquarters in Boston, Massachusetts). Popular VCSEL arrays for use in laser projectors are manufactured by Lumentum Operations (headquarters in San Jose, California).

[0046] FIG. 3A shows a device 300 that measures distance and angles to an object having a surface 342 shown in FIG. 3B and FIG. 3C. The device 300 includes a projector 302, a clock / timer 320, a processor 330, and a TOF camera 310. In some embodiments, the TOF camera 310 includes the clock / timer 320 having timing signals sent to other components in the device 300. The TOF camera 310 includes a lens and a TOF sensor, which is used in an iTOF or dTOF mode. In an embodiment, the projector 302 projects a pattern 312 of light, enabling the light to be concentrated in spots 314 or other patterns having higher power per unit area than would be possible were the light projected homogeneously over a wide angular region. This higher power per unit area enables more accurate measurements to be made at a given limited measurement rate.

[0047] In an embodiment, the projector 302 is spaced apart from the camera 310 by a baseline distance. Triangulation calculations are carried out by the processor 330, which includes one or more computers external to the device 300 as well as processing elements internal to the TOF camera 310. A geometries representative of the system 300 is shown in FIG. 3B and FIG. 3C. In an embodiment, a compensation measurement is made prior to measuring in the field. Such compensation measurements are carried out in a factory during device assembly or in the field at a preliminary stage using compensation aids such as calibrated dot plates. In other embodiments, the compensation measurements are performed in the field (e.g. outside of a controlledmanufacturing setting) using a standard scanning process or a predetermined- altemative scanning process without the use of compensation aids.

[0048] Referring now to FIG. 3C, in an embodiment, compensation measurements assist in determining several characteristics of the measuring device 300. These characteristics include a baseline distance LBof a baseline 390 that extends from a perspective center 304 of the projector 302 to a perspective center 316 of the ToF camera 310. The projector 302 sends bundles of light 385 at angles aPrelative to the baseline 390, and the ToF camera 310 receives the imaged bundles of light 385 at angles acrelative to the baseline 390. Compensation measurements are performed using the device 300 to measure known positions on a reference artifact, which might be positions of dots on a calibration dot plate, for instance. The ToF camera 310 measures a total trip distance LT0Fof light emitted from the perspective center 304 of the projector 302, traveling to the object intersection point 388, and then traveling to the perspective center 316 of the camera 310.

[0049] In an embodiment, the relative positions of features on a calibration artifact are found based at least in part on the measured distance LT0F, a baseline distance LB, projection angles aP, and camera angles ac. The measured distance LT0Fdepends on the sum of the distance from the perspective center 304 to the intersection point 388 and the distance from the intersection point 388 to the perspective center 316. The latter three parameters, LB, aP, and ac, are adjusted numerically along with the relative orientation of the measuring device 300 until errors in the measured 3D coordinates of features on the reference artifact are reduced or minimized. After such a compensation procedure is completed, either in the factory or before measurements in the field, the device 300 determines 3D coordinates of points such as the points 386 on the surface 342.

[0050] A triangulation calculation illustrated in FIG. 3B provides additional advantages if the ToF camera 310 is used in the iToF mode. In this case, it is possible to use the triangulation calculation to help resolve ambiguity in the number of full cycles of the modulated waveform that have elapsed before light reflects off the objectand is detected by an iTOF sensor such as the sensor 120. One way to eliminate such ambiguities is to sequentially modulate the projector 302 at two or more modulation frequencies. By measuring a given distance at this plurality of modulation frequencies, it is possible using mathematical techniques well known in the art to determine the integer number of complete cycles used at each of the modulation frequencies. Although this approach is viable, triangulation often offers a faster way to eliminate ambiguity. It does this by calculating distance to the intersection point 388 accurately enough to eliminate the need for modulation at multiple frequencies or at least to reduce or minimize the number of required modulation frequencies. FIG. 3B shows the projected light 306 marked with circles 307 and lines 308. The distance between any two circles or any two lines represents the distance traveled by one complete modulation period, as explained further herein with reference to FIGs. 6A-6D.

[0051] Another advantage of spacing the projector 302 and the camera 310 apart by the baseline distance LBis to perform triangulation calculations that assist in distinguishing the measured spot 314 from other spots 314 in the pattern 312. FIG. 3C shows that the bundles of light 385 intersect the surface 342 at a collection of points 386. However, only the intersection point 388 is close to the position obtained from the time-of-flight calculations.

[0052] FIG. 3D shows a typical situation in which a spot of light 354 is captured on a TOF sensor array 350 having a collection of pixels 352. The exemplary spot 354 encompasses a width of 2 or 3 pixels and a height of 2 or 3 pixels, although more or fewer pixels are encompassed. The iTOF sensor array 350 provides a distance reading for each pixel 352. For triangulation calculations, these distance readings are used directly or distances found in centroids of a projected spots or other feature. Other mathematical techniques can alternatively be used to map out features of the surface 342.

[0053] FIG. 4A shows a 3D measuring device 400 that measures distance and angles to an object. The device 400 includes a projector 402, a clock / timer 420, a first ToF camera 440, and a second ToF camera 470. Each of the ToF cameras 440, 470includes a lens and a ToF sensor, which are used in an iToF or dToF mode. In an embodiment, the projector 402 emits light picked up by the ToF camera 440 as pattern 412, enabling the light to be concentrated in spots 444 or other pattern elements having higher power per unit area than would be possible were the light projected homogeneously over a wide angular region. This higher power per unit enables more accurate measurements to be made for a given limited measurement rate. In a similar manner, the pattern 472 includes spots 474 or other concentrated pattern elements. FIG. 4C illustrates one triangulation geometry that are used with the device 400 to find the 3D coordinates of a point 488. In some embodiments, the functionality of the clock / timer 420 are incorporated into one TOF camera 440 or 470 as a primary clock / modulator and the projector 402 and other TOF camera acting as a secondary clock / modulator.

[0054] The geometry of the measuring device 400 enables three triangulation calculations to be carried out simultaneously if desired. In the case illustrated in FIG. 4C, one triangulation calculation uses a baseline 490 drawn between the perspective centers 446, 476 of the two cameras. The length of the baseline 490 is LB1and the intersection angles to the ToF cameras 440, 470 are acland aC2, respectively. In two other cases the triangulation calculations use baselines drawn between the projector 402 and one of the cameras 440, 470. If three such triangulation calculations are performed, the results should closely match in each of the three cases. If the results do not match well, then a position or orientation of at least one of the projector 402, TOF camera 440, and TOF camera 470 has probably moved. This result indicates that measurements are not be reliable until another compensation of the device 400 is performed. It is also the case that certain of the three triangulation calculations are found to be more reliable than the others, perhaps because of the mechanical construction of the device 400. For any of these situations, the two-camera device 400 provide advantages over the one- camera device 300.

[0055] Measurements of the distances to multiple projected spots of light with the iToF camera enables the one or more processors to determine correspondences among projected pattern elements. In this case, the correspondences are determinedbased at least in part on the calculated path distances to the projected pattern elements. If the one or more processors determines that the identified correspondences are not consistent with the measured distances, further actions are taken by the system to eliminate the problem. Such actions might include dropping inconsistent measurements, performing additional measurements, or requesting a further compensation step by the operator.

[0056] FIG. 5A depicts the case in which the surface 542 to be measured is relatively close to the 3D measuring device 300. This contrasts to the case shown in FIG. 3 A in which the surface 342 is relatively far from the 3D measuring device 300. When an object 342 is relatively far from the 3D measuring device 300, it is generally advantageous to include iToF phase calculations in calculating distances to the object 342. On the other hand, when an object 542 is relatively close to the 3D measuring device 300, measurements are in some cases more accurately and quickly determined based entirely on triangulation calculations.

[0057] FIG. 5B depicts the case of FIG. 5A but with a second TOF camera 470. In the case depicted in FIG. 5B, the surface 542 is relatively close to the 3D measuring device 300. For this case, measurements are in some cases more accurately and quickly determined based entirely on triangulation calculations. On the other hand, for the surface 442 relatively far from the measuring device 400, it is generally advantageous to use the iToF sensor in a modulated mode to calculate 3D coordinates of points on the surface 542.

[0058] FIGs. 6A-6D illustrate the principle of operation of ToF sensor used in an iToF mode. FIG. 6A shows a temporal modulation pattern 600 for light emitted by the projector 112 in response to a signal generated by the clock / timer 116. The period of the emitted light pulses is shown to be T in this example. The modulation frequency is the reciprocal of this value: fM0D= 1 / T. FIG. 6B shows a temporal pattern of the modulated light of FIG. 6A received by the TOF sensor 120. The light has traveled a whole number n of complete periods T plus a measured portion of the period T, which is a phase shift 612. The total phase in radians accumulated by the modulated light intraveling from the projector 112 to the object and then on to a pixel of the ToF sensor 120 is equal to (ptot= 2nn + A< >, where A corresponds to the phase shift 612 measured by the pixel in units of radians.

[0059] An exemplary TOF sensor, such as the model IMX556 manufactured by Sony Corporation, converts photons to electrons and then creates an alternating drift field that divides electrons between two detection junctions synchronized to the modulation frequency also applied to the projector 112 by the clock / timer 116, which are at least partly contained within the TOF camera 114. The two detection junctions are 180 degrees out of phase, as illustrated in FIG. 6C by the timing diagrams 620, 630 of the two detection junctions. The signal generated by each of the two detection junctions is shown in the hatched regions 622, 632 of FIG. 6C. The exemplary sensor is a further sample of incoming light at the additional phases 0, 90, 180, 270 degrees for each given depth frame. Other sampling strategies are possible and possibly desirable, such as if measurements are made continuously for a rotating system as further exemplary systems discussed herein below. The phase step of 90 degrees is illustrated in the waveforms of FIG. 6D. As can be seen from the line 645, the rising edge of the waveform 640 is 90 degrees of phase with respect to the waveform 620, and the waveform 650 is 90 degrees of phase with respect to 630. For this case, the electrons received by the first junction and the second junction are indicated in the highlighted regions 622, 632, respectively. The integrated electrical signals generated at the regions 622, 632, 642, 652 can be used to determine the distance traveled by a beam of light over a region corresponding to the phase shift A marked 612 in FIGs. 6A, 6B. The additional phase step at 180 degrees is a reversal of the waveforms of 0 degree waveforms of FIGs. 6C and can be combined with the results of FIG. 6C to reduce or minimize signal noise. Likewise, the phase step at 270 degrees is a reversal of the waveforms at 90 degrees and can be combined with the results of FIG. 6D to reduce or minimize signal noise. The method described with respect to the FIGs. 6A, 6B, 6C, 6D provides a way to determine the phase shift A marked 612 in FIGs. 6 A, 6B. However, this still leaves undetermined the integer / ? complete 2n radians phase shift of complete cycles in the distance travelled before the arrival of the reflected bundle of light at theiTOF detector 120. As explained herein above, multiple modulation frequencies can be launched sequentially, but an alternative embodiment performs a triangulation calculation as described herein above in reference to FIGs. 3A-4C. The triangulation calculation, possibly supplemented by one or more additional phase measurements, can be used to determine the whole number n of complete modulation cycles.

[0060] Mathematically, the temporal modulation pattern 600 includes a whole number of adjoined modulation cycles, each adjoined modulation cycle having a period T. The temporal modulation of the optical signal is detected by the iToF sensor and appears after a delay as the detected waveform 610. The phase shift from the launch of the modulated light from the projector to the reception of the modulated light by an iToF pixel is equal to a whole number of modulation cycles plus the part of a modulation cycle. The iToF sensor is configured to determine the phase shift 612, also referred to as the phase shift A or phase offset. As indicated in FIG. 4B, triangulation calculations are used in many cases to directly determine the whole number n of modulation cycles, each of period T.

[0061] In an embodiment illustrated in FIG. 7A, a 3D measuring device 700 includes a housing 705, a rotation stage 710, a first measuring device 720, a second measuring device 730, and a two-dimensional (2D) camera 740. The rotation stage includes a motor 712 operable to rotate the housing 705 about an axis 714. Optionally, the rotation stage 710 further includes an angle measuring device such as an angular encoder 715. In an embodiment, the rotation stage 710 is mounted on a housing or stand 702, which might be a tripod or instrument stand, for example. In an embodiment, the first measuring device 720 and the second measuring device 730 each include a projector 112, a ToF camera 114, a clock / timer 116, and a processor 118 (FIG. 1). In other embodiments, the first measuring device 720 and the second measuring device 730 shares a clock / timer 116, a processor 118, or both. The 3D measuring device 700 further includes the 2D camera 740, which in an embodiment is a color camera having many more pixels than the ToF camera 120 of the first measuring device 720 and further having many more pixels than the ToF camera of the second measuring device 730. In an embodiment, the color camera has about 40 times more pixels than the ToF camera.In an embodiment, the first measuring device 720 is aligned to enable its ToF camera 114 to cover angles up to vertical (e.g. a direction perpendicular to the floor the device 700 is mounted on) as the rotation stage 710 rotates the housing 705 about the axis 714. In an embodiment, the second measuring device 730 is aligned to enable the upper edge of the field of view of ToF camera 114 to slightly overlap the lower edge of the field of view of ToF camera 114 in the first measuring device 720. In consequence of this alignment, in this embodiment the first measuring device 720 and the second measuring device 730 cover all directions around the 3D measuring device 700 except for a conical region below the stand 702. In an embodiment, the 2D camera has a field of view covering nearly the same region, at least in the vertical direction. In an embodiment, the camera 740 has a portrait orientation 742.

[0062] In some embodiments, it is desired to replace the single 2D camera in the 3D measuring device 700 with two 2D cameras, each pointing in a different direction and each having a landscape orientation. It should be appreciated that while embodiments herein refer to imaging devices having a particular orientation (e.g. a landscape orientation), this is for example purposes and the claims should not be so limited. In other embodiments, one or more of the imaging devices are configured with a portrait orientation. Two possible arrangement for two such single 2D cameras are shown in FIG. 7B and FIG. 7C. In FIG. 7B, two 2D cameras 750, 760 are placed one above the other in a common housing 758. The top camera 750 includes a lens assembly 752 and a circuit board 754 onto which is soldered a 2D camera chip (not shown) to produce images in a landscape orientation 756. The bottom camera 760 includes a lens assembly 762 and a circuit board 764 onto which is soldered a 2D camera chip (not shown) to produce images in a landscape orientation 766. In FIG. 7C, two 2D cameras 770, 780 are placed one to the side of the other. The left camera 770 includes a lens assembly 772 and a circuit board 774 onto which is soldered a 2D camera chip (not shown) to produce images in a landscape orientation 776. The right camera 780 includes a lens assembly 782 and a circuit board 784 onto which is soldered a 2D camera chip (not shown) to produce images in a landscape orientation 786.

[0063] With a measuring device 100 including 2D cameras 120 (referred to as optional color cameras 120 in FIG. 1), it is possible to use the 2D images from the cameras 120 to register images taken with the measuring device 100, capturing images from a plurality of orientations. For example, the 2D camera 740 in FIG. 7A is used to identify feature points in the environment. Such techniques, such as scale-invariant feature transform (SIFT) and other techniques well known in the art, are sometimes referred to as photogrammetry techniques. By extracting from 2D images a plurality of feature points in each of several angles of rotation about the axis 714, a 3D image can be reconstructed over 3D space, even without the assistance of an angle measuring device (e.g., the angular encoder 715). For this registration technique to be effective, some overlap is needed among adjacent images so that common feature points can be matched. In general, a greater amount of overlap will help ensure a greater likelihood that a satisfactory registration can be obtained. For the case in which a camera assembly includes two 2D cameras, such as those assemblies shown in FIGs. 7B, 7C, it is particularly useful to have a fairly large horizontal overlap as the cameras in FIGs. 7B, 7C are rotated about a vertical axis such as the axis 714.

[0064] FIGs. 8A, 8B, 8C illustrate three different rotation strategies for a system having two 2D cameras as shown in FIG. 7B or FIG. 7C. In each case, the 2D cameras 750, 760 are assumed to have a field-of-view (FOV) of 120 degrees (horizontal) x 85 degrees (vertical). In each case, there is a relatively small overlap in the vertical direction so that together the two cameras cover an angular area of around 120 degrees (horizontal) x 160 degrees (vertical) at a single position of the axis 714. For the case in which there are four horizontal rotations about the axis 714, the total overlap between any two successive captures is ((4 ■ 120) — 360°) / 4 = 30°, as shown in FIG. 8A. The cases for five and six overlapping rotations are illustrated in FIG. 8B and FIG. 8C, respectively. The overlap shown in FIG. 8B for five successive rotations is 48 degrees (horizontal), and the overlap shown in FIG. 8C for six successive rotations is 60 degrees (horizontal). Having a greater overlap makes it easier to register 2D images captured in successive rotations.

[0065] One method to estimate rotation angle is by using a stepper motor to drive rotation about an axis, which might be a vertical axis, also referred to as a pan axis. For example, a stepper motor might have a step resolution of 0.9 degree or 1.8 degrees. Smaller resolutions are available with micro-stepping motors. For most applications, stepper motors will not on their own provide enough accuracy. A significant improvement in angular accuracy can be obtained by matching feature points obtained on 2D cameras in overlapping regions on 2D cameras obtained in successive rotations, as illustrated in FIGs. 8A, 8B, 8C. By matching feature points and possibly imposing a loop closure requirement for rotation about 360 degrees, it is possible to obtain relatively high angular accuracy, even without an angular encoder.

[0066] It is possible to use stepper motors or direct drive motors in combination with angular encoders such as magnetic angular encoders or optical angular encoders. Very high accuracy optical angular encoders are used to obtain angular accuracies up to around one arcsecond, which is equal to approximately 5 microradians. Even when such higher accuracy angular encoders are used, it can still be useful to include one or more 2D cameras in a 3D measuring system such as the camera 740 in the system 700. For example, such 2D cameras help resolve ambiguities occurring when features change such as when a person walks in front of a rotating 3D measuring device 700. When angular encoders are of the lower accuracy variety such as magnetic angular encoders and some optical angular encoders, the photogrammetric matching of feature points with 2D cameras such as the 2D cameras 740, 750, 760, 770, and 780 can help improve accuracy of 3D measurement. Further advantages provided by 2D cameras are discussed herein below.

[0067] In an embodiment, one or more 2D cameras are compensated at the factory and possibly further compensated on site before making a measurement. The term calibration is sometimes used in place of the term compensation. In this document, both terms are used to mean obtaining parameters to improve accuracy of the cameras. Such parameters include aberration coefficients for the camera as well as coefficients angles and orientations for the projector and camera in the 3D measuring device.

[0068] FIGs. 9A, 9B illustrates one method of correcting images by using compensation parameters for a camera system such as the camera system 742 of FIG. 7A. In an embodiment, the compensation parameters are obtained in a factory compensation procedure. An uncorrected image captured by a 2D camera is shown in FIG. 9A. The corrected image is shown in FIG. 9B. Note that the proportions of the corrected images in FIG. 9B appear to follow the expected rules of perspective. Also, notice that lines are mostly straight in FIG. 9B, as expected, but often curved in FIG. 9A.

[0069] FIG. 10 shows a perspective view of an embodiment of a 3D measuring device 1000 that incorporates some of the elements described herein above. Two projectors 1012, 1014 project a pattern of light onto an object. In some embodiments, the projectors 1012, 1014 include small, motorized prisms 1016, 1018, respectively, operable to rotate each of the projected beams of light in a small cone angle. The motorized prisms 1016 and 1018 are low inertia elements and hence permit very rapid movement of the projected beams of light. Descriptions of representative projectors 1012, 1014 are given in reference to elements 112, 200, 302, 402. An exemplary modulation pattern appropriate for use with an iToF sensor is illustrated in FIG. 6A. It is also possible to modulate, either sequentially or simultaneously, at a plurality of modulation frequencies as a way of eliminating ambiguity in the number of complete modulation cycles that have elapsed before the iToF sensor detects reflected light. An alternative way to remove or reduce this ambiguity is through the use of triangulation as discussed herein above, especially in reference to FIGs. 3B, 4B, 5A, 5B. As illustrated in FIGs. 5 A, 5B, when an object being measured is within a predetermined distance to the 3D measuring device, triangulation measurements of distance to object points are more accurate than distance measurement that rely on an iTOF sensor. In an embodiment, the predetermined distance is between 2 - 3 meters. In this case, it is possible to turn off modulation of the projected light, thereby increasing the average optical power received by the optical detector and further reducing the measurement time.

[0070] In an embodiment, at least some of the patterned light projected by the projector 1012 onto an object is received by each of the iToF cameras 1020, 1022, and at least some of the patterned light projected by the projector 1014 onto the object is received by each of the iToF cameras 1030, 1032. A schematic representation of patterns of projected light 444, 474 picked up by iToF sensors 440, 470, respectively, is shown in FIG. 4A. FIG. 1 shows how timing of image processing by a TOF camera 114 is interconnected through a clock / timer 1167 and processor 118 to the projector 112.

[0071] In an embodiment, elements of the 3D measuring device 1000 are housed in a housing 1002 coupled to a motorized pan-axis rotation assembly 1040. In embodiments, the pan-axis rotation assembly 1040 includes a motor 1042 operable to rotate the housing 1002 about a motor axis 1044. In an embodiment, the 3D measuring device 1000 is affixed to a stand such as an instrument stand or tripod. In an embodiment, the rotation assembly 1040 further includes an angle measuring device such as an optical angular encoder 1046. In an embodiment, the optical angular encoder 1046 includes a read head 1047 and an encoder disk 1048 having a multiplicity of marks. In an embodiment, the angular encoder is designed to operate in a reflection mode, with light sources on the read head projecting light reflected by the encoder disk onto detectors on the read head. The angle of rotation of the disk is determined based on the reflected light received by the read-head detectors. In other embodiments, a different type of angular measuring device or no angular measuring device is used.

[0072] In an embodiment, the 3D measuring system 1000 further comprises an electronic system 1200 that includes an embedded personal computer (PC) 1060. FIG. 12 shows one way to implement system processing within the 3D measuring instrument 1000. Electronics 1200 within the 3D measuring system 1000 includes the embedded PC 1060 that has a control and processing unit 1212, a first MIPI aggregator 1214, a second MIPI aggregator 1216, a solid state drive (SSD) 1216, a wide local area network (WLAN) interface 1218, and a speaker 1219. In an embodiment, the control and processing unit 1212 is a Jetson Orin System on a Module (SoM) manufactured by NVIDIA Corporation. The first and second MIPI aggregators 1214, 1216 providecommunication links between the control and processing unit 1212 and the iToF camera.

[0073] In an embodiment, additional components in communication with the control and processing unit 1212 include the motorized pan-axis rotation assembly 1240, buttons 1242, light emitting diodes (LEDs) 1244, an inertial measurement unit (IMU) 1246, one or more fans 1248, temperature sensors 1250, and a microcontroller 1256. In an embodiment shown in FIG. 2, connections are made with a Mobile Industry Processor Interface (MIPI), an Inter-Integrated Circuit (I2C) interface, a Serial Peripheral Interface (SPI), a Peripheral Component Interconnect express (PCIe) interface, a Controller Area Network (CAN) bus, and a General Purpose Input Output (GPIO) bus. Additional components include a system field programmable gate array (FPGA) 1257 or other processors. Mobile apps 1253 are supported. As shown in FIG. 10, the 3D measuring instrument 1000 further includes a power supply circuit board 1070, a heat sink 1072, and a battery 1074.

[0074] The 3D measuring device 1000 further includes 2D color cameras 1050, 1052. In an embodiment, the field-of-view (FOV) of the 2D color camera 1050 substantially overlaps the coverage of the 3D measuring subsystem that includes the projector 1012 and the TOF cameras 1020, 1030. Further, the FOV of the 2D color camera 1052 substantially overlaps the coverage of the 3D measuring subsystem that includes the projector 1014 and the TOF cameras 1030, 1032. Elements taken together to include the first projector 1016, the first and second TOF cameras 1020, 1022, and the color camera 1050 all together comprise a first unit 1080. Elements taken together to include the second projector 1014, the third and fourth TOF cameras 1030, 1032, and the color camera 1052 all together comprise a second unit 1082. The first unit 1080 and the second unit 1082 are considered to be subunits held within the housing 1002.

[0075] Advantages of 2D color cameras already discussed herein include (1) ability to determine angles to feature points to relatively high accuracy even without the availability of one or more high accuracy angular encoders; (2) obtaining a count of the number of modulation cycles completed before modulated light launched by aprojector reaches the object and then travels to the ToF camera; (3) obtaining a more accurate understanding of the position of the projected spots in relation to nearby spots; and (4) noting capture on unwanted image features such as the presence of people walking through a beam launched by the 3D measuring instrument 1000; (5) identifying possible errors in 3D (iToF) measurement at edges based at least partly on changes in depth or texture as seen in color images; and (6) colorizing an image. In addition, another important advantage of 2D color cameras has been described in commonly owned U.S. Provisional Patent Application No. 63 / 428131 (hereafter ‘ 131) on November 28, 2022, the contents of which are incorporated herein by reference. The disclosure of the ‘ 131 patent enables a high-resolution point cloud to be obtained from the combination of a low-resolution point cloud and a high-resolution image for the case in which each (A) pixel coordinate has at least a color and (B) the color is the correct color in the sense that no parallax errors are visible in the resulting point cloud.

[0076] FIG. 11 shows a front view of an embodiment of a 3D measuring device 1100 that incorporates some of the elements described herein above. The device 1100 is functionally similar to the device 1000. Two projectors 1112, 1114 project a pattern of light onto an object. In an embodiment, at least some of the patterned light projected by the projector 1112 onto an object is received by each of the iToF cameras 1130, 1122, and at least some of the patterned light projected by the projector 1114 onto the object is received by each of the iToF cameras 1120, 1132.

[0077] In an embodiment, elements of the 3D measuring device 1100 are housed in a housing 1102 coupled to a motorized pan-axis rotation assembly 1140. In embodiments, the pan-axis rotation assembly 1140 includes a motor 1142 operable to rotate the housing 1102 about a motor axis 1144. In an embodiment, the rotation assembly 1140 further includes an angle measuring device such as an optical angular encoder 1146. In an embodiment, the optical angular encoder 1146 includes a read head 1147 and an encoder disk 1148, the encoder disk having a multiplicity of marks. In an embodiment, the angular encoder is designed to operate in a reflection mode, with light sources on the read head projecting light reflected by the encoder disk onto detectors on the read head. The angle of rotation of the disk is determined based on the reflectedlight received by the read-head detectors. In other embodiments, a different type of angular measuring device or no angular measuring device is used.

[0078] The 3D measuring device 1100 further includes 2D color cameras 1150, 1152. In an embodiment, the field-of-view (FOV) the 2D color camera 1150 substantially overlaps the coverage of the 3D measuring subsystem that includes the projector 1112 and the TOF cameras 1122, 1130. Further, the FOV of the 2D color camera 1152 substantially overlaps the coverage of the 3D measuring subsystem that includes the projector 1114 and the TOF cameras 1120, 1132. The 3D measuring device may be further considered to comprise a first unit that includes the projector 1112, the TOF cameras 1122, 1130, and the color camera 1150, and a second unit that includes the projector 1114, the TOF cameras 1120, 1132, and the color camera 1152.

[0079] In an embodiment, a method is provided where the color image of a given frame is acquired. The method then proceeds to perform a method on the image that detects features in the color image. Such features can be e.g. lines, comers, areas where a sharp depth increase / decrease (due to prior knowledge on similar features), or other. In the areas of the detected features, one can run additional methods for depth computation (e.g. combining a different set of phase images, check with triangulation) to identify and correct potential errors that occur in these areas. When multiple measurements yield the same distance result, the method determines / concludes that there is no issue and simply use the measurement. Alternatively, the method simply disregards the measurement in areas where different measurement techniques yield different results.

[0080] In another embodiment, multiple depth measurements of a single point in space is performed with the system described herein , i.e. triangulation, iToF with camera 1, iToF with camera 2. In one embodiment, these measurements can be averaged to give a more precise depth measurement. If these measurements yield distance values that are significantly (i.e. larger than the inherent error) different from one another, an issue is detected and either this data point is deleted or refined by otherdepth measurement techniques (e.g. use a different combination of phase images or scan the region again with a different rotation speed of the scanner).

[0081] In still another embodiment, depth measurements from different techniques (iToF camera 1, iToF camera 2, triangulation) are used to detect issues, depth measurements from different combination of iToF phase images are also used to make separate depth measurements, these are compared and issues detected in the image.

[0082] Embodiments described herein further provide for improving the accuracy of a rotating measuring device based on an indirect time-of-flight depth measurement. Such embodiments are useful for iToF measurements of a moving measuring device, such as a rotating measuring device (e.g., the system 100 for measuring 3D coordinates on objects, the device 300, the device 400, the 3D measuring device 700, the 3D measuring device 1000, the 3D measuring device 1100, and / or the like including combinations and / or multiples thereof).

[0083] For example, for an iToF measurement, a certain number of phase frames are used to compute a depth for each pixel of a TOF camera. According to one or more embodiments described herein, it is be desirable to record multiple or all phase frames from the same position and angle of the rotating scanner. Recording multiple or all phase frames from the same position and angle of the rotating scanner can improve the measurement accuracy of the rotating scanner.

[0084] An iToF measurement uses a modulated detector chip (e.g., a sensor) of a camera (e.g., a camera, such as the ToF camera 114, and / or the like including combinations and / or multiples thereof) in combination with a modulated light source (e.g., VCSEL array 202 (see also FIG. 6)). As described herein, a measuring device (e.g., the measuring device 100) that uses an iToF sensor (e.g., the iToF sensors 440, 470) detects multiple phases of light. More specifically, the measuring device emits a single phase of light. The light is then detected by detectors that are modulated at the same frequency but with varying phases. That is, the detectors have varying phase abilities. The measuring device then performs a distance measurement based onreceiving the emitted light back at the iToF sensor. The iToF sensor includes pixels, and each of the pixels can include multiple “wells” for receiving each phase of light. According to one or more embodiments described herein, each well can be associated with multiple detectors. According to one or more embodiments described herein, each of the wells is associated with one of the phases of light. For example, a first well is associated with a 0 degree phase, a second well is associated with a 90 degree phase, a third well is associated with a 180 degree phase, and a fourth well is associated with a 270 degree phase. It should be appreciated that fewer or more wells can be implemented in other embodiments and that wells can be associated with phases other than the examples set forth herein.

[0085] The depth measurement uses a set of “phase frames” where each of the phase frames includes a number of images, known as “phase images,” that are recorded at the same time. For example, multiple images can be recorded with different phases at the same time. As an example, a single detector module can record, at the same item, a first image and a second image that are 180 degrees apart in phase. Other numbers of images and phase offsets can be used in other examples (e.g., a phase frame can include four images that are 90 degrees apart in phase (e.g., 0 degree phase, 90 degree phase, 180 degree phase, 270 degree phase). A phase image indicates the relative phase between the modulated light and the modulated camera and is acquired with each phase frame.

[0086] Conventionally, an iToF measurement is performed in a static situation where the measuring device does not move between phase frames used to make a depth measurement. In a conventional iToF depth measurement, each pixel of the sensor receives data as part of multiple phase images. From these data, a depth measurement can be computed for each pixel. To avoid measurement artefacts, the different phase images are recorded from the same position and angle. However, a rotating measuring device can introduce measurement artifacts due to the moving nature of the measuring device.

[0087] One approach to avoid such artifacts for a rotating measuring device is to record static sets of phase images with small rotating movements in between. However, this approach has several shortcomings, as follows. Before and after each stop, the scanner has to be accelerated / decelerated, which requires more powerful motors than can otherwise be used to cause the measuring device to rotate. If such motors are not used, the resulting point density of the point cloud is unsatisfactory (e.g., not sufficiently dense). Moreover, no movement in the vertical direction is possible.

[0088] To address these and other shortcomings, one or more embodiments described herein provide for accurate depth measurements with a measuring device using an iToF device by implementing a moving projection pattern. For example, rather than moving the entire measuring device, the projected pattern can be moved incrementally while the measuring device stays stationary. Multiple frames can be acquired by moving the projected pattern to different positions and collecting multiple frames at each of the positions, and depth measurements can be computed using those frames. For example, the pattern can move incrementally (e.g., in steps) without moving the measuring device itself such that a set of phase frames is recorded at each position (e.g., a set of phase frames is recorded at each of the steps). Once a desired amount of depth measurements are captured, the measuring device can be rotated to a new orientation or moved to a new position, and the process can repeat to perform additional depth measurements.

[0089] Advantageously, such an approach moves a smaller mass (e.g., a mirror, a tilt / rotation platform) rather than the entire mass of the measuring device in between two sets of phase frames. Such an approach also reduces the time between measurements, meaning the measurements can be made significantly faster. According to one or more embodiments described herein, fast movement of the laser pattern can be performed in both transverse directions. Essentially, a fast scanner is provided over a limited field of view that provides for fast densification of the point cloud. Once the measurement density in the field of view is sufficiently large (e.g., large enough to satisfy a threshold), the measuring device can rotate or be moved to a next measuring position.

[0090] For example, FIG. 13 depicts a flow diagram of a method 1300 for computing depth measurements with a measuring device using an indirect time of flight (iToF) device and a moving pattern according to one or more embodiments described herein. According to one or more embodiments described herein, the moving measuring device can be any one of the systems or devices described herein, such as the system 100 for measuring 3D coordinates on objects, the device 300, the device 400, the 3D measuring device 700, the 3D measuring device 1000, the 3D measuring device 1100, and / or the like including combinations and / or multiples thereof. The method 1300 can be implemented by any suitable device or system as described herein. For example, the method 1300 can be implemented by the measuring device 100 using the processor 118. As another example, the method 1300 can be implemented by the electronics 1200 within the 3D measuring system 1000. Other examples of devices and systems to implement the method 1300 are also possible. The method 1300 is now described with reference to FIGS. 14A-14D, which depict a measuring device 1410 in an environment1400 according to one or more embodiments described herein.

[0091] At block 1302, the measuring device 1410 is arranged at a first position1401 and a first orientation 1403 of an environment 1400. The measuring device 1410 is configured to project a pattern in the environment 1400 as described herein within a first field of view 1412. At block 1304, the measuring device 1410, while at the first position 1401 and the first orientation 1403, projects a pattern 1414 at a first angular position 1414a (see FIG. 14 A). The measuring device 1410 acquires a first frame of the pattern 1414. At block 1306, the measuring device 1410, while at the first position 1401 and the first orientation 1403, projects a pattern 1415 at a second angular position 1415a (see FIG. 14B). The measuring device 1410 acquires a second frame of the pattern 1415. At block 1308, the measuring device 1410 (or another suitable device, such as a processing system) computes first depth measurements to points in the environment 1400 within a field of view 1412, as described herein, based at least in part on at least one of the first frame and the second frame.

[0092] At block 1308, the measuring device 1410 movedto a second orientation1404, from the first orientation 1403, while remaining at the first position 1401. It canbe observed that the field of view 1412 of the measuring device 1410 changes to the field of view 1413 as the orientation of the measuring device 1410 changes from the first orientation 1403 to the second orientation 1404. At block 1310, the measuring device 1410, while at the first position 1410 and the second orientation 1414, projects a pattern 1416 at a third angular position 1416a (see FIG. 14C). The measuring device 1410 acquires a third frame of the pattern 1416. At block 1312, the measuring device 1410, while at the first position 1410 and the second orientation 1414, projects a pattern 1417 at a fourth angular position 1417a (see FIG. 14D). The measuring device 1410 acquires a fourth frame of the pattern 1417. At block 1316, the measuring device 1410 (or another suitable device, such as a processing system) computes second depth measurements to points in the environment 1400 within the field of view 1413, as described herein, based at least in part on at least one of the third frame and the fourth frame.

[0093] According to one or more embodiments described herein, causing the first orientation 1403 of the measuring device 1410 to change to the second orientation 1404 is based at least in part on a measurement density of the first measurements satisfying a threshold. For example, once a threshold number of measurements to points within a field of view (e.g., the field of view 1412) are captured, the measuring device 1410 can be repositioned (e.g., the orientation and / or position of the measuring device 1410 can be changed). As another example, causing the first orientation 1403 of the measuring device 1410 to change to the second orientation 1404 is based at least in part on a user selection (e.g., the user can select when to cause the orientation to change).

[0094] Additional processes also are included, and it should be understood that the process depicted in FIG. 13 represents an illustration, and that other processes are added or existing processes are removed, modified, or rearranged without departing from the scope of the present disclosure.

[0095] FIGS. 15A-15C depict architectures 1501, 1502, 1503 for projecting a moving pattern according to one or more embodiments described herein. Although the architectures 1501 - 1503 are shown, other possible architectures for proj ecting a movingpattern are possible. Each of the architectures 1501-1503 can be implemented in a measuring device, such as the measuring devices described herein (e.g., the system 100 for measuring 3D coordinates on objects, the device 300, the device 400, the 3D measuring device 700, the 3D measuring device 1000, the 3D measuring device 1100, and / or the like including combinations and / or multiples thereof).

[0096] In FIG. 15 A, the architecture 1501 includes a laser source 1510 (e.g., laser emitter elements 204 of the VCSEL array 202), a moveable mirror 1512, and a diffractive optical element (DOE) 1514 configured and arranged as shown. According to an embodiment, projecting the pattern of the method 1300 can include emitting a beam from the laser source 1510 onto the movable mirror 1512 and deflecting, by the movable mirror 1512, the laser beam through the diffractive optical element 1514 into the environment 1400. According to one or more embodiments described herein, the diffractive optical element (DOE) 1514 can be removed such that the movable mirror 1512 deflects the laser beam directly into the environment 1400.

[0097] According to one or more embodiments described herein, the moveable mirror 1512 can instead be a rotating transmissive prism to cause the projected pattern to move. For example, a beam can be emitted from the laser source 1510 onto the rotating transmissive prism, which can deflect the laser beam through the diffractive optical element 1514 into the environment 1400. As another example, The beam can be emitted from the laser source 1510 through the diffractive optical element 1514 onto the rotating transmissive prism, which deflects the laser beam into the environment 1400.

[0098] In FIG. 15B, the architecture 1502 includes the laser source 1510 (e.g., laser emitter elements 204 of the VCSEL array 202), the moveable mirror 1512, and the diffractive optical element 1514 configured and arranged as shown. According to an embodiment, projecting the pattern of the method 1300 can include emitting a beam from the laser source 1510 through the diffractive optical element 1514 onto the movable mirror 1512 and deflecting, by the movable mirror 1512, the laser beam intothe environment 1400. The architecture 1502 of FIG. 15B provides for dynamic beam deflection.

[0099] In FIG. 15C, the architecture 1503 includes the laser source 1510 (e.g., laser emitter elements 204 of the VCSEL array 202) and the diffractive optical element 1514 configured and arranged as shown. In this example, the laser source 1510 and the diffractive optical element 1514 are housed in a tilt / rotation platform 1520 According to an embodiment, projecting the pattern of the method 1300 can include emitting a beam from the laser source 1510 through the diffractive optical element 1514 into the environment 1400.

[0100] According to one or more embodiments described herein, the diffractive optical element 1514 can move (e.g., from a first angular position to a second angular position) to cause the projected pattern to move.

[0101] According to one or more embodiments described herein, the measuring device includes a light control metasurface chip that causes the projected pattern to move, such as from the first angular position to the second angular position. Light control metasurface chips provide beam steering capabilities for directing projected patterns.

[0102] Other architectures are also possible, as are other configurations, arrangements, and implementations of the embodiments of FIGS. 15A-15C. According to one or more embodiments described herein, the measuring device 1410 includes the laser source 1510 and a laser source angle of the laser source 1510 changes using phased-array optics. According to one or more embodiments described herein, the measuring device 1410 can include the laser source 1510 and the diffractive optical element 1514. The laser source 1510 can be tuned in wavelength (e.g., using temperature, current, piezo mirror, and / or the like including combinations and / or multiples thereof). A diffraction angle of the diffractive optical element 1514 can depend on a wavelength of the laser source 1514. The diffractive optical element 1514 is wavelength dependent such that the diffraction angle changes causing the pattern to shift responsive to changes in the wavelength of the laser source 1510.

[0103] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include causing the first orientation of the measuring device to change to the second orientation is based at least in part on a measurement density of the first measurements satisfying a threshold.

[0104] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include causing the first orientation of the measuring device to change to the second orientation is based at least in part on a user selection.

[0105] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include the measuring device comprises a laser source, a movable mirror, and a diffractive optical element.

[0106] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam through the diffractive optical element into the environment.

[0107] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element onto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

[0108] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and a rotating transmissive prism, and a diffractive optical element, and wherein projecting the pattern comprises emitting a beam from the laser source onto the rotating transmissive prism and deflecting, by the rotating transmissive prism, the laser beam through the diffractive optical element into the environment.

[0109] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and a rotating transmissive prism, and a diffractive optical element, and wherein projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element onto the rotating transmissive prism and deflecting, by the rotating transmissive prism, the laser beam into the environment.

[0110] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and a diffractive optical element, and wherein a movement of the pattern from the first angular position to the second angular position is caused by a movement of the diffractive optical element.

[0111] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a light control metasurface chip, wherein the light control metasurface chip causes the pattern to move from the first angular position to the second angular position.

[0112] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a tilt / rotation platform, the tilt / rotation platform comprising a laser source and a diffractive optical element.

[0113] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element into the environment.

[0114] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and wherein a laser source angle of the laser source changes using phased-array optics.

[0115] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and a diffractive optical element, wherein the laser source is tuned in wavelength, wherein a diffraction angle of the diffractive optical element depends on a wavelength of the laser source, wherein the diffractive optical element is wavelength dependent, and wherein the diffraction angle changes causing the pattern to shift responsive to changes in the wavelength of the laser source.

[0116] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method include that the measuring device comprises a laser source and a movable mirror, and wherein projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

[0117] In addition to one or more of the features described herein, or as an alternative, further embodiments of the measuring device include that the measuring device changes from the first orientation to the second orientation is based at least in part on a measurement density of the first measurements satisfying a threshold.

[0118] In addition to one or more of the features described herein, or as an alternative, further embodiments of the measuring device include that the projector is a laser source, wherein the measuring device comprises a movable mirror and a diffractive optical element.

[0119] In addition to one or more of the features described herein, or as an alternative, further embodiments of the measuring device include projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam through the diffractive optical element into the environment.

[0120] In addition to one or more of the features described herein, or as an alternative, further embodiments of the measuring device include projecting the pattern comprises emitting a beam from the laser source through the diffractive optical elementonto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

[0121] The term “about” is intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ± 8% or 5%, or 2% of a given value.

[0122] Additionally, the term "exemplary" is used herein to mean "serving as an example, instance or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" are understood to include any integer number greater than or equal to one, i.e. one, two, three, four, etc. The terms "a plurality" are understood to include any integer number greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include an indirect "connection" and a direct "connection." It should also be noted that the terms “first”, “second”, “third”, “upper”, “lower”, and the like are used herein to modify various elements. These modifiers do not imply a spatial, sequential, or hierarchical order to the modified elements unless specifically stated.

[0123] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and / or groups thereof.

[0124] While the disclosure is provided in detail in connection with only a limited number of embodiments, it should be readily understood that the disclosure is not limited to such disclosed embodiments. Rather, the disclosure can be modified to incorporate any number of variations, alterations, substitutions or equivalentarrangements not heretofore described, but which are commensurate with the spirit and scope of the disclosure. Additionally, while various embodiments of the disclosure have been described, it is to be understood that the exemplary embodiment s) include only some of the described exemplary aspects. Accordingly, the disclosure is not to be seen as limited by the foregoing description but is only limited by the scope of the appended claims.

Claims

CLAIMSWhat is claimed is:

1. A method for computing depth measurements with a measuring device using an indirect time of flight (iToF) device, the method comprising: causing the measuring device to be arranged at a first position and a first orientation of an environment, the measuring device configured to project a pattern in the environment; projecting, while the measuring device is at the first position and the first orientation, the pattern at a first angular position and acquiring, using the iToF device, a first frame of the pattern; projecting, while the measuring device is at the first position and the first orientation, the pattern at a second angular position and acquiring, using the iToF device, a second frame of the pattern; computing first measurements to points in the environment based at least in part on at least one of the first frame and the second frame; causing the first orientation of the measuring device to change to a second orientation; projecting, while the measuring device is at the first position and the second orientation, the pattern at a third angular position and acquiring, using the iToF device, a third frame of the pattern; projecting, while the measuring device is at the first position and the second orientation, the pattern at a fourth angular position and acquiring, using the iToF device, a fourth frame of the pattern; and computing second measurements to points in the environment based at least in part on at least one of the third frame and the fourth frame.

2. The method of claim 1, wherein causing the first orientation of the measuring device to change to the second orientation is based at least in part on a measurement density of the first measurements satisfying a threshold.

3. The method of claim 1, wherein causing the first orientation of the measuring device to change to the second orientation is based at least in part on a user selection.

4. The method of claim 1, wherein the measuring device comprises a laser source, a movable mirror, and a diffractive optical element.

5. The method of claim 4, wherein projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam through the diffractive optical element into the environment.

6. The method of claim 4, wherein projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element onto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

7. The method of claim 1, wherein the measuring device comprises a laser source and a rotating transmissive prism, and a diffractive optical element, and wherein projecting the pattern comprises emitting a beam from the laser source onto the rotating transmissive prism and deflecting, by the rotating transmissive prism, the laser beam through the diffractive optical element into the environment.

8. The method of claim 1, wherein the measuring device comprises a laser source and a rotating transmissive prism, and a diffractive optical element, and wherein projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element onto the rotating transmissive prism and deflecting, by the rotating transmissive prism, the laser beam into the environment.

9. The method of claim 1, wherein the measuring device comprises a laser source and a diffractive optical element, and wherein a movement of the pattern from the first angular position to the second angular position is caused by a movement of the diffractive optical element.

10. The method of claim 1, wherein the measuring device comprises a light control metasurface chip, wherein the light control metasurface chip causes the pattern to move from the first angular position to the second angular position.

11. The method of claim 1, wherein the measuring device comprises a tilt / rotation platform, the tilt / rotation platform comprising a laser source and a diffractive optical element.

12. The method of claim 11, wherein projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element into the environment.

13. The method of claim 1, wherein the measuring device comprises a laser source and wherein a laser source angle of the laser source changes using phased-array optics.

14. The method of claim 1, wherein the measuring device comprises a laser source and a diffractive optical element, wherein the laser source is tuned in wavelength, wherein a diffraction angle of the diffractive optical element depends on a wavelength of the laser source, wherein the diffractive optical element is wavelength dependent, and wherein the diffraction angle changes causing the pattern to shift responsive to changes in the wavelength of the laser source.

15. The method of claim 1, wherein the measuring device comprises a laser source and a movable mirror, and wherein projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

16. A measuring device comprising: a housing; a projector operably coupled to the housing, the projector configured to project a pattern into an environment;an indirect time-of-flight (ToF) camera coupled to the housing, the indirect ToF camera having a lens and a two-dimensional array of pixels, each pixel configured to convert reflected light into an electrical signal; and at least one processor operably coupled to the housing, the projector and the indirect ToF camera, the at least one processor configured to compute depth measurements of points of the environment by performing operations comprising: projecting, while the measuring device is at a first position and a first orientation of the environment, the pattern at a first angular position and acquiring, using the iToF camera, a first frame of the pattern; projecting, while the measuring device is at the first position and the first orientation, the pattern at a second angular position and acquiring, using the iToF camera, a second frame of the pattern; computing first measurements to the points in the environment based at least in part on at least one of the first frame and the second frame; projecting, while the measuring device is at the first position and a second orientation, the pattern at a third angular position and acquiring, using the iToF camera, a third frame of the pattern; projecting, while the measuring device is at the first position and the second orientation, the pattern at a fourth angular position and acquiring, using the iToF camera, a fourth frame of the pattern; and computing second measurements to the points in the environment based at least in part on at least one of the third frame and the fourth frame.

17. The measuring device of claim 16, wherein the measuring device changes from the first orientation to the second orientation is based at least in part on a measurement density of the first measurements satisfying a threshold.

18. The measuring device of claim 16, wherein the projector is a laser source, wherein the measuring device comprises a movable mirror and a diffractive optical element.

19. The measuring device of claim 18, wherein projecting the pattern comprises emitting a beam from the laser source onto the movable mirror and deflecting, by the movable mirror, the laser beam through the diffractive optical element into the environment.

20. The measuring device of claim 18, wherein projecting the pattern comprises emitting a beam from the laser source through the diffractive optical element onto the movable mirror and deflecting, by the movable mirror, the laser beam into the environment.

Images