Coordinate measurement device that performs time of flight measurements

Abstract

An entry-level scanning system includes a first scanner device coupled to a support structure and has a first projector, a first tracking camera and first TOF cameras, while a second scanner device is coupled to the support structure and includes a second projector, a second tracking camera and a second ToF camera, and a processor determines a position and orientation of the system along with 3D coordinates of an object in an environment using image data and 3D coordinate data acquired by either or both of the first scanner device and the second scanner device. The first ToF camera(s) have a first vertical orientation and the second ToF cameras have a second vertical orientation such that the first ToF cameras and the second ToF cameras have a cumulative vertical field of view of about 180 degrees.

Description

138178-70720COORDINATE MEASUREMENT DEVICE THAT PERFORMS TIME OF FLIGHT MEASUREMENTSCROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of Provisional Patent Application Serial No. 63 / 695,983 titled, “COORDINATE MEASUREMENT DEVICE THAT PERFORMS TIME OF FLIGHT MEASUREMENTS” filed on September 18, 2024, the content of which is incorporated by reference herein.BACKGROUND

[0002] The subject matter disclosed herein relates to a system for measuring three- dimensional (3D) coordinates in an environment, and in particular to a system and method for measuring a pattern of light using a time-of-flight (ToF) sensor.

[0003] A traditional ToF scanner is a scanner in which the distance to a target point is determined based on the speed of light in air of a beam of light traveling between the scanner and a target point. Traditional ToF scanners are typically used for scanning closed and open spaces such as interior areas of buildings, industrial installations and tunnels. They are used, for example, in industrial applications and accident reconstruction applications. A laser scanner optically scans and measures objects in a volume around the scanner through the acquisition of data points representing object surfaces within the volume. Such data points are obtained by transmitting a beam of light onto the objects and collecting the reflected or scattered light to determine the distance, two-angles (i.e., an azimuth and a zenith angle), and optionally a gray-scale value. This raw scan data is collected, stored and sent to at least one processor to generate a 3D image representing the scanned area / object. For the case in which the light source within a scanner is a laser, such a scanner is often referred to as a laser scanner. The term laser scanner is often also used for scanners that use light sources that are not lasers, such as light sources using superluminescent diodes for example.

[0004] ToF measuring systems such as those used in laser scanners are typically one of two types: a phased-based ToF scanner or a pulsed ToF scanner. In a typical phase-based ToF scanner, a beam of light is modulated at a plurality of frequencies before being launched to aMEl\57949330.vl138178-70720 target. After the modulated beam of light has completed a round trip to and from the target, it is demodulated to determine the returning phase of the each of the plurality of frequencies. A processor within the ToF scanner uses the demodulated frequencies and the speed of light in air to determine a distance from the scanner to the target. In contrast, a pulsed ToF scanner typically emits a short pulse of light and measures the elapsed time between launch of the pulse and return of the pulse after having completed a round trip to the target. A processor within the pulsed ToF scanner determines the distance from the scanner to the target based at based at least in part on the measured elapsed time and the speed of light in air. The ToF scanners used in laser scanners today typically include a single optical detector that measures the signal returned from the target. Such optical detectors typically measure up to frequencies from several hundred mega-hertz (MHz) down to pulse widths of a few picoseconds to nanoseconds.

[0005] More recently, ToF methods are being employed in camera sensors having a collection / array of photosensitive elements. Each of the photosensors in the array serves the same function as the single optical detector in a traditional ToF laser scanner, but the photosensors typically are more limited in the speed of their response and their electronic bandwidths. On the other hand, arrays of photosensors are relatively inexpensive, thereby offering advantages where the range and accuracy requirements are not as stringent as for traditional laser scanners.

[0006] A device that uses an array of sensors to measure a stream of modulated light is said to be an indirect ToF (iToF) device, while a device that uses an array of sensors to measure pulsed light is said to be a direct ToF (dToF) device. If an array of pixels using a ToF is included within a camera having a camera lens, then both distances to the target points are determined based on the signals received by the array of pixels.

[0007] While existing systems for measuring distance and angles to an object are suitable for their intended purposes, the need for improvement remains, particularly in providing 3D measurement system having the features described herein.BRIEF DESCRIPTION OF THE DISCLOSURE

[0008] According to one aspect of the present disclosure, a system can comprise a support structure, a first scanner device operably coupled to the support structure, a second scanner device operably coupled to the support structure, and at least one processor. The first scanner device includes a first ToF system including a first projector that emits a first light2MEl\57949330.vl138178-70720 beam into an environment of the system and a ToF camera that detects light reflected from objects in the environment, and a first tracking camera having a fixed connection to the first ToF system and that captures first images of the environment. The second scanner device can include a second ToF system including a second projector that emits a second light beam into the environment and a ToF camera that detects light reflected from the objects in the environment, and a second tracking camera having a fixed connection to the second ToF system and that captures second images of the environment. The at least one processor in communication with the first ToF system, the first tracking camera, the second ToF system and the second tracking camera. The at least one processor determines a three-dimensional (3D) position and orientation of the system using at least one of the first image or the second image of the environment along with output signals from at least one of the first ToF camera or the second ToF camera, and determine 3D coordinates of an object in the environment based at least in part on the output signals from at least one of the first ToF camera or the second ToF camera. The first ToF camera has a first vertical orientation and the second ToF camera has a second vertical orientation such that the first ToF camera and the second ToF camera have a cumulative vertical field of view (VFOV) of substantially 180 degrees.

[0009] In some embodiments, the first vertical orientation is an upward orientation such that a first VFOV of the first ToF camera includes an upper portion of the cumulative VFOV, and the second vertical orientation is a downward orientation such that a second VFOV of the second ToF camera includes a lower portion of the cumulative VFOV.

[0010] In some embodiments, a vertical orientation of the first projector is an upward orientation such that the first light beam spans an upper portion of the cumulative VFOV, and a vertical orientation of the second projector is a downward orientation such that the second light beam spans a lower portion of the cumulative VFOV.

[0011] In some embodiments, a vertical orientation of the first projector is a downward orientation such that the first light beam spans an upper portion of the cumulative VFOV, and a vertical orientation of the second projector is an upward orientation such that the second light beam spans an upper portion of the cumulative VFOV.

[0012] In some embodiments, a vertical orientation of the first tracking camera is an upward orientation, and a vertical orientation of the second tracking camera is a downward orientation.3MEl\57949330.vl138178-70720

[0013] In some embodiments, the first scanner device has a first horizontal orientation, and the second scanner device has a second horizontal orientation that is opposite to the first horizontal orientation.

[0014] In some embodiments, the at least one processor uses a visual simultaneous localization and mapping (SLAM) algorithm to determine the 3D position and orientation of the system and the 3D coordinates of the object in the environment.

[0015] In some embodiments, the support structure includes a motor that rotates the first scanner device and the second scanner device simultaneously, and the at least one processor causes the motor to regularly stop rotation of the first and second scanner devices, and causes the first and second tracking cameras to capture the first and second images of the environment while the rotation of the first and second scanner devices is stopped.

[0016] In some embodiments, the support structure includes a motor that rotates the first scanner device and the second scanner device simultaneously, and the at least one processor causes the first ToF camera and the second ToF camera to acquire 3D data during a first rotation cycle, and_causes the first and second tracking cameras to capture the first and second images of the environment during a second rotation cycle.

[0017] In some embodiments, the at least one processor synchronizes acquisition of 3D data by the first ToF camera and the second ToF camera with acquisition of the first images and the second images by the first tracking camera and the second tracking camera.

[0018] According to another aspect of the present disclosure, a method comprises acquiring first image data and first 3 -dimensional (3D) data of an environment by a first scanner device coupled to a support structure, the first scanner device including a first projector, a first ToF camera and a first tracking camera, acquiring second image data and second 3D data of the environment by a second scanner device coupled to the support structure, the second scanner device including a second projector, a second ToF camera and a second tracking camera, determining a position and orientation of at least one of the first scanner device or the second scanner device using at least one of the first image data and the second image data along with at least one of the first 3D data or the second 3D data, and determining 3D coordinates of an object in the environment based at least in part on at least one of the first 3D data or the second 3D data. The first ToF camera has a first vertical orientation and the second ToF camera has a second vertical orientation such that the first ToF camera and the4MEl\57949330.vl138178-70720 second ToF camera have a cumulative vertical field of view (VFOV) of substantially 180 degrees.

[0019] In some embodiments, the first vertical orientation is an upward orientation such that a first VFOV of the first ToF camera includes an upper portion of the cumulative VFOV, and the second vertical orientation is a downward orientation such that a second VFOV of the second ToF camera includes a lower portion of the cumulative VFOV.

[0020] In some embodiments, a vertical orientation of the first projector is an upward orientation such that the first light beam spans an upper portion of the cumulative VFOV, and a vertical orientation of the second projector is a downward orientation such that the second light beam spans a lower portion of the cumulative VFOV.

[0021] In some embodiments, a vertical orientation of the first projector is a downward orientation such that the first light beam spans a lower portion of the cumulative VFOV, and a vertical orientation of the second projector is an upward orientation such that the second light beam spans an upper portion of the cumulative VFOV.

[0022] In some embodiments, a vertical orientation of the first tracking camera is an upward orientation, and a vertical orientation of the second tracking camera is a downward orientation.

[0023] In some embodiments, the first scanner device has a first horizontal orientation, and the second scanner device has a second horizontal orientation that is opposite to the first horizontal orientation.

[0024] In some embodiments, the method comprises executing a visual simultaneous localization and mapping (SLAM) algorithm to determine the 3D position and orientation of the system and further determine the 3D coordinates of the object in the environment.

[0025] In some embodiments, the method comprises rotating the first scanner device and the second scanner device simultaneously, stopping rotation of the first and second scanner devices, and causing the first and second tracking cameras to capture the first and second images of the environment while the rotation of the first and second scanner devices is stopped.

[0026] In some embodiments, the method comprises rotating the first scanner device and the second scanner device simultaneously, stopping the first ToF camera and the second5MEl\57949330.vl138178-70720ToF camera to acquire 3D data during a first rotation cycle, and causing the first and second tracking cameras to capture the first and second images of the environment during a second rotation cycle.

[0027] In some embodiments, the method comprises synchronizing acquisition of 3D data by the first ToF camera and the second ToF camera with acquisition of the first images and the second images by the first tracking camera and the second tracking camera, respectively.BRIEF DESCRIPTION OF THE DRAWINGS

[0028] These and other features and advantages of the disclosed coordinate measurement device that performs time of flight and triangulation measurements will be more fully understood by reference to the following detailed description in conjunction with the attached drawings in which like reference numerals refer to like elements throughout the different views.

[0029] FIG. 1 is a block diagram of a scanner device, according to an embodiment of the current disclosure;

[0030] FIG. 2 is a block diagram of another scanner device, according to an embodiment of the current disclosure;

[0031] FIG. 3 A is a scanning system including a single scanner device, according to an embodiment of the current disclosure;

[0032] FIGS. 3B and 3C show other scanning systems each of which including two scanner devices, according to embodiments of the current disclosure;

[0033] FIGS. 4A and 4B show another scanning system having two scanner devices, according to embodiments of the current disclosure;

[0034] FIG. 5 is a flow chart of a method for scanning an environment, according to an embodiment of the current disclosure;

[0035] FIG. 6 is a block diagram of scanning system in communication with a mobile device and / or a cloud system, according to an embodiment of the current disclosure;6MEl\57949330.vl138178-70720

[0036] FIG. 7 is a diagram illustrating functionalities of the scanning system the mobile device and the cloud system of FIG. 6, according to an embodiment of the current disclosure.DETAILED DESCRIPTION

[0037] In the following description of various example embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration various example devices, systems, and environments in which aspects of exemplary embodiments disclosed herein are practiced. It is to be understood that other specific arrangements of parts, example devices, systems, and environments are utilized and structural and functional modifications are made without departing from the scope of the present disclosure. Like-numbered call outs refer to identical components in various instances.

[0038] Embodiments described herein relate to systems and methods for an entry level scanning system, or an entry level scanner for measuring 3D coordinates in an environment. The entry level scanner is a low-cost laser scanner with a relatively superior visualization quality and capability for delivering easy and complete workflows to customers. The scanning system is operated like a laser scanner that is mounted on a support structure, such as a tripod. The scanning structure is used in combination with a mobile phone or other mobile device that provides control, visualization and / or additional processing unit(s). A cloud system, e.g., SPHERE or SPHERE XG reality capture data platform hosted and provided by FARO Technologies, Inc of Lake Mary, Florida, is used for processing, storing and / or visualizing data generated by the scanning system in various embodiments.

[0039] The scanning system described herein includes two area scanners, or two scanner devices, that together cover a VFOV of about 180°. For example, the two scanner devices have a cumulative VFOV between 170° and 190°, between 160° and 200° or between 150° and 210°. Each scanner device has a horizontal field of view (HFOV) between 60° and 120°. To scan a complete 360° HFOV, the two scanner devices are rotated around a vertical axis of the support structure.

[0040] Each scanner device includes a color camera, a light projector, e.g., a laser projector including at least one laser, and at least one ToF camera, e.g., iToF camera(s). The ToF camera(s) and the projector form a 3D acquisition system or a ToF system. The color camera records color and tracking images. Example ToF systems include those ToF systems disclosed in the commonly owned United States Provisional Patent Application No.7MEl\57949330.vl138178-7072063 / 545,027 filed on October 20, 2023, and the commonly owned International Application No. PCT / US / 2024 / 051985, the contents of both applications are incorporated by reference herein in their entirety.

[0041] Embodiments described herein provide a 3D coordinate measurement system, whose position and orientation are tracked using at least one optical tracking camera. The tracking camera(s) allows for the tracking of the positions and / or orientation of the scanning system when placed on a movable support structure or fixture, such as a rotational axis to allow registration of detected or measured points in an environment into a common coordinate frame of reference. The scanning system includes a camera that allows for colorization of a resulting point cloud to provide a more realistic visual appearance and provide for texture details that are not visible in the point cloud. The camera is further used for tracking and for colorization of the point cloud.

[0042] Turning now to FIG. 1, a block diagram of an exemplary scanner device 100 is shown, according to an embodiment of the current disclosure. The scanner device 100 measures distances and angles to various points of an object having a surface and located in the environment of the scanner device 100. The scanner device 100 includes a projector 102 (e.g., a laser projector), a tracking camera 104, a clock or timer 112, a processor 114, and a ToF camera 106. The projector 102 emits a light beam (e.g., a laser beam), in the environment, and the ToF camera 106 receives reflections of the light beam from object surfaces or points in the environment. The ToF camera 106 is sometimes referred to as a 3D camera or a red, green, blue, and depth (RGBD) camera. In some implementations, the ToF camera 106 includes a light detection and ranging (LiDAR) sensor. In some embodiments, the ToF camera 106 includes the clock or timer 112, which sends timing signals to other components of the scanner device 100, e.g., the tracking camera 104 and / or the processor 114. The ToF camera 106 typically includes a lens assembly and a ToF sensor, where the ToF sensor operates in at least one of an iToF mode or a dToF mode.

[0043] Conventional ToF cameras use a homogeneous light to measure distances to detected objects or points thereon. In some implementations, the projector 102 projects a pattern 108 enabling the light to be concentrated, such as in spots 110 that have higher power per unit area than homogeneous light projected over a wide angular region. The relatively higher power per unit area patterns enable more accurate measurements to be made compared to conventional ToF cameras. The higher power per unit area patterns also allow for larger8MEl\57949330.vl138178-70720 measurement distances. In some implementations, the projector 102 uses a diffractive optical element in combination with a light source for generating the light pattern 108. In some implementations, the scanner device 100 includes at least one ToF camera 106.

[0044] In some implementations, the scanner device 100 includes an energy source (such as a battery), at least one light emitting diode (LED) and / or additional infrastructure elements. The scanner device 100 includes at least one processor or processing unit 114. The processor(s) 114 receive and process data acquired by the tracking camera 104 and / or the ToF camera(s) 106. The processor(s) 114 perform or execute multiple tasks related to optical tracking and 3D acquisition. The scanner device 100 includes a memory storing executable instructions, which when executed by the processor(s) 114, cause the processor(s) 114 to perform the tasks related to optical tracking and 3D acquisition described herein. The processor(s) 114 use data received from the tracking camera 104 and the ToF camera(s) 106 to determine 3D coordinates of objects, or respective points thereon, in the environment and / or determine a location and / or orientation(s) of the scanner device 100 in the environment. For example, the processor(s) 114 use the data acquired the tracking camera 104 and the ToF camera(s) 106 to simultaneously build a map of the environment and determine the location of the scanner device 100 within the map, e.g., by employing a SLAM-based approach or algorithm. In some implementations, the scanner service 100 includes a wireless interface to communicate to a mobile phone that runs a user interface application, such as the mobile application STREAM manufactured by FARO Technologies, Inc. of Lake Mary, Florida, to a cloud system and / or other remote devices or systems with corresponding fuunctionalities.

[0045] Referring now to FIG. 2, a block diagram of another scanner device 200 is shown, according to an embodiment of the current disclosure. Similar to the scanner device 100, the scanner device 200, in various embodiments, measures distance(s) and angles to an object in the environment or points thereon. The scanner device 200 includes a projector 202, a colorization camera 204, a clock / timer 212, a processor 214, and a ToF camera 206. The ToF camera 206 is similar to the ToF camera 106 and includes the clock / timer 212 that sends timing signals to other components of the scanner device 200 (e.g., colorization camera 204 and / or processor 212). The ToF camera 206 includes a lens assembly and a ToF sensor, which can operate in an iToF mode or a dToF mode. The projector 202, in similar manner to projector 102, projects a pattern 208 that enables the light to be concentrated, such as in spots 210, which have a higher power per unit area than homogeneous light proj ected over a wide angular region.9MEl\57949330.vl138178-70720As discussed above in relation to FIG. 1, the higher power per unit area patterns 208 enable more accurate measurements and a larger distance range compared to homogeneous light emitted by conventional ToF cameras. In some implementations, the colorization camera 204 is used as a tracking camera.

[0046] In the scanner device 100, the tracking camera 104 is a monochrome camera and the scanner device 100 outputs an uncolorized or monochrome point cloud. To have a colorized point cloud, a color camera 204 (e.g., a high-resolution color camera) is used in the scanner device 200. The color camera 204 is used both for point cloud colorization and tracking in various instances. In some embodiments, the color camera 204 is arranged internally or inside the scanner device 200. In other embodiments, the color camera 204 is an external camera, e.g., a camera of a mobile phone or other mobile device. The color camera 204 has a fixed spatial and temporal connection to the projector 202 and the ToF camera 206. For example, the spatial separations or distances between the projector 202, the color camera 204 and the ToF camera 206 is fixed. The relative orientations of the projector 202, the color camera 204 and the ToF camera 206 is fixed. The fixed temporal connection has the result that time delays or time shifts between acquisition time instances or frames of the color camera 204 and acquisition time instances or frames of the ToF camera 206 are fixed. Since the spatial connections between the projector 202 and cameras 204, 206 are known, and temporal association of frames acquired by the ToF camera 206 and colorization camera 204 are known, the colors from the images acquired by color camera 204 are mapped, e.g., by using ray-tracing to the three-dimension points of the point cloud. It should be appreciated that colorizing the point cloud, e.g., computing a colorized mesh, provides advantages for an improved and more realistic visual appearance of the resolution of texture details that are not visible in the uncolored point cloud. Referring back to FIG. 1, the tracking camera 104 has a fixed spatial and temporal connection to the projector 102 and the ToF camera 106.

[0047] A ToF system can measure the 3D data in a plurality of serially acquired frames. As used herein, a “frame” includes at least one image acquired by the TOF camera 106, 206 with a time stamp indicative of when the image was acquired. For certain embodiments involving a moving scanner, such frames are aligned to define a point cloud in a common coordinate frame of reference. In an embodiment, the scanner is rotated on a support such as a tripod. In some implementations, the entry scanner is handheld and carried by a user. The combination of the tracking camera and the ToF system allows features to be detected in10MEl\57949330.vl138178-70720 multiple frames of the tracking camera when used in combination with 3D data from the ToF system to align the single frames. The tracking camera is, in alternate embodiments, a monochromatic or a color camera.

[0048] Referring now to FIG. 3A, a scanning system 300A is shown, according to an embodiment of the current disclosure. The scanning system 300A includes a scanner device 302 that is coupled to, mounted on or integrated with a support structure 310. The support structure 310 provides an axis of rotation 312 for the scanner device 302. For example, the support structure 310 includes a tripod on which the scanner device 302 is mounted. The system 300A furtherincludes a motor 314 that is couple to or mounted on the support structure 310 to rotate the scanner device 302 along the axis of rotation 312. The scanner device 302 is the same as the scanner device 100 of FIG. 1 or the scanner device 200 of FIG. 2 in various instances. It should be appreciated that allowing the scanner device 302 to rotate along the axis of rotation 312 allows the scanner device 302 to achieve a larger measurement field, e.g., a 360° measurement field. For instance, for a scanner device 302 having a HFOV between 60° and 120°, rotation of the scanner device 302 around the rotational axis 312 allows for a 360 degrees scan of the environment along the VFOV of the scanning device 302. In other words, by rotating the scanner device 302, the respective VFOV is rotated horizontally to span 360°. The measurement zone / field is viewed as the field of view in a first direction, e.g., VFOV, that is rotated 360 degrees in a transverse direction, e.g., horizontally. The rotation of the scanning device 302 provides a band of measurement data of the surrounding environment.

[0049] The VFOV of the tracking / color camera 104, 204 and / or the ToF camera(s) 106, 206 is between 60° and 120°. In general, the VFOV of the tracking / color camera 104, 204 and / or the ToF camera(s) 106 is less than 180°. Ultra-wide cameras may be used but are typically too expensive to include in a low-cost scanning solution. In order to fully scan the surrounding environment, using a single scanning device 302 implies longer scanning time as well as more complex mechanical systems that allow horizontal and vertical rotations.

[0050] Referring now to FIGS. 3B and 3C, other scanning systems 300B and 300C are shown, according to certain embodiments of the current disclosure. Each of the scanning systems 300B and 300C include a first scanner device 302a and a second scanner device 302a. Each of the first scanner device 302a and the second scanner device 302b are similar to or the same as the scanner device 100 of FIG. 1 or the scanner device 200 of FIG. 2. Each of the scanner devices 302a and 302b are coupled to (e.g., mounted on, fixated to or integrated with)11MEl\57949330.vl138178-70720 the support structure 310, and include (i) a respective projector 102, 202 that emits a light beam or light pattern into the environment, (ii) at least one respective ToF cameras 106, 206 that detect light reflected from objects in the environment, and (iii) a respective tracking camera 104, 204. The respective projector 102, 202 and the ToF camera(s) 106, 206 forms a 3D acquisition system or ToF system. The respective tracking camera 104, 204 has a fixed temporal and spatial connection to the respective ToF system, e.g., the respective projector 102, 202 and the respective ToF camera(s) 106, 206. The ToF camera(s) 106, 206 of the scanner device 302a has a first vertical orientation and the ToF camera(s) 106, 206 of the scanner device 302b has a second vertical orientation such that the ToF cameras 106, 206 of both scanner devices 302a and 302b have a cumulative VFOV of about 180 degrees.

[0051] In some implementations, the cumulative VFOV of the ToF cameras 106, 206 of both scanner devices 302a and 302b is greater than or equal to 180 degrees. In various instances, the cumulative VFOV of the ToF cameras 106, 206 of both scanner devices 302a and 302b is between 170° and 190°, between 160° and 200° or between 150° and 210°. In some implementations, the tracking camera 104, 204 of the scanner device 302a and the tracking camera 104, 204 of the scanner device 302b have different vertical orientations such that the cumulative VFOV of the tracking cameras of both scanner devices 302a and 302b is about 180 degrees ( e.g., greater than or equal to 180° , between 170° and 190°, between 160° and 200° or between 150° and 210°).

[0052] As depicted in FIGS. 3B and 3C, the vertical orientation of the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302a is an upward orientation (e.g. away from the ground surface on which the systems 300A, 300B is placed) such that the VFOV of the ToF camera(s) and / or the tracking camera 104, 204 of the scanner device 302a covers an upper portion of the cumulative VFOV, while the vertical orientation of the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b is a downward orientation (e.g. towards the ground surface that the systems 300A, 300B are placed) such that the VFOV of the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b covers an lower portion of the cumulative VFOV. In other words, the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302a is oriented or directed at least partially upward to scan an upper half space of the environment, and the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b is oriented or directed at least partially downward to scan a lower half12MEl\57949330.vl138178-70720 space of the environment. For example, the plane 308 is viewed as defining the boundary of the upper half space (where, for example, objects 304 are located) and the lower half space (where objects 306 are located). The ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302a is oriented or directed at least partially upward to scan objects 304 in the upper half space of the environment, and the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b is oriented or directed at least partially downward to scan objects 306 in the lower half space of the environment. It should be appreciated that the use of the terms “upward” and “downward” are not intended to be limiting and the systems 300 A, 300B may be used in any orientation.

[0053] In some implementations, the VFOV of the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302a and the VFOV of ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b overlap. For example, the VFOV of the ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302a include a portion of the lower half space, and the VFOV of ToF camera(s) 106, 206 and / or the tracking camera 104, 204 of the scanner device 302b include a portion of the upper half space.

[0054] In some implementations, the vertical orientation of the projector 102, 202 of the scanner device 302a is an upward orientation such that the respective light beam spans an upper portion of the cumulative VFOV (i.e., the upper half space), and the vertical orientation of the projector 102, 202 of the scanner device 302b is a downward orientation such that the respective light beam spans a lower portion of the cumulative VFOV (i.e., the lower half space). In some implementations, the vertical orientation of the projector 102, 202 of the scanner device 302a is a downward orientation such that the respective light beam spans a lower portion of the cumulative VFOV (i.e., the lower half space), and the vertical orientation of the projector 102, 202 of the scanner device 302b is an upward orientation such that the respective light beam spans the upper portion of the cumulative VFOV (i.e., the upper half space). In general, the vertical orientation of projector 102, 202 is similar to or different from the vertical orientation of the tracking camera 104, 204 and / or ToF camera(s) 106, 206 of the same scanner device 302a or 302b. For example, while the tracking camera 104, 204 and / or ToF camera(s) 106, 206 of the scanner device 302a is oriented or directed to scan the upper half space, the projector 102, 202 of the same scanner device 302a is oriented or directed to illuminate the lower half space.13MEl\57949330.vl138178-70720

[0055] In FIG 3B, the scanner devices 302a and 302b have a common (or similar) horizontal orientation, e.g., such that the HFOVs of both scanner devices 302a and 302b match or substantially match (e.g., within less than 5 degrees or 10 degrees error). In FIG. 3C, for example, the second scanner device has a second horizontal orientation that is opposite to the first horizontal orientation. In some implementations, the scanner device 302a (or the corresponding tracking camera 104, 204 and the corresponding ToF camera(s) 106, 206) have a first horizontal orientation and the scanner device 302b (or the corresponding tracking camera 104, 204 and the corresponding ToF camera(s) 106, 206) has a second horizontal orientation opposite to the first horizontal orientation, as depicted in the example shown in FIG. 3C. As discussed in further detail below, the configuration of FIG. 3C has a technical advantage that allows for faster or more frequent correction(s) of offsets or drifts over time in collected scan data.

[0056] Each of the systems 300B and 300C include a motor 314 that rotates the scanner devices 302a and 302b simultaneously about the axis 312. As the scanner devices 302a and 302b rotate about the axis 312, the respective tracking cameras 104, 204 and the respective ToF cameras 106, 206 fully scan the environment. In some implementations, the systems 300A, 300B and / or 300C include a housing that contains the scanner device(s) 302, 302a and / or 302b as well as various components thereof. In some implementations, each of the systems 300B and 300C include more than two scanner devices. In some implementations, the support structure 310 is moved around and within the environment, e.g., in addition to its rotational movements around the axis 312.

[0057] Referring now to FIGS. 4A and 4B, another scanning system 400 is shown, according to embodiments of the current disclosure. The scanning system 400 includes projectors 402a and 402b, tracking cameras 404a and 404b as well as ToF cameras 406a and 406b that are coupled to the support structure 410. The projector 402a, the tracking camera 404a and the ToF camera(s) 406a form a first scanner device while projector 402b, the tracking camera 404b and the ToF camera(s) 406b form a second scanner device. The support structure 410 include a housing that contains or hosts the first and second scanner devices. In some implementations, the support structure 410 includes a first portion 408a hosting the projector 402a, the tracking camera 404a and the ToF camera(s) 406a, and a second portion hosting the projector 402b, the tracking camera 404b and the ToF camera(s) 406b. The first and second portions 408a and 408b have different orientations or inclination angles in various14MEl\57949330.vl138178-70720 implementations. As depicted in FIG. 4B, the projector 402a is oriented or directed to illuminate the upper half space of the environment while the tracking camera 404a and the ToF camera(s) 406a is oriented or directed to scan objects mainly in the lower half space. The projector 402b is oriented or directed to illuminate the lower half space of the environment while the tracking camera 404b and the ToF camera(s) 406b is oriented or directed to scan objects mainly in the upper half space. In some implementations, the projector 402a is oriented or directed to illuminate the lower half space of the environment while the tracking camera 404a and the ToF camera(s) 406a is oriented or directed to scan objects mainly in the upper half space, and the projector 402b is oriented or directed to illuminate the upper half space of the environment while the tracking camera 404b and the ToF camera(s) 406b is oriented or directed to scan objects mainly in the lower half space. As discussed above, the use of the term “upper” and “lower” is not intended to imply a particular orientation of the system 400 and the system 400 may be used in any orientation.

[0058] The scanning system 400 includes a motor 414 that rotates the portions 408a and 408b or the first and second scanner devices about a rotation axis 412. In some implementations, the scanning system 400 or the support structure 410 includes a handle 416 for holding and moving the support structure 410 around the environment.

[0059] Referring now to FIG. 5, a flow chart of a method 500 for scanning an environment is shown, according to an example embodiment of the current disclosure. The method 500 is performed by or associated with any of the scanning systems described herein. In brief overview, the method 500 includes acquiring (by a first scanner device 302a coupled to a support structure 310, 410 and including a first projector 102, 202, 402a, a first tracking camera 104, 204, 404a, and at least one first ToF cameras 106, 206, 406a) first image data and first 3D data of the environment (operation 502). The method then acquires (by a second scanner device 302b coupled to the support structure 310, 410 including a second projector 102, 202, 402b, a second tracking camera 104, 204, 404b, and at least one second ToF cameras 106, 206, 406b) second image data and second 3D data of the environment (operation 504). Next, the method 500 then determines a 3D position and orientation of at least one of the first scanner device and the second scanner device using at least one of the first image data or the second image data and at least one of the first 3D data or the second 3D data (operation 506). Finally, 3D coordinates of an object in the environment are determined based at least in part on one or more of the first 3D data and / or the second 3D data (operation 508).15MEl\57949330.vl138178-70720

[0060] The first and second image data are acquired by the first tracking camera and the second tracking camera, respectively. The first and second 3D data are acquired by the first ToF camera(s) and the second ToF camera(s), respectively. The 3D data is acquired using an iToF scanner using structured light, triangulation or a combination thereof. For the iToF scanner measurements, at least one iToF camera is used in combination with a projector. The iToF scanner depth measurement is based on measuring the relative phase of a modulated laser with respect to the modulated camera acquisition. The iToF scanner measurement(s), in combination with a high intensity structured light pattern, allows for improved measuring distances at ranges of several tens of meters. The iToF scanner depth measurement is unambiguous within a certain depth interval. Typically, multiple light frequencies are used to resolve ambiguities. However, the use of multiple light frequencies implies longer acquisition time since more camera exposures at the separate frequencies are required.

[0061] The other option for acquiring the 3D data is to use triangulation to resolve any ambiguities of the iToF measurement system. Each element of the structured light pattern detected by the iToF camera(s) is used to triangulate a 3D position of an object or a point thereof in the environment. For relatively short distances, the triangulation allows for higher accuracy and lower noise than the iToF measurement. For relatively large distances, the triangulation measurement is less accurate but allows for resolving the ambiguities of the iToF measurement.

[0062] Triangulation is described in detail in the commonly owned United States Provisional Patent Application No. 63 / 545,027 filed on October 20, 2023, and the commonly owned International Application No. PCT / US / 2024 / 051985, the contents of which are incorporated by reference herein in their entirety. It is to be noted that triangulation and / or the use of multiple frequencies can also be used with the dToF mode. The projector 102, 202, 402 is spaced apart from the ToF camera 106, 206, 406 of the same scanner device by a first distance. Also, in the case where a scanner device includes two ToF cameras, the ToF cameras are spaced apart by a second distance. The processor 114, 214 employs triangulation to determine a distance from a scanner device to the surface of an object reflecting the light pattern emitted from the corresponding projector 102, 202 using the first distance or the second distance. The processor(s) 114, 214 typically use the ToF distance measurement from the ToF camera 106, 206 when applying triangulation. In some implementations, the processor(s) 114, 214 average the ToF distance measurement and the distance measurement obtained by triangulation to16MEl\57949330.vl138178-70720 obtain a useful value. The processor(s) 114, 214 compare the triangulation distance measurement with the ToF distance measurement to confirm that the measurements are within a predetermined threshold, and determine, based on the comparison to an average, whether the camera calibration parameters (e.g. distortion) is within a desired range / threshold.

[0063] The depth measurement with an iToF camera relies on multiple phase frames that are typically recorded sequentially. Since the scanning system is sometimes a moving system, the scanning system and / or the processor(s) 114, 214, take the details of the movement into account when computing the 3D data in such instances. One option is to track the position and orientation of the scanning system (or the respective scanner devices) and use different pixels at different points in time for the depth measurement of one position in space. Tracking allows for a good position and orientation measurement of the scanning system and / or the respective scanner devices. The depth measurement from the triangulation is used as a reliable piece of 3D information that is independent of the movement of the scanning system and is acquired using a single frame.

[0064] In low(er)-cost scanning systems, the use of highly accurate position encoders that measure the exact rotation angle about the rotation axis 312, 412 is typically prohibitive due to higher component costs. As such, tracking is used to determine the position(s) and / or orientation(s) of the scanning system 300 A, 300B, 300C, 400 in a more economical manner, without sacrificing accuracy. In addition, it is procedurally advantageous to track the pose of the scanning devices during the movement between scan positions. Visual tracking, or visual SLAM, supported by 3D data offers a low-cost solution that can deliver high-accuracy measurements of the position and orientation of the scanning system or the respective scanner devices. The processor(s) 114, 214 employ a SLAM-based (or visual SLAM-based) approach or algorithm to determine the 3D position and orientation of the scanning system and / or the 3D coordinates of objects in the environment. The processor(s) 114, 214 use the images acquired by the color camera(s) during the movement of the scanning system, identify image features in the acquired images, and track movement or displacement of the features across the acquired images. The processor(s) 114, 214 use the corresponding 3D data to determine or estimate movement or rotation of the scanning system. For example, the processor(s) 114, 214 determine a rotation angle of the scanning system 300 A, 300B, 300C, 400 based on detected displacement of image features across consecutive frames and 3D data corresponding to the same consecutive frames.17MEl\57949330.vl138178-70720

[0065] In some implementations, the same camera is used for tracking and for colorization / visualization. The processor(s) 114, 214 downsamples the high-resolution images of the tracking camera to limit or reduce the performance requirements of data transfer and processing. The scanning system 300A, 300B, 300C, 400 record tracking images at the same time, or substantially at the same time, as the 3D data. The ToF camera(s) and the tracking camera in each scanner device or in both scanner devices are synchronized in time. For example, timing information generated by the clock / timer 112, 212 is shared among or between the tracking camera and ToF camera(s) of the same or both scanner devices of the scanning system.

[0066] Multiple scans with a spatial separation of several meters are recorded by the scanning system. The processor(s) 114, 214 automatically registers the scans using the information from the system or device tracking. Different algorithms are used in various implementations for registration, such as texture-based approaches, cloud-to-cloud, or targetbased registrations.

[0067] It should be noted that the determining steps 506 and 508 of FIG. 5 involve correcting an offset or a drift in scan positions of objects or points thereof in the environment. Offset correction is described in detail the commonly owned U.S. Patent Application No. 18 / 469,258, the content of which is incorporated by reference herein in its entirety. As described in FIGS 9 and 10 thereof, the offset correction is performed once a complete motion loop of 360 degrees is reached by the scanning system or the scanner device. However, when using the configuration of FIG. 3C, for example, where the scanner devices 302a and 302b have opposite horizontal directions or orientations, the processor 114, 214 can perform offset correction after only a 180 degrees rotation. For example, if the VFOVs of the scanner devices 302a and 302b (or the respective ToF cameras and tracking cameras) overlap, the HFOV of the scanner device 302a after a rotation by 180 degrees matches the HFOV of the scanner device 302b at its starting point. In the case of an overlap between the VFOV of both scanner device 302a and 302b, the processor(s) 114, 214 advantageously uses the overlap in image data and / or 3D data to detect or determine that the scanning system rotated by 180 degrees, and corrects any offset or drift in the scan positions. As such, the processor(s) 114, 214 performs the offset correction after 180 degrees rotation instead of after 360 degrees rotation. As a result, the offset correction is performed twice, instead of once, within a rotation of 360 degrees. The offset correction after rotating by 180 degrees allows for more frequent corrections, thereby resulting18MEl\57949330.vl138178-70720 in more accurate determination of the position and / or orientation(s) of scanning system, along with positions and / or mapping of objects in the environment.

[0068] Color cameras further allow for a high-quality visualization. For example, the scanning system 300B, 300C, 400 has two color cameras with a resolution of about 20 megapixels (MP). One color camera is used for scanning the upper half space and the other for scanning the lower half space. Different options for the color acquisition mode are sometimes used. According to a first mode, the scanning system 300B, 300C, 400 and / or the processor(s) 114, 214 stops the rotation of the scanner devices and causes the cameras 104, 204, 404 to record the color images when the scanner devices 302a, 302b are not moving. This mode has the advantage that for longer exposure times and motion, blur does not degrade the quality of the acquired image. Also, for high dynamic range (HDR) acquisition, color capture without movement is easier to perform. When stopping the rotation for the color acquisition, either the rotation is stopped in between parts of the 3D acquisition or a separate rotation is done for the color acquisition.

[0069] In some implementations, color images for visualization are recorded during the rotation movement. The processor(s) 114, 214 then employ deconvolution techniques to compensate for motion blur. In some implementations, artificial intelligence (Al) or other techniques are additionally used to compute super-resolution images from multiple overlapping images.

[0070] Referring now to FIG. 6, a block diagram of scanning system 600 in communication with a mobile device 610 and / or a cloud system 612 is shown, according to an embodiment of the current disclosure. The scanning system 600 is similar to any of the scanning systems 300B, 300C, 400 described above. The scanning system 600 includes a first 3D acquisition system 601a, a second 3D acquisition system 601b, a 2D sensor (or color imaging system) 603, a circuit board 605 and an infrastructure system 607. The infrastructure system 607 includes, in various embodiments, various of the components described herein later below. The circuit board is communicatively coupled to the mobile device 610 and / or the cloud system 612. The first 3D acquisition system 601a includes a respective laser projector and two respective ToF cameras, while the second 3D acquisition system 601b includes a respective laser projector and two respective ToF cameras. The color imaging system or 2D sensor 603 includes two color images that are used for tracking and colorization. The circuit board 605 includes a control and processing unit, a USB interface, a Wi-Fi interface, an embedded19MEl\57949330.vl138178-70720 multimedia card (eMMC), a memory or storage device, and / or a speaker, among other components. The infrastructure system 607 includes the support structure, a battery, a cooling system, environment sensor(s), an inertial measurement unit (MU), buttons, LEDs, a cooling system, a power supply unit, and / or a PAN axis, among others.

[0071] The scanning system 600 performs a significant amount of image-heavy processing. Part of the processing, such as image raw data processing or tracking, is performed during scanning, whereas part of the processing is performed as a post-processing step in some instances. Where processing resources are limited in the scanning system 600, only some tasks are performed live (such as tasks related to data size reduction and preview generation). Three components are used for the processing of the data including the scanning system 600, which include (i) a JETSON processing board manufactured by NVDIA Corporation of Santa Clara, California, (ii) the mobile device 610 which can run or execute an application like STREAM with enhanced processing capabilities, and the cloud system 612, e.g., SPHERE. In particular, a good integration with SPHERE or similar software allows for the generation of the final workflows and deliverables.

[0072] FIG. 7 is a diagram illustrating functionalities of the scanning system 600, the mobile device 610 and the cloud system 612 of FIG. 6, according to embodiments of the current disclosure. In particular, FIG. 7 illustrates an example distribution of tasks among the scanning system 600, the mobile device 610 and the cloud system 612 of FIG. 6.

[0073] A mobile phone (or tablet) 610 is used as the user interface and control unit in some implementations. The mobile device 610 runs STREAM (or similar software), visualizes the recorded data, supports processing, and serves as an interface to the cloud system 612, where data is further processed and stored. In some implementations, complete on-site workflows are offered to the customer, where the application running on the mobile device 610, e.g., STREAM, generates final deliverables, such as a rendered floorplan.

[0074] The cloud system or platform 612, e.g., SPHERE (or similar), is the central platform where data from the scanning system 600 is stored, enhanced, processed, combined with other data, and so on. The cloud system or platform 612 provide advantages to users by allowing access to a number of processing applications, including several final deliverables that are generated from the acquired scan and image data.20MEl\57949330.vl138178-70720

[0075] The term “about” is intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ± 8% or 5%, or 2% of a given value.

[0076] Additionally, the term "exemplary" is used herein to mean "serving as an example, instance or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "at least one" are understood to include any integer number greater than or equal to one, i.e. one, two, three, four, etc. The terms "a plurality" are understood to include any integer number greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include an indirect "connection" and a direct "connection." It should also be noted that the terms “first”, “second”, “third”, “upper”, “lower”, and the like are used herein to modify various elements. These modifiers do not imply a spatial, sequential, or hierarchical order to the modified elements unless specifically stated.

[0077] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and / or groups thereof.

[0078] For the sake of brevity, conventional techniques related to making and using aspects of the invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and / or process details.

[0079] The present invention may be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable21MEl\57949330.vl138178-70720 program instructions thereon for causing a processor to carry out aspects of the present invention.

[0080] The computer readable storage medium is, in various instances, a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiberoptic cable), or electrical signals transmitted through a wire.

[0081] Computer readable program instructions described herein are typically downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0082] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, statesetting data, configuration data for integrated circuitry, or either source code or object code22MEl\57949330.vl138178-70720 written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user’s computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instruction by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0083] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, are typically implemented by computer readable program instructions.

[0084] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.23MEl\57949330.vl138178-70720

[0085] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0086] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, is implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0087] While the disclosure is provided in detail in connection with only a limited number of embodiments, it should be readily understood that the disclosure is not limited to such disclosed embodiments. Rather, the disclosure is modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the disclosure. Additionally, while various embodiments of the disclosure have been described, it is to be understood that the exemplary embodiment(s) include only some of the described exemplary aspects. Accordingly, the disclosure is not to be seen as limited by the foregoing description but is only limited by the scope of the appended claims.24MEl\57949330.vl

Claims

138178-70720CLAIMSWhat is claimed:

1. A system comprising: a support structure; a first scanner device operably coupled to the support structure and including; a first time-of-flight (ToF) system including a first projector that emits a first light beam into an environment and a ToF camera that detects light reflected from objects in the environment; and a first tracking camera having a fixed connection to the first ToF system, and captures first images of the environment; a second scanner device operably coupled to the support structure and including; a second ToF system including a second projector that emits a second light beam into the environment and a second ToF camera that detects light reflected from the objects in the environment; and a second tracking camera having a fixed connection to the second ToF system, and captures second images of the environment; and at least one processor in communication with the first ToF system, the first tracking camera, the second ToF system and the second tracking camera, where the at least one processor: determines a three-dimensional (3D) position and orientation of the system using at least one of the first images and the second images of the environment along with output signals from at least one of the first ToF camera and the second ToF camera; and determines 3D coordinates of an object in the environment based at least in part on the output signals from at least one of the first ToF camera and the second ToF camera, wherein the first ToF camera has a first vertical orientation and the second ToF camera has a second vertical orientation such that the first ToF camera and the second ToF camera have a cumulative vertical field of view (VFOV) of substantially 180 degrees.

2. The system of claim 1, wherein the first vertical orientation is an upward orientation such that a first VFOV of the first ToF camera includes an upper portion of the cumulative VFOV, and25MEl\57949330.vl138178-70720 wherein the second vertical orientation is a downward orientation such that a second VFOV of the second ToF camera includes a lower portion of the cumulative VFOV.

3. The system of claim 1, wherein a vertical orientation of the first projector is an upward orientation such that the first light beam spans an upper portion of the cumulative VFOV, and a vertical orientation of the second projector is a downward orientation such that the second light beam spans a lower portion of the cumulative VFOV.

4. The system of claim 1 , wherein a vertical orientation of the first proj ector is a downward orientation such that the first light beam spans a lower portion of the cumulative VFOV, and a vertical orientation of the second projector is an upward orientation such that the second light beam spans an upper portion of the cumulative VFOV.

5. The system of claim 1, wherein a vertical orientation of the first tracking camera is an upward orientation, and a vertical orientation of the second tracking camera is a downward orientation.

6. The system of claim 1 , wherein the first scanner device has a first horizontal orientation, and the second scanner device has a second horizontal orientation that is opposite to the first horizontal orientation.

7. The system of claim 1, wherein the at least one processor uses a visual simultaneous localization and mapping (SLAM) algorithm to determine the 3D position and orientation of the system and the 3D coordinates of the object in the environment.

8. The system of claim 1, wherein the support structure includes a motor that rotates the first scanner device and the second scanner device simultaneously, and the at least one processor: causes the motor to regularly stop rotation of the first and second scanner devices; and causes the first and second tracking cameras to capture the first and second images of the environment while the rotation of the first and second scanner devices is stopped.26MEl\57949330.vl138178-707209. The system of claim 1, wherein the support structure includes a motor that rotates the first scanner device and the second scanner device simultaneously, and the at least one processor: causes the first ToF camera and the second ToF camera to acquire 3D data during a first rotation cycle; and causing the first tracking camera and the second tracking camera to capture the first image and second image of the environment during a second rotation cycle.

10. The system of claim 1, wherein the at least one processor synchronizes acquisition of 3D data by the first ToF camera and the second ToF camera with acquisition of the first images by the first tracking camera and acquisition of the second images by the second tracking camera.

11. A method comprising: acquiring first image data and first 3 -dimensional (3D) data of an environment by a first scanner device coupled to a support structure, the first scanner device including a first projector, a first time-of-flight (ToF) camera and a first tracking camera; acquiring second image data and second 3D data of the environment by a second scanner device coupled to the support structure, the second scanner device including a second projector, a second time-of-flight (ToF) camera and a second tracking camera; determining a position and orientation of at least one of the first scanner device and the second scanner device using at least one of the first image data and the second image data along with at least one of the first 3D data and the second 3D data; and determining 3D coordinates of an object in the environment based at least in part on at least one of the first 3D data and the second 3D data, wherein the first ToF camera has a first vertical orientation and the second ToF camera has a second vertical orientation such that the first ToF camera and the second ToF camera have a cumulative vertical field of view (VFOV) of substantially 180 degrees.

12. The method of claim 11, wherein the first vertical orientation is an upward orientation such that a first VFOV of the first ToF camera includes an upper portion of the cumulative VFOV, and the second vertical orientation is a downward orientation such that a second VFOV of the second ToF camera includes a lower portion of the cumulative VFOV.27MEl\57949330.vl138178-7072013. The method of claim 11 , wherein a vertical orientation of the first proj ector i s an upward orientation such that the first light beam spans an upper portion of the cumulative VFOV, and a vertical orientation of the second projector is a downward orientation such that the second light beam spans a lower portion of the cumulative VFOV.

14. The method of claim 11, wherein a vertical orientation of the first projector is a downward orientation such that the first light beam spans a lower portion of the cumulative VFOV, and a vertical orientation of the second projector is an upward orientation such that the second light beam spans an upper portion of the cumulative VFOV.

15. The method of claim 11, wherein a vertical orientation of the first tracking camera is an upward orientation, and a vertical orientation of the second tracking camera is a downward orientation.

16. The method of claim 11, wherein the first scanner device has a first horizontal orientation, and the second scanner device has a second horizontal orientation that is opposed to the first horizontal orientation.

17. The method of claim 11, further comprising: executing visual simultaneous localization and mapping (SLAM) algorithms to determine the position and orientation of at least one of the first scanner device and the second scanner device and further determine the 3D coordinates of the object in the environment.

18. The method of claim 11, further comprising: rotating the first scanner device and the second scanner device simultaneously; stopping rotation of the first scanner and second scanner devices; and causing the first tracking camera and second tracking camera to capture the first and second images of the environment while the rotation of the first scanner device and second scanner device is stopped.

19. The method of claim 11, further comprising: rotating the first scanner device and the second scanner device simultaneously;28MEl\57949330.vl138178-70720 causing the first ToF camera and the second ToF camera to acquire 3D data during a first rotation cycle; and causing the first tracking camera to capture the first image data and the second tracking camera to capture the second image data of the environment during a second rotation cycle.

20. The method of claim 11, further comprising: synchronizing acquisition of the first 3D data with acquisition of the first image data; and synchronizing acquisition of the second 3D data with acquisition of the second image data.29MEl\57949330.vl

Images