Method and apparatus for optimizing image acquisition of an object subjected to an illumination pattern - Patents.com

Abstract

A method, apparatus, and computer-readable medium configured to determine parameters for image acquisition are provided. One or more image sensors (106A, 106B) are each arranged to project a moving pattern onto a scene and acquire a set of images of the scene projected by a projector (104) having a set of adjustable projector parameters, each image sensor having a set of adjustable imaging parameters. The set of adjustable projector parameters and the set of adjustable imaging parameters are determined based on one or more sets of constraints to reduce noise in 3D data generated based on the set of images.

Description

[Technical field]

[0001] The technology described herein generally relates to methods and apparatus for optimizing machine vision image acquisition, including determining optimal parameters for acquisition of images of objects subjected to randomized illumination patterns. [Background technology]

[0002] Advanced machine vision systems and their underlying software are increasingly being used in a variety of manufacturing and quality control processes. Machine vision enables faster, more accurate and repeatable results in the production of both mass produced and custom products. A typical machine vision system includes one or more cameras aimed at the area of ​​interest, an illumination source to direct appropriate illumination at the area of ​​interest, a frame grabber / image processing element to acquire and transmit the images, a computer or on-board processing device to run machine vision software applications to operate on the acquired images, and a user interface for interaction.

[0003] One form of 3D vision system is based on stereo cameras, which employ at least two cameras aligned side-by-side with a camera separation of one to several inches on a baseline. Stereo vision-based systems are typically based on epipolar geometry and image rectification. They can use correlation-based methods or be combined with relaxation techniques to find correspondences in the rectified images from two or more cameras. However, traditional stereo vision systems are limited in their ability to rapidly produce accurate and fast three-dimensional data reconstructions of objects and / or scenes due to the computational effort required. Summary of the Invention

[0004] In accordance with the disclosed subject matter, apparatus, systems, and methods are provided for optimizing machine vision system parameters (e.g., including camera and / or projector / illuminator parameters) for acquiring images for 3D processing. The system can be configured to optimize various parameters to determine the minimum frame time used to acquire a complete sequence of images while imposing various constraints. In some embodiments, these constraints are designed such that the parameters are set such that the system acquires images with sufficient information for 3D reconstruction, conforms to mechanical limitations of the system, and / or enforces illumination power limitations (e.g., to prevent overheating, damage to the system, injury to an operator, etc.).

[0005] Some aspects relate to a system for determining parameters for image capture. The system includes one or more image sensors, each image sensor arranged to capture a set of images of a scene, each image sensor having a set of adjustable imaging parameters. The system includes a projector configured to project a moving pattern onto the scene, the projector having a set of adjustable projector parameters. The system includes a processor in communication with the one or more image sensors and the projector, the processor configured to determine the set of adjustable projector parameters and the set of adjustable imaging parameters based on a set of one or more constraints to reduce noise in 3D data generated based on the set of images.

[0006] In some examples, the set of adjustable imaging parameters includes a frame time indicating a duration of each frame for each image in the set of images, the set of adjustable projector parameters includes a speed at which the pattern moves over the scene, and determining the set of adjustable projector parameters and the set of adjustable imaging parameters includes determining a speed of movement that minimizes the frame time. The processor can be further configured to adjust the frame time based on a minimum time required to read out a field of view of the one or more image sensors. The processor can be further configured to adjust the frame time based on an exposure time at maximum exposure of the projector's illumination device. The processor can be further configured to adjust the frame time based on an exposure time of the one or more image sensors and the set of illumination parameters. The set of illumination parameters can include one or more of an illuminator output during the exposure, a maximum average output of the illuminator, a maximum illuminator output, a maximum average output of the illuminator when the peak illuminator output is set to full output, or a combination thereof.

[0007] In some examples, determining the set of adjustable projector parameters and the set of adjustable imaging parameters can include determining an initial set of operating parameters for the set of adjustable projector parameters and the set of adjustable imaging parameters, determining one or more parameters of the initial set of parameters to violate a thermal constraint of the one or more constraints, and adjusting the one or more parameters to comply with the thermal constraint. Adjusting the one or more parameters to comply with the thermal constraint can include determining a user preference operation and adjusting the one or more parameters to comply with the thermal constraint based on the user preference operation. The user preference operation can include a low latency setting, and adjusting the one or more parameters can include adding a delay between consecutive image sequences. The user preference operation can include a no-blink setting, and adjusting the one or more parameters can include increasing a frame time of one or more image sensors, slowing down a projector speed to slow down a speed at which a pattern moves across a scene, or a combination thereof.

[0008] Certain aspects relate to a computerized method for determining parameters for image capture, the method being executed by a processor configured to store a set of one or more constraints and determine, based on the set of one or more constraints, a set of adjustable imaging parameters for each of one or more image sensors in communication with the processor, each image sensor being positioned to capture a set of images of a scene, and a set of adjustable projector parameters for a projector configured to project a moving pattern onto the scene, where the set of adjustable projector parameters and the set of adjustable imaging parameters reduce noise in 3D data generated based on the set of images.

[0009] In some examples, the set of adjustable imaging parameters includes a frame time indicating a duration of each frame for each image in the set of images, the set of adjustable projector parameters includes a speed at which the pattern moves over the scene, and determining the set of adjustable projector parameters and the set of adjustable imaging parameters includes determining a speed of movement that minimizes the frame time. The method includes adjusting the frame time based on a minimum time required to read out a field of view of the one or more image sensors. The method can include adjusting the frame time based on a combination of a minimum inter-frame distance and a maximum motor speed. The method can include adjusting the frame time based on an exposure time at maximum exposure of an illuminator of the projector.

[0010] In some examples, determining the set of adjustable projector parameters and the set of adjustable imaging parameters may include determining an initial set of operating parameters for the set of adjustable projector parameters and the set of adjustable imaging parameters, determining that one or more parameters in the initial set of parameters violate a thermal constraint of the one or more constraints, and adjusting the one or more parameters to comply with the thermal constraint.

[0011] Some aspects relate to at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform operations of storing a set of one or more constraints; and determining, based on the set of one or more constraints, a set of adjustable imaging parameters for each of one or more image sensors in communication with the processor, each image sensor being positioned to acquire a set of images of a scene, and a set of adjustable projector parameters for a projector configured to project a moving pattern onto the scene, where the set of adjustable projector parameters and the set of adjustable imaging parameters reduce noise in 3D data generated based on the set of images.

[0012] In some examples, the set of adjustable imaging parameters includes a frame time indicating a duration of each frame for each image in the set of images, the set of adjustable projector parameters includes a speed at which the pattern moves over the scene, and determining the set of adjustable projector parameters and the set of adjustable imaging parameters includes determining an operating speed that minimizes the frame time. Determining the set of adjustable projector parameters and the set of adjustable imaging parameters can include determining an initial set of operating parameters for the set of adjustable projector parameters and the set of adjustable imaging parameters, determining one or more parameters in the initial set of parameters violate a thermal constraint of the one or more constraints, and adjusting the one or more parameters to comply with the thermal constraint.

[0013] The foregoing has outlined, rather broadly, the features of the disclosed subject matter so that they may be better understood in the detailed description that follows, and so that the present contribution to the art may be better appreciated. There are, of course, additional features of the disclosed subject matter that will be described below and which form the subject matter of the claims appended hereto. It is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

[0014] In the drawings, each identical or nearly identical component that is shown in various figures is represented by the same reference numeral. For clarity, not every component is shown in every drawing. The drawings are not necessarily to scale, emphasis instead being placed on illustrating various aspects of the techniques and apparatus described herein. [Brief description of the drawings]

[0015] [Figure 1] 1 illustrates an exemplary embodiment in which one projector and two cameras are arranged to capture images of a scene in a manner suitable for generating stereo image correspondence, according to some embodiments.

[0016] [Diagram 2] 1 illustrates example projector components according to some embodiments.

[0017] [Diagram 3] 1 illustrates an exemplary pair of stereo images corresponding to a series of projected light patterns, according to some embodiments.

[0018] [Figure 4] 1 illustrates an exemplary pair of stereo images of a scene, according to some embodiments.

[0019] [Diagram 5] 1 illustrates an exemplary pair of stereo temporal image sequences corresponding to a series of light patterns projected onto a scene, according to some embodiments.

[0020] [Figure 6A] 1 illustrates an exemplary method for determining operational parameters of a vision system, according to some embodiments. [Figure 6B] 1 illustrates an exemplary method for determining operational parameters of a vision system, according to some embodiments.

[0021] [Figure 7] 1 illustrates example images acquired using different operating parameters, according to some embodiments. [Figure 8] 1 illustrates example images acquired using different operating parameters, according to some embodiments. [Figure 9] 1 illustrates example images acquired using different operating parameters, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The techniques described herein can be used to analyze the operating parameters of a stereo vision system, such as parameters related to the camera (e.g., frame time, exposure time, etc.) and / or pattern projector (e.g., illuminator parameters, pattern speed, etc.). The inventors have determined that the configurable parameters for a vision system can generate complex considerations for finding optimal settings for the system. The inventors have further determined that in a machine vision system, some parameters are user configurable while others are not user adjustable, which can make it difficult for a user to control the available parameters to achieve an image with sufficient noise levels. The inventors have further determined that thermal considerations and / or other considerations designed to maintain the integrity of the system may be related to the operating parameters. The inventors have developed technical improvements in a machine vision system that analyzes system operating parameters and determines optimal operating parameters for the system. The techniques can include automatically analyzing and adjusting both user and non-user configurable parameters (e.g., based on a user's setting of a set of user configurable parameters). The techniques can include applying various constraints to the operating parameters to ensure that the system meets thermal limitations and / or other operating constraints of the system to maintain system integrity during operation.

[0023] In the following description, numerous specific details are set forth with respect to the systems and methods of the disclosed subject matter, as well as environments in which such systems and methods may operate, in order to provide a thorough understanding of the disclosed subject matter. In addition, it will be understood that the examples set forth below are merely illustrative, and that other systems and methods are contemplated to exist that are within the scope of the disclosed subject matter.

[0024] FIG. 1 illustrates an exemplary embodiment 100 of a machine vision system in which one projector 104 and two cameras 106A, 106B (collectively referred to as cameras 106) are arranged to capture images of an object or scene 102 in a manner suitable for generating stereo image correspondence. In some embodiments, the projector 104 is used to temporally encode an image sequence of an object captured using multiple cameras. For example, the projector 104 can project a rotating pattern onto the object, and each camera can capture an image sequence that includes 12-16 images (or some other number of images) of the object. Each image contains a set of pixels that make up the image. In some embodiments, the light pattern can move horizontally and / or vertically such that the pattern rotates over the object or scene (e.g., without the pattern itself rotating clockwise or counterclockwise).

[0025] Each of the cameras 106 can include a charge-coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, or other suitable image sensor. In some embodiments, each of the cameras 106 can have a rolling shutter, a global shutter, or other suitable shutter type. In some embodiments, each of the cameras 106 can have a GigE Vision interface, a Universal Serial Bus (USB) interface, a coaxial interface, a FIREWIRE® interface, or other suitable interface. In some embodiments, each of the cameras 106 can have one or more smart features. In some embodiments, each of the cameras 106 can have a C-mount lens, an F-mount lens, an S-mount lens, or other suitable lens type. In some embodiments, each of the cameras 106 can have a spectral filter that matches a projector, such as projector 104, to block ambient light outside the spectral range of the projector.

[0026] There are some general parameters that are applicable to cameras / image sensors. Although the examples described herein are described with respect to image sensors, the techniques are not so limited and are applicable to other types of cameras and imaging devices. The exposure time of an image sensor can refer to the time that light strikes the sensor and is integrated over a period of time to form a signal, such as a charge image. Some image sensors may not have a physical shutter to block light from the sensor and control the exposure time (e.g., as in a conventional camera). Some image sensors may therefore use a global shutter mode in which all pixels are exposed simultaneously, or a rolling shutter mode in which each line is exposed in a rolling manner such that each line is exposed with a delay from the start of the exposure of the previous line. In some embodiments, the system can generate a charge image for a particular exposure period by tracking when the integration of light received at the sensor starts and stops, so that at the end of the exposure time, the image sensor effectively stops integrating information. In some embodiments, the image sensor can generate a charge image by transferring charge signals from each of the light sensitive nodes of the sensor (e.g., because the image sensor may not have a mechanical or optical mechanism to obscure the sensor to block light).

[0027] The image sensor has a readout time during which it scans the stored charge image one row at a time, converts the charges to a signal, and outputs the signal digitally. In some embodiments, the system can be configured so that this readout occurs simultaneously with the exposure. For example, if the timing is correct, the image sensor can expose the next frame N+1 while reading out frame N. In some embodiments, a shutter operation (e.g., a global shutter or a rolling shutter) can terminate the exposure and transfer the charge information to a frame memory, for example, so that the frame exposure can be timed in a manner that does not interrupt the readout. For example, if the image sensor takes 4 ms to perform a readout, but the image sensor is configured to only perform an exposure for 1 ms, then the image sensor can be configured to wait to perform the exposure until 3 ms into the readout time (4 ms).

[0028] In some image sensors, frame time refers to the interval from one frame to the next when the image sensor is configured to provide a series of frames. In sensors with simultaneous exposure and readout, the minimum frame time can be approximately determined based on the larger of the exposure time and readout time. In some embodiments, the frame time is measured in milliseconds. In some embodiments, the frame rate is 1 second / minimum frame time to reflect the number of frames achievable in 1 second for a particular frame time. The frame rate can be measured in Hz. The image sensor can be configured to run slower than the maximum frame rate. For example, when using a system that projects a pattern as described herein, it may be desirable to configure the image sensor to run slower than the maximum frame rate for thermal reasons (e.g., due to the power dissipated in the projector, to avoid overheating, to avoid possible damage to the image sensor, etc.) or to limit the speed at which the pattern can be moved within the image.

[0029] Some systems may allow a user to specify one or more parameters of the image sensor. For example, the system may allow a user to specify the exposure time, region of interest (e.g., a subset of pixels, such as a rectangle of imager pixels), digitizer span and precision, resolution (e.g., full resolution, subsampling, or binning in the X-dimension, Y-dimension, or both dimensions), etc. Some parameters, such as frame time, are not user-configurable because it may be difficult for a user to select an appropriate operating value for the parameter.

[0030] In some embodiments, some imaging parameters may be related to the 3D reconstruction process. For example, as discussed herein, some 3D imaging systems are configured to search for correspondence across different camera sequences to obtain a sequence of images that are ultimately used to determine the 3D data. Therefore, some systems may include parameters related to the image sequence, such as the minimum interval between sequences, the number of images in a sequence, the delay between successive sequences, etc.

[0031] FIG. 2 illustrates exemplary components of a projector, according to some embodiments. FIG. 2 illustrates two views of the projector components: a first view 202, which is a perspective view of the projector components from above, and a second view 204, which is slightly rotated from above and shows a slide 206. The components include a motor 208 that is used to controllably move the slide 206 by means of a first component that moves the slide 206 linearly in the x-direction, and a second component 212 that moves the slide 206 linearly in the y-direction (e.g., a linear bearing). The motor 208 rotates a piece 214 that is rotatably attached to the motor 208 via a ball bearing connection 216. Piece 214 has an eccentric shape such that when the motor rotates shaft 214A of piece 214 and a larger portion 214B coupled to the shaft off-axis, a movement is induced by larger portion 214B, and when motor 208 rotates shaft 214A, first and second components 210 and 212 cooperate to shift slide 206 along the X and Y axes, but do not rotate slide 206. The projector components are mounted and supported by a base plate 218.

[0032] In some embodiments, the projector may include various parameters and / or constraints. In some embodiments, the parameters or constraints may be based on motor speed, desired movement of the mask, parameters related to the projector illuminators (e.g., LED illuminators), etc. For example, some parameters may be related to motor speed, such as a maximum motor speed and / or a minimum motor speed. The maximum motor speed may specify the maximum speed in millirevolutions per second that the motor can achieve while being controlled to a desired accuracy (e.g., ±20%) (e.g., 6000 mil revolutions / second). The minimum motor speed may specify the minimum speed that the motor can run while being controlled to a desired accuracy (e.g., ±20%).

[0033] As another example, some parameters may relate to mask motion, such as sequence motion and frame motion. Sequence motion may specify a desired amount of mask motion over an entire sequence of images. Sequence motion may be specified in units of image pixels (e.g., 180 pixels, 240 pixels, 300 pixels, etc.). In some embodiments, sequence motion may be specified in motor revolutions. Frame motion may specify a desired amount of motion over one complete frame time (e.g., from the start of one frame to the start of the next frame). Frame motion may be specified in units of image pixels (e.g., 5 pixels, 10 pixels, 15 pixels, etc.). In some embodiments, sequence motion constraints are used when acquiring a sequence of images for 3D reconstruction. In some embodiments, frame motion constraints are used when acquiring a single frame (e.g., for checking purposes). In some embodiments, projector parameters and / or constraints may be set by the manufacturer and therefore may not be adjusted by the user.

[0034] Some projector parameters may be based on the illumination intensity of the illuminator used to illuminate the mask and project the resulting pattern onto the scene. Exemplary LED illuminator operating parameters may include maximum LED input power, maximum continuous power, maximum average power (100%) when peak pulse power is set to full power, maximum peak power duration (e.g., maximum duration when using full power (100%)), etc. Radiant power can be approximated as 0.3 times the input, which means that the thermal power dissipated by the LED is approximately 0.7 times the input. LED illuminators often include cooling devices such as fan-cooled heat sinks or passive heat sinks. Depending on how the LED illuminator is cooled, the illuminator operating parameters may change. The system may track whether the fan is detected and working properly. Below is an exemplary set of LED illuminator parameters for an LED illuminator with a fan-cooled heat sink and an LED illuminator with a passive heat sink:

[0035] TIFF0007676349000001.tif37153

[0036] FIG. 3 illustrates an exemplary pair of stereo images 300 and 350 corresponding to one of a series of projected light patterns. For example, projector 104 can project a light pattern onto an object, and camera 106 can capture stereo images 300 and 350. In some embodiments, to reconstruct three-dimensional data from a stereo image sequence from two cameras, it may be necessary to find pairs of corresponding pixels, such as pixels 302 and 352, between the images from each camera. An exemplary pair of stereo images 300 and 350 corresponding to one of a set of projected light patterns is illustrated. For example, projector 104 can project a light pattern onto an object, and camera 106 can capture stereo images 300 and 350. In some embodiments, to reconstruct three-dimensional data from a stereo image sequence from two cameras, it may be necessary to find pairs of corresponding pixels, such as pixels 302 and 352, between the images from each camera.

[0037] In general terms, some 3D techniques match left and right features of images 300 and 350. In some embodiments, sharp edges can be used to determine the disparity between images to perform the matching. However, smooth surfaces (e.g., surfaces without markings and / or sharp angles) may not return enough information, such as the circles shown in images 300 and 350. The pattern projection techniques described herein can be used to obtain a sequence of images with enough information to match left / right image features, e.g., a time sequence of pixel values ​​indicating brightness over time to produce a unique signature along a possible path of matching pixels. Thus, for example, a pattern can provide enough information to perform a correspondence search, even if the objects in the scene do not normally have enough naturally occurring features to perform a stereo image correspondence search.

[0038] 4 illustrates an example pair of stereo images 400 and 450 (and associated pixels) and corresponding pixels 402 and 452, which represent the same portion of a pattern projected in the two images 400 and 450. For example, as described above, projector 104 can project a light pattern onto a scene and camera 106 can capture stereo images 400 and 450. The captured stereo images 400 and 450 can be used to identify correspondence between the two pixels. In some embodiments, a sequence of stereo images captured over time is used to identify correspondence.

[0039] Continuing from the single stereo image pair shown in FIG. 4, FIG. 5 shows that as the projector 104 successively projects different light patterns over time onto the scene, the cameras 106 can acquire time-lapse stereo image sequences 500 and 550 and corresponding time-lapse pixels 502 and 552. Each camera 106 can acquire a series of images 1, 2, 3, 4, ... N over time. The time-lapse pixels 502 and 552 are based on pixels (i,j) and (i',j') across the time-lapse stereo image sequences 500 and 550, respectively. The time-lapse pixels each include an ordered list G_i_j_t of gray values ​​over time, where t indicates a discrete time-lapse instance 1, 2, 3, 4, ... N. The system can work out the correspondence between the cameras based on the changes in illumination intensity generated within the scene. In some embodiments, 3D data can be generated by nearly arbitrary movements of the mask / pattern.

[0040] While 3D imaging techniques can use moving patterns to improve correspondence search, various parameters can affect the noise level of the image, which in turn affects the noise associated with the correspondence search and the final 3D data. In some embodiments, the accuracy of the 3D data (e.g., the noise of the 3D data) can be affected based on projector and / or camera parameters. For example, the 3D data can be affected based on the mask movement in combination with one or more of the exposure time, frame time, and illumination intensity. The techniques described herein can be used to determine optimal parameters for a system including a projector and / or a camera. For example, the techniques can determine projector-related parameters, such as mask movement, based on other system parameters, such as the exposure time and / or frame rate of the image sensor, and other projector parameters, such as illumination intensity.

[0041] One or more various parameters of the image sensor and / or projector can be adjusted (e.g., by a user and / or at the factory prior to shipment to a user). As described herein, some of the parameters may be user-configurable, while others may not be user-configurable. For example, the parameters can be adjusted to control the amount of signal collected from an object. The amount of signal collected is affected by a number of factors, such as how dark or bright the object is, how far the object is from the camera, how powerful the LED illuminating the object is, the f-number of the lens (e.g., the ratio of the focal length of the system to the effective pupil diameter), and / or the responsivity of the sensor. Generally speaking, it may be desirable to configure a system including a camera and a projector to obtain an image with a good signal. For example, if the image is too dark, the value may not be discernible, while if the image is too bright, the sensor was saturated.

[0042] FIG. 6A illustrates an exemplary computerized method 600 for configuring a system including a camera and / or a projector, according to some embodiments. The techniques described herein, including method 600 and method 650 described in connection with FIG. 6B, may be performed on any suitable computing system (e.g., a general purpose computing device (CPU), a graphics processing unit (GPU), a field programmable gate array device (FPGA), an application specific integrated circuit device (ASIC), an ARM-based device, or other suitable computing system), and aspects of the techniques described herein are not limited in this respect. The computing device may be a computing device connected to a camera and a projector, and / or a portion of a camera (e.g., a smart camera). In general terms, the system is configured to control the projector and the sensor in a manner that creates a sufficient pattern by the projector to obtain a sufficient image from the sensor for processing.

[0043] In step 602, the computing device receives an image sequence from each camera of the set of cameras (e.g., camera 106 of FIG. 1). In step 604, the computing device generates 3D data based on the image sequence. As described herein, the 3D generation process may include finding a correspondence between the image sequences to generate the 3D data. In step 606, the system determines one or more updates to the operating parameters of the system based on the 3D data. For example, the system may adjust the operating parameters of the camera and / or projector, as described further herein. In some embodiments, the system may adjust the operating parameters without generating the 3D data. For example, the system may analyze changes to the user-configurable parameters to determine whether the parameters are not at the proper settings and / or violate system constraints. For example, the system may perform method 650 shown in FIG. 6B, as described further below. In step 608, the system determines whether the operating parameters require further refinement. For example, if the operating parameters have not achieved a noise level in the 3D data below the noise threshold, the method may return to step 602 with the parameters updated based on step 606. Otherwise, the method proceeds to step 610 to set the operating parameters.

[0044] Referring to step 606, in some embodiments the system can determine that the 3D is noisy. For example, in some embodiments the system can determine noise by analyzing the 3D data, e.g., noting that there are missing pixels (e.g., if the 3D extent could not be reconstructed), and minimizing parameters to minimize the area of ​​the missing pixels. As described herein, these techniques can include adjusting the total exposure, e.g., by adjusting the exposure time and / or the brightness of the projector.

[0045] Method 600 of FIG. 6A illustrates an example of how a system can be configured to use feedback (e.g., via step 608) to determine how to configure the system. This potentially iterative process is described for illustrative purposes only and is not intended to be limiting. For example, in some embodiments, these techniques can derive a set of constraints (e.g., pattern transfer constraints, thermal constraints) and create a set of acquisition parameters that satisfy those constraints without the use of an iterative process.

[0046] 6B illustrates an exemplary computerized method 650 for checking parameter settings, including camera and / or projector configurations, according to some embodiments. As described above, method 650 may be performed as part of analyzing noise in the 3D data (e.g., as part of step 606) and / or without generating the 3D data. At step 652, the computing device determines a set of operational parameters for the camera and / or projector. At step 654, the computing device determines whether any operational parameters in the set of operational parameters require adjustment based on a set of thermal constraints for the camera. If no, the method proceeds to step 656, where the computing device maintains the set of operational parameters. If yes, the method proceeds to step 658, where the computing device adjusts one or more operational parameters in the set of operational parameters to comply with the set of thermal constraints.

[0047] As described in more detail herein, once an image acquisition sequence has been set up for the system, the system may determine a particular set of initial operating parameters. In some embodiments, the system may initially set a frame time based on the system parameters. For example, the system may set the frame time to the minimum time required to read the field of view. As described further below in connection with step 654, the system then evaluates the initial set of operating parameters against various constraints. These constraints may include physical system constraints (e.g., motor speed, illuminator power, etc.) as well as other constraints such as thermal constraints.

[0048] With reference to step 652, the system may determine sensor hardware parameters (e.g., for use in calculating operational parameters). For example, the system may determine an exposure limit time, a readout limit time, and / or a minimum frame time. The minimum frame time may be calculated based on the maximum of the exposure limit time and the readout limit time. The exposure limit time may be calculated based on the sum of the exposure time plus a small constant overhead (e.g., which can be used to achieve a minimum interval between the end of one exposure and the start of the next exposure). The readout limit time may vary from sensor to sensor depending on the modes supported by the sensor. For example, a sensor may have a frame readout time for a full resolution mode and a frame readout time for a sub-sampling mode (e.g., a 2x2 binning mode). The sub-sampling mode may, for example, output values ​​for only some pixels and ignore the other pixels. For example, the 2x2 binning mode may divide the region of interest into 2x2 pixel chunks and output only the top left pixel of each chunk and ignore the other three pixels. As another example, the 2x2b innings mode divides the region of interest into 2x2 pixel chunks and can only output one value for each chunk (e.g. the sum of the 4 pixels in the chunk).

[0049] In some instances, the readout area can be adjusted depending on the mode. For example, for some sensors, the readout area can be adjusted to be smaller than the full sensor size in full resolution mode, with a correspondingly shorter readout limited frame time (e.g., for speed). As another example, for some sensors, the binning modes may not support limited readout areas.

[0050] In some embodiments, the user may control projector and / or image sensor parameters as described above, while other parameters are not adjustable by the user. For example, the user may be able to control the camera exposure time and the projector illumination intensity. In general terms, the product of these two parameters can be considered to control the final image, since for example, 25% intensity with a 4 ms exposure time can achieve the same image in terms of grayscale brightness as 50% intensity with a 2 ms exposure time. However, this is not always the case. For example, if there is a moving object in the field of view, or a moving mask / pattern as described herein, a 2 ms exposure time may be different compared to a 4 ms exposure time (e.g., a 4 ms exposure time may cause blurring).

[0051] In some embodiments, the system can determine system operating parameters. The system can review and / or adjust user-specified settings and / or other system parameters to provide optimal image acquisition (e.g., handling motion blur to accommodate the physical constraints of the projector). For example, the system can evaluate operating parameters based on one or more constraints that must be met to achieve a clear enough image acquisition for processing. In some embodiments, the system can determine constraints related to a minimum amount of motion of the projected pattern. For example, some considerations can include a minimum amount for the pattern to move between frames, a minimum amount that the pattern moves across an entire image sequence, and / or both. For example, if the pattern does not move enough across individual frames and / or sequences, the images may be too close together to capture enough movement of the pattern to obtain unique signature information for correspondence search. Therefore, the system can be configured to set a minimum threshold for how much the pattern should move throughout the entire sequence and / or across individual frames.

[0052] In some embodiments, the system can determine a sequence motion constraint, which can set a minimum amount of apparent motion of the projected pattern in images in an image sequence, such that the images are sufficiently different from each other and provide enough information in the sequence for proper depth extraction. As mentioned above, in some embodiments, the projector moves the pattern to maintain the same orientation (e.g., the pattern stays in the focal plane of the lens and moves in a 2 mm diameter circle projected into the field of view). In some embodiments, the sequence motion can be represented by the number of revolutions per second of the motor (e.g., the pattern moves in an arc at "X" rps). In some embodiments, the sequence motion can be represented by a certain number of pixels that the pattern moves in total across the image sequence. For example, since the pattern is projected to vary smoothly with a characteristic spatial frequency, the system can determine to take a certain number of pixels of motion of the pattern in the images such that the average change in the image pixels is enough to provide useful new information to the reconstruction process and to extrapolate that motion across the entire image sequence. For example, if it takes 10 pixels to go from black to white, it might take around 5 pixels to change the pattern so that the next image has enough new information to add to the reconstruction. This number can be extrapolated to specify the amount of momentum throughout the sequence.

[0053] In some embodiments, the constraint may control the pattern from moving too much during a single exposure time (e.g., too much movement may cause blurring such that the pattern in the image is not discernible across an entire sequence). For example, the system may determine a frame exposure ratio (FER) constraint, which sets a desired amount of motion during an exposure to avoid excessive blurring within a single image. For example, the FER may specify a minimum ratio of frame period to exposure time. The FER may also be specified using a range and / or minimum and maximum values. For example, the FER may be set with a minimum, default, and maximum value. In some embodiments, the FER constraint sets a maximum amount of motion. In some embodiments, the FER constraint may specify that the system should not seek more than a certain percentage of motion during an exposure, determined based on a calculated minimum amount of motion required for a frame.

[0054] In some embodiments, the system can use the FER constraint to determine the readout time. For example, an FER of 2 can specify that the readout time for each frame is at least twice the exposure time. If the system did not need to meet the FER, it would run at a frame rate limited only by the exposure time and readout time. In some embodiments, the FER configures the system to make the frame time longer compared to the frame time normally allowed by the readout time. For example, with a 4 ms readout time and a 1 ms exposure time, the system in that configuration meets an FER of 2. If the user starts to increase the exposure time (e.g., to get a bright image), the system can increase the exposure time up to 2 ms and still meet an FER of 2 with the readout time remaining at 4 ms. If the system sets the exposure time to be greater than 2 ms, the system can be configured to increase the frame time to still maintain an FER of 2.0. For example, if the system sets the exposure time to 3 ms for a good exposure, the system can also set the minimum frame time to 6 ms (i.e., twice the 3 ms exposure time) to meet an FER of 2.0. Even if the system can read out data in 4 ms and therefore maintain an exposure time of 3 ms, the FER adds extra idling time to the system so that it can only acquire one frame every 6 ms.

[0055] In some embodiments, the projector speed can be adjusted to control blur. As mentioned above, the projector can have a certain range of motor speeds, so the system can set the motor speed within that range to achieve smooth pattern movement (e.g., the motor cannot move the pattern smoothly at speeds outside of this range). The system can take the motor speed into account when adjusting the settings of the imaging device, and / or vice versa. For example, in the case of a very fast frame time (e.g., the system can only process the central part of the image and therefore read out a reduced amount of data, e.g., 1 / 4 of the acquired data), the system can increase the motor speed to move the pattern fast enough so that the system can obtain the required pattern movement distance within the fast frame time. As the frame rate increases, the system uses the motor speed limit to prevent the frame time from becoming too long (e.g., the motor cannot move fast enough for a certain frame time, but still provide smooth pattern movement).

[0056] Thus, as described herein, constraints can be imposed on system parameters, including imager parameters and / or projector parameters. The system can use the determined constraints (e.g., sequence motion and / or frame exposure ratio constraints) to select one or more operating parameters of the system to provide a particular image quality, control the projected mask pattern, etc. In some embodiments, the constraints can cause the system to reduce certain parameters that would normally be allowed for use in the system. For example, the system can select a frame time for the sensor to be longer than a minimum frame time determined by the sensor's limitations. For example, the FER constraint can be used to determine camera settings that compensate for blurring of the pattern in and / or across images. As another example, the system can select a motor speed for a motor (e.g., motor 208 in FIG. 2) that is within the effective operating range of the projector's mechanical system, even if the maximum motor speed is not used. For example, if the system is increasing the exposure time to maintain a particular amount of pattern movement over an image sequence, the system can reduce the motor speed to move the pattern slower (e.g., the pattern moves less for a longer exposure time to reduce blurring).

[0057] Referring to step 654, during normal operation the sensor acquires a sequence of images (e.g., 12-48 images) as described above, each of which is used to create a single 3D image. In acquiring these images, the LED illuminators in the projector are turned on to project the pattern from the slide 206 onto the scene. Since LEDs create thermal considerations, in some embodiments the system may check the set of operating parameters determined in step 652 against a set of one or more thermal constraints. As described herein, the operating parameters of the illuminators may include various parameters such as maximum power, maximum continuous power, maximum average power at peak power, and / or maximum power duration. In some embodiments, the operating parameters of the LED illuminators are not user adjustable. For example, the parameters of the LED illuminators may be adjusted during the manufacturing process (e.g., if the parameters are determined to be too loose or too strong), but cannot be adjusted once deployed. As described herein with respect to step 652, the system allows the user to control certain parameters of the imaging device, such as exposure time and / or LED brightness settings (e.g., what percentage of maximum power is effective to use). In step 654, the system determines whether any of the output constraints are violated based on the user's desired settings (e.g., if the system is to continuously stream the sequence at the user's desired frame rate), and depending on the result, the method proceeds to steps 656 or 658, which are described further herein.

[0058] In some embodiments, there may be different thermal considerations for different system components. For example, an LED illuminator may be able to withstand an operating temperature of 150°C, but operating at such a temperature may damage the image sensor (e.g., because the image sensor can only operate up to 60-70°C) and / or the imaging box may become too hot for humans to touch in certain applications, making it unsafe. In some embodiments, long-term constraints, such as a low long-term power constraint, may be set. For example, an LED illuminator may operate fine at a certain setting, but the LED illuminator may burn out if left for a long time (e.g., continuous illumination for more than 1 second).

[0059] In some embodiments, the system may check other system parameters when performing step 654. For example, the system may allow the user to specify preference actions that may affect steps 654-658. In some embodiments, the user may be able to configure the acquisition timing, which determines whether the user prefers to acquire image sequences using a high frame rate (e.g., to acquire the sequence as quickly as possible with a frame rate as high as desired), or whether the user prefers a slower frame rate. In some embodiments, the user's preferences may affect whether the system has perceptible flickering caused by LED illuminators. For example, the acquisition timing may be set to a low latency setting to acquire individual image sequences as quickly as possible. In some embodiments, extra time required to enforce thermal limitations as described in conjunction with step 658 may be added to the end of the sequence, which allows for fast sequence acquisition, but may result in perceptible pauses between sequence acquisitions, as described below. As another example, the acquisition timing may be set to a non-flickering setting to avoid illumination flickering that is perceptible to the user. In some embodiments, other parameters may be adjusted, such as to slow sequence acquisition times but still be compatible with thermal constraints, as described further herein.

[0060] Further with reference to step 654, the system may perform a calibration step. For example, during calibration, the sensor may be configured to acquire images without illumination (e.g., when the projector's LED illumination is in an "off" state). For such calibration images, since the LED illuminators are "off", the system does not need to check thermal constraints as mentioned in step 654. For example, the frame time may be a minimum value set by the limitations of the sensor hardware, without considering thermal constraints. The system may perform a calibration step to, for example, generate a transformation from 2D coordinates on the sensor to 3D physical coordinates in the measurement volume of the system.

[0061] With reference to step 666, in some embodiments, in step 654, the system determines that none of the sequences at the determined frame rate will violate thermal constraints, and as a result, in step 666, the system maintains the operating parameters. In some embodiments, while an individual image sequence will not violate thermal constraints, over time the sequence may cause damage to the system. For example, depending on the LED illuminator (e.g., an LED illuminator without a fan-cooled heat sink), if the LED illuminator is on at 50% power for 50% of the frame time, the LED illuminator will not overheat immediately, but these settings may eventually cause the system to overheat. In some embodiments, the system determines whether the operating parameters need to be adjusted to protect the system from long-term overheating, etc.

[0062] In some embodiments, if the system needs to adjust parameters, the system can take into account user-configurable parameters. For example, the system may determine that the acquisition timing is set to a low latency setting, as described herein, such that the user prefers to acquire individual sequences as quickly as possible. The system can keep the frame rate unchanged and add an extra period so that the LED illuminators are "off" at the end of each sequence before the next sequence is triggered. As described herein, adding additional delays between sequence acquisitions may cause perceptible blinking of the illuminators when acquiring consecutive sequences. In some embodiments, the benefits of the additional delays may outweigh the delays between sequence acquisitions (e.g., including perceptible blinking). Fast sequence acquisitions may be desirable, for example, in industrial situations. For example, if a robot moves a part, it may be desirable to acquire information about the moved part as soon as possible. Even if there is a delay between sequence acquisitions, such a delay may be negligible, because once a 3D calculation is performed, it takes time for the robot to move and perform the next calculation. Thus, such fast sequence acquisitions allow the system to perform 3D calculations as quickly as possible, with delays between calculations.

[0063] With reference to step 658, if the image sequence violates the thermal constraints, the computing device adjusts one or more operational parameters in the set of operational parameters to comply with the set of thermal constraints (including, for example, when only one image sequence violates the thermal constraints). In some embodiments, the system can determine to adjust one or more operational parameters by considering user-configurable parameters. For example, the system may determine that the acquisition timing is set to a non-blinking setting, as described herein. In some embodiments, the system can increase the frame time, for example, to adjust the operational parameters. In some embodiments, the system can decrease the rotor speed, as needed, to maintain the motion constraints of the sequence. Adjusting the operational parameters, including the frame time and / or the rotor speed, can generally achieve the same sequence cycle rate as in step 656, but with a longer latency from triggering the acquisition of a series of images to completing the sequence acquisition. In some embodiments, the system can pulse the LED illuminators at a frame rate high enough to avoid perceptible blinking of the LED illuminators. Such modification of the operational parameters is beneficial for applications where the user perceives the system illumination. For example, such modifications may be appropriate for images involving people, e.g., in medical and / or industrial applications where people are exposed to imaging (and where, e.g., a shroud may be required to avoid constant perceptible blinking). If a system is configured to blink for 1 ms, e.g., every 4 ms, such a configuration would be perceived by the human eye as a constant brightness, but constant strong illumination for 50 ms followed by an off period of 150 ms would be perceived as a strong 5 Hz blinking.

[0064] In some embodiments, the frame time can be increased to adjust the parameters. The system can analyze the exposure time of the sequence based on the specific projector settings to determine how to adjust the parameters. For example, the system can determine if the product of the LED illuminator setting and the total exposure time required for the image sequence is greater than the product of the maximum peak time and maximum power. If so, the system may determine that thermal limits may be exceeded during the sequence. The system can enforce the average thermal limit per frame by increasing the frame time.

[0065] In some embodiments, the parameters can be adjusted depending on the application of the 3D vision system. In some embodiments, the parameters can be adjusted to ensure a maximum number of highest quality images that can be acquired per second. This may be desirable for robotic guidance or high speed machine positioning. In some embodiments, it may be desirable to acquire a certain number of images per second at a certain quality (e.g., not necessarily the highest quality achievable). For example, in medical imaging applications, it may be desirable to acquire a certain number of images.

[0066] Returning to step 606, in some embodiments, these techniques can empirically adjust parameters based on the reconstructed 3D data. For example, these techniques can empirically determine optimal parameters (e.g., exposure time, frame time, etc.) by measuring the accuracy of the 3D reconstruction. For example, the system can be configured to allow the user to adjust the frame time of the imager and the speed at which the projector moves the pattern over the scene. The system can determine a set of projector and / or imaging parameters by determining the speed of movement that minimizes the frame time. In some embodiments, the system can adjust the frame time based on a minimum time required to read out the field of view of one or more image sensors. In some embodiments, the system can adjust the frame time based on a combination of a minimum inter-frame distance and a maximum motor speed. For example, the system can adjust the frame time to a frame time that is longer than normally used by the system. In some embodiments, the system can adjust the frame time based on the exposure time at maximum exposure of the LED illuminator. For example, the system can reduce the motor speed to make the frame time longer than normally used by the system (e.g., to accommodate the reduced motor speed). In some embodiments, the system can adjust the frame time based on the exposure time of the image sensor and a set of illumination parameters (eg, parameters of the LED illuminators described herein).

[0067] In some embodiments, the system can be configured to allow multiple consecutive acquisition sequences (e.g., two or more) with different acquisition parameters commanded by a single trigger event. It may be desirable to specify different parameters for consecutive sequences, for example to enable HDR (high dynamic range) imaging. In some embodiments, the system can be configured to optimize the sequences independently. For example, each sequence can be configured with a different motor speed. In some embodiments, different motor speeds for consecutive image sequences may result in a short pause between sequences while the motor changes speed. In some embodiments, the system can be configured to determine the same motor speed for both sequences that provides acceptable images. In some embodiments, using the same motor speed can avoid a pause between sequences.

[0068] In some embodiments, the system can be configured to allow a general setting to be adjusted so that a set of constraints (e.g., both the projector and / or camera constraints) are adjusted. For example, a single setting can adjust both the exposure time and the LED illuminator output to provide an integrated exposure. In some embodiments, the settings can be adjusted by a user. In some embodiments, the settings can be adjusted by an automatic exposure control algorithm (e.g., designed to provide a desired image grayscale response).

[0069] Thus, generally referring to steps 654-658, the system may or may not adjust operational parameters. For example, if the system determines that the motor and / or frame time parameters allow for continuous system operation without violating thermal limits, the system may run continuously as configured (e.g., at maximum frame rate). As another example, if the system determines that thermal limits do not allow for even a single image sequence to be acquired at that rate, the system must increase the frame time. As another example, if the system determines that thermal limits allow for a single sequence at the configured rate, but continuous operation would violate the thermal limits, the system may adjust operational parameters (e.g., according to user-specified settings) to either increase the frame time (e.g., so that there is no perceptible flickering) or add illuminator off time between sequences while leaving the frame time unadjusted, as described further herein.

[0070] Below are two non-limiting examples that provide determined frame times and sequence times based on the parameters specified for frame count (e.g., the number of frames acquired for each sequence), LED source brightness, exposure time, and acquisition mode (in this example, low latency as described herein).

[0071] Example 1 Frame count = 24 LED original brightness = 255 (100% of maximum output) Exposure time = 1 ms Acquisition mode = Low Latency No fan Frame time = 3.6 ms (limited by readout speed) The average output during the sequence is approximately 27 watts (96W*28% duty cycle, duty cycle is 1ms on, 2.6ms off). Additional illuminator off time is required at the end of each sequence to meet the fanless continuous output limit of W12. Sequence time = 213 ms (88 ms latency from trigger to perform sequence acquisition to completing sequence acquisition, plus 125 ms illuminator off time before the next trigger can be accepted).

[0072] Example 2 Frame count = 24 LED original brightness = 255 (100% of maximum output) Exposure time = 1 ms Acquisition mode = non-blinking No fan In this case the frame time is increased and the LED duty cycle is reduced to 12% during the sequence, allowing continuous operation at the 12% thermal limit. Frame time = 8.8 ms Sequence time = 211 ms (8.8 ms frame time * 24 frames)

[0073] 7-9 show example image pairs 700 and 750, 800 and 850, and 900 and 950, respectively. Images 700, 800 and 900 are images successively acquired by a first camera, and images 750, 850 and 950 are images of the same scene and pattern successively acquired by a second camera. In general terms, these image sequences show some spatial shift between the pattern acquired by the first camera and the pattern acquired by the second camera, and also show the movement of the pattern over time across the image pair. In particular, images 700, 800 and 900 acquired by the first camera show a first perspective of first and second contours 702 and 704 of an object 706, which run vertically along the object 706 in the images. Images 750, 850 and 950 acquired by the second camera show different spatial perspectives of the first and second contours 702 and 704. For example, comparing these image pairs, the circular portion of the pattern marked by 760 in the images is more toward contour 702 in images 700, 800 and 900 compared to toward contour 704 in images 750, 850 and 950 (e.g., contours 702, 704 contain vertical components that protrude "out" from the page, because the first camera views object 706 from a left vantage point while the second camera views object 706 from a right vantage point). Furthermore, comparing the image sequences, the pattern moves across the image pairs from right to left over time, and small shifts of the pattern can be seen at the instants captured by the image pairs of Figures 7, 8 and 9, respectively (e.g., portion 760 does not appear to overlap contour 702 in image 700, but does overlap contour 702 in image 900).

[0074] Techniques operating according to the principles described herein may be implemented in any suitable manner. The process and decision blocks of the flow charts above represent steps and operations that may be included in algorithms that perform these various processes. The algorithms derived from these processes may be implemented as software integrated with and directing the operation of one or more dedicated or general-purpose processors, as functionally equivalent circuitry such as digital signal processing (DSP) circuits or application specific integrated circuit devices (ASICs), or in other suitable manners. It should be understood that the flow charts included herein do not represent the syntax or operations of any particular circuitry or any particular programming language or type of programming language. Rather, the flow charts are illustrative of functional information that may be used to fabricate circuits or implement computer software algorithms for processing a particular device performing the techniques of the types described herein. It should also be understood that unless otherwise indicated herein, the particular sequence of steps and / or operations depicted in each flow chart is merely illustrative of algorithms that may be implemented, and that implementations and embodiments of the principles described herein may vary.

[0075] Thus, in some embodiments, the techniques described herein may be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer-executable instructions may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine.

[0076] When the techniques described herein are embodied as computer-executable instructions, these computer-executable instructions can be implemented in any suitable manner, including many utility functions, each of which provides one or more operations to complete the execution of an algorithm that operates according to these techniques. However, an instantiated "utility function" is a structural element of a computer system that, when integrated with and executed by one or more computers, causes the one or more computers to perform a particular operational role. A utility function can be part or all of a software element. For example, a utility function may be implemented as a function of a process, or as a separate process, or as other suitable processing units. When the techniques described herein are implemented as multiple utility functions, each utility function may be implemented in a unique manner, and need not all be implemented in the same manner. Furthermore, these utility functions may be executed in parallel and / or serially as needed, and may pass information between each other using a shared memory of the computer on which they are executed, using a message passing protocol, or in other suitable manners.

[0077] Generally, utility functions include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of utility functions can be combined and distributed as desired in the systems in which they operate. In some implementations, one or more utility functions that perform the techniques herein can together form a complete software package. These utility functions may be adapted to interact with other unrelated utility functions and / or processes to implement software program applications in alternative embodiments.

[0078] Several exemplary convenience features have been described herein to perform one or more tasks. However, it should be understood that the described convenience features and task divisions are merely illustrative of the types of convenience features that may implement the exemplary techniques described herein, and that embodiments are not limited to a particular number, division, or type of convenience features. In some implementations, all functionality may be implemented in a single convenience feature. Also, in some implementations, some of the convenience features described herein may be implemented together or separately from other convenience features (i.e., as a single unit or separate units), or some of these convenience features may not be implemented.

[0079] Computer-executable instructions implementing the techniques described herein (whether implemented as one or more utility features or otherwise) are encoded in one or more computer-readable media in some embodiments to provide functionality to the media. Computer-readable media include magnetic media such as hard disk drives, optical media such as compact disks (CDs) or digital versatile disks (DVDs), persistent or non-persistent solid-state memory (e.g., flash memory, magnetic RAM), or other suitable storage media. Such computer-readable media may be implemented in any suitable manner. As used herein, a "computer-readable medium" (also referred to as a "computer-readable storage medium") refers to a tangible storage medium. A tangible storage medium is non-transitory and has at least one physical structural element. In a "computer-readable medium" as used herein, the at least one physical structural element has at least one physical characteristic that can be changed in some way during the process of creating a medium with embedded information, recording information on the medium, or encoding the medium with information. For example, the magnetization state of a portion of the physical structure of the computer-readable medium can be changed during the recording process.

[0080] Additionally, some of the technologies described above include operations for storing information (e.g., data and / or instructions) in a particular manner for use in those technologies. In some implementations of those technologies (e.g., implementations in which the technologies are embodied as computer-executable instructions), the information is encoded on a computer-readable storage medium. Where particular structures are described herein as advantageous formats for storing this information, those structures can be used to provide a physical organization of the information when encoded on the storage medium. Those advantageous structures can then impart functionality to the storage medium by affecting the operation of one or more processors that interact with the information, for example, by increasing the efficiency of computer operations performed by the processors.

[0081] In some implementations (but not all implementations) in which the techniques may be embodied as computer-executable instructions, these instructions may be executed in one or more suitable computing devices operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) may be programmed to execute the computer-executable instructions. A computing device or processor may be programmed to execute the instructions when the instructions are stored in a manner accessible to the computing device or processor, such as a data store (e.g., on-chip cache or instruction registers, computer-readable storage accessible via a bus, computer-readable storage media accessible via one or more networks, and media accessible by the device / processor, etc.). These utility functions, including computer-executable instructions, may be integrated with and direct the operation of a single general-purpose programmable digital computing device, a cooperative system of two or more general-purpose computing devices sharing processing power to jointly perform the techniques described herein, a single computing device or a cooperative system of computing devices (co-located or geographically distributed) solely for performing the techniques described herein, one or more field programmable gate arrays (FPGAs) for performing the techniques described herein, or any other suitable system.

[0082] A computing device may include at least one processor, a network adapter, and a computer-readable storage medium. The computing device may be, for example, a desktop or laptop personal computer, a personal digital assistant (PDA), a smart phone, a mobile phone, a server, or any other suitable computing device. The network adapter may be any suitable hardware and / or software that enables the computing device to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. The computing network may include wireless access points, switches, routers, gateways, and / or other network equipment, and any suitable wired and / or wireless communication medium for exchanging data between two or more computers, including the Internet. The computer-readable medium may be adapted to store data to be processed and / or instructions to be executed by the processor. The processor enables the processing of data and execution of instructions. The data and instructions may be stored in the computer-readable storage medium.

[0083] A computing device may further include one or more components and peripherals, including input / output devices. These devices may be used, among other things, to provide a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visually displaying output, and a speaker or other sound generating device for audibly displaying output. Examples of input devices that may be used in a user interface are keyboards, pointing devices such as mice and touchpads, and digitizing tablets. As another example, a computing device may receive input information via voice recognition or other audible forms.

[0084] The embodiments described above are techniques implemented with circuits and / or computer-executable instructions. It should be understood that some embodiments may be in the form of a method, of which at least one example is provided. Operations performed as part of a method may be ordered in any suitable manner. Thus, while shown as sequential operations in the illustrated embodiments, embodiments may be configured to perform operations in an order different from that illustrated, including performing some operations simultaneously.

[0085] Various aspects of the above-described embodiments may be used alone, in combination, or in various configurations not specifically discussed in the above-described embodiments, and therefore are not limited in their application to the details and arrangements of components set forth in the above description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0086] The use of ordinal numbers such as "first," "second," "third," etc. to modify claim elements in the claims does not, in itself, imply a priority, precedence, or ranking of one claim element over another, or the chronological order in which method operations are performed, but is merely used as a descriptive term to distinguish a claim element having a particular name from other elements having the same name (except for the use of ordinal numbers) to distinguish the claim elements.

[0087] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "having," "having," "including," "involving," and variations thereof herein is meant to encompass the items listed thereafter, and equivalents thereof, as well as additional items.

[0088] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Thus, any embodiments, implementations, processes, features, etc. described herein as exemplary are to be understood as illustrative examples and not as preferred or advantageous examples, unless expressly stated otherwise.

[0089] Having described several aspects of at least one embodiment, it should be understood that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims

1. 1. A system comprising: The system includes one or more image sensors, each of the image sensors comprising: arranged to acquire a set of images of the scene; and a frame time indicating a frame interval for each image in the image set; The system further comprises a projector configured to project a moving pattern onto a scene, the projector being adjustable in a speed at which the moving pattern moves across a screen; The system further includes a processor in communication with the one or more image sensors and the projector, the processor configured to determine the rate of movement based at least in part on the frame time compared to one or more other possible rates of movement to reduce noise in 3D data generated based on the set of images. The above system.

2. The system of claim 1 , wherein the processor is further configured to adjust the frame time based on a minimum time required to read out a field of view of the one or more image sensors.

3. The system of claim 1 , wherein the processor is further configured to adjust the frame time based on a combination of a minimum inter-frame distance and a maximum motor speed.

4. The system of claim 1 , wherein the processor is further configured to adjust the frame time based on an exposure time at maximum exposure of an illumination system of the projector.

5. The system of claim 1 , wherein the processor is further configured to adjust the frame time based on an exposure time of the one or more image sensors and a set of illumination parameters.

6. 6. The system of claim 5, wherein the set of illumination parameters includes an illuminator output during exposure, a maximum average output of the illuminator, a maximum illuminator output, and / or a maximum average output of the illuminator when peak illuminator power is set to full power.

7. 1. A processor-implemented computerized method comprising: The processor is configured to store a set of one or more constraints; the processor is configured to determine, based on the set of one or more constraints, a frame time indicative of a spacing between each frame for each image in the set of images, for each of one or more image sensors in communication with the processor and arranged to acquire a set of images of a scene; each said image sensor configured to determine, for a projector configured to project a moving pattern onto the scene, a speed of movement of the moving pattern across a screen based at least in part on the frame time; the determined motion velocity is compared to one or more other possible motion velocities to reduce noise in 3D data generated based on the image set. The above method.

8. The method of claim 7 , further comprising adjusting the frame time based on a minimum time required to read out a field of view of the one or more image sensors.

9. The method of claim 7 , further comprising adjusting the frame time based on a combination of a minimum inter-frame distance and a maximum motor speed.

10. 8. The method of claim 7, further comprising adjusting the frame time based on an exposure time at maximum exposure of an illumination system of the projector.

11. The method of claim 7 , further comprising adjusting the frame time based on an exposure time of the one or more image sensors and a set of illumination parameters.

12. The method of claim 11 , wherein the set of illumination parameters includes an illuminator output during exposure, a maximum average output of the illuminator, a maximum illuminator output, and / or a maximum average output of the illuminator when peak illuminator power is set to full power.

13. The system of claim 1, wherein the possible motion velocities include an initial velocity and an optimal velocity.

Images