System and method for three-dimensional scanning of moving objects longer than the field of view

Abstract

To provide a system and method for using an area scan sensor of a vision system, in conjunction with an encoder or other knowledge of motion, to capture accurate measurement of an object larger than a single field of view (FOV) of the sensor.SOLUTION: Logic automatically determines if an object 120 is longer than the FOV, and thereby moves / conveys the object 120 until the object is no longer present in the FOV, and causes a sequence of image acquisition snapshots to occur while the object 120 remains within the FOV. At that point when the object is no longer present, acquisition is ended and the individual images are combined as segments in an overall image. These images can be processed to derive overall dimensions of the object based on input application details.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to machine vision systems that analyze objects in three-dimensional (3D) space, and more particularly to systems and methods for analyzing objects transported through an inspection zone on a conveyor. [Background technology]

[0002] Machine vision systems (also referred to herein as "vision systems") that measure, inspect, align objects, and / or decode symbology information (e.g., barcodes, also known as "ID codes") are used in a wide range of applications and industries. These systems are based on the use of image sensors to capture images (typically grayscale or color, one-, two-, or three-dimensional) of a target or object, which are then processed using an on-board or interconnected vision system processor. This processor generally includes both processing hardware and non-transitory computer-readable program instructions to execute one or more vision system processes to generate a desired output based on the image processing information. This image information is typically presented in an array of image pixels, each having a different color and / or intensity.

[0003] As noted above, one or more vision system cameras can be positioned to capture two-dimensional (2D) or three-dimensional (3D) images of objects in an imaged scene. 2D images are typically characterized as pixels having x and y components within an overall N x M image array (often defined by the pixel array of the camera's image sensor). When images are captured in 3D, there is a height, or z-axis, component in addition to the x and y components. 3D image data can be acquired using a variety of mechanisms / techniques, including stereo camera triangulation, LiDAR (light detection and ranging), time-of-flight sensors, and (for example) laser displacement profiling.

[0004] A 3D camera is typically positioned to capture 3D image information for objects that fall within its field of view (FOV), which constitutes a volumetric space that fans out along the horizontal x and y dimensions as a function of distance from the camera sensor in the vertical z dimension. Sensors that acquire images of the entire volumetric space in parallel / simultaneous (i.e., "snapshots") are called "area scan sensors." Such area scan sensors are distinct from line scan sensors (e.g., profilers) that capture 3D information slice by slice and use motion (e.g., conveyor movement) and measurement of this motion (e.g., by motion encoders or steppers) to move objects through the inspection field / FOV.

[0005] One advantage of line-scan sensors is that the inspected object can be any length, provided that the length of the object is taken along the direction of conveyor motion. Conversely, area-scan sensors, which take image snapshots of volumetric space, do not require an encoder to capture the 3D scene, but if the object is longer than the field of view, they cannot image the entire object in a single snapshot. If only part of the object is captured in a single snapshot, additional snapshots (or multiple snapshots) must be taken for the remaining length as the remaining (not yet imaged) parts of the object pass into the FOV. With multiple snapshots, the challenge becomes how to efficiently align (stitch) the multiple 3D images together so that the overall 3D image accurately represents the object's features. Summary of the Invention

[0006] The present invention overcomes the deficiencies of the prior art by providing a system and method for using a vision system's area scan sensor, in conjunction with encoders or other knowledge of motion, to capture accurate measurements of objects larger than the sensor's single field of view (FOV). The system and method specifically address the deficiency that snapshot-based area scan vision systems define a limited field of view, which typically requires the system to acquire multiple snapshots and other data necessary to combine the snapshots. This avoids the potentially computationally intensive task of combining raw image data and then post-processing such data. Instead, exemplary embodiments identify object features / edges or edges (also referred to as "vertices" in reference to identified polygonal shapes) and track them from image to image, thereby providing a lightweight way to process the entire extent of the object for the purposes of displaying or measuring its size and position in space (dimensioning). The system and method can employ logic to automatically determine if the object is longer (in the transport direction) than the FOV, thereby generating a series of image capture snapshots while the moving / transporting object remains within the FOV until the object is no longer present in the FOV. Once no longer present, acquisition ends and the individual images can optionally be combined as segments into an overall image. The overall image data can be used in various downstream processes. Feature data collected from the individual image snapshots, derived with or without the creation of an actual overall image, can be processed to derive the overall dimensions of the object based on the input application details.The feature data thus gathered can be used to determine other properties and characteristics of the object, such as, but not limited to, skew, length, width and / or height tolerance exceedance, confidence score, liquid volume, categorization, quantity (QTY) of data actually imaged from the object versus expected data imaged from the object, positional characteristics of the object, and / or detection of damage to the object. The system and method can also combine complex / multi-level objects that would typically be separated by conventional imaging using area scan sensors due to loss of 3D data in shadows, etc.

[0007] In an exemplary embodiment, a vision system and a method for using the same are provided. The vision system and method can include a 3D camera assembly arranged as an area scan sensor and a vision system processor that receives 3D data from images of an object acquired within a field of view (FOV) of the 3D camera assembly. The object can be conveyed through the FOV in a conveying direction, and the object can define an overall length in the conveying direction that is longer than the FOV. A dimensioning processor measures the overall length based on motion tracking information derived from the object's conveyance through the FOV combined with multiple 3D images of the object. These images can be acquired sequentially by the 3D camera assembly with a predetermined amount of conveying momentum between the 3D images. A presence detector associated with the FOV can provide a presence signal when the object is positioned adjacent to it. The dimensioning processor can be configured to determine whether the object appears in more than one image as it moves in the conveying direction in response to the presence signal. The dimensioning processor can be configured to determine whether the object is longer than the FOV as it moves in the conveying direction in response to information related to features on the object. An image processor can combine information about features about the object from successive image captures by the 3D camera that generate a collection of feature data to determine the dimensions of the object as a whole rather than combining separate individual images into an overall image. By way of example, the image processor can be configured to take a series of image capture snapshots while the object remains within the FOV, in response to a total length of the object being greater than the FOV, until the object exits the FOV. The image processor can be further configured to use the collected feature data and, based on input application data, to derive overall attributes of the object, where the overall attributes include at least one of a confidence score, an object classification, an object dimension, a skew, and an object volume.The object manipulation process can perform tasks on the object based on its overall attributes, including at least one of transferring the object, rejecting the object, issuing a warning, and correcting skew of the object. By way of example, the object can be transported by a mechanical conveyor or manually, and / or the tracking information can be generated by an encoder operatively connected to the conveyor. The motion detection device can be operatively connected to the conveyor, an external feature detection device, and / or a feature-based detection device. The presence signal can be used by a dimensioning processor to determine the continuity of the object between each of the images as the object moves in the transport direction. The images can be acquired by a 3D camera with a predetermined overlap between them, and a removal process can use the tracking information to remove the overlap from the object's dimensions and determine its overall length. The image rejection process can reject the last of the acquired images as a result of the presence signal being asserted after the previous image included the trailing edge of the object. The dimensioning processor can be further configured to use information related to features on the object to determine the continuity of the object between each of the images as the object moves in the transport direction. The dimensioning system may further define a minimum spacing between objects in the image below which the multiple objects are considered to be a single object with missing 3D image data. Illustratively, the image processor may be configured to generate a collection of feature data for the object related to: (a) out-of-length data, (b) out-of-width data, (c) out-of-height data, (d) out-of-volume data, (e) a confidence score, (f) a liquid volume, (g) a classification, (h) a quantity (QTY) of data actually imaged from the object relative to the expected data imaged from the object, (i) a positional feature of the object, and / or (j) detection of damage to the object.

[0008] In an exemplary embodiment, the vision system, and associated method, may include a 3D camera assembly arranged as an area scan sensor with a field of view (FOV) capable of performing an acquisition process that captures one or more images of an object as the object passes through the FOV, and (a) determines whether the object occupies more than one image, (b) determines when the object no longer occupies the next image, and (c) calculates the size and relative angle of the object from the acquired one or more images.

[0009] In an exemplary embodiment, a method for displaying or measuring (dimensioning) the spatial size and position of an object using a vision system having a 3D camera assembly configured as an area scan sensor is provided, wherein a vision system processor receives 3D data from images of the object acquired within a field of view (FOV) of the 3D camera assembly. The object is transportable through the FOV in a transport direction, and the object can define an overall length in the transport direction that is longer than the FOV. The method further includes measuring the overall length based on motion tracking information derived from the object's transport through the FOV in combination with multiple 3D images of the object acquired sequentially by the 3D camera assembly with a predetermined amount of transport motion between the 3D images. A presence signal can be generated when the object is positioned adjacent the FOV, and in response to the presence signal, it can be determined whether the object appears in more than one image as the object moves in the transport direction. Depending on information related to features on the object, it can be determined whether the object is longer than the FOV as the object moves in the transport direction. Information relating to features on the object from successive captures of images by the 3D camera can be combined to generate a collection of feature data that provides dimensions of the object as a whole, rather than combining separate individual images into an overall image. Illustratively, if the overall length of the object is longer than the FOV, a series of image capture snapshots can be taken while the object remains within the FOV until the object exits the FOV. Using the collected feature data and based on the input application data, global attributes of the object can be derived, and the global attributes can include at least one of a confidence score, a classification of the object, object dimensions, skew, and object volume. Illustratively, the method can perform tasks related to the object based on the global attributes, including at least one of forwarding the object, rejecting the object, issuing a warning, and correcting the skew of the object.Illustratively, the presence signal can be used to determine the continuity of the object between each of the images as the object moves in the transport direction. In combining the information, the method can further generate (a) out-of-length data, (b) out-of-width data, (c) out-of-height data, (d) out-of-volume data, (e) a confidence score, (f) a liquid volume, (g) a classification, (h) a quantity (QTY) of data actually imaged from the object relative to the expected data imaged from the object, (i) a positional characteristic of the object, and / or (j) detection of damage to the object. A minimum spacing between objects in the images can be defined, below which multiple imaged objects are considered a single object with missing 3D image data. [Brief explanation of the drawings]

[0010] The following description of the invention refers to the accompanying drawings, in which:

[0011] FIG. 1 is a schematic of a system for acquiring and processing 3D images of objects longer than the field of view using an area scan sensor, using a dimensioning process (processor) to generate an overall image from multiple snapshots as the object moves through the field of view;

[0012] FIG. 2 is a side view of the conveyor and area scan sensor arrangement of FIG. 1 , showing details of the FOV, a region of interest (ROI) usable for 3D imaging of an object trigger plane for image acquisition according to an exemplary embodiment;

[0013] FIG. 3 is a diagram illustrating the operational behavior of the configurations of FIGS. 1 and 2 for an exemplary object having a length in the transport direction greater than the available ROI, which has reached the trigger plane to trigger (cause) a first snapshot;

[0014] FIG. 4 is a diagram illustrating the driving behavior of FIG. 3 in which an exemplary object passes partially out of the ROI, indicating sufficient object movement for the encoder to trigger a second snapshot;

[0015] FIG. 5 is a diagram illustrating operational operation of the configurations of FIGS. 1 and 2 for an exemplary object exhibiting a complex upper surface, which would result in loss of 3D image data and / or the appearance of two separate objects in the FOV;

[0016] FIG. 6 is a flow chart illustrating a procedure for performing a dimensioning process according to an exemplary embodiment, including when multiple objects and / or complex objects are imaged;

[0017] FIG. 7 is a top-down view of an exemplary imaged conveyor surface with overlapping image portions representing the entire imaged object; and

[0018] FIG. 8 is a flow diagram of the overall image acquisition procedure for the configurations of FIGS. 1 and 2 adapted to handle overlapping image portions as shown in FIG. 7 to generate the full length of the object, as well as maximum width, maximum height and relative angle with respect to the conveyance direction (or other coordinate system). DETAILED DESCRIPTION OF THE INVENTION

[0019] I. System Overview

[0020] FIG. 1 shows a schematic diagram of a configuration 100 in which a vision system camera assembly (also referred to simply as a “camera” or “sensor”) 110 captures 3D image data of an exemplary object 120 as it passes below a field of view (FOV) relative to a moving conveyor 130 moving in a downstream direction (arrow 132). A local / relative 3D coordinate system 138 is shown illustratively comprising x-y-z Cartesian coordinates. Other coordinate systems, such as a polar coordinate system, may also be used to represent this 3D space. Note that in this example, the object 120 defines an upstream-to-downstream length L (roughly along the y-axis / direction of movement) that is longer than the upstream and downstream boundaries BF and BR of the FOV, as will be described in further detail below.

[0021] The 3D camera / imaging assembly 110 discussed herein can be any assembly that captures 3D images of an object, including, but not limited to, stereoscopic cameras, time-of-flight cameras, LiDAR, ultrasonic ranging cameras, structured lighting systems, and laser displacement sensors (profilers); thus, the term 3D camera should be broadly understood to include these systems and any other system that generates height information in association with a 2D image of an object. It can also comprise a single camera or an array of cameras, and the terms "camera" and / or "camera assembly" can refer to one or more cameras that capture an image (or images) in a manner that generates the desired 3D image data of the scene. The illustrated camera assembly 110 is shown mounted over the surface of the conveyor 130, such as a checkpoint or inspection station, to image flowing objects as they pass by. In this embodiment, the camera assembly 110 defines an optical axis OA that is generally perpendicular (along the z-axis) to the surface of the conveyor 130. Other non-perpendicular directions of axis OA relative to the conveyor surface are explicitly contemplated. Depending on the conveyor's operating speed and the acquisition time of the camera's image sensor S and associated electronics 110 (depending in part on the frame rate and aperture setting), the object 120 can remain in motion (typically) or can stop momentarily for imaging. The camera 110 acquires a 3D image when the object 120 is sufficiently within its FOV, which can be triggered by a photodetector or other triggering mechanism 136, which sends a trigger signal to the camera 110 and associated processor(s). Thus, the camera assembly 110 in this embodiment is arranged as an area scan sensor.

[0022] The camera assembly 110 includes an image sensor S adapted to generate 3D image data 134. The camera assembly also (optionally) includes an integral illumination assembly I, e.g., an LED ring illuminator that projects light in a predictable direction relative to the axis OA. In an alternative configuration, external illumination (not shown) may be provided. A suitable optical package O is shown in optical communication with the sensor S along the axis OA. The sensor S is in communication with an internal and / or external vision system process (processor) 140 that receives image data 134 from the camera 110 and performs various vision system tasks on the data in accordance with the present systems and methods. The process (processor) 140 includes underlying process / processors or functional modules, such as a set of vision system tools 142, which may include various standard and custom tools that identify and analyze features in the image data, including, but not limited to, edge detectors, blob tools, pattern recognition tools, deep learning networks, ID (e.g., barcode) detectors and decoders, and the like. In accordance with the present system and method, the vision system process (processor) 140 may further include a dimensioning process (processor) 144. This process (processor) 144 may perform various analysis and measurement tasks on features identified in the 3D image data to determine the presence of specific features and calculate further results therefrom. According to an exemplary embodiment, the process (processor) interfaces with a variety of conventional and custom (e.g., 3D) vision system tools 142.

[0023] System configuration and result display can be handled by a separate computing device 150, such as a server (e.g., cloud-based or local), PC, laptop, tablet, and / or smartphone. Computing device 150 is depicted (by way of non-limiting example) with a conventional display or touchscreen 152, keyboard 154, and mouse 156, which collectively provide graphical user interface (GUI) functionality. Alternative implementations of device 150 can include a variety of interface devices and / or form factors. According to the present exemplary configuration, the GUI can be driven in part by a web browser application that resides on the device's operating system and displays web pages along with control and data information from process (processor) 140.

[0024] It should be noted that the process (processor) 140 can be fully or partially on-board the housing of the camera assembly 110, and the various process modules / tools 142 and 144 can be instantiated in whole or in part either on the on-board process (processor) 140 or on a remote computing device 150, as appropriate. In an exemplary embodiment, all vision system and interface functionality can be instantiated on the on-board process (processor) 140, and the computing device 150 can be used primarily for training, monitoring, and related operations on interface web pages (e.g., HTML) generated by the on-board process (processor) 140 and communicated to the computing device via a wired or wireless network link. Alternatively, all or part of the process (processor) 140 can reside within the computing device 150. The results of the processor's analysis can be communicated to a downstream utilization device or process 160. Such a device / process can use the results 162 to manipulate the object / package, for example, gating the conveyor 130 to direct the object to a different destination based on the analyzed characteristics and / or rejecting defective objects.

[0025] The camera assembly includes on-board calibration data, established by factory and / or field calibration procedures, that maps the x, y, and z coordinate axes of the imaged pixels to the camera's coordinate space. This calibration data 170 is provided to the processor and used to analyze the image data. Also, in an exemplary embodiment, the conveyor and / or its drive mechanism (e.g., stepper motor) includes encoders or other motion tracking mechanisms 180 that report relative motion data 182 to the processor 140. The motion data can be communicated in a variety of ways, such as distance-based pulses, each defining a predetermined increment of conveyor movement. The pulses can be summed to determine the total movement over a given period of time.

[0026] It should be noted that, as used herein, the term "conveyor" should be understood broadly to include configurations in which objects are passed through the FOV by another technique, e.g., manual manipulation. Thus, an "encoder," as defined herein, can be any acceptable motion measurement / tracking device, including steppers, mark readers, and / or devices that track features or fiducials on the conveyor or object as it passes through the FOV, including various internal techniques that use knowledge of the underlying vision system application (e.g., of features in the image) to determine the degree of movement between image snapshots.

[0027] II. Dimensioning Process (Processor)

[0028] A. Settings

[0029] FIG. 2 shows a configuration 200 adapted to image objects longer than the FOV in the direction of motion 132. The camera assembly 110 is shown over a portion of the conveyor 130 that forms an inspection station for such objects. The conveyor 130 transmits a resettable encoder count 210 for each trigger event, at which the conveyor 130 is depicted in the area of ​​the 3D camera assembly. A trigger (in this case, the trigger state 220 is negative, with a red or other color / shade indicator represented by point cloud shading) is emitted each time an object passes through a trigger plane 230 aligned with the presence detector 136 ( FIG. 1 ) described above. The 3D FOV defines a usable region of interest (ROI) 250 extending from the surface of the conveyor 130 along the camera axis OA and upward on either side of the axis OA. In this example, the ROI defines a usable height that can be defined by the user based on the maximum expected object height. The points 256 and 258 where the top 254 of the ROI 250 intersects with the front and rear boundaries BF and BR of the FOV, respectively, essentially define the usable length LR of the ROI in the direction of motion 132 (y-axis). Thus, the greater the maximum height HR of the ROI, the shorter the length LR.

[0030] B. Object size inference and trigger / encoder logic

[0031] Once the usable ROI is defined, the processor determines how many pulses the conveyor will travel to reach the length LR. Referring to FIG. 3, the system is shown operating in run-time mode to estimate the size (length) of the object 120, which is depicted with its leading edge 310 reaching the trigger plane 230. The trailing edge 320 is outside the rear boundary 340 of the usable ROI 250. At this point, the 3D camera assembly is triggered to acquire the first image (snapshot) of the object, and the encoder count 330 is set to zero in the processor. The snapshot is analyzed to determine (using vision system tools, height change, etc.) whether the trailing edge of the object is within the ROI. If the object 120 ends within one ROI 250, the snapshot is reported as a complete 3D image for that object, and the system waits for the next object / trigger. Conversely, if the object 120 does not appear to terminate within the usable ROI (e.g., there is no substantial change in the object's geometry within the rear boundary 340 of the ROI 250), i.e., as in FIG. 3 , the system stores the first image and counts encoder pulses for one ROI length (LR) (or a known fraction of this length LR). This count is referenced in FIG. 4 , where the object's rear end 320 has just passed into the usable ROI 250 (past the rear ROI boundary 340). At this point (shown in FIG. 4 ), pulses for one ROI length LR have been counted. In this example, the pulse count 430 is 300 mm. This length is equal to or less than the length LR. As explained below, if overlap between snapshot images is desired, this pulse count can be less than the length LR, thereby ensuring that detail at the edges of the images is not lost. As shown in FIG. 4, the trigger state 420 is also positive at this stage (represented by green or another color / shade and by hatching shading) since the object is present in the trigger plane 230.The logical combination of these events (a positive trigger state and a full encoder count) triggers the camera assembly to take a second snapshot of object 120. This second snapshot is also stored in association with the object from the first snapshot.

[0032] The system analyzes the second snapshot to determine whether the trailing edge 320 of the object 120 is now downstream from the rear ROI boundary 340. If so, then the entire length of the object has been imaged, and the two snapshots can be combined and processed as described further below. If the trailing edge of the object is not within the ROI of the second snapshot, the encoder count is reset, the trigger state 420 remains high since the object continues to be present, and the system counts until the next ROI length LR is reached. Another snapshot is taken, and the above process is repeated until the nth snapshot, in which the trailing edge of the object is finally detected within the ROI 250. At this point, the object has completely passed out of the FOV, so the trigger state goes low and the imaging result is communicated. Special cases sometimes exist due to overlap between snapshot images, but these can be handled by evaluating the encoder count between separate image acquisitions. If a previous snapshot imaged the edge of the object and reported dimensions, but the trigger state is still high (positive), and a new snapshot is taken, and in this case the encoder count for the current snapshot is equal to the length of the ROI, the new snapshot is discarded and no dimensions are reported. This is done because it is an extra / unused snapshot resulting only from overlap between snapshot images. In general, all snapshots can be combined and processed in relation to one object image, as described below.

[0033] C. Complex and / or multi-layered objects

[0034] In some embodiments of runtime operation, an object may exhibit a complex 3D shape. Referring to FIG. 5, an object 520 with varying heights 522 and / or overhanging (occluding) features 525 along its upper surface is shown reaching the trigger plane 230. At this point, the trigger state 540 becomes positive (green, represented by hatched shading), and the encoder count 530 starts at 0. However, this complex shape of the object may result in loss of 3D image data or confuse the vision system processor about the actual location of the object's trailing edge 526—which, in this example, is located outside the ROI 250 still available when the first snapshot is acquired by the camera assembly 110. Also referring to procedure 600 of FIG. 6, an object initially triggers the camera assembly to acquire a 3D snapshot in step 610. A single object (of simple or complex shape) or multiple objects may be present in this initial image. The vision system analyzes the image to determine whether multiple objects are detected, typically by looking for boundaries that extend to the conveyor surface (reference line) and gaps between separated boundaries. If two objects are not detected, the vision system treats the object as a single object (decision step 620). Conversely, if two or more objects are detected, decision step 620 of procedure 600 branches to a further decision step 630, where the gap(s) are analyzed to determine whether the gap(s) are smaller than a minimum distance (determined using camera calibration data and conventional measurement techniques). If the gap(s) are larger than the minimum distance—set by the user or automatically—the system indicates there are multiple objects in the image (step 640).For two objects with a greater than minimum gap distance (step 640) or a single object (decision step 620), procedure 600 branches to decision step 650, where the vision system determines whether all object edges are in the image and the trigger condition goes negative. If so, procedure 600 branches to step 660, where the imaging results are output and the encoder is reset to await the next object / trigger. Conversely, if the trigger condition does not go negative after the current (first) image and the trailing edge of the object(s) remains outside the FOV, decision step 650 branches to step 670, where the camera assembly counts encoder pulses and takes another snapshot of the FOV. This continues until the trigger condition goes negative and the trailing edge of the object is detected.

[0035] Referring back to decision step 630, if a gap exists but is less than the minimum distance, procedure 600 presumes that the image contains a single object and that the gap is the result of missing or missing 3D data in the imaged object. Thus, the object is treated as if it were a single entity, and the decision step branches to a further decision step 650 (described above), in which the presence or absence of the object's trailing edge in the image determines the next step. It should be noted that one technique for predicting and providing missing or missing 3D image data is described in co-pending U.S. Provisional Application No. 62 / 972,114, entitled "Composite 3D Blob Tool and Method of Operation Thereof," filed February 10, 2020, and assigned to the present applicant, the teachings of which are incorporated herein by reference as useful background information. Such a blob tool can be used to inform further utilization steps in generating imaging results.

[0036] D. Image data tracking and resulting overlap

[0037] Referring to FIG. 7, a continuous top-down (xy-plane) view 710 of a conveyor belt 720 is shown, constructed by (for example) acquiring three consecutive images in the presence of a long object. The three acquired images 730, 732, and 734 of the entire object are shown sequentially. In particular, the encoder counts are set so that a first predetermined overlap distance D1 exists between a first pair of images 730 and 732, and a second predetermined overlap distance D2 exists between a second pair of images 732 and 734. Using the overlap distance between the object images ensures that the acquired object features are not inadvertently lost or obscured at the edges of the 3D region of interest. As described below with reference to FIG. 8, the vision system can appropriately remove or combine the overlapping regions to calculate the actual object dimensions and features. It should be noted that, because a captured image of an object typically defines one or more polygons with associated vertices that are used to derive the shape boundary, the term "feature" as used herein should be understood to include or can be used interchangeably with the term "vertex."

[0038] In operation, procedure 800 of FIG. 8 begins in step 810 with acquiring a 3D image in the manner described above. That is, a trigger is generated and the encoder is set to count the movement of an object through the FOV. After the count reaches a distance value that allows overlap with the next image, the count is reset in step 820, and the system determines whether it is the last image—the trailing edge of the object is in the ROI. If not (via decision step 830), procedure 800 branches back to step 810, where another overlapping image is acquired. If the image is the last image in a series of images, procedure 800 branches to step 840 (via decision step 830), where the series of 3D images (segments within the entire object) are passed to a dimensioning process (processor) and associated tools to determine the object's dimensions and (optionally) resolve features. Using the distances established by the encoder to the image pixel locations, the x-y positions of each segment of the object and the height of the box defining the boundary are mapped in step 850. Based on this mapping, overlaps (which constitute a given known encoder distance between segments) are then removed to calculate the actual object length. The resulting data can be defined as a feature data set that effectively combines data from individual shapes to establish an overall view or characterization of the object in the absence of an actual image of the combined object. This resulting feature data set can also be used to determine (for example) the maximum width and height of the entire object and its relative angle to the direction of transport movement. This information is particularly useful in various application processes, such as logistics management to ensure proper handling of objects (e.g., packages).

[0039] It should be noted that, as explicitly contemplated by the present systems and methods, generating an actual overall (composite or stitched) image from the individual image captures (snapshots) is optional. The collection of feature data can be used independently of generating an overall image based on NxM pixels to provide suitable results for use in determining object dimensions and / or other processes described below. The overall image can be generated and used in further processes and / or to provide a visual record of all or part of the imaged object, if desired.

[0040] E. Application of Results

[0041] It is contemplated that the resulting feature data (and / or vertex data) collections from the above-described operations are applicable to a variety of tasks and functions involving imaged objects, including, but not limited to, streams of packages (packages) of different sizes and shapes. In particular, the present process can be used to determine (for example) the skew angle of an object / package relative to the direction of travel and / or the boundaries of surrounding support surfaces. One potential item that can be identified and measured using the present feature data collections is the skew angle of the object relative to the direction of conveyor travel (and parallel side edges) or other support surfaces. In addition to potentially causing jams in narrow chutes, gates, or other transitions, skew data can cause an object to appear longer than its normal, typical dimensions, causing the system to erroneously generate a defect. The skew angle data must be used so that corrective action (i.e., ignoring the false defect or straightening the object) can be taken. In particular, the skew angle information (and / or other measured characteristics) can be part of a metadata tag applied to the results (feature data collection) for use in various downstream operations. Other relevant characteristic data may include out-of-length, out-of-width, out-of-height, and / or out-of-volume values, which indicate that the object is too long, too wide, too tall, or too voluminous for the parameter limits. Further relevant data may relate to, but is not limited to: (a) a reliability score, which can give information on either the shape of the object or the quality of the data received; (b) liquid volume, which can give information about the shape or true (rather than the smallest cube) volume of an object; (c) classification, which can give information on whether the surface shape of the object is flat or not; (d) the quantity of data displayed / imaged (QTY) relative to how much display / image is expected; (e) location features (e.g., distance from a reference point such as a corner, center of gravity, or edge of a conveyor belt); and / or (f) Damage detection (e.g., presence of bulging / collapsed packages based on the actual imaged shape versus the expected shape).

[0042] The use of feature data collections generally avoids the need to include more detailed pixel-based image data for the combined object, thereby allowing additional processing to be performed on the object. Handling and manipulating such feature data collections allows for faster processing and less processor overhead due to the overall size of the data. Some example tasks can include automating warehouse processes, such as rejecting and / or redirecting objects that are mismatched in size and / or shape. Using object-diverting data in this manner allows a server to avoid oversized objects from becoming jammed in a chute or conveyor bend. Similarly, feature data collections derived by the present systems and methods can assist automated labeling processes in ensuring accurate object positioning. Also, as noted above, skew information can be used to avoid false defect conditions and / or allow a system or user to orient an object within a conveyor stream, thus avoiding minor jam conditions. Other object / package handling tasks that rely on this data can use this data in ways that are apparent to those skilled in the art.

[0043] III. Conclusion

[0044] It should be apparent that the system and method described thus far provides an efficient, reliable, and robust technique for determining the length of oversized objects that do not fit entirely within the FOV of a 3D area scan sensor along the direction of conveyor travel. The system and method uses conventional encoder data and detector triggers to generate an accurate set of object dimensions, and is operable even when 3D data is lost or missing due to complex shapes and / or multiple objects present in the conveyor stream.

[0045] The foregoing is a detailed description of exemplary embodiments of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above may be combined with features of other described embodiments as appropriate to produce numerous combinations of features relating to new embodiments. Furthermore, while several separate embodiments of the apparatus and method of the present invention have been described above, what has been described herein is merely an exemplary illustration of the application of the principles of the present invention. For example, as used herein, the terms "process" and / or "processor" should be understood broadly to include various electronic hardware and / or software-based functions and components (and may alternatively be referred to as functional "modules" or "elements"). Furthermore, depicted processes or processors may be combined with other processes and / or processors or divided into various partial processes or processors. Such partial processes and / or partial processors may be combined in various ways according to embodiments of the present application. Similarly, it is expressly contemplated that any functions, processes, and / or processors herein may be implemented using electronic hardware, software consisting of a non-transitory computer-readable medium of program instructions, or a combination of hardware and software. Additionally, as used herein, various directional and orientation terms such as "vertical," "horizontal," "upper," "lower," "bottom," "top," "side," "front," "rear," "left side," "right side," etc. are used only as relative conventions and not as absolute orientations / orientations with respect to a fixed coordinate space such as the direction of gravity. Additionally, when the term "substantially" or "approximately" is used in connection with a given measurement, value, or characteristic, it refers to an amount that is within the normal operating range to achieve a desired result, but includes some variation (e.g., 1-5 percent) due to inherent uncertainties and errors within the tolerances of the system.Accordingly, the description herein is intended to be taken by way of example only and is not intended to otherwise limit the scope of the present invention.

[0046] In the following, exemplary embodiments of the present invention are presented, each of which is made up of a combination of various elements. 1. A vision system comprising a 3D camera assembly arranged as an area scan sensor and a vision system processor that receives 3D data from images of an object acquired within a field of view (FOV) of the 3D camera assembly, the object being conveyed through the FOV in a conveying direction, the object defining a total length between opposing edges of the object in the conveying direction, the total length being longer than the FOV, and the FOV defining a usable region of interest (ROI); a dimensioning processor that measures the total length based on motion tracking information derived from transport of the object through the FOV in combination with multiple 3D images of the object acquired sequentially by the 3D camera assembly with a predetermined transport momentum between the multiple 3D images; a presence detector associated with the FOV, the presence detector providing a presence signal when the object is positioned adjacent thereto; the dimensioning processor is configured to determine, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; the dimensioning processor is configured to determine, in response to information relating to features on the object, whether the object is longer than the FOV as the object moves in the conveying direction; the dimensioning processor is configured to determine a length LR of the usable ROI based on the motion tracking information; if the length LR of the available ROI is greater than the predetermined transfer momentum between the plurality of 3D images, the dimensioning processor is configured to determine an overlap of the plurality of 3D images such that the opposing edges are included in the overlapping 3D images; A vision system in which the vision system processor combines information about features on the object from successive image captures by the 3D camera assembly that generate a collection of feature data and determines the dimensions of the object as a whole, rather than combining separate individual images into an overall image. 2. The vision system of claim 1, wherein the vision system processor is configured to, in response to the object having a total length greater than the FOV, take a series of image capture snapshots while the object remains within the FOV until the object exits the FOV. 3. The vision system of claim 2, wherein the vision system processor is configured to use the collection of feature data and based on input application data to derive overall attributes of the object, the overall attributes including at least one of the classification of the object, the dimensions of the object, the skew and the volume of the object. 4. The vision system of claim 3, further comprising an object manipulation process that performs a task on the object based on the overall attributes, the task including at least one of transferring the object, rejecting the object, issuing a warning, and correcting the object's skew. 5. The vision system of any one of items 1 to 4 above, wherein the object is transported by a conveyor or manually. 6. The vision system of claim 5, wherein said motion tracking information is generated by an encoder operatively connected to said conveyor. 7. A vision system according to any one of claims 1 to 6, wherein the presence signal is used by the dimensioning processor to determine the continuity of the object between each of the plurality of 3D images as the object moves in the conveying direction. 8. The vision system of claim 7, wherein the plurality of 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and further comprising a removal process that uses the motion tracking information to remove the overlap from the dimensions of the object to determine the total length. 9. The vision system of claim 8, further comprising an image rejection process that rejects the last of the plurality of 3D images acquired as a result of the presence signal being asserted after the previous of the plurality of 3D images contained the trailing edge of the object. 10. A vision system of any one of 1 to 9 above, wherein the dimensioning processor is configured to use information relating to features on the object to determine continuity of the object between each of the multiple 3D images as the object moves in the conveying direction. 11. A vision system according to any one of claims 1 to 10, wherein the dimensioning processor defines a minimum spacing between the objects in the multiple 3D images, below which the multiple objects are considered to be a single object with lost 3D image data. 12. The vision system processor: (a) Data outside the length limit, (b) Data outside the width limit, (c) Data outside the height limit, (d) Data outside the volume limit; (e) liquid volume, (f) classification; (g) the quantity (QTY) of data actually imaged from said object relative to the expected data imaged from said object; (h) the location characteristics of said object; or (i) detecting damage to said object; 12. The vision system of any one of claims 1 to 11, configured to generate the collection of feature data for the object, comprising: 13. A method for dimensioning an object using a vision system having a 3D camera assembly arranged as an area scan sensor and a vision system processor that receives 3D data from images of the object acquired within a field of view (FOV) of the 3D camera assembly, the object being conveyed through the FOV in a conveying direction, and the object defining a total length between opposing edges of the object in the conveying direction, the total length being longer than the FOV, the FOV defining a usable region of interest (ROI), comprising: measuring the total length based on motion tracking information derived from transport of the object through the FOV in combination with a plurality of 3D images of the object acquired successively by the 3D camera assembly with a predetermined transport momentum between the plurality of 3D images; generating a presence signal when the object is positioned adjacent to the FOV, and determining, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; determining whether the object is longer than the FOV as the object moves in the conveying direction in response to information relating to features on the object; determining a length LR of the usable ROI based on the motion tracking information, and determining an overlap of the plurality of 3D images such that the opposing edges are included in overlapping 3D images if the length LR of the usable ROI is greater than the predetermined amount of transported motion between the plurality of 3D images; combining information relating to features on the object from successive acquisitions of images by the 3D camera assembly to generate a collection of feature data that provides dimensions of the object as a whole rather than combining separate individual images into an overall image. 14. The method of claim 13, further comprising, in response to the total length of the object being greater than the FOV, taking a series of image capture snapshots while the object remains within the FOV until the object exits the FOV. 15. The method of claim 14, further comprising deriving global attributes of the object using the collection of feature data and based on input application data, the global attributes including at least one of the classification of the object, the size of the object, the skew and the volume of the object. 16. The method of claim 15, further comprising performing a task on the object based on the overall attributes, including at least one of forwarding the object, rejecting the object, issuing a warning, and correcting skew of the object. 17. A method according to any one of claims 13 to 16, wherein the presence signal is used to determine the continuity of the object between each of the plurality of 3D images as the object moves in the conveying direction. 18. The step of combining the information comprises: (a) Data outside the length limit, (b) Data outside the width limit, (c) Data outside the height limit, (d) Data outside the volume limit; (e) liquid volume, (f) classification; (g) the quantity (QTY) of data actually imaged from said object relative to the expected data imaged from said object; (h) the location characteristics of said object; or (i) detecting damage to said object; 18. The method of any one of items 13 to 17, comprising producing 19. A method according to any one of claims 13 to 18, further comprising specifying a minimum spacing between the objects in the plurality of 3D images, below which the plurality of objects are considered to be a single object with lost 3D image data. 20. A vision system comprising a 3D camera assembly arranged as an area scan sensor and a vision system processor receiving 3D data from images of an object acquired within a field of view (FOV) of said 3D camera assembly, said object being conveyed in a conveying direction through said FOV, and said object defining an overall length in said conveying direction that is greater than said FOV; a dimensioning processor that measures the total length based on motion tracking information derived from transport of the object through the FOV in combination with multiple 3D images of the object acquired sequentially by the 3D camera assembly with a predetermined transport momentum between the multiple 3D images; a presence detector associated with the FOV, the presence detector providing a presence signal when the object is positioned adjacent thereto; the dimensioning processor is configured to determine, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; the dimensioning processor is configured to determine, in response to information relating to features on the object, whether the object is longer than the FOV as the object moves in the conveying direction; the presence signal is used by the dimensioning processor to determine continuity of the object between each of the plurality of 3D images as the object moves in the conveying direction; a subtraction process in which the plurality of 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and the motion tracking information is used to subtract the overlap from the object dimensions to determine the total length; an image rejection process that rejects a last one of the plurality of 3D images acquired as a result of the presence signal being asserted after a previous one of the plurality of 3D images contained the trailing edge of the object; A vision system in which the vision system processor combines information about the features on the object from successive image captures by the 3D camera assembly that generate a collection of feature data and determines the dimensions of the object as a whole, rather than combining separate individual images into an overall image.

Claims

1. a 3D camera assembly having a field of view (FOV) defining a usable region of interest (ROI), an area scan sensor configured to acquire a 3D image of an object moving through the FOV, the object defining a total length between opposing edges of the object that is longer than the FOV; and a vision system processor in communication with the area scan sensor, the vision system processor comprising: receiving a plurality of 3D images from the area scan sensor having a known amount of object movement between image captures; determining the total length of the object based on motion tracking information derived from movement of the object through the FOV in combination with the plurality of 3D images; receiving a presence signal indicative of the object being positioned adjacent to the FOV; and determining, in response to the presence signal, whether the object appears in more than one image as the object moves through the FOV; determining whether the object is longer than the FOV as the object moves through the FOV in response to information relating to features on the object; determining a length LR of the usable ROI based on the motion tracking information, and if the length LR of the usable ROI is greater than a known amount of the object movement between the plurality of 3D images, determining the overlap of the plurality of 3D images such that the opposing edges are included in the overlap; A 3D camera assembly that combines information about features on the object from the multiple 3D images to generate a collection of feature data that provides dimensions of the object without combining separate individual images into an overall image.

2. 2. The 3D camera assembly of claim 1, wherein the vision system processor is configured to, in response to the object having a total length greater than the FOV, take a series of image capture snapshots while the object remains within the FOV until the object exits the FOV.

3. 3. The 3D camera assembly of claim 2, wherein the vision system processor is configured to use the collection of feature data and based on input application data to derive global attributes of the object, the global attributes including at least one of a confidence score, a classification of the object, a dimension of the object, a skew, or a volume of the object.

4. 4. The 3D camera assembly of claim 3, further comprising an object manipulation process that performs a task on the object based on the overall attributes, including at least one of transferring the object, rejecting the object, issuing a warning, or correcting skew of the object.

5. 5. The 3D camera assembly of claim 1, wherein the object is transported by a mechanical conveyor or manually.

6. 6. The 3D camera assembly of claim 5, wherein the motion tracking information is generated by an encoder operatively connected to the mechanical conveyor, and wherein a motion detection device is operatively connected to the mechanical conveyor, an external feature detection device, or a feature-based detection device.

7. 7. The 3D camera assembly of claim 1, wherein the presence signal is used by a dimensioning process to determine the continuity of the object between each of the plurality of 3D images as the object moves in a direction of motion.

8. 8. The 3D camera assembly of claim 7, wherein the plurality of 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and the vision system processor is further configured to remove the overlap from the dimensions of the object to determine the total length.

9. 9. The 3D camera assembly of claim 8, wherein the vision system processor is further configured to employ an image rejection process that rejects a final one of the plurality of 3D images acquired as a result of the presence signal being asserted after a previous one of the plurality of 3D images included the trailing edge of the object.

10. 10. The 3D camera assembly of claim 1, wherein the vision system processor is further configured to use a dimensioning process that uses information related to features on the object to determine continuity of the object between each of the plurality of 3D images as the object moves in a direction of motion.

11. 11. The 3D camera assembly of claim 1, wherein the vision system processor is further configured to use a dimensioning process that defines a minimum spacing between the objects in the multiple 3D images, below which the multiple objects are considered to be a single object with lost 3D image data.

12. the vision system processor (a) Data outside the length limit, (b) Data outside the width limit; (c) Height limit outside data, (d) data outside the volume limit; (e) reliability score; (f) liquid volume; (g) classification; (h) the quantity of data actually imaged from said object relative to the expected data imaged from said object (QTY); (i) the positional characteristics of the object; or (j) detecting damage to said object; 12. The 3D camera assembly of claim 1, configured to generate the collection of feature data for the object, comprising:

Images