System and method for three-dimensional scan of moving objects longer than the field of view

Abstract

To provide a system and method for using an area scan sensor of a vision system, in conjunction with an encoder or other knowledge of motion, to capture accurate measurement of an object larger than a single field of view (FOV) of the sensor.SOLUTION: Logic automatically determines if an object 120 is longer than the FOV, and thereby moves / conveys the object 120 until the object is no longer present in the FOV, and causes a sequence of image acquisition snapshots to occur while the object 120 remains within the FOV. At that point when the object is no longer present, acquisition is ended and the individual images are combined as segments in an overall image. These images can be processed to derive overall dimensions of the object based on input application details.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This invention relates to machine vision systems that analyze objects in three-dimensional (3D) space, and more particularly to systems and methods for analyzing objects transported through an inspection zone on a conveyor. [Background technology]

[0002] Machine vision systems (also referred to herein as "vision systems") that measure, inspect, align objects and / or decode symbolic information (e.g., bar codes, also referred to as "ID codes") are used in a wide variety of applications and industries. These systems are based on the use of image sensors that capture images (typically grayscale or color, one-, two- or three-dimensional) of targets or objects, and process the images using an on-board or interconnected vision system processor. The processor typically includes both processing hardware and non-transitory computer-readable program instructions, and executes one or more vision system processes to generate a desired output based on the image processing information. This image information is typically presented in an array of image pixels, each having a different color and / or intensity.

[0003] As noted above, one or more vision system cameras can be positioned to capture two-dimensional (2D) or three-dimensional (3D) images of objects in an imaged scene. A 2D image is typically characterized as a pixel having an x- and y-component within a total N×M image array (often defined by a pixel array of the camera's image sensor). If an image is captured in 3D, then in addition to the x- and y-components there is a height, or z-axis component. 3D image data can be captured using a variety of mechanisms / techniques, such as stereo camera triangulation, LiDAR (Light Detection and Ranging), time-of-flight sensors, and laser displacement profiling (for example).

[0004] A 3D camera is typically positioned to capture 3D image information for objects that fall within its field of view (FOV), which constitutes a volumetric space that fans outward along the horizontal x and y dimensions as a function of distance from the camera sensor in the vertical z dimension. Sensors that acquire images of the entire volumetric space in parallel / simultaneous (i.e., in "snapshots") are called "area scan sensors." Such area scan sensors are distinct from line scan sensors (e.g., profilers) that capture 3D information slice by slice, and use motion (e.g., conveyor movement) and measurement of this motion (e.g., by motion encoders or steppers) to move objects through the inspection field / FOV.

[0005] One advantage of line-scan sensors is that the object being inspected can be of any length, provided that the length of the object is taken along the direction of conveyor motion. Conversely, area-scan sensors, which take image snapshots of a volumetric space, do not require an encoder to capture the 3D scene, but if the object is longer than the field of view, the entire object cannot be imaged in one snapshot. If only part of the object is captured in one snapshot, an additional snapshot (or multiple snapshots) must be taken for the remaining length as the rest of the object (not yet imaged) passes into the FOV. With multiple snapshots, the challenge is how to efficiently align (stitch) the multiple 3D images together so that the entire 3D image accurately represents the object's features. Summary of the Invention

[0006] The present invention overcomes the deficiencies of the prior art by providing a system and method for using an area scan sensor of a vision system in conjunction with an encoder or other knowledge of motion to capture accurate measurements of objects larger than a single field of view (FOV) of the sensor. The system and method specifically address the deficiencies that snapshot-based area scan vision systems define a limited field of view, which typically requires the system to take multiple snapshots and other data required to combine the snapshots. This avoids the potentially computationally intensive task of combining raw image data and then post-processing such data. Instead, the exemplary embodiment provides a lightweight way to identify object features / borders or edges (also referred to as "vertices" in reference to identified polygonal shapes) and track them from image to image, thereby processing the entire extent of the object for purposes of displaying or measuring its size and position in space (dimensioning). The system and method may employ logic to automatically determine if the object is longer (in the transport direction) than the FOV, thereby resulting in a series of image capture snapshots while the moving / transporting object remains within the FOV until the object is no longer present in the FOV. Once no longer present, acquisition ends and the individual images can optionally be combined as segments into the overall image. The entire image data can be used in various downstream processes. Feature data collected from the individual image snapshots, derived with or without the generation of an actual overall image, can be processed to derive the overall dimensions of the object based on input application details.The feature data so collected can be used to determine other properties and characteristics of the object, such as, but not limited to, skew, length, width and / or height tolerance exceedance, confidence score, liquid volume, categorization, quantity of data actually imaged from the object versus expected data imaged from the object (QTY), positional characteristics of the object, and / or detection of damage to the object. The system and method can also combine complex / multi-level objects that would typically be separated by conventional imaging using area scan sensors due to loss of 3D data in shadows, etc.

[0007] In an exemplary embodiment, a vision system and a method for using the same are provided. The vision system and method can include a 3D camera assembly arranged as an area scan sensor and a vision system processor that receives 3D data from images of an object acquired within a field of view (FOV) of the 3D camera assembly. The object can be conveyed through the FOV in a conveying direction, and the object can define an overall length in the conveying direction that is longer than the FOV. A dimensioning processor measures the overall length based on motion tracking information derived from conveying the object through the FOV combined with multiple 3D images of the object. These images can be acquired sequentially by the 3D camera assembly with a predetermined conveying motion between the 3D images. A presence detector associated with the FOV can provide a presence signal when the object is positioned adjacent to it. The dimensioning processor can be configured to determine whether the object appears in more than one image as it moves in the conveying direction in response to the presence signal. The dimensioning processor can be configured to determine whether the object is longer than the FOV as it moves in the conveying direction in response to information related to features on the object. An image processor can combine information about features about the object from successive image captures by the 3D camera that generate a collection of feature data to determine the dimensions of the object as a whole, rather than combining separate individual images into an overall image. By way of example, the image processor can be configured to take a series of image capture snapshots while the object remains within the FOV, in response to a total length of the object being greater than the FOV, until the object moves out of the FOV. The image processor can be further configured to use the collected feature data and based on the inputted application data to derive global attributes of the object, where the global attributes include at least one of a confidence score, a classification of the object, a dimension of the object, a skew, and a volume of the object.The object manipulation process can perform tasks on the object based on the global attributes, including at least one of transferring the object, rejecting the object, issuing a warning, and correcting the skew of the object. Illustratively, the object can be conveyed by a mechanical conveyor or manually, and / or the tracking information can be generated by an encoder operatively connected to the conveyor. The motion detection device can be operatively connected to the conveyor, an external feature detection device, and / or a feature-based detection device. The presence signal can be used by the dimensioning processor to determine the continuity of the object between each of the images as the object moves in the conveying direction. The images can be acquired by a 3D camera with a predetermined overlap between each other, and the removal process can use the tracking information to remove the overlap from the dimensions of the object to determine the overall length. The image rejection process can reject the last of the acquired images as a result of the presence signal being asserted after a previous one of the images included the trailing edge of the object. The dimensioning processor can be further configured to use information related to features on the object to determine the continuity of the object between each of the images as the object moves in the conveying direction. The dimensioning system may further define a minimum spacing between objects in the image below which the multiple objects are considered to be a single object with missing 3D image data. Illustratively, the image processor may be configured to generate a collection of feature data for the object related to: (a) out of length limit data, (b) out of width limit data, (c) out of height limit data, (d) out of volume limit data, (e) confidence score, (f) liquid volume, (g) classification, (h) quantity of data actually imaged from the object versus expected data imaged from the object (QTY), (i) positional characteristics of the object, and / or (j) detection of damage to the object.

[0008] In an exemplary embodiment, the vision system, and associated methods, may include a 3D camera assembly arranged as an area scan sensor with a field of view (FOV) capable of performing an acquisition process that captures one or more images of an object as the object passes through the FOV, and (a) determines if the object occupies more than one image, (b) determines when the object no longer occupies the next image, and (c) calculates the size and relative angle of the object from the acquired one or more images.

[0009] In an exemplary embodiment, a method for displaying or measuring (dimensioning) a size and position in space of an object using a vision system having a 3D camera assembly arranged as an area scan sensor is provided, where a vision system processor receives 3D data from images of the object captured within a field of view (FOV) of the 3D camera assembly. The object is transportable through the FOV in a conveying direction, and the object can define an overall length in the conveying direction that is longer than the FOV. The method further includes measuring the overall length based on motion tracking information derived from the conveying of the object through the FOV in combination with multiple 3D images of the object captured successively by the 3D camera assembly with a predetermined amount of conveying motion between the 3D images. A presence signal can be generated when the object is positioned adjacent the FOV, and in response to the presence signal, it can be determined whether the object appears in more than one image as the object moves in the conveying direction. Depending on information related to features on the object, it can be determined whether the object is longer than the FOV as the object moves in the conveying direction. Information relating to features on the object from successive captures of images by the 3D camera can be combined to generate a collection of feature data to provide dimensions of the object as a whole, rather than combining separate individual images into an overall image. Illustratively, in response to the object's overall length being greater than the FOV, a series of image capture snapshots can be taken while the object remains within the FOV until the object exits the FOV. Using the collected feature data and based on the inputted application data, global attributes of the object can be derived, which can include at least one of a confidence score, a classification of the object, object dimensions, skew, and object volume. Illustratively, the method can perform tasks related to the object, including at least one of forwarding the object, rejecting the object, issuing a warning, and correcting the skew of the object, based on the global attributes.Illustratively, the presence signal can be used to determine the continuity of the object between each of the images as the object moves in the conveying direction. In combining the information, the method can further generate (a) out of length limit data, (b) out of width limit data, (c) out of height limit data, (d) out of volume limit data, (e) confidence score, (f) liquid volume, (g) classification, (h) quantity of data actually imaged from the object versus expected data imaged from the object (QTY), (i) positional characteristics of the object, and / or (j) detection of damage to the object. A minimum spacing between objects in the image can be defined below which the imaged objects are considered to be a single object with missing 3D image data. [Brief description of the drawings]

[0010] The following description of the invention refers to the accompanying drawings, in which:

[0011] FIG. 1 is a schematic of a system for acquiring and processing 3D images of objects longer than the field of view using an area scan sensor, which uses a dimensioning process (processor) to generate an overall image from multiple snapshots as the object moves through the field of view;

[0012] FIG. 2 is a side view of the conveyor and area scan sensor arrangement of FIG. 1 showing details of a FOV, region of interest (ROI) usable for 3D imaging of an object trigger plane for image acquisition according to an exemplary embodiment;

[0013] FIG. 3 is a diagram illustrating the operational behavior of the configurations of FIGS. 1 and 2 for an exemplary object having a length in the transport direction greater than the available ROI that has reached the trigger plane to trigger (cause) a first snapshot;

[0014] FIG. 4 is a diagram illustrating the driving action of FIG. 3 in which an exemplary object passes partially out of the ROI, indicating sufficient motion of the object for the encoder to trigger a second snapshot;

[0015] FIG. 5 is a diagram illustrating operational operation of the configurations of FIGS. 1 and 2 for an example object exhibiting a complex upper surface that would result in loss of 3D image data and / or showing two separate objects in the FOV;

[0016] FIG. 6 is a flow diagram illustrating a procedure for performing a dimensioning process according to an exemplary embodiment, including when multiple objects and / or complex objects are imaged;

[0017] FIG. 7 is a top-down view of an exemplary imaged conveyor surface with overlapping image portions representing the entire imaged object; and

[0018] FIG. 8 is a flow diagram of the overall image acquisition procedure for the configurations of FIGS. 1 and 2 adapted to handle overlapping image portions as shown in FIG. 7 to generate the full length of the object, as well as maximum width, maximum height and relative angles with respect to the conveying direction (or other coordinate system). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] I. System Overview

[0020] FIG. 1 shows a schematic of an arrangement 100 in which a vision system camera assembly (also referred to simply as a "camera" or "sensor") 110 captures 3D image data of an exemplary object 120 as it passes below a field of view (FOV) with respect to a moving conveyor 130 moving in a downstream direction (arrow 132). A local / relative 3D coordinate system 138 is shown with xyz Cartesian coordinates by way of example. Other coordinate systems, such as a polar coordinate system, may also be used to represent this 3D space. It should be noted that in this example, the object 120 defines an upstream-to-downstream length LO (roughly along the y-axis / direction of movement) that is longer than the upstream and downstream boundaries BF and BR of the FOV, which are described in more detail below.

[0021] The 3D camera / imaging assembly 110 discussed herein can be any assembly that captures a 3D image of an object, including but not limited to stereo cameras, time-of-flight cameras, LiDAR, ultrasonic ranging cameras, structured lighting systems, and laser displacement sensors (profilers), and thus the term 3D camera should be broadly understood to include these systems and any other system that generates height information in conjunction with a 2D image of an object. It can also comprise a camera or an array of cameras, and the terms "camera" and / or "camera assembly" can refer to one or more cameras that capture an image (or images) in a manner that generates the desired 3D image data for the scene. The illustrated camera assembly 110 is shown as being mounted over the surface of the conveyor 130, such as a checkpoint or inspection station, to image objects as they pass by. In this embodiment, the camera assembly 110 defines an optical axis OA that is approximately perpendicular (along the z-axis) to the surface of the conveyor 130. Other non-perpendicular directions of axis OA relative to the conveyor surface are also explicitly contemplated. Depending on the speed of conveyor movement and the acquisition time of the camera's image sensor S and associated electronics 110 (depending in part on the frame rate and aperture settings), the object 120 can remain in motion (typically) or can stop for a moment for imaging. The camera 110 captures a 3D image when the object 120 is sufficiently within the FOV, which can be triggered by a photodetector or other triggering mechanism 136, which sends a trigger signal to the camera 110 and associated processor (or processors). Thus, the camera assembly 110 in this embodiment is arranged as an area scan sensor.

[0022] The camera assembly 110 includes an image sensor S adapted to generate 3D image data 134. The camera assembly also (optionally) includes an integral illumination assembly I, e.g., a ring illuminator of LEDs that projects light in a predictable direction relative to the axis OA. In an alternative configuration, external illumination (not shown) can be provided. A suitable optical package O is shown in optical communication with the sensor S along the axis OA. The sensor S is in communication with an internal and / or external vision system process (processor) 140 that receives image data 134 from the camera 110 and performs various vision system tasks on the data in accordance with the systems and methods of the present application. The process (processor) 140 includes underlying process / processors or functional modules, such as a set of vision system tools 142, which can include various standard and custom tools that identify and analyze features in the image data, including, but not limited to, edge detectors, blob tools, pattern recognition tools, deep learning networks, ID (e.g., barcode) detectors and decoders, and others. The vision system process (processor) 140 may further include a dimensioning process (processor) 144 in accordance with the present system and method. This process (processor) 144 may perform various analysis and measurement tasks on features identified in the 3D image data to determine the presence of specific features and to compute further results therefrom. According to an exemplary embodiment, the process (processor) interfaces with a variety of conventional and custom (e.g., 3D) vision system tools 142.

[0023] System configuration and display of results may be handled by a separate computing device 150, such as a server (e.g., cloud-based or local), PC, laptop, tablet, and / or smartphone. Computing device 150 is depicted (by way of non-limiting example) with a conventional display or touch screen 152, keyboard 154, and mouse 156, which collectively provide a graphical user interface (GUI) function. Alternative implementations of device 150 may include a variety of interface devices and / or form factors. According to an exemplary configuration of the present application, the GUI may be driven in part by a web browser application that resides on the device's operating system and displays web pages along with control and data information from process (processor) 140.

[0024] It should be noted that the process (processor) 140 can be fully or partially on-board the housing of the camera assembly 110, and the various process modules / tools 142 and 144 can be instantiated in whole or in part either on the on-board process (processor) 140 or on a remote computing device 150, as appropriate. In an exemplary embodiment, all vision system and interface functions can be instantiated on the on-board process (processor) 140, and the computing device 150 can be used primarily for training, monitoring and related operations on the interface web pages (e.g., HTML) generated by the on-board process (processor) 140 and communicated to the computing device via a wired or wireless network link. Alternatively, all or part of the process (processor) 140 can reside within the computing device 150. Results of the analysis by the processor can be communicated to a downstream utilization device or process 160. Such device / process can use the results 162 to manipulate the object / package, for example, gating the conveyor 130 to direct the object to a different destination based on the analyzed characteristics and / or rejecting defective objects.

[0025] The camera assembly includes on-board calibration data, established by factory and / or field calibration procedures, that maps the x, y, and z coordinate axes of the imaged pixels to the camera's coordinate space. This calibration data 170 is provided to the processor and used to analyze the image data. Also, in an exemplary embodiment, the conveyor and / or its drive mechanism (e.g., stepper motor) includes an encoder or other motion tracking mechanism 180 to report relative motion data 182 to the processor 140. The motion data can be communicated in a variety of ways, such as distance-based pulses, each of which defines a predetermined increment of conveyor movement. The pulses can be summed to determine the total movement within a given period of time.

[0026] It should be noted that, as used herein, the term "conveyor" should be understood broadly to include configurations in which objects are passed through the FOV by another technique, such as manual manipulation. Thus, an "encoder," as defined herein, can be any acceptable motion measurement / tracking device, including steppers, mark readers, and / or devices that track features or fiducials on the conveyor or objects as they pass through the FOV, including a variety of internal techniques that use knowledge of the underlying vision system application (e.g., of features in the image) to determine the degree of movement between image snapshots.

[0027] II. Dimensioning Process (Processor)

[0028] A. Settings

[0029] FIG. 2 shows a configuration 200 set up adapted to image objects longer than the FOV in the direction of motion 132. The camera assembly 110 is shown covering a portion of the conveyor 130 that forms an inspection station for such objects. The conveyor 130 transmits a resettable encoder count 210 for each trigger event at which the conveyor 130 is depicted in the area of ​​the 3D camera assembly. A trigger (in this case the trigger state 220 is negative, with a red or other color / shade indicator represented by the point cloud shading) is fired each time an object passes through a trigger plane 230 that is aligned with the presence detector 136 (FIG. 1) described above. The 3D FOV defines a usable region of interest (ROI) 250 that extends from the surface of the conveyor 130 along the camera axis OA and upwards on either side of the axis OA. In this example, the ROI defines a usable height that is user definable based on the maximum expected object height. The points 256 and 258 where the top 254 of the ROI 250 intersects with the front and rear boundaries BF and BR of the FOV, respectively, essentially define the usable length LR of the ROI in the direction of motion 132 (y-axis). Thus, the higher the maximum height HR of the ROI, the shorter the length LR.

[0030] B. Object size estimation and trigger / encoder logic

[0031] Once the usable ROI is defined, the processor determines how many pulses the conveyor will travel to reach the length LR. Referring to FIG. 3, the system is shown operating in a run-time mode to estimate the size (length) of the object 120, which is depicted with its leading edge 310 reaching the trigger plane 230. The trailing edge 320 is outside the trailing boundary 340 of the usable ROI 250. At this point, the 3D camera assembly is triggered to take a first image (snapshot) of the object, and the encoder count 330 is set to zero in the processor. The snapshot is analyzed to determine (using vision system tools, height change, etc.) whether the trailing edge of the object is within the ROI. If the object 120 ends within one ROI 250, the snapshot is reported as being a complete 3D image for the object, and the system waits for the next object / trigger. Conversely, if the object 120 does not appear to end within the usable ROI (e.g., there is no substantial change in the object geometry within the rear boundary 340 of the ROI 250), i.e., as in FIG. 3, the system stores the first image and counts encoder pulses for one ROI length (LR) (or a known fraction of this length LR). This count is referenced in FIG. 4, where the rear end 320 of the object has just passed into the usable ROI 250 (past the rear ROI boundary 340). At this point (shown in FIG. 4), pulses for one ROI length LR have been counted. In this example, the pulse count 430 is 300 mm. This length is equal to or less than the length LR. As will be explained below, if overlap between snapshot images is desired, this pulse count can be less than the length LR. This ensures that details at the edges of the images are not lost. As shown in FIG. 4, the trigger state 420 is also positive at this stage (represented by a green or another color / shade and by hatching shading) since the object is present in the trigger plane 230.The logical combination of these events (a positive trigger state and a full encoder count) triggers the camera assembly to take a second snapshot of object 120. This second snapshot is also stored in association with the object from the first snapshot.

[0032] The system analyzes the second snapshot to determine whether the trailing edge 320 of the object 120 is now downstream of the rear ROI boundary 340. If so, then the entire length of the object has been imaged, and the two snapshots can be combined and processed as described further below. If the trailing edge of the object is not within the ROI of the second snapshot, the encoder count is reset, the trigger state 420 remains high since the object continues to be present, and the system counts until the next ROI length LR is reached. Another snapshot is taken, and the process is repeated until the nth snapshot, in which the trailing edge of the object is finally detected within the ROI 250. At this point, the object has passed completely out of the FOV, so the trigger state goes low, and the imaging result is communicated. Special cases sometimes exist due to overlap between snapshot images, but these can be handled by evaluating the encoder count between separate image acquisitions. If a previous snapshot imaged an object edge and reported dimensions, but the trigger state is still high (positive), and a new snapshot is taken, and in this case the encoder count for the current snapshot is equal to the length of the ROI, the new snapshot is discarded and no dimensions are reported. This is done because it is a redundant / unused snapshot resulting only from overlap between snapshot images. In general, all snapshots can be combined and processed in relation to one object image, as described below.

[0033] C. Complex and / or multi-layered objects

[0034] In some embodiments of run-time operation, the object may have a complex 3D shape. Referring to FIG. 5, an object 520 with different heights 522 and / or overhanging (occlusion) features 525 along its upper surface is shown reaching the trigger plane 230. At this point, the trigger state 540 becomes positive (green, represented by hatched shading) and the encoder count 530 starts at 0. However, this complex shape of the object may result in loss of 3D image data or confuse the vision system processor as to the actual location of the object's trailing edge 526, which in this example is located outside the ROI 250 still available when the first snapshot is taken by the camera assembly 110. Referring also to the procedure 600 of FIG. 6, the object first triggers the camera assembly to take a 3D snapshot in step 610. There may be a single object (simple or complex shaped) or multiple objects in this first image. The vision system analyzes the image to determine if multiple objects are detected, typically by looking for gaps between boundary portions that extend to the conveyor surface (reference line) and between separated boundary portions. If two objects are not detected, the vision system treats the object as a single object (decision step 620). Conversely, if two or more objects are detected, decision step 620 of procedure 600 branches to a further decision step 630, where the gap (or gaps) are analyzed to determine if the gap (or gaps) is smaller than a minimum distance (determined using camera calibration data and conventional measurement techniques). If the gap is larger than a minimum distance - set by the user or automatically - the system indicates that there are multiple objects in the image (step 640).For two objects with a greater than minimum gap distance (step 640) or for a single object (decision step 620), procedure 600 branches to decision step 650 where the vision system determines whether all object edges are in the image and the trigger condition goes negative. If so, procedure 600 branches to step 660 where the imaging results are output and the encoder is reset to await the next object / trigger to occur. Conversely, if the trigger condition is not negative after the current (first) image and the trailing edge of the object (or objects) remains outside the FOV, decision step 650 branches to step 670 where the camera assembly counts encoder pulses and takes another snapshot of the FOV. This continues until the trigger condition goes negative and the trailing edge of the object is detected.

[0035] Referring back to decision step 630, if a gap exists but is less than the minimum distance, procedure 600 presumes that the image contains a single object and that the gap is the result of missing or missing 3D data in the imaged object. Thus, the object is treated as if it were a single object, and the decision step branches to a further decision step 650 (described above), where the presence or absence of the object's trailing edge in the image determines the next step. It should be noted that one technique for predicting and providing missing or missing 3D image data is described in co-pending U.S. Provisional Application No. 62 / 972,114, entitled "Composite Three-Dimensional Blob Tool and Method of Operation Thereof," filed February 10, 2020 and assigned to the present applicant, the teachings of which are incorporated herein by reference as useful background information. Such a blob tool can be used to inform further utilization steps in the generation of imaging results.

[0036] D. Image Data Tracking and Resulting Overlap

[0037] Referring to FIG. 7, a continuous top-down (xy-plane) view 710 of a conveyor belt 720 constructed from (for example) three consecutive image acquisitions in the presence of a long object is shown. Three acquired images 730, 732, and 734 of the entire object are shown in sequence. In particular, the encoder counts are set such that a first predetermined overlap distance D1 exists between a first pair of images 730 and 732, and a second predetermined overlap distance D2 exists between a second pair of images 732 and 734. The use of overlap distances between object images ensures that acquired object features are not inadvertently lost or obscured at the edges of the 3D ROI. As described below with reference to FIG. 8, the vision system can appropriately remove or combine overlap areas to calculate actual object dimensions and features. It should be noted that because a captured image of an object typically defines one or more polygons with associated vertices that are used to derive the shape's boundary, the term "feature" as used herein should be understood to include, or can be used interchangeably with, the term "vertex."

[0038] In operation, the procedure 800 of FIG. 8 begins in step 810 with the acquisition of a 3D image in the manner described above. That is, a trigger is generated and the encoder is set to count the movement of the object through the FOV. After the count reaches a distance value that allows overlap with the next image, the count is reset in step 820 and the system determines whether it is the last image - the trailing edge of the object is in the ROI. If not (via decision step 830), the procedure 800 branches back to step 810 and another overlapping image is acquired. If the image is the last image in the series of images, the procedure 800 branches to step 840 (via decision step 830) where the series of 3D images (segments within the entire object) are communicated to a dimensioning process (processor) and associated tools to determine the object's dimensions and (optionally) resolve features. Using the distances established by the encoder to the image pixel locations, the xy location and height of the bounding box of each segment of the object are mapped in step 850. Based on this mapping, overlaps (constituting a given known encoder distance between segments) are then removed and the actual object length is calculated. The resulting data can be defined as a feature data collection that effectively combines data from the individual shapes to establish an overall view or characterization of the object in the absence of an actual image of the combined object. This resulting feature data collection can also be used to determine (for example) the maximum width and height of the entire object and its relative angle to the conveyance movement direction. This information is particularly useful in various application processes, such as logistics to ensure proper handling of objects (e.g. packages).

[0039] It should be noted that, as explicitly contemplated by the systems and methods of the present application, generating an actual overall (composite or stitched) image from the separate image captures (snapshots) is optional. The collection of feature data can be used independently of generating an overall image based on N×M pixels, providing suitable results for use in determining object dimensions and / or other processes described below. The overall image can be generated and used in further processes and / or to provide a visual record of all or part of the imaged object, if desired.

[0040] E. Application of Results

[0041] It is contemplated that the resulting feature data (and / or vertex data) collection from the above described operations can be applied to a variety of tasks and functions related to imaged objects, including, but not limited to, streams of packages of different sizes and shapes. In particular, the process herein can be used to determine (for example) the skew angle of an object / package relative to the direction of travel and / or the boundaries of a surrounding support surface. One possible thing that can be identified and measured using the feature data collection herein is the skew angle of the object relative to the direction of travel (and parallel side edges) of a conveyor or other support surface. In addition to potentially causing jams in narrow chutes, gates or other transitions, the skew data can cause an object to appear longer than its normal normal dimensions, causing the system to erroneously generate a defect. The skew angle data must be used so that corrective action (i.e., ignore the erroneous defect or straighten the object) can be taken. In particular, the skew angle information (and / or other measured characteristics) can be part of a metadata tag applied to the result (feature data collection) for use in various downstream operations. Other relevant characteristic data may include out of length limit, out of width limit, out of height limit, and / or out of volume limit values, which indicate that the object is too long, too wide, too tall, or too voluminous for the parameter limits. Further relevant data may relate to, but is not limited to: (a) A reliability score, which can give information on either the shape of the object or the quality of the data received; (b) liquid volume, which can give information about the shape or true (rather than the smallest cube) volume of an object; (c) classification, which can give information on whether the surface shape of an object is flat or not; (d) the quantity of data displayed / imaged (QTY) versus how much is expected to be displayed / imaged; (e) location features (e.g., distance from a reference point such as a corner, center of gravity, edge of a conveyor belt); and / or (f) Damage detection (e.g., presence of bulging / depressed packages based on actual imaged shape versus expected shape).

[0042] The use of feature data collections generally avoids the need to include more detailed pixel-based image data for the combined object, thereby allowing additional processes to be performed on the object. Handling and manipulating such feature data collections allows for faster processing and less processor overhead due to the overall size of the data. Some example tasks can include automating processes in a warehouse, such as rejecting and / or redirecting objects that are not consistent in size and / or shape. Using the data to deflect objects in this way allows a server to avoid oversized objects becoming jammed in a chute or at a bend in the conveyor. Similarly, feature data collections derived by the systems and methods of the present application can assist an automated labeling process to ensure that the location of the object is accurate. Also, as described above, skew information can be used to avoid false defect conditions and / or allow a system or user to orient an object in a conveyor stream, thus avoiding soft jam conditions. Other object / package handling tasks that rely on data can use this data in a manner that is apparent to one of ordinary skill in the art.

[0043] III. Conclusion

[0044] It should be apparent that the system and method described thus far provides an efficient, reliable and robust technique for determining the length of oversized objects that do not fit entirely within the FOV of a 3D area scan sensor along the direction of conveyor travel. The system and method uses conventional encoder data and detector triggers to generate an accurate set of object dimensions, and is operable in the event of lost or missing 3D data due to complex shapes and / or multiple objects being present in the conveyor stream.

[0045] The above is a detailed description of exemplary embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the present invention. Each feature of the various embodiments described above may be combined with features of other described embodiments to produce numerous combinations of features related to new embodiments. Furthermore, although several separate embodiments of the apparatus and method of the present invention have been described above, what has been described herein is merely an exemplary illustration of the application of the principles of the present invention. For example, as used herein, the terms "process" and / or "processor" should be understood broadly to include various electronic hardware and / or software based functions and components (and may alternatively be referred to as functional "modules" or "elements"). Furthermore, the depicted processes or processors may be combined with other processes and / or processors or divided into various partial processes or processors. Such partial processes and / or partial processors may be combined in various ways according to the embodiments of the present application. Similarly, it is expressly contemplated that any functions, processes and / or processors herein may be implemented using electronic hardware, software consisting of a non-transitory computer-readable medium of program instructions, or a combination of hardware and software. In addition, as used herein, various orientation and configuration terms such as "vertical", "horizontal", "upper", "lower", "lower", "top", "side", "front", "rear", "left", "right", etc. are used only as a relative convention and not as an absolute orientation / configuration with respect to a fixed coordinate space such as the direction of gravity. In addition, when the term "substantially" or "approximately" is used with respect to a given measurement, value, or characteristic, it refers to an amount that is within the normal operating range to achieve a desired result, but includes some variation (e.g., 1-5 percent) due to inherent uncertainties and errors within the tolerances of the system.Accordingly, the description herein is intended to be taken as illustrative only and is not intended to otherwise limit the scope of the present invention.

[0046] In the following, exemplary embodiments of the present invention will be described, which are combinations of various elements. 1. A vision system comprising a 3D camera assembly arranged as an area scan sensor and a vision system processor receiving 3D data from an image of an object acquired within a field of view (FOV) of the 3D camera assembly, the object being conveyed through the FOV in a transport direction and the object defining a length between opposing edges of the object in the transport direction, the length being greater than the FOV, the FOV defining a usable region of interest (ROI), a dimensioning processor that measures the total length based on motion tracking information derived from a translation of the object through the FOV in combination with a plurality of 3D images of the object sequentially acquired by the 3D camera assembly with a predetermined translation momentum between the plurality of 3D images; a presence detector associated with the FOV, the presence detector providing a presence signal when the object is positioned adjacent thereto; the dimensioning processor is configured to determine, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; the dimensioning processor is configured to determine, in response to information relating to features on the object, whether the object is longer than the FOV as the object moves in the conveying direction; the dimensioning processor is configured to determine a length LR of the usable ROI based on the motion tracking information; if the length LR of the available ROI is greater than the predetermined transfer momentum between the plurality of 3D images, the dimensioning processor is configured to determine an overlap of the plurality of 3D images such that the opposing edges are included in the overlapping 3D images; A vision system in which the vision system processor combines information about features on the object from successive image captures by the 3D camera assembly that generate a collection of feature data and determines the dimensions of the object as a whole without combining separate individual images into an overall image. 2. The vision system of claim 1, wherein the vision system processor is configured to, in response to a total length of the object being greater than the FOV, acquire a series of image capture snapshots while the object remains within the FOV until the object exits the FOV. 3. The vision system of claim 2, wherein the vision system processor is configured to use the collection of feature data and based on input application data to derive global attributes of the object, the global attributes including at least one of the classification of the object, the dimensions of the object, the skew and the volume of the object. 4. The vision system of claim 3, further comprising an object manipulation process that performs a task on the object based on the overall attributes, the task including at least one of transferring the object, rejecting the object, issuing a warning, and correcting skew of the object. 5. The vision system of any one of claims 1 to 4, wherein the object is transported by a conveyor or manually. 6. The vision system of claim 5, wherein said motion tracking information is generated by an encoder operatively connected to said conveyor. 7. A vision system of any one of claims 1 to 6, wherein the presence signal is used by the dimensioning processor to determine continuity of the object between each of the multiple 3D images as the object moves in the conveying direction. 8. The vision system of claim 7, wherein the multiple 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and further comprising a removal process that uses the motion tracking information to remove the overlap from the dimensions of the object to determine the total length. 9. The vision system of claim 8, further comprising an image rejection process that rejects the last one of the plurality of 3D images acquired as a result of the presence signal being asserted after a previous one of the plurality of 3D images contained the trailing edge of the object. 10. A vision system of any one of 1 to 9 above, wherein the dimensioning processor is configured to use information relating to features on the object to determine continuity of the object between each of the multiple 3D images as the object moves in the conveying direction. 11. A vision system of any one of 1 to 10 above, wherein the dimensioning processor defines a minimum spacing between the objects in the multiple 3D images, below which the multiple objects are considered to be a single object with lost 3D image data. 12. The vision system processor comprises: (a) Data that exceeds the length limit, (b) Data outside the width limit, (c) Data outside height limits; (d) Data outside the volume limit; (e) liquid volume, (f) classification; (g) the quantity of data actually imaged from said object relative to the expected data imaged from said object (QTY); (h) the location characteristics of the object; or (i) detecting damage to said object; 12. The vision system of any one of claims 1 to 11, configured to generate the collection of feature data relating to the object, comprising: 13. A method for dimensioning an object using a vision system having a 3D camera assembly arranged as an area scan sensor and a vision system processor that receives 3D data from images of the object acquired within a field of view (FOV) of the 3D camera assembly, the object being conveyed through the FOV in a conveying direction and the object defining a total length between opposing edges of the object in the conveying direction, the total length being longer than the FOV, the FOV defining a usable region of interest (ROI), comprising: measuring said total length based on motion tracking information derived from a transport of said object through said FOV in combination with said plurality of 3D images of said object successively acquired by said 3D camera assembly with a predetermined transport momentum between said plurality of 3D images; generating a presence signal when the object is positioned adjacent to the FOV, and determining, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; determining whether the object is longer than the FOV as the object moves in the conveying direction in response to information relating to features on the object; determining a length LR of the usable ROI based on the motion tracking information, and determining an overlap of the plurality of 3D images such that the opposing edges are included in overlapping 3D images if the length LR of the usable ROI is greater than the predetermined amount of transport motion between the plurality of 3D images; a 3D camera assembly configured to capture images of an object from a plurality of images captured by the 3D camera assembly, the 3D camera assembly including a plurality of images captured by the 3D camera assembly, the plurality of images being ... 14. The method of claim 13, further comprising, in response to the total length of the object being greater than the FOV, taking a series of image capture snapshots while the object remains within the FOV until the object moves out of the FOV. 15. The method of claim 14, further comprising deriving global attributes of the object using the collection of feature data and based on inputted application data, the global attributes including at least one of the classification of the object, the dimensions of the object, the skew and the volume of the object. 16. The method of claim 15, further comprising performing a task with respect to the object based on the overall attributes, the task including at least one of forwarding the object, rejecting the object, generating a warning, and correcting skew of the object. 17. A method according to any one of claims 13 to 16, wherein the presence signal is used to determine continuity of the object between each of the plurality of 3D images as the object moves in the conveying direction. 18. The step of combining the information comprises: (a) Data that exceeds the length limit, (b) Data outside the width limit, (c) Data outside height limits; (d) Data outside the volume limit; (e) liquid volume, (f) classification; (g) the quantity of data actually imaged from said object relative to the expected data imaged from said object (QTY); (h) the location characteristics of the object; or (i) detecting damage to said object; 18. The method of any one of claims 13 to 17, comprising producing 19. A method according to any one of claims 13 to 18, further comprising specifying a minimum spacing between the objects in the multiple 3D images, below which the multiple objects are considered to be a single object with lost 3D image data. 20. A vision system comprising a 3D camera assembly arranged as an area scan sensor and a vision system processor receiving 3D data from an image of an object acquired within a field of view (FOV) of said 3D camera assembly, said object being conveyed in a conveying direction through said FOV, and said object defining an overall length in said conveying direction that is longer than said FOV; a dimensioning processor that measures the total length based on motion tracking information derived from a translation of the object through the FOV in combination with a plurality of 3D images of the object sequentially acquired by the 3D camera assembly with a predetermined translation momentum between the plurality of 3D images; a presence detector associated with the FOV, the presence detector providing a presence signal when the object is positioned adjacent thereto; the dimensioning processor is configured to determine, in response to the presence signal, whether the object appears in more than one image as the object moves in the conveying direction; the dimensioning processor is configured to determine, in response to information relating to features on the object, whether the object is longer than the FOV as the object moves in the conveying direction; the presence signal is used by the dimensioning processor to determine continuity of the object between each of the plurality of 3D images as the object moves in the conveying direction; a subtraction process in which the plurality of 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and the motion tracking information is used to subtract the overlap from the dimensions of the object to determine the total length; and an image rejection process for rejecting a last one of the plurality of 3D images acquired as a result of the presence signal being asserted after a previous one of the plurality of 3D images contained the trailing edge of the object. A vision system in which the vision system processor combines information about the features on the object from successive image captures by the 3D camera assembly that generate a collection of feature data and determines the dimensions of the object as a whole without combining separate individual images into an overall image.

Claims

1. A 3D camera assembly having a field of view (FOV) that defines a usable region of interest (ROI), comprising: an area scan sensor configured to acquire a 3D image of an object moving through the FOV, the object defining a total length between opposing edges of the object that is longer than the FOV; and and at least one processor in communication with the area scan sensor, the at least one processor comprising: receiving a plurality of 3D images from the area scan sensor having a known amount of object movement between image captures; determining the total length of the object based on motion tracking information derived from movement of the object through the FOV in combination with the plurality of 3D images; receiving a presence signal indicative of the object being positioned adjacent to the FOV, and determining, in response to the presence signal, whether the object appears in more than one image as the object moves through the FOV; determining whether the object is longer than the FOV as the object moves through the FOV in response to information relating to features on the object; determining a length LR of the usable ROI based on the motion tracking information, and if the length LR of the usable ROI is greater than a known amount of the object movement between the plurality of 3D images, determining an overlap of the plurality of 3D images such that the opposing edges are included in the overlap; A 3D camera assembly that combines information about features on the object from the multiple 3D images to generate a collection of feature data that provides dimensions of the object in a manner other than by combining separate individual images into an overall image.

2. A 3D camera assembly as claimed in claim 1, wherein the at least one processor is configured to, in response to a total length of the object being longer than the FOV, obtain a series of image capture snapshots while the object remains within the FOV until the object exits the FOV.

3. The 3D camera assembly of claim 2, wherein the at least one processor is configured to use the collection of feature data and based on input application data to derive global attributes of the object, the global attributes including at least one of a confidence score, a classification of the object, dimensions of the object, skew or volume of the object.

4. The 3D camera assembly of claim 1, further comprising an object manipulation process that performs a task on the object based on the overall attributes, the task including at least one of transferring the object, rejecting the object, generating a warning, or correcting a skew of the object.

5. The 3D camera assembly of claim 1, wherein the object is transported by a mechanical conveyor or manually.

6. The 3D camera assembly of claim 5, wherein the motion tracking information is generated by an encoder connected to operate in conjunction with the mechanical conveyor, and a motion detection device is connected to operate in conjunction with the mechanical conveyor, an external feature detection device or a feature based detection device.

7. A 3D camera assembly as claimed in claim 1, wherein the presence signal is used by a dimensioning process to determine the continuity of the object between each of the multiple 3D images as the object moves in the direction of motion.

8. A 3D camera assembly as claimed in claim 7, wherein the multiple 3D images are acquired by the 3D camera assembly with a predetermined overlap between them, and the at least one processor is further configured to remove the overlap from the dimensions of the object to determine the total length.

9. The 3D camera assembly of claim 8, wherein the at least one processor is further configured to employ an image rejection process to reject a last one of the plurality of acquired 3D images as a result of the presence signal being asserted after a previous one of the plurality of 3D images included the rear end of the object.

10. A 3D camera assembly as claimed in claim 1, wherein the at least one processor is further configured to use a dimensioning process that uses information relating to features on the object to determine continuity of the object between each of the multiple 3D images as the object moves in a direction of motion.

11. The 3D camera assembly of claim 1, wherein the at least one processor is further configured to use a dimensioning process that determines a minimum spacing between the objects in the multiple 3D images, below which the multiple objects are considered to be a single object with 3D image data lost.

12. The at least one processor, (a) Data outside the length limit, (b) Data outside the width limit; (c) Height limit exceeded data; (d) Data outside the volume limit; (e) the reliability score; (f) liquid volume; (g) classification; (h) the amount of data actually imaged from said object relative to the expected data imaged from said object (QTY); (i) a positional feature of the object; or (j) detecting damage to the object; The 3D camera assembly of claim 1 , configured to generate the collection of feature data for the object, the collection comprising:

Images