Composite three-dimensional blob tool and method of operation thereof

CN115362469BActive Publication Date: 2026-10-27COGNEX CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202180025743.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-02-10
Filing Date
2021-02-10
Publication Date
2026-10-27
Estimated Expiration
2041-02-10

AI Technical Summary

Technical Problem

通常,诸如水瓶、镜面状表面或特定空间频率的小特征的复杂对象会导致视觉系统中3D数据捕获的丢失

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115362469B_ABST
    Figure CN115362469B_ABST
Patent Text Reader

Abstract

The present invention provides a system and method for performing 3D imaging of complex objects with possible missing image data. Available 3D image data combined with the absence / loss of image data allows for the calculation of x, y, and z dimensions. The absence / loss of data is assumed to be just other types of image data and represents something that prevents the generation of accurate data in the subject image. Data segments can be connected to the area of missing data and generate a maximum bounding box. The shadow of the object generated can be represented as negative or missing data but does not represent a physical object. The height of the positive data is estimated, the size of the object shadow based on the height, the location in the FOV, and the angle of the light ray generating the image, and the size of the object shadow is removed from the result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a machine vision system for analyzing objects in three-dimensional (3D) space, and more specifically, to a system and method for analyzing objects with complex shapes. Background Technology

[0002] Machine vision systems (also referred to herein as "vision systems") that perform the measurement, inspection, alignment, and / or decoding of symbols (e.g., barcodes, also known as "identity document (ID) codes") of objects are widely used in a variety of applications and industries. These systems are based on the use of image sensors that acquire images of subjects or objects (typically grayscale or color, and one-dimensional, two-dimensional, or three-dimensional) and process these acquired images using onboard or interconnected vision system processors. The processor typically includes processing hardware and non-transitory computer-readable program instructions that perform one or more vision system processes to generate a desired output based on the image processing information. This image information is typically provided within an array of image pixels, each pixel having various colors and / or intensities.

[0003] As described above, one or more vision system cameras can be used to acquire two-dimensional (2D) or three-dimensional (3D) images of objects in an imaging scene. A 2D image is typically characterized as pixels with x and y components throughout the entire NXM image array (typically defined by the pixel array of a camera image sensor). In the case of acquiring images in a 3D manner, in addition to the x and y components, there is also a height or z-axis component. Various mechanisms / techniques, including stereo cameras, LiDAR, time-of-flight sensors, and triangulation (e.g., laser displacement profiles), can be used to acquire 3D image data.

[0004] A typical problem with 3D camera systems when measuring complex arrangements of similar or dissimilar items is that image data can be lost between objects arranged (e.g., in a group). For example, a typical complex object is a bag of water bottles, where the cap might be half the diameter of the bottle or smaller. When viewed with a 3D image sensor, the resulting image data typically comes from the cap, which is fully exposed at the top of the image height. However, useful image data (e.g., the space between the bottle and the cap) is lost between the caps. This can be due to shadows in the lighting and the angle at which light travels from the illuminator to the object and back to the image sensor. Complex objects, such as water bottles, mirrored surfaces, or small features with specific spatial frequencies, often result in a loss of 3D data capture in the vision system. In the water bottle example, the cap / top typically provides a solid response to 3D image data, but the area around the cap may not provide usable image data because the water / plastic / plastic packaging blurs the optical feedback that would normally generate a 3D image. Summary of the Invention

[0005] This invention overcomes the shortcomings of existing technologies by providing a system and method for performing 3D imaging on complex objects where image data may be missing. The system and method combine the missing / absent image data with available 3D image data and can treat the entire set of current and missing image data as a single object, allowing for precise calculation of the x, y, and z dimensions of that single object. Typically, the system and method herein assume that missing / absent image data is simply other types of data, and that the absence / absence of image data indicates the presence of something preventing the generation of accurate 3D image data in the subject image. Using this assumption, image data segments can be concatenated to regions of missing data, thereby enabling the generation of maximum bounding boxes. For tall and complex objects, the shadows generated by the object can also be represented as negative or missing data, but not representing the physical object. Using height data from the positive data, the object shadow size, position in the field of view, and ray angles of the generated image are estimated based on that height and removed from the result to provide more accurate 3D image results.

[0006] In an illustrative embodiment, a composite 3D binary large object (blob) tool and its operation method may be provided for a vision system. The vision system may have a processor that uses a 3D camera component to receive acquired 3D image data from an object. The tool and associated method may include blob tool processing that identifies positive and negative 3D image data in the acquired 3D image data and combines the positive and negative 3D image data that define z-dimensional height information to define at least one connected object. The negative 3D image data may at least partially include orthogonal xy dimensions defining data gaps about the object and orthogonal shadows about the object. The z-dimensional height information may be used to refine the xy dimensions of the object based on (1) knowledge of ray angles projected by illumination associated with the 3D camera component and received by an image sensor of the 3D camera component, and (2) the position of the object in the field of view of the image sensor, such that the contribution of shadows to the object changes based on the refined xy dimensions of the object. Illustratively, the blob tool processing can analyze the spatial meaning of acquired 3D image data based on calibration information in the 3D camera assembly. Objects can include at least one of: (a) multiple side-by-side objects, each with a top of approximately similar height; and (b) one or more objects with top surfaces that respectively define multiple different heights. The 3D camera assembly can include at least one of a stereo camera, a structured lighting-based camera, a time-of-flight camera, and a profiler. Shadow and positive 3D image data can define both a bounding box containing the shadow and positive 3D image data residing above a reference surface. Exemplarily, the reference surface can define the conveyor belt surface where the object resides. The object can be a package, and the results of the 3D blob tool are provided to a gating component of the conveyor belt, which guides the object to one of multiple different destinations based on the object's characteristics. These characteristics can indicate defective objects, thereby guiding the gated conveyor belt to defective and / or rejected object locations. Attached Figure Description

[0007] The following description of the invention refers to the accompanying drawings, in which:

[0008] Figure 1 This is an overview of a system for acquiring and processing 3D images of complex objects, which employs a composite 3D blob tool to provide predictions of missing image data;

[0009] Figure 2 This is a perspective view of an exemplary group of objects for which image data is missing due to shadows and other effects that can be imaged and analyzed using the illustrative composite 3D blob tool described herein.

[0010] Figure 3 yes Figure 2 A representation of the 3D point cloud image of the object in the image, showing the image data along the xy plane of the image with missing image data;

[0011] Figure 4 yes Figure 3 The diagram shows a relatively accurate height measurement along the z-axis;

[0012] Figure 5 This is a diagram illustrating the positioning of a 3D camera component on a set of objects and the potential for shadows and missing image data.

[0013] Figure 6 It is by Figure 5 A top-down view of the images captured by the camera component, showing the shadows of individual objects in each of different directions and sizes;

[0014] Figure 7 It is a representation of a 3D point cloud image of a basic rectangular object residing near the edge of the camera component's field of view (FOV), showing the projection and orientation of shadows;

[0015] Figure 8 yes Figure 7 A representation of the 3D point cloud image of the basic rectangular object, which resides in the middle closer to the FOV of the camera component, showing a more complex projection and orientation of shadows;

[0016] Figure 9 This is a flowchart of the process steps for operating a composite 3D blob tool according to an illustrative embodiment; and

[0017] Figure 10 This is a diagram illustrating the creation of bounding boxes using a composite 3D blob tool according to an illustrative embodiment to determine features in a multilevel object or group of objects. Detailed Implementation

[0018] I. System Overview

[0019] Figure 1 An overview of arrangement 100 is shown, in which a vision system camera assembly (also simply referred to as a "camera" or "sensor") 110 acquires 3D image data of an exemplary group 120 of object 122 as object 122 passes below its field of view (FOV) relative to a moving conveyor belt 130. In this example, object 122 is a group of 120 cans or bottles arranged in groups—e.g., within a crate or shrink wrap. It should be noted that any arrangement of objects or a single object with a complex top surface can be imaged and analyzed according to the systems and methods described herein.

[0020] The 3D camera / imaging assembly 110 considered can be any assembly that acquires a 3D image of an object, including but not limited to stereo cameras, time-of-flight cameras, lidar, ultrasonic ranging cameras, structured lighting systems, and laser displacement sensors (profilometers). Therefore, the term 3D camera should be broadly understood to include these systems as well as any other system that generates height information associated with a 2D image of the object. A single camera or an array of cameras may also be provided, and the terms "camera" and / or "camera assembly" may refer to one or more cameras that acquire images in a manner that generates desired 3D image data of the scene. The described camera assembly 110 is shown mounted above the surface of the conveyor belt 130 as a checkpoint or inspection station that images the object as it flows past. The object 122 may remain in motion or temporarily stop imaging, depending on the speed of the conveyor belt and the acquisition time of the camera image sensor(s) and associated electronics (partly depending on the frame rate and aperture settings) 110. In alternative embodiments, the conveyor belt may be omitted, and the object may be located on a non-moving stage or surface. As a non-limiting example, camera 110 defines an optical axis OA that is approximately perpendicular to the surface of conveyor belt 130. Alternatively, the camera axis OA may be oriented at a non-perpendicular angle relative to the surface of the conveyor belt in an alternative arrangement. Camera calibration can be converted between an internal coordinate system and an imaging scene coordinate system. Note that the local x, y, and z axes (or other coordinate systems) 138 are shown for reference only. In this example, the plane of the conveyor belt surface represents the xy dimension, and the height perpendicular to the conveyor belt surface represents the z dimension.

[0021] Camera 110 includes an image sensor S adapted to generate 3D image data 134 within its housing. The camera assembly includes an overall illumination assembly I (e.g., a ring illuminator of LEDs that projects light in a predictable direction relative to axis OA). External illumination (not shown) may be provided in an alternative arrangement. A suitable optical package O for optical communication with sensor S along axis OA is shown. Sensor S communicates with an internal and / or external vision system processor 140, which receives image data 134 from camera 110 and performs various vision system tasks on the data according to the system and method described herein. Processor 140 includes a low-level processing / processor or functional module comprising a set of vision system tools 142, which may include various standard and custom tools for recognizing and analyzing features in the image data, including but not limited to edge detectors, blob tools, pattern recognition tools, deep learning networks, etc. Vision system processor 140 may also include a dimensioning processor according to the system and method. The processor 144 performs various analysis and measurement tasks on features identified in the 3D image data to determine whether specific features exist that can be used to compute further results. The processor uses various conventional and custom (e.g., 3D) vision system tools 142, including a 3D blob tool 144 according to an exemplary embodiment. System setup and results display can be handled by a separate computing device 150, such as a server (e.g., cloud-based or local), a personal computer (PC), a laptop computer, a tablet, and / or a smartphone. The computing device 150 is described (by way of non-limiting example) as having a conventional display or touchscreen 152, a keyboard 154, and a mouse 156, which together provide graphical user interface (GUI) functionality. Various interface devices and / or shape elements may be provided in alternative embodiments of the device 150. The GUI may be partially driven by a web browser application residing on the device's operating system and displaying web pages with control and data information from the processor 140, according to the exemplary arrangement herein.

[0022] Note that the processor 140 may be entirely or partially floor-mounted on the housing of the camera assembly 110, and various processing modules / tools 142 and 144 may be instantiated, as appropriate, entirely or partially, in the onboard processor 140 or in the remote computing device 150. In an exemplary embodiment, all vision system and interface functions may be instantiated on the onboard processor 140, and the computing device 150 may be primarily used for training, monitoring, and operations related to a web interface (e.g., HyperText Markup Language, HTML) generated by the onboard processor 140 and transmitted to the computing device via a wired or wireless network link. Alternatively, all or part of the processor 140 may reside in the computing device 150. The results of the analysis from the processor may be sent to a downstream utilization device or processing 160. Such a device / process may use the results 162 to process objects / packages—for example, gating the conveyor belt 130 based on the analyzed features to direct objects to different destinations and / or reject defective objects.

[0023] II. Exemplary Images and Questions

[0024] Figure 2 An exemplary runtime object, or object group 210, is also described—in this non-limiting example, defining a glass body 212 and a crate of a jar or bottle covered with a top or lid 220. Figure 3 and Figure 4 Showing from Figure 1 An exemplary (e.g., 3D point cloud) representation of this object with a 3D camera component overlaid on it is shown. Specifically, Figure 3 The diagram shows a representation 300, where the top 220 of the can is imaged along the XY plane. The top 220 is defined by an appropriately sized bounding box, which may be defined based on the height difference with the surrounding conveyor surface 320. Regions of interest (rectangles 330) show the individual tops obscured by shadows and other data (can body beneath the lid), making it difficult for vision system tools (e.g., edge detectors) to resolve individual lids. Instead, in… Figure 4 The display 400 shows the overall height data of the same region (rectangle 420 within outer bounding box 410), and defines fairly accurate measurements along the Z direction / dimension. To further illustrate, Figure 5 The camera assembly 510 shown generates a projected illumination fan 512, which the camera assembly receives as reflected light 514 (offset due to the placement of lighting and optical components) in a manner that generates a series of outward fan-shaped shadows 522 from the top (cover) 520. This projected illumination fan 512 overlaps and blurs the final 2D image of the scene. Figure 6As shown in the top view, each top 520 can display a unique size / shape shadow, which is also offset in a discrete direction 520 based on the top's relative position in the scene compared to the lighting angle and optical axis OA. This increases the complexity and uncertainty of the entire image.

[0025] Further reference Figure 7 The 3D display representation 710 described in the top-down view of the basic rectangular object 720 illustrates the challenge of resolving the shape, where the object 720 is sufficiently far from one edge of the FOV, has no shadow on one side 730, a deep shadow on one side (opposite) 740, and two light shadows 750. Similarly, Figure 8 The representation 810 shows the basic object 820 (also a rectangle in the top-down view). This object 820 is closer to the center of the FOV and relative to... Figure 7 Object 720 in the image is slightly rotated. In this representation, object 820 defines a shadow 830 with eight (8) corners, including a four-cornered shadow 850 from the illumination projector that overlaps with the four-cornered shadow 860 from the camera. The presence of complex shadows that vary depending on position in the field of view makes it challenging to achieve resolution of features using 2D image data (even when combined with raw height data), as some height data may be missed due to shadows. However, as stated above, the available height data can be quite accurate.

[0026] III. Blob Tools and Operation Methods

[0027] Figure 9 This is a flowchart illustrating process 900 according to an exemplary embodiment. In step 910, a camera assembly acquires one or more 3D images of an object or group of objects within the field of view (FOV). Using the 3D image data, in step 920, one or more regions of interest—e.g., regions whose height data is above a reference plane (e.g., a conveyor belt surface) and other boundary regions (boxes)—are placed around the object within the regions of interest. In step 930, regions within the regions of interest lacking image data (also referred to as “inverse data”) are identified. This can be achieved using various methods known to those skilled in the art (e.g., segmentation tools and / or other suitable vision system tools). Typically, these are regions where height values ​​and / or xy pixel values ​​are absent. Notably, process 900 then uses the missing image data to indicate connectivity and / or shading. Therefore, in step 940, the process connects all the data (above the reference plane) and all the inverse data in contact. In step 950, the process then determines the position of the boundary boxes within the regions of interest. This gives a relative reference to the object within the entire FOV. This is based on data stored in the camera assembly (during factory calibration steps and / or field calibration at setup). Figure 1The calibration data 170 in the process understands the (stored) path of light rays returning from the illuminator to the camera at each point at different heights within the FOV. Therefore, based on this data and the bounding box's position within the FOV, as well as the height data of each point within the bounding box, the process can refine the bounding box to remove shadows from the main image data (step 960). This requires following the paths of the expected rays in the connected shape and mapping them to known heights in the image data. In step 970, the result of this calculation can be sent as refined image data downstream for processing—such as performing dimensioning, inspection, and other vision system tools.

[0028] IV. Objects of different heights (multi-level)

[0029] Figure 10 This is a schematic diagram illustrating camera assembly 1010 acquiring images of multiple objects or different portions of the same object within its field of view (FOV) 1020. In this example, objects 1030 and 1032, with different heights, are relatively adjacent to each other. Note that the maximum expected height of the objects is pre-programmed by the user, and in this example, they reside entirely within the FOV (as shown). It is further assumed that the main objects 1030 and 1032 are relatively cubic in shape. This assumption can be verified by observing the mass and shape of all top surface points (1034 and 1036) of the objects before initiating any corrections.

[0030] In the first step, using the missing point data caused by the shadows and any observation data, a minimum bounding box 1042 is generated, which includes the shadows and any observation data located above the reference surface A (e.g., conveyor belt surface 1040). Next, using the first bounding box 1042, a plane A coinciding with the topmost surface 1044 of the bounding box 1042 is generated. Then, from each corner / vertex (v[i]) (1046 and 1048) of the bounding box 1042 located on the reference surface 1040, the following steps are performed:

[0031] a. Project v[i] along the ray onto the illumination source and determine the intersection point (point B) where the ray intersects with the top plane that coincides with surface 1044, and the distance between point A and point B (distance B);

[0032] b. Project v[i] onto the imager along the ray and determine the intersection point (point C) where the ray intersects the plane that coincides with the top surface 1044, and the distance between point A and point C (distance C);

[0033] c. Obtain the vertex (u[i]) of box A on the topmost surface 1044 directly above v[i] (located on surface A);

[0034] d. Compare distance B and distance C; use the larger one to check the corresponding point (point B or point C) and determine whether the point is located within the top surface 1044;

[0035] i. If so, create a new point u'[i] corresponding to that point (point B or point C); and

[0036] ii. If not, create a new point u'[i] corresponding to u[i].

[0037] The resulting upper vertices u'[]1050 and 1052 should essentially form a rectangular shape representing the correction shadow; alternatively, the best-fit rectangle is applied to these vertices, and the final correction box (1040) is generated by projecting the rectangle down onto surface A.

[0038] Please note that the perfect square bounding box 1060 provides an incorrect estimate of the geometry because it bisects the upper shadow and lower box 1032. Therefore, the initial box 1042 is drawn in an elongated manner to enclose the entire structure above the reference plane 1040.

[0039] V. Conclusion

[0040] It should be clear that the above-described system and method can effectively provide missing image data in 3D images where the geometry and lighting characteristics of the camera components are known. The system and method work effectively on adjacent, spaced-apart objects and objects defining different heights or portions of different heights.

[0041] The foregoing is a detailed description of illustrative embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the invention. Features of each of the various embodiments described above can be suitably combined with features of other described embodiments to provide multiple feature combinations in associated new embodiments. Furthermore, while several individual embodiments of the apparatus and methods of the present invention have been described above, what is described herein is merely an illustrative application of the principles of the invention. For example, as used herein, the terms “process” and / or “processor” should be broadly understood to include various functions and components based on electronic hardware and / or software (and alternatively referred to as functional “modules” or “elements”). Furthermore, the described processes or processors can be combined with other processes and / or processors, or divided into various subprocesses or processors. These subprocesses and / or subprocessors can be combined in various ways according to the embodiments herein. Likewise, it is expressly contemplated that any function, process, and / or processor herein can be implemented using electronic hardware, software, or a combination of hardware and software comprised of a non-transitory computer-readable medium of program instructions. Furthermore, as used herein, various directional and configuration terms such as “vertical,” “horizontal,” “up,” “down,” “bottom,” “top,” “side,” “front,” “back,” “left,” and “right” are used only as relative specifications with respect to a fixed coordinate space, and not as absolute directions / arrangements, such as the direction of gravity. Additionally, when the terms “substantially” or “approximately” are used for a given measurement, numerical value, or characteristic, it refers to a quantity within the normal operating range to achieve the desired result, but this quantity includes some variation due to inherent inaccuracies and errors within the system's permissible tolerances (e.g., 1-5%). By way of non-limiting example, in cases where the 3D camera includes line-scan type components (e.g., a profilometer), an encoder or other motion-sensing device is used in conjunction with the imager to construct the y-dimensional of a complete 3D image for the field of view line by line. Therefore, this description is by way of example only and is not intended to limit the scope of the invention.

[0042] What is being protected is:

Claims

1. A composite 3D blob tool for a vision system, the composite 3D blob tool having a processor that receives acquired 3D image data from an object using a 3D camera component, comprising: blob tool processing that identifies positive 3D image data and negative 3D image data in the acquired 3D image data, and the blob tool processing combining the positive 3D image data defining z-dimensional height information and the negative 3D image data to define at least one connected object, wherein the negative 3D image data at least partially includes an orthogonal xy-dimensional dimension, the orthogonal xy-dimensional dimension defining data missing information about the object and shadows about the object, and wherein the shadows and the positive 3D image data A bounding box is defined, the bounding box including both the shadow residing above the reference surface and the positive 3D image data, the z-dimensional height information being used to refine the bounding box to remove the shadow based on (1) knowledge of the ray angle projected by illumination associated with the 3D camera assembly and received by the image sensor of the 3D camera assembly, and (2) the position of the object in the field of view of the image sensor, such that the contribution of the shadow to the object is changed based on the refined bounding box, and wherein the blob tool processes the spatial meaning of the connected object based on calibration information in the 3D camera assembly for the acquired 3D image data.

2. The composite 3D blob tool according to claim 1, wherein the object comprises at least one of: (a) a plurality of side-by-side objects, each having a top of substantially similar height; and (b) one or more objects having top surfaces that define a plurality of different heights respectively.

3. The composite 3D blob tool according to claim 1, wherein the 3D camera assembly includes at least one of a stereo camera, a structured lighting-based camera, a time-of-flight camera, and a profiler.

4. A composite 3D blob tool for a vision system, wherein the composite 3D... The blob tool has a processor that uses a 3D camera component to receive acquired 3D image data from an object, including: blob tool processing that identifies positive and negative 3D image data in the acquired 3D image data, and the blob tool processing combines the positive 3D image data and the negative 3D image data that define z-dimensional height information to define at least one connected object, wherein the negative 3D image data at least partially includes an orthogonal xy dimension that defines data gaps and shadows about the object, and wherein the shadows and the positive 3D image data define a bounding box that includes both the shadows and the positive 3D image data residing above a reference surface, the z-dimensional height information being used to refine the bounding box to remove the shadows based on (1) knowledge of ray angles projected by illumination associated with the 3D camera component and received by an image sensor of the 3D camera component, and (2) the position of the object in the field of view of the image sensor, such that the contribution of the shadows to the object changes based on the refined bounding box.

5. The composite 3D blob tool according to claim 4, wherein, The reference surface defines the conveyor belt surface on which the object resides.

6. The composite 3D blob tool according to claim 5, wherein, The object is a package, and the result of the 3D blob tool is provided to a gating component of the conveyor belt, which guides the object to one of several different destinations based on the object's characteristics.

7. A method for operating a 3D blob tool for a vision system with a processor, comprising the following steps: Provide 3D image data of an object acquired using a 3D camera component; identify positive 3D image data and negative 3D image data in the acquired 3D image data, and combine the positive 3D image data and the negative 3D image data; At least one connected object is defined, and the connected object is analyzed for the spatial meaning of the acquired 3D image data based on calibration information in a 3D camera. The positive 3D image data defines z-dimensional height information, and the negative 3D image data at least partially includes orthogonal xy dimensions, which define data gaps and shadows related to the object. The shadows and the positive 3D image data define a bounding box containing both the shadows and the positive 3D image data residing above a reference surface. Based on knowledge of ray angles projected by illumination associated with the 3D camera assembly and received by the image sensor of the 3D camera assembly, and the object's position in the field of view, the bounding box is refined using the z-dimensional height information to remove the shadows, thereby removing the contribution of the shadows to the object based on the refined bounding box. And the connection object is analyzed based on the spatial meaning of the acquired 3D image data using calibration information in the 3D camera component.

8. The method according to claim 7, wherein, The object includes at least one of the following: (a) a plurality of side-by-side objects, each having a top of approximately similar height; and (b) one or more objects having top surfaces that define a plurality of different heights respectively.

9. The method according to claim 7, wherein, The 3D camera assembly includes at least one of a stereo camera, a structured lighting-based camera, a time-of-flight camera, and a profiler.

10. A method for operating a 3D blob tool for a vision system with a processor, comprising the following steps: Provide 3D image data of an object acquired using a 3D camera component; identify positive 3D image data and negative 3D image data in the acquired 3D image data, and combine the positive 3D image data and the negative 3D image data; At least one connected object is defined, and the connected object is analyzed for the spatial meaning of the acquired 3D image data based on calibration information in a 3D camera, wherein the positive 3D image data defines z-dimensional height information, and the negative 3D image data at least partially includes an orthogonal xy-dimensional dimension, which defines data gaps about the object and defines shadows about the object; a bounding box is defined using the shadows and the positive 3D image data, the bounding box including both the shadows and the positive 3D image data residing above a reference surface; based on knowledge of the ray angles projected by illumination associated with the 3D camera assembly and received by the image sensor of the 3D camera assembly, and the position of the object in the field of view, the bounding box is refined using the z-dimensional height information to remove the shadows, thereby removing the contribution of the shadows to the object based on the refined bounding box.

11. The method according to claim 10, wherein, The reference surface defines the conveyor belt surface on which the object resides.

12. The method according to claim 11, wherein, The object is a package, and the method further includes: gating the conveyor belt based on the characteristics of the object to guide the object to one of a plurality of different destinations.

13. The method of claim 12, further comprising: Defects in the object are identified and the conveyor belt is gated to guide the object to a rejected destination.

Citation Information

Patent Citations

  • Method and system for projector calibration

    CN105592310A

  • Image sensing device and measuring system for providing image data and information on 3D-characteristics of an object

    CN106133476A