Symbol reader with multi-core processor and operating system and method thereof

Abstract

The present invention provides a vision system camera having a multi-core processor, high speed and high resolution imager, field of view extender, auto focus lens, and pre-processor connected to the imager for pre-processing image data, and a coordinated method of operation, which provides highly desirable acquisition and processing speed, as well as image clarity, in a wide range of applications. The mechanism is effective for scanning objects requiring a wide field of view, varying in size, and moving relatively rapidly with respect to the system's field of view. The vision system provides a physical package having a variety of physical interconnect options to support a variety of options and control functions. The package is effective in dissipating heat generated internally by arranging components to optimize heat exchange with the surrounding environment, and including heat dissipation structures to facilitate such heat exchange (e.g. fins). The system also allows for a variety of multi-core process optimization and load balancing of image processing and system operation (e.g. automatic adjustment of tasks).

Description

[0001] Divisional Application

[0002] This application is a divisional application of application number 201810200359.1, filed on October 8, 2013, entitled "Symbol Reader with Multi-core Processor and Operating System and Method Thereof". This application is a divisional application of application number 201310465330.3, filed on October 8, 2013, entitled "Symbol Reader with Multi-core Processor and Operating System and Method Thereof" TECHNICAL FIELD

[0003] The present invention relates to machine vision systems, and more particularly to vision systems capable of acquiring, processing, and decoding symbols such as bar codes. BACKGROUND

[0004] Vision systems for measuring, detecting, correcting, and / or decoding symbols (e.g., one- and two-dimensional bar codes, also known as "IDs") are widely used in applications and industries. Such systems are based on the use of an image sensor (also known as an "imager") that acquires an image (typically a grayscale or color image, and one-, two-, or three-dimensional image) of an object or target, and a vision system processor, either on-board or interconnected, that processes the acquired image. The processor typically includes both processing hardware and non-transitory computer readable program instructions that execute one or more vision system processes based on information processed from the image to produce a desired output. The image information is typically provided in an array of image pixels, each having a different color and / or intensity. In the example of a symbol reader (also referred to herein as a "camera"), a user or automated process acquires an image of a target believed to contain one or more bar codes, two-dimensional codes, or other symbol types. The image is processed to identify features of the bar code, and then decoded by a decoding program and / or processor to acquire the underlying alphanumeric data represented by the bar code.

[0005] One common application of ID readers is to track and sort targets moving along a path in production and logistics operations, such as a conveyor. The ID readers can be positioned along the entire path to acquire the respective ID of all objects of interest at the appropriate angle as each object moves within its field of view. Depending on the placement of the reader relative to the moving path and the size (e.g., height) of the objects, the focal length of the reader relative to the object can vary. That is, larger objects can result in the ID thereon being closer to the reader, while smaller / flat objects can contain IDs that are farther from the reader. In each case, the ID should appear at sufficient resolution to be properly imaged and decoded. Unfortunately, the imaging sensors relied upon by the most readily available vision system cameras on the market are defined in pixel arrays that are close to square in size (e.g., close to a 1 : 1 aspect ratio, while more generally 4:3, 5:4, or 16:9). This aspect ratio does not fit well with the requirements of reading applications in which objects pass on a conveyor line with a relatively wide field of view (FOV) relative to the camera. More generally, the height of the FOV should be slightly larger than the ID (or other region of interest), while the width of the FOV should be approximately equal to or slightly larger than the width of the conveyor line. In some instances, line-scan cameras can be employed to accommodate object movement and wide FOVs. However, such a solution is not suitable for certain geometrical configurations of objects and line mechanisms. Likewise, line-scan (i.e., one-dimensional) imaging sensors tend to be more costly than conventional rectangular format sensors.

[0006] In cases where the object and / or line is relatively wide, the lens or imager of a single ID reader can not have sufficient field of view in the lateral direction to cover the entire width of the path while maintaining the resolution required to accurately image and decode the ID. Failure to image the full width can result in the reader missing IDs outside of its field of view, or passing too quickly through the field of view. One high-cost approach to providing the required width is to employ multiple cameras spanning the width of the line, typically networked together to divide the image data and process. Alternatively, a wider field of view aspect ratio of one or more cameras can be achieved by optically expanding the native field of view of the sensor using a field expander that splits the field of view into multiple narrower strips extending across the width of the conveyor line. A challenge in providing such mechanisms is that the narrower sections in the upstream-to-downstream direction of the moving line can require higher frame rates to ensure that the ID is fully captured before it moves out of the section. This can tax the processing speed of the system, while current imager-based decoding systems that acquire over a wider area are substantially lacking in the frame rate required to reliably decode at high object pass speeds.

[0007] A further challenge in operating a vision system based ID reader is that focus and illumination should be set to relative optimal values to provide a readable ID image for the decoding application. This requires a fast analysis method of the focus and illumination conditions so that these parameters can be calculated automatically and / or adjusted automatically. In cases where the field of view is wide and / or the object throughput is high relative to the imaging scene, it can not be possible to achieve the processing speed required to perform such functions using a conventional vision system based reader.

[0008] Typically, to provide such high speed functions, the imager / sensor can acquire images at a relatively high frame rate. It is generally desirable to provide image processing mechanisms / processes that can more efficiently employ the image frames in various ways that can improve the system's ability to adjust parameters and read image data at high rates. SUMMARY

[0009] The present invention overcomes the shortcomings of the prior art by providing a vision system camera having a multi-core processor, high speed and high resolution imager, field of view extender (FOVE), auto-focus lens, and pre-processor connected to the imager for pre-processing image data, and a method of operating the same, that provides highly desirable acquisition and processing speeds, as well as image clarity, in a wide range of applications. The mechanism efficiently scans for objects of interest that require a wide field of view, size, and location of useful features, and that move relatively rapidly with respect to the system's field of view. The vision system provides a physical package that has a variety of physical interconnects to support a variety of options and control functions. The package is arranged to optimize heat exchange with the surrounding environment, effectively dissipating heat generated internally, and includes heat dissipation structures to facilitate such heat exchange (e.g., fins). The system also allows for a variety of multi-core process optimization and load balancing of image processing and system operation (e.g., automatic adjustment of tasks).

[0010] In an exemplary embodiment, the vision system includes a camera housing that houses an imager and a processor mechanism. The processor mechanism includes (a) a preprocessor interconnected with the imager that receives and pre-processes images from the imager at a first frame rate (e.g., 200 to 300 or more images per second) and (b) a multi-core processor (having multiple cores) that receives pre-processed images from the preprocessor and performs vision system tasks thereon. Results related to information in the images can thereby be generated. It should be noted that the term "core" as used herein should be broadly construed to include "groups of cores" that are discrete and assigned a particular task. Exemplarily, the first frame rate is much higher than a second frame rate at which the multi-core processor receives images from the preprocessor. The preprocessor (e.g., FPGA, ASIC, DSP, etc.) can also be interconnected with a data store that buffers images from the imager. In various processes, where the entire image is not necessary for a particular function (e.g., auto-adjustment), portions of the image or sub-images can be buffered based on instructions from the preprocessor. Likewise, sub-sampled image data can be buffered in certain processes, such as auto-adjustment, that do not require full resolution images in performing the task. Further, the multi-core processor can be interconnected with a data store that stores operational instructions for each core of the multi-core processor. The store also stores image data that is to be processed by each core based on a schedule. In particular, the schedule instructs that each image is selectively processed in each core in order to increase efficiency of result generation. The schedule can instruct one or more cores to perform system tasks (also referred to as "system operational tasks" that are not directly related to image processing and decoding tasks), such as auto-adjustment, such as illumination control, brightness exposure, and auto-focusing of a lens. The lens can be a liquid lens or other type of variable focus lens. The preprocessor can be configured and arranged to perform such pre-set auto-adjustment operations based at least in part on information generated by the system tasks performed by at least one of the cores. More particularly, the results generated by the cores can include decoded symbols (IDs / codes) imaged from an object.

[0011] In an exemplary embodiment, the camera assembly lens can be optically coupled to a FOVE that divides the image received at the imager into a plurality of sub-images along an extended width. The sub-images can be vertically stacked on the imager and include overlap in the width direction. The overlap can occur in each sub-image and can be wide enough to completely image the largest ID / code that needs to be observed, ensuring that no symbols are lost due to the division between fields of view. Exemplarily, each sub-image is processed by a separate core (or separate group of cores) of a multi-core processor. To assist in automatic calibration, the FOVE can include a fiducial point at a known focal distance from the imager, located at a position in the optical path that can be selectively or partially exposed to the imager so that image capture at runtime can be accomplished without any significant interference from the fiducial point. A self-calibration process uses the fiducial point to determine the focal length (focus) of the lens. Exemplarily, the fiducial point can be located on an optical component of the FOVE. Optionally, the FOVE housing supports an external illuminator that is removably attached to the housing by interlocking alignment structures and magnets.

[0012] The physical packaging of the camera assembly is built from materials with good thermal conduction to transfer heat faster to the surrounding environment, such as aluminum alloys. The processor mechanism includes an imager board that includes the imager and a main board that includes the multi-core processor, which is biased against an inner side of the camera housing by a spring-loaded bracket assembly, thereby achieving a secure yet removable snap fit, and a tight fit with the inner side walls of the camera assembly housing for improved heat transfer from the main board. To further enhance heat exchange and tight fit, the main board includes a profile of protruding circuit elements that is configured to follow the inner profile of the inner side of the camera housing so as to minimize the distance therebetween. The camera assembly housing also includes a plurality of heat fins on its outer side to exchange heat with the surrounding environment. The housing further supports one or more external fans. The front of the housing is adapted to mount a removable lens assembly. Such a removable lens assembly can include a liquid lens that is connected by a cable to a connector on a side (e.g., front) of the camera assembly housing. Another connector is provided to control optional internal (or external) illumination. The back of the camera includes a separate I / O board that is connected to the main board by an electronic link. The I / O board includes a plurality of connectors that are exposed to the outside for various data and control function interfaces. One such control / function is an external speed signal (e.g., an encoder signal) from a moving pipeline relative to the field of view of the camera assembly. The pre-processor and / or multi-core processor are built and configured to perform at least one of the following operations based on the speed signal and a plurality of images: (a) control focusing of a variable lens; (b) determine a focal distance of an imaged object; (c) correct a focal distance to the pipeline; and (d) determine a relative speed of an imaged object. Typically, the camera housing includes a front face and a back face that are each sealingly attached to the respective joint at each opposite end of the body (using gasket sealing). Optionally, the joint between one (or both together) of the front face and back face and the body includes a ring made of translucent material that is built and configured to illuminate one of a plurality of pre-set colors so as to provide an indicator to the user of a corresponding system status. For example, the ring can illuminate green for a good (successful) ID read, and red for a no (failed) ID read.

[0013] In an embodiment, based on the pre-processor's recognition of a useful feature (e.g., symbol / ID / code), the pre-processor can be adapted to selectively transfer the image from a buffer memory to the multi-core processor for further processing at the cores of the multi-core processor.

[0014] In an exemplary embodiment, a method for processing images in a vision system includes capturing images in an imager of a vision system camera at a first frame rate and transferring at least a portion of the images to a multi-core processor. The transferred images are processed to produce results in each of a plurality of cores of the multi-core processor in accordance with a schedule, which contains information related to the images. The processing steps can further include a step of identifying images containing symbols in at least one of the plurality of cores of the transferred images and a step of performing decoding on the images containing symbols in another of the plurality of cores so that one core identifies whether a symbol is present (and optionally provides other information related to the symbol, such as including resolution, symbol type, etc.) and another core decodes the symbol that has been identified. Optionally, the processing steps can include a step of performing image analysis on the transferred images to identify images having sufficient characteristics for decoding in at least one of the plurality of cores. In other words, the core determines whether the image is sufficiently clear and usable for decoding. Another core performs the step of decoding on images having sufficient characteristics, whereby unusable images are discarded before attempting to locate and / or decode symbols. In an embodiment, the step of decoding is performed in at least one of the plurality of cores using a first decoding process (e.g., algorithm) on the transferred images and the step of decoding is performed in another of the plurality of cores using a second decoding process, such that decoding can occur in at least one decoding process. Illustratively, the step of decoding can require that a image be decoded in at least one of the plurality of cores and, after a preset time interval, if (a) the image has not completed decoding and (b) there is a possibility that the image can be decoded with more time, the image is continued to be decoded in another of the plurality of cores. Optionally, after the time limit has passed, there is a possibility that more time can successfully decode, the system can allow the core to continue decoding and assign the next image to a different core. In a further embodiment, where there are multiple image frames having multiple types of symbols (e.g., one-dimensional and two-dimensional codes), the system can provide load balancing. The cores split the images in a manner that provides a relative load balancing of one-dimensional (ID) codes and two-dimensional (2D) codes to each core.

[0015] In a further embodiment, codes can be assigned to non-decoding system tasks based on a current trigger frequency. A low trigger frequency within a threshold allows a core to be used for system tasks, such as auto-adjustment, while a higher trigger frequency indicates that the core is to be used for decoding (e.g., generating results related to image information). As described above, various processes related to core assignment can intermingle during the operation of the vision system, and processing resources (cores) can be reassigned for various purposes. BRIEF DESCRIPTION OF DRAWINGS

[0016] The following description of the application is provided, together with the accompanying drawings, to provide a basic understanding of various aspects of the application. The

[0017] Figure 1 is a schematic view of a vision system relative to an exemplary mobile pipeline setup having objects of various sizes and shapes including ID or other indicia, each object passing through the field of view of the system according to an exemplary embodiment;

[0018] Figure 2 is a block diagram of circuitry for acquiring and processing image data and for controlling various system functions according to an exemplary embodiment;

[0019] Figure 3 is Figure 1 is a front perspective view of a vision system camera assembly according to an exemplary embodiment;

[0020] Figure 4 is Figure 1 is a rear perspective view of a vision system camera assembly according to an exemplary embodiment;

[0021] Figure 5 is a side sectional view of a vision system camera assembly along line 5-5 of Figure 3

[0022] Figure 5A Figure 3 is a rear sectional view of a vision system camera assembly along line 5A-5A of

[0023] Figure 6 is Figure 1 is a front perspective view of a vision system camera assembly with internal lighting assembly and lens removed;

[0024] Figure 7 is Figure 1 is a perspective view of a vision system according to an exemplary embodiment including a vision system camera assembly and the field of view extender (FOVE) cooperating with an external crossbar type illuminator mounted thereon;

[0025] Figure 7A is a more detailed top sectional view of the coupling between the FOVE housing and the front of the camera assembly according to Figure 7

[0026] Figure 8 is Figure 7 is a perspective view of the optical components of an exemplary FOVE shown with the housing removed;

[0027] Figure 9 is Figure 7 is a plan view of the optical components of an exemplary FOVE shown with the housing removed and acquiring a wide field of view image;

[0028] Figure 10 is a plan view of the optical components of an exemplary FOVE shown with the housing removed and acquiring a wide field of view image;​​​Figure 7 A schematic diagram of the FOVE, which provides multiple fields of view to the imager of the camera assembly.

[0029] Figure 11 for Figure 7 A front view of the FOVE, which has a crossbar-type illuminator mounted on a bracket relative to the FOVE housing, and... Figure 1 A connector that works with the camera assembly;

[0030] Figure 12 For installation Figure 1 A partial top cross-sectional view of a thin-film-based liquid lens assembly controlled by and in a camera assembly according to an exemplary embodiment.

[0031] Figure 13 for Figure 1 A rear-view perspective view of the internal components of the camera assembly, in which the housing body has been removed and the “360-degree” ring indicator structure between the body and its front is shown in detail.

[0032] Figure 14 For, used for Figure 1 A flowchart illustrating the generalized operation of the core allocation system task and the scheduling algorithm / process of the vision system task in the multi-core processor of the vision system.

[0033] Figure 15 The diagram shows a block diagram of a multi-core process, in which an image frame is divided into multiple parts, which are then assigned to multiple cores for processing.

[0034] Figure 16 The diagram shows a block diagram of a multi-core process, in which an image frame is assigned to one core for processing, while another core executes one or more system tasks.

[0035] Figure 17 The flowchart shows how the system dynamically allocates cores for image processing and non-image processing tasks based on the current trigger frequency.

[0036] Figure 18 The diagram shows a block diagram of a multi-core process, in which the ID / code in each image frame is dynamically allocated to the cores in a way that more effectively balances the processing load of the entire core group.

[0037] Figure 19 The flowchart shows that after the decoding process of an identifier code in the first core exceeds a preset time limit, the process is allocated to the second core.

[0038] Figure 20is a flow chart showing continuation of a decoding process of an identifier code to a first core after a preset time limit is exceeded by a first core processing the decoding process of the identifier code;

[0039] Figure 21 is a block diagram of a multi-core process in which ID / code in an image frame is assigned to two cores in parallel, wherein each core performs a different decoding algorithm;

[0040] Figure 22 is a block diagram of a multi-core process in which a series of image frames are each assigned to a different core for processing;

[0041] Figure 23 is a block diagram of a multi-core process in which image frame data is assigned to a first core running an ID / code lookup process in parallel with a second core running an ID / code decoding process based on ID / code information found by the first core;

[0042] Figure 24 is a block diagram of a multi-core process in which image frame data is assigned to a first core running a vision system process in parallel with a second core running an ID / code decoding process based on image information provided by the first core;

[0043] Figure 25 is a block diagram of a multi-core process in which image frame data is assigned to a first core running an ID / code presence / absence process in parallel with a second core running an ID / code localization and decoding process based on ID / code presence / absence information provided by the first core;

[0044] Figure 26 is a block diagram of a multi-core process in which image frame data is assigned to a first core running an image analysis process in parallel with a second core running an ID / code localization and decoding process based on information provided by the first core relating to image frame quality and characteristics;

[0045] Figure 27 is a flow chart of a system process for adjusting focus based on a comparison of measurements from a conveyor / pipeline speed sensor (encoder) and tracking of features on objects passing through the field of view of an exemplary vision system;

[0046] Figure 28 is a flow chart of a process using a pre-processor (FPGA) connected to an imager to locate useful features (ID / code) and send only image frames that appear to contain useful features to a multi-core processor for further processing;

[0047] Figure 29 is Figure 1side view of the vision system showing the self-calibration fiducial provided for the FOVE, and an optional bottom mounted cooling fan on the vision camera assembly;

[0048] Figure 29A more detailed perspective view of the camera assembly according to an exemplary embodiment, including a bottom mounted bracket and cooling fan;

[0049] Figure 29B exploded perspective view of the camera assembly with Figure 29A bracket and cooling fan;

[0050] Figure 30 flowchart of a system process for correcting the curved non-linearity of the lens drive current for focal length / optical power;

[0051] Figure 31 flowchart of a system process for determining focal length from analysis of feature locations in each overlap region of the image projected at the FOVE;

[0052] Figure 32 flowchart of a system process for determining the speed and / or distance of an object through Figure 1 the field of view of the vision system by changes in the size of object features between image frames; and

[0053] Figure 33 schematic diagram of an exemplary master-slave mechanism according to an embodiment showing multiple interconnected camera assemblies and illuminators. DETAILED DESCRIPTION

[0054] I. SYSTEM OVERVIEW

[0055] Figure 1A vision system 100, also referred to as a "machine vision system," according to an exemplary embodiment is described. The vision system 100 includes a vision system camera 110, which illustratively includes an integrated (and / or internal) processor mechanism 114. The processor mechanism 114 enables image data acquired by an imager (e.g., a CMOS or CCD sensor) 112 (shown in dashed lines) to be processed to analyze information within the acquired images. The imager 112 is disposed on a cooperating imager circuit board 113 (also shown in dashed lines), and as described below the processor mechanism 114 in this embodiment includes a multi-core architecture, which includes at least two separate (discrete) processing cores CI and C2, which according to an embodiment can be provided as a single die (e.g., chip). Also as described below, the processor 114 is disposed on a processor board or "main" board 115. Similarly, an interconnect input / output (I / O) board 117 and a user interface (UI) board 123 are provided for communication with remote devices and information display, respectively. The functions of the imager 112 and the multi-core processor 114 are described in further detail below. In general, the processor runs a vision system process 119, which takes advantage of the multi-core processor mechanism 114, as well as an ID lookup and decode process 121. Optionally, all or part of the decode process can be handled by a dedicated decoder chip on a separate die from the processor 114.

[0056] The camera 110 includes a lens assembly 116, which is optionally removable and replaceable with a variety of conventional (or custom) mounting base lens assemblies. The lens assembly can be manually or automatically focused. In an embodiment, the lens assembly 116 can include an auto-focus mechanism based on known systems, such as commercially available liquid lens systems. In an embodiment, the mounting base can be defined as the geometry of a well-known cine or "C-mount" base - other known or custom geometries are expressly contemplated in alternative embodiments.

[0057] As shown, an exemplary field of view extender (FOVE) 118 is mounted in front of the lens assembly 116. The FOVE allows for an extension of the width WF of the field of view 120, which is typically limited by the lens assembly 116 at a given focal length to an initial width that is N times smaller than the width of any overlap region (or regions) between the fields of view, and a length LF of the field of view 120 is reduced to 1 / N of the initial length. The FOVE 118 can be implemented using a variety of mechanisms, generally including a set of prisms that divide the field of view into a series of vertically split portions of the imager. In an embodiment, the FOVE is configured to direct the directions of its outer mirrors to receive light from different lateral portions of the scene, which can be a pipeline with moving objects (e.g., a conveyor belt with moving objects) as shown in FIG. 2A, or a scene with a moving object as shown in FIG. 2B. In an embodiment, the FOVE 118 is configured to direct the directions of its outer mirrors to receive light from different lateral portions of the scene, which can be a pipeline with moving objects (e.g., a conveyor belt with moving objects) as shown in FIG. 2A, or a scene with a moving object as shown in FIG. 2B. Figure 1Thereafter, the outer mirrors direct light to a mating perpendicularly tilted inner mirror of a beamsplitter, which then directs light through an aperture in the FOVE that is substantially aligned in line with the optical axis of the camera to avoid image distortion. The inner mirror directs light from each outer mirror to separate strips on the imager, with one strip stacked vertically, for example, on top of the other, and the vision system then looks for and analyzes features throughout the image. The field of view defined by the mirrors includes a widthwise overlap region that is sized and arranged to ensure that central features are fully present in at least one strip. In another embodiment, the moving mirrors change position between acquired image frames so that the full width of the scene is imaged in successive frames. Exemplary FOVE mechanisms, including those described herein, are shown and described in U.S. Patent Application No. 13,367,141 entitled "System and Method for Vision System Field of View Extension" by Nunnink et al. This application is incorporated by reference herein as useful background information.

[0058] In one embodiment, the FOVE 118 is provided with a first outer mirror that forms an acute angle with respect to the optical axis of the camera and a second outer mirror that forms an opposite acute angle with respect to the opposite side of the optical axis. From the perspective of the vision system camera, a beamsplitter is positioned in front of the first outer mirror and the second outer mirror. The beamsplitter is provided with a first reflective surface and a second reflective surface. Exemplarily, the first outer mirror and the first reflective surface are arranged to direct a first field of view from the scene along the optical axis to the imager. Likewise, exemplarily, the second outer mirror and the second reflective surface are arranged to direct a second field of view from the scene along the optical axis to the imager. In a horizontal direction at the scene, the first field of view is at least partially separated from the second field of view. In addition, the first outer mirror, the second outer mirror, and the beamsplitter are arranged to project each of the first field of view and the second field of view as strips in a vertically stacked relationship to the imager. It should be clear that a wide variety of FOVE implementations are expressly contemplated in the various embodiments herein.

[0059] The FOVE is sufficient to image objects 122, 124 (e.g., boxes) moving at a speed VL across the moving pipeline 126 relative to the camera assembly 110 to properly acquire useful features (e.g., bar codes 130, 132, 134). As an example, the width WF of the FOVE 120 extends to approximately match the width WL of the pipeline 126. It is contemplated in alternative embodiments that the objects remain stationary while the camera assembly moves relative to the objects on a track or other suitable structure (e.g., a robotic arm). For example, two objects 122 and 124 having different heights H01 and H02 are passed through the FOVE 120. As described above, the height difference is a factor that generally requires the camera assembly to change focus. The ability to change focus more rapidly becomes highly desirable when the objects move more rapidly through the FOVE 120. Likewise, the ability to more rapidly identify useful features and process these features using the vision system processor 114 becomes highly desirable. It is expressly contemplated that multiple vision system camera assemblies having cooperating FOVEs, illuminators, and other accessories can be employed to image objects passing through a scene. For example, a second vision system 180 (shown in dashed lines) is provided to image the opposite side of the objects. As shown, this additional vision system 180 is connected (via connection 182) to the system 100 described above. This allows for shared image data and synchronized capture and illumination triggering, among other functions (e.g., master-slave operation of interconnected camera assemblies as described below). Each camera assembly can independently process image data or can perform some or all of the processes in the cores of interconnected camera assemblies according to various multi-core processes as described below. The number, placement, and operation of further vision systems are highly variable in various embodiments.

[0060] II. Electronic portion of the system

[0061] By way of reference Figure 2The circuitry and functionality of the imager board 113, main board 115, I / O board 117, and UI board 123 will be described in greater detail. As shown, the imager 112 is located on the imager board 113 and can comprise a commercially available CMOS 2 megapixel grayscale unit such as model CMV2000 from CMOSIS of Belgium. Other types and sizes of imagers can be provided in alternative embodiments, including imagers of higher or lower resolution, color imagers, multispectral imagers, and the like. The imager is operatively connected to an FPGA 210 (or other programmable circuitry) via control and data connections, which FPGA performs image processing processes in accordance with exemplary embodiments described below. For purposes of this description, the FPGA or equivalent high speed processing logic such as ASIC, DSP, and the like can be referred to as an "imager-interconnected" "preprocessor" that performs initial and / or certain automatic adjustment functions on the received stream of image frames from the imager. Further, although an FPGA is used as an example, any programmable or non-programmable processing logic (or multiple logics) that can perform the desired preprocessing functions can be expressly contemplated for use as a "preprocessor." An exemplary preprocessor circuit is the ECP3 family of FPGAs, available from Lattice Semiconductor of Hillsboro, Oregon. The FPGA 210 is interconnected with an appropriately sized non-volatile memory 212 (Flash) that provides configuration data to the FPGA. The FPGA 210 also controls optional internal illumination 214 (described further below) and optional variable (e.g., liquid) lens assembly 216 for providing fast auto-focus to the camera lens assembly. Also, while the preprocessor described herein is suitable for performing certain functions including but not limited to automatic adjustment, image data conversion, and acquired image data storage operations, a wide variety of additional processes (e.g., vision system processes) related directly to information processing within the image can be performed by the preprocessor, such as feature finding, and the like. More generally, the high frame rate of the imager makes the use of such high speed processors desirable (in various embodiments) to operate on the initial processes relative to the acquired image frames.

[0062] One way to quickly operate a liquid lens assembly is the EL-6-18-VIS-LD thin film bottom liquid lens, available from Optotune AG, Switzerland. In addition to fast operation, this lens defines, for example, a 6 mm aperture, making it very suitable for wide angle imaging and fast operation. This exemplary variable lens package has dimensions of 18 x 18.4 x 8.9 (thickness) mm. Control current is approximately between 0 and 200 mA. Response time is typically less than 2 milliseconds and its settling time is typically less than 10 milliseconds. After integrating this liquid lens into an exemplary lens assembly, the entire lens assembly provides a field of view of approximately 20 degrees and a focal length adjustment range of approximately 60 mm to infinity. In operation, the EL-6-18-VIS-LD is a shape-changing lens. It includes a container that is injection molded, filled with optical liquid, and sealed by an elastic polymer membrane. The deflection of the lens is directly proportional to the pressure in the liquid. The EL-6-18 employs an electromagnetic actuator that exerts pressure on the container. Thus, the focal length of the lens is controlled by the current through the actuator coil. This focal length decreases as the applied current increases.

[0063] A temperature sensor 218 is provided in association with the lens to monitor the operating temperature in the vicinity of the lens. This allows for temperature-based adjustment of the liquid lens, as well as other temperature-related parameters and functions. The temperature sensor is placed on an I2C bus 220, which is also used to control the internal illumination 214 and the liquid lens using appropriate control signals specified by the lens manufacturer. As described below, additional temperature sensors can be provided to one or more circuit boards (e.g., sensor 288) to monitor the temperature status of various components of the system. As shown, the bus 220 is interconnected with the multi-core processor 114 on the motherboard 115. Similarly, the FPGA 210 is bonded to the processor 114 via a serial peripheral interface (SPI) bus 224 and a PCIe bus 226, which respectively carry control and data signals between the units. Illustratively, the SPI 224 bus interface between the FPGA 210 and the processor 114 is employed by the processor 114 to configure the FPGA during system startup. Subsequent configuration, communication of image data, and other system data are carried over the PCIe bus 226. The PCIe bus can be configured as a dual (2X) lane. The FPGA 210 is also interconnected via a 16-bit connection with a 64 MB data memory 228, which allows for buffering of image data in order to support high frame rates of the imagers at the imager board level - and such image frames can then be employed for downstream image processing or auto-adjustment functions as described below. In general, a portion of the auto-adjustment can require the use of lower resolution images. In turn, a sequence of acquired images can be stored in memory 228 at a lower resolution (satisfying the FPGA functionality) while higher resolution images are sent to the processor 114 for processing as described below. The memory 228 can be of any acceptable type, such as DDR3 dynamic random access memory. Alternatively, another memory type can be employed, such as static random access memory (SRAM). Appropriate power supply voltages 230 are also provided for the various imager board components, which are taken from an external voltage source (typically 120-240 VAC wall current with appropriate transformers, rectifiers, etc.).

[0064] Link 232 also illustratively connects FPGA 210 with an external lighting control connector 234 on I / O board 117 and exposed outside the camera assembly 110 housing rear. Likewise, link 232 also interconnects the FPGA with a sync trigger connection 236 on I / O board 117 to synchronize image acquisition (including lighting triggering) with other interconnected camera assemblies. This interconnection can occur in the case of multiple camera assemblies simultaneously imaging multiple sides of a box and / or in the case of a box passing through multiple relatively proximate stations in a flow line. Synchronization avoids cross-talk between illuminators and other undesirable effects. In general, it is noted that in this embodiment, various image acquisition functions and / or processes, including internal and external lighting, focusing, and brightness control are all directly controlled by the fast running FPGA process 245. This allows motherboard processor 114 to focus on vision system tasks and image data decoding. In addition, synchronization of acquisition also allows multiple camera assemblies to share a single illuminator or group of illuminators, as the illuminator (or illuminators) are independently triggered for each camera as each camera acquires an image frame.

[0065] It is noted that an appropriate interface can be provided for an external trigger. Such an external trigger can allow strobing of the camera assembly for image acquisition when a moving target is in the field of view. This strobing avoids acquiring images of the space between objects on the flow line that are not necessary. A detector or other switch device can be used to provide the strobe signal in accordance with conventional techniques.

[0066] FPGA 210 provides certain pre-processing work on the images to improve the speed and efficiency of image data operations. Image data is transferred from imager 112 to FPGA in series. All or a portion of the data can be temporarily stored in data store 228 for analysis by various FPGA operations. FPGA 210 converts the serial image data to PCIe protocol using conventional techniques so that it is compatible with the data bus architecture of the processor and transmitted on PCIe bus 226 to processor 114. The image data is then directly transferred into data store 244 for subsequent processing by processor cores CI and C2. By utilizing multiple cores, many desirable and efficiency enhancing operations in processing image data are enabled, as described in detail below. FPGA 210 is also programmed (e.g., FPGA process 245) to analyze acquired image data to perform specific system auto-adjustment operations such as auto-brightness control (e.g., auto-exposure) and auto-focus control (e.g., when using liquid lens assembly 216). Generally, for situations where the focal length is changed, such as when encountering objects of different heights, both brightness and focus need to be adjusted. Typically, these operations require higher image acquisition rates of imager 112 (e.g., at speeds of about 200-300 image frames per second) to allow for additional operations on the image data, while the net decoding rate at processor 114 is a minimum of 100 frames per second. That is, some images are processed in FPGA, while others are transferred to memory on motherboard 115 for vision system processing (e.g., ID finding and decoding of IDs found in the images) without compromising the maximum frame rate of the processor by stepping down. More generally, data store 228 buffers acquired image frames and takes some of the frames (from the excess number of available image frames resulting from the high frame rate) for auto-adjustment functions of FPGA 210, while transferring others to processor 114 for further processing. The division of functions between FPGA 210 and processor 114 facilitates efficiency and more optimal utilization of system resources.

[0067] In various embodiments, FPGA 210 and memory 228 can be adapted to receive a "burst" of image frames at a high acquisition frame rate, take some of the frames of the image frame burst for performing auto-adjustment, and send other frames to the processor at a speed suitable for the processor processing speed. The high volume of image frames obtained from the burst (e.g., when an object is in the field of view) can be fed to processor 114 during interstitial time before the point in time when the next object reaches the field of view, where the next object reaching the field of view causes the next "burst" to be acquired, stored, and transferred to processor 114.

[0068] The terms "process" and / or "processor" as used herein should be interpreted in their broadest context as encompassing various electronic hardware and / or software-based functions and components. Moreover, the processes or processors can be combined or divided into sub-processes or sub-processors. Various combinations of such sub-processes and / or sub-processors can be implemented in accordance with the embodiments herein. Likewise, it is expressly contemplated that any of the functions, processes and / or processors described herein can be implemented using electronic hardware, software, or a combination of hardware and software, wherein the software comprises non-transitory computer readable medium embodying program instructions.

[0069] Referring to Figure 2 The main board 115 is shown with a multi-core processor 114. Various types, makes and / or configurations of processors can be employed to carry out the teachings of the embodiments herein. In an exemplary embodiment, the processor 114 comprises a dual-core DSP such as model 6672 available from Texas Instruments, Inc. of Dallas, Texas. The processor 114 can be fast enough and cost effective for the purposes of the vision system applications contemplated herein. The term "multi-core" as used herein shall mean two (i.e., "dual-core") or more separate processors implemented on a single die and / or packaged in a single on-board circuit chip. Each core is generally capable of independently processing at least a portion of the data stored in memory 244. The processor 114 is interconnected with non-volatile memory 240 containing appropriate boot-up configuration data. This allows for basic operation of the processor at start-up of the camera system, including when any program code and / or operating system software is loaded. The program code / operating system software is stored in program memory 242, which can be configured to use various solid-state memory devices. In an exemplary embodiment, a NOR Flash memory having 32 MB capacity and a 16-bit interface is employed. At start-up, the program code is loaded from the flash program memory 242 into data memory 244. Image data and other data operated on by the processor is also stored in data memory 244, and can be flushed from data memory when it is no longer needed by the system processes. Various types, sizes and configurations of memory can be employed. In an embodiment, the memory is 256 MB of DDR3 dynamic random access memory having a 64-bit interface.

[0070] Other conventional circuitry for driving the processor and providing other functions, such as code error exclusion, is also provided on the main board 115 and interconnected with the processor 114. This circuitry can be configured in accordance with conventional techniques and can include a core voltage regulator 246 (e.g., model UCD7242 from Texas Instruments), an LVDS clock generator 248 (e.g., model CDCE62005 from Texas Instruments), and a sequential microcontroller 250 (e.g., PIC18F45 from Microchip Technology Inc. of Chandler, Arizona). A JTAG interface 252 (e.g., 60-pin and 14-pin) is also interconnected between a port on the processor 114 and the sequential microcontroller 250. Appropriate voltages (e.g., 1.5V, 1.8V, 2.5V, and 6.2V) are provided to the various circuit elements of the main board 115 by a voltage source 254 on the I / O board, which is connected to a regulator 260 (e.g., 24V to 3.3V regulator). External power is thus received from a power source (e.g., 24V wall transformer) via appropriate cables 262. The main board 115 and cooperating processor 114 are connected to the I / O board via a UART loaded on the processor, which is connected to an RS-232 standard compliant serial connector 266 on the outside of the housing. This port can be used to control external functions, such as warnings, conveyor line shut-off, and the like. The processor also includes a serial gigabit media independent interface (SGMII) to an Ethernet port on the back of the housing via a physical layer chip 268 and a gigabit Ethernet transformer 270. This allows image data and other control information to be transmitted to a remote computer system via a network. Via an interface computer and appropriate user interface (e.g., web-based graphical user interface / browser screen(s)), a user can also program the functions of the system. In various embodiments (not shown), as an option, the camera assembly can also be provided with a wireless Ethernet connection, communications, and the like.

[0071] The processor SPI bus 224 is connected to an appropriate ATTINY microcontroller 272 (e.g., available from Atmel Corporation of San Jose, California) that implements an interface to 4x optical inputs (4X OPTO IN) 274 and 4x optical outputs (4X OPTO OUT) 276 using conventional techniques. This interface provides "slow" I / O operations, including external strobe trigger inputs, good-read and bad-read outputs, encoder inputs (e.g., to count movement pulses on a moving conveyor assembly), target detection, and various other I / O functions. The bus 224 is also connected to a further ATTINY microcontroller 280 on the UI board 123. This microcontroller is connected to user interface (UI) devices that are external to the camera assembly housing. These devices include, but are not limited to, an audible tone generator 282 (e.g., a buzzer), one or more control buttons 284, and one or more indicator lights 286 (e.g., LEDs). These devices allow the user to perform various functions, including vision system training, calibration, and the like, as well as to receive status of system operation. This can include on / off functions, fault warnings, success / failure of read ID, and the like. A common status indicator (LED) can be associated with trigger-on, trigger-off, encoder, and target detection status. Other interface devices (not shown) can also optionally be provided, such as a display screen and / or alphanumeric display. The I / O board 117 includes appropriate temperature sensors to monitor internal temperature.

[0072] It should be apparent that the placement and location of components on each of the various boards, as well as the functions of those components, are highly variable. It is expressly contemplated that more or fewer circuit boards can be employed in various embodiments. Likewise, some or all of the functions of multiple components can be consolidated into a single circuit, or some or all of the functions of a particular component can be split among multiple circuits on one or more boards. Furthermore, Figure 2 The components, interconnections, bus architecture, and functions described are merely examples of various circuit arrangements that can perform similar functions. Those skilled in the art will appreciate the structure of alternative circuit arrangements having similar or identical functions.

[0073] III. Physical Packaging

[0074] Having described the arrangement of electronic components on the various circuit boards of the camera assembly, as well as their respective interconnections and functions, reference is now made to Figures 3-7 which describes the physical structure of the camera assembly 110. Figures 3-6 A camera assembly 110 according to an embodiment is described, having a conventional lens 310 and a surrounding, inboard (annular) illumination assembly 320. Figure 7 For a more detailed external view of the camera assembly 110, having as Figure 1The optional FOVE annex 118.

[0075] The housing 330 of the camera assembly 110 is constructed of a material with suitable rigidity and heat transfer properties. In an exemplary embodiment, an aluminum alloy (e.g., 6061) may be used to construct part or all of the housing. The body 332 is also provided with integrally formed longitudinal fins 339 around its perimeter to further assist heat transfer. The housing 330 consists of three main parts: the body 332, the front portion 334, and the rear portion 336. The body 332 is a single piece with an open interior. The front portion 334 and the rear portion 336 are respectively secured to opposite ends of the body using screws located in holes 338 and 410. The front portion 334 and the rear portion 336 are pressed against the ends of the body to form an hermetically sealed seal that protects the internal electronic components from contact with dust, moisture, and other contaminants that may be present in the manufacturing process or other process environments. Gaskets 510 (e.g., O-rings, see Figure 5 () are placed at each of the respective ends of the body 332 to press tightly seal the front 334 and the rear 336. Note that the body may be made into a protruding structure with appropriate countersunk holes formed by holes and other machined shapes applied to the outer and inner sides.

[0076] like Figure 5 As shown, the imager plate and the mating imager 112 are fixed against the front portion 334, wherein the imager is perpendicular to the optical axis OA defined by the lens assembly 310. In this embodiment, a fixed lens assembly 310 is employed, which has front and rear convex lenses 512 and 514 in a conventional configuration. For example, the lens assembly is a 16mm lens assembly with a C-mount base. It is screwed into a camera assembly lens base 520, which extends from the front portion 334. In the alternative embodiments described below, other lens types and mounting base configurations are explicitly contemplated.

[0077] The lens is surrounded by a ring-shaped inner illumination assembly 320, which has an outer ring 524 and an illumination circuit board 526 at its front end. The circuit board 526 is supported on three supports 528, which are arranged in a triangular orientation about the optical axis OA. In this embodiment, illumination is provided by eight high-output LEDs 530 (e.g., OSRAM Dragon LEDs) with mating lenses 532. These LEDs operate at selected, discontinuous visible and / or near-visible (e.g., infrared) wavelengths. In various embodiments, different LEDs operate at different wavelengths, which can be selected by the illumination control process. For example, some LEDs may operate at green wavelengths, while others may operate at red wavelengths. (See reference...) Figure 6wherein the illumination assembly 320 has been removed, exposing the front face 610 of the camera assembly 110. The front face 610 includes a pair of multi-pin connectors 614 and 616, which are located on the imager board and are similar to the illustrated components 214 and 216 in Figure 2 . That is, the 5-pin connector 614 is interconnected via a cable (not shown) with the illumination board 526. The 8-pin connector 616 is connected to control and power the optional liquid lens assembly described below. The front face 610 also includes three bases 620 (which can be threaded) to support each of the illumination circuit board mounts 528. The threaded C-mount base 520 can also be seen. Note that the described inboard illumination assembly 320 is an optional implementation for the vision system camera assembly. In various embodiments described herein, the inboard illumination assembly can be omitted in favor of one or more outboard illumination assemblies, or, in some special cases, ambient illumination.

[0078] Referring particularly to the cross-sectional view of Figure 5 , the imager board is connected by a ribbon cable 550 to the main board 115, which is illustratively seated against the top side of the interior of the body. The main board exchanges heat with the body 332 and the mating fins 339 in this position to allow for better heat transfer. The main board 115 can be mounted using fasteners, or, as shown, using a cradle element 552, which engages the underside of the main board 115 in a position that does not interfere with the on-board circuit elements. The cradle 552 includes a lower extension 553 having a hole that fits over a vertical post 555 that extends upward in a telescoping fashion from a base 554. The base 554 sits on the underside of the housing body 332. The cradle 552 is biased upward via a compression spring 556 placed between the underside of the cradle and the base 554 and encircling the extension 553 and the post 555. This mechanism allows for the insertion or removal of the board by adjusting the position of the cradle 552 relative to the base 554. That is, to install the board 115, a user presses down on the cradle 552 against the biasing force of the spring 556, slides the board 115 into the interior of the body 332, and then releases the cradle 552 to snap into place against the board 115 and maintain it in position against the top end of the interior of the body 332. Removal is the reverse of this process. The board 115 is held securely in place against the body 332 by the spring 556, thereby ensuring adequate heat exchange. In various embodiments, the main board 115 can also include an on-board heat sink that is connected to the body 332. Likewise, a thermal conducting paste, or another heat transfer medium, can be disposed between the contact portions of the board 115 (e.g., the processor 114) and the interior surface of the body 332. Briefly referring to Figure 13 , as described below, the underside of the main board 115 can include thermal gap pads 1330 that fill the gap between the upper portion of the board 115 and the interior surface of the body.

[0079] More generally, referring also to Figure 5AThe inner surface 580 of the body 332 is shaped relative to the surface of the motherboard 115 such that it closely matches the shape of protrusions, surface-mount components, and circuit elements on the motherboard 115, and these components are mounted to conform to the shape of the body. That is, taller components are placed towards the longitudinal centerline, where the body has a higher profile, while shorter components are placed along either side of the motherboard's longitudinal axis. More generally, components are divided into multiple height regions according to the internal geometry of the body. In cases where some circuits tend to be large or tall (e.g., capacitors), these components may be divided into two or more smaller components with the same overall electronic quantity values ​​as a single larger component. A thermal gap filler (e.g., a pad or other medium) is disposed between the board and the inner top, and this placement of components, based on the internal geometry of the body, ensures that the distance between the body and both short and tall components is minimized. Exemplarily, as shown, a multi-core processor is configured to directly contact the inner side of the body (generally with a thin layer of thermally conductive adhesive therebetween), thus the body acts as an effective heat sink for the processor. As also shown in the figure, the motherboard 115 is indexed laterally relative to the bracket 552 via a vertical rod 582 passing through a hole in the board. This ensures that the bracket and the board maintain a predetermined alignment relative to the body. Note that although cooling is passive in the described embodiment, in further embodiments one or more fan units may participate in cooling the interior or exterior of the housing. In particular, four mounting holes 588 may be provided along the bottom of the body 332 (two of which are located in…). Figure 5A (Shown in dashed lines). In this embodiment, these holes 588 receive a conventional 60x60mm computer fan. Optionally, as described below, the holes 588 may receive an intermediate bracket for mounting the fan and / or other specifically conceived fan mechanisms / sizes. A connector may be provided on the housing, or an external connector may be used to connect a suitable voltage adapter and power the fan (or multiple fans). Furthermore, an auxiliary cooling mechanism (e.g., liquid cooling) may be used in optional embodiments. Typically, the system is designed to operate using ambient cooling up to approximately 40 degrees Celsius. However, in some environments where the operating temperature may exceed this value, the use of at least one cooling fan is activated.

[0080] like Figure 5 As shown, the I / O board 117 is mounted against the rear 336 of the camera assembly housing 330. The I / O board 117 is connected to the rear end of the mainboard 115 by a ribbon cable 560. Its function is as described in reference [reference needed]. Figure 2 The various rear connectors 420, 422, 424, 426 and 428 (see...) Figure 4) from the rear side of the I / O board 117. The I / O board is likewise interconnected with the UI board 123 via a ribbon cable 570. As shown, the UI board is exposed to the user along the angled top surface 440 of the rear portion 336. In other embodiments, the arrangement and location of the circuit boards on and / or within the body can be varied.

[0081] Referring to Figure 7 and Figure 7A in greater detail, the FOVE 118 is shown attached to a coupling 710 that includes a removable L-shaped bracket 712 at the front of the camera assembly. The bracket 712 includes a vertical plate 714 that faces the front 334 of the camera assembly and is secured with fasteners, and a horizontal plate 716 that is adapted to have further mounting brackets and support structures secured thereto. The bracket 712 of the coupling 710 can also be used to mount a removable illuminator 750, as described below. The FOVE housing 730 is supported relative to the camera assembly by a set of four vertical posts 732 that are secured to the base bracket on the side of the camera assembly, and that are secured to the rear wall 736 of the FOVE housing. The flange 736 is secured to the rear of the FOVE housing 730 by suitable fasteners or other securing mechanisms (not shown). The lens assembly 116 is covered by the cylindrical cover 720 that extends between the front face (610) of the camera assembly 110 and the rear of the FOVE housing 730. The cover 720 is removable and serves to seal the lens and FOVE housing from dust and to prevent the ingress of external environmental contaminants. The vertical posts 732 or another acceptable open frame allow the user to access the lens assembly 116 for adjustment and maintenance. The vertical posts 732 movably (heavy arrow 744) support a sliding block 746 that engages the sliding lens cover 1692. A pair of low-friction bushing-containing joints 747 surround the two (or more) vertical posts 732. O-rings 748, 749 are respectively embedded on the inside of the inner circumference of the flange 736 and on the inside of the inner circumference of the vertical face 714 of the opposing L-shaped bracket 712. The lens cover 720 can be slid forward out of the sealed position depicted in the figures to expose the lens assembly 116 (as an exemplary lens type, the Nikon® AF-S DX NIKKOR® 18-105mm f / 3.5-5.6G ED lens). Figure 7AA thrust shoulder 754 is formed on the vertical face 714, which defines a central orifice 756. This shoulder prevents the housing 720 from continuing to move forward after it has sealingly engaged the camera assembly. Likewise, a rear stop 758 is provided on the front end of the housing 720 to engage the inner face of the flange 736. Forward sliding of the housing 720 into the FOVE enclosure 730 is stopped when the slide block engages the outer wall of the flange 736. This provides sufficient space to access the lens 1697 for adjustment and / or maintenance. The FOVE enclosure 730 can be constructed of various materials, including various polymers such as injection molded, glass-filled polycarbonate and / or composites, or metals such as aluminum. In particular, glass-filled polycarbonate minimizes dimensional tolerances due to shrinkage during the molding process. The front end of the FOVE enclosure is open to the scene and includes a cover transparent window 740.

[0082] Further reference is made to Figure 8 and Figure 9 , in which the enclosure 730 is removed to show the FOVE lens geometry in greater detail. In various embodiments, various optical components and mechanisms can be employed to provide the FOVE, and in general it is contemplated that the FOVE will divide a wide image into at least two stacked images (strips), each of which occupies a portion of the imager. In this manner, the image height is reduced by about 1 / 2 (with some overlap), and the width of each strip (again, with some overlap) is the full width of the imager. Given the exemplary camera assembly providing dual core processing capability and high image acquisition speed, various processing techniques can be used to perform efficient and rapid processing of the pair of strips (as described below). Exemplarily, the FOVE 118 is based on the above-incorporated U.S. Patent Application No. 13,367,141 entitled "System and Method for Field of View Extension for Vision Systems" by Nunnink et al. Further embodiments of FOVE mechanisms that can be employed in accordance with the vision system camera assembly, as well as cooperating couplings and accessories, are likewise described in the commonly-assigned, co-pending U.S. Patent Application No. (Docket No. C12-004CIP (119 / 0126P1)) entitled "System and Method for Field of View Extension for Vision Systems" by Nunnink et al., filed on even date herewith, which is expressly incorporated herein by reference.

[0083] As shown in Figure 8 , the optical components of the FOVE include left and right outer mirrors 810 and 812, and stacked and crossed inner mirrors 820 and 822. The outer mirrors 810 and 812 are tilted at different angles. Likewise, the inner mirrors 820, 822 are tilted at different angles. Reference is made to Figure 9, showing the fields of view 910 and 912 of each of the outer side mirrors 810 and 812. A region of slight overlap OR is provided, which is at least as wide as the largest useful feature (e.g. the largest bar code) that is imaged at the focal distance FD. This guarantees that a complete image of the feature appears in at least one of the two fields of view 910, 912. Each of the imaged fields of view 910, 912 is fully reflected by its respective outer side mirror onto the intersecting inner side mirrors 820, 822, as shown. The reflected images are then further reflected to the lens 310, each field of view being vertically stacked with respect to the other (resulting from the relative tilt of each of the mirrors 810, 812, 820, 822). Thus, as shown in the schematic diagram of Fig. 10, each of the fields of view 910, 912 is projected onto each of a pair of stacked strip regions 1010, 1012 on the imager 112. A relatively small, vertical overlap region 1030 can be provided, which includes the images of both fields of view 910, 912. The amount of overlap in the vertical direction depends on the aperture of the overlap lens assembly, and can be minimized using a small aperture setting, such as F:8. The dashed lines 1040 and 1042 on each strip represent the horizontal overlap of the fields of view OR. This region is analyzed in order to obtain a complete feature (e.g. an ID), which can be fully present in one strip and missing in whole or in part in the other strip. Figure 10 As shown in the schematic diagram of Fig. 10, each of the fields of view 910, 912 is projected onto each of a pair of stacked strip regions 1010, 1012 on the imager 112. A relatively small, vertical overlap region 1030 can be provided, which includes the images of both fields of view 910, 912. The amount of overlap in the vertical direction depends on the aperture of the overlap lens assembly, and can be minimized using a small aperture setting, such as F:8. The dashed lines 1040 and 1042 on each strip represent the horizontal overlap of the fields of view OR. This region is analyzed in order to obtain a complete feature (e.g. an ID), which can be fully present in one strip and missing in whole or in part in the other strip. Figure 9 As shown in the schematic diagram of Fig. 10, each of the fields of view 910, 912 is projected onto each of a pair of stacked strip regions 1010, 1012 on the imager 112. A relatively small, vertical overlap region 1030 can be provided, which includes the images of both fields of view 910, 912. The amount of overlap in the vertical direction depends on the aperture of the overlap lens assembly, and can be minimized using a small aperture setting, such as F:8. The dashed lines 1040 and 1042 on each strip represent the horizontal overlap of the fields of view OR. This region is analyzed in order to obtain a complete feature (e.g. an ID), which can be fully present in one strip and missing in whole or in part in the other strip.

[0084] In an exemplary embodiment, each of the outer side mirrors 810, 812 has a horizontal length OML of between 40-120 mm, typically 84 mm, and a vertical height OML of between 20-50 mm, typically 33 mm, as an example of representative dimensions. Similarly, the intersecting inner side mirrors 820, 822 have an exemplary horizontal length CML of 30-60 mm, typically 53 mm, and a vertical height CMH of 10-25 mm, typically 21 mm. In an exemplary embodiment, the total horizontal span of the outer side mirrors 810, 812 is approximately 235 mm, and the spacing MS between each respective outer side mirror face and the mating inner side mirror face (e.g. 210 and 220; 212 and 222) is approximately 100 mm. Based on prior measurements and appropriate focusing adjustments in the selected camera lens 310, a total extended field of view WF of approximately 60-80 cm is covered by the single FOVE camera mechanism at high resolution according to a focal distance FD of approximately 35-40 mm. As shown, the FOVE divides the two fields of view 910, 912 into two stacked strips, each having a height of approximately 600 pixels on the imager, which will provide sufficient resolution or adequate decoding of bar code features on a fast moving pipeline.

[0085] As shown in the schematic diagram of Fig. 10, each of the fields of view 910, 912 is projected onto each of a pair of stacked strip regions 1010, 1012 on the imager 112. A relatively small, vertical overlap region 1030 can be provided, which includes the images of both fields of view 910, 912. The amount of overlap in the vertical direction depends on the aperture of the overlap lens assembly, and can be minimized using a small aperture setting, such as F:8. The dashed lines 1040 and 1042 on each strip represent the horizontal overlap of the fields of view OR. This region is analyzed in order to obtain a complete feature (e.g. an ID), which can be fully present in one strip and missing in whole or in part in the other strip. Figure 11As shown, the FOVE assembly allows for the removable attachment of an accessory crossbar-type illuminator 750. The position of the illuminator 750 (or multiple illuminators) relative to the FOVE housing is variable in further embodiments. In this embodiment, the illuminator 750 is attached to a bracket 1110 that extends forward from the coupler 710 (see Fig. 7) relative to the bottom side of the FOVE housing 730. The bracket 1110 and crossbar-type illuminator can be permanently or removably engaged, for example, using a threaded fastener (not shown) that passes through a top end of the bracket 1110 and is inserted into a threaded hole (not shown) on the top side of the illuminator 750. The bracket can be connected to the mounting hole of the L-shaped bracket 712. Although a crossbar-type illuminator is described, various alternative lighting types and configurations can be employed. The illuminator can include multiple multi-wavelength light sources that are selectively operated and / or light sources that operate at different brightness, angles, or ranges. In alternative embodiments, other attachment mechanisms, such as adhesive strips, hooks and loop-type fasteners, screws, and the like, can be used to provide secure and removable mechanical connections between the lighting and bracket components. For example, U.S. Patent Application No. (Docket No. C12-022), entitled "COMPONENT ATTACHED DEVICES AND RELATED SYSTEMS AND METHODS FOR MACHINE VISION SYSTEMS," by Saul Sanz Rodriguez and Laurens Nunnink, which is commonly assigned to the Applicant of the present application, filed on even date herewith, is incorporated herein by reference as further background information. This application describes techniques for attaching illuminators and other optical accessories to FOVE assemblies or other vision system structures using magnetic assemblies.

[0086] It is noted that, as described herein, the use of a FOVE is one option to expand the FOV to provide a wider aspect ratio relative to height. Another employable option, in addition to (or instead of) a FOVE, is to use an image sensor configured to have an aspect ratio of, for example, 1 :4 or 1 :5. Such a ratio can be optimal for scanning objects that move along a wider pipeline. Thus, in various embodiments, the sensors used for the camera assembly herein can be selected to be sensors having a wide aspect ratio, where the pixel width is a multiple of the pixel height. Exemplary methods and processes for operating image data can be adapted to process data on a wide sensor, for example, operating different regions of the sensor with different cores of a processor.

[0087] Reference is now made to Figure 12According to one embodiment, an exemplary liquid lens assembly 1210 is described, in conjunction with camera assembly 110, and in conjunction with mounting base 520. In this embodiment, a liquid lens unit 1220 (a thin film based unit as described above) is mounted in a housing 1222 that accommodates the rectangular shape of the lens unit 1220 using a cradle structure 1230. Various support structures can be employed to secure the lens within assembly 1210. The liquid lens unit illustratively includes a housing 1232 that supports a front offset lens 1240. Behind offset lens 1240 is mounted a variable, fluid filled thin film lens 1244. This lens varies based on the electro-mechanical actuation of an actuator assembly 1250. The actuator assembly, temperature sensor and other components are connected by a ribbon cable 1256 to an 8 pin connector 616, which extends from the liquid lens housing 1232 out of the lens assembly housing 1222. The routing of the cable and / or the size / shape of the housings and other components are highly variable. A transparent cover glass 1258 is provided at the rear of the liquid lens unit 1220 to seal it. The received light is transmitted to a fixed rear lens 1260 that is suitably supported within housing 1222. The housing includes a mounting assembly 1270 (which can also include a locking ring - not shown in the figure) that threadably secures the lens assembly 1210 to the mounting base 520 at the front of the camera 610. As an application of auto focus, the focusing of the liquid lens assembly 1210 is further described below.

[0088] Although not shown in the figures, any of the lens assemblies described herein can include various optical filters to attenuate certain wavelengths of light or to provide various effects, such as polarization. Likewise, illuminators can be provided with various filters. This allows for selective imaging of objects when certain types of illumination are projected and received through filters appropriate for that illumination type.

[0089] It should be clear that, according to the embodiments herein, the camera assembly can be provided with various optional interfaces and indicators. With particular reference to Figures 3, 4 and 5, and now referring to Figure 13With the front portion 334, body cover 332, and rear portion 336 removed, the internal components of the camera assembly are described. The joint between the body 332 and the rear portion 336 includes a ring 1310 of translucent material (acrylic or polycarbonate) that functions as a light pipe. The translucent ring 1310 can surround a portion of the perimeter of the joint, or, as shown, the entire (e.g., "360 degree indicator") perimeter of the joint. The ring 1310 can be completely transparent or partially transparent. Illustratively, the ring 1310 is illuminated by one of a plurality of differently colored light sources (e.g., LEDs, not shown) that are operatively connected to the imager circuit board 113. The light from the LEDs is directed into the ring 1310 via a light pipe or other light-transmitting conduit. Depending on the color and / or timing of the illumination (e.g., one or more colors flashing at a certain rate or pattern), the ring can be used to indicate various operational states. For example, a good ID read and / or decode can be illuminated in green, while a failed (e.g., bad or erroneous) ID read / decode can be illuminated in red. A flashing red color can indicate a system fault. Other colors, such as yellow, can also be included for various indications. The ring provides a unique and aesthetically pleasing, yet intuitive, way to indicate system status. The number of light sources used to illuminate the ring around the perimeter is highly variable and can be arranged in accordance with conventional techniques. Although the ring 1310 is shown as being sandwiched between the body 332 and the front portion 334, it is expressly contemplated that a similar ring can be sandwiched between the rear portion 336 (not shown) and the body 332 at the joint using the principles described above. Additionally, in various embodiments, a ring can be provided at both the front and rear joints.

[0090] IV. Processing image data in a multi-core processor

[0091] The exemplary multi-core processor 114 provides a high degree of processing independence with respect to each discrete core (Cl, C2). Without specific instructions from the user, there is minimal cross-communication between processes to share data. Typically, each processor operates its own operating system and loaded programs independently of the other. The memory space in the RAM 244 corresponding to each processor is generally non-contiguous and has minimal shared memory space. An internal bus within the processor provides for data exchange between the cores as appropriate based on the user's program instructions. Thus, the processes provide the ability to divide image processing tasks in order to improve the efficiency and speed of processing. The following is a description of various exemplary processes that can be executed using the dual-core functionality of the processor 114.

[0092] Referring to Figure 14As shown, the generalized procedure 1400 allows the processor to dynamically assign different tasks to each processor to perform. The tasks can be operations on a single image frame that is transferred to the processor from the FPGA. The tasks can be vision system tasks such as ID lookup or ID decode tasks. The procedure 1400 can allow the core operations in the multi-core processor 114 to be optimized so that the cores are used efficiently. That is, if ID lookup consumes less processor resources than ID decode, one core can be adapted to look up multiple IDs while another decodes useful image frames with the IDs that were found. Likewise, in the case where a frame represents two halves of a FOVE image, the image can be split between two cores, etc. Generally, the program data includes one or more scheduling algorithms that can be adapted to operate a particular set of image data with the highest efficiency. These scheduling algorithms can help the processor anticipate when each core becomes free to perform a given task. An appropriate scheduling algorithm is determined in step 1410 of the procedure 1400, and the algorithm is well suited to a particular set of tasks, which are loaded to at least one core in step 1420. The core becomes the scheduler for the multiple cores and communicates the schedule over the internal bus. As image frames are transferred from the FPGA to the cores of the processor over the PCIe bus, the frames are monitored and the tasks to be performed on the image data are identified by the scheduling algorithm (step 1430). The scheduling algorithm assigns the image data and tasks to the next available processor (step 1440). The assignment can be based on a prior estimate of when the processor becomes available. When the task on a particular image frame is complete, the scheduling algorithm continues to monitor and assign new tasks and data to the cores. The scheduling algorithm can be employed overtime to monitor the observed results of different types of tasks and to optimize the priority of tasks in each core. One core has the scheduling algorithm that determines which core receives the tasks.

[0093] It should be noted that in this exemplary embodiment, the use of two cores CI and C2 is exemplary of a multi-core processor that can include three or more cores. The procedures described herein can be adapted to scale to three or more cores.

[0094] The following is a description of a further procedure for using a multi-core processor according to an embodiment:

[0095] Referring to Figure 15A schematic diagram of this is shown in Figure 15, which shows a multi-core process 1500 in which the processor 114 receives an image frame 1510 that is divided into two portions 1520, 1522. The portions can be divided vertically (as in the case of two fields of view provided by a FOVE), horizontally, or by another division method (e.g., alternating pixels). The two (or more) image portions 1520, 1522 are passed to each core CI and C2. Each of the two (or more) partial images is processed and decoded in parallel with their respective core CI, C2. The decoded results 1530, 1532 can be combined and provided to a downstream process, such as a good ID read or an indication of no ID read, and the decoded information transmitted to a remote computer. An overlap can generally be provided between the two partial images so that the ID between the images is sufficiently identified in at least one core. The overlap can vary, but is generally large enough to properly encompass an ID of a given size in at least one of the partial images. In the case where the image is divided by the processor itself, the overlap is provided by sending overlapping image data to both cores simultaneously. In the case of a FOVE, the overlap is present in the acquired image, and the image of each field of view can be passed to each core without additional overlap. Communication between the cores (bus link 1540) allows for the combination of results and other cross-core communication as needed.

[0096] In a further embodiment, for cases in which there is little or no overlap between images (e.g., multiple FOVE images that are substantially non-overlapping), the process 1500 can be replaced by a stitching process. Thus, in this embodiment, each FOVE image, possibly including a portion (but not all) of the exemplary ID feature set, and both images collectively containing substantially the entire ID feature set. One or more of the cores are employed to identify the interrelationships between the ID segments in each image and "stitch" into a complete ID. This can occur during the ID finding phase of the process, in which the complete ID is assembled and then decoded by one or more of the cores, or during the decoding process, e.g., the process decodes a portion of the entire ID in each image and attempts to combine each individual decoded result.

[0097] Note that although each multi-core process described in this paper uses discrete cores to execute discrete processes as shown in the figure, it is clearly envisioned that the term "core" used in this paper can broadly refer to a group of cores. Thus, in the case of a quad-core processor, one group of two cores can handle one process task, while a second group of two cores can handle another process task. Alternatively, a group of three cores can handle one (higher processing overhead) task, while a single core can handle different (lower processing overhead) tasks. Alternatively, concurrent tasks or four concurrent tasks can be executed by assigning tasks to appropriate processor cores and / or core groups. The scheduling algorithm can also be programmed to dynamically reassign cores to different tasks based on the current processing needs of a given task. The appropriate level of processing power required for a given task (e.g., multiple cores) can be determined by experimentation, computation of different types of tasks, and monitoring the speed at which different numbers of processors complete the task. This process is described below.

[0098] Reference Figure 16 The diagram illustrates a multi-core process 1600, where processor 114 receives image frames 1610 at one (or a group of) cores (or multiple cores) C1, and C1 performs ID decoding to output a decoding result 1620. Conversely, a second (or a group of) cores (or multiple cores) C2 executes one or more (non-decoding) system-related tasks 1630, which support image acquisition and other system operations by outputting information 1640 for further downstream tasks. Such system tasks 1630 may include (but are not limited to):

[0099] • Focus setting algorithm (including distance measurement / calibration and sharpness calculation) and automatic brightness (which may include exposure, gain and illumination intensity) algorithm;

[0100] • JPEG (or other) image data compression, for example, performed on the image frame and then stored and / or transmitted to a remote computer; and / or

[0101] • Wavefront reconstruction, which is used, for example, in a vision system, to improve depth of field by using known wavefront coding techniques.

[0102] In situations where the system uses one or more cores to perform non-decoding system tasks (e.g.) Figure 16 The process (1600) allocates system tasks to certain cores, which may depend on the current triggering frequency. For example... Figure 17As shown, the scheduling process 1700 determines the current trigger frequency at step 1710. If the trigger frequency is below a certain threshold, such that fewer cores are needed to perform the decoding task, the decision step 1720 allocates one or more cores to non-decoding tasks (step 1730). Conversely, if the trigger frequency exceeds a certain threshold (or thresholds), one or more cores (the number of cores can depend on the frequency) are allocated to decoding tasks (step 1740). As shown in a simplified two-core embodiment, at a low trigger frequency, one core is allocated to decoding and the other core is allocated to system tasks. At a higher trigger frequency, one core (e.g., Cl) is allocated to decoding, while the one or additional core(s) (e.g., C2) can perform both decoding and system tasks. This is particularly applicable to a two-core system. In an exemplary multi-core system employing more than two cores, one or more cores can be allocated to decoding while other core(s) are allocated to both decoding and system tasks.

[0103] Figure 18 The process 1800 is schematically depicted as employing multiple cores when both one-dimensional and two-dimensional codes (or other independent types of features requiring different processing power / decoding time) are present. Typically, two-dimensional codes require more processing resources / time to fully decode. Once the IDs in the images are found, they are scheduled so that the task load is dynamically balanced for each of cores Cl and C2 to optimize the system throughput. For example, as shown, two one-dimensional codes 1810 and 1820 are in respective images 1850 and 1860. Similarly, two two-dimensional codes 1830 and 1840 are in respective images. The codes are organized so that at each next image, the two-dimensional and one-dimensional decoding tasks can be switched between the two cores. In this way, on average, each core Cl, C2 produces the same amount of processing of decoding results 1880, 1890.

[0104] As Figure 19The multi-core process 1900 shown allocates a first (or set) core(s) to decode an image within a maximum time determined by the system's highest throughput (step 1910). If the maximum time is exceeded without completing the decoding, decision step 1920 jumps to decision step 1930 which decides if the image is decodable if given more processing time than the maximum time. If not, then the system indicates no read (step 1940). If decodable is false, a second (or set) core(s) is allocated in step 1950 to attempt to further decode the image or more images that could not be decoded within the maximum time (but have features that could complete decoding with more processing time). In one example of operation, the features that assume the image is decodable with the given more time include: (a) a finder pattern of the code has been found in the image; and / or (b) other codes from a set of codes printed on the object have been found (e.g., Maxicode and bar codes printed on the same package where one has already been found). Alternatively, if an ID is assumed or possible to decode with more time, or by using one or more different decoding algorithms than currently employed, decision step 1930 can jump (shown in dashed line) to step 1960 where the system controls the first core or reallocates the second core to continue processing the ID using a different decoding algorithm. The algorithm can be a default selected or based on certain features in the image and / or in the ID (e.g., apparent image contrast, etc.) where the features make such an algorithm particularly suitable for processing.

[0105] Figure 19 A variation of the process 1900 is shown in Figure 20 In the process 2000 described, the maximum decoding time on a given image has been reached (steps 2010 and 2020). Assuming there are features that give the possibility of completing decoding with more processing time (otherwise an indication of no read is issued in step 2040), the system allows the first (or set) core(s) to continue processing the image and allocates the decoding of the next image to a different (or set) core(s) so that the first (or set) core(s) completes its decoding task (step 2050).

[0106] As Figure 21A multi-core process 2100 is shown that attempts to decode an ID / code 2110 in an image using multiple decoding algorithms. A first (or group) of cores (or multiple cores) Cl attempts to decode the ID / code 2110 using a first decoding algorithm 2120, while a second (or group) of cores (or multiple cores) C2 attempts to decode the same ID / code 2110 using a second decoding algorithm 2130 simultaneously (when available). For example, one core Cl attempts to decode the image using an algorithm optimized for Data Matrix codes with high contrast, while another core C2 uses an algorithm optimized for (DPM) codes with low contrast. The decoding results or decoding failures 2140, 2150 are output from each of the cores (or core groups) Cl, C2. Note that in some instances, the two sets of results from different algorithms can be merged to "stitch" together a complete code or otherwise used to verify the decoding task. This can occur in situations where neither result is a complete (or reliable) read of the ID / code.

[0107] As shown in Figure 22 Another multi-core process 2200 is shown employing cores 1 (Cl) through N (CN). In this process, one (or group) of cores is used to decode each of a series of images 1-N (2210, 2212, 2214). The cores Cl-CN produce decoding results 1-N (2220, 2222, 2224), respectively. As described above, the images can be assigned to the cores in a predetermined order or based on a dynamically determined order. In the case of dynamic assignment (as described above), various factors can be considered, such as code type and speed of decoding a given image (e.g., decoding time exceeds a maximum threshold).

[0108] Figure 23 A multi-core process 2300 is described in which regions containing IDs are located by one (or group) of cores and the IDs in the regions are decoded in another (or group) of cores. Image frame data 2310 is transmitted to cores Cl and C2 simultaneously. One core Cl operates a process 2320 to find regions containing symbol (ID) information, while another core C2 operates an ID decoding process (typically passed between cores via an internal bus) that uses region information 2340 to concentrate information about the IDs and to concentrate transmitted ID features (e.g., bar code orientation, boundaries, etc.) in those regions to speed up the decoding process and efficiently produce decoding results 2350. In cases where more than two cores are used, fewer numbers of cores can be used to find and more cores can be used to decode (or vice versa).

[0109] Figure 24A multi-core process 2400 is described. In this embodiment, a first (or group) of cores Cl processes image frame data 2410 using various conventional and / or specialized vision system tools 2420 to extract relevant image information (e.g., edges, down-sampled pixels, blobs, etc.). The extracted image information 2440 is transferred over a bus to a second (or group) of cores C2, which is decoded by a decode process 2430 that includes processes for interpreting the extracted information to screen for features that resemble an ID. This results in decoded results 2450, if any.

[0110] Figure 25 A multi-core process 2500 similar to processes 2300 and 2400 is described. A first (or group) of cores Cl employs an ID presence / absence process 2520 (e.g., adapted to search for features that resemble an ID, such as tight parallel lines, and / or the geometry of a DataMatrix in the image data) on the transferred image frame data 2510 to determine the presence / absence of an ID / code. This results in presence / absence information 2540, which varies with location, site, or image feature information, uniquely determining the actual presence or absence. This determines whether the image contains an ID / code, which is discarded without further processing if not. The presence / absence information 2540 is transferred to a second (or group) of cores C2. This is used in the second core to perform a process 2530 or discard the image data. If the ID / code is shown to be present, the second (or group) of cores C2 employs an ID localization and decode process 2530 (or processes) to find and decode the image by sufficient resemblance to a symbol presentation. Upon completion of the decode process, any decode results 2550 are output. In addition to (or instead of) ID localization data, this and other processes described herein can transfer other ID-related data between cores. Such other data can include, but is not limited to, image resolution, ID type, etc.

[0111] Further variations of multi-core processes 2300, 2400, and 2500 are described in Figure 26As described in process 2600, the first (or group of) cores C1 analyzes the data of each image frame 2610 to determine whether the image has sufficient quality and / or content for processing by the second (or group of) cores C2. Image analysis process 2620 determines image features and whether it is worthwhile to perform an ID lookup and decoding process. If so, the first (or group of) cores C1 instructs (sends instruction 2640) the second (or group of) cores to be responsible for the ID lookup / location and decoding process 2630, which outputs a decoding result 2650. Possible features used to determine the adequacy of image data include, but are not limited to, image contrast, sharpness / focus quality, etc. As shown, it is also clearly conceivable that at least a portion of the image analysis process 2620 can be operated within the FPGA using a pre-defined algorithm suitable for running within the FPGA. The information derived by the algorithm is then transmitted to one or more cores (e.g., C1, C2, etc.) for ID location and decoding according to process 2630.

[0112] It should be clear that any of the aforementioned multi-core processes can be combined with other multi-core processes in a single runtime operation using scheduling algorithms. For example, autofocus can be run as a system task within a single core. Figure 16 Process 1600 in the image acquisition event is used to acquire a portion of the image of the corresponding object, while processing of the local image (e.g., FOVE two parts of the image) may be performed during a subsequent portion of the image acquisition event. Other processes described above may also be performed as appropriate during other portions of the acquisition event.

[0113] V. Additional system features and functions

[0114] Having described various exemplary embodiments of the electronic and physical packaging and multi-core processes of the vision system described herein, the following further describes exemplary features and functions that may be desirably and advantageously adopted to enhance overall operability and versatility.

[0115] Typically, determining the focal length and quickly adjusting the lens assembly is desirable on a continuous object basis, especially when the objects have different heights and / or orientations (e.g., ...). Figure 1 (As shown in the example). Typically, conveyor systems and other moving assembly lines are adapted to include: encoder signals in pulse form based on travel-distance, the period of which varies with the assembly line speed. By knowing the travel-distance increment between pulses, the speed of the assembly line (and the objects on it) at any given time can be determined. Thus, referring to... Figure 27The encoder signal is input to the camera assembly interface (step 2710) and processed to determine the actual object velocity (step 2720). When features on the object (e.g. ID or other distinguishable shape) are identified, their pixel drift can be tracked between image frames (step 2730). The time between frames is known, so the movement of the pixels in the features between frames allows the system to calculate the relative range to the object (feature). With the diverging camera lens described, pixel drift increases at shorter ranges and decreases at longer ranges. Thus, by measuring the pixel drift, the range can be calculated using the basic equation (step 2740). When the range is calculated, the system can command the FPGA to adjust the liquid lens assembly (or other autofocus lens) appropriately (step 2750). Typically, a list of current values is stored corresponding to preset ranges. Once the range is known, the system sets the current to that value. Lens assembly calibration to ensure that the current adjustment corresponds to the determined range can be performed periodically using conventional or custom techniques. In an exemplary embodiment, a known distance to a conveyor can be used to calibrate the range of the liquid lens. A feature on the conveyor (or an applied fiducial) is sharply focused by the lens, and then this feature is set to the known range. This feature can be fixed (e.g. on the side of the conveyor within the field of view) or can be on the conveyor. In the case where it is on the conveyor, it can optionally be encoded as an encoder position from which the relative precise position of the calibration feature within the field of view (downstream) can be known.

[0116] Referring to Figure 28 The process 2800 of the FPGA (or other pre-processor connected to the imager) can include a program or process that performs a high speed search for features that are similar to an ID / code. This process can use a standard ID lookup program such as searching for a pattern of multiple adjacent parallel lines or edges similar to a datamatrix. The FPGA transfers only the image frames containing such features from the buffer (memory 228) to the processor 114 over the PCIe bus (step 2820), essentially eliminating image frames that do not contain code. The processor then uses the assigned core (or cores) to perform further decoding processes on the received image frames (step 2830). The FPGA can also transfer the relevant ID location data (if any) to shorten the decoding time within the processor 114.

[0117] Referring to Figure 29As shown, vision system 100 has a camera assembly 110, a lens assembly / housing 116, and an additional FOVE 118. FOVE has been provided with one or more applied fiducials 2910, which can include a checkerboard pattern of light and dark elements or another clearly discernible pattern. In this embodiment, fiducials 2910 are applied to a corner of FOVE window 740, a relatively small and distant location relative to the overall field of view (e.g., at a corner). Optionally (or in addition), fiducials 2912 (shown in dashed lines) can be placed at appropriate locations on a mirror (e.g., large mirror 812 - shown in dashed lines). Typically, the fiducials are located on an optical component along the FOVE optical path. The distance between the fiducials and the image plane (sensor 112 - shown in dashed lines) can be accurately determined by focusing on the fiducials, and the focal length of the liquid lens (or other lens assembly) can be accurately calibrated. Additional techniques for providing "closed loop" auto-calibration for liquid lenses (or other variable lens assemblies) are shown and described in commonly assigned U.S. Patent Application No. 13 / 563,499 entitled "System and Method for Determining and Controlling Focal Length in a Vision System Camera" by Laurens Nunnink et al. The teachings therein are incorporated by reference as useful background material herein. Generally, the structures and techniques described in this incorporated application provide a lens assembly with a structure that selectively projects a fiducial pattern to at least a portion of the optical path during calibration (which can occur on-the-fly during run-time operation), but allows some or all of the field of view to remain undisturbed during normal run-time operation. This approach substantially eliminates inaccuracies due to manufacturing tolerances, calibration drift over time of use, temperature of the system and / or lens assembly.

[0118] To further illustrate, in Figure 29 As shown, the optional fan assembly 2920 described above is mounted to the underside of camera assembly 110 by screws or other fasteners 2921. A connecting cable 2922 is connected to appropriate connectors at the rear of the camera assembly. Optionally, cable 2922 can be connected to an external power source.

[0119] With further reference to Figure 29A and 29B A more detailed perspective view of exemplary camera assembly 110 (with exemplary lens 2928) can also include an optional bracket 2930, which provides an intermediary assembly relative to fan 2920. Bracket 2930 includes an annular inlet / outlet 2931 sized to match the diameter of the fan blades so as to allow air flow through the annular inlet / outlet. Bracket 2930 also includes fasteners 2932, which secure the bracket to the threaded holes (not shown) in the bottom of the camera body described above. Figure 5a 588). The fan 2920 is mounted to the outside of the bracket 2930 by fasteners 2936 offset from the bracket fasteners 2932. These fasteners 2938 are inserted into threaded holes 2937 in the bracket 2930. The fasteners 2936 pass through washers 2938 that maintain the rigidity of the mounting flange of the fan. The fasteners 2936 also pass through standoffs 2940 that space the fan 2920 from the outside of the board, allowing air flow to be exhausted from the underside. In one embodiment, the spacing of the standoffs can be between about 0.5 and 2 cm, although a wide range of possible spacings can be explicitly contemplated. Note that a left or right side and / or top side mounting of the bracket and / or fan can also be explicitly contemplated in alternative embodiments. This can depend in part on the mounting mechanism of the camera. The fan can be covered by a conventional safety grille as part of the fastening mechanism. The bracket 2930 also includes a pair of exemplary tabs 2934 with fastening holes 2944 that can be used as part of the mounting mechanism to hang the camera assembly (and any cooperating accessories, such as a FOVE over the imaging scene).

[0120] Referring to Figure 30 The precise operation of a liquid lens (or another variable lens) assembly can be improved by setting a characteristic curve of drive current versus focal length (or lens optical power). That is, the operating curve of drive current for a lens assembly is typically non-linear over its entire range of focal lengths. The process 3000 is used for non-linearity. During manufacturing, or during calibration, the lens is driven to focus on an object / reference point at different known focal lengths (step 3010). The lens is driven to focus on an object / reference point at a known focal length. At this focus, the actual drive current is measured (step 3020). The process continues through an increment of focal lengths (decision step 3030 and step 3040) until all focal lengths have been traversed by the process. The decision step 3030 then jumps to step 3050, in which the data points on drive current are used to generate a characteristic curve of drive current versus focal length (or optical power). The characteristic curve indicates any non-linearity and it can be stored (e.g., a lookup table or a modeling equation) for subsequent use by the lens during runtime using the correction provided by the characteristic curve. It should be clear that the analysis and correction for non-linearity of the lens drive current can be implemented using a wide range of techniques that will be apparent to those skilled in the art.

[0121] Referring to Figure 31A process 3100 is shown that determines focus distance based on the overlap region in the FOVE image. The image frame 3110 is split into two portions 3120 and 3122 corresponding to each side of the overall extent of the FOVE. Each of the image portions 3120 and 3122 contains a cooperating overlap region 3130 and 3132 as described above. Within each of the overlap regions 3130, 3132 are one or more identifiable features (e.g. X 3140 and bar code 3142). These features can be any element of contrast visible in both overlap regions. The system identifies these features in each overlap region and determines their relative position and size (step 3150). These parameters change in known metric scales with different focus distances. In step 3160, the process 3100 compares the position shift (and size difference, if any) of the known corresponding values for the corresponding focus distances. More generally, this process works in the manner of a coincidence range finder. The value of the corresponding focus distance is then used in step 3170 to set the focus distance in the lens assembly. This process and other automatic adjustment processes described herein can be implemented by programming on the FPGA or using the system task function in one or more cores of the processor 114, which returns information to the FPGA so that the FPGA can perform the focus distance adjustment.

[0122] As Figure 32 shown, another process 3200 is used to more generally determine the speed and distance of an object through the field of view, which is useful in the auto focus and other automatic adjustment processes. In this embodiment, the system identifies one or more features in the object, typically some or all of the edges of the object itself or another closed or semi-closed element. In step 3220, the process records and stores the size of the feature (or features). The process then looks for the next image frame with the feature (or features) (decision step 3230) and / or has obtained enough frames to make a determination. If the next frame is to be processed, the process returns to step 3220 and records / stores the size of the feature (or features) in the next frame. This continues until there are no more frames available or enough frames have been processed. The decision step 3230 then jumps to step 3240, in which the size change between image frames is calculated. In step 3250, the process then calculates the relative distance and speed of the object given the knowledge of the time axis between image frames and the relative distance information (e.g. a characteristic curve or lookup table) about the speed of the given change in size over time. This can be used to control the focus of the lens assembly.

[0123] Referring Figure 33, two camera assemblies M and S (FOVE omitted) are positioned on each opposite side of the scene to image the front and back of a 3312 object 3310 having multiple IDs on different surfaces, only some of which are in the field of view of each camera, but all of which (e.g., front 3320, top 3322, and back 3324) are imaged by both camera assemblies M and S. Each camera assembly M and S includes respective illuminators MI and SI. It is noted that cameras M and S are placed in a master-slave configuration, respectively, where a back-mounted RS-485 connector 3330 on assembly M (which is part of the communication interface provided by the camera assembly and communicates with processor 114) is connected to a Y-cable 3332. The Y-cable includes opposing male and female connectors 3334. One of the connectors (3336) connects to the opposing connector 3338, which is connected via a second Y-cable 3340 to assembly S, which has a further connector 3342 to connect additional slave units. To avoid cross-talk between the illuminators, the processor of assembly M controls its imaging collection and its illumination at times TM, and controls the image capture / illumination of assembly S at non-continuous times TS. The capture times TM and TS are offset via a preset time axis that ensures that the image capture of each camera assembly is not interfered with by the other. The images can be processed by either core in each camera assembly, or the image data can be shared between the two camera assemblies using appropriate connections (e.g., network wiring Figure 2 270). For example, one set of cores can be adapted to find IDs in all images, while another set can be adapted to decode all images. Additional camera assemblies can be connected via appropriate cabling to implement an extended master-slave configuration (or other control configuration).

[0124] VI. SUMMARY

[0125] It should be clear that the embodiments described above for vision systems employing vision system cameras having multi-core processors, high speed, high resolution imagers, FOVE, auto-focus lenses, and pre-processors connected to the imagers for pre-processing image data provide highly desirable acquisition and processing speeds, as well as image clarity, in a wide range of applications. More particularly, the mechanism efficiently scans, requiring a wide field of view, size and location of the feature of interest, and objects moving relatively quickly with respect to the system's field of view. The vision system provides a physical package having a variety of physical interconnects to support various options and control functions. The package is arranged to optimize heat exchange with the surrounding environment, effectively dissipating heat generated internally, and includes heat dissipation structures to facilitate such heat exchange (e.g. fins). The system also allows for a variety of multi-core process optimization and load balancing of image processing and system operation (e.g. automatically adjusting tasks). At the same time, it is expressly contemplated that the methods and procedures described above for operating the camera assembly and performing vision system / decoding tasks can be combined in various ways to achieve the desired processing results. Likewise, the procedures can be switched depending on the processing conditions (e.g. procedure 2100 can be used and then switched to procedure 2300 as appropriate, etc.). Likewise, given multiple cores (more than two), multiple procedures can be executed simultaneously (e.g. procedure 2500 is executed in two of the four cores, while procedure 2600 is executed simultaneously in the other two of the four cores).

[0126] Exemplary embodiments of the present application are described above in detail. The embodiments disclosed herein are not intended to be exhaustive or to be construed as restricting the application to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. It is intended that the scope of the application encompass these and other alternatives. Each feature disclosed in this description, and / or the relative arrangement of the features, can be combined with one or more features of any of the illustrated embodiments, and / or the relative arrangement of the features, in any combination, of the illustrated embodiments. In addition, many of the concepts described herein are applicable, with or without modifications, to a variety of other devices and methods. For example, various directional and positional terms, such as "vertical," "horizontal," "up," "down," "bottom," "top," "side," "front," "back," "left," "right," and the like, are used herein with respect to the orientation of the described embodiments. These terms are used for purposes of illustration and convenience only and are not intended to limit the application to the position or orientation described. Also, although not described, it is expressly contemplated that various mounting mechanisms, supported by various structures (e.g., top boom, ceiling pole, beam, etc.), can be used to secure the camera assembly and other vision system components relative to the imaging scene, as appropriate. Also, although the FOVE is shown as a dual field of view expander, it is expressly contemplated that the FOVE can expand the field of view to three or more fields of view, each appropriately projected as a partial image on the imager. Also, although the FOVE expansion is described as occurring in the "width" dimension, it is expressly contemplated that the term "width" can be replaced by "height" herein, where such an application is desired. Thus, the expansion can occur in either the width or height. Also, it is expressly contemplated that the internal or external illumination can include wavelengths that are visible and / or invisible (e.g., near infrared light) for special functions, such as calibration, and that the imager can be adapted to read such wavelengths uniquely during specific tasks, such as calibration. Furthermore, although each of the FPGA and processor is shown herein as performing certain functions, it is expressly contemplated that some functions can be switched between either of these structures. In alternative embodiments, most of the tasks and functions can be performed by a multi-core processor, and the hardware / firmware-based functions performed by the FPGA can be reduced to a minimum, or the FPGA can be omitted altogether, in favor of different circuitry adapted to send image data from the image sensor to the processor in the appropriate format and at the appropriate time. Accordingly, this description is to be construed as illustrative only and is for the purpose of teaching those skilled in the art the general manner of practicing the application. It is to be understood that the forms of the application herein shown and described are to be taken as the presently preferred embodiments. Elements and materials can be substituted for one another herein without departing from the scope of the application.

Claims

1. A vision system comprising: a camera including an imager and a processor mechanism, the camera configured to capture successive images of at least one object having at least one symbol code; a pre-processor interconnected with the imager and configured to receive and pre-process the successive images from the imager at a first frame rate, wherein the processor mechanism is configured to receive the successive images from the pre-processor at a second frame rate, wherein the first frame rate is higher than the second frame rate; the processor mechanism including: a first core configured to receive a first image of the successive pre-processed images from the pre-processor, the first image containing at least a first portion of the at least one symbol code, the first core configured to decode the first portion of the at least one symbol code and generate a first decoding result, a second core configured to receive a second image of the successive pre-processed images from the pre-processor, the second image containing at least a second portion of the at least one symbol code, the second core configured to decode the second portion of the at least one symbol code and generate a second decoding result; and an illumination assembly configured to illuminate at least a portion of the at least one object; wherein the successive images are serially transmitted from the imager to the pre-processor and serially transmitted from the pre-processor to the processor mechanism; wherein the pre-processor is configured to transmit only image frames containing symbol code features to the processor mechanism.

2. The vision system of claim 1, wherein, the processor mechanism further including: a third core configured to receive a third image of the successive pre-processed images from the pre-processor, the third image containing at least a third portion of the at least one symbol code, the third core configured to decode the third portion of the at least one symbol code and generate a third decoding result, a fourth core configured to receive a fourth image of the successive pre-processed images from the pre-processor, the fourth image containing at least a fourth portion of the at least one symbol code, the fourth core configured to decode the fourth portion of the at least one symbol code and generate a fourth decoding result.

3. The vision system of claim 1, wherein, the symbol code includes at least one of a one-dimensional (ID) barcode or a two-dimensional (2D) barcode.

4. The vision system of claim 1, wherein, the first image and the second image are consecutive with respect to the successive images.

5. The vision system of claim 1, wherein, the at least one object includes a plurality of objects having a plurality of heights or a plurality of orientations.

6. The vision system of claim 1, wherein, the imager includes a CMOS sensor.

7. The vision system of claim 6, wherein, the CMOS sensor includes a grayscale CMOS sensor.

8. The vision system of claim 1, wherein, the illumination assembly includes one or more LEDs surrounding a lens of the camera, wherein each of the one or more LEDs corresponds to an associated LED lens.

9. The vision system of claim 1, wherein, the successive images include a burst of image frames at a predetermined acquisition rate when the at least one object is in a field of view of the camera.

10. The vision system of claim 1, wherein, the processor mechanism includes a dual-core DSP.

11. The vision system of claim 1, wherein, the pre-processor is configured to adjust at least one of a brightness of the illumination assembly or a focus of the camera.

12. The vision system of claim 1, wherein, the camera captures the successive images at a rate of 200 to 300 image frames per second.

13. The vision system of claim 1, wherein, There is overlap between the first image and the second image for the symbol code such that the symbol code is completely present in at least one of the first portion or the second portion.

14. The vision system of claim 1, wherein, The at least one object is moved relative to the camera via a moving pipeline.

15. A method of decoding a symbol code, comprising: capturing, by a camera, successive images of at least one object having at least one symbol code; transmitting, by a serial transmission, the successive images from an imager to a preprocessor; receiving and preprocessing, at the preprocessor, the successive images at a first frame rate, wherein the preprocessor is configured to transmit only image frames containing symbol code features to the processor mechanism; transmitting, by a serial transmission, the preprocessed successive images to the processor mechanism; receiving, at a processor mechanism, the preprocessed successive images at a second frame rate, wherein the first frame rate is higher than the second frame rate; receiving, at a first core of the processor mechanism, a first image of the preprocessed successive images, the first image containing at least a first portion of the at least one symbol code; decoding, at the first core of the processor mechanism, the first portion of the at least one symbol code to generate a first decoding result; receiving, at a second core of the processor mechanism, a second image of the preprocessed successive images, the second image containing at least a second portion of the at least one symbol code; decoding, at the second core of the processor mechanism, the second portion of the at least one symbol code to generate a second decoding result.

16. The method of claim 15, further comprising: illuminating at least a portion of the at least one object.

17. The method of claim 15, further comprising: receiving, at a third core of the processor mechanism, a third image of the preprocessed successive images, the third image containing at least a third portion of the at least one symbol code; decoding, at the third core of the processor mechanism, the third portion of the at least one symbol code to generate a third decoding result; receiving, at a fourth core of the processor mechanism, a fourth image of the preprocessed successive images, the fourth image containing at least a fourth portion of the at least one symbol code; decoding, at the fourth core of the processor mechanism, the fourth portion of the at least one symbol code to generate a fourth decoding result.

18. The method of claim 15, wherein, The symbol code comprises at least one of a one-dimensional (ID) barcode or a two-dimensional (2D) barcode.

19. The method of claim 15, wherein, The first image and the second image are consecutive with respect to the successive images.

20. The method of claim 15, wherein, The successive images comprise a burst of image frames at a predetermined acquisition rate when the at least one object is in a field of view of the camera.

21. The method of claim 15, wherein, There is overlap between the first image and the second image for the symbol code such that the symbol code is completely present in at least one of the first portion or the second portion.

22. The method of claim 15, further comprising: The at least one object is moved relative to the camera via a moving pipeline.

Images