Systems and methods for high-resolution multiline imaging

Abstract

Systems and associated methods implement and / or perform acquiring a plurality of images with the camera, including: activating one or more illumination sources to provide different illumination conditions according to a predetermined order, and under each of the plurality of illumination conditions, capturing one of the plurality of images. Between the acquiring of successive images, a transport device is configured to cause the object to move a distance relative to the camera in a movement direction transverse to a row direction of the camera that is equal to (I).

Description

SYSTEMS AND METHODS FOR HIGH-RESOLUTION MULTILINE IMAGINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This claims priority to and the benefit of U.S. Provisional Application No. 63 / 674,073, filed July 22, 2024, and U.S. Provisional Application No. 63 / 665,376, filed June 28, 2024, the entire contents of which are each herein incorporated by reference.STATEMENT REGARDING FEDERALLY FUNDED RESEARCH

[0002] Not applicable.BACKGROUND

[0003] Machine vision systems, also termed “vision systems” herein, are used to perform a variety of tasks in a manufacturing environment. In general, a vision system consists of one or more cameras with an image sensor (or “imager”) that acquires grayscale or color images of a scene that contains an object or surface of interest. Images of the object / surface can be analyzed to provide data / information to users and associated manufacturing processes. The data produced by the image is typically analyzed and processed by the vision system in one or more vision system processors that can be purpose-built, or part of one or more software application(s) using either conventional or deep 1 earning / ALbased processes, instantiated within a general purpose computer (e.g. a PC, laptop, tablet or smartphone), and / or a custom processor. Some types of tasks performed by the vision system can include inspection of objects and surfaces (e.g. web inspection), such as those residing on a moving conveyor arrangement or motion stage, for expected features and / or defects.

[0004] Surface inspection for various markets (e.g., for inspecting coated battery electrodes for electric vehicles) benefits from high scan speed and resolution. Some comparative examples use multiple camera lines to create line scan images with different illuminations, thus providing increased imaging speed. Such comparative examples of inspection modalities may provide high speed at medium resolutions; however, these examples may only be usable with alimited number of expensive high-end area scan cameras. There exists a need to utilize multiline imaging with a wider range of cameras and camera sensors, and to provide high speed, high resolution imaging at a lower cost than the comparative examples.SUMMARY

[0005] In an aspect, the present disclosure provides a method of acquiring images of an object acted upon by a transport device using a camera that includes a plurality of lines of pixels. The method comprises acquiring a plurality of images with the camera, including: activating one or more illumination sources to provide a plurality of illumination conditions according to a predetermined order, and under each of the plurality of illumination conditions, capturing one of the plurality of images, wherein, between the acquiring of successive ones of the plurality of images, the transport device is configured to cause the object to move a distance relative to the camera in a movement direction transverse to a row (or line) direction of the camera that is equalw^ereisatotal number of the plurality of lines (or rows) of pixels, M is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient. (“Row” and “line” are used herein interchangeably relative to image acquisition, including with regard to “row direction” and “line direction” )

[0006] In another aspect, the present disclosure provides an imaging system. The system comprises one or more illumination sources configured to illuminate an object acted upon by a transport device with a plurality of illumination conditions; a camera that includes a plurality of lines of pixels; and at least one control unit in communication with the one or more illumination sources and the camera, the control unit configured to cause the imaging system to perform operations that include: acquiring a plurality of images with the camera, including: activating the one or more illumination sources to provide a predetermined order of different illumination conditions of the plurality of illumination conditions, and under each of illumination conditions of the predetermined order of illumination conditions, capturing a respective one of the plurality of images, wherein, between the acquiring of successive ones of the plurality of images, the transport device is configured to cause the object to move a distance relative to the camera in a movement direction transverse to a row direction of the camera that is equal to where N is a totalnumber of the plurality of lines of pixels, AT is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient.

[0007] In another aspect, the present disclosure provides a method of acquiring images of an object acted upon by a transport device. The method comprises acquiring an image of the object with a line scan camera, including: selecting an illumination source from a plurality of illumination sources, activating one or more illumination sources for a predetermined period of time to provide the selected illumination condition, accumulating a charge in a photosensitive element of the line scan camera, deactivating the select illumination source, and activating a charge transfer mechanism of the line scan camera to read out image data corresponding to the charge; and repeating the operation of acquiring for a plurality of different illumination conditions, including selecting a different illumination source from the plurality of illumination sources, wherein, between successive acquiring operations, the transport device is configured to move a predetermined distance.

[0008] In another aspect, the present disclosure provides an imaging system. The system comprises one or more illumination sources configured to selectively illuminate an object acted upon by a transport device with any of a plurality of illumination conditions; a line scan camera; and at least one control unit in communication with the one or more illumination sources and the line scan camera, the control unit configured to cause the imaging system to perform operations that include: acquiring an image of the object with the line scan camera, including: activating the one or more illumination sources to provide an illumination condition of the plurality of illumination conditions for a predetermined period of time, accumulating a charge in a photosensitive element of the line scan camera corresponding to the activation of the illumination source, deactivating the illumination source to stop providing the selected illumination condition, and activating a charge transfer mechanism of the line scan camera to read out image data corresponding to the charge; and repeating the operation of acquiring for multiple different illumination conditions of the plurality of illumination conditions wherein, between successive iterations of acquiring, the transport device is configured to move a predetermined distance.

[0009] Other features, objects, and advantages of the present invention are apparent in the detailed description that follows. It should be understood, however, that the detailed description, while indicating embodiments of the present invention, is given by way of illustration only, notlimitation. Various changes and modifications within the scope of the invention will become apparent to those skilled in the art from the detailed description.BRIEF DESCRIPTIONS OF THE DRAWINGS

[0010] FIG. 1 illustrates an example vision system arrangement according to various aspects of the present disclosure.

[0011] FIG. 2 illustrates an example procedure for operating an image arrangement according to various aspects of the present disclosure.

[0012] FIG. 3A illustrates an example illumination system according to various aspects of the present disclosure.

[0013] FIG. 3B illustrates an example timing diagram for the system of FIG. 3 A.

[0014] FIG. 4A illustrates an example pixel layout according to various aspects of the present disclosure.

[0015] FIG. 4B illustrates an example pixel layout according to various aspects of the present disclosure.

[0016] FIG. 4C illustrates an example pixel layout according to various aspects of the present disclosure.

[0017] FIG. 4D illustrates an example pixel layout according to various aspects of the present disclosure.

[0018] FIG. 5 illustrates an example imaging method according to various aspects of the present disclosure.

[0019] FIG. 6A illustrates an example illumination sequence according to various aspects of the present disclosure.

[0020] FIG. 6B illustrates an example illumination sequence according to various aspects of the present disclosure.

[0021] FIG. 7 illustrates example imaging results according to various aspects of the present disclosure.

[0022] FIG. 8 illustrates example imaging results according to various aspects of the present disclosure.

[0023] FIG. 9A illustrates an example illumination system according to various aspects of the present disclosure.

[0024] FIG. 9B illustrates an example timing diagram for the system of FIG. 9A.

[0025] FIG. 10 illustrates an example imaging method according to various aspects of the present disclosure.

[0026] FIG. 11 illustrates an example illumination sequence according to various aspects of the present disclosure.

[0027] FIG. 12 illustrates example imaging results according to various aspects of the present disclosure.

[0028] FIG. 13 illustrates example imaging results according to various aspects of the present disclosure.DETAILED DESCRIPTION

[0029] Before the present invention is described in further detail, it is to be understood that the invention is not limited to the particular embodiments described. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. The scope of the present invention will be limited only by the claims. As used herein, the singular forms “a,” “and,” and “the” include plural embodiments unless the context clearly dictates otherwise.

[0030] It should be apparent to those skilled in the art that many additional modifications beside those explicitly described are possible without departing from the inventive concepts. In interpreting this disclosure, all terms should be interpreted in the broadest possible manner consistent with the context. Variations of the term “comprising,” “including,” or “having” should be interpreted as referring to elements, components, or steps in a non-exclusive manner, so the referenced elements, components, or steps may be combined with other elements, components, or steps that are not expressly referenced. Embodiments referenced as “comprising,” “including,” or “having” certain elements are also contemplated as “consisting essentially of’ and “consisting of’ those elements, unless the context clearly dictates otherwise. It should be appreciated that aspects of the disclosure that are described with respect to a system are applicable to the methods, and vice versa, unless the context explicitly dictates otherwise.

[0031] Any citations to publications, patents, or patent applications herein are incorporated by reference in their entirety. Any numerals used in this application with or without about / approximately are meant to cover any normal fluctuations appreciated by one of ordinary skill in the relevant art.

[0032] Numeric ranges disclosed herein are inclusive of their endpoints. For example, a numeric range of between 1 and 10 includes the values 1 and 10. When a series of numeric ranges are disclosed for a given value, the present disclosure expressly contemplates ranges including all combinations of the upper and lower bounds of those ranges. For example, a numeric range of between 1 and 10 or between 2 and 9 is intended to include the numeric ranges of between 1 and 9 and between 2 and 10.

[0033] As used herein, the terms “component,” “system,” “device” and the like are intended to refer to either hardware, firmware, software, software in execution, or any combination thereof. The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0034] Furthermore, the disclosed subject matter may be implemented as a system, method, apparatus, or article of manufacture using standard programming and / or engineering techniques and / or programming to produce hardware, firmware, software, or any combination thereof to control an electronic based device to implement aspects detailed herein.

[0035] Unless specified or limited otherwise, the terms “connected,” “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings. As used herein, unless expressly stated otherwise, “connected” means that one element / feature is directly or indirectly connected to another element / feature, and not necessarily electrically or mechanically. Likewise, unless expressly stated otherwise, “coupled” means that one element / feature is directly or indirectly coupled to another element / feature, and not necessarily electrically or mechanically.

[0036] As used herein, the term “processor” may include one or more processors and memories and / or one or more programmable hardware elements. As used herein, a “processor” may include one or more individual processing units and / or one or more individual processing cores. Where a processor is referred to as performing a method or operation, various procedures,sub-operations, steps, etc. may be performed by the same processing unit / core and / or by different processing units / cores, in series or in parallel, in any combination. As used herein, the term “processor” is intended to include any of types of processors, central processing units (CPUs), graphics processing units (GPUs), microcontrollers, digital signal processors, or other devices capable of executing software instructions. For the avoidance of doubt, cloud processing is contemplated in the definition of a processor.

[0037] As used herein, the term “memory” includes a non-volatile medium, e.g., a magnetic media or hard disk, optical storage, or flash memory; a volatile medium, such as system memory, e.g., random access memory (RAM) such as dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), extended data out (EDO) DRAM, extreme data rate dynamic (XDR) RAM, double data rate (DDR) SDRAM, etc.; or an installation medium, such as software media, e.g., a CD-ROM, or floppy disks, on which programs may be stored and / or data communications may be buffered. The term “memory” may also include other types of memory or combinations thereof. For the avoidance of doubt, cloud storage is contemplated in the definition of memory.

[0038] Multiline imaging modalities may be implemented with a variety of camera and sensor candidates. One candidate class of cameras are time delay integration (TDI) line scan cameras. These cameras have several lines (e.g., 2, 4, 8, 16, 32, 64, 128, 256, or some other number of lines). Though they have multiple lines, according to comparative examples of implementations it is impossible or impractical to read out a small number of lines at high speed separately. The present disclosure provides systems and methods to use and / or modify TDI line scan cameras and read out multiple lines separately at high speed.

[0039] Another type of camera and sensor candidate is line scan cameras and sensors with a small number of lines (e.g., 2, 3, 4, 5, 6, 8, 12, 16, or 32 lines), or area scan cameras that are windowed to a small number of lines. For comparative examples using these cameras and sensors, the lines cannot be read out separately or the number of lines is less than the number of illuminations. The present disclosure provides systems and methods to use and / or modify such cameras and read out multiple lines separately at high speed.

[0040] Yet another candidate class of cameras are contact image sensor (CIS) cameras. A CIS camera may have one or multiple lines (e.g., 2, 3, 4, or more lines), and may be capable of color and / or monochrome imaging. In one example, a CIS camera may have multiple lines andinclude color filters, with each line corresponding to imaging in a different color (e.g., a line of pixels including red color filters, a line of pixels including green color filters, and a line of pixels including blue color filters). In another example, a CIS camera may have multiple lines, with each line corresponding to imaging across the same wavelength range (i.e., monochrome imaging). A CIS camera according to either example may additionally include a line for capturing an image in the infrared wavelength range.

[0041] FIG. 1 shows avision system arrangement 100 according to an illustrative example. In this exemplary system, a moving surface 110 (e.g., of a transport device) moves in a direction of motion (shown by an arrow) through an inspection area. While in the illustrated example, the direction of motion is linear (e.g., the transport device is a conveyor, a linear axis, etc.), in some implementations the direction of motion may be rotational (e.g., the transport device rotates about a rotation axis, etc.). The surface 110 can be a conveyer having a predetermined width carrying objects 112 of one or more sizes / shapes, or a web. The surface of the object 112 or web can be inspected for various defects, details, or features that can define a width, length, and height / depth. Thus, such features may benefit from multiple, discrete illumination conditions to be accurately imaged. The use of photometric techniques can render such features with sufficient detail to perform classical and / or deep-learning-based machine vision analysis.

[0042] As shown, an imaging system is mounted over the surface 110 at the inspection area. The imaging system in the illustrated example includes an illumination source 120. In the illustrated example, the illumination source 120 is shown as a dome, on which one camera 122 and a plurality of illumination devices 124 are mounted. In other examples, as also detailed below, a variety of one or more other illumination sources can be used, with corresponding one or more illumination devices (i.e., one or more LEDs or other devices of various known configurations to provide illumination as part of an illumination source). Further, a camera may in some cases be mounted separately from an illumination device, or an illumination device can be mounted on a camera. Thus, reference herein to the illumination source 120 illustrated in FIG. 1 should be considered to also encompass types of illumination sources other than domes, unless otherwise expressly required. The camera can include an appropriate image sensor (e.g., a CMOS sensor, a CCD sensor, and the like) and optics that should be clear to those of skill. In some examples, the image sensor is a line scan sensor (e.g., a TDI line scan sensor, a line scan sensor with a plurality of, e.g., twenty or fewer, lines of pixels), a contact image sensor (CIS), or an area scan sensorthatis windowed, either by software or hardware, to a plurality of (e.g., forty or fewer) lines of pixels. The camera 122 defines a field of view (FOV) sufficient to image the entire or a part of the surface width of the surface 110. In this example, the optical axis of the camera 122 is approximately perpendicular to the local plane of the surface 110. While FIG. 1 illustrates only one camera 122, in other implementations a plurality of cameras 122 may be provided, for example to provide stereoscopic imaging. Where a plurality of cameras 122 are implemented, individual cameras 122 may image only a portion of the width of the surface 110, such that the cameras 122 collectively image the entirety of the width of the surface 110. Moreover, some or all of the cameras 122 may be oriented such that their respective optical axes are oriented oblique to the local plane of the surface 110, for example to image multiple sides of the object 112.

[0043] The camera 122 can include various on-board vision system processes / processors and / or can be linked (wired or wirelessly), using an appropriate networking protocol, to a vision system process / processor 130. The vision system processor 130 can be a standalone computing device, and / or instantiated on a general purpose PC, laptop, tablet, or other device 150 with an appropriate user interface 152 (e.g., keyboard, touchscreen, etc.) and display 154. The vision system processor 130 may receive image data from the camera 122 and provide various controls over the link(s) to the camera 122. Image data may be analyzed for features and other information using classical and / or deep-learning-based tools / processes 132, which can include defect detection, defect classification, edge-finding, blob tools, ID readers, etc., as well as image-trained neural networks. The tools / processes 132 can be coordinated with information relative to motion of the surface, which can be derived via an encoder 140 and associated encoder signal 142. The encoder 140 can be operatively connected with the drive components for the surface 100, or otherwise responsive to surface motion (e.g., via mechanical, optical, etc. sensing). Other forms of motion sensing can be employed in alternate implementations.

[0044] The tools 132 can employ photometric processes on successive images from the camera 122. They may operate with an illumination control process / processor 134 that is interconnected with the tools 132 and provides and / or receives trigger signals to / from the camera 122 and / or the encoder 140. The illumination devices 124 may be positioned and / or oriented so as to provide an illumination pattern to the inner surface of the dome 120, which in turn illuminates the inspection area 114 that spans the width of the surface 110 or a part thereof. The illumination devices 124 may be any device configured to emit light in response to electrical signals; forexample, LEDs, laser emitters, and the like. The motion signals from the encoder 140 can be used to trigger an illumination control process / processor, the camera 122, and / or different illumination arrays within the overall arrangement (e.g., different individual illumination devices 124 or combinations thereof) as described in detail below. One or more images (e.g., a plurality of images) are acquired of the inspection area scene in each illumination configuration, and the group of images may be analyzed by the tools 132 using photometric and / or other known techniques from which feature information is determined.

[0045] The feature information is used by further analysis processes / processors 136 to generate output results. The output results can be, for example, provided to the display 154. These results can include the surface texture, gradient X / Y, curvature, and / or depth. These results can provide information on object or surface condition, including but not limited to a presence or absence of a three-dimensional (3D) (e.g. physical / structural) defect and / or or a reflectance defect. The 3D defect or the reflectance defect can be related to one or more of a scratch, a dent, a gouge, debris presence, a buff mark, improper material composition, and / or a variation in material thickness. The results can be transmitted to downstream utilization processes and / or devices to perform desired functions; for example, an alarm can be generated, motion can be stopped, a reject action can occur, the results can be displayed visually / graphically or via text, and / or a defective object or surface can be marked. Likewise, analyzed surfaces can be sorted by appropriate gating functions based upon differentiated features.

[0046] The illumination source (e.g., dome) 120 may be provided to illuminate the object 112 in an indirect way, to reduce and / or eliminate the incidence of ambient (i.e., environmental) light on the object 112. Where present, a dome may be constructed from any appropriate material (e.g., metal, polymer, composite, etc.), and can exhibit heat-dissipation characteristics. However, in some implementations (e.g., in which the illumination devices provide sufficient illumination intensity such that ambient light does not affect imaging) a dome may be omitted. In these implementations, for example, one or more illumination sources may be positioned and / or oriented to illuminate the object 112 in a direct way. Moreover, the illumination devices 124 may not be point-like sources, and instead may be generally one-dimensional (e.g., bar-like) or two- dimensional (plane-like) sources. While FIG. 1 illustrates four illumination devices 124, in other implementations one or more illumination sources can include any number of illumination devices124 (e g., so that the illumination source(s) and / or illumination device(s) can provide an integer number M of different illumination conditions).

[0047] In one particular arrangement, the illumination devices 124 may include a plurality of bar lights defining a multi-channel arrangement that produces collimated light beams at multiple oblique angles, respectively, with respect to the vertical (e.g., the camera optical axis). The oblique (i.e., non-parallel with the optical axis of the camera 122) angles can vary. In a non-limiting example, the angles of the opposing oblique light beams may be symmetric about the optical axis. Each light channel / direction may be activated by a separate circuit (which may be directed by the illumination control 134 or another process / processor), so that each channel can independently strobe its array of light sources to propagate light beams at each respective angle. In this example, the plurality of arrays of light sources and associated beams may be interleaved across the width of the transport device 110 in an alternating manner.

[0048] In some examples, a bar light may have illumination devices configured as individual light emitters arranged on a sawtooth-shaped or otherwise concavo-convex surface, such that the bar light is capable of providing illumination at two different illumination angles. In such examples, the two sets of light emitters may be separately connected so as to receive separate illumination signals from the illumination controller 134. In other examples, a bar light may have individual light emitters arranged on a substantially planar surface. In such examples, the light emitters may be overlaid by segments of two pieces of direction turning film (or another ID array / structure of prismatic material or reflective material, such as adjustable mirrors) that alternate between opposing angles of light redirection to provide multi-angle oblique illumination.

[0049] As also noted above, in different examples, one or more illumination sources of a variety of configurations can be used to provide a plurality of illumination conditions. In some examples, the light emitters may be arranged on a grid. For example, some implementations can use lighting systems - and corresponding types and arrangements of illumination sources - disclosed in PCT application no. PCT / US2025 / 031743, titled “SYSTEM AND METHOD FOR ILLUMINATING AND IMAGING SURFACES IN MOTION,” the entirety of which is incorporated herein by reference. In some examples, at least two illumination devices can be configured to generate at least two different illumination distributions or patterns on an emitting surface (e.g., with planar illumination that illuminates objects from several spatial directions), including as detailed in U.S. Patent No. 10,760,901, titled “Device for illuminating objects,” theentirety of which is incorporated herein by reference. In some examples, a tunnel can be arranged above a surface being imaged to provide illumination (e.g., with the tunnel extending to overly a width of the surface) A tunnel can include, in some cases, one or more illumination devices and / or one or more light reflecting or light scattering inner surfaces. In some examples, illumination sources can include technologies disclosed in U.S. Patents 7,808,644 (“Device for optically measuring the shapes of objects and surfaces”), 7,453,777 (“Method and device for optical form measurement and / or estimation“), and 8,830,386 (“Camera chip, camera and method for image recording”), and EP Patent 3034993B1 (“Verfahren zur optischen Formerfassung und / oder Formprufung eines Gegenstands und Beleuchtungseinrichtung”), all of which are incorporated herein by reference, and / or various known light bar technologies, photometric stereo technologies, and / or spectral technologies (i.e., that use light of different wavelengths (e.g., multiple and / or variable wavelengths).

[0050] Regardless of the arrangement of the illumination devices 124, as illustrated in the example configuration of the illumination source 120 of FIG. 1, the illumination devices 124 may generate interleaved arrangements of light so as to generate lighting of the scene by a plurality of different illumination fields. Or, in other examples, other illumination sources can similarly generate lighting with a plurality of different illumination fields. In some cases, these illumination fields can be roughly equivalent to a plurality of distance point sources arranged along the same or different directions, respectively. Sets of one or more illumination devices (or sources) may be successively strobed to acquire a repeating sequence of images as the object (or web) surface 110 is scanned (e.g., via successive strobing of channels, with reference to example discussion above). The image sequence may then be deinterlaced and used to generate (e.g.,) photometric images (texture, gradient X / Y, curvature, depth, etc.).

[0051] FIG. 2 shows a generalized procedure 200 for operating an imaging arrangement such as shown in FIG. 1 and described above. In operation 202, a web (e.g., the transport device 110) or other transport device is advanced. The system may detect motion of the surface under inspection using an encoder signal of an encoder 140 or other technique and quantify the degree of motion. When appropriate motion has occurred, an image capture operation 204 is performed.

[0052] In a general example of operation 204, the system may trigger an illumination control 134, the camera 122, and / or at least one illumination device or array. Illumination can be provided in one or more wavelengths to achieve different effects, assuming illumination deviceswith a plurality of output wavelengths are provided. While the surface is illuminated by the array, at least one image is acquired by the associated camera(s). The procedure 200 may then determine if a next illumination device or array should be triggered as part of the overall image processing task (e.g., photometric imaging). If so, operations 202 and 204 are repeated. Once the last illumination device or array is triggered (e.g., once each illumination condition has been imaged), and all corresponding images have been acquired and stored (e.g., in processor memory), the system may perform photometric and / or other image processing tasks on the stored image. These processes may be used to generate results that are used by downstream processes, as described above. The procedure 200 may repeated any number of times as surface motion continues.

[0053] Operation 204 itself may include several sub-operations, the details of which may depend on the details of the imaging system (e.g., the details of the image sensor). For example, the details of operation 204 may depend on the spacing between adjacent lines of pixels in the camera.

[0054] As discussed above, some examples can include successive selection of different illumination sources (or devices) to provide corresponding illumination patterns. In some examples, the same one or more illumination sources (or devices) can be controlled to selectively provide successive different illumination conditions. Thus, unless otherwise indicated, discussion above and below with regard to selection or operation of different illumination sources (or devices) to provide different illumination conditions should be understood to also include operation of the same one or more illumination sources (or devices) to provide a succession of selected illumination conditions. Thus, generally, different illumination conditions can be selectively provided via selection of different illumination sources (or devices) to provide respective different illumination conditions and / or via control of the same illumination source(s) (or device(s)) to provide multiple different illumination conditions.

[0055] FIG. 3A illustrates one example of an illumination system 300 including a control unit 302 (e.g., processor 130), an imaging unit 304 (e.g., camera 122, which in some embodiments may advantageously be a line scan camera, a CIS camera, or an area scan camera windowed to a small number of lines), and an illumination unit 306 (e.g., illumination devices 124). The example of FIG. 3A is illustrative and not limiting, and in some implementations the control unit 302, imaging unit 304, and / or illumination unit 306 may have a different structure and / or architecturethan shown in FIG. 3 A. For example, the illumination unit 306 may use any variety of appropriate illumination sources or illumination devices, as discussed above.

[0056] As illustrated, the imaging unit 304 includes a plurality of lines of pixels, of which two are shown. The imaging unit 304 is configured for parallel readout using a transfer gate, which in the illustrated example is implemented as a transistor that operates in response to a transfer control signal XFER. When the control signal XFER is high (in the illustrated example), a signal from a line of pixels is transferred to the output OUT. In examples, a plurality of lines can be read out in parallel. The illumination unit 306 includes a plurality of illumination devices. Each illumination device comprises a light emitting diode and a transistor arranged between a power supply voltage Vccand a ground voltage. In response to an illumination control signal (IL1 to ILM), the light emitting diode selectively emits light. The light may be in the visible and / or infrared wavelength ranges. The control signals TRG and IL 1 -ILM may be provided by the control unit 302 to the imaging unit 304 and / or the illumination unit 306, respectively.

[0057] FIG. 3B illustrates an example timing diagram for the imaging system 300. When an illumination control signal is pulsed high, the corresponding transistor becomes conductive and the corresponding light emitting device emits light. In synchronicity with the illumination, the imaging unit 304 captures an image of the inspection area. For each image that has been captured corresponding to a different illumination condition, the transfer control signal XFER is pulsed high to cause the imaging unit 304 to output imaging data.

[0058] In the illustrated example, the lines of pixels in the imaging unit 304 are separated by an extra spacing. FIG. 4A illustrates the pixel layout for such an imaging unit, which has a total of N lines of pixels. FIGS. 4A-D should be understood as a projection of the physical pixel layout into the object space of the imaging system (i.e., the three-dimensional space in which the object and the imaging system are located). As shown in FIG. 4A, adjacent lines of pixels are separated by an extra spacing S, which is a multiple n times the height H of a single line of pixels. The multiple n may be any number, including integers and fractions. The distance between the top of one line of pixels and the top of the next line of pixels is referred to as the pixel pitch P, which is equal to the extra spacing S plus the height H. The extra spacing S and pixel height H, and thus the pixel pitch P, are usually the same between each adjacent pair of lines. The quantities H, S, and P refer to the object space (and not the image space) of the imaging system. Moreover, inimplementations in which pixel binning in the row and / or column directions were implemented, H corresponds to the effective pixel height and P corresponds to the effective pixel pitch.

[0059] For the imaging system 300 as shown in FIGS. 3A-4A, FIG. 5 shows an example method 500 of the imaging operation 204. The method 500 includes acquiring a first plurality of images with the camera, which in turn includes operation 502 of activating respective ones of the plurality of illumination sources (e.g., illumination devices 124) according to a first predetermined order (also referred to herein as a first sub-order), each of the plurality of illumination sources corresponding to a different illumination condition (e.g., a different intensity property: current control, voltage control, switch-on timing, dimming, e.g., using pulse width modulation; modulation property: amplitude modulation, frequency modulation, pulse width modulation or other types of modulation; position property: local distribution of one or several illumination sources; angular illumination distribution property; spectral and wavelength properties; polarization property; coherence property; or any other illumination property or combination thereof, etc.) and operation 504 of, under each of the plurality of illumination conditions, capturing one of the first plurality of images. The method 500 includes acquiring a second plurality of images with the camera, which in turn includes operation 506 of activating respective ones of the plurality of illumination sources according to a second predetermined order (also referred to herein as a second sub-order); and operation 508 of, under each of the plurality of illumination conditions, capturing one of the second plurality of images. Accordingly, each portion of the object 112 is imaged at least once for each illumination condition.

[0060] In other examples, such as when a CIS camera is used, the lines of pixels in the imaging unit may or may not be separated by an extra spacing. FIGS. 4B-D illustrate the pixel layout for such an imaging unit, which has a total of 3 lines of pixels. In other implementations, any number / V of lines may be present. A CIS camera may include several pixel units, in particular in the form of sensor chips, each of which includes a number of columns of pixels. By arranging multiple pixel units next to one another, the total number of columns may be increased and thus a larger area may be imaged. In FIGS. 4B-D, the “row” direction is referred to as X, and extends in a direction transverse (e.g., perpendicular) to the direction of motion of the transport device; and the “column” direction is referred to as Y, and extends in a direction generally along (e.g., parallel to) the direction of motion of the transport device. In general, for FIGS. 4B-D, the quantity AX refers to the offset distance between adjacent pixel units in the row direction, and may be a multiplei of the height H. Thus, the offset distance in the row direction may have the general form AX = i x H. In this expression, z may be negative (e.g., if the pixel units are arranged such that the rows are overlapped, as shown in FIGS. 4C-D), positive (e.g., as shown in FIG. 4B), or zero. Moreover, the quantity AY refers to the offset distance between adjacent pixel units in the column direction, and may be a multiple / of the height H. Thus, the offset distance in the column direction may have the general form AY = j x H. The quantities AX and AY refer to the object space (and not the image space) of the imaging system. Moreover, in implementations in which pixel binning in the row and / or column directions were implemented, AX and AY correspond to the effective offset distances.

[0061] As shown in FIG. 4B, within each pixel unit neighboring lines of pixels abut one another such that 5 = 0 x H and the pixel pitch P = H + S is equal to the height H of a single line of pixels. The adjacent pixel units may be separated by a spacing AX in the row direction, which is a multiple i times the height / / of a single line of pixels (e.g., to allow space for interconnections between units, associated circuitry, etc.), such that they are not directly adjacent. Pixel data for one or multiple columns of pixels that are not present due to the spacing AX may be interpolated from neighboring pixels. If required, interpolation may also be used to compensate for tolerances in placement of pixel units. The multiple i may be any number, including integers and fractions, including positive numbers, negative numbers and zero. In the illustrated example of FIG. 4B, z = 1. In some implementations, the adjacent pixel units may also be offset by a spacing AY in the column direction, which is a multiple j times the height / / of a single line of pixels. The multiple j may be any number, including integers and fractions, including positive numbers, negative numbers and zero. In the illustrated example of FIG. 4B,j = 0. Moreover, in implementations in which pixel binning in the row and / or column directions is implemented, AY corresponds to the effective spacing in the row direction and / or AF corresponds to the effective spacing in the column direction. The relative arrangement of adjacent pixel units in this example can be repeated for multiple pixel units such that, for example, the lines of pixels of the first pixel unit are aligned with the lines of pixels of the third pixel unit (and the fifth pixel unit, and the seventh pixel unit, and so on) and the lines of pixels of the second pixel unit are aligned with the lines of pixels of the fourth pixel unit (and the sixth pixel unit, and the eighth pixel unit, and so on).

[0062] By using a staggered arrangement, it may be possible to avoid a separation gap in row direction between adjacent pixel units, in particular by choosing AX < 0 x H. One example of such an arrangement is shown in FIG. 4C. As shown in FIG. 4C, the pixel unit overlaps lines of another pixel unit such that there is an overlap of a certain number of pixels |z|. In the illustrated example, z = -3 and there is an overlap of three pixels. Because there is an overlap in the row direction, there are no columns with “missing” pixels. In this example, there is usually no need to interpolate “missing” pixel data. Still, interpolation may be used to compensate for tolerances in placement of pixel units. Moreover, in FIG. 4C, j = 4 , and there is an offset in the column direction equal to AF = 4 x H between the two pixel units. The example of FIG. 4C may have advantages of multiple lines, high resolution, and no “missing” pixels due to the staggered alignment. In this example, one illumination sub-order may be sufficient, because there is no extra spacing S between lines. The relative arrangement of adjacent pixel units in this example can be repeated for multiple pixel units such that, for example, the lines of pixels of the first pixel unit are aligned with the lines of pixels of the third pixel unit (and the fifth pixel unit, and the seventh pixel unit, and so on) and the lines of pixels of the second pixel unit are aligned with the lines of pixels of the fourth pixel unit (and the sixth pixel unit, and the eighth pixel unit, and so on).

[0063] A staggered arrangement is also possible if there is an extra spacing S between rows within a pixel unit, as illustrated in FIG. 4D. In FIG. 4D, similar to the example of FIG. 4C, the pixel lines of one pixel unit overlap with the pixel lines of another pixel unit such that there is an overlap of a certain number of pixels, equal to \i\. In the illustrated example, z = -3 and there is an overlap of three pixels. In FIG. 4C,j = 6 (i.e., there are three rows of pixels with height / 7, separated by an extra spacing equal XoH, and an offset in the column direction equal to AF = 6 X H between the two pixel units). Because there is an extra spacing S between lines of pixels within a pixel unit, more than one illumination sub-order may be used. The relative arrangement of adjacent pixel units in this example can be repeated for multiple pixel units such that, for example, the lines of pixels of the first pixel unit are aligned with the lines of pixels of the third pixel unit (and the fifth pixel unit, and the seventh pixel unit, and so on) and the lines of pixels of the second pixel unit are aligned with the lines of pixels of the fourth pixel unit (and the sixth pixel unit, and the eighth pixel unit, and so on).

[0064] While FIGS. 4A and 4D illustrate an extra spacing S that is equal to H, in other examples S may be only approximately equal to H (e.g., within 25%). In this case, S = zz x H andn ~ 1 . In such an example, P ~ 2 x H . In yet other examples there may be S = n x H and n > 1 (e.g. n = 2, 3, .. . ). In yet other examples there may be a combination of different values of S for different lines. Moreover, while FIGS. 4C and 4D illustrate a separating gap of H in the column direction between the two pixel units (resulting in an offset of pixel units of AY = 4 x H for FIG. 4C and AY = 6 x H for FIG. 4C) this is merely an example. In other implementations, the pixel units may be separated by a larger or smaller gap, resulting in a larger or smaller offset of pixel units in the column direction. Thus, for example, while FIG. 4D illustrates an example where AY = 6 x H, in other examples AY > 6 x H.

[0065] FIG. 5 illustrates an example in which the first and second pluralities of images are acquired in an interleaved manner, such that a first portion of operation 502 is performed to activate one illumination source (to provide a first illumination condition), followed by a first portion of operation 504 to acquire a first one of the plurality of first images, then a first portion of operation 506 is performed to activate one illumination source, followed by a first portion of operation 508 to acquire a first one of the plurality of second images, then followed by a second portion of operation 502 to activate another illumination source (or differently control the one illumination source) to provide a second illumination condition, and so on. In alternatives, the images may be instead acquired in parallel or in series, such that operations 506 and 508 are performed simultaneously with or following, respectively, operations 502 and 504.

[0066] If the lines of pixels are not separated by an extra space, as is the case for certain CIS camera examples (i.e., A = 0 x H, P = H + A = H), it may not be necessary to use both first and second sub-orders. In such an example, the method 500 may omit operations 506, 508, and (depending on the implementation) 510. Because there is no extra spacing S, one cycle of illumination conditions will be sufficient to capture information about the entirety of the object under all illumination conditions. However, the method 500 may include multiple image capture operations for each illumination condition (e.g., to increase the signal -to-noise ratio (SNR)), in which case operation 510 may be retained.

[0067] Between acquiring successive images, as noted above with regard to operation 202, the transport device moves a certain distance and causes the object to move the certain distance relative to the camera in a movement direction that is transverse to (e.g., perpendicular to) a row direction of the camera. This distance may be proportional to the quantity X H, where N, M, and H are the quantities described above with regard to FIGS. 3 A and 4A. Thus, in general the distancemay be expressed as where C is an oversampling coefficient that may or may not bean integer. The oversampling coefficient C is a factor for sampling, i.e., oversampling, undersampling, or regular sampling in a direction of movement. The value of C may be selected based on the desired amount of sampling in the movement direction. For example, C may be selected to be larger than 1 (e.g., 2, 3, 4, 1.5, etc.) for oversampling, i.e., to capture additional images and thereby increase the SNR and / or increase the resolution in the movement direction. In other examples, C may be selected to be smaller than 1 (e.g., 2 / 3) to provide an undersampling, which would result in combined images having lower resolution but which may provide the ability to utilize the systems and methods of the present disclosure with cameras having lower imaging framerate capabilities. However, if C is selected to be too small, the resolution may be too degraded to provide suitable image quality. Thus, in some examples, C > 1 / 2 (e.g., C > 2 / 3). In still other examples C may be selected to be equal to 1 for regular sampling.

[0068] In particular examples, this distance may be equal to the quantity X H. As anN alternative, this distance may be an integer or other fraction of the quantity — X H. If a CIS camera is used that has multiple pixel units arranged in a staggered manner (e.g., with an offset AY), the distance may also be equal to the quantity This distance may be determined by anoutput of an encoder (e.g., encoder 140) in either case. A predetermined period of time may elapse between successive activations of the same illumination source to reduce or prevent a variation in brightness of light emitted by the illumination source, which would otherwise result in a reduction in image quality.

[0069] For a camera with the same number of lines of pixels as number of different illuminations (N = AT), the transport device may travel a distance corresponding to one pixel height H from one illumination to the next. If the number of the lines of pixels is less than a number of illuminations (N < M), the transport device may travel a fraction of a pixel height H between illuminations. For example, for a camera sensor with N = 2 lines and M = 4 illuminations, the transport device (and thus the imaged object thereon) travels for a distance corresponding to A pixel height H (or an integer fraction thereof) from one illumination to the next. For a camera sensor with N = 2 lines and M = 5 illuminations (e.g., four visible illuminations and one IR illumination), the transport device (and thus the imaged object thereon) travels for a distance corresponding to 0.4 pixel spacing (or an integer fraction thereof) from one illumination to thenext. Similarly, for a camera sensor with N = 3 lines and M= 4 illuminations, the transport device (and thus the imaged object thereon) travels for a distance corresponding to % pixel height H (or an integer fraction thereof) from one illumination to the next. For a camera sensor with N= 3 lines and M = 5 illuminations (e.g., four visible illuminations and one IR illumination), the transport device (and thus the imaged object thereon) travels for a distance corresponding to 0.6 pixel height H (or an integer fraction thereof) from one illumination to the next.

[0070] If the lines of pixels are separated by an extra space of one pixel height H (i.e., 5 = H and P = 2H), the first and second sub-orders may be the same. This is illustrated in FIG. 6A, in which the first sub-order and the second sub-order are both 1-2-3-4, such that the overall order is 1-1-2-2-3-3-4-4. Alternatively, the second sub-order may be different from the first sub-order. This is illustrated, for one example, in FIG. 6B, in which the first sub-order is 1 -2-3-4 and the second sub-order is 3-4-1-2, such that the overall order is 1-3-2-4-3-1-4-2. Both FIGS. 6A and B show that the first and second sub-orders are interleaved with one another (i.e., illumination conditions of the first sub-order alternate with illumination conditions of the second sub-order).

[0071] In FIGS. 6A and B, it is assumed that the imaging system is implemented as a dome (see FIG. 1) divided into four quadrants, each corresponding to a different illumination device having a different illumination condition. In multiline mode, if there were no pixel spacing, one could operate the quadrants according to the order 1-2-3 -4 to acquire one image of each portion of the object. However, if there is an extra spacing, this order would not be suitable as image information for certain illuminations would be missing for certain lines. In one example, the quadrants could be illuminated according to the order 1-1-2 -2-3-3 -4-4 (see FIG. 6A). However, switching off a lighting quadrant and immediately switching it on again (or even leaving the quadrant on for two cycles) can result in a variation of brightness, which can reduce image quality. FIG. 6B illustrates a lighting sequence that ensures that each quadrant is switched off for a number of periods before being switched on again, such that a predetermined period of time elapses between successive activations of the same illumination source. This period may be sufficiently long to ensure consistent brightness. The lighting sequence may be symmetrical regarding all quadrants. In other words, the second sub-order follows the same sequence as the first sub-order, but may be “out of phase” such that it starts at a different illumination. In the illustrated example, the sequence is 1-3 -2-4-3 -1-4-2, meaning that the second sub-order follows the same 1-2-3-4 sequence but begins with illumination condition 3. Note that this scheme is not limited to quadrantsand may be applied to an illumination unit having any number of sections. There are alternative light sequences with similar properties.

[0072] FIGS. 7-8 illustrates how such a sequence results in imaging the entirety of the transport device, for an example in which there are four different illumination conditions and in which the camera has four lines of pixels separated by a line spacing S (i.e., N = M = 4). In FIGS. 7-8, an overhead view of the transport device is shown at left with a plurality of lettered index blocks A-O, for explanatory purposes. For ease of explanation, a line spacing S = H is assumed. The illumination order of FIGS. 7-8 correspond to the example shown in FIG. 6B, in which the first sub-order is 1-2-3-4 and the second sub-order is 3-4-1-2, such that the overall order is 1-3-2- 4-3-1-4-2. At a first time, the first illumination device (corresponding to the first illumination condition) is activated and an image is captured. Because of the extra spacing, rows A, C, E, and G of the transport device are captured. After the transport device has moved a distance corresponding to the pixel height H (e.g., as determined by a signal from an encoder), a different illumination device (corresponding to the third illumination condition) is activated and another image is captured. This image includes rows C, E, G, and I.

[0073] Note that, if the distance traveled between successive image capture operations is equal to X H, each portion of the object will only be captured once. If the distance traveled . . . . .pbetween successive image capture operations is an integer tractionX H, where k is an integer), each portion of the object will be captured multiple times (and in particular, k times). By capturing each portion of the object multiple times, the SNR can be improved.

[0074] This sequence of operations continues, and FIG. 7 illustrates the end result after each illumination condition has been imaged once. FIG. 8 illustrates the end result after each illumination condition has been imaged twice. For example, after two imaging operations for the first illumination condition, rows A, C, E-H, J, and L are captured, such that the entirety of the object between rows E and H have been properly captured. FIGS. 7 and 8 show the first iteration of the sequence (e.g., the first order followed by the second order), and thus includes some gaps where rows have not yet been imaged. By repeating this sequence, the entirety of the object for all rows will be captured at least once.

[0075] In any event, returning to FIG. 5, once all images of a portion of the transport device have been captured, at operation 510 corresponding portions of the first and second plurality of images (e.g., portions which show the same portion of the transport device) may be combined andoutput. Output may occur after each portion has been imaged a threshold number of times (e.g., one or greater).

[0076] The illumination order may be determined based on an input from a user. For example, the system may be configured to receive a proposed order from the user. In some examples, the system may be configured to determine whether the proposed order will result in successful imaging of the object (e.g., suitably acquiring image information for all illuminations and all lines of pixels), and may be configured to flag the sequence and / or to alert the user if the proposed order will be unsuccessful. In such cases, the system may further be configured to recommend a lighting sequence (e.g., to recommend first and second orders) that would generate deinterlaced images of high quality.

[0077] The systems and methods set forth herein may be implemented with TDI line scan cameras as well, which are cameras and sensors designed to take images with high sensitivity for a single illumination. Therefore, light received by several lines is summed up using charge accumulation. For photometric deflectometry or photometric stereo images, several illuminations (e.g., four) may be used. A summation of light under several illuminations according to comparative examples would destroy the image information needed. Thus, the present disclosure further provides systems and methods for illuminating and reading out TDI line scan sensors in such a way that image information under different illuminations is preserved.

[0078] FIG. 9A illustrates one example of an imaging system 900 including a control unit 902 (e.g., processor 130), an imaging unit 904 (e.g., camera 122, including a TDI line scan sensor), and an illumination unit 906 (e.g., illumination devices 124). The example of FIG. 9A is illustrative and not limiting, and in some implementations the control unit 902, imaging unit 904, and / or illumination unit 906 may have a different structure and / or architecture than shown in FIG. 9A.

[0079] As illustrated, the imaging unit 904 includes a plurality of lines of pixels, of which two are shown. Each pixel in the imaging unit 904 may be configured to accumulate charge in response to detected illumination and may transfer the charge to subsequent lines of pixels as illustrated by the arrows. The imaging unit 904 is configured for readout using a transfer gate, which in the illustrated example is implemented as a transistor connected to the last line that operates in response to a transfer control signal XFER. When the control signal XFER is high (in the illustrated example), a signal from a line of pixels is transferred to the output OUT. In this example, a plurality of lines can be read out one after the other. The illumination unit 906 includesa plurality of illumination devices. Each illumination device comprises a light emitting diode and a transistor arranged between a power supply voltage Vccand a ground voltage. In response to an illumination control signal (IL1 to ILM), and in accordance with a timing control signal TRIG (e.g., from an encoder), the light emitting diode selectively emits light. The light may be in the visible and / or infrared wavelength ranges. The control signals, TRIG, XFER, and IL1-ILM may be provided by the control unit 902 to the imaging unit 904 and / or the illumination unit 906, respectively.

[0080] FIG. 9B illustrates an example timing diagram for the imaging system 900. When an illumination control signal is pulsed high, the corresponding transistor becomes conductive and the corresponding light emitting device emits light. In synchronicity with the illumination, the imaging unit 904 captures an image of the inspection area. Between each individual illumination operation, the transfer control signal TRG is pulsed high, in particular it is pulsed high a number of times, in particular N times, to cause the imaging unit 904 to output imaging data.

[0081] FIG. 10 illustrates an example method 1000 of the imaging operation 204 for the imaging system 900 as shown in FIGS. 9A-B. The method 1000 results in a scheme for illuminating and reading out TDI line scan sensors in such a way that image information under different illuminations is preserved and includes an operation of acquiring an image of the object with the camera that is repeated for a plurality of different illumination conditions.

[0082] At operation 1002, an illumination source i from the plurality of illumination sources AT (e.g., illumination devices 124) is selected and then activated for a predetermined period of time. At operation 1004, a number of lines of the sensor captures light from the active illumination, for example by accumulating a charge in a photosensitive element of the line scan camera. During this time, the charge transfer mechanism of the TDI sensor is not active. Next, at operation 1006, the illumination source z is deactivated. At operation 1008, the charge transfer mechanism of the TDI sensor is activated a plurality of times, corresponding to the number of lines N (or a multiple thereof). The image information is shifted by a distance of N lines (or a multiple thereof) and read out. In one example implementation, N = 4.

[0083] Operations 1002 to 1008 are then repeated for each of the plurality of different illumination conditions (e.g., with a different illumination source selected for each iteration), thus preserving the image information under different illuminations. In the method 1000, this is accomplished by, at operation 1010, determining whether the illumination source (or condition) isthe last illumination source (i.e., if 7 = AY) (or the last illumination condition). If so, the method 1000 terminates or reinitializes (e.g., as shown in FIG. 2 to provide for continuous operation). If not, however, the index variable i is incremented and the method 1000 returns to operation 1002 for the new illumination source (or condition). Operations 1002-1008 may be repeated at a predetermined interval such that, between successive iterations of operation 1002, a time for the transport device to an integer multiple of pixels of the line scan camera elapses. Once all images of a portion of the object have been captured, corresponding portions of each iteration of capturing images (e.g., portions which show the same portion of the transport device) may be combined and output. Output may occur after each portion has been imaged a threshold number of times (e.g., two, four, or greater).

[0084] FIG. 11-13 illustrates how such a sequence results in imaging the entirety of the object. FIG. 11 illustrates an example implementation in which four illumination sources (corresponding to four different illumination conditions) are arranged as quadrants of a dome. In this example, because there is no extra spacing between lines of pixels, the quadrants may be illuminated in any order, such as 1-2-3-4.

[0085] In FIG. 12-13, an overhead view of the transport device is shown at left with a plurality of lettered index blocks A-O, for explanatory purposes. At a first time, the first illumination device (corresponding to the first illumination condition) is activated and an image is captured. Because of the pixel spacing, rows A-D of the transport device are captured. After the transport device has moved a distance corresponding to the pixel height H (e.g., as determined by a signal from an encoder), the second illumination device (corresponding to the second illumination condition) is activated and another image is captured. This image includes rows B-E.

[0086] This sequence of operations continues, and FIG. 12 illustrates the end result after each lighting condition has been imaged once. FIG. 13 illustrates the end result after each lighting condition has been imaged twice. For example, after two imaging operations for the first lighting condition, rows A-H are captured, such that the entirety of the transport device between these rows has been properly captured. By repeating this sequence, the entirety of the transport device for all rows will be captured once or multiple times. By capturing each portion of the object multiple times, the SNR can be improved.

[0087] Accordingly, the systems and methods set forth above provide a scan speed that is multiple times higher than with a single camera (e.g., a single line camera). The above systemsand methods may further be used with a wider variety of camera models and sensors for multiline technology and may result in significantly higher resolution than with comparative examples. Moreover, the above systems and methods enable lower cost solutions for cameras and sensors.

[0088] The particular aspects disclosed above are illustrative only, as the technology may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. Furthermore, no limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular aspects disclosed above may be altered or modified and all such variations are considered within the scope and spirit of the technology. Accordingly, the protection sought herein is as set forth in the claims below.

Claims

CLAIMSWhat is claimed is:

1. A method of acquiring images of an object acted upon by a transport device using a camera that includes a plurality of lines of pixels, the method comprising: acquiring a plurality of images with the camera, including:(a) activating one or more illumination sources to provide a plurality of illumination conditions according to a predetermined order, and(b) under each of the plurality of illumination conditions, acquiring a corresponding one of the plurality of images; wherein, between the acquiring of successive ones of the plurality of images, the transport device is configured to cause the object to move a distance relative to the camera in a movement direction transverse to a row direction of the camera that is equal to Qj- x where N is atotal number of the plurality of lines of pixels, M is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient.

2. The method of claim 1, wherein the predetermined order includes a first sub-order and a second sub-order interleaved with one another.

3. The method of claim 1 or claim 2, wherein a predetermined period of time elapses between successive activations of the one or more illumination sources to provide the same illumination condition.

4. The method of any one of claims 1 to 3, wherein the first sub-order and the second sub-order follow the same sequence of illumination conditions.

5. The method of claim 4, wherein the second sub-order is different than the first suborder.

6. The method of claim 4, wherein the second sub-order is the same as the first suborder.

7. The method of any one of claims 1 to 6, wherein the camera is a line scan camera including the plurality of lines of pixels, or an area scan camera windowed to the plurality of lines of pixels.

8. The method of any of one claims 1 to 7, wherein N9. The method of any one of claims 1 to 8, wherein N is less than M.

10. The method of any one of claims 1 to 9, wherein adjacent ones of the plurality of lines of pixels are separated by a line spacing S.

11. The method of claim 10, wherein S = H.

12. The method of any one of claims 1 to 11, further comprising combining portions of one of the plurality of images with portions of another of the plurality of images, corresponding to the same illumination condition.

13. The method of any one of claims 1 to 12, wherein a timing of acquiring the plurality of images is based on an input from an encoder associated with the transport device.

14. The method of any one of claims 1 to 13, further comprising receiving a proposed order from a user interface.

15. The method of claim 14, further comprising providing, to the user interface, a recommendation for the first predetermined order and a recommendation for the second predetermined order, based on the proposed order.

16. The method of claim 14 or claim 15, further comprising determining whether the proposed order results in successful imaging of the object based on a parameter of the camera and / or a parameter of the plurality of illumination sources.

17. The method of any one of claims 1 to 16, comprising: repeating the operation of acquiring the plurality of images a plurality of times during a continuous operation of the transport device.

18. The method of any one of claims 1 to 17, wherein the plurality of illumination sources are arranged in at least one of a dome arrangement, a bar arrangement, a sawtooth arrangement, or a grid arrangement.

19. The method of any one of claims 1 to 18, wherein the plurality of illumination sources are arranged to indirectly illuminate the object.

20. The method of any one of claims 1 to 19, wherein the camera is a contact image sensor (CIS) camera.

21. The method of claim 20, wherein the camera includes a first pixel unit and a second pixel unit, wherein the second pixel unit is offset from the first pixel unit by an offset distance AX, wherein AX = i 'zH in a direction transverse to a direction of motion of the transport device.

22. The method of claim 20 or claim 21, wherein the camera includes a first pixel unit and a second pixel unit, wherein the second pixel unit is offset from the first pixel unit by an offset distance AY, wherein AY =j Hin a direction along a direction of motion of the transport device.

23. The method of any one of claims 1 to 22, wherein C > 1 / 2.

24. An imaging system, comprising: one or more illumination sources configured to illuminate an object acted upon by a transport device with a plurality of illumination conditions;a camera that includes a plurality of lines of pixels; and at least one control unit in communication with the one or more illumination sources and the camera, the control unit configured to cause the imaging system to perform operations that include: acquiring a plurality of images with the camera, including:(a) activating the one or more illumination sources to provide a predetermined order of different illumination conditions of the plurality of illumination conditions, and(b) under each of the illumination conditions of the predetermined order of illumination conditions, capturing a respective one of the plurality of images; wherein, between the acquiring of successive images of the plurality of images, the transport device is configured to cause the object to move a distance relative to the camera in a.. . .. . I 1 • , 1 (N . -\ , movement direction transverse to a row direction of the camera that is equal to - 1 — x H i, whereJVis a total number of the plurality of lines of pixels, is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient.

25. The system of claim 24, wherein the predetermined order includes a first sub-order and a second sub-order interleaved with one another.

26. The system of claim 25, wherein the first sub-order and the second sub-order follow the same sequence of illumination conditions.

27. The system of claim 26, wherein the second sub-order is different than the first suborder.

28. The system of claim 26, wherein the second sub-order is the same as the first suborder.

29. The system of any of claims 24 to claim 28, wherein a predetermined period of time elapses between successive activations of the one or more illumination sources to provide the same illumination condition.

30. The system of any one of claims 24 to 29, wherein the camera is a line scan camera including the plurality of lines of pixels, or an area scan camera windowed to the plurality of lines of pixels.

31. The system of any one of claims 24 to 30, wherein N M.

32. The system of any one of claims 24 to 31, wherein N is less than M.

33. The system of any one of claims 24 to 32, wherein adjacent ones of the plurality of lines of pixels are separated by a line spacing S.

34. The system of claim 33, wherein S = H.

35. The system of any one of claims 24 to 34, the operations further including combining portions of one of the plurality of images with portions of another of the plurality of images, corresponding to the same illumination condition.

36. The system of any one of claims 24 to 35, wherein a timing of acquiring the plurality of images is based on an input from an encoder associated with the transport device.

37. The system of any one of claims 24 to 36, the operations further including receiving a proposed order for the predetermined order from a user interface.

38. The system of claim 37, the operations further including providing, to the user interface, a recommendation for a first predetermined order and a recommendation for a second predetermined order, based on the proposed order.

39. The system of claim 37 or claim 38, the operations further including determining whether the proposed order results in successful imaging of the object based on a parameter of the camera and / or a parameter of the one or more illumination sources.

40. The system of any one of claims 24 to 39, the operations including: repeating the operation of acquiring the plurality of images a plurality of times during a continuous operation of the transport device.

41. The system of any one of claims 24 to 40, wherein the one or more illumination sources include a plurality of illumination sources that are arranged in at least one of a dome arrangement, a bar arrangement, a sawtooth arrangement, or a grid arrangement.

42. The system of any one of claims 24 to 41, wherein the one or more illumination sources are arranged to indirectly illuminate the object.

43. The system of any one of claims 24 to 42, wherein the camera is a contact image sensor (CIS) camera.

44. The system of claim 43, wherein the camera includes a first pixel unit and a second pixel unit, wherein the second pixel unit is offset from the first pixel unit by an offset distance AX, wherein AX = i 'zH in a direction transverse to a direction of motion of the transport device.

45. The system of claim 43 or claim 44, wherein the camera includes a first pixel unit and a second pixel unit, wherein the second pixel unit is offset from the first pixel unit by an offset distance AY, wherein AY = J x H in a direction along the direction of motion of the transport device.

46. The system of any one of claims 24 to 45, wherein C > 1 / 2.

47. A method of acquiring images of an object acted upon by a transport device, comprising:(i) acquiring an image of the object with a line scan camera, including:(a) selecting an illumination condition from a plurality of illumination conditions,(b) activating one or more illumination sources for a predetermined period of time to provide the selected illumination condition,(c) accumulating a charge in a photosensitive element of the line scan camera,(d) deactivating the one or more illumination sources to stop providing the selected illumination condition, and(e) activating a charge transfer mechanism of the line scan camera to read out image data corresponding to the charge; and(ii) repeating (i) for a plurality of different illumination conditions, including selecting a respective different illumination condition from the plurality of illumination conditions, wherein, between successive iterations of (i), the transport device is configured to move a predetermined distance.

48. The method of claim 47, wherein, during (b)-(d), the charge transfer mechanism is deactivated.

49. The method of claim 47 or claim 48, wherein, the predetermined distance is equalatotal number of the plurality of lines of pixels, AT is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient.

50. The method of any one of claims 47 to 49, wherein the plurality of different illumination conditions includes four illumination conditions.

51. The method of any one of claims 47 to 50, comprising: repeating (i) and (ii) a plurality of times during a continuous operation of the transport device.

52. The method of claim 51, comprising: combining portions of the image data corresponding to iterations of (ii) having the same illumination conditions.

53. The method of any one of claims 47 to 52, wherein the line scan camera is a time delay integration line scan camera.

54. The method of any one of claims 47 to 53, wherein selecting the illumination condition from the plurality of illumination conditions includes selecting the corresponding one or more illumination sources from a plurality of illumination sources; and wherein repeating (i) for the plurality of different illumination conditions includes selecting respective different one or more illumination sources from the plurality of illumination sources.

55. An imaging system, comprising: one or more illumination sources configured to selectively illuminate an object acted upon by a transport device with any of a plurality of illumination conditions; a line scan camera; and at least one control unit in communication with the one or more illumination sources and the line scan camera, the control unit configured to cause the imaging system to perform operations that include:(i) acquiring an image of the object with the line scan camera, including:(a) activating the one or more illumination sources to provide an illumination condition of the plurality of illumination conditions for a predetermined period of time,(b) accumulating a charge in a photosensitive element of the line scan camera corresponding to the activation of the illumination source,(c) deactivating the illumination source to stop providing the illumination condition, and(d) activating a charge transfer mechanism of the line scan camera to read out image data corresponding to the charge; and(ii) repeating (i) for multiple different illumination conditions of the plurality of illumination conditions, wherein, between successive iterations of (i), the transport device is configured to move a predetermined distance.

56. The system of claim 55, wherein, during (a)-(c), the charge transfer mechanism is deactivated.

57. The system of claim 55 or claim 56, wherein the predetermined distance is a equal's atotal number of the plurality of lines of pixels, AT is a total number of the plurality of different illumination conditions, H is a height of a pixel in object space, and C is an oversampling coefficient.

58. The system of any one of claims 55 to 57, wherein the plurality of different illumination conditions includes four illumination conditions.

59. The system of any one of claims 55 to 58, the operations further including: repeating (i) and (ii) a plurality of times during a continuous operation of the transport device.

60. The system of any one of claims 55 to 59, the operations further including: combining portions of the image data corresponding to iterations of (ii) having the same illumination conditions.

61. The system of any one of claims 55 to 60, wherein the line scan camera is a time delay integration line scan camera.

62. The system of any one of claims 55 to 61, wherein the one or more illumination sources include a plurality of illumination sources are arranged in at least one of a dome arrangement, a bar arrangement, a sawtooth arrangement, or a grid arrangement.

Images