System and method for auto-focusing a vision system camera on barcodes

Abstract

To provide a system and method for detecting and acquiring one or more in-focus images of one or more barcodes within the field of view of an imaging device.SOLUTION: The present invention relates to a method in a vision system. A measurement process measures depth-of-field of barcode detection. A plurality of nominal coarse focus settings of a variable lens allow sampling, in steps, of a lens adjustment range corresponding to allowable distances between the one or more barcodes and the image sensor, so that a step size of the sampling is less than a fraction of the depth-of-field of barcode detection. An acquisition process acquires a nominal coarse focus image for each nominal coarse focus setting. A barcode detection process detects one or more barcode-like regions and respective likelihoods. A fine focus process fine-adjusts, for each high-likelihood barcode, the variable lens near a location of the barcode-like regions. The process acquires an image for decoding using the fine adjusted setting.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a machine vision system for detecting and decoding 1D and 2D barcodes and other forms of symbols or ID codes, and more particularly to a camera focusing system for use in reading barcodes and / or other ID codes.

Background Art

[0002] A vision system that performs measurement, inspection, and / or alignment of objects and / or decoding of symbols in the form of machine-readable symbols (also referred to as "barcodes", "ID codes", "codes", and / or simply "ID"). Often, these machine-readable symbols take the form of well-known 1D and 2D barcodes such as 2D data matrix codes, QR codes (registered trademark), DPM codes, etc. More generally, barcodes can be defined as optical machine-readable data representations, and the data often describes something about the object with the barcode. Conventional barcodes systematically represent data by varying the width and spacing of parallel lines and are sometimes called linear or one-dimensional (1D). Later, 2D variants were developed using rectangles, dots, hexagons, or other geometric patterns and were called matrix codes or 2D barcodes, but they do not use bars themselves. This includes, but is not limited to, data matrix codes, QR codes (registered trademark), and DPM codes. These are all used in a wide range of applications and industries. Barcode reading systems generally rely on the use of image sensors, which acquire images of the target or object (usually grayscale or color and 1D, 2D, or 3D), and process these acquired images using an in-vehicle or interconnected vision system processor. The processor generally includes both processing hardware and non-transitory computer-readable program instructions that perform one or more vision system processes to generate a desired output based on the processed information of the image. This image information is usually provided within an array of image pixels, each having various colors and / or intensities. In the example of a barcode reader (also referred to as a "camera" in this specification), a user or an automated process acquires an image of an object that is considered to contain one or more barcodes. The image is processed to identify barcode features, and then the barcode features are decoded by a decoding process and / or a processor to obtain the unique alphanumeric data represented by the code.

[0003] When in operation, a barcode reader typically functions to illuminate a scene that includes one or more barcodes. This illumination can include an aimer that projects colored dots onto the region of interest within the imaged scene, thereby allowing the user to center the image axis of the reader on the barcode within the captured scene. This illuminated scene is then acquired by an image sensor within a camera assembly through an optical system. The array sensor pixels are exposed, and the electronic values generated for each pixel by the exposure are stored in an array of memory cells that can be referred to as an “image” of the scene. In the context of barcode reading applications, the scene includes an object of interest having one or more barcodes of appropriate dimensions and types. The barcode(s) is / are part of the stored image.

[0004] A common use of barcode readers is to track and classify objects moving along a line (such as a conveyor) in manufacturing and logistics operations. A barcode reader, or more typically a plurality of readers (a constellation), can be placed at appropriate viewing angles along the line to acquire the expected barcode on the surface of each object as it passes through the respective field of view. The focal distance of the reader with respect to the object can vary depending on the placement of the reader with respect to the line and the size of the object. Therefore, it is desirable to use an autofocus mechanism in combination with the barcode reader to achieve an appropriate focal distance.

[0005] Existing techniques for providing an autofocus function to a barcode reader involve sweeping through a potentially large number of focus settings and selecting the setting that maximizes the image sharpness for the selected region of interest. The procedure for autofocus is described by useful background information in “Generalized Autofocus” by Daniel Vaquero, Natasha Gelfand, Marius Ticol, Kari Pullil, and Matthew Turk, Applications of Computer Vision (WACV), 2011 IEEE.

[0006] Regions of interest are often selected using discovery contrast measurements, which often misidentify other high-contrast structures (such as text and other graphics) as barcodes. Both contrast measurements and sharpness metrics are only meaningful when the focus is approximately correct, which is why it is necessary to consider a number of focus settings during a sweep. Additionally, sharpness metrics are sensitive to lighting, which can also change (for example, during the process of an auto-adjustment operation even when the lighting is optimized). Therefore, such methods are time-consuming and may result in focusing on the wrong area of the scene. Additionally, commercially available techniques for identifying barcodes were too slow to be used directly during an autofocus process loop. Generally, first the focus on a certain area is determined, and then barcode detection and decoding procedures operate on the focused image of the region of interest. Summary of the Invention

[0007] The present invention overcomes the drawbacks of the prior art by providing a system and method for detecting and acquiring one or more focused images of one or more barcodes within the field of view of an imaging device that acquires one or more images of a scene containing barcodes. The variable lens receives focus adjustment information from the focusing process and directs the received light towards the image sensor. The measurement process measures the effective depth of field for barcode detection as the maximum focus error that can reliably detect one or more barcodes by the focusing process. This depth of field measurement process can be predicted, in whole or in part, by an automated system or manual input from a system user, which can be an evaluation based on heuristics, observations, or other calculations (for example). A plurality of nominal coarse focus settings of the variable lens enable sampling at sampling steps of a lens adjustment range corresponding to an allowable distance range between one or more barcodes and the image sensor, and as a result, the step size of the sampling steps is smaller than a part of the depth of field for barcode detection. The acquisition process uses the image sensor to acquire a nominal coarse focus image for at least one coarse focus setting of the variable lens. The barcode detection process detects one or more barcode-like regions (which may be part of the entire barcode pattern) within each nominal coarse focus image. The fine focus process, for each barcode-like region having a sufficiently high likelihood in the coarse focus image, starts from the coarse focus setting of the variable lens and finely adjusts in order from the highest likelihood to the lowest likelihood to optimize the focus of the image close to the position of the barcode-like region. Next, the fine focus process acquires an image for decoding using the optimized fine adjustment focus setting of the variable lens.

[0008] In an exemplary embodiment, a system and method are provided for detecting and acquiring one or more focused images of one or more barcodes within the field of view of an imaging device having an image sensor that acquires one or more images of a scene including one or more barcodes. The system and method include a variable lens that receives focus adjustment information from a focusing process and directs the received light toward the image sensor. The measurement process measures the effective depth of field of barcode detection as the maximum focus error by which one or more barcodes can be reliably detected by the focusing process. A plurality of nominal coarse focus settings of the variable lens enable sampling in sampling steps of a lens adjustment range corresponding to an allowable distance range between one or more barcodes and the image sensor. The step size of the sampling steps is smaller than a part of the depth of field of normal barcode detection. The acquisition process acquires a nominal coarse focus image for at least one coarse focus setting of the variable lens using the image sensor. The fine focus process fine-tunes the variable lens in order from the highest likelihood to the lowest likelihood starting from the coarse focus setting for each barcode-like region having a sufficiently high likelihood in the coarse focus image to optimize the focus of an image close to the position of the barcode-like region. The fine focus process acquires an image for decoding using the optimized fine-tuned focus setting of the variable lens. Exemplarily, the barcode detection process is configured to measure the likelihood of each barcode-like region, and at least one of the barcode-like regions defines a likelihood exceeding a predetermined level. The barcode detection process can operate at a substantially maximum frame rate that reliably identifies the position of the barcode-like region in the image reliably acquired by the image sensor. The depth of field can be measured in visibility, and the adjustment information defines the visibility. In an embodiment, the part can include a value that is about half or less of the depth of field of barcode detection. The variable lens can include a high-speed liquid lens. Exemplarily, the fine focus process can be configured to optimize the focus of an image by acquiring an image at one or more adjusted focus settings, and the one or more adjusted focus settings can be configured to maximize the image sharpness score in a local region around each barcode-like region.A variable lens and a processing unit operatively connected to an image sensor can be provided, and this processing unit can operate in at least one of a focusing process, a barcode detection process, and an acquisition process. Exemplarily, the processing unit can include at least one of a GPU and an FPGA. The variable lens can be adjusted based on an iterative and / or numerical step-based search technique for a coarse focus setting. The fine focus process fine-tunes the variable lens starting from a coarse setting in order from the highest likelihood to the lowest likelihood. The barcode can include at least one of a 1D barcode and a 2D barcode. The coarse focus setting can be based on a distance metric generated by a distance sensor for the imaging device. The distance sensor can include at least one of a time-of-flight sensor, LIDAR, radar, ultrasonic sensor, stereoscopic sensor, and sonar sensor.

[0009] In an exemplary embodiment, a system and method are provided for aligning the focus of an imaging system to one or more barcodes. For each of at least two nominal coarse focus settings, a nominal coarse focus image is acquired. The nominal coarse focus settings are selected to sample an allowable distance range between each barcode and the imaging system. A barcode detector determines barcode-shaped regions within each nominal coarse focus image and evaluates the likelihood that each of the barcode-shaped regions is an actual barcode. The barcode detector operates within an effective depth of field that enables the barcode to be reliably detected in the presence of defocus over a distance range between the sampled distances corresponding to the nominal coarse focus settings. For each barcode-shaped region having a sufficiently high likelihood, a fine focus procedure optimizes the focus in order from the highest likelihood to the lowest likelihood to obtain an image sharp enough to successfully decode one or more barcodes in each barcode-shaped region. The fine focus procedure maximizes sharpness measurements in a local region surrounding each barcode-shaped region by acquiring an image at one or more adjusted focus settings that are substantially close to the nominal coarse focus settings used to acquire the nominal coarse focus image in which each barcode-shaped region was detected by the barcode detector, thereby optimizing the focus. Exemplarily, a vision system processor receives the acquired nominal coarse focus image and operates the barcode detector. The vision system processor can include at least one of a GPU and an FPGA. An electronically controlled variable focus lens assembly can be used to image the nominal coarse focus image at a selected focal distance. The lens can further include a high-speed liquid lens whose focal distance can be set based on a diopter value. Exemplarily, the operation of the system and method can be performed according to a plurality of priorities.

[0010] The following description of the present invention refers to the accompanying drawings.

Brief Description of the Drawings

[0011]

Figure 1

[0012]

Figure 2

[0013]

Figure 2A

[0014]

Figure 3

[0015]

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0016] I. System Overview

[0017] FIG. 1 illustrates a configuration 100 for reading and decoding one or more barcodes (1D and 2D barcodes, such as DPM and QR codes (registered trademarks), and other related symbol codes) 110 and 114 disposed on an exemplary object and / or other imaging scene 112. The exemplary object 112 is here shown by a plurality of surfaces 115, 117, and 119 that define a three-dimensional structure. The configuration 100 includes a vision system camera assembly 120, which may be a handheld unit or a fixed unit. In alternative embodiments, multiple cameras may be employed to image the scene, each operating the systems and methods of this specification. As will be further explained below, the systems and methods of this specification can effectively distinguish barcode-like features 110 and 114 from other geometric and text information, such as text 116, adjacent to them. Thus, the systems and methods of this specification enable one or more barcodes within the field of view (FOV) to be rapidly detected and decoded, despite the presence of other potential interfering features that are similarly quickly discarded as barcode-like regions (and / or barcode candidates) by the process. As used herein, the term “barcode-like region” refers to all or a portion of an image that includes features representing all or a portion of any suitable type / format of barcode. Thus, a barcode-like region can be identified based on distinguishable portions and / or fragments of the entire barcode occurring therein. Note that barcode 114 is disposed on upright surface 117 so as to be at a different distance from barcode 110 on object surface 115 with respect to the image sensor. The systems and methods of this specification can effectively find multiple barcodes with different effective distances within the same overall field of view (FOV) using the autofocus techniques described herein.

[0018] The camera assembly can include various functional elements including a built-in and / or separate illumination / aimer 122, an image sensor S (implementable using CMOS or other suitable technology), and an optical module O. The sensor S can define an acceptable format and pixel array size and can be implemented to supply image data 124 in either color or grayscale format.

[0019] The optical system O can be in any acceptable package that is either integrated or removable (e.g., a screw-type lens base assembly). The optical system O includes a variable focus lens unit 130 that physically varies the focal length of the optical system to receive an operating distance along the optical axis OA between the image plane of the sensor S and the plane including the barcode 110. The variable focus lens 130 can be based on various operating principles such as a mechanical electric lens that moves along the optical axis to change the focus, or a so-called liquid lens that changes the contour shape of the lens based on the movement of a magnetic fluid in the presence of an electromagnetic field. Liquid lenses are commercially available from various vendors such as Varioptic in France or Optotune in Switzerland. The variable focus lens 130 is controlled via a signal 132 generated according to the specifications of the lens manufacturer. Typically, such a signal 132 is supplied as a variable current and / or voltage level, typically based on digital values generated by a vision system process (processor) 140 (or other process (processor)) further described below. The variable focus function can operate such that the field of view (FOV) is sized to completely image the barcode while the process (processor) 140 maintains sufficient resolution to decode the barcode features.

[0020] The vision system process (processor) 140 can be partially or fully housed within the housing of a camera assembly or any attached device (e.g., an attached smartphone). Alternatively or additionally, the processor can be partially or fully instantiated within a separate stand-alone computing device 150 such as a server, PC, laptop, tablet, or smartphone that is interconnected with the camera assembly via suitable wired and / or wireless data links 152. Such a computing device 150 can also be removably attached to the on-board processor within the camera assembly 120 when needed to monitor, adjust, or set up the vision system. The computing device 150 can include suitable user interface components such as a display / touch screen 154, keyboard 156, and / or mouse 158. The process (processor) 140, computing device 150, and / or other devices within the configuration 100 can be linked with one or more downstream utilization devices and / or processes that employ decoded data derived from barcodes to perform tasks such as part recognition, logistics, object classification, etc.

[0021] In an embodiment, the processor 140 can comprise or include a graphics processing unit (GPU) of a field programmable gate array (FPGA) that executes the desired program instructions used herein.

[0022] The vision system process (processor) 140 includes a plurality of functional processes (processors) for performing the operation of the vision system arrangement 100. The vision system process (processor) 140 can include a vision system tool module 142, and this vision system tool module 142 can have one or more tools used for the detection and analysis of image features, such as an edge detector, a blob analyzer, calipers, alignment, and pattern recognition processes (processors). The vision system process (processor) 140 also includes a barcode detection and decoding process (processor) 144, which can be based on a commercially available or customized (e.g., software) application. The barcode detector and decoder locate barcode candidate features within the acquired image of the scene 112. Note that the vision tool 142 can be used to refine and analyze the features to better locate the barcode candidates. When the barcode candidate features are resolved, the decoding process (processor) within the module 144 can be used to generate appropriate information (e.g., an alphanumeric data string) from the read barcode 110.

[0023] As part of the image acquisition and barcode detection process, the camera optics O can be adjusted to account for the distance between the imaging plane and the sensor image plane. In order to avoid missing barcodes that may appear within the field of view only for a short time and / or in relative motion, it is desirable to detect and decode the barcode as quickly as possible. Thus, in this specification, the focus process (processor) 146 attempts to adjust the variable lens focal length 130 according to a predetermined procedure described below.

[0024] Optionally, one or more additional distance measurement devices (e.g., LIDAR, time-of-flight sensors, stereoscopic devices, ultrasonic and / or radar) 170 can be used to supply distance information 172 to an appropriate functional module of the vision system process (processor) 140. The distance information 172 can define the physical distance (double-dashed arrow 174) between the camera assembly 120 and the surface of the imaging object 112 (and with respect to the sensor image plane). This data 172 can be used to supplement the pixel-based focus information used herein.

[0025] In particular, the barcode detection and decoding process (processor) 144 can be based on a commercially available or customized high-speed bar-code / symbol detector, such as those based on the Hotbars™ available from Cognex Corporation (Natick, Massachusetts). Such detectors can rapidly identify the location / region of barcode features within an image despite significant blur and can order them according to the likelihood that they are actual barcodes.

[0026] In addition to the rapid line detection capabilities of the above-described Hotbars™ barcode detector, several other techniques that can be used in combination with the appropriate processor configurations described herein to more rapidly identify and decode barcode (e.g., 2D barcode) features include the following. 1. Use edge detection and Hough transform-based line detection techniques in combination with clustering to find parallel lines within code candidates. See the following as background information: "Improvement of barcode detection combined with a simple detector" by Peter Bodnar and Laszlo G. Nyul, 8th International Conference on Signal Image Technology and Internet-Based Systems (2012); "Image analysis methods for visual code localization" by Peter Bodnar, Doctoral Thesis, Department of Image Processing and Computer Graphics, Faculty of Informatics, University of Szeged, Hungary (2015), available at URL address http: / / doktori.bibl.u-szeged.hu / 2825 / 8 / tezisfuzet.pdf on the World Wide Web. 2. Use the Histogram of Oriented Gradients (HOG) function on tiles of the acquired image containing candidate barcodes to detect adjacent tiles with similar dominant orientations. See the following as background information: "Image analysis methods for visual code localization" by Peter Bodnar (supra). 3. Use an adaptive image threshold to binarize bars and spaces within the captured barcode and use an image morphology that merges into the barcode area. See the following as background information: "Automatic localization for multi-symbol and multiple 1D and 2D barcodes" by Daw-Tung Lin, Chin-Lin Lin, Journal of Ocean Science and Technology, Vol. 21, No. 6, pp. 663 - 668 (2013); "Implementation of barcode localization technique using morphological operations" by Savreet Kaur and Raman Maini, International Journal of Computer Applications (0975 - 8887), Vol. 97, No. 13, July 2014; "Image analysis methods for visual code localization" by Peter Bodnar (supra). 4. Using deep learning to perform texture classification. See the following for background information: Hansen, Daniel Kold, Nasrollahi, Kamal, Rasmussen, Christoffer Bogelund, Moeslund, Thomas B., "Real-time barcode detection and classification using deep learning", published in the proceedings of the 9th International Conference on Computational Intelligence IJCCI (2017), Volume 1: Peter Bodnar, "Image analysis method for visual code localization" (supra).

[0027] Another technique for more quickly detecting and decoding a selected type of barcode under specific conditions is described, for example, in Yunhua Gu, Weixiang Zhang, "QR code recognition based on image processing", published on May 10, 2011, in the International Conference on Information Science and Technology IEEE.

[0028] II. Generalized autofocus operation procedure

[0029] FIG. 2 shows a generalized procedure for performing an autofocus procedure 200 according to one embodiment. In step 210, a user or other process triggers image acquisition in the presence of a barcode (or more generally, in the presence of a "barcode-like region", a term that should be used interchangeably with "barcode" as needed). Next, procedure 200 applies a coarse focus adjustment in step 220. This adjustment involves sending a signal to the lens to achieve a predetermined focal distance (expressed as a normal focus / visibility value). Next, in step 222, at least one image is acquired by the sensor at this selected focal distance. Next, the system uses appropriate vision system tools to search for barcode candidate features, and if no barcode feature is detected and / or the predetermined confidence score / likelihood is not met (decision step 224), the next coarse focus value (described below in the focus plane scan) is selected (step 226), and the adjustment is applied in step 220. Another image is acquired until a barcode is detected (or until the coarse focus values are exhausted), and the barcode position / determination process (decision step 224) is repeated. If there is a high confidence and / or likelihood of the presence of a barcode in the region of interest (a confidence value / score exceeding a predetermined threshold) (via decision step 224), procedure 200 applies a fine focus adjustment procedure to the lens in step 230. In the embodiments described below, an initial set of nominal focus planes can be determined, and the coarse focusing process scans such planes until a barcode in a region is identified with sufficient confidence / likelihood. Further, the procedure can prioritize the regions to be searched based on the likelihood of the barcode or other metrics, such as a user-defined metric regarding regions with a high likelihood of containing the barcode in the acquired image, and based on such prioritization, the regions are scanned by moving between focus planes until all values are exhausted or a barcode with sufficient confidence is found.

[0030] In step 232, an image is acquired with this fine-tuning value. Next, the barcode detection and decoding procedure determines whether the barcode located from the coarse focus step is decoded to provide appropriate information (decision step 234). If not, a new fine focus value is selected in step 236 and applied to the lens (step 230), and then the acquisition step 232 and the decision step 234 are repeated until the barcode is decoded or all fine-tuning values are exhausted. The decoded barcode result / information (if any) is then sent to an appropriate downstream process in step 240 (via decision step 234). Referring to the exemplary field of view 250 of FIG. 2A, if no area encompassing a decodable barcode / barcode-like area (fragment) is detected during the coarse focus process, the central quarter of the field of view 250 (substantially enclosed box 260) or the entire region of interest selected by the user (exemplary dashed box 270) can be used as the target area for subsequent barcode searches. The process then proceeds to a conventional contrast / sharpness-based search algorithm / procedure to locate barcodes within the new area 260 or 270.

[0031] As outlined above and described in further detail below, it should be noted that in any acquired image of a scene, there may be several regions of interest that may contain features that may require decoding. As an example, FIG. 4 shows an acquired imaging scene 400 of an underlying object in which at least three regions of interest 410, 420, and 430 are depicted, each containing a 2D barcode, a 1D barcode, and text, respectively. If it is confirmed that no barcode features are present, the text can be excluded from subsequent processing. The boundaries of each region may vary and can be set by default or scaled to include barcode-like areas (barcodes and / or fragments thereof) located within them.

[0032] Advantageously, a small number of images (referred to as nominal focal planes) acquired at various roughness settings over the entire possible focus range of the imaging system during operation define a relatively large depth of field for the reading optical system, and are used for barcode detection to detect all barcode-like regions within the scene regardless of distance. Such an optical system with a high depth of field is described in U.S. Patent Application No. 15 / 844448, entitled "Dual Imaging Vision System Camera and Method of Using the Same," filed on December 15, 2017. The process of acquiring images at these nominal focal planes is particularly efficient when using a high-speed variable lens configuration, such as a liquid lens, that can quickly respond to changes in visibility values.

[0033] As described below, step 200 can also use optional distance measurements (e.g., range data 172) from available sensors (such as time-of-flight and / or other range detection devices 170) to limit the coarse focus range so that initial values and incremental changes in values occur within and around the detection range.

[0034] In an embodiment, the use of fine focus steps is employed to refine the coarse focus visibility value settings for each barcode-like region. Images are acquired at this setting, and then the barcode is attempted to be decoded using a decoding process. The barcode-like regions are processed in order of the highest likelihood of the presence of barcode features. This configuration is essentially an intelligent mechanism for visual attention (for both the position within the field of view of the imaging system / camera assembly and the distance within the focus range).

[0035] III. AUTO-FOCUS PROCESS

[0036] A more detailed flowchart of the autofocus procedure 300 is shown in FIG. 3. Procedure 300 receives an input including the full distance range 310 in the case of a camera / imaging system, an optional input 312 of a distance measurement value from a range detection sensor, an input 314 of a technique used in a coarse detection mode, i.e., the search for barcode-like regions, and an input 316 of whether the coarse search looks beyond (closer and / or farther) a given distance defined by an optional distance measurement value and (optionally) examines distances other than the available distance measurement values (closer and / or farther from the image sensor). Note that the input elements 312, 314, and 316 may be optional in the overall procedure 300 and are shown as such (i.e., “optional”) in FIG. 3. As will be explained below, other optional procedure steps / inputs are similarly marked. Such optional inputs, steps, and processes can be omitted in the overall procedure and / or selected by the user at setup or runtime. More specifically, the coarse search mode input 314 determines the priority of different coarse plane distances, as will be further described below. Generally, the use of priorities when processing the coarse plane is optional, as will be further explained below. The “look beyond the distance measurement value” input 316 can decide to treat the distance measurement value (if any) as accurate and not look for barcodes elsewhere. Alternatively, the input 316 can decide to treat the distance measurement value (if any) as a hint and first look for barcodes at these positions, but if the process fails to successfully decode the required barcode, look for barcodes at other distances within the range.

[0037] The information elements 310 - 316 are used / usable to calculate a list of coarse focus planes used in the coarse focus procedure (step 320). After calculating the coarse focus planes, step 324 extracts a distance from the coarse plane list. This can start from an initial plane value that can be default (e.g., the plane measured by a rangefinder and / or the central distance within the range of the measured plane and / or distance) (step 324). Alternatively, the distance measurement can be based on (or include) a predetermined distance distribution according to a suitable discovery method operating within the specified operating distance of the sensor. This part of procedure 300 can be ranked, and it should be noted that step 324 extracts the distance with the current priority. If no priority is provided, only a single set of focus planes is used. Next, the camera optics is focused on the relevant visibility value of the current (priority) focus plane, and at least one image is acquired in step 326. The procedure uses the acquired image to determine whether a barcode exists in the image and where it exists (decision step 330), and / or to determine the likelihood that such a barcode exists (step 332). If it exists, these barcodes / likelihoods are saved (added to the list of regions) in addition to the corresponding focal distance values for each region. It should be noted that the determination of the barcode can be based on an automatic detection function or can be performed outside the region of interest of the entire image (input 336). Next, in step 334, the focus region is calculated and added with the overall likelihood for the region of interest (input 338). It should be noted that the input and specification of these regions of interest 338 and 334 are optional in the overall procedure 300. Such optional regions of interest can be used if there is existing knowledge about the possible or approximate location of barcodes in the image, for example, based on the location in a previously acquired image. Further, the automatic detection input 336 and process steps 330, 332 are optional. The optional automatic search processes 330, 332, and 338 can be used when the procedure can handle the region of interest (if any) as accurate, and thus does not search for barcodes at another location outside these regions.If automatic detection is not used (decision step 330), the regions of interest (if any) can only be treated as hints, and procedure 300 can first look for barcodes in those regions, but if the required barcodes are not decoded, procedure 300 searches for barcode-like regions within the regions of interest across the entire acquired image (for example).

[0038] Procedure 300 queries whether all planar distances have been manipulated (at decision step 340). Optionally, if multiple priorities are provided in procedure 300, this query can be limited to the current priority. If more planar distances are manipulated in the coarse focus process (optionally within the current priority if any), procedure 300 returns to step 324 to extract the next incremental distance. Next, the coarse detection process (steps 324, 326, 330, 332, and 334) repeats via decision step 340. Conversely, if the distances are exhausted (optionally at the current priority), decision step 340 branches to step 350 where the most likely region of the overall coarse image is extracted (and removed) from the previously saved list of image regions. Next, the camera lens is focused to the best coarse focus for that region and one or more images are acquired at step 354. At this stage, the fine focus process (dashed box 358) is operated and a micro-sharpness score is calculated at step 356. In this calculation, the sharpness measurement values can include (a) gradient measure, (b) frequency domain measure, (c) autocorrelation measure, (d) autocorrelation measure, (e) statistical measure, and / or (d) edge-based measure. As background information, see "Evaluation of Sharpness Measures and Search Algorithms for Auto-Focusing of High-Magnification Images" by Yi Yao, Besma Abidi, Narjes Doggaz, and Mongi Abidi, Proceedings of SPIE - The International Society for Optical Engineering 6246, June 2006. If the focus generates a sharpness score (a value below a predetermined threshold) that may not be sufficient to result in a readable barcode, the next fine focus setting is incremented (step 362), the lens is refocused with the one or more new images acquired (step 354), and then a new sharpness score is calculated (step 356).

[0039] Once a sufficient sharpness score has been generated (decision step 360), an attempt is made to decode the barcode features within the region by a related decoding process (step 370). In addition to using the region of interest, note that the coarse focus process and / or the fine focus process (barcode search) can be performed on the full-size acquired image or the subsampled image. When using a subsampled image for the focus process, the full-size image should be captured at the desired focal plane for decoding. If the required (requested) predetermined number of barcodes are properly detected / decoded and the result is provided to the system (no other barcodes are present in the image to be decoded), the focus and decoding processes are complete (step 380 via decision step 372). If not, the procedure branches to decision step 390 to determine whether any further saved regions remain. If so, after extracting the next region with the highest likelihood, the fine focus process 358 is repeated (step 350). Once all regions have been processed, the decision step branches to decision step 392 to determine whether any further coarse distances exist. If not, procedure 300 is complete (step 380). If another coarse distance exists (e.g., an imaging object that requires reading multiple barcodes at different distances but not all distances have been processed yet), procedure 300 branches to step 394 where the priority (if employed) is raised and the original region list is erased. This process can include, for example, reading a lower priority coarse focal plane, e.g., a focal plane outside the specified range determined around the input distance measurement by a range sensor (and / or other discovery methods) from a distance from a range detection device. In procedure 300, the distance is extracted again at the current priority in step 324, and both the coarse focus process and the fine focus process are repeated as described above until all coarse distances have been processed.

[0040] It is optional to provide an optional priority according to steps 324, 340, and 394, whereby procedure 300 can determine the priority of the rough surface / distance, and it should be noted that it determines how quickly a specific type of code can locate, focus, and decrypt. In a basic implementation, no priority is provided. In this scenario, step 324 selects the next rough surface, decision step 340 simply queries whether there are any more rough surfaces / distances to be processed, and step 394 can be omitted when the procedure branches from decision step 392 directly back to step 324. By adopting a priority, a priority can be assigned to each distance (based on the rough search mode and / or distance distribution provided by the system user as described above). Regions within a given distance / plane will only be processed after regions for distances / planes with a higher priority have been processed first.

[0041] As part of the decryption process, procedure 300 can employ one or more fine search algorithms. As non-limiting examples, such algorithms can include (a) fixed step size search, (b) fixed step size search by interpolation, (c) iterative search, (d) variable step size search, and (e) other acceptable iterative and / or numerical step-based search techniques. For background information, refer to "Active Computer Vision by Cooperative Focusing and Stereo Occlusion" by Krotkov, EP, Springer-Verlag, New York (1989).

[0042] IV. Conclusion

[0043] It will be apparent that the above-described system and method can effectively detect and decode various sizes and / or types of barcodes within a wide range of operating distances often encountered using a hand-held barcode reader. This system and method effectively utilize the advantageous performance of a high-speed variable (e.g., liquid) lens and more recent technology processors, such as the HotBars™ barcode detection algorithm and similar approaches / software. Accordingly, this system and method provide an advancement and advantage over prior art for barcode identification that is typically too slow for use in a focus adjustment loop. Additional instances of exemplary systems and methods using (e.g.) FPGA hardware (executing program instructions using hardware / firmware) can further improve and speed up the operation of barcode detection within the focus adjustment loop of an imaging device.

[0044] The above is a detailed description of exemplary embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the present invention. Each feature of the various embodiments described above may be combined with the features of another described embodiment as long as it is suitable for providing combinations of a number of features in related new embodiments. Further, although a number of separate embodiments of the apparatus and method of the present invention have been described above, what is described herein is merely illustrative of the application of the principles of the present invention. For example, it should be noted that it is necessary to acquire images at all coarse focus settings before attempting decoding. That is, in various exemplary embodiments, it is assumed that images can be acquired for a subset (one or more) of the coarse focus settings, and that subset can be processed until one or more barcodes are identified and decoded. Also, the terms "process" and / or "processor" as used herein should be interpreted broadly to include various electronic hardware and / or software-based functional elements and components (which can alternatively be referred to as functional "modules" or "elements"). Further, the illustrated process or processor can be combined with other processes and / or processors, or can be divided into various sub-processes or processors. Such sub-processes and / or sub-processors can be combined in various ways according to the embodiments herein. Similarly, it is clearly contemplated that any function, process, and / or processor herein can be implemented using electronic hardware, software consisting of non-transitory computer-readable program instructions, or a combination of hardware and software. Further, the various terms used herein to represent directions and / or orientations, such as "vertical", "horizontal", "up", "down", "bottom", "top", "side", "front", "rear", "left", "right", "forward", "backward" and the like are used only as relative expressions and do not represent absolute orientations based on a fixed coordinate system such as the direction of the action of gravity.In addition, when the terms "substantially" or "approximately" are used with respect to a given measurement, value or characteristic, it refers to an amount within the normal operating range for achieving the intended result, but includes a certain degree of variation due to inherent inaccuracies or errors within the tolerance range allowed for the system (for example, 1 to 5 percent).

[0045] The claims are set out below.

Claims

1. 1. A system for detecting and capturing one or more focused images of one or more barcodes within a field of view of an imaging device having an image sensor that captures one or more images of a scene including the one or more barcodes, comprising: a variable lens that receives focus adjustment information from the focus process and directs the received light to the image sensor; a measurement process for measuring an effective depth of field of the barcode as the maximum focus error that the one or more barcodes can be reliably detected by the focus process; a plurality of nominal coarse focus settings of a variable lens that enable sampling with sampling steps of a lens adjustment range corresponding to an allowable distance between the one or more barcodes and the image sensor, such that a step size of the sampling steps is less than a portion of a barcode detection depth of field; an acquisition process for acquiring a nominal coarse focus image for at least one coarse focus setting of the variable lens by the image sensor; a barcode detection process that detects one or more barcode-like regions in each nominal coarse-focused image; a fine focus process for at least one of the barcode-like regions in a coarsely focused image by fine-tuning the variable lens from a coarse focus setting to optimize image focus proximate the location of the barcode-like region and acquiring an image for decoding using the optimized fine-tuned focus setting of the variable lens; The above system comprising:

2. The system of claim 1 , wherein the barcode detection process is configured to measure a likelihood of each barcode-like region, and at least one barcode-like region defines a likelihood above a predetermined level.

3. 2. The system of claim 1, wherein the barcode detection process operates at substantially a maximum frame rate that reliably identifies barcode-like regions within images captured by an image sensor.

4. The system of claim 3 , wherein the portion of the barcode detection depth of field is less than or equal to about half the barcode detection depth of field.

5. 2. The system of claim 1, wherein at least one of: (a) the variable lens comprises a high speed liquid lens; and (b) the depth of field is measured in diopters and the adjustment information defines diopters.

6. 6. The system of claim 5, wherein the fine focus process is configured to optimize image focus by acquiring images at one or more adjusted focus settings and selecting the one or more adjusted focus settings that maximizes an image sharpness score in a local region surrounding each barcode-like region.

7. The system of claim 1 , further comprising a processing unit operatively connected to said variable lens and said image sensor to operate in at least one of said focusing process, barcode detection process, and acquisition process.

8. The system of claim 7 , wherein the processing unit comprises at least one of a GPU and an FPGA.

9. The system of claim 1 , wherein the variable lens is adjusted based on an iterative numerical step-based search technique over a coarse focus setting.

10. The system of claim 1 , wherein the fine focus process fine-tunes the variable lens starting from a coarse setting and in order from most likely to least likely.

11. The system of claim 1 , wherein the barcode comprises at least one of a 1D barcode and a 2D barcode.

12. The system of claim 1 , wherein the coarse focus setting is based on a distance measurement generated by a distance sensor relative to the imaging device.

13. The system of claim 12 , wherein the distance sensor includes at least one of a time-of-flight sensor, a LIDAR, a radar, an ultrasonic sensor, a stereo sensor, and a sonar sensor.

14. 1. A method for focusing an imaging system on one or more bar codes, comprising: acquiring a nominal coarse focus image for each of at least two nominal coarse focus settings, the nominal coarse focus settings being selected to sample a range of acceptable distances between each barcode and an imaging system; employing a barcode detector to determine barcode-like regions in each of the nominal coarse focus images and estimating respective likelihoods that each of the locations of the barcode-like regions represents an actual barcode, the barcode detector operating within an effective depth of field that allows barcodes at a distance range between sampled distances corresponding to a nominal coarse focus setting to be reliably detected in the presence of defocus; for each barcode-like region having a sufficiently high likelihood, in order from highest to lowest likelihood, optimizing focus using a fine focus procedure to acquire an image sharp enough to successfully decode one or more barcodes in each barcode-like region, said fine focus procedure optimizing focus by maximizing a sharpness measure in a local region around each barcode-like region by acquiring images at one or more adjusted focus settings substantially close to a nominal coarse focus setting used to acquire a nominal coarse focus image at which each barcode-like region is detected by a barcode detector; The above method comprising the steps of:

15. 15. The method of claim 14, further comprising the steps of receiving the captured nominal coarsely focused image with a vision system processor and activating a barcode detector with the vision system processor.

16. 16. The method of claim 15, wherein the step of receiving in a vision system processor comprises receiving in a processor including at least one of a GPU and an FPGA.

17. 16. The method of claim 15, further comprising an electronically controlled variable focus lens assembly for capturing said nominal coarse focus image at a selected focal length.

18. The method of claim 17 , wherein the lens comprises a high speed liquid lens whose focal length can be set based on a diopter value.

19. 20. The method of claim 17, further comprising adjusting the variable focus lens assembly based on an iterative numerical step-based search technique over a coarse focus setting.

20. The method of claim 14 , wherein the obtaining, employing, and using steps are performed according to a number of priorities.

Images