SYSTEM AND METHOD FOR AUTOFOCUSING A VISION SYSTEM CAMERA ON BARCODES

Abstract

System for detecting and capturing one or more sharply focused images of one or more barcodes within the field of view of an imaging device with an image sensor which captures one or more images of a scene containing the one or more barcodes, comprising: an adjustable lens that receives focus adjustment data from a focusing process and directs the received light to the image sensor; a measurement process that measures an effective depth of field for barcode detection as the maximum focus error for which one or more barcodes can be reliably detected by the focus process; a plurality of coarse focus nominal settings of the adjusting lens, which allows scanning in sampling steps of a lens adjustment range corresponding to the range of permissible distances between the one or more barcodes and the image sensor, so that a step size of the sampling steps is less than a fraction of the depth of field of the barcode detection, a capture process that uses the image sensor to capture a roughly focused nominal image for at least one coarse focus setting of the adjusting lens; a barcode detection process that detects one or more barcode-like regions in each coarsely focused nominal image; a fine-focus process which, starting from the coarse setting, performs a fine adjustment of the adjusting lens on at least one of the barcode-like regions with a coarsely focused image in order to optimize the focus of the image near a point of the barcode-like regions, and captures an image for decoding with an optimized, finely adjusted focus setting of the adjusting lens.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to machine vision systems for locating and decoding 1D and 2D barcodes and other types of symbol sets or ID codes, and more specifically to camera focusing systems for use in reading barcodes and / or other ID codes. BACKGROUND OF THE INVENTION

[0002] Vision systems are used to perform measurements, inspections, object orientations, and / or the decoding of symbol sets in the form of machine-readable symbols (also called "barcodes," "ID codes," "codes," and / or simply "IDs"). These machine-readable symbols often take the form of commonly known one-dimensional (1D) and two-dimensional (2D) barcodes, such as 2D DataMatrix codes, QR codes, DPM codes, etc. More generally, a barcode can be defined as an optical, machine-readable representation of data; the data often describes something related to the object bearing the barcode. Conventional barcodes represent data by varying the widths and spacing of parallel lines and can therefore be described as linear or one-dimensional (1D).Later, two-dimensional (2D) versions were also developed, using rectangles, dots, hexagons, or other geometric patterns. These are called matrix codes or 2D barcodes, although no actual lines are used. Examples include DataMatrix codes, QR codes, and DPM codes, all of which have a wide range of applications in commerce and industry. Barcode reading systems generally rely on an image sensor that captures images (usually one-, two-, or three-dimensional grayscale or color images) of the object. These images are then processed by a built-in or external vision processor.The processor generally contains both processing hardware and non-volatile, machine-readable program instructions that execute one or more vision system processes to generate a desired output based on the processed image information. This image information is typically provided within an array of image pixels, each with different colors and / or intensities. In the example of a barcode reader (also referred to here as a "camera"), the user or automated process captures an image of an object, assuming that it contains one or more barcodes. The image is processed to detect barcode features, which are then decoded by a decoding process and / or processor to extract the alphanumeric data represented in the code.

[0003] In operation, a barcode reader typically serves to illuminate the scene containing one or more barcodes. This illumination may include aiming devices that project a colored dot onto the region of interest in the imaged scene, allowing the user to center the reader's image axis on the barcode within the scene. This illuminated scene is then captured by an image sensor within the camera assembly through an optical system. The sensor's pixel array is exposed, and the electronic value(s) generated for each pixel by the exposure are stored in an array of memory cells, which can be referred to as the "image" of the scene. In the context of a barcode reading application, the scene comprises an object of interest that has one or more barcodes of an appropriate type and size. The barcode(s) is / are part of the stored image.

[0004] A common application for barcode readers is tracking and sorting objects along a line (e.g., a conveyor belt) during manufacturing and logistics processes. The barcode reader, or more typically a group of readers (constellation), can be positioned across the conveyor line at appropriate viewing angles to capture any expected barcodes on the surfaces of corresponding objects as they move through the field of view. The focal length of the reader relative to the object can vary depending on its placement along the conveyor line and the size of the object. Therefore, the use of an autofocus mechanism in conjunction with the barcode reader is desirable to achieve the correct focal length.

[0005] Existing techniques for providing autofocus functionality for barcode readers involve cycling through a potentially large number of focus settings and selecting the one that offers maximum image sharpness for a chosen region of interest. For relevant background information, an autofocus method is described in Daniel Vaquero, Natasha Gelfand, Marius Tico1, Kari Pulli1 and Matthew Turk, Generalized Autofocus, Applications of Computer Vision (WACV), 2011 IEEE.

[0006] The region of interest is often selected using heuristic contrast measurements, which frequently mistake other high-contrast structures (e.g., text or graphics) for barcodes. Both contrast measurements and sharpness metrics are only useful if the focus is already nearly correct, which necessitates considering a large number of focus settings during the selection process. Furthermore, sharpness metrics are sensitive to lighting, an aspect that can also vary (e.g., during an autofocus process that also optimizes lighting). Such methods are therefore time-consuming and can also lead to focusing on the wrong area of ​​the scene. Moreover, commercially available barcode detection techniques are currently too slow for direct use during an autofocus process loop.Generally, the focus is first set on a region, and then the barcode detection and decoding methods work with a focused image of the region of interest.

[0007] The paper by Daniel Vaquero et al., “Generalized Autofocus”, in: IEEE Workshop on applications of computer vision, IEEE, 2011, pp. 511-518, concerns a focusing system for computer-aided photography in which the focus is initially varied in low-resolution images of a scene by changing the distance to the lens. From this, a computer determines an ideal lens position that renders all objects in the scene sharply. DE 699 31 532 T2 and DE 100 41 847 A1 represent further state of the art. OVERVIEW OF THE INVENTION

[0008] The invention is defined by the independent claims. Preferred embodiments are defined by the dependent claims.

[0009] The present invention overcomes the disadvantages associated with the prior art by providing a system and associated method for detecting and capturing one or more sharply focused images of one or more barcodes within the field of view of the imaging device, which captures one or more images of a scene containing the barcodes. An adjusting lens receives focus adjustment data from a focusing process and directs the received light to an image sensor. A measurement process measures the effective depth of field for barcode detection as the maximum focus error for which the one or more barcodes can be reliably detected by the focusing process. This depth-of-field measurement process can be based wholly or partially on automated systems or on manual input from (e.g.) a system user, which may involve estimates (e.g.,) based on heuristics, observations, or other calculations. A plurality of coarse-focus nominal settings of the adjusting lens allows scanning in sampling steps of a lens adjustment range corresponding to the range of permissible distances between the one or more barcodes and the image sensor, such that the step size of the sampling steps is less than a fraction of the depth of field for barcode detection. A capture process uses the image sensor to acquire a coarsely focused nominal image for at least one coarse-focus setting of the adjusting lens. A barcode detection process detects one or more barcode-like regions (which may also be a segment of a complete barcode pattern) within each coarsely focused nominal image.A fine-focus process, for each barcode-like region with a sufficiently high probability of the coarsely focused image, starting from the highest to the lowest probability, fine-tunes the focusing lens to optimize the focus of the image near a point within the barcode-like region. The fine-focus process then captures an image for decoding using this optimized, finely adjusted focusing lens.

[0010] In an exemplary embodiment, the system provides a system and associated method for detecting and acquiring one or more sharply focused images of one or more barcodes within the field of view of an imaging device with an image sensor that acquires one or more images of a scene containing the one or more barcodes. The system and associated method comprise an adjusting lens that receives focus adjustment data from a focusing process and directs the received light to the image sensor. A measuring process measures the effective depth of field for barcode detection as the maximum focus error for which the one or more barcodes can be reliably detected by the focusing process.Multiple coarse focus nominal settings of the focusing lens allow scanning in steps across a lens adjustment range corresponding to the permissible distances between the one or more barcodes and the image sensor. The step size of the scanning steps is typically smaller than a fraction of the depth of field required for barcode detection. A capture process uses the image sensor to acquire a coarsely focused nominal image for at least one coarse focus setting of the focusing lens. A barcode detection process detects one or more barcode-like regions and their respective probabilities in each coarsely focused nominal image. A fine focus process, starting from the coarse setting, fine-tunes the focusing lens for each barcode-like region with a sufficiently high probability in the coarsely focused image to optimize the image focus near that point in the barcode-like region.The fine-focus process captures an image for decoding using an optimized, finely tuned focus setting of the adjustment lens. For illustrative purposes, the barcode detection process is configured to measure the probability of each barcode-like region, with at least one of the barcode-like regions defining a probability above a predetermined level. The barcode detection process can essentially operate at the maximum frame rate at which the barcode-like regions within an image are reliably located by the image sensor. The depth of field can be measured in diopters, and the setting data defines diopters. In some embodiments, the fraction can be a value no greater than approximately half the depth of field of the barcode detection. The adjustment lens can be a high-speed liquid lens.For illustrative purposes, the fine-focusing process can be arranged to optimize image focus by capturing an image with one or more adjusted focus settings. These one or more adjusted focus settings can be arranged to maximize the image sharpness rating in a local region around a corresponding barcode-like region. A processing unit can be provided that is functionally connected to the focusing lens and the image sensor. This processing unit can execute the focusing process, the barcode detection process, and / or the capture process. For illustrative purposes, the processing unit can include a GPU (graphics processing unit) and / or an FPGA (field-programmable gate array).The focusing lens can be adjusted across coarse focus settings using an iterative search technique and / or a numerical step-based search technique. The fine focus process fine-tunes the focusing lens from the coarse setting, sequentially from highest probability to lowest probability. The barcode can be a 1D barcode and / or a 2D barcode. The coarse focus adjustment can be based on a distance measurement generated by a distance sensor relative to the imaging device. This distance sensor can be a time-of-flight (TOF) sensor, a lidar sensor, a radar sensor, an ultrasonic sensor, a stereoscopic sensor, and / or a sonar sensor.

[0011] In an exemplary embodiment, a system and method for focusing an imaging system on one or more barcodes are provided. Coarsely focused nominal images are acquired for each of at least two coarse focus nominal settings. The coarse focus nominal settings are selected such that the range of permissible distances between the barcodes and the imaging system is scanned in each case. A barcode detector identifies barcode-like regions within each coarsely focused nominal image and estimates the probability that each of these barcode-like regions is actually a barcode.The barcode detector operates within an effective depth of field that allows a barcode to be reliably detected even in the presence of focus blur, within a range of scanned distances corresponding to the coarse focus nominal settings. For each barcode-like region with a sufficiently high probability, from highest to lowest, a fine focus procedure optimizes the focus and captures an image sharp enough to successfully decode the one or more barcodes present in that region.The fine-focus method optimizes focus by maximizing sharpness measurement in a local region around the barcode-like area. This is achieved by capturing images with one or more adjusted focus settings that are substantially close to the coarse-focus nominal setting used to capture the coarsely focused nominal images in which the barcode-like area is detected by the barcode detector. To illustrate, a vision system processor receives the captured, coarsely focused nominal images and activates the barcode detector. The vision system processor may include a GPU and / or an FPGA. An electronically controlled, focus-adjustable lens assembly can be used to image the coarsely focused nominal images at selected focal lengths. The lens may further include a high-speed liquid lens whose focal length can be adjusted based on a diopter value.To illustrate, the operation of the system and process can be carried out according to a multiple priority levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The following description of the invention refers to the accompanying drawings, in which: Fig. 1 a diagram of a vision system arrangement in which a stationary or portable vision system camera is shown which, using an autofocus system and method according to an embodiment, images an object containing a barcode; Fig. 2 a flowchart of a generalized procedure for applying coarse focus and fine focus together with barcode detection and decoding according to an embodiment; Fig. 2A is a diagram showing the field of view of the camera assembly. Fig. 1 and a central target region as well as a user-defined region of interest for use in locating barcode-like regions; Fig. 3 a more detailed method for the autofocus process according to an illustrative embodiment; and Fig. 4 is an exemplary captured image of a scene containing an object with a barcode on it, showing various regions of interest with which the methods from Fig. 2 and Fig. 3 are operated. DETAILED DESCRIPTION I. System Overview

[0013] Fig. Figure 1 shows an arrangement 100 for reading and decoding one or more barcodes (one-dimensional and two-dimensional barcodes (1D and 2D barcodes), such as DPM and QR codes and other codes with related symbol sets) 110 and 114, which are located on an exemplary object and / or other depicted scene 112 according to an embodiment. The exemplary object 112 is shown here with several surfaces 115, 117, and 119 that define a three-dimensional structure. The arrangement 100 comprises a vision system camera assembly 120, which can be a portable or a stationary unit. A plurality of cameras can be used to image the scene in alternative embodiments, each operating the present system and method.As described in more detail below, the present system and method are capable of effectively distinguishing between barcode-like features 110 and 114 relative to other adjacent geometric and textual data, such as text 116. Thus, the present system and method effectively enable the rapid detection and decoding of one or more barcodes within the field of view (FOV), even when other potentially confusing features are present, which are just as quickly discarded by the process as barcode-like regions (and / or potential barcodes). The term "barcode-like region" as used here refers to all or part of an image containing features that represent all or part of a barcode of any suitable type / format.Therefore, the barcode-like region can be identified based on a recognizable section and / or fragment of the entire barcode present on the object. It should be noted that barcode 114 is located on the upright surface 117, so that it has a different distance relative to the image sensor compared to barcode 110 on the object surface 115. The present system and method are also capable of effectively detecting multiple barcodes at different distances within the same overall field of view (FOV) using the autofocus techniques described herein.

[0014] The camera assembly can include various functional elements, such as built-in and / or separate illumination / aiming devices 122, an image sensor S (which may be implemented using CMOS or another suitable technology), and an optical module O. The sensor S can define any usable format and any pixel arrangement size and can be implemented to provide image data 124 either in color or in grayscale form.

[0015] The optics O can be any suitable optical assembly, which can be either permanently installed or removable (for example, a screw-in lens base assembly). The optics O comprises a focus-adjustable lens unit 130, which physically changes the focal length of the optics to adapt to the working distance along the optical axis OA between the image plane of the sensor S and the surface containing the barcode 110. The focus-adjustable lens 130 can be based on various operating principles—for example, as a mechanical, motor-driven lens that moves along the optical axis to vary the focus, or as a so-called liquid lens, which changes the shape of the lens profile based on the movement of a ferrofluid in the presence of an electromagnetic field.Liquid lenses are commercially available from various suppliers, such as the French company Varioptic SA or the Swiss company Optotune AG. The focus-adjustable lens 130 is controlled by signals 132, which are generated according to the respective manufacturer's specifications. Typically, such signals 132 are a variable current and / or voltage level, based (usually) on a digital value provided by the vision system process 140 (or another process), which are described below. The focus adjustment function can be used to adjust the size of the field of view (FOV) so that the barcode is fully imaged while maintaining sufficient resolution for the process 140 to decode the barcode features.

[0016] The vision system processor 140 may be partially or completely contained within the housing of the camera assembly or another device (for example, a connected smartphone). Alternatively, or additionally, the processor may be partially or completely instantiated in a separate standalone computing device 150, such as a server, PC, laptop computer, tablet, or smartphone, which is connected to the camera assembly 120 via a suitable wired and / or wireless data connection 152. Such a computing device 150 may also be detachably attached to an embedded processor within the camera assembly if required for monitoring, adjusting, or configuring the vision system. The computing device 150 may include suitable user interface components, such as a display / touchscreen 154, a keyboard 156, and / or a mouse 158.The process(or) 140, the computing device 150 and / or another device within the arrangement 100 can be connected to one or more downstream user devices and / or processes which use decoded data obtained from the barcodes to perform tasks such as part recognition, logistics, object sorting, etc.

[0017] In some embodiments, the processor 140 may include a graphics processing unit (GPU) or a field-programmable gate array (FPGA) which executes the desired program instructions used here.

[0018] The vision system process 140 comprises a plurality of functional processes for executing the operations of the vision system arrangement 100. The vision system process 140 can include a vision system tool module 142, which may include one or more tools used in the detection and analysis of image features—such as edge finders, blob analyzers, calipers, alignment and pattern recognition processes or processors. The vision system process 140 also includes a barcode detection and decoding process 144, which may be based on a commercially available or customized (e.g., software) application. The barcode detection and decoding device locates potential barcode features within the captured image of scene 112. It should be noted that the vision system tools 142 can be used to refine and parse these features to better locate elements that could be barcodes.If relevant barcode features are resolved, a decoding process(or) within module 144 can then be used to generate corresponding data (e.g. alphanumeric data strings) from the read barcode(s) 110.

[0019] As part of the image acquisition and barcode detection process, the camera optics can be adjusted to account for the distance between the imaged area and the sensor image plane. Since it is desirable to locate barcodes as quickly as possible to avoid overlooking barcodes that appear in the field of view only briefly and / or while in relative motion, the present focus process 146 attempts to adjust the focal length of the adjustable lens 130 according to the predetermined procedure described below.

[0020] Optionally, one or more area-finding devices (e.g., a lidar, time-of-flight sensor, stereoscopic array, ultrasound, and / or radar) 170 can be used to provide area data 172 to a suitable functional module or modules of the vision system process or processor 140. The area data 172 can define the physical distance (double dashed line 174) between the camera assembly 120 (and relative to the sensor image plane) and the surface of the imaged object 112. This data 172 can be used to supplement the pixel-based focal length data used here.

[0021] It should be noted that the Barcode Detection and Decoding Process (or) 144 can be based on a commercially available or custom-built high-speed barcode / symbol detector, such as the Hotbars™ Barcode Detection Device, commercially available from Cognex Corporation of Natick, MA. Such a detector is capable of rapidly locating barcode-containing areas / regions in an image, even in the presence of blurring, and sorting them according to the probability that they are actual barcodes.

[0022] In addition to the fast line-finding capabilities of the Hotbars™ barcode finding device described above, there are several other techniques that can be used in conjunction with suitable processor arrangements described here to locate and decode barcode features (e.g., 2D barcodes) more quickly: 1. The use of line detection techniques based on edge detection and Hough transform in combination with clustering to detect parallel lines in a potential code element. For background information, see Peter Bodnar and Laszlo G. Nyul, Improving Barcode Detection with Combination of Simple Detectors, Eighth International Conference on Signal Image Technology and Internet Based Systems (2012); and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes, Dissertation, PhD School in Computer Science, Department of Image Processing and Computer Graphics, Faculty of Science and Informatics, University of Szeged, Hungary (2015), available online at http: / / doktori.bibl.uszeged.hu / 2825 / 8 / tezisfuzet.pdf; 2. The use of a histogram of Gaussian features (HoG features) in tiles of the captured image containing the element that could be the barcode, in order to detect neighboring tiles with a similar dominant orientation. For background information, see Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above). 3. The use of adaptive image thresholding to binarize the lines and spaces in the mapped barcode and to map morphology in order to merge them with the barcode regions. For background information, see Daw-Tung Lin, Chin-Lin Lin, Automatic Location for Multi-Symbology and Multiple 1D and 2D Barcodes, Journal of Marine Science and Technology, Vol. 21, No. 6, pp. 663–668 (2013); Savreet Kaur and Raman Maini, Implementation of Barcode Localization Technique using Morphological Operations, International Journal of Computer Applications (0975-8887), Vol. 97, No. 13, July 2014; and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above). 4. The use of deep learning for texture classification. For background information, see Hansen, Daniel Kold; Nasrollahi, Kamal; Rasmussen, Christoffer Bøgelund; Moeslund, Thomas B., Real-Time Barcode Detection and Classification Using Deep Learning, published in the Proceedings of the 9th International Joint Conference on Computational Intelligence, Volume 1: IJCCI (2017); and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above).

[0023] Further techniques for faster detection and decoding of selected types of barcodes under certain conditions are described by way of example in Yuhua Gu, Weixiang Zhang, QR Code Recognition Based On Image Processing, International Conference on Information Science and Technology, IEEE, published on May 10, 2011. II. General Autofocus Operating Procedure

[0024] Fig. Figure 2 shows a general method for performing an autofocus method 200 according to one embodiment. The user, or another process, triggers image acquisition upon the presence of a barcode (or, more generally, upon the presence of a "barcode-like region," this term being used synonymously with "barcode" where appropriate) in a scene 210. The method 200 then applies a coarse focus adjustment in step 220: This adjustment instructs the lens to assume a predetermined focal length (usually expressed as a focus / diopter value). Subsequently, in step 222, at least one image is acquired by the sensor at this selected focal length.The system then searches for potential barcode features using appropriate vision system tools. If no barcode features are found and / or the found features do not meet a predetermined confidence score / probability (decision step 224), the next coarse focus value (focus plane, described below) is selected (step 226), and the adjustment in step 220 is applied. Another image is acquired, and the barcode localization / identification process (decision step 224) is repeated until a barcode is found (or until the scanning of all coarse focus values ​​is complete).If there is a high degree of confidence and / or probability of presence (confidence value / rating above a predetermined threshold) that a barcode is present in a region of interest (via step 224), the procedure 200 applies a fine focus adjustment procedure to the lens in step 230. In an embodiment described in more detail below, an initial set of nominal focus planes can be determined, and a coarse focus process scans these planes sequentially until a barcode for a region has been identified with sufficient confidence / probability.Furthermore, the procedure can prioritize regions to be scanned based on barcode probability or other metrics – for example, user-defined metrics concerning regions with a high barcode probability within the captured image – and scan regions based on such priorities with respect to coarse focus, continuing between focus planes until either the scanning of all is completed or a barcode is found with sufficient confidence.

[0025] In step 232, an image is captured using this fine-adjustment value. The barcode detection and decoding procedure then determines whether the barcode(s) located during the coarse focus step will be decoded to provide relevant information (decision step 234). If not, a new fine focus value is selected and applied to the lens in step 236 (step 230), and then detection step 232 and determination step 234 are repeated until a barcode is decoded or the scanning of all fine values ​​is complete. The decoded barcode results / data (if any) are then transferred to an appropriate downstream process in step 240 (via decision step 234). In the example field of view 250 in Fig. 2A, which is now referred to, can be used as a target region for a subsequent barcode search if no region containing a decodable barcode or a barcode-like region (or a fragment thereof) is found during the coarse focus process. This target region is the central quarter (approximately the outlined box 260) of the field of view 250, or a user-selected region of interest (for example, the dashed box 270). This process then switches to a traditional contrast / sharpness-based search algorithm / method to locate one or more barcodes within the new region 260 or 270.

[0026] It should be noted, as described generally above and explained in more detail below, that each captured image of the scene may contain several regions of interest, which may contain features that require decoding. For example, Fig. 4. A captured image scene 400 of an underlying object, in which at least three regions of interest 410, 420, and 430 are depicted, each containing a 2D barcode, a 1D barcode, and text. The text can be excluded from further processing once it is ensured that it does not contain any barcode features. The boundaries of each region can vary and can have a default setting or be sized to include the barcode-like regions (barcodes and / or fragments thereof) contained within it.

[0027] Advantageously, during operation, a small number of captured images at various coarse diopter settings across the entire possible focus range of the imaging system (called nominal focus planes) are used. These allow the barcode detector to detect all barcode-like regions in the scene, regardless of distance, by defining a relatively large depth of field for the reading optics. Such high-depth-of-field optics are described, as relevant background information, in U.S. Patent Application No. 15 / 844,448, entitled "DUAL-IMAGING VISION SYSTEM CAMERA AND METHOD FOR USING THE SAME," filed on December 15, 2017. The process of capturing images at these nominal focus planes is particularly efficient when combined with the use of a high-speed adjusting lens assembly capable of rapidly responding to changes in diopter value—similar to a liquid lens.

[0028] As described below, the method 200 can also use an optional distance measurement (e.g. the range data 172) from available sensors (e.g. TOF sensor or other range finding device 170) to limit the coarse focus range such that the initial value and the incremental value change are within and around a detected range.

[0029] In some embodiments, a fine-focus step is used to refine the coarse-focus diopter setting for each barcode-like region. An image is captured with this setting, and then the decoding process attempts to decode the barcode. Barcode-like areas are processed in order of the highest probability of barcode features being present. This arrangement essentially represents an intelligent mechanism for visual attention (both for localization within the field of view and for distance within the focal length range of the imaging system or camera assembly). III. Autofocus process

[0030] A more detailed flowchart of an autofocus procedure 300 is in Fig. Figure 3 shows that the method 300 receives input data, including the total distance range 310 of the camera or imaging system, optional distance measurements 312 from a range-finding sensor, the coarse search modes—i.e., the technique(s) to be used when searching for barcode-like regions 314 and when searching—whether the coarse search should look beyond a given distance defined by the optional distance measurements (closer and / or farther away), and (optionally) whether any other distances (closer to and / or farther away from the image sensor) beyond the available distance measurements 316 should be checked. It should be noted that the input elements 312, 314, and 316 in the overall method 300 may be optional and thus in Fig.3 are indicated as such (i.e., “OPTIONAL”). Other optional process steps / inputs are also indicated, as described below. Such optional inputs, steps, and processes can be omitted from the overall procedure and / or selected by the user during setup or at runtime. More specifically, the coarse search mode input 314 determines the priorities of the various coarse level distances, as described below. The use of priorities in coarse level processing is generally optional, as described below. The look-beyond-distance input 316 can decide to consider the distance measurements (if any) as precise and not to search for barcodes elsewhere.Alternatively, input 316 can decide to consider the distance measurements (if any) as clues and potentially search for barcodes at these locations first, but then, if the process has not successfully decoded the required barcodes, search for barcodes at other distances within the area.

[0031] The data from elements 310-316 are used, or can be used, to calculate a list of coarse focus planes used in the coarse focus procedure (step 320). After calculating the coarse focus planes, a distance is then taken from the coarse plane list in step 324. This begins with an initial plane value, which may be a predefined value (e.g., an average distance within the plane range and / or the distance measured by the rangefinder (step 324)). Alternatively, the distance measurement may be based on (or include) a predetermined distance distribution according to a suitable heuristic operating within the sensor's predefined working distance. It should be noted that this part of procedure 300 may be arranged in priority levels, with step 324 taking a distance corresponding to the current priority level.If no priority level is provided, only a single set of focus planes is used. The camera optics are then focused to the diopter value assigned to the current (priority level) focus plane, and at least one image is captured in step 326. The procedure uses the captured image(s) to determine (decision step 330) whether and, if so, where any barcodes are present in the image, or whether there is a probability of their presence (step 332). If so, these barcodes / probabilities are stored (added to the region list) along with their respective region(s) and their corresponding focal length value(s). It should be noted that barcode detection can be based on an automatic detection function or can take place outside of regions of interest in the overall image (input 336).Subsequently, in step 334, one or more focus regions are calculated and appended along with the overall probabilities for regions of interest (input 338). It should be noted that the input and designation of regions of interest 338 and 334 are optional in the overall procedure 300. Such optional regions of interest can be used if prior knowledge exists regarding a possible or approximate location of the barcode(s) in the image—for example, based on their location in the previously acquired image(s). It should also be noted that the 'Automatically detect' input 336 and process steps 330 and 332 are optional.The optional 'Auto-Find' process steps 330, 332, and 338 can be used if the procedure is able to interpret the regions of interest (if any) as precise and therefore does not search for barcodes in other locations outside these regions. If 'Auto-Find' is not used (decision step 330), the regions of interest (if any) can only be considered as clues, and procedure 300 can initially search for barcodes in these regions. However, if the required barcodes are not decoded, procedure 300 will subsequently search within the regions of interest, across (e.g.) the entire captured image, for barcode-like regions.

[0032] Procedure 300 then checks whether all level spacings have been sampled (decision step 340). Optionally, this check can be restricted to the current priority level if multiple priority levels are provided in procedure 300. If multiple level spacings need to be sampled in the coarse focus process (optionally within the current priority level, if one is provided), procedure 300 returns to step 324 and retrieves the next incremental spacing. The coarse detection process (steps 324, 326, 330, 332, and 334) is then repeated via decision step 340.Conversely, once the distance sampling (optionally at the current priority level) is complete, decision step 340 branches to step 350, in which the region of the overall coarse image with the highest probability is selected (and removed) from the previously stored list of image regions. The camera lens is then focused to the best coarse focus for this region, and one or more images are acquired in step 354. At this stage, the fine-focusing process (dashed box 358) is performed, with the fine-focus score being calculated in step 356. Sharpness measurements used in this calculation may include (a) a gradient measurement, (b) a frequency domain measurement, (c) an autocorrelation measurement, (d) a variance measurement, (e) a statistical measurement, and / or (f) an edge-based measurement.For background information, see Yi Yao, Besma Abidi, Narjes Doggaz, and Mongi Abidi, Evaluation of Sharpness Measures and Search Algorithms for the Auto-Focusing of High Magnification Images, Proceedings of the SPIE - The International Society for Optical Engineering 6246, June 2006. If the focus generates a sharpness rating that may not be sufficient (below a predetermined threshold) to produce a readable barcode (decision step 360), the next fine focus setting is incremented (step 362), and the lens is refocused, capturing one or more new images (step 354) and subsequently calculating a new sharpness rating (step 360).

[0033] If a sufficient sharpness score is generated (decision step 360), an attempt is made to decode the barcode features within the region using a corresponding decoding process (step 370). It should be noted that, in addition to using regions of interest, the coarse and / or fine focus process (barcode search) can be performed using captured full-size images or partially scanned images. If partially scanned images are used for the focus process, a full-size image should be captured at the desired focus plane for decoding. If the given number of required (requested) barcodes are correctly located / decoded and the results are provided to the system (and no further barcodes are present in the image to be decoded), the focus and decoding process is complete (step 380 via decision step 372).Otherwise, the process branches to decision step 390 to determine whether any further stored regions remain. If so, the fine-focus process 358 is repeated after the next most likely region has been extracted (step 350). Once all regions have been processed, the decision step branches to decision step 392 to determine whether any further coarse distances exist. If not, process 300 is completed (step 380). If another coarse distance exists (for example, a mapped object where multiple barcodes with different distances need to be read, but not all distances have yet been processed), process 300 branches to step 394, in which the priority level (if one is used) is incremented and the original region list is deleted.This process can, for example, include reading coarse focus planes with lower priority levels from distance measurements taken by rangefinders. These focus planes may lie outside the specified area around a distance measurement input from a range sensor (and / or another heuristic). In procedure 300, the distance is again taken at the current priority level in step 324, and the coarse and fine focus processes are repeated as described above until all coarse distances have been processed.

[0034] It should be noted that the provision of optional priority levels according to steps 324, 340, and 394 is optional, enabling procedure 300 to establish a priority order for the coarse levels / distances, which determines how quickly certain types of codes can be located, focused, and decoded. In a basic implementation, no priority is provided. In this scenario, step 324 selects the next coarse level, decision step 340 simply queries whether further coarse levels / distances need to be processed, and step 394 can be omitted, with the procedure branch from decision step 392 directly returning to step 324. Priority levels can be used, assigning a priority level to each distance (based on the coarse search mode and / or distance distribution determined by the system).(provided by the user, as described above). Regions at a given distance or level will only be processed after the regions for the distances / levels with higher priority levels have been processed.

[0035] As part of the decoding process, Procedure 300 can employ one or more fine-search algorithms. Examples, without limitation, include: (a) a fixed-step size search, (b) a fixed-step size search in interpolation, (c) an iterative search, (d) a variable-step size search, and (e) another usable iterative and / or numerically step-based search technique. For background information, see E.P. Krotkov, Active Computer Vision by Cooperative Focus and Stereo, New York, Springer-Verlag (1989). IV. Concluding Remarks

[0036] It is understood that the system and method described above effectively detect and decode barcodes of various sizes and / or types within a wide range of working distances commonly encountered when using a portable barcode reader. The system and method effectively utilize the advantageous operating characteristics of high-speed adjustable lenses (e.g., liquid lenses) and technically advanced processors, such as the Hotbars™ barcode detection algorithm and similar approaches or software. This system and method thus represent advancements and advantages over conventional barcode localization techniques, which were generally too slow for use in a focusing loop. By additionally instantiating the exemplary system and method using (e.g.,) FPGA hardware (which executes the program commands described here using hardware / firmware) allows for further improvement and acceleration of barcode detection within the focus loop of the imaging device.

[0037] The foregoing is a detailed description of exemplary embodiments of the invention. Various modifications and additions can be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above can, where appropriate, be combined with features of other described embodiments to provide a multitude of feature combinations in associated new embodiments. Furthermore, while the foregoing describes a number of separate embodiments of the device and method of the present invention, what is described here is merely illustrative of the application of the principles of the present invention. For example, it is noted that images must be acquired at all coarse focus settings before any decoding is attempted.This means that in various exemplary embodiments, it is considered that images can be acquired for a subset of (one or more) coarse focus settings and that this subset can be processed until one or more barcodes are located and decoded. Furthermore, the terms "process" and / or "processor" used herein should be interpreted broadly to encompass a variety of electronic hardware- and / or software-based functions and components (and may alternatively be referred to as functional "modules" or "elements"). Additionally, a described process or processor may be combined with other processes and / or processors or subdivided into various subprocesses or subprocessors. Such subprocesses and / or subprocessors may be combined with one another in various ways according to the embodiments described herein.It is also expressly considered that each function, process, and / or processor herein may be implemented using electronic hardware, software consisting of a non-volatile, computer-readable medium of program instructions, or a combination of hardware and software. Furthermore, various directional and arrangement-related terms used herein, such as "vertical," "horizontal," "upward," "downward," "lower," "upper," "lateral," "front," "rear," "left," "right," and the like, are used only as relative references and not as absolute directions / arrangements with respect to a fixed coordinate space, such as the direction of gravity.Furthermore, the terms "essentially" or "approximately," when applied to a specified measure, value, or property, are intended to refer to a quantity that lies within a normal operating range for achieving desired results, although fluctuations due to inherent inaccuracies and errors within the permissible tolerances of the system (e.g., 1-5 percent) are possible. Accordingly, this description is to be considered merely exemplary and is not intended to otherwise limit the scope of protection of the present invention.

Claims

[1] A system for detecting and capturing one or more focused images of one or more barcodes within the field of view of an imaging device having an image sensor that captures one or more images of a scene containing the one or more barcodes, comprising: an adjustment lens that receives focus adjustment data from a focus process and directs received light to the image sensor; a measurement process that measures an effective depth of field of the barcode detection as the maximum focus error for which the one or more barcodes can be reliably detected by the focus process; a plurality of coarse focus nominal settings of the adjustment lens, which allows scanning in scanning steps of a lens adjustment range corresponding to the range of permissible distances between the one or more barcodes and the image sensor, such that a step size of the scanning steps is less than a fraction of the depth of field of the barcode detection, a capture process that captures a coarsely focused nominal image with the image sensor for at least one coarse focus setting of the adjustment lens; a barcode detection process that detects one or more barcode-like regions in each coarsely focused nominal image; a fine focus process that, for at least one of the barcode-like regions with a roughly focused image, finely adjusts the adjustment lens from the coarse adjustment to optimize the focus of the image near a location of the barcode-like regions, and acquires an image for decoding with an optimized, finely adjusted focus setting of the adjustment lens. [2] The system of claim 1, wherein the barcode detection process is arranged to measure a respective probability of the barcode-like regions, and wherein the at least one of the barcode-like regions defines a probability above a predetermined level. [3] The system of claim 1, wherein the barcode detection process is operated at substantially the maximum frame rate at which the barcode-like regions within an image are reliably located by the image sensor. [4] The system of claim 3, wherein the fraction is no greater than approximately one-half of the depth of field of the barcode detection. [5] The system of claim 1, wherein (a) the adjustment lens comprises a high-speed liquid lens and / or (b) the depth of field is measured in diopters and the adjustment data defines diopters. [6] The system of claim 5, wherein the fine focus process is arranged to optimize the focus of the image by acquiring an image with one or more adjusted focus settings, and selecting the one or more adjusted focus settings that maximize an image sharpness rating in a local region around the corresponding one of the barcode-like regions. [7] The system of claim 1, further comprising a processing unit in functional connection with the adjustment lens and with the image sensor, which executes the focus process and / or the barcode detection process and / or the capture process. [8] The system of claim 7, wherein the processing unit comprises a GPU and / or FPGA. [9] The system of claim 1, wherein the adjustment lens is adjusted based on an iterative, numerical step-based search technique across the coarse focus settings. [10] The system of claim 1, wherein the fine focus process performs a fine adjustment of the adjustment lens starting from the coarse adjustment, sequentially from a highest probability to a lowest probability. [11] The system of claim 1, wherein the barcode comprises a 1D barcode and / or a 2D barcode. [12] The system of claim 1, wherein the coarse focus adjustment is based on a distance measurement generated by a distance sensor relative to the imaging device. [13] The system of claim 12, wherein the distance sensor comprises a TOF sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor and / or a stereoscopic sensor and / or a sonar sensor. [14] A method for focusing an imaging system on one or more barcodes, the method comprising the steps of: coarsely focused nominal images are acquired for each of at least two coarse focus nominal settings, the coarse focus nominal settings being selected to scan the range of permissible distances between corresponding ones of the barcodes and the imaging system; a barcode detector is used to determine barcode-like regions within each of the coarsely focused nominal images and to estimate a corresponding probability that each of the locations of the barcode-like regions actually display barcodes, wherein the barcode detector is operated within an effective depth of field that allows a barcode to be reliably detected at a range of distances between the sampled distances corresponding to the coarse focus nominal settings, even in the presence of focus blur; and for each of the barcode-like regions with a sufficiently high probability, from highest to lowest probability, a fine focus method is used to optimize the focus and acquire an image that is sufficiently sharp to successfully decode the one or more barcodes at corresponding ones of the barcode-like regions, wherein the fine focus method optimizes the focus by maximizing a sharpness measurement in a local region around a corresponding one of the barcode-like regions by acquiring images with one or more adjusted focus settings that are substantially close to the coarse focus nominal setting used to acquire the coarsely focused nominal images in which the respective one of the barcode-like regions is detected by the barcode detector. [15] The method of claim 14, further comprising receiving the acquired coarsely focused nominal images at a vision system processor and operating the barcode detector with the vision system processor. [16] The method of claim 15, wherein the step of receiving at the vision system processor comprises receiving at a processor comprising a GPU and / or an FPGA. [17] The method of claim 15, further comprising an electronically controlled, focus-adjustable lens assembly that images the coarsely focused nominal images at selected focal lengths. [18] The method of claim 17, wherein the lens comprises a high-speed liquid lens whose focal length can be adjusted based on a diopter value. [19] The system of claim 17, further comprising adjusting the focus-adjustable lens assembly based on an iterative, numerical step-based search technique across the coarse focus settings. [20] The method of claim 14, wherein the steps of detecting, using and utilizing are performed according to a plurality of priority levels.

Images