SYSTEM AND METHOD FOR AUTOFOCUSING A VISION SYSTEM CAMERA ON BARCODES

Abstract

The present invention provides a system and associated method for detecting and acquiring one or more sharply focused images of one or more barcodes within the field of view of an imaging device. A measurement process measures the depth of field for barcode detection. A plurality of coarse focus nominal settings of an adjustable lens allows stepwise scanning of a lens adjustment range, which corresponds to permissible distances between the one or more barcodes and the image sensor, such that the step size of the scanning is less than a fraction of the depth of field for barcode detection. An acquisition process captures a coarsely focused nominal image for each coarse focus nominal setting. A barcode detection process detects one or more barcode-like regions and their corresponding probabilities.A fine-focus process fine-tunes the focusing lens for each barcode with a high probability of being detected, positioning it near a point in the barcode-like regions. Using this fine-tuned setting, the process captures an image for decoding.

Description

AREA OF INVENTION

[0001] The present invention relates to machine vision systems for locating and decoding 1D and 2D barcodes and other types of symbol sets or ID codes, and more specifically to camera focusing systems for use in reading barcodes and / or other ID codes. BACKGROUND OF THE INVENTION

[0002] Vision systems are used to perform measurements, inspections, object orientations, and / or the decoding of symbol sets in the form of machine-readable symbols (also called "barcodes," "ID codes," "codes," and / or simply "IDs"). These machine-readable symbols often take the form of commonly known one-dimensional (1D) and two-dimensional (2D) barcodes, such as 2D DataMatrix codes, QR codes, DPM codes, etc. More generally, a barcode can be defined as an optical, machine-readable representation of data; the data often describes something related to the object bearing the barcode. Conventional barcodes represent data by varying the widths and spacing of parallel lines and can therefore be described as linear or one-dimensional (1D).Later, two-dimensional (2D) versions were also developed, using rectangles, dots, hexagons, or other geometric patterns. These are called matrix codes or 2D barcodes, although no actual lines are used. Examples include DataMatrix codes, QR codes, and DPM codes, all of which have a wide range of applications in commerce and industry. Barcode reading systems generally rely on an image sensor that captures images (usually one-, two-, or three-dimensional grayscale or color images) of the object. These images are then processed by a built-in or external vision processor.The processor generally contains both processing hardware and non-volatile, machine-readable program instructions that execute one or more vision system processes to generate a desired output based on the processed image information. This image information is typically provided within an array of image pixels, each with different colors and / or intensities. In the example of a barcode reader (also referred to here as a "camera"), the user or automated process captures an image of an object, assuming that it contains one or more barcodes. The image is processed to recognize barcode features, which are then decoded by a decoding process and / or processor to extract the alphanumeric data represented in the code.

[0003] In operation, a barcode reader typically serves to illuminate the scene containing one or more barcodes. This illumination may include aiming devices that project a colored dot onto the region of interest in the imaged scene, allowing the user to center the reader's image axis on the barcode within the scene. This illuminated scene is then captured by an image sensor within the camera assembly through an optical system. The sensor's pixel array is exposed, and the electronic value(s) generated for each pixel by the exposure are stored in an array of memory cells, which can be referred to as the "image" of the scene. In the context of a barcode reading application, the scene comprises an object of interest that has one or more barcodes of an appropriate type and size. The barcode(s) is / are part of the stored image.

[0004] A common application for barcode readers is tracking and sorting objects along a line (e.g., a conveyor belt) during manufacturing and logistics processes. The barcode reader, or more typically a group of readers (constellation), can be positioned across the conveyor line at appropriate viewing angles to capture any expected barcodes on the surfaces of corresponding objects as they move through the field of view. The focal length of the reader relative to the object can vary depending on its placement along the conveyor line and the size of the object. Therefore, the use of an autofocus mechanism in conjunction with the barcode reader is desirable to achieve the correct focal length.

[0005] Existing techniques for providing autofocus functionality for barcode readers involve cycling through a potentially large number of focus settings and selecting the one that offers maximum image sharpness for a chosen region of interest. For relevant background information, an autofocus method is described in Daniel Vaquero, Natasha Gelfand, Marius Tico1, Kari Pulli1 and Matthew Turk, Generalized Autofocus, Applications of Computer Vision (WACV), 2011 IEEE.

[0006] The region of interest is often selected using heuristic contrast measurements, which frequently mistake other high-contrast structures (e.g., text or graphics) for barcodes. Both contrast measurements and sharpness metrics are only useful if the focus is already nearly correct, which necessitates considering a large number of focus settings during the selection process. Furthermore, sharpness metrics are sensitive to lighting, an aspect that can also vary (e.g., during an autofocus process that also optimizes lighting). Such methods are therefore time-consuming and can also lead to focusing on the wrong area of ​​the scene. Moreover, commercially available barcode detection techniques are currently too slow for direct use during an autofocus process loop.Generally, the focus is first set on a region, and then the barcode detection and decoding methods work with a focused image of the region of interest. OVERVIEW OF THE INVENTION

[0007] The present invention overcomes the disadvantages associated with the prior art by providing a system and associated method for detecting and capturing one or more sharply focused images of one or more barcodes within the field of view of the imaging device, which captures one or more images of a scene containing the barcodes. An adjusting lens receives focus adjustment data from a focusing process and directs the received light to an image sensor. A measuring process measures the effective depth of field for barcode detection as the maximum focus error for which the one or more barcodes can be reliably detected by the focusing process. This depth-of-field measurement process can be based wholly or partially on automated systems or on manual input from (e.g.) a system user, which may involve estimates (e.g.,) based on heuristics, observations, or other calculations. A plurality of coarse-focus nominal settings of the adjusting lens allows scanning in sampling steps of a lens adjustment range corresponding to the range of permissible distances between the one or more barcodes and the image sensor, such that the step size of the sampling steps is less than a fraction of the depth of field for barcode detection. A capture process uses the image sensor to acquire a coarsely focused nominal image for at least one coarse-focus setting of the adjusting lens. A barcode detection process detects one or more barcode-like regions (which may also be a segment of a complete barcode pattern) within each coarsely focused nominal image.A fine-focus process, for each barcode-like region with a sufficiently high probability of the coarsely focused image, starting from the highest to the lowest probability, fine-tunes the focusing lens to optimize the focus of the image near a point within the barcode-like region. The fine-focus process then captures an image for decoding using this optimized, finely adjusted focusing lens.

[0008] In an exemplary embodiment, the system provides a system and associated method for detecting and acquiring one or more sharply focused images of one or more barcodes within the field of view of an imaging device with an image sensor that acquires one or more images of a scene containing the one or more barcodes. The system and associated method comprise an adjusting lens that receives focus adjustment data from a focusing process and directs the received light to the image sensor. A measuring process measures the effective depth of field for barcode detection as the maximum focus error for which the one or more barcodes can be reliably detected by the focusing process.Multiple coarse focus nominal settings of the focusing lens allow scanning in steps across a lens adjustment range corresponding to the permissible distances between the one or more barcodes and the image sensor. The step size of the scanning steps is typically smaller than a fraction of the depth of field required for barcode detection. A capture process uses the image sensor to acquire a coarsely focused nominal image for at least one coarse focus setting of the focusing lens. A barcode detection process detects one or more barcode-like regions and their respective probabilities in each coarsely focused nominal image. A fine focus process, starting from the coarse setting, fine-tunes the focusing lens for each barcode-like region with a sufficiently high probability in the coarsely focused image to optimize the image focus near a point within the barcode-like region.The fine-focus process captures an image for decoding using an optimized, finely tuned focus setting of the adjustment lens. For illustrative purposes, the barcode detection process is configured to measure the probability of each barcode-like region, with at least one of the barcode-like regions defining a probability above a predetermined level. The barcode detection process can essentially operate at the maximum frame rate at which the barcode-like regions within an image are reliably located by the image sensor. The depth of field can be measured in diopters, and the setting data defines diopters. In some embodiments, the fraction can be a value no greater than approximately half the depth of field of the barcode detection. The adjustment lens can be a high-speed liquid lens.For illustrative purposes, the fine-focusing process can be arranged to optimize image focus by capturing an image with one or more adjusted focus settings. These one or more adjusted focus settings can be arranged to maximize the image sharpness rating in a local region around a corresponding barcode-like region. A processing unit can be provided that is functionally connected to the focusing lens and the image sensor. This processing unit can execute the focusing process, the barcode detection process, and / or the capture process. For illustrative purposes, the processing unit can include a GPU (graphics processing unit) and / or an FPGA (field-programmable gate array).The focusing lens can be adjusted across coarse focus settings using an iterative search technique and / or a numerical step-based search technique. The fine focus process fine-tunes the focusing lens from the coarse setting, sequentially from highest probability to lowest probability. The barcode can be a 1D barcode and / or a 2D barcode. The coarse focus adjustment can be based on a distance measurement generated by a distance sensor relative to the imaging device. This distance sensor can be a time-of-flight (TOF) sensor, a lidar sensor, a radar sensor, an ultrasonic sensor, a stereoscopic sensor, and / or a sonar sensor.

[0009] In an exemplary embodiment, a system including a method for focusing an imaging system on one or more barcodes is provided. Coarsely focused nominal images are acquired for each of at least two coarse focus nominal settings. The coarse focus nominal settings are selected such that the range of permissible distances between the barcodes and the imaging system is scanned in each case. A barcode detector identifies barcode-like regions within each coarsely focused nominal image and estimates the probability that each of the barcode-like regions is actually a barcode.The barcode detector operates within an effective depth of field that allows a barcode to be reliably detected even in the presence of focus blur, within a range of scanned distances corresponding to the coarse focus nominal settings. For each barcode-like region with a sufficiently high probability, from highest to lowest, a fine focus procedure optimizes the focus and captures an image sharp enough to successfully decode the one or more barcodes present in that region.The fine-focus method optimizes focus by maximizing sharpness measurement in a local region around the barcode-like area. This is achieved by capturing images with one or more adjusted focus settings that are substantially close to the coarse-focus nominal setting. This coarse-focus nominal setting is used to capture the coarsely focused nominal images in which the barcode-like area is detected by the barcode detector. To illustrate, a vision system processor receives the captured, coarsely focused nominal images and activates the barcode detector. The vision system processor may include a GPU and / or an FPGA. An electronically controlled, focus-adjustable lens assembly can be used to image the coarsely focused nominal images at selected focal lengths. The lens assembly may further include a high-speed liquid lens whose focal length can be adjusted based on a diopter value.To illustrate, the operation of the system and process can be carried out according to a multiple priority levels. List of characters

[0010] The following description of the invention refers to the accompanying drawings, in which: Fig. 1 a diagram of a vision system arrangement in which a stationary or portable vision system camera is shown which, using an autofocus system and method according to an embodiment, images an object containing a barcode; Fig. 2 a flowchart of a generalized procedure for applying coarse focus and fine focus together with barcode detection and decoding according to an embodiment; Fig. 2A is a diagram showing the field of view of the camera assembly. Fig. 1 and a central target region as well as a user-defined region of interest for use in locating barcode-like regions; Fig. 3 a more detailed method for the autofocus process according to an illustrative embodiment; and Fig. 4 is an exemplary captured image of a scene containing an object with a barcode on it, showing various regions of interest with which the methods from Fig. 2 and Fig. 3 are operated. DETAILED DESCRIPTION System Overview

[0011] Fig. Figure 1 shows an arrangement 100 for reading and decoding one or more barcodes (one-dimensional and two-dimensional barcodes ( 1D - and 2D barcodes), such as DPM and QR codes and other codes with related symbol sets) 110 and 114, which are based on an exemplary object and / or other depicted scene 112 according to one embodiment. The exemplary object 112 This is thus several areas 115 , 117 and 119 shown, defining a three-dimensional structure. The arrangement 100 includes a vision system camera assembly 120 , which can be a portable or a stationary unit. A plurality of cameras can be used to image the scene in alternative embodiments, each operating the present system and method. As described in more detail below, the present system and method are capable of effectively distinguishing between barcode-like features. 110 and 114 relative to other adjacent geometric and textual data, such as the text 116to distinguish between barcodes. Thus, the present system and method effectively enable the rapid detection and decoding of one or more barcodes located within the field of view (FOV), even when other, potentially confusing features are present, which are just as quickly discarded by the process as barcode-like regions (and / or potential barcodes). The term "barcode-like region" as used here refers to all or part of an image containing features that represent all or part of a barcode of any suitable type / format. Therefore, the barcode-like region can be identified based on a recognizable section and / or fragment of the entire barcode present on the object. It should be noted that the barcode 114 on the upright surface 117 is located so that it is relative to the barcode 110 on the object surface115 exhibits a different distance in relation to the image sensor. The present system and method are also capable of effectively detecting multiple barcodes at different distances within the same overall field of view (FOV) using the autofocus techniques described here.

[0012] The camera assembly can include a variety of functional elements, such as built-in and / or separately available lighting / aiming devices. 122 , an image sensor S (which may be implemented using CMOS or another suitable technology), and an optical module O. The sensor S can define any usable format and any pixel arrangement size and can be implemented to acquire image data 124 to be delivered either in color or in grayscale form.

[0013] Regarding optics OIt can be any usable optical package, which can be either permanently installed or removable (for example, a screw-in lens base assembly). The optics O includes a focus-adjustable lens unit 130 , which physically changes the focal length of the optics to adapt to the working distance along the optical axis OA between the image plane of the sensor S and the barcode 110 to adapt to the surface it contains. The focus-adjustable lens 130Lenses can be based on many different operating principles – for example, as a mechanical, motor-driven lens that moves along the optical axis to vary the focus, or as a so-called liquid lens, which changes the shape of the lens profile based on the movement of a ferrofluid in the presence of an electromagnetic field. Liquid lenses are commercially available from various suppliers, such as the French company Varioptic. SA or the Swiss company Optotune AG The focus-adjustable lens 130 is transmitted via signals 132 controlled signals that are generated according to the respective manufacturer's specifications. These signals are typically... 132 as a variable current and / or voltage level, based (usually) on a digital value provided by the vision system process(or) 140(or by another process(or)), which are described below. The focus adjustment function can be used to adjust the size of the field of view (FOV) so that the barcode is fully displayed while maintaining sufficient resolution for the process(or) 140 enables the decoding of barcode features.

[0014] The vision system process(or) 140 The processor can be partially or entirely contained within the housing of the camera assembly or another device (for example, a connected smartphone). Alternatively, or additionally, the processor can be partially or entirely housed in a separate, standalone computing unit. 150 , such as a server, a PC, a laptop computer, a tablet or smartphone, instantiated via a suitable wired and / or wireless data connection 152 with the camera assembly 120is connected. Such a computing device 150 It can also be detachably attached to a built-in processor within the camera assembly when needed for monitoring, adjusting, or setting up the viewing system. The computing unit 150 suitable user interface components, such as a display / touchscreen 154 , a keyboard 156 and / or a mouse 158 , include. The process(or) 140, the computing device 150 and / or another device within the arrangement 100 can be connected to one or more downstream devices and / or processes that use decoded data obtained from the barcodes to perform tasks such as part recognition, logistics, object sorting, etc.

[0015] In some designs, the processor 140a graphics processing unit (GPU) or a field-programmable gate array (FPGA) which executes the desired program instructions used here.

[0016] The vision system process(or) 140 includes for the execution of the processes of the vision system arrangement 100 a plurality of functional processes. The visual system process. 140 can a vision system tool module 142 This includes one or more tools used in the detection and analysis of image features—such as edge finders, blob analyzers, calipers, alignment and pattern recognition processes or processors. The vision system process(or) 140 It also includes a barcode detection and decoding process. 144, which can be based on a commercially available or custom (e.g., software) application. The barcode detection and decoding unit locates potential barcode features within the captured image of the scene. 112 It should be noted that the vision system tools 142 These features can be used to refine and parse them in order to better locate elements that could be barcodes. Once potential barcode features have been resolved, a decoding process can then be performed within the module. 144 to be used from the read barcode(s) 110 to generate corresponding data (e.g. alphanumeric data strings).

[0017] As part of the image acquisition and barcode detection process, the camera optics can be adjusted to account for the distance between the imaged area and the sensor image plane. Since it is desirable to locate barcodes as quickly as possible to avoid overlooking barcodes that appear in the field of view only briefly and / or while in relative motion, the present focusing process attempts to... 146 , the adjustable lens focal length 130 to be adapted according to the predetermined procedure described below.

[0018] Optionally, one or more area-finding devices (e.g., a LIDAR, TOF sensor, a stereoscopic array, ultrasound and / or radar) can be used. 170 can be used to process area data 172 to a suitable functional module or modules of the vision system process or processor 140 to provide. The area data 172can the physical distance (double dashed line) 174 ) between the camera assembly 120 (and relative to the sensor image plane) and the area of ​​the imaged object 112 define this data 172 can be used to supplement the pixel-based focal length data used here.

[0019] It should be noted that the barcode detection and decoding process(or) 144 This can be based on a commercially available or custom-built high-speed barcode / symbol detector, such as the Hotbars™ barcode locator, commercially available from Cognex Corporation of Natick, MA. Such a detector is capable of rapidly locating barcode-containing areas / regions in an image, even in the presence of blurring, and sorting them according to the probability that they are actual barcodes.

[0020] In addition to the fast line-finding capabilities of the Hotbars™ barcode finding device described above, there are several other techniques that can be used in conjunction with suitable processor arrangements described here to locate and decode barcode features (e.g., 2D barcodes) more quickly: 1. The use of line detection techniques based on edge detection and Hough transform in combination with clustering to detect parallel lines in a potential code element. For background information, see Peter Bodnar and Laszlo G. Nyul, Improving Barcode Detection with Combination of Simple Detectors, Eighth International Conference on Signal Image Technology and Internet Based Systems (2012); and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes, Dissertation, PhD School in Computer Science, Department of Image Processing and Computer Graphics, Faculty of Science and Informatics, University of Szeged, Hungary (2015), available online at http: / / doktori.bibl.u-szeged.hu / 2825 / 8 / tezisfuzet.pdf; 2. The use of a histogram of Gaussian features (HoG features) in tiles of the captured image containing the element that could be the barcode, in order to detect neighboring tiles with a similar dominant orientation. For background information, see Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above). 3. The use of adaptive image thresholding to binarize the lines and spaces in the mapped barcode and to map morphology in order to merge them with the barcode regions. For background information, see Daw-Tung Lin, Chin-Lin Lin, Automatic Location for Multi-Symbology and Multiple 1D and 2D Barcodes, Journal of Marine Science and Technology, Vol. 21, No. 6, pp. 663–668 (2013); Savreet Kaur and Raman Maini, Implementation of Barcode Localization Technique using Morphological Operations, International Journal of Computer Applications (0975-8887), Vol. 97, No. 13, July 2014; and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above). 4. The use of deep learning for texture classification. For background information, see Hansen, Daniel Kold; Nasrollahi, Kamal; Rasmussen, Christoffer B0gelund; Moeslund, Thomas B., Real-Time Barcode Detection and Classification Using Deep Learning, published in the Proceedings of the 9th International Joint Conference on Computational Intelligence, Volume 1: IJCCI (2017); and Peter Bodnar, Image Analysis Methods for Localization of Visual Codes (cited above).

[0021] Further techniques for faster detection and decoding of selected types of barcodes under certain conditions are described by way of example in Yuhua Gu, Weixiang Zhang, QR Code Recognition Based On Image Processing, International Conference on Information Science and Technology, IEEE, published on May 10, 2011. General autofocus operating procedure

[0022] Fig. Figure 2 shows a general procedure for performing an autofocus procedure. 200 According to one embodiment. The user, or another process, triggers an action upon the presence of a barcode (or, more generally, upon the presence of a "barcode-like region", this term being used, where appropriate, as a synonym for "barcode") in a scene. 210 an image capture process. 200 then turns to step 220 A coarse focus adjustment is performed: This adjustment instructs the lens to assume a predetermined focal length (usually expressed as a focus / diopter value). Then, in step... 222The sensor captures at least one image with this selected focal length. The system then searches for potential barcode features using suitable vision system tools, and if no barcode features are found and / or the found features have a predetermined confidence score / probability (decision step) 224 If the requirements are not met, then the next coarse focus value (focus plane, described below) is selected (step 226 ) and the adjustment will be made in step 220 applied. Another image is captured and the barcode localization / identification process (decision step) begins. 224The process is repeated until a barcode is found (or until the scanning of all coarse focus values ​​is complete). If there is a high confidence and / or probability of presence (confidence value / rating above a predetermined threshold) that a barcode is present in a region of interest (via step 1), the process is repeated. 224 ), applies the procedure 200 in step 230A fine focus adjustment procedure is applied to the lens. In one embodiment, described in more detail below, an initial set of nominal focus planes can be determined, and a coarse focus process scans these planes sequentially until a barcode for a region has been identified with sufficient confidence / probability. Furthermore, the procedure can prioritize regions to be scanned based on barcode probability or other metrics—for example, user-defined metrics relating to regions with a high barcode probability within the captured image—and coarse focus is performed on regions based on such priorities, advancing between focus planes until either the scanning of all is complete or a barcode is found with sufficient confidence.

[0023] In step 232An image is captured using this fine-tuning value. The barcode detection and decoding process then determines whether the barcode(s) located during the coarse focus step is / are decoded to provide the corresponding information (decision step). 234 If not, the next step will be... 236 A new fine focus value was selected and applied to the lens (step 230 ) and then the data collection step 232 and the determination step 234 This process is repeated until a barcode is decoded or the scanning of all fine values ​​is complete. The decoded barcode results / data (if any are available) are then displayed in step [number]. 240 (about the decision step) 234 ) transferred to an appropriate, subordinate process. In the exemplary field of vision 250 in Fig. 2A, to which reference is now made, can, if no region containing a decodable barcode or a barcode-like region (or a fragment thereof) is found during the coarse focus process, use the middle quarter (approximately the outlined box). 260 ) of the field of vision 250 , or a region of interest entirely selected by the user (for example, the dashed box) 270 ) is used as the target region for a subsequent barcode search. This process then switches to a traditional contrast / sharpness-based search algorithm / method to find one or more barcodes within the new region. 260 or 270 to locate.

[0024] It should be noted, as described generally above and explained in more detail below, that each captured image of the scene may contain several regions of interest, which may contain features that require decoding. For example, Fig. 4 a captured image scene 400 an underlying object in which at least three regions of interest 410 , 420 and 430 The images show regions containing a 2D barcode, a 1D barcode, and text. The text can be excluded from further processing once it has been ensured that it does not contain any barcode features. The boundaries of each region can vary and may have a default setting or be sized to include the barcode-like regions (barcodes and / or fragments thereof) within it.

[0025] Advantageously, during operation, a small number of captured images at various coarse diopter settings across the entire possible focus range of the imaging system (called nominal focus planes) are used. These allow the barcode detector to detect all barcode-like regions in the scene, regardless of distance, by defining a relatively large depth of field for the reading optics. Such high-depth-of-field optics are described, as relevant background information, in U.S. Patent Application No. 15 / 844,448, entitled "DUAL-IMAGING VISION SYSTEM CAMERA AND METHOD FOR USING THE SAME," filed on December 15, 2017. The process of capturing images at these nominal focus planes is particularly efficient when combined with the use of a high-speed adjusting lens assembly capable of rapidly responding to changes in diopter value—similar to a liquid lens.

[0026] As described below, the procedure 200 also an optional distance measurement (e.g. the area data) 172 ) of available sensors (e.g., TOF sensor or other area detection device) 170 ) to narrow the coarse focus area so that the initial value and the incremental change in value lie within and around a captured area.

[0027] In some embodiments, a fine-focus step is used to refine the coarse-focus diopter setting for each barcode-like region. An image is captured with this setting, and then the decoding process attempts to decode the barcode. Barcode-like areas are processed in order of the highest probability of barcode features being present. This arrangement essentially represents an intelligent mechanism for visual attention (both for localization within the field of view and for distance within the focal length range of the imaging system or camera assembly). Autofocus process

[0028] A more detailed flowchart of an autofocus process 300 is in Fig. 3 shown. The procedure 300 receives input data, including the total distance range 310the camera or imaging system, optional distance measurements 312 from an area-finding sensor that offers coarse search modes - i.e., the technique(s) to be used when searching for barcode-like regions 314 and when searching for whether the coarse search should look beyond a given distance defined by the optional distance measurements (closer and / or further away), and (optionally) whether any distance measurements beyond the available distance measurements should be used. 316 Other distances (closer to and / or further away from the image sensor) should be checked. It should be noted that the input elements 312 , 314 and 316 in the overall process 300 They can be optional and therefore in Fig.3 are indicated as such (i.e., "OPTIONAL"). Other optional procedure steps / inputs are also indicated as described below. Such optional inputs, steps, and processes can be omitted from the overall procedure and / or selected by the user during setup or at runtime. More specifically, the coarse search mode input determines 314 The priorities of the various coarse plane distances are described below. The use of priorities when processing coarse planes is generally optional, as described below. The "Look Beyond Distance Measurements" input 316 can decide to consider the distance measurements (if available) as precise and not search for barcodes elsewhere. Alternatively, the input can 316decide to use the distance measurements (if any) as clues and potentially search for barcodes at these locations first, but then, if the process has not successfully decoded the required barcodes, search for barcodes at other distances within the area.

[0029] The data of the elements 310-316 are used, or can be used, to calculate a list of coarse focus planes that are used in the coarse focus procedure (step 320 ) are used. After calculating the coarse focus planes, the following steps are then taken. 324 A distance is taken from the coarse level list. This starts with an initial level value, which is a default value (e.g., an average distance within the level range and / or the distance measured by the rangefinder (step)). 324)) can act. Alternatively, the distance measurement can be based on (or contain) a predetermined distance distribution, according to a suitable heuristic that operates within the specified working distance of the sensor. It should be noted that this part of the procedure 300 can be arranged in priority levels, with the step 324 a distance is taken that corresponds to the current priority level. If no priority level is provided, only a single set of focus planes is used. The camera optics are then focused to the diopter value associated with the current (priority level) focus plane, and in step 326 at least one image is captured. The process uses the captured image(s) to determine (decision step). 330), whether and, if so, where any barcodes are present in the image, or whether there is a probability that they are present (step 332 If this is the case, these barcodes / probabilities are saved (added to the region list) in addition to their respective region(s) and their corresponding focal length value(s). It should be noted that barcode detection can be based on an automatic detection function or can take place outside of regions of interest within the overall image (input). 336 ). Then, in step 334 one or more focus regions are calculated and appended together with the overall probabilities for regions of interest (input) 338 It should be noted that the input and designation of regions of interest 338 and 334 in the overall process 300These are optional. Such optional regions of interest can be used when there is prior knowledge about a possible or approximate location of the barcode(s) in the image – for example, based on their location in the previously captured image(s). It should also be noted that the 'Auto-detect' input 336 and process steps 330 , 332 are optional. The optional 'Automatically locate' process step 330 , 332 and 338 This can be used if the method is able to accurately define the regions of interest (if any) and therefore does not search for barcodes in other locations outside these regions. 'Automatic detection' is not used (decision step). 330 ), so the regions of interest (if any exist) can only be considered as indicators and the procedure can 300Initially, the process searches for barcodes in these regions, but the procedure then searches 300 , if the required barcodes are not decoded, then search within the regions of interest across (e.g.) the entire captured image for barcode-like regions.

[0030] The procedure 300 It then checks whether all plane distances have been sampled (decision step). 340 Optionally, this query can be restricted to the current priority level if the procedure requires it. 300 Multiple priority levels are provided. If multiple plane distances are to be sampled in the coarse focus process (optionally within the current priority level, if one is provided), the process reverses. 300 to step 324 back and takes the next incremental distance. The rough finding process (steps) 324 , 326 , 330 , 332 and334 ) is then discussed in the decision step 340 repeated. Conversely, when the sampling of the distances (optionally at the current priority level) is completed, the decision step branches. 340 to the step 350 , in which the region of the overall coarse image with the highest probability is selected (and removed) from the previously saved list of image regions. The camera lens is then focused on the best coarse focus for this region, and in step 1, the following steps are taken: 354 one or more images are captured. At this stage, the fine-focusing process (dashed box) begins. 358 ) operated, with the fineness rating in step 356is calculated. The calculation can include sharpness measurements (a) a gradient measurement, (b) a frequency domain measurement, (c) an autocorrelation measurement, (d) a variance measurement, (e) a statistical measurement, and / or (f) an edge-based measurement. For background information, see Yi Yao, Besma Abidi, Narjes Doggaz, and Mongi Abidi, Evaluation of Sharpness Measures and Search Algorithms for the Auto-Focusing of High Magnification Images, SPIE - The International Society for Optical Engineering Proceedings 6246, June 2006. If the focus generates a sharpness rating that may not be sufficient (below a predetermined threshold) to produce a readable barcode (decision step). 360 ), so the next fine focus setting is incremented (step 362 ) and the lens is refocused, capturing one or more new images (step 354) and then a new spiciness rating is calculated (step 360 ).

[0031] If a sufficient severity rating is generated (decision step) 360 ), an attempt is made to decode the barcode features within the region through a corresponding decoding process (step 370It should be noted that, in addition to using regions of interest, the coarse and / or fine focus process (barcode search) can be performed using captured full-size images or partially scanned images. If partially scanned images are used for the focus process, a full-size image should be captured at the desired focus plane for decoding. Once the required number of barcodes has been correctly located / decoded and the results have been provided to the system (and no further barcodes are present in the image to be decoded), the focus and decode process is complete (step 1). 380 about decision step 372 Otherwise, the procedure branches off to the decision step. 390 to determine if any further stored regions remain. If so, the fine-focusing process is initiated. 358repeated after the next region with the highest probability has been sampled (step 350 Once all regions have been processed, the decision step branches to the next decision step. 392 , to determine whether further coarse distances exist. If this is not the case, the procedure 300 completed (step 380 If a further coarse distance exists (for example, a depicted object where multiple barcodes with different distances need to be read, but not all distances have been processed yet), then the procedure branches. 300 to step 394, in which the priority level (if one is used) is incremented and the original region list is deleted. This process can, for example, include reading coarse focus planes with lower priority levels from a distance measurement by rangefinders, such as focus planes that are outside the specified area defined around a distance measurement input from a range sensor (and / or another heuristic). In this procedure 300 The gap will be increased again at the current priority level in step 324 The sample is removed and the coarse and fine focusing processes are repeated as described above until all coarse distances have been processed.

[0032] It should be noted that the provision of optional priority levels is carried out according to the steps 324 , 340 and 394 is optional, which makes the procedure 300is able to establish a priority order for the coarse levels / distances, which determines how quickly certain types of codes can be located, focused on, and decoded. In a basic implementation, no priority is provided. In this scenario, in step 324 The next rough level is selected, and the decision step 340 The system merely queries whether further coarse levels / distances need to be processed, and the step can be completed. 394 omitted, whereby the procedural branch depends on the decision step 392 directly to the step 324This leads back to the previous step. Priority levels can be used, assigning a priority level to each distance (based on the coarse search mode and / or the distance distribution provided by the system or the user, as described above). Regions at a given distance or level are only processed after regions at higher priority levels have been processed.

[0033] As part of the decoding process, the procedure can 300Use one or more fine-tuning algorithms. Examples, without being limiting, may include: (a) a fixed-step size search, (b) a fixed-step size search in interpolation, (c) an iterative search, (d) a variable-step size search, and (e) another usable iterative and / or numerically step-based search technique. For background information, see E.P. Krotkov, Active Computer Vision by Cooperative Focus and Stereo, New York, Springer-Verlag (1989). Concluding remarks

[0034] It is understood that the system and method described above effectively detect and decode barcodes of various sizes and / or types within a wide range of working distances commonly encountered when using a portable barcode reader. The system and method effectively utilize the advantageous operating characteristics of high-speed focusing lenses (e.g., liquid lenses) and technically advanced processors, such as the Hotbars™ barcode detection algorithm and similar approaches or software. This system and method thus represent advancements and advantages over conventional barcode localization techniques, which were generally too slow for use in a focusing loop. By additionally instantiating the exemplary system and method using (e.g.,) FPGA hardware (which executes the program commands described here using hardware / firmware) allows for further improvement and acceleration of barcode detection within the focus loop of the imaging device.

[0035] The foregoing is a detailed description of exemplary embodiments of the invention. Various modifications and additions can be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above can, where appropriate, be combined with features of other described embodiments to provide a multitude of feature combinations in associated new embodiments. Furthermore, while the foregoing describes a number of separate embodiments of the device and method of the present invention, what is described here is merely illustrative of the application of the principles of the present invention. For example, it is noted that images must be acquired at all coarse focus settings before any decoding is attempted.This means that in various exemplary embodiments, it is considered that images can be acquired for a subset of (one or more) coarse focus settings and that this subset can be processed until one or more barcodes are located and decoded. Furthermore, the terms "process" and / or "processor" used herein should be interpreted broadly to encompass a variety of electronic hardware- and / or software-based functions and components (and may alternatively be referred to as functional "modules" or "elements"). Additionally, a described process or processor may be combined with other processes and / or processors or subdivided into various subprocesses or subprocessors. Such subprocesses and / or subprocessors may be combined with one another in various ways according to the embodiments described herein.It is also expressly considered that each function, process, and / or processor herein may be implemented using electronic hardware, software consisting of a non-volatile, computer-readable medium of program instructions, or a combination of hardware and software. Furthermore, various directional and arrangement-related terms used herein, such as "vertical," "horizontal," "upward," "downward," "lower," "upper," "lateral," "front," "rear," "left," "right," and the like, are used only as relative references and not as absolute directions / arrangements with respect to a fixed coordinate space, such as the direction of gravity.Furthermore, the terms "essentially" or "approximately," when applied to a specified measure, value, or property, are intended to refer to a quantity that lies within a normal operating range for achieving desired results, although fluctuations due to inherent inaccuracies and errors within the permissible tolerances of the system (e.g., 1-5 percent) are possible. Accordingly, this description is to be considered merely exemplary and is not intended to otherwise limit the scope of protection of the present invention.

Claims

[] Claimed is: [1] System for detecting and capturing one or more sharply focused images of one or more barcodes within the field of view of an imaging device with an image sensor which captures one or more images of a scene containing the one or more barcodes, comprising: an adjustable lens that receives focus adjustment data from a focusing process and directs the received light to the image sensor; a measurement process that measures an effective depth of field for barcode detection as the maximum focus error for which one or more barcodes can be reliably detected by the focus process; a plurality of coarse focus nominal settings of the adjusting lens, which allows scanning in sampling steps of a lens adjustment range corresponding to the range of permissible distances between the one or more barcodes and the image sensor, so that a step size of the sampling steps is less than a fraction of the depth of field of the barcode detection, a capture process that uses the image sensor to capture a roughly focused nominal image for at least one coarse focus setting of the adjusting lens; a barcode detection process that detects one or more barcode-like regions in each coarsely focused nominal image; a fine-focus process which, starting from the coarse setting, performs a fine adjustment of the adjusting lens on at least one of the barcode-like regions with a coarsely focused image in order to optimize the focus of the image near a point of the barcode-like regions, and captures an image for decoding with an optimized, finely adjusted focus setting of the adjusting lens. [2] System according to claim 1, wherein the barcode detection process is arranged to measure a respective probability of the barcode-like regions, and wherein the at least one of the barcode-like regions defines a probability above a predetermined degree. [3] System according to claim 1, wherein the barcode detection process is essentially operated at the maximum frame rate at which the barcode-like regions within an image are reliably located by the image sensor. [4] System according to claim 3, wherein the fraction is not greater than approximately half the depth of field of the barcode detection. [5] System according to claim 1, wherein (a) the adjusting lens comprises a high-speed liquid lens and / or (b) the depth of field is measured in diopters and the adjustment data define diopters. [6] System according to claim 5, wherein the fine focus process is arranged to optimize the focus of the image by capturing an image with one or more adapted focus settings and selecting the one or more adapted focus settings by which an image sharpness rating is / are maximized in a local region around the corresponding barcode-like region(s). [7] System according to claim 1, further comprising a processing unit in functional connection with the adjusting lens and with the image sensor, which allows the focusing process and / or the barcode detection process and / or the capture process to take place. [8] System according to claim 7, wherein the processing unit comprises a GPU and / or FPGA. [9] System according to claim 1, wherein the adjusting lens is adjusted across the coarse focus settings based on an iterative search technique based on numerical steps. [10] System according to claim 1, wherein the fine focusing process, starting from the coarse adjustment, performs a fine adjustment of the adjusting lens, successively from a highest probability to a lowest probability. [11] System according to claim 1, wherein the barcode comprises a 1D barcode and / or a 2D barcode. [12] System according to claim 1, wherein the coarse focus adjustment is based on a distance measurement generated by a distance sensor relative to the imaging device. [13] System according to claim 12, wherein the distance sensor comprises a TOF sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor and / or a stereoscopic sensor and / or a sonar sensor. [14] Method for focusing an imaging system on one or more barcodes, the method comprising the steps of: coarsely focused nominal images are captured for each of at least two coarse focus nominal settings, wherein the coarse focus nominal settings are chosen such that the range of permissible distances between the corresponding barcodes and the imaging system is scanned in each case; a barcode detector is used to determine barcode-like regions within each of the coarsely focused nominal images and to estimate a corresponding probability that each of the locations of the barcode-like regions actually displays barcodes, the barcode detector being operated within an effective depth of field which allows a barcode to be reliably detected in a range of distances between the sampled distances corresponding to the coarse focus nominal settings, even in the presence of focus blur; and For each of the barcode-like regions, with a sufficiently high probability, from highest to lowest, a fine-focus procedure is used to optimize the focus and capture an image that is sufficiently sharp to successfully decode the one or more barcodes on the corresponding barcode-like regions. The fine-focus procedure optimizes the focus by maximizing a sharpness measurement in a local region around a corresponding barcode-like region by capturing images with one or more adjusted focus settings that are substantially close to the coarse-focus nominal setting used to capture the coarsely focused nominal images in which the respective barcode-like region is detected by the barcode detector. [15] Method according to claim 14, further comprising that the captured, coarsely focused nominal images are received at a vision system processor and the barcode detector is put into operation with the vision system processor. [16] Method according to claim 15, wherein the receiving step at the vision system processor comprises receiving at a processor which includes a GPU and / or an FPGA. [17] Method according to claim 15, further comprising an electronically controlled, focus-adjustable lens assembly which images the coarsely focused nominal images at selected focal lengths. [18] Method according to claim 17, wherein the lens comprises a high-speed liquid lens in which the focal length can be adjusted based on a diopter value. [19] System according to claim 17, further comprising that the focus-adjustable lens assembly is adjusted across the coarse focus settings on the basis of an iterative search technique based on numerical steps. [20] Method according to claim 14, wherein the steps of acquiring, using and utilizing are carried out according to a plurality of priority levels.