System and method for tracking optical codes

Abstract

The present invention provides a computerized method and system for locating a barcode applied to an object. The method may receive a first image of a first barcode segment applied to a first object captured at a first time and identify a first position of the first barcode segment. The method also receives a second image of a second barcode segment captured at a second time and identifies a second position of the second barcode segment. The method may also predict a range of possible positions of the first barcode segment at the second time based on a tracking model, wherein the tracking model tracks the first barcode segment based on the first position and determines that if the second position is within the range of possible positions of the first barcode segment at the second time, then the first barcode segment and the second barcode segment correspond to the same barcode.

Description

[0001] This application is a divisional application of the patent application with application number "201510670126.4", application date "October 9, 2015", and title "System and method for tracking optical codes".

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] The present invention relates to a U.S. application filed on October 9, 2014, with application number TBD, applicants Ivan A. Bachelder and Savithri Vaidyanathan, and invention name “BARCODE DECODING”, the contents of which are incorporated into the present application by reference. Technical Field

[0004] The present invention relates to tracking a target and, more particularly, to predicting the range of possible future positions of an optical code. Background Art

[0005] There are many applications for automatic identification and tracking of targets, for example in products using optical codes. An optical code is a pattern of elements with different light reflections or light emissions assembled according to some predetermined rules. Known optical codes are linear bar codes used in different consumer products. Linear bar codes include bars or spaces in a linear manner. Optical codes can also be two-dimensional. Linear and two-dimensional bar codes are generally referred to as symbols. Two-dimensional optical codes can include open / closed patterns in two-dimensional matrix codes. The bar code or symbol can be printed on a label placed on the product packaging or directly on the product itself.

[0006] The information encoded in the barcode or symbol can be decoded by using an optical reader in a fixed frame device or a portable handheld device. For example, in the case of a fixed frame device, a transmission line moves a target marked with a code or symbol within the range of a fixed frame reader, which can produce an image of the code or symbol. Image-based reader devices typically include at least one camera capable of producing a two-dimensional image of a field of view (FOV). For example, some systems currently use a two-dimensional charge coupled device (CCD) image sensor that acquires an image that is subsequently received by a processor. The processor is programmed to examine image data to identify code candidates (such as barcode candidates or symbol candidates) and decode those code candidates. The reader device can be programmed to acquire an image of the field of view (FOV) in rapid succession and decode any acquired code candidate as quickly as possible. The processor runs one or more decoding algorithms to decode the code candidate.

[0007] In a fixed frame device, the optical reader may acquire a large number of images of the same target and application code. The target and application code may be in different positions along the direction through the FOV. In addition, some of the targets and application codes in the image are new to the FOV, e.g., those that were located outside the FOV during a previous image, and other targets and application codes leave the FOV before a subsequent image is generated.

[0008] When an image of a code is obtained, the quality of the image depends on several factors, such as the angle of the reader relative to the surface to which the code is applied, the material and texture of the surface to which the code is applied, the quality of the code marking or damage after marking, environmental and device lighting characteristics, distortions in the applied symbol, transmission line speed, distance from the surface to which the code is applied, optical blur, camera resolution, sensor noise, motion blur (partial motion during sensor exposure), etc. The image quality affects the ability of the processor to run a specific algorithm to decode the code. For example, in many cases, a single decoding algorithm cannot successfully decode the code in the image unless the environment surrounding the image acquisition is essentially ideal. In addition, the image may not include the entire code, but only a fragment of it. Summary of the invention

[0009] Multiple images of the code are necessary to reconstruct the entire code. To compensate for incomplete and incomplete image acquisition of the code, tracking algorithms can help locate where the code is in subsequent captured images. After different fragments of the same code are identified in subsequent images, stitching algorithms can be used to stitch the different fragments to produce the entire code.

[0010] In some aspects, a method for locating a barcode applied to a target is provided. The method includes receiving, by a processor, a first image of a first barcode segment applied to a first target captured by a camera at a first time, and identifying, by the processor, a first position of the first barcode segment based on the first image. The method also includes receiving, by the processor, a second image of a second barcode segment captured by the camera at a second time, and identifying, by the processor, a second position of the second barcode segment based on the second image. The method includes predicting, by the processor, a range of possible positions of the first barcode segment at the second time based on a tracking model, wherein the tracking model tracks the first barcode segment based on the first position, and determining, by the processor, that if the second position is within the range of possible positions of the first barcode segment at the second time, then the first barcode segment and the second barcode segment correspond to the same barcode applied to the first target.

[0011] In some aspects, a method for locating a barcode applied to an object in a system configured to extend a field of view of a scene imaged by a camera having an image sensor is provided, the method comprising: receiving a first image captured by the camera at a first time, the first image comprising a first field of view and a second field of view, wherein the first field of view of the first image depicts a first barcode segment and the second field of view of the first image depicts a second barcode segment; identifying a first position of the first barcode segment within the first field of view of the first image based on the first image; identifying a first position of the second barcode segment within the second field of view of the first image based on the first image. The method may include: receiving a second image captured by the camera at a second time, the second image comprising a first field of view and a second field of view, wherein the first field of view of the second image depicts a third barcode segment and the second field of view of the second image depicts a fourth barcode segment; identifying a second position of the third barcode segment within the first field of view of the second image based on the second image; identifying a second position of the fourth barcode segment within the second field of view of the second image based on the second image. The method includes: predicting a range of possible positions of a first barcode fragment at a second time based on a first tracking model, wherein the first tracking model tracks the first barcode fragment based on the first position of the first barcode fragment; predicting a range of possible positions of a second barcode fragment at the second time based on a second tracking model, wherein the second tracking model tracks the second barcode fragment based on the first position of the second barcode fragment; determining whether the second position of the third barcode fragment is within the range of possible positions of the first barcode fragment at the second time; determining whether the second position of the fourth barcode fragment is within the range of possible positions of the second barcode fragment at the second time; and if the second position of the third barcode fragment is within the range of possible positions of the first barcode fragment at the second time and the second position of the fourth barcode fragment is within the range of possible positions of the second barcode fragment at the second time, determining that the first barcode fragment, the second barcode fragment, the third barcode fragment and the fourth barcode fragment correspond to the same barcode.

[0012] In some aspects, a system for locating a barcode applied to a target is provided. The system includes: a camera configured to capture an image of a barcode applied to a target; and a processor in communication with the camera. The processor is configured to: receive a first image of a first barcode segment applied to a first target captured by the camera at a first time; identify a first position of the first barcode segment based on the first image; receive a second image of a second barcode segment captured by the camera at a second time; identify a second position of the second barcode segment based on the second image; predict a range of possible positions of the first barcode segment at the second time based on a tracking model, wherein the tracking model tracks the first barcode segment based on the first position; and if the second position is within the range of possible positions of the first barcode segment at the second time, determine that the first barcode segment and the second barcode segment correspond to the same barcode applied to the first target.

[0013] In some aspects, a system for locating a barcode applied to a target is provided. The system is configured to extend the field of view of a scene imaged by a camera having an image sensor. The system includes: a camera configured to capture an image of a barcode applied to a target; and a processor in communication with the camera. The processor is configured to: receive a first image captured by the camera at a first time, the first image including a first field of view and a second field of view, wherein the first field of view of the first image depicts a first barcode segment and the second field of view of the first image depicts a second barcode segment; identify a first position of the first barcode segment within the first field of view of the first image based on the first image; identify a first position of the second barcode segment within the second field of view of the first image based on the first image; receive a second image captured by the camera at a second time, the second image including a first field of view and a second field of view, wherein the first field of view of the second image depicts a third barcode segment and the second field of view of the second image depicts a fourth barcode segment; identify a second position of the third barcode segment within the first field of view of the second image based on the second image; identify a second position of the fourth barcode segment within the second field of view of the second image based on the second image. The processor is configured to: predict a range of possible positions of the first barcode segment at the second time based on a first tracking model, wherein the first tracking model tracks the first barcode segment based on the first position of the first barcode segment; predict a range of possible positions of the second barcode segment at the second time based on a second tracking model, wherein the second tracking model tracks the second barcode segment based on the first position of the second barcode segment. The processor is configured to: determine whether the second position of the third barcode segment is within the range of possible positions of the first barcode segment at the second time; determine whether the second position of the fourth barcode segment is within the range of possible positions of the second barcode segment at the second time; and if the second position of the third barcode segment is within the range of possible positions of the first barcode segment at the second time and the second position of the fourth barcode segment is within the range of possible positions of the second barcode segment at the second time, determine that the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment correspond to the same barcode.

[0014] These and other embodiments will be described in more detail in the remainder of the specification, with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An exemplary image processing system according to some embodiments is shown;

[0016] Figure 2illustrates an exemplary adaptive tracking model according to some embodiments;

[0017] Figure 3 An exemplary adaptive tracking model is shown in accordance with some embodiments. DETAILED DESCRIPTION

[0018] Various aspects of the present invention will now be described with reference to the accompanying drawings, wherein the same reference numerals are used throughout the various drawings to correspond to similar elements. It should be understood, however, that the drawings and detailed descriptions hereinafter referred to are not intended to limit the claimed subject matter to the specific form disclosed. On the contrary, the intent is to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the claimed subject matter.

[0019] The present invention relates to systems and methods for tracking codes in optical reader systems. According to aspects of the present disclosure, a tracking model predicts a possible range, and sometimes a relative likelihood, of the pose of an optical code or group of optical codes based on at least one or more previously measured poses of the same optical code or group. This predictive capability of the tracking model constrains the range of possible correspondences or matches between a moving optical code and an optical code identified at a subsequent time or at a different spatial reference frame. More specifically, the tracking model can constrain the range of possible correspondences between a group of interest and a candidate optical code group identified at a subsequent time or at a different spatial reference frame. According to optional aspects of the present disclosure, in order to simultaneously track one or more distinct, but potentially similar optical codes or groups of optical codes, the tracking model can prioritize possible correspondences by likelihood.

[0020] A gesture may include the position, orientation, and / or apparent size (scale) of a light code or group. The range of gestures may be indicated by a single predicted gesture associated with some uncertainty, e.g., a position other than an expected error of that position described by a distribution (e.g., a normal distribution). The range of gestures may alternatively be indicated simply by a set or trajectory of possible gestures (e.g., by a line segment or circular area representing possible positions) using a joint probability distribution (e.g., a uniform distribution).

[0021] refer to Figure 1 , various aspects of the present invention will be described in the context of an exemplary imaging system / transmission line 10, wherein a transmission line 30 carries moving targets 26a, 26b, 26c, etc. along a travel direction 25. It should be understood by those skilled in the art that the imaging system 10 is an exemplary application of the present invention and that the disclosed systems and methods for tracking optical codes on targets may be applied in other applications.

[0022] In the present example, each target has similar physical characteristics, and therefore only one target, such as target 26b, will be described in detail. Specifically, target 26 includes a surface 27 that faces generally upward when target 26b is moved by transmission line 30. A matrix code 24a is applied to top surface 27 for identification purposes. A similar type of matrix code 24a is applied to the top surface of each target 26a and 26c.

[0023] refer to Figure 1 , the image processing system includes a camera 22 having an optics 24 that defines a field of view 28 through which a transmission line 30 moves the targets 26a, 26b, 26c, etc., and below the camera 22. Thus, as the targets move along the travel direction 25, each top surface 27 enters the field of view 28. The field of view 28 may be large enough so that the entire top surface 27 is within the field of view 28 at one point or another, and thus any code applied to the top surface 27 of the target passes through the field of view 28 and can be captured in the image by the camera 22. As the target moves along the travel direction 25, the camera 22 may capture partial fragments of the code applied to the top surface 27. It will be appreciated by those skilled in the art that the field of view may be large enough so that it can capture more than one target. In addition, the image processing system 10 may include more than one camera and / or a camera with a field of view expander, which is described in US Pat. No. 8,646,690, entitled "System and Method for Expansion of Field View in a Vision System", by Nunnink et al., the contents of which are incorporated by reference into the present application. US Pat. No. 8,646,690, for example, provides details of the field of view expander at column 8, line 43 to column 9, line 28 and Figures 5-7.

[0024] The image processing system 10 also includes a computer or processor 14 (or multiple processors) that receives the image from the camera 22, examines the image to identify sub-portions of the image that may include instances of a matrix code that are code candidates, and then attempts to decode each code candidate in an effort to identify the target currently located within the field of view 28. To this end, the camera 22 is linked to the processor 14. The interface device 16 / 18 may also be linked to the processor 14 to provide visual and audio output to a system user, as well as to provide input to the user to control the imaging system, set imaging system operating parameters, troubleshoot imaging system problems, etc. In at least some embodiments, the imaging system also includes a tachometer (encoder) 33 positioned proximate to the transmission line 30, which can be used to identify the direction of travel 25 and / or the speed at which the transmission line 30 is transporting the target through the field of view.

[0025] Figure 2 An exemplary method 200 for predicting a light code gesture or group gesture based on an adaptive tracking model according to aspects of the present invention is shown. Specifically, the process starts at 202, and the range of the next possible gesture is predicted by determining an initial tracking model 204. The method determines the initial light code feature or light code group feature and gesture 206 and predicts the range of the possible next tracked light code (or group) gesture 208. Details on how the method predicts the range of the possible next light code gesture are provided below. The method then identifies a candidate light code with a gesture within the prediction range 210 and, for example, determines a new light code gesture by matching the features of the candidate light code to the features 212 of the tracked light code. The method can then optionally modify the tracking model 214. The method can be iteratively repeated to determine the subsequent range of gestures for a specific light code or group.

[0026] According to aspects of the invention, the disclosed systems and methods can track a single physical linear (barcode) or 2D identification code / symbol, or a collective arrangement of such symbols within a field of view or one or more images. The disclosed systems and methods can track optical codes based on at least one of: direction of motion (in image coordinates or physical coordinates), optical code speed (in image or physical coordinates), range of motion or movement (between images, in pixels, or in physical spaces), optical code acceleration or deceleration, smoothness of motion, rotational speed, or characteristics of a particular system.

[0027] According to aspects of the invention, system parameters may include characteristics of an optical reader system (e.g., a camera system). For example, a camera system may include multiple cameras. Optionally, a camera system may include a camera with a field of view expander, whose characteristics may include accurate or approximate boundaries between fields of view in acquired images, accurate or appropriate orientation in acquired images, and any calibration with pixel-to-pixel correspondence information. Additional camera system parameters may include relative camera positions and calibrations of cameras relative to other cameras in a multi-camera system, relative camera positions relative to a conveyor belt (e.g., orientation, viewing angle, distance, calibration, which may include pixel size related to the distance between the camera and the conveyor belt), and generally, any relationship between a conveyor plane and an image plane. Camera calibration may include a mapping of sensor imager pixels to physical spaces. Such calibration may include modeling of lenses, field of view expanders, and other optical distortions.

[0028] Other system characteristics may include information about the conveyor system, which may include, for example, conveyor belt speed (in image coordinates or physical coordinates), conveyor belt width and length.

[0029] The disclosed tracking systems and methods can track a code associated with a target based on code-related inputs. For example, as the target moves on a conveyor belt, the tracking model can track an optical code based on information about the motion of the code. The image captured by the camera can include a timestamp indicating the time when the image was captured. The image capture typically indicates the time when the sensor was exposed, but can indicate the time when the image was transferred to the memory, or another consistent event within the image capture sequence. The information can include the position and velocity of the code from the encoder reading, including the timestamp of the image of the specific code tracked by the tracking model.

[0030] Code features that can be used by the tracking model may also include invariant measurements or information relating to the identity and appearance of the code regardless of its pose. This information may include, for example, the type of code, which may be defined by symbology or code classification (e.g., two-dimensional symbology, multi-width barcode, two-width barcode). If more than one code is present in the setup, additional information may include dimensional properties of the code, such as aspect ratio, initial length, width, and any known data string information, including, for example, field length, range of character values, character characteristics, or spatial relationships to other codes.

[0031] According to aspects of the invention, additional parameters that can be used by the disclosed tracking system and method may include information about a particular code pose. This information may include, for example, the location of a center, a corner, or some identifiable point on the code in any type of coordinate (x, y) in the image. Typically, in a linear barcode, the location in the image is imprecise in the direction parallel to its bars, and is accurate within half a character in the direction perpendicular to the bars. For two-dimensional symbols, the location is typically accurate within half a module in x and y. Additional code-related information may include the direction of the code in the image, any skew or warping of the code, and the scale of the code (such as the appearance module size). The appearance module size is the size of the smallest code unit (such as a bar or a two-dimensional module) in units of pixels per module (PPM).

[0032] Figure 3An exemplary method 300 for predicting the range of possible code poses based on an adaptive tracking model according to aspects of the present invention is shown. Specifically, the process starts at 302 by determining an initial tracking model 304 to predict the range of the next possible pose and obtaining an initial field of view image 305. The method then determines the initial tracked code features and poses in the initial FOV image 306 by using, for example, a symbol candidate search algorithm. The search algorithm is described in the following documents: U.S. Patent No. 4,948,955, the name of the applicant Bodnár et al., "A Novel Method for Barcode Localization in Image Domain," the name of the applicant Zhang et al., "Automatic Real-Time Barcode Localization in Complex Scenes," the name of the applicant Kutiyanawala et al., "A Simple and Efficient Approach to Barcode Localization," the name of the applicant Juett et al., "Barcode Localization Using a Bottom Hat Filter," and the name of the applicant Kutiyanawala et al., "A simple and efficient approach to barcode localization," all of which are incorporated by reference into this application. At this point, it is not necessary to decode each candidate code, only to identify the symbol as a possible code. In addition, the disclosed method and system can track the code even if only a portion of the code is visible, as long as it is identified as a viable code.

[0033] Subsequently, the disclosed method may predict a possible next tracked code pose 308, for example, based on the currently tracked code pose, optional code motion input, optional system input, and / or a code tracking model. According to aspects of the invention, the range may optionally include a relative likelihood of poses within the predicted range. The method may then acquire the next FOV image 309, and then identify possible codes 310 in the next FOV image having poses within the predicted pose range. At this step, the method may filter out false candidates by, for example, looking for candidates with evidence of bars, processing the candidates to measure features (e.g., edges or evidence of bars that can be used for comparison with the tracked code, including type of symbology, module size, and length), and attempting to decode the candidates to additionally identify and remove false candidates and provide additional information for matching of code fragments.

[0034] After identifying the possible codes, the method may then match the features of the candidate code to the tracked code features to determine the new current pose of the tracked light code 312. Additionally, at this step, the method may disambiguate two or more matching candidate codes by using pose likelihoods, and, when character offsets are ambiguous, such as due to repeated substrings, poses may be used to disambiguate the way to match the string characters of the tracked barcode to the candidates.

[0035] The method may then adapt the code features and / or tracking model 314. For example, the method may modify the code tracking model based on the error between the predicted pose of the code and the actual pose within the predicted area. The method may modify the code features by, for example, combining decoded or partially decoded strings and / or measured code features. Finally, the method may optionally associate a tracked code in one field of view with a tracked code in another field of view, for example, by comparing the code features or by using a camera or field of view calibration. This method may be iteratively repeated to determine a subsequent range of poses for a particular code.

[0036] Example of a tracking model

[0037] The following examples of tracking models are provided for exemplary purposes only and are not limiting.For each of the following examples, a physical environment is provided and parameters of the tracking model are discussed.

[0038] According to aspects of the present disclosure, the disclosed systems and methods can track codes applied to boxes where the boxes move on a conveyor belt in the system and a single camera provides a single field of view. On each box, there are two linear bar codes in a "picket fence" direction, and the belt moves vertically relative to the camera. The picket fence direction is an orientation in which the bar code is essentially horizontal (bars vertical).

[0039] The tracking model can use the direction of motion, the minimum and maximum movement between images, and the maximum error in vertical movement (in pixels) to predict the range of possible next code poses. The tracking model can further assume that the next code pose can be located within the predicted range under uniform distribution conditions. The initial tracking model can be a tracking model in which the direction of motion is equal to 90 degrees, the minimum movement is 0 meters per second, and the maximum movement can be half the image height.

[0040] A simple prediction based on the exemplary model can identify the next position of each barcode. For example, the model can predict that the "y" position can be along a vertical line segment starting and ending at the minimum and maximum movement distances from the current position of each barcode, respectively, and that the "x" position does not exceed the maximum error from the vertical line. Possible corrections in the model can include replacing the direction of movement in the model with the actual measured angle between the last two positions or possibly a running average of the angle.

[0041] According to some aspects of the present disclosure, the disclosed systems and methods can track codes applied to boxes randomly moving on a conveyor belt, where up to two boxes may be visible at a single time. Assume that the system includes a single camera with a field of view extender and no calibration information, as described in U.S. Application No. 8,646,690, the conveyor has an directional encoder, and the conveyor belt has a constant speed and moves the boxes continuously. Furthermore, in this example, each code has an unknown direction that does not change as the box moves on the conveyor belt.

[0042] The disclosed systems and methods can use, for example, information about the encoder (conveyor position) in physical units per pixel at the conveyor and at the highest box height, the relative direction and area of ​​the field of view in the captured image, the range of possible box heights, and camera calibration information.

[0043] The tracking model can use the direction of motion, the speed of the box in the image, and information about the maximum deviation from the expected position to predict the range of possible next code poses for each tracked code in each field of view. The tracking model can also assume a Gaussian likelihood as a function of the distance from the expected position to predict the range of possible next code poses.

[0044] The initial tracking model may include a direction of motion equal to 90 degrees, a maximum deviation equal to 20 pixels, and a speed in the image equal to the average of the conveyor's speed and a maximum box height, each of which is determined by calculating the motion for each encoder unit using appropriate camera calibration.

[0045] Simple predictions based on the exemplary model can identify the next position of each code. For example, the model can predict the next position of the code by multiplying the speed times with the difference between the current encoder reading and the previous encoder reading, with a direction similar to the direction in the code feature. The likelihood of the code within the prediction range can be determined by a Gaussian likelihood function relative to the predicted position. Possible corrections to the module may include correcting the direction of the symbols in the code feature by taking a running average of the measured directions and correcting the speed by using a running average of the measured distances between the previous position and the current position divided by the encoder variation.

[0046] According to some aspects of the present disclosure, the disclosed systems and methods can track codes applied to cases and read by handheld readers or display readers (such as readers at grocery store checkouts.) In this example, each case has a single bar code with garbled text, and there are arbitrary but smooth changes in the orientation of the code relative to the camera and unknown but smoothly varying code direction and speed.

[0047] The tracking model can use discrete and independent Kalman filters for x and y velocities and angular velocity. Kalman filtering is well known in the art as a method for object tracking. Examples of using Kalman filtering for tracking can be found in Gutman, P., Velger, M., "Tracking Targets Using Adaptive Kalman Filtering," IEEE Transactions on Aerospace and Electronic Systems Vol. 26, No. 5: pp. 691-699 (1990). The initial tracking model can set the velocities in the x and y dimensions to zero, the angular velocity to zero, the variance measurement results for the velocity to a large value, and the variance measurement results for the position to a value that approximates the orientation measurement error (such as half a character width).

[0048] Simple predictions based on the exemplary model may apply a constant velocity or acceleration module to predict the next position of the code.Possible tracking model corrections may include simply updating the Kalman filter variance in a standard manner.

[0049] According to some aspects of the present disclosure, the disclosed systems and methods may include marks applied to boxes that are randomly moved on a conveyor belt. Each mark has a cluster of printed bar codes and two-dimensional symbols, where some bar codes and symbols are stacked, such as printed parallel to other bar codes and symbols. The marks can be located on the top or side of the box, and some marks are located on top of other marks. The conveyor belt has a constant speed, but can be stopped periodically. The camera is set with an oblique viewing angle. The optical system has two cameras, each of which has a field of view expander and there is no calibration information. Each mark is in an unknown independent direction, which can change smoothly as the box catches up with the side of the conveyor.

[0050] The disclosed systems and methods can use the relative direction and area of ​​the field of view in the images and the timestamp of each image to predict the possible range of the pose of the marker. The tracking model can further use as parameters the direction of motion of the marker in the captured images, the velocity of the marker in the captured images, the angular velocity of the marker, the maximum deviation from the desired position, and the maximum angular deviation. The tracking model can also assume a Gaussian likelihood as a function of the distance from the desired position and the angular deviation.

[0051] The initial tracking model may include a direction of motion equal to 90 degrees (with 15 degree uncertainty), unknown values ​​for velocity and angular velocity in the captured images, and a maximum deviation of 20 pixels. The module may also assume a spatial arrangement of the codes on the markers in each field of view (as measured in the first image).

[0052] Simple predictions based on the exemplary module can identify the next position of the center of the marker under different assumptions. If the velocity is unknown, the marker can be anywhere along the line segment between the center of the previous marker and the edge of the field of view along the direction of motion. Alternatively, if the velocity is known, the center of the marker can be located at a point on the line segment that is away from the distance determined by multiplying the velocity with the difference in the timestamp between the current image and the previous image. The model can also predict the orientation of the next marker. Specifically, if the angular velocity is unknown, the next marker can have an arbitrary orientation. If the angular velocity is known, the marker is at an angle determined by multiplying the angular velocity with the difference in the timestamp between the current image and the previous image and adding the previous angle.

[0053] The likelihood of a tag within the predicted range of the pose can be determined by a Gaussian likelihood function relative to the predicted position and angle. The predicted position of each code on the tag can be determined by using the spatial arrangement. Possible corrections that can be applied to the module may include determining the pose of the tag by adapting the spatial arrangement of the discovered code (or at least the visible code) to the original spatial arrangement, correcting the orientation of the tag in the code feature by taking a running average of the measured orientation, correcting angular velocity and linear velocity by using a running average of the measured distance between the previous position and the current position divided by the timestamp change, and correcting the spatial arrangement by averaging the measured relative position and direction of each code in the setup.

[0054] In an image-based automatic recognition system, an image-based reader may sometimes not be able to read an optical code based on a captured image. For example, some captured images may not include an optical code. The camera in a modern image-based optical code reader typically operates at a high frequency. Due to the high rate at which images are captured, some images may not include an optical code (such as an image may have a conveyor belt or box, or a barcode that enters the field of view when it enters the field of view). Naturally, in those cases, analyzing the captured image will not identify the optical code in the image. In other cases, the captured image may include an optical code that may or may not be located by an image-based system. In addition, even in some cases where an optical code has been located by an image-based system, an image-based system may not be able to decode the optical code.

[0055] According to aspects of the present disclosure, the disclosed systems and methods can identify images with optical codes in the images and can store these images. This can be done to improve image buffering. The optical code in the image can be decoded or not. In addition, the disclosed systems and methods can identify secondary images, such as images in which no optical code has been located within the image. The disclosed systems and methods can collect the saved images and determine the cause of the unsuccessful optical code reading.

[0056] Typical optical reader systems store and transmit all captured images to a processing unit for further processing, such as for analyzing the reasons why a particular optical code did not decode. However, those systems have limited bandwidth and limited storage space for images. Therefore, it is not practical to transmit all images offline to the processing unit or to store all captured images in the optical reader system memory. In addition, not all captured images may be useful for the processing unit to identify the reason for the decoding failure, so when transmitting or storing those images, important optical reader resources can end up wasting. The disclosed systems and methods can identify only a subset of the captured images, such as to analyze why a particular code failed to decode.

[0057] According to some aspects of the disclosure, an optical reader may have within its field of view a target with an optical code applied to the target. Specifically, the optical reader may capture multiple images or frames of the target. Through the entire set of captured images, the disclosed system may first identify images with optical codes and separate them from other images without optical codes.

[0058] The disclosed system and method can attempt to decode an image including an optical code. The disclosed method can successfully decode some images including the code. Other images including the code but not successfully decoded can be used to analyze the cause of the decoding failure. According to some aspects disclosed, images with codes but not successfully decoded can be divided into two groups. The first group can include the most interesting images, and the second group includes redundant images. The images of the first group and the second group can be stored in different memory locations. The most interesting images can be stored in, for example, an internal memory or transmitted to a remote location, for example, via the Internet or a local area network. The memory location in which the redundant images of the second group are stored can be reused to store other images.

[0059] In a typical application, for each code that is identified but not successfully decoded, the optical reader is configured to identify a single image of most interest. However, a single image may not provide enough information about why a particular code was not decoded. For example, one image may include a fragment of the code, while a second image may include a different fragment of the code. In addition, in addition to the actual code, there may be other information that may not be captured by a single image and may be useful to the processing system, such as an address block. According to some aspects of the disclosure, the system identifies multiple images of most interest, wherein the multiple images of most interest may be separated from redundant images based on different criteria, and the different criteria may include, for example, the distance of the barcode fragment from the center of the field of view, a portion of the symbol visible in the image, the quality of the barcode, a portion of the decodable characters, the confidence level of the decodable characters, and the image quality in the barcode area. According to aspects of the disclosure, the tracking model discussed above can be used to identify the images of most interest. For example, the tracking model can be used to identify which images correspond to the same code that was not decoded.

[0060] The image may be of the entire optical code or even a fragment of the optical code. The set of image frames of interest may include one image frame for each unsuccessfully decoded symbol. Furthermore, the images are matched to identify duplicate images having the optical code.

[0061] The disclosed system may include more than one optical reader. All optical readers may be configured to have targets within their visual range. The system may synchronize images captured from different optical readers corresponding to the same target. If there is more than one optical reader in the system, the optical readers may operate in a master / slave array. This may facilitate image synchronization, such as sharing compatible lighting, or avoiding incompatible lighting. In addition, the system in a master / slave array may provide combined data results, for example, in a multi-reader scanning tunnel.

[0062] The disclosed reader can match the decoding results with the image candidates that failed to decode the optical code. Among the images that the system has successfully decoded, the disclosed system and method can analyze the images to determine the best image for recording and preservation. In addition, the disclosed system determines the best image among the images that were not successfully decoded.

[0063] According to aspects of the present disclosure, an image that was not successfully decoded can be used to provide feedback about why decoding has failed. For example, possible reasons why an image did not decode can include one or more characteristics of the bar code reader system as well as physical code and adjacent characteristics. Optical reader characteristics that can cause decoding failures include insufficient illumination, non-uniform illumination, sensor noise, high viewing direction relative to the bar code, etc. Physical characteristics and scenarios of bar codes that can cause decoding failures include: code marking quality or damage after marking, distortions in applied symbols, insufficient quiet areas, features or edges such as being too close to a "start" character or an "end" character. The start and end characters are separator characters, also known as guard patterns, which appear at the beginning and end of a bar code. The "start" character and the "end" character can be used to enable a reader to determine where to start reading and where to end reading.

[0064] According to some aspects of the disclosure, the disclosed system can also identify multiple images that include codes and are successfully decoded. The system can pass the multiple images that are successfully decoded to a processor for further analysis. For example, the processor can analyze the code quality in the successfully decoded images. Among the images that successfully decode the code, the disclosed system can identify the most interesting images, such as those that can provide the most information for analyzing the code quality. The tracking model can be used to identify which images correspond to the same code that is successfully decoded.

[0065] According to some embodiments, the camera and reader image capture are synchronized so that illumination light from the read can be used to illuminate the camera image. The reader array communicates the status of the read (such as whether there is a good or bad read) to the camera to provide both good read / bad read status to the camera. The camera can then store, transmit, or discard the image based on the read status.

[0066] The disclosed system can provide real-time / historical overviews of system performance and any aggregated collected data to a reader group. The disclosed system can be built on a distributed architecture that can minimize network load and distribute computation and storage to multiple readers and collected data.

[0067] The disclosed system may include a data collection module that is responsible for collecting read performance data from multiple readers and may also collect unread images. Historical data is stored to allow the system to examine how read performance has changed and how interventions in service processes (such as replacing a printer) affect read performance. Read performance may be measured, for example, by a decode rate (percentage). Additional criteria for read performance may include evaluating the quality or confidence level of bar code decoding.

[0068] The disclosed system may further include an image analysis and classification module and a data synchronization module. The image analysis and classification module may perform analysis on one or more corresponding images and generate a classification for a decoding failure. A decoding failure may include the absence of a case during a trigger sequence, the absence of a label on a case, misplacement of a label (such as if it is suspended on the case), and poor label printing quality. The data synchronization module may be responsible for synchronizing data between different units of the system.

[0069] An exemplary system for collecting performance data and collecting images from one or more code reading devices is the COGNEX EXPLORER RTM.

[0070] According to some aspects of the present invention, the disclosed systems and methods can improve yield, such as the yield of successful reads of an optical reader system. The predicted range of the possible next tracked code posture is used to sequence the decoding attempts on the code candidates. For example, the code decoded in the previous image can be sequenced later or ignored completely, so as to sequence the decoding candidates of other potential codes in the early stage. In a system such as a high frame rate area based on a camera for optical code reading, the time available for decoding each image is limited. In this case, improving the sequence of analyzing code candidates (and avoiding unnecessary processing of previously decoded symbols) can improve the optical reader system yield. Similarly, the predicted range of the possible next tracked code posture can be used to improve the sequence of candidates for codes that have not been successfully decoded. Other methods of customizing code candidates based on the predicted range of the possible next tracked code posture are expected.

[0071] The subject matter described herein can be implemented in digital electronic circuits or in computer software, firmware or hardware or their respective combinations, and the computer software, firmware or hardware includes the structural device disclosed in the specification and its structural equivalent or their combination. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs embodied in an information carrier (such as in a machine-readable storage device) or embodied in a signal for propagation, which is executed by the operation of a control data processing device (such as a programmable processor, a computer or a plurality of computers) or used to control the operation of the data processing device. Computer programs (also referred to as programs, software, software applications or codes) can be rewritten in any form of programming language (including compiled code or translated code), and the programming program can be deployed in any form, including deployment as independent programs or modules, components, subroutines or other units suitable for use in a computing environment. Computer programs do not necessarily correspond to files. Programs can be stored in a portion of a file that holds other programs or data, in a single file for the program in question, or in multiple coordinated files (such as files that store a portion of one or more modules, subroutines or codes). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

[0072] The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors running one or more computer programs to perform the functions of the subject matter described herein by operating input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits), and the apparatus of the subject matter described herein can be implemented as special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0073] Processors suitable for executing computer programs include, for example, general purpose microprocessors and special purpose processors and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and / or data. Typically, a computer will also include or be operatively coupled to receive data from and transfer data to one or more mass storage devices (such as magneto-optical or magneto-optical disks) for storing data or both. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices (such as EPROM, EEPROM and flash memory devices); magnetic disks, (such as internal hard disks or removable disks); magneto-optical disks; and optical disks (such as CD and DVD disks). The processor and memory may be supplemented by or incorporated into dedicated logic circuits.

[0074] In order to provide interaction with a user, the subject matter described herein can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user, and a keyboard and a pointing device (such as a mouse or trackball), and the user can provide input to the computer through the keyboard and pointing device. Other types of devices can also be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback (such as visual feedback, auditory feedback or tactile feedback), and the input from the user can be received in any form, including sound, speech or tactile input.

[0075] The subject matter described herein can be implemented with a computing system that includes a back-end component (e.g., a data server), an intermediate component (e.g., an application server), or a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of such back-end components, intermediate components, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include a local area network ("LAN") or a wide area network ("WAN"), such as the Internet.

[0076] It should be understood that the disclosed subject matter is not limited to application to the details of construction and to the arrangement of components given in the following description or shown in the accompanying drawings. The disclosed subject matter can be used in other embodiments and can be implemented and executed in various ways. In addition, it should be understood that the words and terms used herein are for descriptive purposes and should not be construed as limiting.

[0077] As such, those skilled in the art will appreciate that the concepts upon which this disclosure is based may readily be utilized as a basis for the designing of other structures, methods, and systems for carrying out the several purposes of the disclosed subject matter. It is important, therefore, that the claims be regarded as including such equivalent structures that do not depart from the spirit and scope of the disclosed subject matter.

[0078] While the disclosed subject matter has been described and illustrated in the foregoing exemplary embodiments, it is to be understood that the present disclosure has been made by way of example only and that numerous changes in details of the implementation of the disclosed subject matter may be made without departing from the spirit and scope of the disclosed subject matter, which is limited only by the following claims.

Claims

1. A method for locating a barcode applied to an object, comprising: receiving, by a processor, a first image of a first bar code segment applied to a first target captured by a camera at a first time, wherein the first image does not include a second bar code segment different from the first bar code segment; identifying, by the processor, a first position of a first barcode segment based on the first image; receiving, by the processor, a second image of the second bar code segment applied to the first target captured by the camera at a second time, wherein the second image does not include the first bar code segment; identifying, by the processor, a second position of a second barcode segment based on the second image; predicting, by the processor, a range of possible positions of the first barcode segment at a second time based on a tracking model, wherein the tracking model tracks the first barcode segment based on the first position; and It is determined by the processor that if the second position is within a range of possible positions of the first bar code segment at the second time, then the first bar code segment and the second bar code segment correspond to the same bar code applied to the first target.

2. The method according to claim 1, wherein: The tracking model further tracks the first barcode segment based on at least one of: a fixed constraint, a constructible constraint, an adaptive constraint, and a user constraint.

3. The method according to claim 1, wherein: The tracking model further tracks the first barcode fragment based on at least one of: additional information from the captured first image and the captured second image, information from at least one previous image sequence, a time interval between the first time and the second time, a distance between the first position and the second position, information from a current image sequence, image calibration information, image scaling information, time differences of image exposure sequences, symbology, and camera movement information.

4. The method according to claim 1, wherein: The tracking model further tracks the first barcode segment based on at least one of an estimate of motion in image coordinates or an estimate of motion in three-dimensional physical space.

5. The method according to claim 1, wherein: The tracking model further tracks the first barcode segment based on at least one of: a position of a camera that captured the first image and the second image, an orientation of the camera, a camera calibration, an angle of the camera relative to the first object, and a distance of the camera from the first object. 6 . The method of claim 1 , further comprising combining the first barcode segment and the second barcode segment if the first barcode segment and the second barcode segment correspond to the same barcode.

7. The method of claim 1, further comprising determining that if the first barcode segment and the second barcode segment do not correspond to the same barcode, then the first barcode segment and the second barcode segment correspond to different targets.

8. The method according to claim 1, wherein: The tracking model tracks the target in the image coordinate system.

9. The method according to claim 1, wherein: The tracking model tracks an object moving on a conveyor belt, wherein the conveyor belt moves the object in a first direction of travel.

10. The method according to claim 9, wherein: The tracking model further tracks the target based on the physical properties of the conveyor belt.

11. The method according to claim 10, wherein: The physical characteristics of the conveyor belt include at least one of: the first direction of travel, a linear speed of the conveyor belt, a length of the conveyor belt, and a width of the conveyor belt.

12. A method for locating a barcode applied to an object in a system configured to extend the field of view of a scene imaged by a camera having an image sensor, the method comprising: receiving a first image captured by the camera at a first time, the first image comprising a first field of view and a second field of view, wherein the first field of view of the first image depicts a first barcode segment and the second field of view of the first image depicts a second barcode segment, wherein the first image does not include a third barcode segment and a fourth barcode segment, wherein the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment are different from one another; identifying, based on the first image, a first position of a first barcode segment within a first field of view of the first image; identifying a first position of a second barcode segment within a second field of view of the first image based on the first image; receiving a second image captured by the camera at a second time, the second image comprising a first field of view and a second field of view, wherein the first field of view of the second image depicts the third bar code segment and the second field of view of the second image depicts the fourth bar code segment, wherein the second image does not include the first bar code segment and the second bar code segment; identifying a second position of a third barcode segment within the first field of view of the second image based on the second image; identifying a second position of a fourth barcode segment within a second field of view of the second image based on the second image; predicting a range of possible positions of the first barcode segment at a second time based on a first tracking model, wherein the first tracking model tracks the first barcode segment based on a first position of the first barcode segment; predicting a range of possible positions of the second barcode segment at a second time based on a second tracking model, wherein the second tracking model tracks the second barcode segment based on the first position of the second barcode segment; determining whether a second position of the third barcode segment is within a range of possible positions of the first barcode segment at a second time; determining whether a second position of the fourth barcode segment is within a range of possible positions of the second barcode segment at a second time; and If the second position of the third barcode segment is within the range of possible positions of the first barcode segment at the second time and the second position of the fourth barcode segment is within the range of possible positions of the second barcode segment at the second time, it is determined that the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment correspond to the same barcode.

13. The method according to claim 12, wherein: The first tracking model and the second tracking model further track the barcode fragment based on at least one of: a fixed constraint, a configurable constraint, an adaptive constraint, and a user constraint.

14. The method according to claim 12, wherein: The first tracking model and the second tracking model further track the barcode fragments based on at least one of the following: additional information from the captured first image and the captured second image, information from at least one previous image sequence, a time interval between the first time and the second time, a distance between a first position of the first barcode fragment and a second position of the first barcode fragment, a distance between a first position of the second barcode fragment and a second position of the second barcode fragment, information from a current image sequence, image calibration information, image scaling information, time differences of an image exposure sequence, symbology, and camera movement information.

15. The method according to claim 12, wherein: The first tracking model and the second tracking model further track the barcode segment based on at least one of an estimate of motion in image coordinates or an estimate of motion in three-dimensional physical space.

16. The method of claim 12, wherein the first tracking model and the second tracking model further track the barcode fragment based on at least one of: a position of a camera that captured the first image and the second image, an orientation of the camera, a camera calibration, an angle of the camera relative to the first target, and a distance of the camera from the first target.

17. The method of claim 12, further comprising combining the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment if the barcode segments correspond to the same barcode.

18. The method of claim 12, further comprising determining that if the barcode segments do not correspond to the same barcode, the barcode segments correspond to different targets.

19. The method according to claim 12, wherein: The tracking model tracks the target in the image coordinate system.

20. The method according to claim 12, wherein: The tracking model tracks an object moving on a conveyor belt, wherein the conveyor belt moves the object in a first direction of travel.

21. The method according to claim 20, wherein: The tracking model further tracks the target based on the physical properties of the conveyor belt.

22. The method according to claim 21, wherein: The physical characteristics of the conveyor belt include at least one of: the first direction of travel, a linear speed of the conveyor belt, a length of the conveyor belt, and a width of the conveyor belt.

23. A system for locating a bar code applied to an object, comprising: a camera configured to capture an image of a bar code applied to a target; as well as a processor in communication with the camera and configured to: receiving a first image of a first bar code segment applied to a first target captured by the camera at a first time, wherein the first image does not include a second bar code segment different from the first bar code segment; identifying a first position of a first barcode segment based on the first image; receiving a second image of the second bar code segment applied to the first target captured by the camera at a second time, wherein the second image does not include the first bar code segment; identifying a second position of a second barcode segment based on the second image; predicting a range of possible positions of the first barcode segment at a second time based on a tracking model, wherein the tracking model tracks the first barcode segment based on the first position; and If the second position is within a range of possible positions of the first bar code segment at a second time, then it is determined that the first bar code segment and the second bar code segment correspond to the same bar code applied to the first target.

24. A system for locating a barcode applied to an object, the system being configured to extend the field of view of a scene imaged by a camera having an image sensor, the system comprising: a camera configured to capture an image of a bar code applied to a target; as well as a processor in communication with the camera and configured to: receiving a first image captured by the camera at a first time, the first image comprising a first field of view and a second field of view, wherein the first field of view of the first image depicts a first barcode segment and the second field of view of the first image depicts a second barcode segment, wherein the first image does not include a third barcode segment and a fourth barcode segment, wherein the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment are different from one another; identifying, based on the first image, a first position of a first barcode segment within a first field of view of the first image; identifying a first position of a second barcode segment within a second field of view of the first image based on the first image; receiving a second image captured by the camera at a second time, the second image comprising a first field of view and a second field of view, wherein the first field of view of the second image depicts the third bar code segment and the second field of view of the second image depicts the fourth bar code segment, wherein the second image does not include the first bar code segment and the second bar code segment; identifying a second position of a third barcode segment within the first field of view of the second image based on the second image; identifying a second position of a fourth barcode segment within a second field of view of the second image based on the second image; predicting a range of possible positions of the first barcode segment at a second time based on a first tracking model, wherein the first tracking model tracks the first barcode segment based on a first position of the first barcode segment; predicting a range of possible positions of the second barcode segment at a second time based on a second tracking model, wherein the second tracking model tracks the second barcode segment based on the first position of the second barcode segment; determining whether a second position of the third barcode segment is within a range of possible positions of the first barcode segment at a second time; determining whether a second position of the fourth barcode segment is within a range of possible positions of the second barcode segment at a second time; and If the second position of the third barcode segment is within the range of possible positions of the first barcode segment at the second time and the second position of the fourth barcode segment is within the range of possible positions of the second barcode segment at the second time, it is determined that the first barcode segment, the second barcode segment, the third barcode segment, and the fourth barcode segment correspond to the same barcode.

Images