READING OPTICAL CODES

Description

[0001] The invention relates to an optoelectronic code reader and a method for reading optical codes according to the preamble of claim 1 and 15 respectively.

[0002] Code readers are commonly found at supermarket checkouts, for automatic package identification, mail sorting, baggage handling at airports, and in other logistics applications. In a code scanner, a reading beam is guided across the code using a rotating mirror or a polygonal mirror wheel. A camera-based code reader uses an image sensor to capture images of objects with their codes, and image analysis software extracts the code information from these images. Camera-based code readers can easily handle code types other than one-dimensional barcodes, such as matrix codes, which are also two-dimensional and provide more information.

[0003] In one important application, the code-bearing objects are conveyed past the code reader. A code scanner captures the codes as they are successively guided into its reading area. Alternatively, in a camera-based code reader, a line scan camera reads the object images containing the code information successively and line by line, capturing the relative movement. A two-dimensional image sensor regularly records image data, which overlaps to a greater or lesser extent depending on the recording frequency and conveying speed. To allow the objects to be arranged in any orientation on the conveyor, several code readers are often installed on a single reading tunnel to capture objects from multiple or all sides.

[0004] As preparation for reading codes, a captured image of a code-bearing object is scanned for areas of interest, or code image areas—that is, those areas in the image that could potentially contain a code. This step is called segmentation or pre-segmentation. Humans find it very easy to recognize code image areas. However, from an algorithmic perspective, this is an extremely challenging task. This is because patterns or structures in the background often exhibit similar properties to the code image areas themselves. For example, 2D codes with very small modules have many similar properties to text areas, logos, packaging tape, and the like.

[0005] In a code-reading application, only a limited amount of decoding time is typically available because new objects and codes are constantly being captured, requiring processing in near real-time. A certain degree of temporal flexibility is permissible, but the time gap must never become so large that codes are missed. For example, with a matrix camera, the time between two images sets an upper limit on the decoding time. This holds true at least on average; a processing buffer allows for more decoding time to be allocated to an image in exceptional cases, but this time is then unavailable for subsequent images. The situation is quite similar when using a line scan camera with successive image compositing. If the offset between the processed image sections and the currently captured line becomes too large, a gap eventually arises that cannot be closed under near real-time conditions.

[0006] If the aforementioned segmentation detects false-positive areas of interest where no optical code is actually present, this increases decoding time. Codes may then go unread because the decoder has to spend too much of its decoding time attempting to read these false-positive areas of interest.

[0007] Therefore, it is desirable to avoid false-positive areas of interest. This does not, of course, mean shifting the balance towards false-negative areas of interest, i.e., overlooking optical codes during segmentation from the outset. One possible approach to reducing the overall error rate in finding areas of interest is to improve the segmentation process. EP 4 231 195 A1, for example, uses a combination of classical image processing and a machine learning method for this purpose.

[0008] In practical applications, optical codes are usually printed on labels. They are therefore not typically located directly on the object itself, but rather grouped across the label. Furthermore, the labels exhibit regularities. Most shipping companies or logistics providers, for example, use a largely fixed layout. However, information that can be derived from a label is currently not considered for code reading.

[0009] EP 3 534 291 A1 addresses the reading of codes during baggage handling at airports, where multiple readings can occur due to so-called tear-off codes—older codes that have not been completely removed. In the event of a multiple read, the geometry of the code or of a tag or label on which the code is located is checked. However, these are all steps downstream of segmentation and decoding and do not contribute to reducing false positives in areas of interest or improving the utilization of decoding time.

[0010] In US 2008 / 0121688 A1, the information to be encoded is distributed, at least partially overlapping, across two barcodes of a label. This is intended to ensure that the information can still be read even if one of the codes is partially obscured. However, this does not address the aforementioned issues of limited decoding time and potential false positives in areas of interest.

[0011] EP 3 428 835 A1 and EP 4 258 160 A1 deal with prior knowledge about codes or code schemes, which may allow codes to still be read if they are damaged or if other malfunctions cause a reading error. This involves reconstructing part of the code from the prior knowledge or code scheme, or alternatively, at least determining that a reading result is incorrect. All of this relates to the decoding itself, not the segmentation, and does not utilize any information related to labels.

[0012] US Patent 2010 / 0044439 A1 (D1) discloses an image processing method for reading barcodes in which the probable image area is located along with the barcode to restrict decoding to this area. The detection of the probable image area is based on the assumed light background of a barcode. Specifically, the values ​​of a binarized image are simply added together, where black is represented by a zero and white by a one, so that light rows or columns can be identified by a high sum.

[0013] EP 2 003 599 A1 (D2) describes a real-time binarization method in an optoelectronic sensor for code detection. As a preferred preprocessing step for the actual image processing, areas of interest are then identified. An example of an area of ​​interest is a sticker identifiable by its optical properties.

[0014] It is therefore the purpose of the invention to further improve the reading of optical codes.

[0015] This problem is solved by an optoelectronic code reader and a method for reading optical codes according to claim 1 and 15, respectively. A light-receiving element captures image data of code-bearing objects, particularly during a conveying movement of the objects. The light-receiving element can be an image sensor of a matrix camera or a line sensor that successively captures image lines. A code scanner can also capture image data line by line. A control and evaluation unit segments the image data, thus identifying regions of interest (ROIs) containing optical codes or code candidates. Whether a region of interest actually contains a code is not known at this stage; this is initially assessed solely based on the segmentation criteria, which may result in false positives.In some situations, the image data as a whole will contain no code, so that the segmentation results in no areas of interest or only false positives of interest. The areas of interest are processed with a decoder or decoding method to read the code content of any optical code contained within them. In the case of a false positive area of ​​interest, the decoding method will only consume decoding time without producing a reading result. A decoder preferably uses a variety of decoding methods, which is why the decoding method is repeatedly referred to with an indefinite article in the following text.

[0016] The invention is based on the fundamental idea of ​​recognizing a label in the image data. As described in the introduction, a label contains optical codes and often other markings on a common substrate, such as a tag, sticker, or other printed material attached to the code-bearing object. The aim is to assign the areas of interest to a label, thereby distinguishing between areas of interest that are part of a label and those that are not. In the case of multiple recognized labels, a distinction is preferably made as to which label an area of ​​interest belongs. Further processing of the areas of interest using the decoding method then depends on whether the area of ​​interest belongs to a label and, if so, to which label.Differences may arise, for example, with regard to the decoding method, the sequence of decoding, the allocated decoding time and the number of retries, as explained below using various embodiments.

[0017] The invention has the advantage that knowing which label a data belongs to enables improved decoding. The available decoding time can be better utilized to focus more on genuine code areas rather than false positives. This increases the overall read rate. Furthermore, the labels provide a logical grouping of areas of interest, optical codes, and their code content, which can also be made available for output. In particular, information can be output indicating which label a read result belongs to, which label was detected, and / or whether it was fully decoded or not.

[0018] The control and evaluation unit is preferably designed to group the areas of interest according to labels. Thus, it does not simply make a binary decision as to whether an area of ​​interest is located on a label or not. Rather, a grouping is found that assigns each area of ​​interest to a specific label. This may result in some "lone wolves" remaining, namely areas of interest that cannot be assigned to any label.

[0019] The control and evaluation unit is preferably designed to determine a score for the areas of interest, indicating how reliably an optical code is detected in that area, and to consider this score in the sequence of processing the areas of interest using a decoding method. The score depends on how well the segmentation criteria have been met and is therefore a secondary result of the segmentation process. According to this embodiment, unlike in the prior art, the decoding sequence does not depend solely on the score. Instead, the label assignment is advantageously taken into account, at least in part.

[0020] The control and evaluation unit is preferably designed to assign a higher value to areas of interest that are part of a label. This is a particularly simple way to incorporate label affiliation into the decoder's processing sequence. The value can be increased even further if a code has already been read on the same label or if the label has been more precisely identified. These are clear indications that the area of ​​interest in question is not a false positive result of the segmentation, so it is likely worthwhile to invest further decoding time here. The surcharge to the value can be understood or implemented conversely as a discount for areas of interest outside of labels.This means that such lone wolves are not completely excluded from decoding, but they must show very clear signs of an optical code in order to be allocated decoding time.

[0021] The control and evaluation unit is preferably designed to process the relevant areas of a label sequentially using a decoding method, so that the relevant areas of a label are processed as a group. "Sequentially" or "simultaneously" means that areas of interest that do not belong to the currently processed label are not decoded in the interim. The decoder's sequence thus differs significantly from the conventional approach, which ignores label affiliation. According to this embodiment, the logical relationship of optical codes from a single label is also reflected in their simultaneous processing by the decoder. Within the group of relevant areas of the same label, the sequence can, for example, be prioritized according to numerical values.If multiple labels are detected simultaneously, the order of the labels can be determined randomly or based on the numerical values ​​of the areas of interest on the label. For example, the process starts with the label containing the area of ​​interest with the highest numerical value, or with the highest average numerical value, or with the most or fewest areas of interest, and so on.

[0022] The control and evaluation unit is preferably designed to recognize a label based on its brightness and homogeneous texture. These are two particularly defining characteristics of a label. Typically, optical codes are printed on paper or a comparable white or at least very light background. Accordingly, searching for a label in a captured image or image area essentially means searching for a bright, uniform area. Of course, a homogeneous texture is not required throughout, as otherwise only blank labels without codes would be found. Rather, the requirement is that a relevant proportion—for example, 50%, 25%, or even just 10%—of the label forms its homogeneous background.If the code reader can detect colors, it will preferably further differentiate and search for a white or light gray background, since optical codes are generally not applied to colored backgrounds, not even light-colored backgrounds such as yellow or light pink. On the other hand, the code reader may exceptionally know that labels exist in a specific color; in this case, that color becomes a strong indicator that identifies the label as such.

[0023] The control and evaluation unit is preferably designed to recognize a label using at least one of the following criteria: dark structures, shape, size, contained code structures, and the orientation of the code structures within the label. Dark structures correspond to the codes being sought, which are preferably printed or applied in black. A label usually has a rectangular shape or at least a well-defined geometry, which may also include other geometric properties such as an aspect ratio. The size or dimensions of a label may also be assumed or known. Contained code structures mean that signatures of optical codes are recognized, in particular typical light-dark transitions in expected patterns.Finally, code structures on a label are not randomly placed at an angle, but are applied systematically, usually horizontally or sometimes vertically. This brings the total to five possible criteria for a label; including brightness and homogeneity of the introductory paragraph, there are seven criteria in total. These are not necessarily exhaustive, but particularly suitable. To avoid having to address all criteria, advantageous sub-combinations can be created, for example, brightness, shape, and the code structures contained within.

[0024] The control and evaluation unit preferably has access to at least one label template that identifies a known label type, wherein a label template comprises a number of optical codes present on this label type, in particular including the respective code type and / or position of the optical codes within a label. Access means that the control and evaluation unit includes a corresponding memory of any design or has at least indirect access to such a memory. So far, it has only been recognized that it is a label. With the aid of label templates, specific labels can be recognized and distinguished. Label templates can include various features, the number of optical codes contained therein being particularly important, preferably including the associated code type.Furthermore, prior knowledge about the respective code can be specified, for example, a recurring fixed part of the code or a description of what type of characters are expected at at least some positions within the code. Such prior knowledge facilitates decoding or, in the case of damage or disrupted image capture, makes it possible in the first place. For further details, reference is made to the aforementioned documents EP 3 428 835 A1 and EP 4 258 160 A1. Preferably, the label template also indicates a position of the code on the label. This position is preferably relative, for example, from the center of the label or from a code selected as a reference on the label. For this purpose, rays radiating from the reference point can be specified with angle and length, indicating where another code is located according to the label template.For implementation reasons alone, a label template preferably includes an identifier, which can also be designed to be descriptive, to allow for association with a manufacturer or a label class recognizable to the user. A label template can optionally include one or more of the seven characteristics mentioned above for label identification, in particular the expected background brightness, the size and shape of the label, and the orientation of the codes.

[0025] The control and evaluation unit has preferential access to at least one label template that features at least one logo. A logo is therefore provided as an alternative or supplementary feature of a label template. A logo is understood to be a graphic symbol that recognizably identifies a specific label manufacturer, in particular an image-text combination. Such logos are applied to numerous labels to indicate the label manufacturer, logistics provider, or shipper. The logo is then usually in a fixed geometric relationship to the label and the codes applied to it. This allows relevant areas of a label to be verified, validated, or even initially located based on the logo's position on the label type known from the label template.

[0026] The control and evaluation unit is preferably designed for automatic learning, in which a label template or a property of a label template is learned from captured image data. In this process, the code reader is preferably shown specific examples of labels with codes before operation. Learning or further learning during operation is also conceivable, for example, with an initial label template that only specifies the number and type of codes, and to which the positions of the codes are subsequently learned during operation using labels that match the template. As an alternative to automatic learning, manual parameterization or input with the support of a graphical user interface (GUI) is conceivable. Hybrid systems with automatic suggestions to a user or with initially manually created label templates that are then automatically refined are also possible.

[0027] The control and evaluation unit is preferably designed to identify a label using a label template, in particular with the aid of at least one previously decoded optical code of an area of ​​interest on the label or a logo. The label templates make it possible to identify a label of a specific type within the image data. This can in itself be useful output information, indicating which labels have been detected or which reading result corresponds to which label. This is preferably used to assign further properties to the label based on the label template or to verify a decoding result. With each read code, hypotheses can be formulated or refined regarding which label template a label detected in the image data belongs to.Identifying a label using a label template is particularly easy and robust, either by using the code content of one of the codes on the label or by using a logo.

[0028] The control and evaluation unit is preferably designed to process areas of interest within a label that has been at least partially identified using a decoding method until all optical codes of the label have been decoded. "At least partially" means that there is at least a hypothesis as to which label template(s) a label recognized in the image data corresponds to. From a matching label template, it is then possible to deduce which further codes should still be present, preferably even with a positional indication within the label. Further read attempts (retries) can therefore be carried out in a targeted manner to ultimately capture and identify a label completely, i.e., with all its codes.In this context, a label template provides a dual condition in the sense of completeness and sufficiency: Firstly, further areas of interest must be processed with a decoding method if not all codes have yet been read according to the label template's specifications. Secondly, however, processing of further areas of interest within a label detected in the image data can cease if the read codes correspond to the label template. There may still be further areas of interest within the label, but these are now identified as false positives, and this is recognized by the label template, thus avoiding unnecessary decoding time.

[0029] The control and evaluation unit is preferably designed to transfer the code content of one area of ​​interest to at least one other area of ​​interest if, according to the label template, several optical codes with the same code content are present in a label. Some labels are designed to have the same code content redundantly applied to multiple codes on the label. It is known via the label template whether such a case exists. If such a label is then recognized in image data and one of the redundant codes has already been read, it is unnecessary to read the other redundant codes as well. Instead, according to this embodiment, the reading result is transferred to all redundant codes without any additional decoding time expenditure.

[0030] The control and evaluation unit is specifically designed to adapt the areas of interest on a label using the label template. Thanks to a label template with positions, it is known for a given label where the codes, and specifically the unread codes, must be located. This allows for further improvement of the segmentation; dedicated areas of interest can be created at precisely the right locations on the label, enabling successful decoding in the next step, ideally through targeted retries.This is conceivable in several respects: Areas of interest that were previously overlooked as false negatives may be added because the relevant location is examined more closely or weaker structures are now accepted; areas of interest that have already been segmented may be sorted out as false positives because they are located in a place where no code is provided in the label; or an already segmented area of ​​interest may be improved in position and size.

[0031] The method according to the invention can be further developed in a similar manner and exhibits similar advantages. Such advantageous features are described by way of example, but not exhaustively, in the dependent claims following the independent claims.

[0032] The invention is further explained below with regard to additional features and advantages by way of example embodiments and with reference to the accompanying drawing. The illustrations in the drawing show: Fig. 1: A schematic three-dimensional overview of an exemplary assembly of a code reader above a conveyor belt on which objects with codes to be read are conveyed; Fig. 2: An exemplary flowchart for the recognition and decoding of labels; Figs. 3a-e: Example images at various stages of label recognition, namely in Figure 3a the unprocessed original image, in Figure 3b a close-up of interference structures that can be mistaken for codes, in Figure 3c the highlighted structures brighter than a threshold, in Figure 3d the application of an erosion filter and in Figure 3eFigure 4 shows the four possible labels remaining in this example; Figure 4 shows an exemplary flowchart for identifying a label with a label stencil and for decoding it; Figure 5 shows an example image of a standalone code not applied to a label; Figure 6 shows an example image of a label with multiple codes to illustrate the indication of code positions on a label using rays of angle and length; Figure 7a shows an illustration of the shape and position information of a label stencil; Figure 7b shows an illustration of the test to determine whether a label stencil matches a picked-up label with respect to the shape and position information shown in Figure 7a; and Figure 8 shows an example image of a label with redundant codes.

[0033] Figure 1Figure 1 shows an optoelectronic code reader 10 in a preferred application situation mounted above a conveyor belt 12, which conveys objects 14, as indicated by arrow 16, through the detection area 18 of the code reader 10. The objects 14 bear codes 20a-b on their outer surfaces, which are detected and evaluated by the code reader 10. Many of the codes 20a are located in groups on a label 22, although there may exceptionally be individual codes 20b outside the labels 22. The codes 20a-b and labels 22 can only be detected by the code reader 10 if they are affixed to the top surface or at least visible from above. Therefore, unlike the illustration in Figure 1, the following applies: Figure 1To read a code located, for example, to the side or bottom, a plurality of code readers 10 are mounted from different directions to enable so-called omnidirectional reading from all directions. In practice, the arrangement of the multiple code readers 10 into a reading system is usually implemented as a reading tunnel. This stationary application of the code reader 10 on a conveyor belt is very common in practice. However, the invention initially relates to the code reader 10 itself or the code-reading method implemented therein, so this example should not be understood as limiting.

[0034] The code reader 10 uses an image sensor 24 to capture image data of the conveyed objects 14, including codes 20a-b and labels 22. This data is then further processed by a control and evaluation unit 26 using image evaluation and decoding methods. The specific imaging method is not essential to the invention, so the code reader 10 can be constructed according to any known principle. For example, only one line is captured at a time, either using a line-shaped image sensor or a scanning method. In the latter case, a simple light receiver such as a photodiode suffices as the image sensor 24. The control and evaluation unit 26 combines the lines captured during the conveying movement into the image data. A matrix-shaped image sensor allows a larger area to be captured in a single image, and here, too, the merging of images is possible both in the conveying direction and perpendicular to it.The multiple images are captured sequentially and / or by multiple code readers 10, whose detection ranges 18, for example, only together cover the entire width of the conveyor belt 12, with each code reader 10 capturing only a subsection of the overall image, and in particular, the subsections being stitched together by image processing. Fragmentary decoding within individual subsections followed by stitching of the code fragments is also conceivable.

[0035] The code reader 10 outputs information via an interface 28, such as read codes 20a-b, recognized labels 22 or types of labels, as well as image data or excerpts thereof. It is also conceivable that the control and evaluation unit 26 is not located in the actual code reader 10, i.e., the one in Figure 1The control and evaluation unit 26 is not arranged as shown in the camera, but is connected as a separate control device to one or more code readers 10. In this case, the interface 28 also serves as a connection between internal and external control and evaluation. The control and evaluation functionality can be distributed across virtually any internal and external components, with the external components also being able to be connected via a network or cloud. All of this is not further differentiated here, and the control and evaluation unit 26 is considered part of the code reader 10, regardless of the specific implementation. The control and evaluation unit 26 can comprise several components, such as an FPGA (Field Programmable Gadget Array), a microprocessor (CPU), and the like.In particular, for the segmentation with a neural network, which will be described later, specialized hardware components can be used, such as an AI processor, an NPU (Neural Processing Unit), a GPU (Graphics Processing Unit), a VPU (Video Processing Unit), or the like.

[0036] The task of code reader 10 is to read codes 20a-b. As a preprocessing step of the image data, areas of interest are first identified where a code 20a-b is likely to be found. This is also known as segmentation or pre-segmentation and is a known method, for example, as a combination of classical image processing and a machine learning method according to EP 4 231 195 A1 mentioned in the introduction. Segmentation can simultaneously determine a score indicating the probability, based on the segmentation criteria, that a code 20a-b is present in the area of ​​interest. The areas of interest, or code candidates, are then processed by a decoder, which uses one or more decoding methods to read the code content. False positives can occur in areas of interest where no code is actually present.Then part of the available decoding time is wasted unnecessarily, and the decoding time is limited, as already described in the introduction, because the application constantly generates new image data and therefore requires evaluation at least in near real time.

[0037] According to the invention, segmentation and decoding are supported by the recognition of labels 22 and the derivation of certain properties or processing steps depending on whether a code 20a-b is present on a label 22 or not, and preferably in a more differentiated manner depending on the type of label 22. This will now first be presented in the form of a brief overview and then with reference to the Figures 2 to 8 explained in more detail.

[0038] In a relatively simple embodiment, labels 22 are recognized based on general image features, such as their light-colored areas. Only areas of interest with presumed codes 20a on labels 22 are then decoded, or at least these are prioritized with regard to decoding time. A solitary area of ​​interest, i.e., an area of ​​interest outside of labels 22, must, for example, provide very clear indications of a code 20b through a high value before decoding is attempted. Generally, apart from the aforementioned exception, it is assumed that areas of interest outside of labels are interference textures or false-positive areas of interest. The codes 20a are grouped by the labels 22; each label forms a kind of visual bracket.This allows for a predefined decoding sequence for each group, label 22 by label 22, instead of the conventional, effectively random order with respect to labels 22. Specific prior knowledge about particular labels 22 beyond the general image features for label 22 recognition is not required.

[0039] In another embodiment, label types are described by prior knowledge, so-called label templates. This allows labels to be classified and this information to be shared externally. Furthermore, improvements for decoding, the decoding sequence, and the areas of interest can be derived from the prior knowledge about a label, and highly targeted read retries can be performed, which have a higher probability of success and thus make better use of the available decoding time.

[0040] In yet another embodiment, additional information about the expected positions of the codes 20a of a label 22 is included in the label templates. This allows particularly promising areas of interest to be identified at positions where a code 20a should be located but has not yet been read, in order to acquire areas of interest, to eliminate those that have not yet been read, to improve their position and form, and ultimately to perform particularly promising read retries.

[0041] In an extension of this embodiment, unnecessary decoding calls for redundant codes 20a are avoided. Numerous types of labels 22 convey the same code information in several redundant codes, and the corresponding information is stored in the label template. This makes it possible to read only one of these redundant codes 20a or to switch to another of the redundant codes 20a if decoding proves difficult. The reading result can be transferred directly to the other redundant codes without requiring separate decoding.

[0042] Figure 2 Figure 1 shows an exemplary flowchart for the recognition of labels 22 and for decoding them. In this embodiment, no specific information is yet available for particular types of labels 22. The individual steps are only mandatory if explicitly described as such.

[0043] In step S1, bright, at least partially structureless areas are detected. For this purpose, corresponding connected components are created, particularly in the form of BLOBs (Binary Large Objects). Up to this point, the labels 22 are recognized solely based on the two general characteristics of brightness and homogeneity. It should be noted that the labels 22 being sought are by no means entirely structureless due to the codes 20a they contain, but only exhibit corresponding proportions, which is sufficient as a characteristic for the recognition of a label 22.

[0044] In step S2, the related components that can be considered candidates for label 22 are checked against further label characteristics in order to exclude as many related components as possible that do not correspond to any label 22. The two characteristics already mentioned, brightness and homogeneous components, or even just one of these characteristics, are sometimes sufficient, but often not enough on their own to identify label 22 with sufficient clarity.Therefore, preferably up to five additional features are added: dark structures or distinct black values, since corresponding printed codes 20a are expected; a shape or contour such as that of a rectangle or other expected shape of a label 22 with or without an expected aspect ratio; expectations regarding the size or area, because codes 20a are supposed to fit within it; recognizable code structures with typical texture features, light-dark transitions, or texture signatures of the expected code types; and the orientation of the code structures within the label 22, measured, for example, by the contour or shape of the feature listed above, since the orientation is typically horizontal or vertical, but not oblique. Information on the extent to which typical texture features of a code 20a are present and how they are oriented can already be obtained as a partial result of the segmentation.The seven features mentioned can only be used partially in any combination, and conversely, this is not an exhaustive list of possible features that can be checked. A particularly preferred combination of features, in addition to checking brightness and / or homogeneous components in step S1, also checks the shape and presence of texture features typical of codes in step S2.

[0045] The question of whether a connected component is large enough to accommodate codes 20a can be estimated from minimum symbol sizes, which indicate how many code modules a smallest assumed code 20a contains, and estimated module sizes. The module size specifies the size of the individual code modules in pixels and is estimated, for example, from previous segmentation results. Smaller connected components cannot be labels 22 with codes 20a, so this feature excludes small light areas such as reflections on transparencies and the like.

[0046] If the code reader 10 is capable of detecting color, further characteristics are conceivable. In particular, there are light areas that are not actually white or light gray, but, for example, yellow, and which therefore usually do not qualify as label 22. However, in a specific application, a particular color for label 22 may be known; in this case, colored, homogeneous areas are even a particularly strong indicator of a label 22.

[0047] In step S3, it is assumed that the remaining related components correspond to labels 22. The areas of interest with potential codes 20a-b from the preceding segmentation are now assigned to labels 22. Alternatively, step S1 is not preceded by segmentation for codes 20a-b, but only by a rough identification of light areas as the basis for step S1, and the code-related segmentation only takes place now and only within each respective label 22. In both cases, groups of areas of interest with possible codes 20a result, with each group being assigned to a recognized label 22.

[0048] In step S4, the decoder now processes the areas of interest label by label 22. This group-based or label-22-based sequence differs from the conventional sequence, which, for example, relies solely on a value used by the segmentation to estimate the probability of a code 20a in the respective area of ​​interest. Such a value can also prioritize the decoding order within a label 22. Similarly, the processing order of the labels 22 can be made dependent on such values, and, for example, a label 22 with the highest value, the highest average value, or similar criteria might be processed first.

[0049] In step S5, the decoder can optionally and exceptionally process areas of interest that do not belong to any label 22. This requires that segmentation has also been performed outside of label 22 and that corresponding individual areas have been found—that is, areas of interest with a likely code 20b outside of label 22. To avoid undermining the effect of the decoding based on label 22, these individual areas must be particularly promising, for example, by exhibiting an exceptionally high value from the segmentation. Step S5 does not necessarily have to follow step S4. An individual area can also be decoded first or in between.This is particularly interesting in the case of a line scan camera, where it can easily happen that a lone individual has already been detected, but not yet a complete label 22, so that the time can be used to decode the lone individual in advance.

[0050] As an alternative or supplement to the described process of Figure 2 In addition to purely classical image processing, the use of machine learning methods or a neural network is also conceivable. This applies particularly to steps S1 and S2 for the recognition of labels 22. The machine learning method is trained with images in which the position of labels 22 is known, so that these images can serve as positive examples and, simultaneously, as negative examples in areas without labels 22, along with appropriate annotations for supervised learning.

[0051] Figure 3a-eTo further illustrate the procedure just described, some example images at different stages of label recognition are shown (22). First, the Figure 3a an unedited original image. Figure 3b Figure 2 illustrates a texture, here from a foil, that can be confused with codes 20a-b and which would therefore require a great deal of unnecessary decoding time for false-positive areas of interest using conventional segmentation and decoding methods. Since these textures are not located on a label 22 and are not assigned exceptionally high values ​​by the segmentation, the invention is able to exclude such areas of interest from decoding. Figure 3c Structures brighter than a certain brightness threshold are highlighted in white. Figure 3dThis was further processed with an erosion filter using a 5x5 pixel filter kernel. Without a separate image, a minimum size, an approximately rectangular shape, and the presence of textures for code structures were then required, and connected components were formed from the remaining light structures. Only the four in Figure 3e The candidates marked for label 22 are three of which are also recognizable as label 22 with the naked eye. The elongated, light-colored adhesive strip is still correctly considered label 22 under the discussed criteria, although this is not objectively correct, and could be further filtered out based on an aspect ratio or a stricter specification for code-like structures.

[0052] Figure 4Figure 1 shows an exemplary flowchart for identifying a label 22 using a label template and for the subsequent decoding. The individual steps are only mandatory if explicitly described as such. In this embodiment, the system does not rely solely on general image features to recognize labels 22. Instead, the label templates incorporate prior knowledge about specific expected types of labels 22. This prior knowledge allows for even more optimized use of the available decoding time. For example, once the codes 20a located on the recognized label 22 according to the label template have been read, further reading efforts on that label 22 can be discontinued, as any other areas of interest on that label 22 must be false positives. Furthermore, highly targeted, particularly promising retries can be performed.Finally, additional output information can be made available, indicating which labels 22 were captured, for example through configurable names or numbers of the label templates.

[0053] A label template comprises various pieces of information that describe a label 22 of the relevant type. For implementation reasons, this preferably includes a designation, which is preferably descriptive and refers, for example, to the manufacturer of the label 22. Preferably, it is further specified how many codes 20a are to be decoded on a label 22 of the corresponding type and what their code type is. Optionally, symbol sizes or code lengths can be specified for each code type. To support the decoder, prior knowledge for certain codes can be stored; for this, reference is made to the aforementioned documents EP 3 428 835 A1 and EP 4 258 160 A1. Finally, redundancies can be specified, which are exploited in a later embodiment. Optionally, label properties can be specified according to the requirements. Figure 2The image features presented must be included, for example, that a label of the corresponding type is yellow and rectangular with an aspect ratio of 3:2. This list is not exhaustive. In particular, code positions will be added later as another possible property in a label template. Examples of simple label templates are therefore: Type 1 - 4x DMx redundant, 1x C128, or Type 2 - 6x EAN129 redundant.

[0054] Label templates are defined by the user, either through parameterization, as a data field, or via a graphical user interface that, where possible, makes semi-automatic suggestions. It is also conceivable to automatically learn label templates from existing examples, possibly based on user suggestions and / or with their modifications.

[0055] In step S11 of the process according to Figure 4A label 22 has already been identified, particularly based on the procedure that corresponds to steps S1 and S2 of the Figure 2 As already explained, decoding is delayed until this recognition is complete and the areas of interest on labels 22 are known through segmentation, with a possible exception for isolated codes, i.e., codes 20b outside of labels 22, which are recognized with high reliability as codes 20b and which may be decoded in the meantime.

[0056] In step S12, the areas of interest are grouped into labels 22 and sorted according to their numerical values. This sorting can also be done sequentially, for example, by identifying only the area of ​​interest with the highest numerical value.

[0057] In step S13, the decoder processes the relevant areas of a label 22 according to their sorting order. Decoding is therefore performed in groups for each label 22; see also the description of step S14. Figure 2 .

[0058] In step S14, it is checked whether a first, or in later iterations, a further Code 20a could be read on a label 22. If no Code 20a can be read, then no comparison with label templates is possible; the remaining decoding time must then be allocated using conventional methods until a Code 20a can be read or no more decoding time is available.

[0059] In step S15, it is checked whether a label template can be identified and, consequently, whether all codes 20a on label 22 have been read. This then constitutes a complete success (Good Read) for label 22. In step S16, information about the identity of label 22 and all code contents of the codes 20a on it can be output. If there is another label 22, it will be read without an explicit arrow being displayed in the process. Figure 4 Decoding continues; otherwise, wait for new image data.

[0060] In step S17, as an alternative to fully identifying and processing label 22, it is checked whether there are any candidates among the label templates that are partially matched by the previously read codes 20a. If this is not the case, label 22 is classified as unknown in step S18. If decoding time is still available, an attempt can be made to continue decoding with further areas of interest. However, this cannot change the result that label 22 is unknown and does not match any label template. It is, however, quite possible that all codes of even an unknown label 22a can be successfully decoded.

[0061] If, on the other hand, at least one label template matches the previous read results, then the most suitable or a randomly selected one of these label templates is provisionally assigned, and in step S18, areas of interest on label 19 are derived from it that promise a particularly high probability of a successful read on a retry. The process then iterates to step S13 to read as many codes 20a as possible. If no more decoding time is available, the process is aborted at step S18, leaving an unknown label 22 and only partially processed areas of interest on this label 22.

[0062] Figure 5 Figure 1 shows an example image of a lone code, i.e., a code 20b, which is applied directly to a package and not to a label 22. This illustrates an exception to the decoding based solely on label 22, which was already discussed in step S5 of the Figure 2was explained and which also in Figure 4 can be supplemented. An attempt is then made to decode a certain number of lone wolves and / or lone wolves with a particularly high value, preferably during image acquisition and label recognition 22, otherwise by allocating a certain proportion of the subsequently available decoding time.

[0063] Figure 6 Figure 1 shows an example image of a label with multiple codes to illustrate the specification of code positions on a label using rays with angles and lengths. This is a particularly advantageous embodiment for a further embodiment of the invention in which label templates contain information about the internal geometry of a label. Particularly interesting geometric properties are the shape of the label and the positions of the codes on the label.

[0064] The geometric properties must first be learned. As already described for the other information of a label template, this can be parameterized manually, preferably supported by an input mask, an editor, or a graphical user interface in which, for example, code positions on a sample image are manually marked.

[0065] Preferably, however, the positions are captured automatically. Suitable example images of a label for a label template can be presented specifically, for example, during commissioning. Alternatively, label templates can be completed or updated with labels 22 based on any objects not specifically selected for learning, either before or during operation. A label template already exists for this purpose, but it is not yet fully populated. If a snapshot is then assigned to the label template, such as... Figure 4 If the codes are described, their position can be determined based on the codes read, and the corresponding information can be added or modified in the label template.

[0066] The positions should preferably be stored in the label template in a rotation- and scaling-invariant manner, because during reading, labels are read in any orientation and from various distances. The positions of all codes in a label template can be advantageously defined with the desired invariances if an arbitrarily selected code from the label template is chosen as the reference code. This is in Figure 6This is illustrated. Reference point 30 is the center of gravity of the reference code. The scan direction 32, perpendicular to the bars of the reference code, can serve as an anchor for orientation. For a 2D code, its horizontal orientation, for example, can be used analogously. Alternatively, the outer shape 34 of the label 22 could also be used, but the scan direction 32 is more stable and is known anyway during code reading. Starting from this reference point, rays 36 are now drawn through the centers of gravity 38 of the other codes of the label, whose orientation is described, for example, by a respective angle to the scan direction 32. This fulfills the condition of rotational invariance. To also achieve scale invariance, the distance from reference point 30 to another code along the associated ray 36 can be measured in units of the module sizes of the reference code.This module size is estimated during the decoding of the reference code and is even known exactly after its successful encoding.

[0067] To teach the positions for a label template, the codes 20a of a label 22 are first read from a sample image. One of the codes 20a is selected as the reference code, and its center point is defined as the reference point 30. The angle to the scan direction 32 of the reference code is determined for the rays 36 or connecting lines to the centers of the other codes 20a. The distances along the rays 36 are first measured in pixels and then converted into units of the module size of the reference code, which is known from decoding the reference code. Perspective effects, for example in the case of side reading, can be corrected using the shape 34 of the label 22 or that of the codes 20a.

[0068] Accordingly, the label templates store positional data for each code, such as an angle and a distance in units of the module size of a reference code, as well as the identity of the reference code. Multiple such positional data points can be stored for each code, corresponding to different reference codes. This redundancy allows the system to select which currently read code can be used as the reference code. Furthermore, it is conceivable to capture and store additional geometric information in the label template, such as distances to the edges of label 22 or the sizes of adjacent codes in units of their own module size.

[0069] Figure 7a This shows an illustration of the shape and position-related information of a label template. The same reference code is selected as in Figure 6. The positions of the other codes 20a are now simply marked by a cross, although the described rotation- and scaling-invariant position data are still preferably used.

[0070] Figure 7b shows an illustration of the test to determine whether a label template complies with the requirements of the... Figure 7a The information shown, relating to shape and position, matches a recorded label 22a. In order to be able to apply a label template at all, there must be at least one hypothesis as to which label template might fit, for example, as described in steps S15 and S17 of the Figure 4The position data of the label template and the centers of gravity of the 38 read codes 22a of label 22 can be used in various ways. Target positions of the label template can be compared with the actual positions of the read codes 20a to confirm the hypothesis about the fit of the label template to label 22. Once the identity is established, the actual positions can be used to complete or update the label template. And, starting from a successfully read reference code, it is possible to predict where further codes should be located.

[0071] The latter can be used to perform targeted read retries specifically at the expected positions of codes. This allocates additional decoding time to those areas of interest that are not false positives.

[0072] Further segmentation can be performed to obtain better areas of interest at the expected code positions, or to create areas of interest that were overlooked in the earlier segmentation. Based on the label template and after reading the reference code, prior knowledge is available regarding the location of an area of ​​interest within the missing code. With an area of ​​interest identified or corrected using this prior knowledge, further reading attempts can then be made. It is helpful for post-segmentation if the label template contains information about the extent of its codes. This, like the position information, can be specified in units of module sizes, making it scale-invariant. This primarily refers to the vertical direction, as the horizontal direction varies depending on the code content.

[0073] Finally, there is the option to verify labels 22, i.e., to provide feedback on misprints or similar issues. Often, codes 20a are applied too close to the edge of the label 22. If this results in missing code elements that were never printed within the area of ​​the label 22, the code 20a may become unreadable. Such positioning errors are detected using the label template in order to correct corresponding printing errors in future labels 22.

[0074] Figure 8Figure 1 shows an example image of a label 22 with four redundantly applied DMx codes 40 to illustrate a possible extension of the embodiments described so far. Such redundancies are common, and the label stencil can contain corresponding information. If one of the redundant codes 40 is successfully read, unnecessary decoding calls for the other redundant codes 40 can be avoided. The code content is known after the first reading of one of the redundant codes 40 and can be directly transferred to the remaining redundant codes 40.

[0075] In addition to outputting code content, the code reader 10 is often expected to also provide image data as an output. However, this generates a large amount of image data, especially if the original high resolution is maintained. Conversely, reducing the resolution impairs potential subsequent image analysis. Label recognition 22 now allows images to be retained at high resolution for data reduction and cropped precisely to the area of ​​the label 22. Additionally, a full-size image at a reduced resolution can be output.

[0076] It is conceivable not to use the described processing and decoding related to label 22 continuously, but rather to offer it as a label decoding mode in addition to the conventional procedure. The system could then switch to the new mode manually, for example, if it is known that numerous label 22s will follow in the application, or automatically if there are too many false positives of interest in the standard mode.

[0077] For objects that are entirely light-colored or those covered with a label 22 across their entire surface, many advantages of the invention are not realized, since there are no areas of interest outside of the labels 22. In some cases, however, a label 22 is still detected and identified, so that, for example, code positions can still be checked or predicted. If it is not possible to identify the label 22, the entire object is treated as having an unknown label 22, which ultimately leads to the same procedure as conventionally without considering labels 22. However, the decoding can also explicitly revert to the standard mode in such situations.

[0078] As an alternative or supplement, a logo on the label can be used for identification. In this case, the label template contains information about the logo, such as a sample image of the logo or descriptive characteristics that allow the logo to be recognized. The process can then be carried out by Figure 4 This differs because a label is identified by its logo and not by a read code. It is also conceivable that identification via read codes and logo complement each other or, depending on the situation, replace each other. Once a label, with at least its rough geometry, has been found within the image data, as in... Figure 2As explained, for each potentially suitable label template, hypotheses can already be derived regarding where within the label the respective logo should be located. A comparison between the expected logo and the image information within the label area can therefore be carried out very specifically and only for a few small image sections.

[0079] Once a logo has been recognized and a label identified via its label template, the position of the logo on the label template can be used to predict where the relevant areas of the codes on the label are located. The logo thus replaces or supplements the read codes, which are stored in the label. Figure 4The logo can be used as an anchor or reference code. The mechanisms described above can be used analogously to locate the areas of interest in a rotation- and scale-invariant manner, starting from the logo as an anchor. The logo can thus simplify and improve the location and reading of codes in many situations. Furthermore, it is conceivable that a label can be identified by its logo even in the unfavorable situation where not a single code on it is readable. This facilitates troubleshooting and, if necessary, manual re-decoding.

Claims

1. An optoelectronic code reader (10) for reading optical codes (20a-b) that has at least one light reception element (24) for generating image data from received light and a control and evaluation unit (26) that is configured to segment the image data to locate regions of interest having assumed optical codes (20a-b) and to process the regions of interest by a decoding process to read the code content of an optical code (20a-b) in the region of interest, characterized in that the control and evaluation unit (26) is further configured to recognize a label (22) in the image data, to assign a label (22) to the regions of interest, and thus to distinguish between regions of interest that are part of a label (22) and those that are not, and to process a region of interest differently by the decoding process when the region of interest is part of a label (22) than when the region of interest is not part of a label (22).

2. A code reader (10) in accordance with claim 1, wherein the control and evaluation unit (26) is configured to group the regions of interest by labels (22).

3. A code reader (10) in accordance with claim 1 or claim 2, wherein the control and evaluation unit (26) is configured to determine a score for the regions of interest on how reliable an optical code (20a-b) has been detected in the region of interest, and to take account of the score in the order of the processing of the regions of interest by a decoding process.

4. A code reader (10) in accordance with claim 3, wherein the control and evaluation unit (26) is configured to assign a higher score to regions of interest that are part of a label (22).

5. A code reader (10) in accordance with any one of the preceding claims, wherein the control and evaluation unit (26) is configured to process regions of interest of a label (22) directly after one another by a decoding process so that regions of interest of a label (22) are worked through as a group.

6. A code reader (10) in accordance with any one of the preceding claims, wherein the control and evaluation unit (26) is configured to recognize a label (22) using its brightness and homogeneous texture.

7. A code reader (10) in accordance with any one of the preceding claims, wherein the control and evaluation unit (26) is configured to recognize a label (22) with the aid of at least one of the following criteria: dark structures, shape, size, code structures contained, orientation of the code structures within the label (22).

8. A code reader (10) in accordance with any one of the preceding claims, wherein the control and evaluation unit (26) has access to at least one label template that characterizes a known label type, with a label template having a plurality of optical codes (20a) present on this label type, in particular including the respective code type and / or position of the optical code (20a) within a label (22).

9. A code reader (10) in accordance with claim 8, wherein the control and evaluation unit (26) has access to at least one label template that has at least one logo.

10. A code reader (10) in accordance with claim 8 or claim 9, wherein the control and evaluation unit (26) is configured for an automatic teaching in which a label template or a property of a label template is taught from detected image data.

11. A code reader (10) in accordance with any one of the claims 8 to 10, wherein the control and evaluation unit (26) is configured to identify a label (22) with reference to a label template, in particular with the aid of at least one already decoded optical code (20a) of a region of interest of the label (22) or of a logo.

12. A code reader (10) in accordance with claim 11, wherein the control and evaluation unit (26) is configured to process regions of interest in an at least partially identified label (22) for so long by a decoding process until all the optical codes (20a) of the label (22) have been decoded.

13. A code reader (10) in accordance with any one of the claims 8 to 12, wherein the control and evaluation unit (26) is configured to transfer the code content of a region of interest having one of the plurality of optical codes (20a, 40) of the same code content to at least one further region of interest in the case that a plurality of optical codes (20a, 40) of the same code content (20) are present in a label (20) in accordance with the label template.

14. A code reader (10) in accordance with any one of the claims 8 to 13, wherein the control and evaluation unit (26) is configured to adapt the regions of interest of a label (22) with the aid of a label template.

15. A method of reading optical codes (20a-b) in which image data from received light are generated, the image data are segmented to locate regions of interest having assumed optical codes (20a-b), and the regions of interest are processed by a decoding process to read the code content of an optical code (20a-b) in the region of interest, characterized in that a label (22) is recognized in the image data, in that the regions of interest is assigned a label (22), and a distinction is thus made between regions of interest that are part of a label (22) and those that are not, and in that a region of interest is processed differently by the decoding process when the region of interest is part of a label (22) than when the region of interest is not part of a label (22).