READING OPTICALLY READABLE CODES
Patent Information
- Application Number
- DE502022006894
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-02-10
- Filing Date
- 2022-12-15
- Publication Date
- 2026-02-19
- Estimated Expiration
- 2042-12-15
AI Technical Summary
Optically readable codes embedded in the surfaces of objects, particularly on plant and animal products, are difficult to read due to distortions, reflections, and surface irregularities, making them challenging for consumers to decode using smartphones.
A method and system utilizing a trained machine learning model to perform transformations on images of optically readable codes, reducing distortions and reflections by learning from reference images to generate clearer, more readable codes.
The method significantly reduces decoding errors by optimizing code contrast and eliminating distortions, enabling reliable reading of embedded codes on various objects.
Description
[0001] The present invention relates to the technical field of marking objects with optically readable codes and reading (decoding) these codes. The invention comprises a method, a system, and a computer program for decoding optically readable codes embedded in the surfaces of objects.
[0002] Tracking goods and merchandise plays an important role in many sectors of the economy.
[0003] GB2585938A discloses a method for object identification that includes the detection of an object candidate in an image and the comparison of candidate data with locally stored templates.
[0004] Goods and merchandise, or their packaging and / or containers, are often marked with a unique identifier (e.g., a serial number) (serialization) in order to track them along their supply chain and to be able to record incoming and outgoing goods automatically.
[0005] In many cases, the identifier is applied to the goods and merchandise or their packaging and / or containers in an optically readable form, e.g. in the form of a barcode (e.g. EAN-8 barcode) or a matrix code (e.g. QR code, Data Matrix code).
[0006] US20050082370A1 discloses a system and method for decoding barcodes from images taken with a mobile device.
[0007] H. Pu et al. reveal a method for deblowing QR codes using a neural network (H. Pu et al.: Quick response barcode deblurring via double convolutional neural network, Multimed Tools Appl (2019) 78: 897-912).
[0008] Behind the barcode or matrix code is often a serial number, which usually provides information about the type of goods or product and its origin.
[0009] For some pharmaceutical products, even individual labeling of individual packages is required. According to Article 54a(1) of Directive 2001 / 83 / EC, as amended by the so-called EU Falsified Medicines Directive 2011 / 62 / EU (FMD), at least prescription medicines must be marked with an individual identifier (English: ). Unique Identifier ) to mark, which in particular enables verification of authenticity and identification of individual packages.
[0010] Plant and animal products are usually only marked with a serial code. For plant and animal products, marking is usually applied to packaging and / or containers, for example in the form of stickers.
[0011] In some cases, identifiers are also applied directly to plant or animal products. For example, eggs in the European Union are marked with a producer code from which the hen's farming method, the country of origin, and the farm where the egg was produced can be deduced. Increasingly, the peels of fruit and vegetables are also being marked (see, for example, E. Etxeberria). et al.: Anatomical and Morphological Characteristics of Laser Etching Depressions for Fruit Labeling, 2006, HortTechnology. 16, 10.21273 / HORTTECH.16.3.0527). Direct marking of plant or animal products (without the use of stickers) conserves resources and serves to prevent counterfeiting, since an identifier incorporated into the surface of a plant or animal product cannot be removed and / or replaced as easily as a sticker or a print.
[0012] Consumers are showing increasing interest in the origin and supply chain of plant and animal products. For example, they want to know where the product comes from, whether and how it was treated (e.g., with pesticides), how long the transport took, what conditions prevailed during transport, and / or similar information.
[0013] EP3896629A1 proposes marking plant and animal products with a unique identifier in the form of an optically readable code. Consumers can read this code, for example, using their smartphone camera. Based on the read code, consumers can access various information about the plant or animal product. EP3896629A1 suggests embedding the optically readable code into the surface of the plant or animal product, for example, using a laser. An optical code embedded in the surface of a plant or animal product has lower contrast with the surrounding tissue and is therefore more difficult to read than, for example, an optical code printed in black ink on a white background, as is commonly found on stickers or tags.Furthermore, optical codes appear differently on different objects. For example, if optically readable codes are applied to curved surfaces, such as those found on many fruits and vegetables, using a laser, the codes can become distorted, making them difficult to read. Similarly, if optically readable codes are applied to smooth surfaces, reflections (e.g., from ambient light) can occur during the reading process, further complicating the reading. The surfaces of fruits and vegetables can also be uneven; for instance, apples may have specks and spots, and potatoes may have irregularities. Such inconsistencies and irregularities can make reading the codes more difficult.
[0014] The task therefore arises to provide means by which optically readable codes on a wide variety of different objects, especially on a wide variety of plant and animal products, can be reliably read by a consumer using simple means, such as a smartphone.
[0015] This problem is solved by the subject matter of the independent claims. Preferred embodiments are found in the dependent claims, the present description, and the drawings.
[0016] A first object of the present invention is a computer-implemented method comprising the steps Receiving at least one image of an object, wherein the object includes an optically readable code, the optically readable code being embedded in a surface of the object; identifying the object based on the at least one image; reading transformation parameters for the identified object from a database; performing one or more transformations of the at least one image according to the transformation parameters, thereby generating a transformed image, wherein the transformation parameters determine which transformation(s) are performed and, in the case of multiple transformations, in which order they are performed, wherein at least one transformation is performed using a trained machine learning model.where performing a transformation using the trained machine learning model comprises: ▪ feeding the at least one image acquisition to the trained machine learning model, ▪ receiving a transformed image acquisition from the trained machine learning model, ∘ where the trained machine learning model was trained on the basis of training data, wherein the training data for each object of a plurality of objects comprises i) at least one reference image acquisition of an optical code incorporated into a surface of the object as input data and ii) a transformed reference image acquisition of the optical code as target data, wherein decoding the optical code in the transformed reference image acquisition produces fewer decoding errors than decoding the optical code in the reference image acquisition,where training for each of the many objects comprises: ▪ Inputting the at least one reference image into the machine learning model, ▪ Receiving a predicted transformed reference image from the machine learning model, ▪ Calculating a deviation between the transformed reference image and the predicted transformed reference image, ▪ Modifying the model parameters with a view to reducing the deviation, ▪ Decoding the optically readable code represented in the transformed image.
[0017] Another object of the present invention is a system comprising at least one processor, wherein the processor is configured to receive at least one image of an object, wherein the object includes an optically readable code, the optically readable code being embedded in a surface of the object, to identify the object based on the at least one image, to read transformation parameters for the identified object from a database, to perform one or more transformations of the at least one image according to the transformation parameters and thereby produce a transformed image, ∘ wherein the transformation parameters determine which transformation(s) are performed, and in the case of multiple transformations, in which order they are performed, ∘ wherein at least one transformation is performed using a trained machine learning model,where performing a transformation using the trained machine learning model comprises: ▪ feeding the at least one image acquisition to the trained machine learning model, ▪ receiving a transformed image acquisition from the trained machine learning model, ∘ where the trained machine learning model was trained on the basis of training data, wherein the training data for each object of a plurality of objects comprises i) at least one reference image acquisition of an optical code incorporated into a surface of the object as input data and ii) a transformed reference image acquisition of the optical code as target data, wherein decoding the optical code in the transformed reference image acquisition produces fewer decoding errors than decoding the optical code in the reference image acquisition,where training for each of the many objects comprises: ▪ Inputting the at least one reference image into the machine learning model, ▪ Receiving a predicted transformed reference image from the machine learning model, ▪ Calculating a deviation between the transformed reference image and the predicted transformed reference image, ▪ Modifying the model parameters with a view to reducing the deviation in decoding the optically readable code represented in the transformed image.
[0018] Another object of the present invention is a computer program product comprising a data carrier on which a computer program is stored, wherein the computer program can be loaded into a computer's main memory and causes the computer to perform the following steps: Receiving at least one image of an object, wherein the object includes an optically readable code, the optically readable code being embedded in a surface of the object; identifying the object based on the at least one image; reading transformation parameters for the identified object from a database; performing one or more transformations of the at least one image according to the transformation parameters, thereby generating a transformed image, wherein the transformation parameters determine which transformation(s) are performed and, in the case of multiple transformations, in which order they are performed, wherein at least one transformation is performed using a trained machine learning model.where performing a transformation using the trained machine learning model comprises: ▪ feeding the at least one image acquisition to the trained machine learning model, ▪ receiving a transformed image acquisition from the trained machine learning model, o where the trained machine learning model was trained on the basis of training data, wherein the training data for each object of a plurality of objects comprises i) at least one reference image acquisition of an optical code incorporated into a surface of the object as input data and ii) a transformed reference image acquisition of the optical code as target data, wherein decoding the optical code in the transformed reference image acquisition produces fewer decoding errors than decoding the optical code in the reference image acquisition,where training for each of the many objects comprises: ▪ Inputting the at least one reference image into the machine learning model, ▪ Receiving a predicted transformed reference image from the machine learning model, ▪ Calculating a deviation between the transformed reference image and the predicted transformed reference image, ▪ Modifying the model parameters with a view to reducing the deviation, ▪ Decoding the optically readable code represented in the transformed image.
[0019] The invention is explained in more detail below, without distinguishing between the subject matter of the invention (system, method, computer program product). Rather, the following explanations are intended to apply analogously to all subject matter of the invention, regardless of the context in which they are made (system, method, computer program product).
[0020] If the present description or the claims mention steps in a sequence, this does not necessarily mean that the invention is limited to that sequence. Rather, it is conceivable that the steps could also be carried out in a different sequence or even in parallel; unless one step builds upon another, which makes it essential that the building step be carried out subsequently (which will be clear in the specific case). The sequences mentioned thus represent preferred embodiments of the invention.
[0021] The present invention provides means for reading an optically readable code. The terms "reading" and "decoding" are used synonymously in this description.
[0022] The term "optically readable code" refers to markings that can preferably be detected with the help of a camera and converted, for example, into alphanumeric characters.
[0023] Examples of optically readable codes include barcodes, stacked codes, composite codes, matrix codes, and 3D codes. Alphanumeric characters that can be read using automated text recognition (text recognition) are also examples. optical character recognition, Codes that can be captured (interpreted, read) and digitized using the abbreviation OCR fall under the term optically readable codes.
[0024] Optically readable codes are a type of machine-readable code, meaning codes that can be captured and processed by a machine. In the case of optically readable codes, such a "machine" typically includes a camera.
[0025] A camera typically comprises an image sensor and optical elements. The image sensor is a device for capturing two-dimensional images of light electrically. It is usually a semiconductor-based image sensor, such as a CCD (centralized CD) sensor. = charge-coupled device ) or CMOS sensor (CMOS = complementary metal-oxidesemiconductor The optical elements (lenses, apertures and the like) serve to create the sharpest possible image of the object from which a digital image is to be taken on the image sensor.
[0026] Optically readable codes (e.g., compared to RFID transponders) have the advantage that they can be read by many consumers using simple means. For example, many consumers own a smartphone equipped with one or more cameras. Using such a camera, an image of the optically readable code can be created on the camera's image sensor. This image can be digitized and processed and / or stored by a computer program located on the smartphone. Such a computer program can be configured to identify and interpret the optically readable code, i.e., to translate it into another form, such as a sequence of numbers, a sequence of letters, and / or the like, depending on the information contained in the optically readable code.
[0027] According to the present invention, the optically readable code is incorporated into the surface of an object. Such an object is a real, physical, tangible object. It can be a natural object or an industrially manufactured object. Examples of objects within the meaning of the present invention are: tools, machine components, circuit boards, chips, containers, packaging, jewelry, design objects, works of art, pharmaceuticals (e.g., in the form of tablets or capsules), pharmaceutical packaging, and plant and animal products.
[0028] In a preferred embodiment of the present invention, the subject matter is a medicinal product (e.g. in the form of a tablet or a capsule).
[0029] In another preferred embodiment of the present invention, the object is an industrially manufactured product such as a component for a machine.
[0030] In another preferred embodiment of the present invention, the subject matter is a plant or animal product.
[0031] A "plant product" is a single plant or part of a plant (e.g., a fruit), or a group of plants or a group of parts of a plant. An "animal product" is an animal, part of an animal, or a group of animals or a group of parts of an animal, or an item produced by an animal (such as a chicken egg). Industrially processed products, such as cheese and sausages, are also included in this description under the term "plant or animal product."
[0032] The plant or animal product is usually a part of a plant or animal that is suitable and / or intended for consumption by a human or an animal.
[0033] Preferably, the plant product is at least part of a cultivated plant. The term "cultivated plant" is understood to mean a plant that is purposefully grown as a crop through human intervention. In a preferred embodiment, the cultivated plant is a fruit plant or a vegetable plant. Although fungi are not biologically classified as plants, fungi, and in particular their fruiting bodies, are also to be included under the term "plant product."
[0034] Preferably, the cultivated plant is one of the plants listed in the following encyclopedia: Christopher Cumo: Encyclopedia of Cultivated Plants: From Acacia to Zinnia, Volumes 1 to 3, ABC-CLIO, 2013, ISBN 9781598847758.
[0035] A plant or animal product could be, for example, an apple, a pear, a lemon, an orange, a mandarin orange, a lime, a grapefruit, a kiwi, a banana, a peach, a plum, a mirabelle plum, an apricot, a tomato, a cabbage (a cauliflower, a white cabbage, a red cabbage, a kale, Brussels sprouts or the like), a melon, a pumpkin, a cucumber, a bell pepper, a zucchini, an eggplant, a potato, a sweet potato, a leek, celery, a kohlrabi, a radish, a carrot, a parsnip, a black salsify, an asparagus, a sugar beet, a ginger root, rhubarb, a coconut, a Brazil nut, a walnut, a hazelnut, a chestnut, an egg, a fish, a piece of meat, a piece of cheese, a sausage and / or the like.
[0036] The optically readable code is embedded in the object's surface. This means that the optically readable code is not attached to the object as a tag, nor is it applied as a sticker, nor is it printed onto the object with dye. Instead, a surface of the object has been modified so that the surface itself bears the optically readable code.
[0037] In the case of plant products and eggs, the surface can be, for example, the shell.
[0038] The optically readable code can be engraved, etched, burned, embossed, and / or otherwise incorporated into the object's surface. Preferably, the optically readable code is introduced into the object's surface (for example, the peel of a fruit or vegetable) using a laser. The laser can alter dye molecules in the object's surface (e.g., bleach and / or destroy them) and / or cause localized burns and / or destruction and / or chemical and / or physical changes to the tissue (e.g., evaporation of water, denaturation of proteins, and / or similar processes), creating a contrast with the surrounding (unaltered) part of the surface.
[0039] The optically readable code can also be applied to the surface of the object using a water jet or sandblasting.
[0040] The optically readable code can also be mechanically introduced into the surface of the object by scratching, piercing, cutting, rasping, scraping, stamping and / or the like.
[0041] Details regarding the marking of articles, in particular plant or animal products, can be found in the prior art (see, for example, EP2281468A1, WO2015 / 117438A1, WO2015 / 117541A1, WO2016 / 118962A1, WO2016 / 118973A1, DE102005019008A, WO2007 / 130968A2, US5660747, EP1737306A2, US10481589, US20080124433).
[0042] Preferably, the optically readable code has been introduced into the surface of the object using a carbon dioxide laser (CO2 laser).
[0043] At least one image is captured of the object or part of the object. This image shows the optically readable code embedded in the object's surface.
[0044] The term "image capture" preferably refers to a two-dimensional representation of the object or a part thereof. Image capture is usually digital. The term "digital" means that the image can be processed by a machine, typically a computer system. "Processing" refers to the known methods of electronic data processing (EDP).
[0045] Digital images can be processed, edited, and reproduced using computer systems and software, as well as converted into standardized data formats, such as JPEG (graphics format of the Joint Photographie Experts Group), PNG ( Portable Network Graphics ) or SVG ( Scalable Vector Graphics ) . Digital images can be visualized using suitable display devices, such as computer monitors, projectors and / or printers.
[0046] In digital image capture, image content is typically represented and stored using integers. In most cases, these are two-dimensional images, which are binary encoded and may be compressed. Digital image captures are usually raster graphics, in which the image information is stored in a uniform grid. Raster graphics consist of a grid-like arrangement of so-called pixels in the case of two-dimensional representations or voxels in the case of three-dimensional representations, each assigned a color or a grayscale value. The main characteristics of a 2D raster graphic are therefore the image size (width and height measured in pixels, also commonly referred to as image resolution) and the color depth. Each pixel in a digital image file is typically assigned a color.The color encoding used for a pixel is defined, among other things, by the color space and color depth. The simplest case is a binary image, where a pixel stores a black-and-white value. In an image whose color is defined by the so-called RGB color space (RGB stands for the primary colors red, green, and blue), each pixel consists of three color values: one for red, one for green, and one for blue. The color of a pixel is created by superimposing (additively mixing) these three color values. The individual color value is discretized, for example, into 256 distinguishable levels called tonal values, which typically range from 0 to 255. The color nuance "0" of each color channel is the darkest. If all three channels have a tonal value of 0, the corresponding pixel appears black; if all three channels have a tonal value of 255, the corresponding pixel appears white.In the implementation of the present invention, digital image recordings are subjected to certain operations (transformations). These operations primarily concern the pixels as so-called spatial operators, as in the case of an edge detector, or the tonal values of the individual pixels, as in the case of color space transformations. There are numerous possible digital image formats and color encodings. For the sake of simplicity, this description assumes that the images in question are RGB raster graphics with a specific number of pixels. However, this assumption should in no way be considered limiting. Those skilled in image processing will understand how to apply the teachings of this description to image files that are in other image formats and / or in which the color values are encoded differently.
[0047] The at least one image capture may also be one or more excerpts from a video sequence.
[0048] The at least one image is captured using one or more cameras. Preferably, the at least one image is captured by one or more cameras of a smartphone.
[0049] Using multiple cameras that view an object from different directions and capture images from different perspectives has the advantage of capturing depth information. This depth information can then be used to derive and / or extract information about any curvature of the object being photographed.
[0050] The at least one image shows the optically readable code embedded in the surface of the object.
[0051] Typically, the appearance of an optically readable code embedded in the surface of an object differs from an optically readable code printed in black ink onto a flat white surface.
[0052] Embedding an optically readable code into the surface of an object typically results in a marking with lower contrast than, for example, an optically readable code printed in black or color on a white sticker. While the contrast can be optimized by customizing the sticker or tag, a black marking (such as a black barcode or matrix code) on a white background offers very high contrast. Such high contrast is not usually achieved when embedding optically readable codes into the surface of an object, particularly in the case of plant or animal products, pharmaceuticals, and / or components. This can lead to difficulties in reading the codes.Furthermore, optically readable codes, which are, for example, etched into the surface of an object using a laser, have different appearances depending on the object. In other words, optically readable codes etched into an apple typically look different from those etched into a banana, a tomato, a pumpkin, or a potato. The specific variety of fruit can also influence the appearance of the optically readable code: optically readable codes in a Granny Smith apple look different from those in a Pink Lady apple.The surface structure of an object, especially a plant or animal product, can also influence the appearance of an optically readable code: the relatively rough surface of a kiwi can make reading an optically readable code just as difficult as unevenness and blemishes in the skin of potatoes and apples. The surfaces of fruits, vegetables, tablets, capsules, and many components are typically curved. If an optical code is applied to a curved surface, it can distort the code; for example, the code may exhibit pincushion or barrel distortion. Such distortions can make code decoding difficult. The degree of distortion usually depends on the degree of curvature.If an optically readable code measuring 2 cm x 2 cm is etched into an apple, the distortion will be greater than if the same code is etched into a melon. Furthermore, distortions will differ between nearly spherical products (e.g., apples, tomatoes, melons) and nearly cylindrical products (e.g., cucumbers). Smooth surfaces (such as those of apples) produce more reflections (e.g., from ambient light) than rough surfaces (such as those of a kiwi).
[0053] The aforementioned properties of the object (unevenness, inconsistencies, stains, curved surfaces, smooth (reflective) surfaces, coloring, texture, surface roughness and / or the like) are also referred to as disruptive factors in this description.
[0054] Interfering factors are characteristics of the object and / or the optically readable code incorporated into a surface of the object that lead to disturbances in at least one image recording.
[0055] Such disturbances include, for example, distortions, light reflections, code elements with different colors and / or sizes, and / or the like.
[0056] Interfering factors and disturbances can lead to elements of an optically readable code, as depicted in the at least one image, not being recognized or being misinterpreted.
[0057] Interfering factors and disturbances can lead to features of the object, as depicted in the at least one image, being interpreted as features of an optical code, even though they are not part of the optically readable code.
[0058] The present invention reduces and / or eliminates one or more disturbances in at least one image acquisition. This is achieved by means of one or more transformations.
[0059] A "transformation" is a function or operator that takes one or more images as input and produces one image as output. The transformation can ensure that an optically readable code, represented by one or more input images, exhibits fewer disturbances in the output image. For example, a transformation can reduce light reflections on the surface of the object in the output image compared to the input image. Similarly, a transformation can make irregularities and / or inconsistencies on the object's surface less noticeable in the output image than in the input image. In general, the transformation reduces the probability of the output image generating a decoding error when decoding the represented optically readable code compared to the input image.
[0060] Examples of transformations are described below.
[0061] Imperfections can be reduced and / or eliminated using classic image processing techniques. One example is the correction of distortion. Optically readable codes applied to curved surfaces often exhibit distortion. This is shown schematically using an example in Fig. 1 shown. Fig. 1 (a) It displays an optically readable code. This is a QR code. The QR code exhibits pincushion distortion. The cause of the pincushion distortion could be a curved surface of the object. In Fig. 1 (b) Four envelopes H1, H2, H3, and H4 were determined (step (110)) that delimit the optically readable code on the four sides. The four envelopes enclose an area. The area within the envelopes carries the optically readable code. The envelopes can be, for example, circular arcs, elliptical segments, polynomials, or the like. In a directory correction, the pincushion-distorted code within the envelopes can be subjected to a transformation (step (120)) that maps the pincushion-shaped area to a square ( Fig. 1 (c) Such corrections are described in image processing textbooks and in scientific publications on the subject (see, for example: J. Park et al.: Lens Distortion Correction Using Ideal Image Coordinates, Consumer Electronics IEEE Transactions on, 55, 2009, 987-991, 10.1109 / TCE.2009.5278053; X. Wang, R. Klette: Geometry Correction of Projected Rectangular Pictures, 2005, http: / / citr.auckland.ac.nz / techreports / 2005 / CITR-TR-167.pdf). Fig. 1 (d) This shows the transformed QR code, which no longer exhibits any distortion. It can be decoded in a further step. The probability of decoding errors occurring during decoding is low for the code shown in Fig. 1 (d) compared to the in Fig. 1 (a) The code shown is reduced.
[0062] Light reflections can also be reduced and / or eliminated using classic image editing techniques.
[0063] "Classical" image processing techniques encompass all measures and operations that do not rely on the use of a machine learning model trained on reference images. Classical image processing techniques can be found in textbooks such as... " Digital Image Processing - An Algorithmic Introduction using Java" by W. Burger and MJ Burge, Springer-Verlag, 2016, ISBN: 9781447166849, 1447166841.
[0064] According to the invention, at least one transformation is carried out using a machine learning model.
[0065] Such a model can be trained in a supervised learning process to generate a transformed image from one or more image recordings.
[0066] A "machine learning model" can be understood as a computer-implemented data processing architecture. The model can receive input data and provide output data based on this input data and model parameters. Through training, the model can learn a relationship between the input data and the output data. During training, the model parameters can be adjusted to produce a desired output for a given input.
[0067] When training such a model, it is presented with training data from which it can learn. The trained machine learning model is the result of the training process. In addition to input data, the training data includes the correct output data (target data) that the model is to generate based on the input data. During training, patterns are recognized that map the input data to the target data.
[0068] During the training process, the input data for the training data is fed into the model, and the model generates output data. The output data is then compared to the target data (so-called target data). Ground Truth Data) are compared. Model parameters are modified so that the deviations between the output data and the target data are reduced to a (defined) minimum.
[0069] During training, an error function (English: loss function The error function can be used to evaluate the predictive quality of the model. It can be chosen to reward a desired relationship between output data and target data and / or penalize an undesired relationship. Such a relationship could be, for example, similarity, dissimilarity, or another type of relationship.
[0070] An error function can be used to report an error (English: loss The goal of the training process is to calculate the error for a given pair of output and target data. This can involve modifying (adjusting) the parameters of the machine learning model to reduce the error to a (defined) minimum for all pairs in the training dataset.
[0071] An error function can, for example, quantify the deviation between the model's output data and the target data for specific input data. If both the output and target data are numbers, the error function can be the absolute difference between these numbers. In this case, a high absolute value for the error function may indicate that one or more model parameters need to be significantly modified.
[0072] For output data in the form of vectors, difference metrics between vectors such as the mean squared error, a cosine distance, a norm of the difference vector such as a Euclidean distance, a Chebyshev distance, an Lp norm of a difference vector, a weighted norm, or any other type of difference metric of two vectors can be chosen as the error function.
[0073] For higher-dimensional outputs, such as two-dimensional, three-dimensional, or even higher-dimensional outputs, an element-wise difference metric can be used. Alternatively or additionally, the output data can be transformed before calculating a loss value, for example, into a one-dimensional vector.
[0074] In this case, the machine learning model can be trained to generate a transformed image from one or more image acquisitions. Training can be performed using training data. The training data can comprise a large number of reference images and transformed reference images. The term "large number" preferably means more than 100. The term "reference image" serves only to distinguish between images used to train a machine learning model and images used when the trained machine learning model is applied, in order to optimize the images for error-free decoding.
[0075] Each reference image can display an optically readable code embedded in the surface of a reference object. One or more reference images serve as a set of input data for the machine learning model. For each set of input data, there is a transformed reference image as target data. The transformed reference image displays the same optically readable code as the one or more reference images of the input dataset. Transformed reference images can be generated by one or more optical image processing experts based on the reference images. The one or more experts can apply one or more transformations to the reference images to produce transformed reference images with less noise.The transformed reference images can make the optically readable codes appear, for example, more distinct, clearer, with higher contrast, less distortion, fewer reflections, and fewer features that can lead to decoding errors than in the case of the untransformed reference images. One or more experts can combine several reference images showing the same optically readable code on the surface of the same object to create a transformed reference image. The goal of these transformations and combinations is to produce transformed reference images that generate fewer decoding errors when reading the depicted optical codes than the untransformed reference images.
[0076] The decoding error can be determined empirically. It can be, for example, the percentage of codes that could not be decoded (correctly). So, if, for instance, 10 out of 100 codes represented in 100 reference images could not be decoded or produced one or more reading errors, then the percentage of codes that could not be decoded correctly is 10%. From the 100 reference images, 100 transformed reference images are created by one or more image processing experts. The codes represented in the transformed reference images are then decoded. If the experts have performed their work correctly, more than 90, ideally all 100, of the 100 codes in the 100 transformed reference images should be decoded correctly.
[0077] Many optically readable codes have error correction mechanisms. The decoding error can also refer to the number of corrected bits in a code.
[0078] One or more image processing experts can define transformations for individual reference images that increase the contrast of the optically readable code compared to its surroundings, and / or reduce / eliminate distortions, and / or reduce / eliminate reflections, and / or reduce / eliminate other characteristics that can lead to decoding errors. Each transformation can be tested; if it does not produce the desired result, it can be discarded, refined, modified, and / or extended with further transformations.
[0079] Examples of transformations include: spatial low-pass filtering, spatial high-pass filtering, sharpening, blurring (e.g., Gaussian blurring), unsharp masking, erosion, median filter, maximum filter, contrast range reduction, edge detection, color depth reduction, grayscale conversion, negative creation, color corrections (color balance, gamma correction, saturation), color replacement, Fourier transformation, Fourier low-pass filter, Fourier high-pass filter, inverse Fourier transformation.
[0080] Some transformations can be achieved by convolving an image in raster graphic form with one or more convolution matrices (English: convolution kernel These transformations are typically performed on square matrices of odd dimensions, which can have different sizes (for example, 3x3, 5x5, 9x9, and / or the like). Some transformations can be represented as a linear system, using discrete convolution, a linear operation. For discrete two-dimensional functions (digital images), the following formula applies to the calculation of discrete convolution: I * x y = ∑ i = 1 n ∑ j = 1 n I x − i + a , y − 1 + a k i j where I *( x, y ) represent the result pixels of the transformed image capture and I The original image capture to which the transformation is applied. a specifies the coordinate of the center point in the square convolution matrix and k(i,j) is an element of the convolution matrix. For 3x3 convolution matrices, n=3 and a=2; for 5x5 matrices, n=5 and a=3.
[0081] The following are transformations and sequences of transformations that, for example in the case of apples, result in the transformed reference image producing fewer decoding errors than the untransformed reference image: Color transformation into intensity-linear RGB signals: Color transformation of the linear RGB signals into, for example, a reflection channel and / or an illumination channel, and at least two color channels to distinguish between the coded and uncoded surface areas. For this purpose, the intensity-linear RGB color signals are combined linearly with each other to achieve the best possible differentiation between coded and uncoded surface areas. Reflection correction is achieved by subtracting the reflection channel from the at least two color channels (additive correction). Illumination correction is achieved by normalizing the at least two color channels to the illumination channel (multiplicative correction). Imperfections in the apple surface are corrected by detecting the imperfections and spatially interpolating them from the surrounding area. The illumination- and reflection-corrected image section is unsharp-masked with a filter mask of a certain extent, so that spatial inhomogeneities, e.g.,The image brightness is well balanced by the curved surface shape of the apple, and the image contrast between encoded and unencoded areas is enhanced and optimized by increasing the high-frequency image components.
[0082] Further examples of transformations can be found in the numerous publications on the subject of digital image processing.
[0083] If a large number of transformed reference images with less noise than the original (untransformed) reference images have been generated, the training data can be used to train the machine learning model. The (untransformed) reference images are fed into the model as input data. The model is configured to generate an output image from one or more reference images (as input data). The output image is then compared to a transformed reference image (the target data). The deviations between the output image and the transformed reference image can be quantified using an error function.
[0084] The identified errors can be used to adjust the machine learning model parameters to reduce them. Once the errors reach a predefined minimum, the model is trained and can be used to generate new transformed images based on previously unused images. "New" in this context means that the corresponding images were not used during the model's initial training.
[0085] The machine learning model can, for example, be or include an artificial neural network.
[0086] An artificial neural network comprises at least three layers of processing elements: a first layer with input neurons (nodes), an N-th layer with at least one output neuron (node), and N-2 inner layers, where N is a natural number and greater than 2.
[0087] The input neurons are used to receive one or more image images. The output neurons are used to output transformed image images.
[0088] The processing elements of the layers between the input neurons and the output neurons are interconnected in a predetermined pattern with predetermined connection weights.
[0089] Training a neural network can be performed, for example, using a backpropagation method. The goal is to achieve the most reliable possible mapping of given input data to given output data. The quality of the mapping is described by an error function. The aim is to minimize this error function. In the backpropagation method, the artificial neural network is trained by changing the connection weights.
[0090] In the trained state, the connection weights between the processing elements contain information regarding the relationship between image acquisitions and transformed image acquisitions.
[0091] A cross-validation method can be used to split the data into training and validation datasets. The training dataset is used for backpropagation training of the network weights. The validation dataset is used to verify the prediction accuracy of the trained network when applied to unknown data.
[0092] In a particularly preferred embodiment, the machine learning model comprises a generative, adversarial network (English: generative adversarial network, Abbreviation: GAN). Details on these and other artificial neural networks can be found in the state of the art (see, e.g., M.-Y. Liu). et al.: Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications, arXiv:2008.02793; J. Henry et al.: Pix2Pix GAN for Image-to-Image Translation, DOI: 10.13140 / RG.2.2.32286.66887).
[0093] Once the machine learning model is trained, it can be used to generate new transformed image recordings based on new image recordings.
[0094] The new image (or several new image acquisitions) are fed into the trained model and the model generates a transformed image.
[0095] In the transformed image, the optically readable code is more easily recognizable and readable than in the original (non-transformed) image (or the original images in the case of multiple images), i.e., the probability of a decoding error occurring is reduced.
[0096] It is conceivable that a machine learning model is trained to perform a defined task, such as contrast optimization, directory correction, reduction or elimination of light reflections, and / or the like. It is conceivable that several machine learning models exist, each trained independently to perform a defined task. It is also conceivable that these independently trained machine learning models are combined to perform several defined tasks sequentially. For example, the machine learning models can be connected in series, so that the transformed image from one machine learning model serves as input for the next model in the series.Preferably, the training of the models is also carried out in such a way that the transformed image image generated by one model serves as input for the model immediately following in the sequence.
[0097] However, it is also conceivable that a machine learning model is trained to perform several defined tasks (e.g., simultaneously).
[0098] As described, several transformations of the received image are performed, with at least one transformation being carried out using a trained machine learning model.
[0099] It is conceivable to combine transformations based on classical image processing techniques with transformations using a machine learning model. In a first step, one or more transformations based on classical image processing techniques could be performed, followed by one or more transformations using one or more machine learning models. Alternatively, one or more transformations using one or more machine learning models could be performed in a first step, followed by one or more transformations based on classical image processing techniques. Alternating between classical image processing methods and the use of one or more machine learning models is also possible.
[0100] In a preferred embodiment, one or more transformation parameters are identified in a first step based on the at least one image acquisition. The one or more transformation parameters then determine which transformation(s) are performed, and, in the case of multiple transformations, in which order they are performed.
[0101] The determination of the transformation parameters can be carried out in two steps: in a first step, the object that is (at least partially) depicted in the first image can be identified; in a second step, the transformations to be carried out, and if necessary the order of the transformations, can then be read from a data storage device.
[0102] The object can be identified, for example, by its characteristic properties depicted in the initial image. Such characteristic properties include, for example, its color and / or color distribution, shape, size, texture, and / or other features. Pattern recognition techniques, as known to those skilled in image processing, can be used to identify the object based on these depicted characteristic features.
[0103] Preferably, a trained machine learning model is used to identify the object. Such a model for identifying the object is also referred to as a recognition model in this description.
[0104] Such a model can be trained in a supervised learning process to output information, based on at least one image capture, indicating what object is depicted in the at least one image capture.
[0105] The machine learning model can be designed (configured) as a classification model that assigns at least one image as input data to a specific object as output data. For example, the model can output a number that represents the specific object depicted in the image.
[0106] Such classification models are described in the prior art (see e.g. F. Sultana). et al.: Advancements in Image Classification using Convolutional Neural Network, arXiv:1905.03288v1 [cs.CV], 2019; A. Khan et al.: A Survey of the Recent Architectures of Deep Convolutional Neural Networks, Artificial Intelligence Review, DOI: https: / / doi.org / 10.1007 / s10462-020-09825-6).
[0107] The process of assigning at least one image to the object depicted in the first image is also referred to in this description as "identifying the object." The at least one image can be assigned to the respective object and / or to the respective part of an object depicted in the at least one image. In the case of plant products, the at least one image can also be assigned to a stage of ripeness and / or a visual appearance; for example, in the case of bananas, to a green or yellow banana; in the case of bell peppers, to a green, yellow, or red bell pepper; in the case of an apple, to the specific apple variety, and / or the like.
[0108] Identifying the object in at least one image serves to determine transformation parameters. These parameters specify which transformations, and, in the case of multiple transformations, in what order, are performed.
[0109] The transformation parameters can be stored in a data repository, such as a relational database. The machine learning model can be configured and trained to output an identifier based on at least one image capture, which is linked to the transformation parameters in the database. In other words, using an identifier that represents the object depicted in the at least one image capture, transformation parameters associated with that identifier can be determined from a database and retrieved.
[0110] It is also conceivable that the machine learning model is trained to provide / output the transformation parameters itself.
[0111] In a further preferred embodiment, one or more transformation parameters are determined based on interfering factors and / or disturbances. In a first step, one or more interfering factors and / or disturbances can be identified in the at least one image acquisition. In a second step, one or more transformations can then be selected that reduce and / or eliminate disturbances resulting from the interfering factors in the at least one image acquisition.
[0112] For example, it is conceivable that the first step involves analyzing whether the object depicted in the at least one image has a curvature, such as whether it is round. If a curvature (e.g., a round shape) is identified, this indicates that distortions may occur in the depicted optically readable code. In such a case, a transformation can be selected that corrects the distortion. If one or more light reflections are identified in the at least one image, a transformation can be selected that reduces or eliminates these reflections. If low contrast is identified between the optically readable code embedded in the object's surface and its surroundings, or if the object's surface is generally dark, a transformation can be selected that increases the contrast of the optically readable code against its surroundings.To detect a "dark color", for example, the tonal values of the image elements representing the object can be compared with one or more reference values.
[0113] Selecting a transformation can mean choosing a classic image processing method to reduce and / or remove the noise from the at least one image capture. However, selecting a transformation can also mean choosing a machine learning model that has been trained to reduce and / or remove the noise from the at least one image capture.
[0114] Once a transformed image has been generated from at least one image capture, the code represented in the transformed image can be read (decoded) in a subsequent step. Depending on the code used, there are already existing methods for reading (decoding) the respective code (see, for example: https: / / www.gs1.org / docs / barcodes / GS1 DataMatrix Guideline.pdf).
[0115] The read (decoded) code can contain information about the object on whose surface the code is embedded.
[0116] In a preferred embodiment, the read code comprises a (unique) identifier that allows a consumer to obtain further information about the item, for example, from a database. The read code and / or information associated with the read code can be output, i.e., displayed on a screen, printed on a printer, and / or stored in a data storage device.
[0117] Further information on such an (individual) identifier and the information that can be stored about the object associated with the identifier and displayed to a consumer is described in patent application EP3896629A1.
[0118] The invention is explained in more detail below with reference to the drawings, without limiting the invention to the features and combinations of features shown in the drawings. Statements made with regard to a drawing and / or a feature in a drawing are not limited to that particular drawing, but are intended to apply generally.
[0119] Fig. 2 schematically and exemplarily shows, in the form of a flowchart, the reading of an optically readable code that is embedded in the surface of an object.
[0120] In a first step (210), a digital image I of the object O is generated using a camera C. In a second step (220), the digital image I is fed into a machine learning model (MLM). The MLM model is configured and trained to generate a transformed image I* based on the original image I. In a third step (230), the MLM model provides the transformed image I*, in which an optically readable code (in this case, a QR code) embedded in the surface of the object O has a higher contrast with its surroundings and is therefore more clearly visible and easier to read than the code in the untransformed image I. In a fourth step (240), the optically readable code is extracted, and the extracted code OI is provided.The extracted code OI can be displayed and / or information about the object O can be retrieved from a database using the extracted code OI (e.g., transmitted and displayed).
[0121] Fig. 3 This schematically and exemplarily illustrates, in the form of a flowchart, the creation of a transformed image and the reading of an optically readable code embedded in the transformed image. The starting point is at least one image of an object whose surface contains an optically readable code. In the present example, there are two images, I1 and I2; however, there could be more or fewer. It is conceivable that the images I1 and I2 were created from different perspectives, or that an optical filter (e.g., a polarizing filter or a color filter) was used in one image, while no filter or a different filter was used in the other. In step (310), the images I1 and I2 undergo a first transformation.In the present example, the transformation of the two image acquisitions I1 and I2 generates a first transformed image acquisition I*. In step (320), the transformed image acquisition I* undergoes a second transformation. The second transformation generates a second transformed image acquisition I** from the first transformed image acquisition. In step (330), the second transformed image acquisition I** undergoes decoding, which decodes the optically readable code depicted in the second transformed image acquisition. The result of the decoding is a read code OI. According to the invention, at least one transformation is performed using a machine learning model.In the present example, the first transformation could be distortion correction and / or reduction of light reflections using classical image processing techniques, and the second transformation could be contrast optimization using a machine learning model. However, it is also possible that the first transformation is performed using a machine learning model, and the second transformation is performed either using another machine learning model or using classical image processing techniques.
[0122] Fig. 4 schematically shows, in the form of a flowchart, another example of optimizing at least one image capture with regard to error-free reading of an optically readable code.
[0123] The starting point is at least one image I of an object whose surface contains an optically readable code. In step (410), the at least one image I is fed to an identification unit IU. The identification unit IU can be a recognition unit configured to identify the object depicted in the at least one image I. The object can be identified, for example, using a machine learning model (with a recognition model) that is configured and trained to associate the at least one image with the object depicted in it. Alternatively, the identification unit IU can be a unit configured to detect one or more disturbances and / or interference factors in the at least one image.Transformation parameters can be determined based on the results of the identification unit analysis. These transformation parameters define which transformation(s) are performed and, in the case of multiple transformations, in what order they are performed. In this example, the transformation parameters are stored in a data store DB. The identification unit IU can, for example, output an identifier (step (420)) that identifies the object depicted in the image, or it can output an identifier (step (420)) that identifies one or more disturbances and / or one or more disturbance factors. The transformation parameters can then be determined from the data store based on the identifier (step (430)). In this example, two transformations, T1 and T2, result. The at least one image I is fed into the first transformation, T1, in step (440).The at least one image I supplied to the first transformation T1 in step (440) need not necessarily be the same at least one image I supplied to the identification unit IU. It is conceivable that the at least one image I comprises two images, a first image and a second image, both showing the same object but, for example, at different times. The first image can be supplied to the identification unit IU to determine transformation parameters, and the second image can be supplied to the first transformation T1.For example, it is conceivable that the camera used to generate the at least one image is configured to continuously digitize images falling on the camera's image sensor at a defined rate, generate digital images, and feed these digital images to an identification unit. The identification unit can transmit an identifier to a control unit, which uses this identifier to determine transformation parameters. The determined transformation parameters provide information about the transformation(s) (and, if applicable, the sequence of transformations) that are then applied to one or more of the subsequently generated images. The result of transformation T1 of the at least one image I is a first transformed image, which is fed to the second transformation T2 (step (450)).The result of the second transformation T2 is a second transformed image, which is fed to a decoder D (step (460)). Decoder D is configured to decode the optically readable code represented in the second transformed image. Decoder D outputs the decoded code OI in step (470).
[0124] Fig. 5 This diagram illustrates, schematically and using an example, the creation of a training dataset for training a machine learning model. The training dataset is used to train the machine learning model to perform one or more transformations of image data. The training process is described in... Fig. 6 shown.
[0125] Creating a training dataset can be a manual process performed by one or more experts. In this example, the starting point is a number. n from reference image recordings RI 1 to RI n , where n is an integer, preferably greater than 100. Each reference image shows an optical code embedded in the surface of an object. Preferably, each reference image shows an optically readable code embedded in a different instance of the object. The object may, for example, be an apple; in that case, each reference image preferably shows an optically readable code in different apple instances.
[0126] If the reference object is always the same, such as an apple, then the training data set can be used to train a machine learning model to reduce and / or eliminate disturbances in image recordings of optically readable codes embedded in apples.
[0127] If different reference objects are used, such as different fruits (e.g., apples and pears), the training dataset can be used to train a machine learning model to reduce and / or eliminate noise in images of optically readable codes embedded in various fruits. The more diverse (varied) the reference objects depicted in the reference images, the more training data is required, and the more versatile the trained machine learning model can be. A machine learning model trained solely on reference images of apples of a defined variety will perform less well when used to read codes on bananas than a model trained on reference images of apples of different varieties and bananas.The optically readable code shown in the reference images can be the same or different in all reference images.
[0128] In the present example ( Fig. 5 An expert creates a transformed reference image from a single reference image. As described in this document, multiple reference images can also be combined to form a single transformed reference image. A transformed reference image is created by subjecting the reference image to one or more transformations. The expert determines which transformation(s) are performed and, if multiple transformations are performed, in what order. The goal of the transformation(s) is to create a transformed reference image from at least one reference image that contains fewer disturbances and thus reduces the probability of decoding errors.
[0129] Fig. 6 This diagram schematically illustrates a step in training a machine learning model. Training is based on a training dataset. The training dataset comprises a large number of reference images and transformed reference images. The reference images and transformed reference images form pairs; that is, for each reference image, there is a corresponding transformed reference image. When multiple reference images are combined to create a transformed reference image, there are tuples, where each tuple contains a transformed reference image and a number of reference images used to generate the transformed reference image. For simplicity, this example uses pairs.
[0130] In the present example, such a pair consists of a reference image RI. i and a transformed reference image RI* i shown. The reference image RI i is fed into the MLM machine learning model. The MLM machine learning model is configured with a transformed image acquisition I*. i to generate. The transformed image capture I* i is used with the transformed reference image RI* i The results are compared. Using an error function LF, an error L is calculated, which represents the deviations of the transformed image I*. i from the transformed reference image RI* i The error L can be quantified. For example, it can be used in a backpropagation process to modify the model parameter MP of the machine learning model so that the error is reduced to a (defined) minimum. The described process is repeated for a large number of reference images and transformed reference images until the resulting errors are reduced to a (defined) minimum. Once the machine learning model is trained, it can be used to generate transformed images for new images, where "new" means that the corresponding images have not already been used in training.
[0131] Fig. 7 shows an exemplary and schematic system according to the invention.
[0132] The system (1) comprises a computer system (10), a camera (20), and one or more data storage devices (30). The camera (20) can be used to create images of objects. The camera (20) is connected to the computer system (10) so that the captured images can be transferred to the computer system (10). The camera (20) can be connected to the computer system (10) via a cable connection and / or a wireless connection. A connection via one or more networks is also conceivable. Furthermore, it is conceivable that the camera (20) is an integral part of the computer system (10), as is the case, for example, with today's smartphones and tablet computers.
[0133] The computer system (10) is configured (for example by means of a computer program) to receive one or more image recordings (from the camera or from a data storage device), to produce a transformed image recording, to decode the optical code in the transformed image recording and to output the decoded code and / or to provide information associated with the decoded code.
[0134] The data storage device (30) can contain images, models, model parameters, computer programs, transformation parameters, and / or other information. The data storage device (30) can be connected to the computer system (10) via a cable connection and / or a wireless connection. A connection via one or more networks is also conceivable. Furthermore, the data storage device (30) could be an integral part of the computer system (10). It is also conceivable that multiple data storage devices are present.
[0135] Fig. 8 Figure 10 schematically shows a computer system (10). Such a computer system (10) can comprise one or more stationary or portable electronic devices. The computer system (10) can include one or more components, such as a processing unit (11) connected to a memory (15).
[0136] The processing unit (11) (English: processing unit)The processing unit (11) may comprise one or more processors alone or in combination with one or more memories. The processing unit (11) may be ordinary computer hardware capable of processing information such as digital images, computer programs, and / or other digital information. The processing unit (11) typically consists of an arrangement of electronic circuits, some of which may be implemented as an integrated circuit or as several interconnected integrated circuits (an integrated circuit is sometimes referred to as a "chip"). The processing unit (11) may be configured to execute computer programs, which may be stored in a working memory of the processing unit (11) or in the memory (15) of the same or another computer system.
[0137] The memory (15) can be ordinary computer hardware capable of storing information such as digital images, data, computer programs, and / or other digital information, either temporarily and / or permanently. The memory (15) can be volatile and / or non-volatile and can be permanently installed or removable. Examples of suitable memory include RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, flash memory, a removable computer disk, an optical disc, a magnetic tape, or a combination of the above. Optical discs can include read-only compact discs (CD-ROM), read / write compact discs (CD-R / W), DVDs, Blu-ray discs, and similar media.
[0138] In addition to the memory (15), the processing unit (11) can also be connected to one or more interfaces (12, 13, 14, 17, 18) to display, transmit, and / or receive information. The interfaces can include one or more communication interfaces (17, 18) and / or one or more user interfaces (12, 13, 14). The one or more communication interfaces can be configured to send and / or receive information, e.g., to and / or from a camera, other computers, networks, data storage devices, or the like. The one or more communication interfaces can be configured to transmit and / or receive information via physical (wired) and / or wireless communication links. The one or more communication interfaces can include one or more interfaces for connecting to a network, e.g.,using technologies such as mobile phone, Wi-Fi, satellite, cable, DSL, fiber optic, and / or the like. In some examples, the one or more communication interfaces may include one or more near-field communication interfaces configured to connect devices using near-field communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, infrared (e.g., IrDA), or similar.
[0139] The user interfaces (12, 13, 14) may include a display (14). A display (14) may be configured to show information to a user. Suitable examples include a liquid crystal display (LCD), a light-emitting diode (LED) display, a plasma display (PDP), or similar. The user input interface(s) (12, 13) may be wired or wireless and may be configured to receive information from a user into the computer system (10), for example, for processing, storage, and / or display. Suitable examples of user input interfaces include a microphone, an image or video recording device (e.g., a camera), a keyboard or keypad, a joystick, a touch-sensitive surface (separate from or integrated into a touchscreen), or similar.In some examples, the user interfaces may include automatic identification and data capture (AIDC) technology for machine-readable information. This could include barcodes, radio frequency identification (RFID), magnetic stripes, optical character recognition (OCR), integrated circuit cards (ICC), and similar technologies. The user interfaces may also include one or more interfaces for communication with peripheral devices such as printers and the like.
[0140] One or more computer programs (16) can be stored in memory (15) and executed by the processing unit (11), which is programmed to perform the functions described in this description. The retrieval, loading, and execution of instructions from the computer program (16) can be performed sequentially, with one instruction being retrieved, loaded, and executed at a time. However, the retrieval, loading, and / or execution can also be performed in parallel.
[0141] The system according to the invention can be configured as a laptop, notebook, netbook, tablet PC and / or handheld device (e.g., smartphone). Preferably, the system according to the invention includes a camera.
Claims
1. Computer-implemented method comprising the steps of - receiving at least one image recording (I) of an object (O), wherein the object (O) comprises an optically readable code, wherein the optically readable code is introduced into a surface of the object (O), - identifying the object (O) on the basis of the at least one image recording (I), - reading out transformation parameters for the identified object (O) from a database, - carrying out one or more transformations (T1, T2) of the at least one image recording (I) in accordance with the transformation parameters and generating a transformed image recording (I*) in the process, ∘ wherein the transformation parameters stipulate what transformation / transformations (T1, T2) is / are carried out and, in the case of a plurality of transformations (T1, T2), in what order they are carried out, ∘ wherein at least one transformation (T1, T2) is carried out with the aid of a trained machine learning model (MLM), wherein carrying out a transformation (T1, T2) with the aid of the trained machine learning model (MLM) comprises: ▪ feeding the at least one image recording (I) to the trained machine learning model (MLM), ▪ receiving a transformed image recording (I*) from the trained machine learning model (MLM), ∘ wherein the trained machine learning model (MLM) was trained on the basis of training data (TD), wherein the training data (TD) for each object of a multiplicity of objects comprise i) at least one reference image recording (RIi) of an optical code introduced into a surface of the object as input data and ii) a transformed reference image recording (RI*i) of the optical code as target data, wherein decoding the optical code in the transformed reference image recording (RI*i) generates fewer decoding errors than decoding the optical code in the reference image recording (RIi), wherein the training for each object of the multiplicity of objects comprises: ▪ inputting the at least one reference image recording (RIi) into the machine learning model (MLM), ▪ receiving a predicted transformed reference image recording (I*i) from the machine learning model (MLM), ▪ calculating a deviation between the transformed reference image recording (RI*i) and the predicted transformed reference image recording (I*i), ▪ modifying the model parameters (MP) with regard to reducing the deviation, - decoding the optically readable code imaged in the transformed image recording (I*).
2. Method according to Claim 1, wherein the object (O) is a plant or animal product.
3. Method according to Claim 1, wherein the object (O) is a medicament in the form of a tablet or capsule.
4. Method according to Claim 1, wherein the object (O) is a tool, a machine component, a circuit board, a semiconductor chip, a container, a packaging, a piece of jewellery, a design object or an art object.
5. Method according to any of Claims 1 to 4, wherein for each object of the multiplicity of objects the transformed reference image recording (RI*i) was generated by applying one or more transformations (T1, T2) to the at least one reference image recording (RIi), wherein the one or more transformations (T1, T2) were chosen such that reading out the optically readable code in the transformed reference image recording (RI*i) leads to fewer decoding errors than reading out the optically readable code in the at least one reference image recording (RIi).
6. Method according to Claim 5, wherein the one or more transformations (T1, T2) were determined empirically.
7. Method according to any of Claims 1 to 6, wherein the optically readable code has been introduced into the surface of the object (O) by means of a laser.
8. Method according to any of Claims 1 to 7, furthermore comprising the step of: - outputting the decoded optically readable code (OI) and / or information linked with the decoded code (OI).
9. Method according to any of Claims 1 to 8, wherein identifying the object (O) shown in the at least one image recording (I) is effected by means of a machine learning model (MLM), wherein the machine learning model (MLM) is trained to assign an image recording (I) to the object (O) shown in the at least one image recording (I).
10. Method according to any of Claims 1 to 9, wherein the optically readable code is a matrix code.
11. Method according to any of Claims 1 to 10, wherein the transformations (T1, T2) have one or more of the following effects: - distortion correction, - reducing and / or eliminating light reflections, - increasing the contrast between the optically readable code and its surroundings.
12. Method according to any of Claims 1 to 11, wherein the machine learning model (MLM) is an artificial neural network or comprises such a network.
13. Method according to any of Claims 1 to 12, wherein a portion of the transformations (T1, T2) is effected by means of image processing methods that are not based on the use of a machine learning model (MLM).
14. System comprising at least one processor, wherein the processor is configured - to receive at least one image recording (I) of an object (O), wherein the object (O) comprises an optically readable code, wherein the optically readable code is introduced into a surface of the object (O), - to identify the object (O) on the basis of the at least one image recording (I), - to read out transformation parameters for the identified object (O) from a database, - to carry out one or more transformations (T1, T2) of the at least one image recording (I) in accordance with the transformation parameters and to generate a transformed image recording (I*) in the process, ∘ wherein the transformation parameters stipulate what transformation / transformations (T1, T2) is / are carried out and, in the case of a plurality of transformations (T1, T2), in what order they are carried out, ∘ wherein at least one transformation (T1, T2) is carried out with the aid of a trained machine learning model (MLM) , wherein carrying out a transformation (T1, T2) with the aid of the trained machine learning model (MLM) comprises: ▪ feeding the at least one image recording (I) to the trained machine learning model (MLM), ▪ receiving a transformed image recording (I*) from the trained machine learning model (MLM), ∘ wherein the trained machine learning model (MLM) was trained on the basis of training data (TD), wherein the training data (TD) for each object of a multiplicity of objects comprise i) at least one reference image recording (RIi) of an optical code introduced into a surface of the object as input data and ii) a transformed reference image recording (RI*i) of the optical code as target data, wherein decoding the optical code in the transformed reference image recording (RI*i) generates fewer decoding errors than decoding the optical code in the reference image recording (RIi), wherein the training for each object of the multiplicity of objects comprises: ▪ inputting the at least one reference image recording (RIi) into the machine learning model (MLM), ▪ receiving a predicted transformed reference image recording (I*i) from the machine learning model (MLM), ▪ calculating a deviation between the transformed reference image recording (RI*i) and the predicted transformed reference image recording (I*i), ▪ modifying the model parameters (MP) with regard to reducing the deviation, - to decode the optically readable code imaged in the transformed image recording (I*).
15. Computer program product comprising a data carrier on which a computer program is stored, wherein the computer program, when it is loaded into a main memory of a computer and executed by the computer, causes the computer to execute the following steps: - receiving at least one image recording (I) of an object (O), wherein the object (O) comprises an optically readable code, wherein the optically readable code is introduced into a surface of the object (O), - identifying the object (O) on the basis of the at least one image recording (I), - reading out transformation parameters for the identified object (O) from a database, - carrying out one or more transformations (T1, T2) of the at least one image recording (I) in accordance with the transformation parameters and generating a transformed image recording (I*) in the process, ∘ wherein the transformation parameters stipulate what transformation / transformations (T1, T2) is / are carried out and, in the case of a plurality of transformations (T1, T2), in what order they are carried out, ∘ wherein at least one transformation (T1, T2) is carried out with the aid of a trained machine learning model (MLM), wherein carrying out a transformation (T1, T2) with the aid of the trained machine learning model (MLM) comprises: ▪ feeding the at least one image recording (I) to the trained machine learning model (MLM), ▪ receiving a transformed image recording (I*) from the trained machine learning model (MLM), ∘ wherein the trained machine learning model (MLM) was trained on the basis of training data (TD), wherein the training data (TD) for each object of a multiplicity of objects comprise i) at least one reference image recording (RIi) of an optical code introduced into a surface of the object as input data and ii) a transformed reference image recording (RI*i) of the optical code as target data, wherein decoding the optical code in the transformed reference image recording (RI*i) generates fewer decoding errors than decoding the optical code in the reference image recording (RIi), wherein the training for each object of the multiplicity of objects comprises: ∘ inputting the at least one reference image recording (RIi) into the machine learning model (MLM), ∘ receiving a predicted transformed reference image recording (I*i) from the machine learning model (MLM), ∘ calculating a deviation between the transformed reference image recording (RI*i) and the predicted transformed reference image recording (I*i), ∘ modifying the model parameters (MP) with regard to reducing the deviation, - decoding the optically readable code imaged in the transformed image recording (I*).