Reading optically readable codes

Abstract

The present invention relates to the technical field of marking articles with light-readable codes and reading the codes. The present invention relates to a method, system and computer program product for detecting and interpreting a light readable code that has been introduced into the surface of an article.

Description

Technical Field

[0001] The present invention relates to the technical field of marking optical-readable codes on an object and reading the codes. The protection object of the present invention is a method, a system, and a computer program product for imaging and interpreting an optical-readable code introduced on the surface of an object.

Background Art

[0002] The tracking of goods and products plays an important role in many areas of the economy. Products, goods, their packaging and / or containers are provided with unique identifiers (e.g., serial numbers) (serialization) so that they can be tracked across the supply chain and the incoming and outgoing products can be imaged by machines.

[0003] In many cases, the identifier is added to products, goods, their packaging and / or containers in an optically readable form, such as in the form of a barcode (e.g., EAN-8 barcode) or a matrix code (e.g., QR code (registered trademark)) (see, for example, European Patent Application Publication No. 3640901). Barcodes and matrix codes often conceal a serial number that provides information about the type and origin of the product or goods.

[0004] In some cases of pharmaceutical products, it is even necessary to individually mark each individual package. In accordance with Article 54a(1) of Directive 2001 / 83 / EC, as amended by the so-called EU Falsified Medicines Directive 2011 / 62 / EU (FMD), at least for medical products subject to a prescription, it is necessary to mark them using individual recognition features (referred to as unique identifiers) that enable, in particular, checking the authenticity and identifying each individual pack.

[0005] Plant and animal products are typically only provided with serial codes. For plant and animal products, markings are usually added or attached to the packaging and / or containers, for example, in the form of stickers or imprints.

[0006] In some cases, identifiers are directly added to plant or animal products. For example, in the European Union, eggs are assigned producer codes, from which it is possible to deduce that the producer is a poultry farmer, the country of origin of the eggs, and the producer of the eggs. The peels of fruits and vegetables are also being marked (see, for example, E. Etxeberria et al., "Anatomical and Morphological Characteristics of Laser Etching Depressions for Fruit Labeling," 2006, HortTechnology.16,10.21273 / HORTTECH.16.3.0527).

[0007] Consumers are increasingly interested in the origin and supply chain of plant and animal products. For example, consumers want to know where each product came from, whether it was processed, and how it was processed (e.g., using crop protection compounds), how long it took to transport, what conditions were prevalent during transport, and / or similar information.

[0008] European Patent Application Publication No. 3896629 proposes a method for providing unique identifiers to plant and animal products in the form of optically readable codes. Consumers can read the codes, for example, using the camera on their smartphones. Based on the read codes, various information about the plant or animal product can be displayed to the consumer. European Patent Application Publication No. 3896629 also proposes introducing optically readable codes to the surface of plant or animal products, for example, using a laser. The contrast of optical codes introduced to the surface of plant or animal products is low relative to the surrounding tissue, making them more difficult to read than optical codes that are added in black on a white background, such as those typically used for stickers or tags. Furthermore, the appearance of optical codes differs depending on the object they are applied to. For example, when optically readable codes are applied by laser to curved surfaces, such as those found in many fruits and vegetable varieties, the codes may become distorted, which can hinder reading. When applying an optically readable code to a smooth surface, reflections from the surface (for example, reflections caused by ambient light) may occur during reading, interfering with the reading process. The surfaces of fruits and vegetables may not be uniform; for example, apples may have bitter pits or spots. Potatoes may have uneven surfaces. Such unevenness and irregularities may interfere with the reading of the code. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] European Patent Application Publication No. 3640901 [Patent Document 2] European Patent Application Publication No. 3896629 [Non-patent literature]

[0010] [Non-Patent Document 1] E. Etxeberria et al., "Anatomical and Morphological Characteristics of Laser Etching Depressions for Fruit Labeling" 2006, HortTechnology.16,10.21273 / HORTTECH.16.3.0527 [Overview of the project] [Problems that the invention aims to solve]

[0011] Therefore, the challenge to be addressed is to provide a means by which consumers can reliably read optically readable codes on multiple different objects, particularly multiple plant and animal products, using simple means such as a smartphone. [Means for solving the problem]

[0012] This problem is solved by the subject matter of the independent claim. Preferred embodiments are shown in the dependent claims, this specification and the drawings.

[0013] The first object of protection according to the present invention is a computer-based method for reading an optically readable code introduced on the surface of an object, The first step of receiving the image recording of the subject, A step of identifying the object based on the first image recording, The steps include: reading the transformation parameters of the identified target from the database, A step of converting a target first image record and / or second image record based on conversion parameters, wherein the converted image record is obtained. The steps include decoding the optically readable code that is imaged in the converted image recording and This is a method that provides [something].

[0014] Further protections of the present invention extend to systems comprising at least one processor, wherein the processor is: Receiving a first image record of an object, the object comprising an optically readable code, the optically readable code being introduced onto the surface of the object, and Identifying the object imaged in the first image record, and Reading conversion parameters of the identified object from a database, and Converting the first image record and / or a second image record of the object according to the conversion parameters, whereby a converted image record is obtained, and Decoding the optically readable code imaged in the converted image record A system configured to perform.

[0015] A further subject of protection of the present invention is a computer program product comprising a data carrier storing a computer program, the computer program being loadable into the main memory of a computer system, the computer program comprising Receiving a first image record of an object, the object comprising an optically readable code, the optically readable code being introduced onto the surface of the object, the step of Identifying the object based on the first image record, and Reading conversion parameters of the identified object from a database, and Converting the first image record and / or a second image record of the object based on the conversion parameters, whereby a converted image record is obtained, the step of Decoding the optically readable code imaged in the converted image record and Causing a computer system to execute. A computer program product.

Brief Description of the Drawings

[0016] [Figure 1] Schematically shows, as an example in the form of a flowchart, the reading of an optically readable code introduced onto the surface of an object. [Figure 2]The system according to the present invention is schematically shown as an example. [Figure 3] A computer system is schematically shown.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, the present invention will be described in more detail without distinguishing the protection objects (systems, methods, computer program products) of the present invention. Furthermore, it should be noted that the following description applies equally to all protection objects of the present invention regardless of the context (system, method, computer program product) in which the description is made.

[0018] If steps are described in a predetermined order in this specification or claims, this does not necessarily mean that the present invention is limited to the described order. Furthermore, it is conceivable that the steps are executed in a different order or in parallel with each other. An exception is when a given step is based on another step, according to which the step based on the previous step must be executed next (however, this will become clear in individual cases). Therefore, the described order is a preferred embodiment of the present invention.

[0019] In the present invention, means for reading an optically readable code are provided. In this specification, the terms "reading" and "decoding" are used synonymously.

[0020] The term "optically readable code" is understood to mean, for example, something marked that can be imaged using a camera and converted into alphanumeric characters.

[0021] Examples of optically readable codes are barcodes, stacked codes, composite codes, matrix codes, 3D codes. Alphanumeric characters that can be captured (interpreted, read) and digitized by automatic text recognition (abbreviation: OCR, also known as optical character recognition) are also included in the term "optically readable code".

[0022] Readable codes belong to the category of codes that can be read by machines, that is, codes that can be captured and processed by machines. In the case of optically readable codes, such "machines" typically include cameras.

[0023] A camera typically comprises an image sensor and optical elements. An image sensor is a device that records a two-dimensional image from light using electrical means. This usually includes semiconductor image sensors, such as CCDs (Charge-Coupled Devices) and CMOS sensors (Complementary Metal-Oxide-Semiconductors). Optical elements (lenses, apertures, etc.) are used to maximize the sharpness of the image being digitally recorded on the image sensor.

[0024] Optically readable codes have the advantage of being easily readable by many consumers. In this regard, many consumers, for example, own smartphones equipped with one or more cameras. Using such cameras, it is possible to generate an image representation of the optically readable code on an image sensor. A computer program stored in the smartphone can digitize, process, and / or store this image representation. Such a computer program can be configured to identify and interpret the optically readable code, that is, to convert it into various forms, such as sequences of numbers, characters, and / or similar, depending on the information present in the form of the optically readable code.

[0025] The optically readable code is introduced onto the surface of an object. Such objects are real, physical, tangible objects. Such objects can be natural or industrially manufactured. Examples of objects included in the significance of this invention are tools, machine parts, circuit boards, chips, containers, packaging, jewelry, design objects, works of art, pharmaceuticals (e.g., pharmaceuticals in tablet form), pharmaceutical packaging, and plant and animal products.

[0026] In one preferred embodiment of the present invention, the subject is a drug (for example, a drug in the form of a tablet or capsule) or drug packaging.

[0027] In a more preferred embodiment of the present invention, the subject is an industrially manufactured product such as a machine part.

[0028] In a more preferred embodiment of the present invention, the subject is a plant product or an animal product.

[0029] Plant products include individual plants, parts of plants (e.g., fruits), groups of plants, and groups of plant parts. Animal products include animals, parts of animals, groups of animals, groups of animal parts, and products produced by animals (e.g., eggs). Industrially processed products such as cheese and sausages are also intended to be included in the term "plant products or animal products."

[0030] Plant or animal products are typically parts of plants or animals that are suitable for and / or intended for consumption by humans or animals.

[0031] Plant products are preferably at least a part of cultivated plants. The term "cultivated plants" is understood to mean plants that are specifically cultivated as useful plants through human intervention. In a preferred embodiment, cultivated plants are fruit plants or vegetable plants. Although fungi are not considered plants from a biological standpoint, fungi, in particular fungal fruiting bodies, are also intended to be included in the term, plant products.

[0032] Preferably, the plant to be cultivated is one of the plants listed in the encyclopedia "Encyclopedia of Cultivated Plants: From Acacia to Zinnia" Volumes 1 to 3 by Christopher Cumo, ABC-CLIO, 2013, ISBN 9781598847758.

[0033] Plant or animal products may include, for example, apples, pears, lemons, oranges, tangerines, limes, grapefruits, kiwis, bananas, peaches, plums, mirabelles, apricots, tomatoes, cabbage (such as cauliflower, white cabbage, purple cabbage, kale, Brussels sprouts), melons, pumpkins, cucumbers, bell peppers, zucchini, eggplants, potatoes, sweet potatoes, leeks, celery, kohlrabi, radishes, carrots, parsnips, scorzonera, asparagus, sugar beets, ginger rhizomes, rhubarb, coconuts, Brazil nuts, walnuts, hazelnuts, European chestnuts, eggs, fish, meat pieces, cheese pieces, sausages and / or similar items.

[0034] The optically readable code is introduced onto the surface of the object. This means that the optically readable code is neither attached to the object in the form of a tag nor added to it in the form of a sticker. Furthermore, the surface of the object itself is modified to carry the optically readable code.

[0035] In the case of plant-based products and eggs, the surface can be, for example, the epidermis / outer shell.

[0036] An optically readable code can be engraved, etched, branded, and / or imprinted onto an object, and / or introduced to the surface of the object in any other way. Preferably, the optically readable code is introduced to the surface of the object (e.g., the epidermis in the case of fruits and vegetables) by laser. In this case, the laser modifies (e.g., bleaches or destroys) the pigment molecules on the surface of the object and / or causes localized burning and / or destruction of tissue and / or chemical and / or physical modification (e.g., evaporation of water, denaturation of proteins and / or similar), thereby creating a contrast with the surrounding parts of the surface (parts not modified by the laser).

[0037] Optically readable codes can also be introduced to a target surface by water jets or sand jets.

[0038] Optically readable codes can also be mechanically introduced to the surface of an object by scribing, puncturing, parting, rasping, scraping, stamping, and / or similar methods.

[0039] Details regarding the marking of the target, particularly plant or animal products, can be obtained from the prior art (see, for example, EP2281468A1, WO2015 / 117438A1, WO2015 / 117541A1, WO2016 / 118962A1, WO2016 / 118973A1, DE102005019008A, WO2007 / 130968A2, US5660747, EP1737306A2, US10481589, US20080124433).

[0040] It is preferable that the optically readable code is introduced to the surface of the object by a carbon dioxide laser (CO2 laser).

[0041] Typically, when optically readable codes are introduced onto a surface, the resulting markings have lower contrast than, for example, optically readable codes in the form of black or color imprints on a white sticker. Because stickers can be individually designed, the contrast can be optimized; for example, black markings on a white background (e.g., black barcodes or black matrix codes) result in extremely high contrast. Such high contrast is usually not achieved when introducing optically readable codes onto a surface, particularly in the case of plant or animal products, and pharmaceuticals. This can make reading the codes difficult. Furthermore, the appearance of optically readable codes introduced onto a surface by a laser varies from object to object. In other words, the appearance of optically readable codes introduced on an apple, for example, is usually different from that of optically readable codes introduced on bananas, tomatoes, pumpkins, or potatoes. The variety of fruit can also affect the appearance of the optically readable codes; the appearance of optically readable codes on a Granny Smith apple will differ from that of a Pink Lady apple. The surface structure of plant or animal products can also affect the appearance of optically readable codes. The relatively rough surface structure of kiwifruit can interfere with the reading of optically readable codes, much like the bumps and spots on the skin of potatoes or apples. The surfaces of fruits and vegetables are usually curved. When optical codes are introduced into curved surfaces, the codes may become distorted, for example, pincushion distortion or barrel distortion. Such distortion can interfere with the decoding of the codes. In this case, the degree of distortion usually depends on the degree of curvature. An optically readable code measuring 2cm x 2cm will be more distorted when introduced into an apple than when the same code is introduced into a melon. Furthermore, the distortion that occurs in nearly spherical products (e.g., apples, tomatoes, melons) is different from the distortion that occurs in nearly cylindrical products (e.g., cucumbers). Smooth surfaces (e.g., the surface of an apple) produce more reflections (e.g., reflections caused by ambient light) than rough surfaces (e.g., the surface of a kiwifruit).

[0042] Accordingly, according to the present invention, the image recording of the optically readable code introduced on the surface of the object is subjected to one or more transformations before the code is read. Such transformations can enhance the contrast between the optically readable code and the surrounding parts of the surface (parts of the surface that do not have the code), and / or reduce or eliminate distortion, and / or reduce or eliminate reflection, and / or reduce or eliminate artificial elements caused by the presence of a feature object. In this case, one or more transformations are selected for each individual object depending on its features.

[0043] A "conversion" is a function or operator that takes an image as input and generates an image as output. Conversion ensures that, when an optically readable code is imaged in the input image, the contrast of the readable code relative to its surroundings is higher in the output image than in the input image, and / or the distortion is lower in the output image than in the input image. Conversion ensures that light reflection from the surface of the object is reduced in the output image compared to the input image. Conversion ensures that the clarity of the surface of the object is lower in the output image than in the input image when surface irregularities and / or non-uniformities are present. Generally, conversion ensures that decoding errors that occur during the decoding of an imaged optically readable code are less in the output image than in the input image. Such decoding errors occur, for example, when the white squares of a QR code are interpreted as black squares by the image sensor.

[0044] One or more transformation parameters determine what kind of transformations are performed, and if there are multiple transformations, one or more transformation parameters determine the order in which these transformations are performed.

[0045] First, the first image record of the target is generated.

[0046] The term "image recording" is preferably understood to mean a two-dimensional image representation of an object or part thereof. Image recordings are usually digital image recordings. The term "digital" means that the image recording can be processed by a machine, usually a computer system. "Processing" is understood to mean known methods of electronic data processing (EDP).

[0047] Using computer systems and software, digital image recordings can be processed, edited, and played back, and can also be converted to standardized data formats such as JPEG, Portable Network Graphics (PNG), and Scalable Vector Graphics (SVG). Digital image recordings can be visualized using appropriate display devices such as computer monitors, projectors, and / or printers.

[0048] In digital image recording, image content is typically represented and stored as integers. In most cases, digital image recording includes two-dimensional images that can be binary encoded and compressed as appropriate. Digital image recording usually includes raster graphics, where image information is stored in a uniform raster grid. Raster graphics consist of a raster arrangement of pixels in two-dimensional representations and a raster arrangement of voxels in three-dimensional representations, with color or grayscale values ​​assigned in each case. Therefore, the main characteristics of 2D raster graphics are image size (width and height measured in pixels, commonly known as image resolution) and color depth. Typically, colors are assigned to pixels in a digital image file. The encoding of the colors used for pixels is determined specifically using color space and color depth representations. The simplest example is a binary image where black and white values ​​are stored in pixels. In images where colors are defined using the so-called RGB color space (RGB represents the primary colors red, green, and blue), each pixel consists of three color values: one color value for red, one for green, and one for blue. The color of a pixel arises from the superposition of these three color values ​​(additive color mixing). Each color value is discretized to, for example, 256 distinguishable levels, which are called tonal values ​​and typically range from 0 to 255. A hue of "0" in each color channel is the darkest. If the tonal values ​​of all three channels are 0, the corresponding pixel appears black, and if the tonal values ​​of all three channels are 255, the corresponding pixel appears white. When the present invention is implemented, a specific operation (transformation) is performed on the digital image recording. In this case, the operation mainly involves pixels as so-called spatial operators, such as edge detectors, or as tonal values ​​of individual pixels, such as in the case of color space transformations. Multiple possible digital image formats and color encodings exist. For simplicity, this specification assumes that the images used herein are RGB raster graphics having a certain number of pixels. However, this assumption is not meant to be understood as a limitation.It will be obvious to those skilled in image processing how the teachings herein can be applied to image files that exist in other image formats, and / or image files in which color values ​​are encoded in a different way.

[0049] At least one image recording may be one or more excerpts from a video sequence.

[0050] At least one image recording is generated using one or more cameras. Preferably, at least one image recording is generated by one or more cameras of a smartphone.

[0051] Using multiple cameras to view an object from different directions and generate image recordings from those different viewing directions has the effect of capturing depth information. For example, from such depth information, it is possible to derive and / or collect information about the curvature of an imaged object in front of the viewer.

[0052] The first image recording shows an optically readable code introduced onto the surface of the object.

[0053] The first image record is used to determine the transformation parameters. Based on the first image record, the transformation parameters are determined, and in the process of transforming the first image record (and / or the second image record) according to the transformation parameters, a transformed image record is obtained that produces fewer decoding errors than the first image record when decoding the imaged optical code, for example, because the contrast of the optically readable code against the surroundings of the optically readable code is higher in the transformed image record than in the first image record and / or the resulting distortion / reflection is smaller.

[0054] The conversion parameters can be determined in two steps: in the first step, the object being imaged (at least in a symmetrical state) in the first image recording can be identified; and in the second step, for the identified object, the conversion parameters that result in an increase in the contrast of the optically readable code relative to the surrounding optically readable code can be read from the data storage medium.

[0055] Identification can be achieved, for example, based on the characteristics of the object being imaged in the first image recording. Such characteristic characteristics may include, for example, color and / or color distribution, shape, size, texture, and / or other characteristics. Pattern recognition methods known in image processing can be used to identify the object based on the imaged features.

[0056] It is preferable to use a trained machine learning model for object identification. In this specification, such a model is also referred to as a recognition model.

[0057] Such a model can be trained using a supervised learning method to produce at least a balanced output based on first image recording information that indicates what objects are present in the first image recording.

[0058] A "machine learning model" can be understood as a data processing architecture performed by a computer. The model can receive input data and supply output data based on the input data and model parameters. Through training, the model can learn the relationship between input and output data. During training, the model parameters can be adapted to supply a desired output for a specific input.

[0059] During the training of such a model, training data is provided to the model that can be used to learn the model. The trained machine learning model is the result of the training process. The training data includes not only the input data but also the correct output data (target data) that the model should generate based on the input data. During training, patterns that map the input data to the target data are recognized.

[0060] In the training process, training data is input to the model, and the model generates output data. The output data is compared to the target data (so-called ground truth data). Model parameters are modified to reduce the deviation between the output data and the target data to the minimum value (default value).

[0061] During training, a loss function can be used to evaluate the predictive quality of the model. The loss function can be chosen to reward desirable relationships between output data and target data, and / or punish undesirable relationships. Such relationships can be similarity, difference, or other types of relationships.

[0062] A loss function can be used to calculate the loss for a specific pair of output and target data. The goal of the training process is to modify (adapt) the parameters of the machine learning model so that the loss values ​​for all pairs in the training dataset decrease to a minimum (default) value.

[0063] A loss function can, for example, quantify the deviation between a model's output data and target data for specific input data. For instance, if the output and target data are numerical, the loss function is the absolute difference between these numbers. In this case, a large absolute value of the loss function may mean that one or more model parameters must be significantly altered.

[0064] For output data in vector format, the loss function can be selected from, for example, the norm of the difference vector, such as mean squared error, cosine distance, Euclidean distance, Chebyshev distance, Lp norm of the difference vector, weighted norm, or any other type of difference metric between two vectors.

[0065] A machine learning model can be implemented (configured) as a classification model that assigns image recordings as input data to specific objects as output data. For example, the model can output a numerical value representing a specific object that is imaged in the image recording.

[0066] Such classification models have been described using conventional techniques (see, for example, F. Sultana et al.: Advancements in Image Classification using Convolutional Neural Network, arXiv:1905.03288v1[cs.CV], 2019, and A. Khan et al.: A Survey of the Recent Architectures of Deep Convolutional Neural Networks, Artificial Intelligence Review, DOI:https: / / doi.org / 10.1007 / s10462-020-09825-6).

[0067] In this specification, assigning the first image record to an object that is imaged in the first image record is also referred to as "identifying an object." In this case, the first image record can be assigned to each part of the object and / or the object that is imaged in the image record. In the case of plant products, the first image record can also be assigned to the degree of maturity and / or the appearance; in the case of bananas, for example, to green or yellow bananas; in the case of bell peppers, for example, to green, yellow or red bell peppers; in the case of apples, to individual existing apple varieties and / or similar.

[0068] Finally, the identification of the object in the first image recording is used to determine the transformation parameters that are intended to be applied to the first image recording (or the second image recording) to, for example, enhance the contrast of the optically readable code against its surroundings, reduce distortion, reduce reflections, and / or reduce / remove other features that may lead to decoding errors. Thus, the identification is intended to identify the characteristics of the object being imaged that enable the selection of the above transformation parameters that ensure the optically readable code is decoded with the least possible errors.

[0069] Transformation parameters are typically stored in a data storage medium, such as a relational database. A machine learning model can be configured and trained to output identifiers associated with transformation parameters in the database, based on a first image recording. In other words, based on the identifiers representing the objects imaged in the first image recording, the transformation parameters assigned to those identifiers can be determined and retrieved from the database.

[0070] It is also conceivable to train a machine learning model to supply / output the conversion parameters themselves.

[0071] As already explained, the transformation parameters determine what transformation is intended to be applied to the first image record (or second image record), and, in the case of multiple transformations, the order in which the transformations are intended to be applied.

[0072] For a specific object, the above transformation can be empirically determined to ensure that the transformed image recording produces fewer decoding errors than the untransformed image recording.

[0073] Therefore, for multiple objects, it is possible to generate image records in which the optically readable codes introduced on the surface of the objects are imaged in at least a uniform manner. A person skilled in the art of image processing can specify for each record transformations that result in increased contrast of the optically readable codes relative to their surroundings, and / or reduction / removal of distortion, and / or reduction / removal of reflections and / or reduction / removal of other features that may result in decoding errors.

[0074] Decoding errors can be quantified empirically. For example, the decoding error can be the percentage of code that could not be decoded. Many optically readable codes have error correction mechanisms. The decoding error can also represent the number of corrected bits in the code.

[0075] Each transformation can be verified, and if the desired result is not obtained through a transformation, it can be discarded, improved, modified, and / or extended through further transformations.

[0076] Examples of transformations include spatial low-pass filtering, spatial high-pass filtering, sharpening, blurring (e.g., Gaussian blur), unsharp masking, erosion, median filtering, maximum filter, contrast range reduction, edge detection, color depth reduction, grayscale level conversion, negative creation, color correction (color balance, gamma correction, saturation), color replacement, Fourier transform, Fourier low-pass filter, Fourier high-pass filter, and inverse Fourier transform.

[0077] Several transformations can be performed by convolving raster graphics using one or more convolution matrices (called convolution kernels). Convolution matrices are typically square matrices with an odd number of dimensions and can have various sizes (e.g., 3x3, 5x5, 9x9, and / or similar). Some transformations can be represented as linear systems to which discrete convolution (linear operation) is applied. For discrete two-dimensional functions (digital images), the following formula is used for discrete convolution.

number

[0078] For example, in the case of an apple, the following shows a transformation and transformation sequence that results in fewer decoding errors in the transformed image recording than in the untransformed image recording. • Intensity, i.e., color conversion to a linear RGB signal. • A color conversion that converts a linear RGB signal into, for example, a reflection channel and / or illumination channel, and at least two color channels used to differentiate the encoded surface portion from the unencoded surface portion. To do this, the intensity, i.e., the linear RGB color signals, are linearly combined with each other so that the best possible differentiation between the encoded surface portion and the unencoded surface portion is achieved. • Reflection correction (additive correction) by subtracting the reflection channel from at least two color channels. • Illumination correction (multiplicative correction) by normalizing at least two color channels to the illumination channel. • Detection of surface irregularities in apples, correction of surface irregularities, and spatial interpolation from around defects. • Unsharp masking of illumination-corrected and reflection-corrected image segments using a range-based filter mask. This appropriately compensates for spatial non-uniformity of image brightness, for example, caused by the curved shape of the apple, and amplifies and optimizes the image contrast between encoded and unencoded surface areas by increasing the high-frequency image portion.

[0079] Further examples of transformations can be found in numerous publications on digital image processing.

[0080] It is also conceivable to analyze the first image recording in order to directly derive transformation parameters from the image recording. If the object being imaged in the image recording is identified, the transformation parameters are determined based on the identified object; if the transformation parameters are directly derived, they can be derived from the way the object is imaged. For example, it is possible to determine the average luminance, contrast range, color distribution and / or similar, and based on this, one or more transformation parameters that minimize decoding errors are selected, set, or determined. The optimization of contrast between encoded and unencoded surface regions can also be directly achieved by correlation analysis, for example, by the transformation of the color principal axes of the recorded object in this image region.

[0081] In yet another step, one or more transformations are applied to the first or second image record depending on the transformation parameters. As already revealed, it is not necessary to apply the transformation to the first image record, i.e., the image record based on the time when the transformation parameters were determined. Furthermore, it is also conceivable to apply the transformation to the second image record, which preferably shows the same subject as the first image record, but at a different point in time (for example, an earlier or later point in time). The camera used to generate the first image record may be configured to continuously digitize the image representation incident on the camera's image sensor at a predetermined rate, generate digital image records, and supply these digital image records to a recognition model that recognizes the imaged subject. The recognition model can pass an identifier of the recognized (identified) subject to a control unit, which determines transformation parameters based on the identifier. The determined transformation parameters can then be applied to one or more subsequently generated digital image records. Such subsequently generated image records are referred to herein as "second image records."

[0082] It is also conceivable that the captured object is identified not based on a single first image record, but on multiple first image records, for example, a series of image records that are consecutive in time. It is also conceivable that the transformation parameters are determined not based on a single first image record, but on multiple first image records, for example, a series of image records that are consecutive in time. It is also conceivable that the transformation is applied not to a single (first or second) image record, but to an image record among multiple image records, for example, an image record in a series of image records that are consecutive in time.

[0083] The converted image record is the result of applying one or more conversions to the image record of the optically readable code on the target surface.

[0084] The converted image recording can recognize and read optically readable codes more effectively than the original (unconverted) image recording.

[0085] In the next step, the code in the converted image recording is read (decoded). This specification describes existing methods for reading (decode) each code, depending on the code being used.

[0086] The read (decoded) code may contain information about the object on which the code is applied.

[0087] In a preferred embodiment, the read code includes an identifier (unique identifier), which the consumer can use to obtain further information about the subject from, for example, a database. The read code and / or information associated with the read code can be output, i.e., displayed on a screen, printed by a printer, and / or stored in a data storage medium.

[0088] Further information relating to such identifiers (unique identifiers) and information that can be stored and displayed to consumers as relating to an object associated with the identifier is described in the patent application published in European Patent Application No. 3896629, the entire contents of which are incorporated herein by reference.

[0089] The present invention will be described in more detail below with reference to the drawings, but the present invention is not limited to the features and combinations of features shown in the drawings.

[0090] Figure 1 schematically illustrates the reading of an optically readable code introduced on a target surface in the form of a flowchart.

[0091] In the first step (110), a digital image recording (I) of the object (O) is generated using a camera (C). In the second step (120), the digital image recording (I) is supplied to a recognition model (IM). The recognition model (IM) is configured to identify the captured object (O) in the image recording (I) based on the image recording (I). In the third step (130), the recognition model (IM) supplies information (OI1) about the identified object (O). In the fourth step (140), transformation parameters (TP) are determined from a database (DB) based on the information (OI1) about the identified object (O). In the fifth step (150), the determined transformation parameters (TP) are supplied to a process that transforms the image recording (I). In the sixth step (160), one or more transformations are applied to the image recording (I) according to the transformation parameters (TP). As a result, the transformed image recording (I * ) is obtained, and the contrast of the optically readable code (in this case, a QR code) introduced on the surface of the object (O) is high relative to its surroundings, making it more clearly recognizable and easier to read than the code in the case of an unconverted image recording (i). In the seventh step (170), the optically readable code is read and the read code (OI2) is provided. The read code (OI2) can be displayed and / or information about the object O based on the read code OI can be provided, for example, by reading from a database (e.g., communicating and displaying).

[0092] Figure 2 schematically illustrates a system according to the present invention.

[0093] The system (1) comprises a computer system (10), a camera (20), and one or more data storage media (30). The camera (20) can be used to generate image recordings of a target. The camera (20) is connected to the computer system (10) so that the generated image recordings can be sent to the computer system (10). The camera (20) can be connected to the computer system (10) via cable and / or wireless connection. Connection via one or more networks is also possible. It is also possible that the camera (20) is an integral part of the computer system (10), for example, as in today's smartphones and tablet computers.

[0094] The computer system (10) is configured to receive an image recording (from a camera or data storage medium), identify the object being imaged in the image recording, determine transformation parameters based on the identified object, transform the image recording and / or further image recordings of the object according to the transformation parameters, acquire the transformed image recording, decode the optical code in the transformed image recording, output the decoded code and / or provide information associated with the decoded code (for example, configured by a computer program).

[0095] The data storage medium (30) can store image recordings, models, transformation parameters, transformation operators and / or transformation functions, recognition models, computer programs and / or other information / and other information. The data storage medium (30) can be connected to the computer system (10) via cable and / or wireless connection. Connection via one or more networks is also possible. It is also possible that the data storage medium (30) is an integral part of the computer system 10. It is also possible that there are multiple data storage mediums.

[0096] Figure 3 schematically shows a computer system (10). Such a computer system (10) may include one or more stationary or portable electronic devices. The computer system (10) may include one or more components, such as a processing unit (11) connected to a storage medium (15).

[0097] The processing unit (11) may comprise one or more processors, either individually or in combination with one or more storage media. The processing unit (11) can be general computer hardware capable of processing information such as digital image recordings, computer programs, and / or other digital information. The processing unit (11) typically consists of an array of electronic circuits, some of which may be designed as an integrated circuit, or as a group of integrated circuits connected to one another (an integrated circuit is also referred to as a "chip"). The processing unit (11) can be configured to execute computer programs that can be stored in the main memory of the processing unit (11) or in storage media (15) of the same computer system or different computer systems.

[0098] The storage medium (15) can be general computer hardware capable of temporarily and / or permanently storing information such as digital image recordings, data, computer programs and / or other digital information. The storage medium (15) may comprise volatile and / or non-volatile storage media and may be built-in and fixed or removable. Examples of suitable storage media include RAM (random access memory), ROM (read-only memory), hard disks, flash memory, replaceable computer floppy disks, optical disks, magnetic tapes, and combinations thereof. Optical disks include compact disks with read-only memory (CD-ROM), compact disks with read / write capabilities (CD-R / W), DVDs, and Blu-ray® discs.

[0099] The processing unit (11) can be connected not only to the storage medium (15) but also to one or more interfaces (12, 13, 14, 17, 18) for displaying, sending and / or receiving information. The interfaces may comprise one or more communication interfaces (17, 18) and / or one or more user interfaces (12, 13, 14). One or more communication interfaces may be configured to send and / or receive information to and from, for example, a camera, another computer, a network, a data storage medium, etc. One or more communication interfaces may be configured to send and / or receive information via physical connections (wired connections) and / or wireless connections. One or more communication interfaces may comprise one or more interfaces for connecting to a network using technologies such as mobile phones, Wi-Fi, satellite, cable, DSL, optical fiber, and / or similar technologies. In some examples, one or more communication interfaces may comprise one or more near-field communication interfaces configured to connect devices using near-field communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, and infrared (e.g., IrDA).

[0100] The user interface (12, 13, 14) may include a display (14). The display (14) can be configured to display information to the user. Preferred examples include liquid crystal displays (LCDs), light-emitting diode displays (LEDs), and plasma display panels (PDPs). The user input interface (12, 13) can be wired or wireless and can be configured in the computer system (10) to receive information from the user for processing, storage, and / or display. Preferred examples of the user input interface include microphones, image and video recording devices (e.g., cameras), keyboards and keypads, joysticks, and touch-sensitive screens (which may be separate from or integrated into touchscreens). In some examples, the user interface may include automatic identification and data capture technology (AIDC) used for machine-readable information. AIDC includes barcodes, radio frequency identification (RFID), magnetic stripes, optical character recognition (OCR), and integrated circuit cards (ICCs). The user interface may further include one or more interfaces used for communication with peripheral devices such as printers.

[0101] One or more computer programs (16) can be stored in a storage medium (15) and executed by a processing unit (11), thereby programming the processing unit (11) to perform the functions described herein. Instructions for the computer programs (16) can be acquired, loaded, and executed sequentially, however, acquisition, loading, and / or execution can also be performed in parallel.

[0102] The system according to the present invention can be implemented as a laptop, notebook, netbook, tablet PC, and / or portable device (e.g., smartphone). Preferably, the system according to the present invention includes a camera. [Explanation of Symbols]

[0103] 1 System 10 Computer Systems 11 Processing Section 12 Interfaces 13 Interfaces 14 Interfaces 15 Storage medium 16 Computer Programs 17 Interfaces 18 Interfaces 20 cameras 30 Data storage media 110 First Step 120 Second Step 130 The Third Step 140 The Fourth Step 150 The Fifth Step 160 The 6th Step 170 The 7th Step C Camera DB Database I. Digital Image Recording i Unconverted image recording I * Converted image record IM Recognition Model O Target Information regarding the OI1 target OI2 read code TP conversion parameters

Claims

1. A method performed by a computer, A step of receiving a first image recording of an object at a first point in time, wherein the object comprises an optically readable code, and the optically readable code is introduced onto the surface of the object. The steps include identifying the object based on the first image recording, The steps include: reading the conversion parameters of the identified target from the database; A step of converting a second image recording of the target at a second time point different from the first time point based on the conversion parameters, The steps include decoding the optically readable code that is imaged in the converted image recording, and A method for providing this.

2. The method according to claim 1, wherein the subject is a plant product or an animal product.

3. The method according to claim 2, wherein the subject is a plant product, and the light-readable code is introduced into the epidermis of the plant product.

4. The method according to claim 1, wherein the subject is a pharmaceutical drug.

5. The method according to claim 1, wherein the optically readable code is introduced to the surface of the object by a laser.

6. The method according to claim 1, wherein the optically readable code is a barcode or a matrix code.

7. The method according to claim 1, wherein the optically readable code is alphanumeric.

8. The method according to claim 1, wherein the object is identified based on features imaged in the first image recording.

9. The method according to claim 1, wherein the objects are identified using a trained machine learning model, the machine learning model is trained on training data to assign image recordings to objects, and the training data used for each of a plurality of objects comprises i) at least one image recording of the object, and ii) information about which objects are imaged in the image recording.

10. The training of the aforementioned machine learning model is The steps include inputting the target image recording into the machine learning model, The steps include receiving output data from the aforementioned machine learning model, A step of calculating the deviation between the output data and the information regarding what objects are being imaged in the image recording using a loss function, The steps include modifying the parameters of the machine learning model with respect to reducing the aforementioned deviation, The steps include storing the trained machine learning model in a data storage medium and The method according to claim 9, comprising:

11. The method according to claim 1, wherein the conversion parameters are determined empirically, and when determining the conversion parameters empirically, the conversion parameters selected are such that reading the optically readable codes in the converted image recording results in fewer decoding errors than reading the optically readable codes in the unconverted image recording.

12. The step of converting the second image recording is: The steps include reducing the distortion of the optically readable code being imaged in the second image recording, The steps include reducing reflections in the second image recording, A step of increasing the contrast of the optically readable code as it is imaged in the second image recording with respect to the surrounding area of ​​the optically readable code. The method according to claim 1, comprising one or more of the following.

13. A system comprising at least one processor, wherein the processor is Receiving a first image recording of the object at a first point in time, wherein the object comprises an optically readable code, and the optically readable code is introduced onto the surface of the object. Identifying the object being imaged in the first image recording, The conversion parameters of the identified target are read from the database, Based on the conversion parameters, the second image recording of the target at a second time point different from the first time point is converted. Decoding the optically readable code that is imaged in the converted image recording. A system configured to perform the following actions.

14. A computer program, wherein the computer program can be loaded into the main memory of a computer system, and the computer program is A step of receiving a first image recording of an object at a first point in time, wherein the object comprises an optically readable code, and the optically readable code is introduced onto the surface of the object. The steps include identifying the object based on the first image recording, The steps include: reading the conversion parameters of the identified target from the database; A step of converting a second image recording of the target at a second time point different from the first time point based on the conversion parameters, The steps include decoding the optically readable code that is imaged in the converted image recording, and A computer program that causes the aforementioned computer system to execute.

Images