SYSTEMS AND METHODS FOR DETERMINING INFORMATION REPRESENTED BY AN OPTICALLY MACHINE-READY CODE

Abstract

A system for retrieving information represented by an optically machine-readable code may include a memory and a processor.The processor can be configured to: determine a digital feature description of a candidate for an optically machine-readable code; for at least one optically machine-readable reference code, where each of the multiple reference codes is associated with a physical object to uniquely identify the respective physical object, where each reference code is described by a respective reference code feature description in the feature space: compare the feature description of the candidate code with the reference code feature description to generate a comparison result; and determine that the reference code represents the candidate code if the comparison result satisfies a predefined similarity criterion; if it is determined that the reference code represents the candidate code, identify the information represented by the reference code as the information represented by the code.

Description

Technical field

[0001] Several aspects of this disclosure relate generally to systems and methods for obtaining information represented by an optically machine-readable code. background

[0002] The recognition of barcodes, Quick Response (QR) codes, or AprilTag codes is a well-established technical process for which numerous solutions already exist. In industry, for example, near-field scanners are a de facto standard. There are also many solutions that recognize codes within a digital camera image (RGB). A conventional solution uses artificial intelligence (AI) models (e.g., YoloV8 in the case of a QReader) to recognize a code, combined with a decoder library to provide the encoded information.

[0003] In a modern industrial setting, the main limitation of barcode readers is their maximum range of approximately 30 cm, which has significant financial implications. In a typical application, for example, the scanning process requires a forklift operator transporting goods with such a code to dismount and approach a shelf to scan the items, which can be very time-consuming. Furthermore, increasing the size of a barcode, QR code, or AprilTag code is not an option, as the size of these codes is standardized, or the associated product or code representation is too small to accommodate a larger code. Another conventional solution is to use one scanner / camera per shelf location.This would, for example, lead to an enormous demand for cameras in a warehouse and would therefore also be neither economical nor feasible. Brief description of the drawings

[0004] In the drawings, identical reference numerals generally refer to identical parts in the different views. The drawings are not necessarily to scale but primarily serve to illustrate the principles of the invention. The following description details various embodiments of the invention with reference to the following drawings, in which: Fig. Figure 1A shows a test setup used to test various conventional solutions for detecting and decoding a QR code; Fig. 1B the results of the test according to the test setup of Fig. 1A shows various conventional solutions for recognizing and decoding a QR code; Fig. 2 shows a system according to different aspects of this revelation; Fig. 3 shows an exemplary digital black and white image; Fig. 4 shows a part of a system according to various aspects of this revelation; Fig. 5 shows a diagram illustrating a test result that tests a system according to various aspects of this revelation; Fig. 6 shows a part of a system according to various aspects of this revelation; Fig. 7 shows a system for determining information represented by an optically machine-readable code; Fig. 8 shows a procedure according to various aspects of this revelation; and Fig. 9 shows a procedure according to various aspects of this revelation. Description

[0005] The following detailed description refers to the attached drawings, which show specific details and embodiments in which the invention can be practiced.

[0006] The term "exemplary" is used here to mean "serving as an example, case, or illustration." Each embodiment or design described herein as "exemplary" is not necessarily to be understood as preferred or advantageous over other embodiments or designs.

[0007] The word "over" in reference to deposited material formed "over" a side or surface can be used here to mean that the deposited material may be formed "directly on," e.g., in direct contact with, the implied side or surface. The word "over" in reference to deposited material formed "over" a side or surface can also be used here to mean that the deposited material may be formed "indirectly on" the implied side or surface, with one or more additional layers arranged between the implied side or surface and the deposited material.

[0008] The terms "processor" or "controller," as used here, can be understood as any type of unit that enables the processing of data. The data can be processed according to one or more specific functions performed by the processor or controller. Furthermore, a processor or controller, as used here, can be understood as any type of circuit, e.g., any type of analog or digital circuit. A processor or controller can thus be or contain an analog circuit, a digital circuit, a mixed-signal circuit, a logic circuit, a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), an integrated circuit, an application-specific integrated circuit (ASIC), etc., or any combination thereof.Any other type of implementation of the respective functions described in more detail below can also be understood as a processor, controller, or logic circuit. It is understood that two (or more) of the processors, controllers, or logic circuits described herein may be implemented as a single unit with equivalent functionality or the like, and conversely, that each of the processors, controllers, or logic circuits described herein may be implemented as two (or more) separate units with equivalent functionality or the like.

[0009] Various implementations can use one or more machine learning models to perform vehicle functions (or other functions described herein). The term "model," as used here, can be understood as any type of algorithm that produces output data from input data. A machine learning model can be executed by a computer system to progressively improve the performance of a given task. In some aspects, parameters of a machine learning model can be adjusted during a training phase based on training data, and a trained machine learning model can then be used during an inference phase to make predictions or decisions based on input data.In some aspects, the trained machine learning model can be used to generate additional training data, and this additional machine learning model can be refined during a second training phase based on the generated additional training data. This additional trained machine learning model can then be used during an inference phase to make predictions or decisions based on input data.

[0010] The machine learning models described here can take any suitable form or use any suitable technique. For example, each of the machine learning models can use techniques of supervised learning, semi-supervised learning, unsupervised learning, or reinforcement learning.

[0011] In supervised learning, the model can be built using a training dataset that contains both the inputs and the corresponding desired outputs. Each training instance can include one or more inputs and one desired output. Training can involve iterating through training instances and using an objective function to teach the model to predict the output for new inputs. In semi-supervised learning, some inputs in the training set may lack the desired outputs.

[0012] In unsupervised learning, the model can be built from a dataset containing only inputs and no desired outputs. The unsupervised model can be used to find structures in the data (e.g., grouping or clustering of data points) by discovering patterns. Techniques that can be implemented in an unsupervised learning model include, for example, self-organizing maps, nearest-neighbor mapping, k-means clustering, and singular value decomposition.

[0013] Reinforcement learning models can receive positive or negative feedback to improve accuracy. A reinforcement learning model may attempt to maximize one or more goals / rewards. Techniques that can be implemented in a reinforcement learning model include, for example, Q-learning, temporary differences (TD), and deep adversarial networks.

[0014] A machine learning model described here (e.g., a deep learning model) can be a neural network. The neural network can be any type of neural network, such as a convolutional neural network, an autoencoder network, a variational autoencoder network, a sparse autoencoder network, a recurrent neural network, a deconvolutional neural network, a generative adversarial network, a predictive neural network, a transform neural network, or a sum-product neural network, and the like. The neural network can comprise any number of layers, and the training of the neural network, i.e., the adaptation of the neural network layers, can be based on any training principle, such as backpropagation, i.e., the backpropagation algorithm.

[0015] For the purposes of this description, an optically machine-readable code is understood to be a machine-readable graphical code that can be optically read by an optical sensor, such as a camera. An optically machine-readable code is, for example, a visual representation of data that can be read and interpreted by a machine, typically using an optical scanning device, such as an optical sensor like a camera. Such a code is designed to store information in a format that allows for fast and accurate data retrieval without human intervention. Examples of an optically machine-readable code include a 1D barcode (one-dimensional, e.g., a Universal Product Code (UPC), a Code 39, or a Code 128) or a 2D barcode (two-dimensional, e.g., a Quick Response (QR) code, a Data Matrix code, or an Aztec code).

[0016] In the context of this description, one or more deep image features (as examples of feature descriptions of an optically machine-readable code) are complex and abstract representations of visual information (e.g., an optically machine-readable code) that are automatically learned by one or more deep neural networks (e.g., through one or more digital (e.g., optical) filters and / or through one or more convolutional neural networks and / or through one or more transformation neural networks) during the training process using digital image data. These deep image features capture hierarchical patterns and structures within digital images, thus enabling more effective and nuanced image analysis than traditional, manually created features.Deep image features can be hierarchically organized, with lower neural network layers capturing simple elements in the digital image data, such as edges and textures, while deeper neural network layers recognize more complex patterns, such as object parts or entire objects. Unlike traditional methods that rely on manual feature engineering, one or more deep learning models automatically extract relevant features from raw digital image data, reducing the need for expert knowledge. As the information traverses the neural network layers of a deep neural network, the deep image features become increasingly abstract, enabling one or more deep learning models to capture high-level concepts and semantics within one or more digital images. The deep image features can be provided, for example, by an encoder stage of the deep neural network.in the so-called latency space as an example of a feature space.

[0017] A transformation of pixel information (color information and / or luminance information and / or hue information) associated with one or more pixels in a digital image in a pixel information space, into a feature description in a feature space, can be performed by one or more of the following methods: one or more digital (e.g., optical) filters and / or one or more convolutional neural networks and / or one or more transformation neural networks and / or one or more autoencoder networks and / or one or more variational autoencoder networks and / or one or more sparse autoencoder networks and / or one or more generative antagonistic networks or the like. In general, any filter structure that effects a transformation from a pixel space to a feature space can be used in various aspects of this disclosure.

[0018] In the context of this disclosure, information represented by an optically machine-readable code is to be understood as information that is optically encoded by the optically machine-readable code.

[0019] The information represented by an optically machine-readable code may include one or more of the following elements: - one or more Uniform Resource Locators (URLs), i.e., one or more links to a website that allow users to quickly access online content by scanning the optically machine-readable code; - Contact details, e.g. in digital vCard or MeCard format, which may contain information such as one or more names, one or more telephone numbers, one or more email addresses and one or more physical addresses; - Text information, e.g., a plain text message that can be directly encoded into an optically machine-readable format for easy forwarding or display; - Information on social media links that directly point to a social media profile or page and allow quick access to a social media account such as Facebook, Twitter or Instagram; - WLAN access data, e.g., one or more names and passwords of WLAN communication networks, which allow users to connect to one or more WLAN communication networks without manual input; - SMS and WhatsApp messages that may contain one or more phone numbers and one or more pre-filled messages that can open a messaging app when scanned; - Payment information that can be used for mobile payments, e.g., through direct linking to a payment platform or a cryptocurrency wallet address; - Detailed event information, e.g., about an event such as date, time, and location, which may be encrypted and often allows a user to save the event directly to their calendar; - a PDF or other format of a digital document, e.g. a link to a downloadable file or document for easy access; - Vouchers or promotional information, e.g. information on the distribution of discount vouchers or promotional offers that can be redeemed in-store or online; - Information about the physical item (e.g. goods) to which the optically machine-readable code is assigned, such as the name of the physical item, a location where the physical item is to be stored, price information about the physical item, or other logistical information about the physical item, and the like.

[0020] The recognition of an optically machine-readable code, such as a barcode, QR code, or AprilTag code, is a well-established technical step for which many solutions exist. In industry, for example, an optical near-field scanner is a de facto standard. A conventional solution recognizes codes within digital camera images (RGB). These conventional solutions utilize artificial intelligence (AI) models (e.g., YoloV8 in the case of QReader) to recognize the code in combination with a decoder library and provide the encoded information based on pixel information in pixel information space.

[0021] In current industrial applications, the limitations of barcode or tag code readers lie in their maximum range of approximately 30 cm, which impacts actual performance and financial consequences. For example, the conventional scanning process requires a forklift operator transporting goods with such a code to dismount and approach a shelf to scan the items, which can be very time-consuming. Furthermore, enlarging the optical machine-readable code is not an option, as the size of an optical machine-readable code or tag is standardized. Another conventional solution is to use one scanner / camera per shelf location. This would result in an enormous number of cameras in a warehouse and would therefore also be neither economical nor feasible.

[0022] Conventional solutions share a common drawback: the optical sensor must be positioned relatively close to the optically machine-readable code, otherwise recognition and decoding will fail. A test was conducted using a commercially available optical camera with a 2MP (megapixel) RGB (red, green, blue) optical sensor and a maximum resolution of 1920 x 1080 pixels (generally, a higher resolution helps to detect smaller optically machine-readable codes at greater distances). A QR code 102, measuring 25 mm × 25 mm, was used as an example optically machine-readable code and was placed at various positions in front of the camera (in the center of the image). A test setup 100 is shown in Fig. 1A. The results, shown in diagram 110 in Fig. Figure 1A shows that the recognition system had difficulty decoding the QR code at distances greater than 0.5 meters, and that an embedded artificial intelligence (AI) QReader in this setup cannot even find relevant areas at distances greater than 0.9 meters. Diagram 110 shows the respective relative number of detections achieved in relation to the distance between the optical camera and the QR code 102. A first feature 112 shows the results for ZBar decodings, a second feature 114 shows the results for QReader decodings, and a third feature 116 shows the results for QReader detections.

[0023] It should be noted that in this test example, the input image has already been cropped to a relevant area. If the entire image is provided, the results are even worse. There are several reasons for the relatively poor recognition capabilities compared to conventional solutions. First, the optical camera's RGB sensor has a rather low resolution with only moderate image quality. Second, the tested QR code has 25 x 25 pixels (px) and, at a distance of 0.9 meters from the area of ​​interest (ROI), measures only 32 x 32 px, which also includes an additional border. Therefore, at this distance, not all information is correctly captured in the image, significantly complicating decoding. Finally, all AI networks are trained using specific data, which requires certain image and code sizes.Even slight deviations in an industrial environment can significantly impair detection and thus reduce the successful decoding rate.

[0024] Therefore, conventional recognition systems must be relatively close to the optical machine-readable code to successfully read and decode the information encoded in the optical machine-readable reference code. As a result, scanning the optical machine-readable code for a large number of physical objects (e.g., goods) is often a tedious and time-consuming process. For example, when an employee restocks a shelf with physical objects (e.g., goods), each physical object must be manually scanned and placed in the correct position. If scanning is forgotten or the physical object (e.g., the goods) is placed in the wrong position, the information in the warehouse management system will be incorrect, which can ultimately lead to significant financial losses.

[0025] The automation of processes such as industrial production and warehouse management depends heavily on the ability to reliably identify and track physical objects (e.g., goods). An industry standard is the use of optically machine-readable codes, such as 1D codes (e.g., barcodes) or 2D codes (e.g., QR codes, data matrix codes, Aztec codes), or AprilTag codes, to (re)identify and / or locate physical codes relating to goods or other physical objects, features, or locations using a camera-based system.

[0026] At the same time, many warehouses and production facilities already have numerous cameras for various purposes, such as security, production monitoring, etc. Ideally, various aspects of this disclosure could reuse these cameras for the code-scanning task. However, since these cameras are typically located further away from the physical object (e.g., goods), it is not possible to use them for optical scanning of machine-readable codes with a conventional recognition system.

[0027] For illustration, some aspects of this disclosure provide a system and a procedure which do not directly decode the pixel information from a determined candidate for an optically machine-readable code in one or more captured digital images (in the pixel information space), but determine an optically machine-readable code by comparing a feature description of an area of ​​interest (e.g., a feature description of a candidate for an optically machine-readable code) with a corresponding feature description of one or more stored optically machine-readable reference codes (in a feature space) and then determine the information associated with a determined (e.g., best-matching) optically machine-readable reference code and use this information as the information associated with the recognized optically machine-readable reference code.

[0028] A system for the optical recognition of machine-readable codes using conventional optical sensors, e.g., conventional cameras, is provided, which can be configured to perform the procedures described in more detail below. The system in Fig. System 200, as depicted, clearly divides the problem of optical machine-readable code scanning into capturing a code area (also called an area of ​​interest, e.g., an area containing and / or representing an optical machine-readable candidate code) within a camera image (i.e., one or more two-dimensional images 204 and / or one or more depth images 206) and subsequently re-identifying the optical machine-readable code from a known set of optical machine-readable codes (also called optical machine-readable reference codes) (e.g., by accessing a warehouse / production line management system, generally by accessing a database containing at least the optical machine-readable codes used in the system to identify and / or locate the physical objects used in the system (e.g.,(goods or autonomous mobile robots) as optically machine-readable reference codes). This allows the system to correctly identify optically machine-readable codes that are much farther away than is currently possible with conventional systems, enabling a significantly higher degree of automation and considerably better performance.

[0029] In other words, according to various aspects of this revelation, a system can improve the detection and identification of an optically machine-readable code at a greater distance than a conventional system. The system can use two independent stages.

[0030] First, a stage to capture all relevant areas (also referred to as “areas of interest”) in one or more digital images (e.g., one or more two-dimensional images 204 and / or one or more depth images 206) that represent possible optically machine-readable codes.

[0031] Secondly, a stage compares these image areas (i.e., the identified area of ​​interest, e.g., the identified candidate for an optical machine-readable code) with a database of existing optical machine-readable reference codes. The database can be located in a warehouse or production environment, as all physical objects (e.g., goods) and their associated optical machine-readable reference codes are known and stored digitally. By comparing, for example, deep image features, the system can calculate a confidence score for each area of ​​interest (e.g., for each candidate for an optical machine-readable code) and provide this score along with the best match. This can then be used to retrieve all relevant information from the database for physical objects (e.g., inventory management).

[0032] As in Fig. As shown in Figure 2, System 200 comprises two main stages: a stage to locate all relevant image areas that may contain candidates for optically machine-readable codes, and a second stage to identify whether the identified areas contain a valid optically machine-readable code and to link this code to a physical object (e.g., a product) from a database (e.g., a goods / production / warehouse management database). These stages are explained in more detail below.

[0033] Fig. Figure 2 shows the System 200 for determining information represented by an optically machine-readable code.

[0034] The system 200 can include one or more optical sensors 202, e.g., one or more optical depth sensors 202, e.g., one or more (e.g., digital) cameras 202, e.g., one or more (e.g., digital) depth cameras 202. Furthermore, the system 200 can include one or more memory units and one or more processors (in Fig. (2 not shown). It should be noted that the one or more memories and / or the one or more processors may be part of the one or more optical sensors 202. Alternatively, the one or more memories and / or the one or more processors may be located outside of the one or more optical sensors 202.

[0035] The one or more optical sensors 202 can be configured to capture one or more (e.g. digital) two-dimensional images 204, e.g. one or more digital two-dimensional color images 204, and / or one or more (e.g. digital) depth images 206.

[0036] Every two-dimensional image can contain a multitude of image elements (also called pixels). Color information and / or brightness information and / or hue information can be assigned to one or more of the pixels in a pixel information space. The color information can be color information from any desired color space, e.g., the RGB (red-green-blue) color space, the sRGB (standard red-green-blue) color space, the CMY (cyan-magenta-yellow) color space, the CMYK (cyan-magenta-yellow-black) color space, or any other desired color space.

[0037] Each depth image 206 can also contain a variety of image elements (also called pixels). Color information and / or brightness information and / or hue information can be assigned to one or more of the pixels in the pixel information space. The color information can be color information from any desired color space, e.g., the RGB (red-green-blue) color space, the sRGB (standard red-green-blue) color space, the CMY (cyan-magenta-yellow) color space, the CMYK (cyan-magenta-yellow-black) color space, or any other desired color space. In addition, each depth image 206 can contain further depth information 216 assigned to one or more of the pixels.

[0038] The one or more optical sensors 202, e.g., a processor of the one or more optical sensors 202 (or a processor outside the one or more optical sensors 202), can be configured to identify one or more areas of interest 208 in the one or more digital images 204, 206. For example, the processor can be configured to perform an object segmentation process on the digital two-dimensional image 204 and / or on the one or more digital depth images 206. In general, the processor can be configured to perform any conventional object segmentation process.Furthermore, in various aspects of this disclosure, the processor may be configured to receive the one or more digital depth images 206 containing color and / or luminance information associated with one or more pixels of the digital depth image. The processor may also be configured to identify a candidate for an optically machine-readable code 214 (as an example of the scope of interest 214) by performing the object segmentation process on at least a portion of the one or more digital depth images 206 using the color and / or luminance information associated with pixels of at least that portion of the digital depth image.Furthermore, the processor can be configured to perform the object segmentation process using the depth information assigned to the pixels of at least part of the digital depth image. To achieve more reliable object segmentation for determining the candidate for the optical machine-readable code 214, the processor can also be configured to perform the object segmentation process using a reference form of an optical machine-readable reference code 210 and a reference size (e.g., reference size 404 in ). Fig. 4) of the optically machine-readable reference code 210.

[0039] In this context, it should be noted that optical machine-readable reference codes (OPCs) typically have a predefined shape and size, which may even be standardized. For example, a QR code (as an example of an OPC) can have a rectangular shape and a size of, say, 25 mm × 25 mm. However, other predefined shapes are also possible for OPCs, such as triangular, circular, elliptical, or the like. Furthermore, other predefined sizes are also possible for OPCs, such as any predefined two-dimensional value. The processor can use such a predefined size and shape of the OPC to refine the determination of the area of ​​interest (AFI), such as the candidate for an OPC.This results from the pre-stored knowledge about the shape and size of the area of ​​interest 214, e.g., the candidate for an optical machine-readable code 214, in which the one or more depth images 206 are searching for and which the processor uses as part of the object segmentation process. Furthermore, the processor may be configured to use additional prior knowledge about the optical machine-readable reference codes 210, e.g., the fact that every optical machine-readable code in an image is normally surrounded and bounded by a white perimeter that runs around the optical machine-readable code in an image (such as the one or more depth images 206).

[0040] In addition, the processor uses, for example, the depth information to project a potential area of ​​interest into the one or more depth images 206 and compares the projected size of the potential area of ​​interest with the sizes of the one or more optically machine-readable reference codes 210.

[0041] According to various aspects of this disclosure, a database 212 is provided which stores the following information for all optically machine-readable reference codes 210 provided in the system (e.g., a warehouse or a rack storage system); in other words, the database 212 stores, for example, the following information for each optically machine-readable reference code 210 that is associated with (or assigned to) a physical object in the system that is to be uniquely identified in the system: - a form information (also called reference form) that describes the form of the respective optically machine-readable reference code 210; - a size specification (also called reference size, e.g. reference size 404 in Fig. 4), which describes the size of the respective optically machine-readable reference code 210; - the information (e.g. in digital or binary form) (graphical, i.e. optical) that is encoded by the respective optically machine-readable reference code 210.

[0042] Intuitively, the processor is configured to identify a region of interest 214 (e.g., a candidate for an optical machine-readable code 214) by comparing a portion of the one or more depth images 206 with the one or more stored optical machine-readable reference codes 210. The processor can determine a projection of the potential region of interest using the depth information associated with the pixels contained within the potential region of interest in the one or more depth images 206, and can then compare the projected potential region of interest with the one or more stored optical machine-readable reference codes 210, e.g., by comparing the projected shape and / or size of the projected potential region of interest with the reference shape and / or size (e.g., reference size 404 in ). Fig. 4) of the stored optical machine-readable reference codes 210, one or more optical machine-readable reference codes 210, to identify one or more areas of interest 214 that may contain and / or represent a candidate for an optical machine-readable code 214.

[0043] In summary, the first stage of System 200 aims to identify all areas of interest within one or more digital images 204, 206 (e.g., a recognized digital RGB image that may contain an optically machine-readable code). As mentioned above, identifying these areas of interest can involve an object segmentation process, which may include various approaches such as principal component analysis, template matching, or artificial intelligence (AI) to identify areas of interest (which may contain one or more optically machine-readable codes) within the one or more digital images 204, 206 (e.g., a digital RGB image 204, 206). It should be noted that the first and second stages can be implemented and performed independently.

[0044] It should be noted that the conventional approaches to identifying candidates for optically machine-readable codes 214 may require that the respective optically machine-readable code in the one or more digital images 204, 206 (e.g. the digital RGB image 204, 206) is sufficiently large, i.e., "number of image pixels of a candidate for an optically machine-readable code >> number of pixels contained in an optically machine-readable reference code".

[0045] In various aspects of this revelation, the first stage of System 200 may also be capable of identifying regions of interest 214 (e.g., candidates for optically machine-readable codes 214) where the region of interest 214 is very small. Therefore, the processor may be configured not to search for specific optically machine-readable code patterns in the first place. Instead, the one or more digital images 204, 206 may optionally be converted into one or more digital black-and-white images through several processing steps (see, e.g., digital (binary) black-and-white image 302 in Fig. 3) Regions of interest 304 (such as a region containing a candidate for an optically machine-readable code 304) may be represented as white rectangles (as an example of a reference shape). The processor can use this prior knowledge of the code characteristic that an optically machine-readable code is usually on a white background (environment). In the digital (binary) black-and-white image 302, there may be a large number of potential regions of interest 214, 304, represented, for example, as white rectangles as candidates for optically machine-readable codes. Therefore, the processor may be configured to use an additional depth sensor that provides depth data (e.g., depth information associated with one or more pixels of the digital (binary) black-and-white image 302). By knowing the dimensions (i.e., the reference size (e.g., reference size 404 in Fig. 4)) of the optically machine-readable reference codes 210, which may be present in the scene (originating from the management system), it is possible to calculate for each distance the expected rectangle sizes in the one or more images 204, 206, 302.

[0046] A System 400 can determine the digital (binary) black-and-white image 302, for example by converting one or more two-dimensional images 204 into the digital (binary) black-and-white image 302 (in block 402). System 400 is similar to System 200. Fig. 2 and additionally includes the processor which is set up to convert one or more two-dimensional images 204 into the digital (binary) black and white image 302.

[0047] By further considering aspects such as an aspect ratio (in other words, the size of the area of ​​interest) (the area of ​​interest 304 should have a shape that is approximately square) or the color of the interior of the area of ​​interest 304, it may be possible to significantly reduce false positives.

[0048] Finally, all remaining areas of interest 304 are provided as output 214, i.e., the identified area of ​​interest 214 that may represent or contain a candidate for an optically machine-readable code 214, as in Fig. 2 and Fig. Figure 4 shows that it may be provided that an optically machine-readable candidate code and / or an optically machine-readable reference code is rotated (see System 400 in [reference]). Fig. 4 and rotated areas of interest, namely a first area of ​​interest 602 (e.g., rotated by 90°), a second area of ​​interest 604 (e.g., rotated by 180°), a third area of ​​interest 606 (e.g., rotated by 270°), and the rotated versions (e.g., additionally) are compared. This will be covered in the next system component 218.

[0049] Back to Fig. 2: After completion of the object segmentation process, the processor uses the identified areas of interest 214 (e.g., the identified candidates for optical machine-readable codes 214) to determine the desired information represented by an optical machine-readable code. The processor implements, by way of example, the function of an optical machine-readable code matcher.

[0050] The processor aims to identify, from a range of image areas that may contain "valid" optical machine-readable codes (i.e., optical machine-readable reference codes 212 stored in database 212), i.e., the identified areas of interest 214, all areas of interest 214 that do not contain a "valid" optical machine-readable code (i.e., to ignore these areas of interest 214), and to identify the correct optical machine-readable code and the associated physical object (e.g., goods) using the stored reference data from database 212 (e.g., inventory management database 212), e.g., using the feature description of the stored optical machine-readable reference codes 212.

[0051] Therefore, the processor can first load all available optically machine-readable reference codes 210 from the (e.g., inventory management) database 212. Subsequently, for a determined area of ​​interest 214 (e.g., for each area of ​​interest 214), all optically machine-readable reference codes 210 are reduced to the size of the respective determined area of ​​interest 214. It should be noted that the processor can alternatively also enlarge the respective determined area of ​​interest 214; however, reducing the size can yield better results, as enlargement always introduces artifacts.

[0052] In other words, the processor can be configured to project each stored optical machine-readable reference code 210 onto the size of the respective area of ​​interest 214 (using the depth information associated with the pixels contained in the respective area of ​​interest 214) and then compare the reduced optical machine-readable reference codes 212 with the respective area of ​​interest 214 (at original size).Alternatively, the processor can be configured to project the respective area of ​​interest 214 (using the depth information assigned to the pixels contained in the respective area of ​​interest 214) onto the size of the stored optical machine-readable reference codes 210 (if the sizes of the stored optical machine-readable reference codes 210 differ from each other, the projection onto the respective size of the respective stored optical machine-readable reference code 210 can be performed) and then compare the upscaled respective area of ​​interest 214 with the optical machine-readable reference codes 210 (in original size).

[0053] The processor is further configured to then compare each (e.g., reduced) optical machine-readable reference code 210 with the identified area of ​​interest 214 (e.g., enlarged) and to evaluate the similarity, and then to select the best-matching optical machine-readable reference code (block 218 in Fig. 2).

[0054] In various aspects of this disclosure, the processor may be configured to compare the optically machine-readable reference code 210 with the identified area of ​​interest 214 by comparing a respective feature description of each optically machine-readable reference code 210 with a feature description of the identified area of ​​interest 214 (e.g., the feature description of the candidate for the optically machine-readable code 214). For this purpose, the processor may be configured to transform the pixel information (e.g., the color information and / or the brightness information and / or the hue information) from the pixel information space into a feature space (e.g.,by applying one or more optical filters and / or a neural transformation network and / or an encoding stage of a convolutional neural network (CNN) to the pixel information in the pixel information space to generate the respective feature description in the feature space. Thus, the processor can be configured to transform the pixel information associated with the respective area of ​​interest 214 and / or to transform the pixel information associated with the respective (i.e., each) optically machine-readable reference code 210 (or its respective projection). In various aspects of this disclosure, it may also be provided that another processor, e.g., a sensor-external processor or even a system-external processor, can perform the respective transformation(s).For example, the feature description of a respective optically machine-readable reference code 210 can be stored together with the respective optically machine-readable reference code 210 in the database 212, and the preprocessor can receive the feature description of the respective optically machine-readable reference code 210 and compare it with the respective area of ​​interest 214.

[0055] In some examples, the processor can be configured to determine the mean absolute difference of the respective feature descriptions (which may be in the form of a respective feature vector) to evaluate the similarity between a given area of ​​interest 214 and a given (e.g., scaled-down) optical machine-readable reference code 210. Alternatively, the processor can be configured to implement a template-matching approach, e.g., from OpenCV. However, if the area of ​​interest 214 is too small, it can become difficult to robustly identify the correct optical machine-readable reference code 210 by comparison in the pixel information space, as the evaluations are more or less random.

[0056] Therefore, in various aspects of this disclosure, the processor can be configured to use deep image features (e.g., using a CNN) to compute the similarity between a given area of ​​interest 214 and a given (e.g., reduced) optical machine-readable reference code 210. For example, the processor can be configured to use a pre-trained version of ResNet18 (or ResNet50) to obtain a neural network, e.g., a CNN, capable of determining the feature vectors for both the area of ​​interest 214 (froi) and the respective optical machine-readable reference code 210 (fref). Subsequently, a similarity value is defined, e.g., as the mean distance between the two vectors, with lower values ​​representing higher similarity. Mathematically expressed, this is: s=1n‖froi−fref‖2, where “n” is the number of feature entries (e.g., a standard ResNet18 feature vector has a resolution of 64 * 8 * 8 pixels).

[0057] With this feature-matching approach, it is possible to reliably identify the correct optically machine-readable reference code 210 even if the area of ​​interest 214 has fewer pixels (e.g. 20 * 20 pixels) than the original optically machine-readable reference code 210 (e.g. 25 * 25 pixels).

[0058] The system described above was tested with 400 randomly generated QR codes in addition to the one present in the digital image, and compared to a conventional detector (zbar) with template matching and feature matching. As shown in diagram 500 in Fig. As shown in Figure 5, feature matching leads to good recognition of the QR codes even at a distance of more than 1.5 meters (see first feature 502, which indicates an average rating of "good match", and second feature 504, which indicates an average rating of "false match").

[0059] It should be noted that the evaluation can be further refined, for example by introducing different weights for different features of the feature descriptions, for example depending on the variance observed in the optically machine-readable reference codes 210.

[0060] One aspect of the evaluation can be to consider the orientation of the code within the given region of interest. In this respect, the processor can be configured to distinguish between clearly identifiable rotations, e.g., 45 degrees, and barely perceptible rotations, i.e., 90 degrees, 180 degrees, and 270 degrees, with respect to the normal orientation of an optically machine-readable reference code. The reason the latter three are rather difficult to identify is that a region of interest, e.g., in a digital (e.g., black and white) image, is usually an axis-parallel oriented rectangle. In the first case, the processor can easily calculate the rotation by considering the rotation of the rectangle in the digital (e.g., black-xml-ph-0000@deepl.internalimage).One might think that a QR code or a data matrix code has a clearly defined layout with markings for top and bottom, however, these markings are hardly recognizable from a greater distance, which is why these features cannot be used.

[0061] Instead, in various aspects of this disclosure, the processor may be configured to correct each identified area of ​​interest 214 with respect to a rotation if the rotation is not close to 90 degrees, 180 degrees, or 270 degrees. The processor may be configured not to make any correction for multiples of 90 degrees. Instead, the processor may be configured to determine the evaluation for four areas (derived from a given area of ​​interest 214): the original rotated area of ​​interest 214 (r), r rotated a further n times 90 degrees (n = 1, 2, 3), e.g., the first area of ​​interest 602 (e.g., rotated by 90°), the second area of ​​interest 604 (e.g., rotated by 180°), and the third area of ​​interest 606 (e.g., rotated by 270°), as in Fig. Figure 6 shows that a correct alignment of the area of ​​interest 214 (e.g., a correct alignment of the candidate for the optical machine-readable code) (i.e., aligned with the respective optical machine-readable reference code 210) achieves the lowest false detection results (i.e., the best rating).

[0062] Finally, for each area of ​​interest 214 that contains a valid optical machine-readable reference code 210 (i.e., determines a match value below a predefined threshold), the processor implementing the code-matcher provides the best-matching optical machine-readable reference code 210 and the associated determined match value (which can also be called the associated confidence value). With this data, the (e.g., goods / production management) system 200 can be queried, and the next appropriate steps in, for example, the production line or warehouse management can be initiated, as described in Fig. 2 is illustrated, for example, by generating instructions for controlling a robot 220, e.g., to deliver the respective physical object to a location associated with the optical machine-readable code that has been matched with a respective optical machine-readable reference code 210. For example, the processor can be configured to provide a list 222 of the comparison results of the (e.g., best-matching, e.g., 5 best-matching, e.g., 10 best-matching) optical machine-readable reference codes 210 for a respective area of ​​interest 214. The processor can be configured to select the optical machine-readable reference code 210 with the best match value from the list 222 as the best-matching optical machine-readable reference code 224 and to read the information encoded by the best-matching optical machine-readable reference code 224 from the database 212.

[0063] Furthermore, the processor can be configured to generate control instructions according to information encoded by the best matching optical machine-readable reference code 224, or according to information associated with a physical object associated with the best matching optical machine-readable reference code 224, which is stored, for example, in the database 212.

[0064] Although the processor can be configured to use deep image features, no feature extractor training (e.g., of a CNN) is required. Instead, pre-trained models such as ResNet18 on ImageNet (or ResNet50) or any other pre-trained neural network, e.g., any other pre-trained convolutional neural network, can be used immediately. This makes it very convenient to use, as the System 200 is ready to go right out of the box.

[0065] In various aspects of this disclosure, the processor may be configured to implement one or more of the following measures to further improve the recognition rate: It may be possible to use or implement a camera's depth sensor to obtain additional geometric data of the various objects in the field of view. This could be, for example, the dimensions of a boundary box in 3D (i.e., width, height, length) or a 3D point cloud. This information, combined with the set of best-case matches, can be used to select the most probable object / code. For instance, if several optically machine-readable reference codes have more or less the same score, this additional 3D data can help resolve ambiguities. - The (e.g., inventory) database 212 can be represented as a knowledge graph, which can help to accelerate the retrieval of relevant optical machine-readable reference codes 610, and, in combination with additional functions as explained in the first point, can help to further improve the reliability of the optical machine-readable code and the associated physical object.

[0066] In various aspects of this revelation, the System 200 can increase the distance at which, for example, the camera can be placed, thereby also increasing the area covered. This reduces the number of cameras required or allows the use of lower-resolution cameras, meaning customers can also use depth cameras for this purpose. Both options can lead to significant cost savings.

[0067] Furthermore, the system can use QR codes for internal tracking of goods, so such a solution also benefits our own merchandise management systems.

[0068] Fig. Figure 7 shows a System 700 for retrieving information represented by an optically machine-readable code. The System 700 can include a memory 702 (e.g., one that stores instructions 706 which, when executed by a processor 704, implement the following process) and the processor 704 coupled to the memory 702. The processor can be configured to retrieve a digital feature description of a candidate for an optically machine-readable code;for at least one optically machine-readable reference code from a plurality of optically machine-readable reference codes, wherein each optically machine-readable reference code from the plurality of optically machine-readable reference codes is assigned to a physical object to uniquely identify the respective physical object, wherein each optically machine-readable reference code is described by a respective reference code feature description in the feature space: comparing the feature description of the optically machine-readable candidate code with the reference code feature description to generate a comparison result; and determining that the at least one optically machine-readable reference code represents the optically machine-readable candidate code if the comparison result satisfies a predefined similarity criterion;If it is determined that the at least one optically machine-readable reference code represents the optically machine-readable candidate code, the information represented by the optically machine-readable reference code shall be determined to be the information represented by the optically machine-readable code.

[0069] In various aspects of this disclosure, the 704 processor may be configured to: receive a digital depth image that includes color and / or luminance information and depth information associated with one or more pixels of the digital depth image; identify the candidate for an optically machine-readable code by performing an object segmentation process on at least a portion of the digital depth image using the color and / or luminance information associated with the pixels of at least that portion of the digital depth image; the depth information associated with the pixels of at least that portion of the digital depth image; a reference form of an optically machine-readable reference code; and a reference size of the optically machine-readable reference code.

[0070] Fig. Figure 8 shows a block diagram illustrating a method 800 for retrieving information represented by an optical machine-readable code according to various aspects of this disclosure. The method 800 may include: retrieving 802 a digital feature description of a candidate for an optical machine-readable code, for at least one optical machine-readable reference code from a plurality of optical machine-readable reference codes 804, wherein each optical machine-readable reference code of the plurality of optical machine-readable reference codes is associated with a physical object to uniquely identify the respective physical object, wherein each optical machine-readable reference code is described by a respective reference code feature description in the feature space; comparing 806 the feature description of the optical machine-readable candidate code with the reference code feature description to produce a comparison result;and determine 808 that the at least one optically machine-readable reference code represents the optically machine-readable candidate code if the comparison result satisfies a predefined similarity criterion; if it is determined that the at least one optically machine-readable reference code represents the optically machine-readable candidate code, determine 810 information represented by the optically machine-readable reference code than the information represented by the optically machine-readable code.;

[0071] Fig.Figure 9 shows a block diagram illustrating a procedure 900 for identifying a candidate for an optically machine-readable code according to various aspects of this disclosure.Method 900 can be described in 902 as receiving a digital depth image that has color information and / or luminance information and depth information associated with one or more pixels of the digital depth image, and in 904 as identifying the candidate for an optically machine-readable code by performing an object segmentation process on at least one part of the digital depth image using the color information and / or luminance information associated with the pixels of at least the part of the digital depth image, the depth information associated with the pixels of at least the part of the digital depth image, a reference shape of an optically machine-readable reference code, and a reference size of the optically machine-readable reference code.

[0072] The following explains various aspects of this revelation:

[0073] Example 1 is a system for retrieving information represented by an optically machine-readable code. The system may include a memory and a processor coupled to the memory. The processor may be configured to retrieve a digital feature description of a candidate for an optically machine-readable code;for at least one optically machine-readable reference code from a plurality of optically machine-readable reference codes, wherein each optically machine-readable reference code of the plurality of optically machine-readable reference codes is assigned to a physical object in order to uniquely identify the respective physical object, wherein each optically machine-readable reference code is described by a respective reference code feature description in the feature space: comparing the feature description of the candidate for an optically machine-readable code with the feature description of the reference code to generate a comparison result; and determining that the at least one optically machine-readable reference code represents the candidate for an optically machine-readable code if the comparison result satisfies a predefined similarity criterion;If it is determined that the at least one optically machine-readable reference code represents the optically machine-readable candidate code, information represented by the optically machine-readable reference code is determined to be different from the information represented by the optically machine-readable code.

[0074] In Example 2, the subject of Example 1 may optionally include the processor being further configured to determine the digital feature description by determining the digital feature description that describes the features of the candidate for an optically machine-readable code in a feature space, wherein the candidate for an optically machine-readable code is determined from a digital image.

[0075] In Example 3, the subject of Example 2 may optionally include the processor being further configured to determine a digital feature description that describes features of a candidate for an optically machine-readable code in a feature space by transforming pixel information associated with pixels of the candidate for an optically machine-readable code of at least one part of a digital image from a pixel information space into the feature space.

[0076] In Example 4, the subject of Example 3 may optionally include the processor being further configured to transform the pixel information using one or more digital filters.

[0077] In Example 5, the subject of any of Examples 3 or 4 may optionally include the processor being further configured to transform the pixel information using a neural transformation network, for example, the neural transformation network implementing a deep learning model, in an encoder stage of a convolutional neural network.

[0078] In Example 6, the subject of any of Examples 3 or 4 may optionally include the processor being further configured to transform the pixel information using a neural transformation network, for example, the neural transformation network implementing a deep learning model.

[0079] In Example 7, the subject of any of Examples 1 to 6 may optionally include the selection of multiple optically machine-readable reference codes from a group of optically machine-readable codes consisting of a barcode, a quick response code, and a data matrix code.

[0080] In Example 8, the subject of any of Examples 1 to 7 may optionally include that the pixel information in the pixel information space includes color information and / or brightness information and / or hue information associated with a pixel.

[0081] In Example 9, the subject of any of Examples 1 to 8 may optionally include that the processor is further configured to determine a feature description that describes features of a candidate for an optical machine-readable code by performing a segmentation process for candidates for optical machine-readable codes at least on that part of the digital image.

[0082] In Example 10, the subject of Example 9 may optionally include that the digital image is a digital depth image that contains color information and / or luminance information and depth information associated with one or more pixels of the digital depth image.The processor is further configured to perform the segmentation process for candidates for optical machine-readable codes, which includes: receiving the digital depth image; identifying candidates for optical machine-readable codes () as candidates for optical machine-readable codes using the following information: the color information and / or luminance information associated with the pixels of at least part of the digital depth image; the depth information associated with the pixels of at least part of the digital depth image; a reference shape of an optical machine-readable reference code; and a reference size of the optical machine-readable reference code.

[0083] In Example 11, the subject of any of Examples 9 or 10 may optionally include that the processor is further configured to perform the optical machine-readable candidate code segmentation process, which includes: determining a binary image of at least the section of the digital image; and performing the optical machine-readable candidate code segmentation process on at least one section of the binary image.

[0084] In Example 11, the subject of any of Examples 9 to 11 may optionally include that the reference size is 25 mm * 25 mm.

[0085] In Example 13, the subject of any of Examples 9 to 12 can optionally include that the reference shape is a rectangle.

[0086] In Example 14, the subject of any of Examples 1 to 13 may optionally include that the processor is further configured to rotate the candidate for an optically machine-readable code and to determine a feature description of the rotated candidate for an optically machine-readable code in the feature space as a feature description of the candidate for an optically machine-readable code.

[0087] In Example 15, the subject of any of Examples 1 to 14 may optionally include that the processor is further configured to perform the following for at least one rotated optical machine-readable reference code: comparing the feature description of the candidate optical machine-readable code with the feature description of the rotated optical machine-readable reference code to produce a comparison result; and determining that the at least one optical machine-readable reference code represents the candidate optical machine-readable code if the comparison result satisfies a predefined similarity criterion.

[0088] In Example 16, the subject of any of Examples 1 to 15 may optionally include that the processor is further configured to: generate a confidence value for the comparison result; and determine that the at least one optically machine-readable reference code represents the optically machine-readable reference code if the confidence value satisfies a predefined confidence criterion.

[0089] In Example 17, the subject of any of Examples 1 to 16 may optionally include that the processor is further configured to determine that the at least one optical machine-readable reference code representing the candidate for the optical machine-readable code uses at least one of the following pieces of information about the physical object: weight of the physical object; size of the physical object; shape of the physical object; and / or color of the physical object.

[0090] In Example 18, the subject of any of Examples 1 to 17 may optionally include that the processor is further configured to generate instructions to control an actor, e.g. a transporter, e.g. a transport vehicle, according to information stored in conjunction with the determined optically machine-readable reference code.

[0091] Example 19 is a system for identifying candidates for optically machine-readable code. The system can include a memory and a processor coupled to the memory.The processor can be configured to: receive a digital depth image containing color and / or luminance information and depth information associated with one or more pixels of the digital depth image; identify the candidate for an optically machine-readable code by performing an object segmentation process on at least a portion of the digital depth image using the color and / or luminance information associated with the pixels of at least that portion of the digital depth image; the depth information associated with the pixels of at least that portion of the digital depth image; a reference form of an optically machine-readable reference code; and a reference size of the optically machine-readable reference code.

[0092] In Example 20, the subject of Example 19 may optionally include the processor being further configured to perform the object segmentation process, which includes: determining a binary image of at least the section of the digital image; and performing the object segmentation process on at least one section of the binary image.

[0093] In Example 21, the subject of any of Examples 19 or 20 may optionally include that the reference size is 25 mm * 25 mm.

[0094] In Example 22, the subject of any of Examples 19 to 21 may optionally include that the reference shape is a rectangle.

[0095] Example 23 is a method for retrieving information represented by an optically machine-readable code. The method may include: retrieving a digital feature description of a candidate for an optically machine-readable code; for at least one optically machine-readable reference code from a plurality of optically machine-readable reference codes, wherein each optically machine-readable reference code from the plurality of optically machine-readable reference codes is associated with a physical object to uniquely identify the respective physical object, wherein each optically machine-readable reference code is described by a respective reference code feature description in the feature space; comparing the feature description of the candidate for an optically machine-readable code with the feature description of the reference code to generate a comparison result;and determine that the at least one optically machine-readable reference code represents the candidate for an optically machine-readable code if the comparison result satisfies a predefined similarity criterion; if it is determined that the at least one optically machine-readable reference code represents the optically machine-readable candidate code, determine information represented by the optically machine-readable reference code than the information represented by the optically machine-readable code.

[0096] In Example 24, the subject of Example 23 may optionally include determining the digital feature description, which includes determining the features of the candidate for an optically machine-readable code in a feature space, wherein the candidate for an optically machine-readable code is determined from a digital image.

[0097] In Example 25, the subject of Example 24 may optionally include the process of determining a digital feature description that describes features of a candidate for an optically machine-readable code in a feature space, which involves transforming pixel information associated with pixels of the candidate for an optically machine-readable code of at least one part of a digital image from a pixel information space into the feature space.

[0098] In Example 26, the subject of Example 25 may optionally include the transformation of pixel information using one or more digital filters.

[0099] In Example 27, the subject of any of Examples 25 or 26 may optionally include the transformation of pixel information using an encoder stage of a convolutional neural network, for example, the encoder stage of the convolutional neural network implementing a deep learning model.

[0100] In Example 28, the subject of any of Examples 25 or 26 may optionally include the transformation of pixel information using a neural transformation network, e.g., the neural transformation network implementing a deep learning model.

[0101] In Example 29, the subject of any of Examples 23 to 28 may optionally include the selection of multiple optically machine-readable reference codes from a group of optically machine-readable codes consisting of a barcode, a quick response code, and a data matrix code.

[0102] In Example 30, the subject of any of Examples 23 to 29 may optionally include that the pixel information in the pixel information space includes color information and / or luminance information and / or hue information associated with a pixel.

[0103] In Example 31, the subject of any of Examples 23 to 30 may optionally include that determining a feature description that describes features of a candidate for an optically machine-readable code includes performing a segmentation process for candidates for optically machine-readable codes, at least on that part of the digital image.

[0104] In Example 32, the subject of Example 31 may optionally include that the digital image is a digital depth image that has color information and / or luminance information and depth information associated with one or more pixels of the digital depth image, and that performing the segmentation process for a candidate for an optical machine-readable code includes: receiving the digital depth image; identifying the candidate for an optical machine-readable code as a candidate for an optical machine-readable code using the following information: the color information and / or luminance information associated with the pixels of at least part of the digital depth image, the depth information associated with the pixels of at least part of the digital depth image, a reference shape of an optical machine-readable reference code, and a reference size of the optical machine-readable reference code.

[0105] In Example 33, the subject of any of Examples 31 or 32 may optionally include that the implementation process for segmenting the candidate for an optically machine-readable code comprises: determining a binary image of at least part of the digital image; and performing the segmentation process for the candidate for an optically machine-readable code on at least part of the binary image.

[0106] In Example 34, the subject of any of Examples 31 to 33 may optionally include that the reference size is 25 mm * 25 mm.

[0107] In Example 35, the subject of any of Examples 31 to 34 may optionally include that the reference shape is a rectangle.

[0108] In Example 36, the subject of any of Examples 23 to 35 may optionally include that the procedure further comprises: rotating the candidate for an optically machine-readable code; and determining a feature description of the rotated candidate for an optically machine-readable code in the feature space as a feature description of the candidate for an optically machine-readable code.

[0109] In Example 37, the subject matter of any of Examples 23 to 36 may optionally include the procedure further comprising: performing the following steps for at least one rotated optical machine-readable reference code: comparing the feature descriptions of the candidate optical machine-readable code with the feature descriptions of the rotated optical machine-readable reference code to produce a comparison result; and determining that the at least one optical machine-readable reference code represents the candidate optical machine-readable code if the comparison result satisfies a predefined similarity criterion.

[0110] In Example 38, the subject of any of Examples 23 to 37 may optionally include that the procedure further comprises: generating a confidence value for the comparison result; and determining that the at least one optically machine-readable reference code represents the optically machine-readable candidate code if the confidence value satisfies a predefined confidence criterion.

[0111] In Example 39, the subject of any of Examples 23 to 38 may optionally include that the at least one optically machine-readable reference code representing the candidate for the optically machine-readable code is determined using at least one of the following pieces of information about the physical object: weight of the physical object; size of the physical object; shape of the physical object; and / or color of the physical object.

[0112] In Example 40, the subject of any of Examples 23 to 39 may optionally include that the processor is further configured to generate instructions for controlling an actor, e.g., a transporter, e.g., a transport vehicle, according to information stored in conjunction with the determined optically machine-readable reference code.

[0113] Example 41 is a method for identifying a candidate optical machine-readable code. The method may include: receiving a digital depth image that has color and / or luminance information and depth information associated with one or more pixels of the digital depth image; identifying the candidate optical machine-readable code by performing an object segmentation process on at least a portion of the digital depth image using the color and / or luminance information associated with the pixels of at least that portion of the digital depth image; the depth information associated with the pixels of at least that portion of the digital depth image; a reference form of an optical machine-readable reference code; and a reference size of the optical machine-readable reference code.

[0114] In Example 42, the subject of Example 41 may optionally include the object segmentation process comprising: determining a binary image of at least part of the digital image; and performing the object segmentation process on at least part of the binary image.

[0115] In Example 43, the item of one of Examples 41 or 42 may optionally include that the reference size is 25 mm * 25 mm.

[0116] In Example 44, the subject of any of Examples 41 to 43 may optionally include that the reference shape is a rectangle.

[0117] Example 45 is a computer-readable medium that stores instructions which, when executed by a processor, implement a procedure for determining information represented by an optically machine-readable code from any of Examples 23 to 40.

[0118] Example 46 is a computer-readable medium that stores instructions which, when executed by a processor, implement a procedure for determining information represented by an optically machine-readable code from any of Examples 41 to 44.

[0119] Example 47 is a computer program element that is set up such that, when executed by a processor, it implements a procedure for determining information represented by an optically machine-readable code from any of Examples 23 to 40.

[0120] Example 48 is a computer program element that is set up such that, when executed by a processor, it implements a procedure for determining information represented by an optically machine-readable code from any of Examples 41 to 44.

[0121] Example 49 is a system for retrieving information represented by an optically machine-readable code. The system may include: means for retrieving a digital feature description of a candidate for an optically machine-readable code; means to uniquely identify, for at least one optically machine-readable reference code from a plurality of optically machine-readable reference codes, wherein each optically machine-readable reference code of the plurality of optically machine-readable reference codes is associated with a physical object, and wherein each optically machine-readable reference code is described by a respective reference code feature description in the feature space: comparing the feature description of the optically machine-readable candidate code with the reference code feature description to generate a comparison result;and determining that the at least one optically machine-readable reference code represents the optically machine-readable candidate code if the comparison result satisfies a predefined similarity criterion; means to determine, if it is determined that the at least one optically machine-readable reference code represents the optically machine-readable candidate code, information represented by the optically machine-readable reference code as the information represented by the optically machine-readable code.

[0122] In Example 50, the subject of Example 49 may optionally include the means of determining the digital feature description by determining the digital feature description that describes the features of the candidate for an optically machine-readable code in a feature space, and determining the candidate for an optically machine-readable code from a digital image.

[0123] In Example 51, the subject of Example 50 may optionally include the means of determining a digital feature description that describes features of a candidate for an optically machine-readable code in a feature space, transforming pixel information associated with pixels of the candidate for an optically machine-readable code of at least one part of a digital image from a pixel information space into the feature space.

[0124] In Example 52, the subject of Example 51 may optionally include the use of one or more digital filters as the means for transforming the pixel information.

[0125] In Example 53, the subject of one of Examples 50 or 51 may optionally include the means of transforming the pixel information using an encoder stage of a convolutional neural network, e.g., the encoder stage of the convolutional neural network implementing a deep learning model.

[0126] In Example 54, the subject of any of Examples 50 or 51 may optionally include the use of a neural transformation network as the means for transforming the pixel information, for example, the neural transformation network that implements a deep learning model.

[0127] In Example 55, the subject of any of Examples 49 to 54 may optionally include that the multiple optically machine-readable reference codes are selected from a group of optically machine-readable codes consisting of a barcode, a quick response code, and a data matrix code.

[0128] In Example 56, the subject of any of Examples 49 to 55 may optionally include that the pixel information in the pixel information space includes color information and / or brightness information and / or hue information associated with a pixel.

[0129] In Example 57, the subject of any of Examples 49 to 56 may optionally include that the means for determining a feature description that describes features of a candidate for an optically machine-readable code include means for carrying out a segmentation process for candidates for optically machine-readable codes at least on that part of the digital image.

[0130] In Example 58, the subject of Example 57 may optionally include that the digital image is a digital depth image containing color information and / or luminance information and depth information assigned to one or more pixels of the digital depth image.The means for performing the segmentation process for candidates for optical machine-readable codes include: means for receiving the digital depth image; means for identifying the candidate for an optical machine-readable code as a candidate for an optical machine-readable code using the following information: the color information and / or luminance information associated with the pixels of at least part of the digital depth image; the depth information associated with the pixels of at least part of the digital depth image; a reference shape of an optical machine-readable reference code; and a reference size of the optical machine-readable reference code.

[0131] In Example 59, the subject of one of Examples 57 or 58 may optionally include that the means for performing the segmentation process for the optically machine-readable candidate code comprise: means for determining a binary image of at least part of the digital image; and means for performing the segmentation process for the optically machine-readable candidate code on at least part of the binary image.

[0132] In Example 60, the subject of any of Examples 57 to 59 may optionally include that the reference size is 25 mm * 25 mm.

[0133] In Example 61, the subject of any of Examples 57 to 60 may optionally include that the reference shape is a rectangle.

[0134] In Example 62, the subject of any of Examples 49 to 61 may optionally include that the system further comprises: means for rotating the candidate for an optically machine-readable code; and means for determining a feature description of the rotated candidate for an optically machine-readable code in the feature space as a feature description of the candidate for an optically machine-readable code.

[0135] In Example 63, the subject of any of Examples 49 to 62 may optionally include that the system further comprises: means for performing the following steps for at least one rotated optical machine-readable reference code: comparing the feature description of the candidate optical machine-readable code with the feature description of the rotated optical machine-readable reference code to produce a comparison result; and determining that the at least one optical machine-readable reference code represents the candidate optical machine-readable code if the comparison result satisfies a predefined similarity criterion.

[0136] In Example 64, the subject of any of Examples 49 to 63 may optionally include that the system further comprises: means for generating a confidence value for the comparison result; and means for determining that the at least one optically machine-readable reference code represents the optically machine-readable candidate code if the confidence value satisfies a predefined confidence criterion.

[0137] In Example 65, the subject of any of Examples 49 to 64 may optionally include the means of determining that the at least one optically machine-readable reference code representing the candidate for the optically machine-readable code uses at least one of the following pieces of information about the physical object: weight of the physical object; size of the physical object; shape of the physical object; and / or color of the physical object.

[0138] In Example 66, the subject of any of Examples 49 to 65 may optionally include that the system further comprises means for generating instructions for controlling an actor, e.g. a transporter, e.g. a transport vehicle, according to information stored in conjunction with the determined optically machine-readable reference code.

[0139] Example 67 is a system for identifying a candidate optical machine-readable code. The system may include: means for receiving a digital depth image that has color and / or luminance information and depth information associated with one or more pixels of the digital depth image; means for identifying the candidate optical machine-readable code by performing an object segmentation process on at least a part of the digital depth image using the color and / or luminance information associated with the pixels of at least that part of the digital depth image; the depth information associated with the pixels of at least that part of the digital depth image; a reference form of an optical machine-readable reference code; and a reference size of the optical machine-readable reference code.

[0140] In Example 68, the subject of Example 67 may optionally include the means of performing the object segmentation process: means of determining a binary image of at least part of the digital image; and means of performing the object segmentation process on at least part of the binary image.

[0141] In Example 69, the subject of any Example 67 or 68 may optionally include that the reference size is 25 mm * 25 mm.

[0142] In Example 70, the subject of any of Examples 67 to 69 may optionally include that the reference shape is a rectangle.

[0143] Although the invention has been demonstrated and described with reference to specific embodiments, it should be clear to those skilled in the art that various modifications in form and detail can be made without departing from the spirit and scope of the invention as defined by the accompanying claims. The scope of the invention is thus specified by the accompanying claims, and all modifications that fall within the meaning and equivalence of the claims are therefore included.

Images