Method and system for collecting data for training machine learning (ML) models

Abstract

The present invention is entitled "Methods and systems for collecting data for training machine learning (ML) models". Various embodiments disclosed herein describe a method comprising receiving marker data from a marker scanner. The marker data includes at least decoded data obtained based on decoding a marker in an image, an image tile including a portion of the marker, and / or the position of one or more corners of the portion of the marker in the image. In addition, the method includes generating an image of an ideal marker based at least on the decoded data. Thereafter, the image of the ideal marker is modified to generate a modified image of the ideal marker. In addition, the portion of the marker is retrieved from the modified image. A clean image tile includes a portion of the ideal marker. In addition, the method includes generating training data, wherein the training data includes the portion of the marker and the portion of the ideal marker.

Description

Technical Field

[0001] Exemplary embodiments of the present disclosure relate generally to machine learning (ML) models, and more particularly, to systems and methods for acquiring data for training ML models. Background Art

[0002] Generally speaking, a machine learning (ML) model may correspond to a mathematical model and / or a probabilistic model that is capable of predicting an outcome. Furthermore, the ML model may be optimized based on the domain in which the ML model is to be used. For example, in a tag scanning domain, the ML model may be used to identify tags in the field of view of a tag scanner. Furthermore, in some cases, the ML model may be used to decode tags. Prior to deploying the ML model, the ML model may be trained on training data. Summary of the invention

[0003] Various embodiments disclosed herein describe a method comprising receiving marker data from a marker scanner. The marker data comprises at least decoded data obtained based on decoding a marker in an image, an image tile comprising a portion of the marker, and / or the position of one or more corners of the portion of the marker in the image. In addition, the method comprises generating an image of an ideal marker based at least on the decoded data. Thereafter, modifying the image of the ideal marker to generate a modified image of the ideal marker. In addition, retrieving the portion of the marker from the modified image. A clean image tile comprises a portion of the ideal marker. In addition, the method comprises generating training data, wherein the training data comprises the portion of the marker and the portion of the ideal marker.

[0004] Various embodiments shown herein describe a central server, which includes a memory device with a set of instructions. A processor is configured to execute the set of instructions, wherein the processor is configured to receive tag data from a tag scanner, wherein the tag data at least includes decoded data obtained based on decoding the tag in the image, an image tile including a portion of the tag and / or the position of one or more corners of the portion of the tag in the image. The processor is also configured to generate an image of an ideal tag based at least on the decoded data. In addition, the processor is configured to modify the image of the ideal tag to generate a modified image of the ideal tag, wherein the orientation of the ideal tag in the modified image is the same as the orientation of the tag in the image. In addition, the processor is configured to retrieve a clean image tile from the modified image of the ideal tag based on the position of one or more corners of the portion of the tag in the image, wherein the clean image tile includes a portion of the ideal tag. In addition, the processor is configured to generate training data, wherein the training data includes the portion of the tag and the portion of the ideal tag.

[0005] Various embodiments shown herein describe a marker scanner comprising an image capture unit configured to capture an image of a field of view of the marker scanner, wherein the image comprises a marker. In addition, the marker scanner comprises a processor communicatively coupled to the image capture unit. The processor is configured to segment the image into a plurality of image tiles. In addition, the processor is configured to determine a quality score for each of one or more of the plurality of image tiles, wherein the one or more image tiles comprise a portion of the marker. In addition, the processor is configured to select an image tile of the one or more image tiles based on the quality score associated with each of the one or more image tiles. In addition, the processor is configured to transmit the image tile to a central server, wherein the image tile constitutes training data for training a machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The description of the exemplary embodiments may be read in conjunction with the accompanying drawings. It should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some of the elements are exaggerated relative to other elements. Embodiments incorporating the teachings of the present disclosure are shown and described with respect to the accompanying drawings set forth herein, in which:

[0007] Figure 1 A system environment is shown in which various embodiments of the present disclosure are implemented;

[0008] Figure 2 shows a block diagram of a tag scanner according to one or more embodiments described herein;

[0009] Figure 3 A flow chart illustrating a method for operating a marking scanner according to one or more embodiments described herein is shown;

[0010] Figure 4 shows an exemplary image according to one or more embodiments described herein;

[0011] Figure 5 A flowchart of a method for retrieving an image tile from an image according to one or more embodiments described herein is shown;

[0012] Figure 6 shows an exemplary image according to one or more embodiments described herein;

[0013] Figure 7 A block diagram of a central server according to one or more embodiments described herein is shown;

[0014] Figure 8A flow chart illustrating a method for operating a central server according to one or more embodiments described herein;

[0015] Fig. 9 A flowchart of a method for rendering a modified image according to one or more embodiments described herein is shown;

[0016] Fig.10 A flowchart showing a method for removing non-marked image portions (including auxiliary information) from marked portions according to one or more embodiments described herein; and

[0017] Fig.11 is an exemplary flow chart illustrating operations performed at a central server according to one or more embodiments described herein. DETAILED DESCRIPTION

[0018] Some embodiments of the present disclosure will be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, embodiments of the present disclosure. In fact, these disclosures may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure satisfies applicable legal requirements. Throughout the content, similar reference numerals refer to similar elements.

[0019]

[0026] Unless the context requires otherwise, throughout this specification and the claims that follow, the word "comprise" and variations such as "include" and "have" are to be interpreted in an open-ended sense, ie, to mean "including, but not limited to."

[0020] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, one or more particular features, structures, or characteristics from one or more embodiments may be combined in any suitable manner in one or more other embodiments.

[0021] The word “example” or “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other implementations.

[0022] If the specification states that a component or feature "may", "can", "could", "should", "will", "preferably", "likely", "usually", "optionally", "for example", "often" or "might" (or other such language) be included or have a property, the specific component or feature is not required to be included or have that property. Such components or features may optionally be included in some embodiments, or may be excluded.

[0023] The terms "electronically coupled", "electronically coupling", "electronically couple", "communicate with", "electronically communicate with", or "connect" in this disclosure refer to two or more components being connected (directly or indirectly) via wired means (such as but not limited to a system bus, wired Ethernet) and / or wireless means (such as but not limited to Wi-Fi, Bluetooth, ZigBee) so that data and / or information can be transmitted to and / or received from these components.

[0024] The term "marking" is broadly intended to include any marking or machine-readable code, including linear symbols, 2D barcodes (such as QR codes and data matrix codes), RFID tags, IR tags, near field communication (NFC) tags, and characters that can be read by a computing device (e.g., a marking scanner). A marking is typically a graphical representation of information (e.g., data), such as a product number, a package tracking number, a patient barcode symbol identifier, a medication tracking identifier, a personnel barcode symbol identifier, etc.

[0025] The term "quality" may refer to a standard or protocol based on which content may be evaluated or compared to one another. For example, the quality of an image may be evaluated based on the sharpness of the image, the noise in the image, the dynamic range of the image, etc. In some examples, the quality of the image may be further evaluated based on the quality of certain parts of the image. For example, the quality of the image may be evaluated based on the quality of the markers in the image. To this end, the quality of the markers may be evaluated based on the ANSI X3.182, ISO 15415, and ISO / IEC 15416 standards.

[0026] The term "position" may refer to a point or a single coordinate in an image. In some examples, a position in an image may be represented in one or more coordinate systems, such as, but not limited to, a Cartesian coordinate system, a polar coordinate system, and the like.

[0027] The processor in the tag scanner can use a machine learning (ML) model optimized for the tag scanning operation to perform various operations. For example, the processor of the tag scanner can use the ML model to identify tags in the field of view of the tag scanner.

[0028] Embodiments of the present invention recognize that, in general, ML models are trained on training data before being deployed in a tagging scanner. One of the main obstacles to creating robust solutions based on machine learning is the lack of sufficient amounts of high-quality training data. Creating robust solutions for applications requires hundreds of thousands of samples or more to fully prepare the ML model for unconstrained operation in the real world.

[0029] In supervised training, the difficulty of obtaining sufficient training data is exacerbated by the need to obtain training pairs rather than just single unrelated samples. For example, in order to train an image reconstruction ML model, training pairs consisting of a "corrupted" image and a corresponding "clean" image are required. The ML model is trained to transform a given corrupted image into the desired clean image. Clean images are often not easily available, which poses a huge challenge to creating a robust supervised learning solution.

[0030] Additionally, it is important to note that in order for the training data to adequately represent actual images seen in a work environment, the training data must actually come from the work environment. ML models trained with laboratory and simulated data may have limited accuracy and tend to deliver subpar results.

[0031] In order to create robust ML applications in the barcode decoding application space, new techniques are clearly needed to overcome the hurdle of obtaining a sufficiently large, representative and diverse set of training image pairs from a variety of devices operating in their native working environments.

[0032] In some examples, the training data may correspond to data that may include one or more features and / or one or more labels specific to the domain used to train the ML model. In some examples, the one or more features may correspond to parameters that may constitute input to the ML model. In addition, the one or more labels in the training data may correspond to expected outputs of the ML model. For example, in a tag scanning domain, the one or more features in the training data (i.e., the input to the ML model) may correspond to images of tags captured by a tag scanner. In addition, the one or more labels in the training data may include clean images of the tags (i.e., the expected outputs of the ML model). However, due to various operational factors associated with tag scanners, such as, but not limited to, ambient lighting conditions in which the tag scanner operates, the orientation of the tag scanner relative to the tag (to be captured), etc., capturing clean images using a tag scanner may be a challenge.

[0033] In addition, due to limited memory space and limited processor processing power in the tag scanner, the tag scanner may not be able to store and / or process a large number of images. For this reason, the tag scanner may not be able to generate training data and therefore may not be able to train the ML model using the training data. Therefore, in some examples, the tag scanner may be configured to transmit the images to a central server, where the central server may generate training data and the ML model may be trained using the training data. In one example, the tag scanner captures images that are approximately 2.3MB in size. Therefore, if a worker (using a tag scanner) scans approximately 1,500 tags during a working day, the tag scanner may generate approximately 3.5GB of data. Transmitting such a large amount of data to a central server every day may be inefficient and will require a large amount of network bandwidth.

[0034] According to the embodiments disclosed herein, a tag scanner may scan and decode a tag in an image to generate decoded data. During scanning and decoding of the tag, the tag scanner may be configured to determine one or more tag positions in the image. In some examples, the one or more tag positions may surround the tag in the image. Additionally or alternatively, the tag scanner may be configured to determine a barcode symbol identifier of the tag. The barcode symbol identifier of the tag may indicate the type of tag scanned by the tag reader. Some examples of the type of tag may include, but are not limited to, code 39, code 128, code 11, PDF417, data matrix code, QR code, Aztec code. Additionally or alternatively, the tag scanner may be configured to retrieve the portion of the tag (referred to as an image tile) from an image including a portion of the tag. In addition, the tag scanner may be configured to determine one or more image tile positions surrounding the portion of the tag in the image (i.e., the image tile). In an exemplary embodiment, the tag scanner may be configured to generate tag data. In some examples, the tag data may include the decoded data, the one or more tag locations in the image, the one or more image tile locations in the image, a bar code symbol identifier for the tag, the image tile, and the size of the image (e.g., the resolution of the image). The tag scanner may then be configured to transmit the tag data to the central server. The bandwidth required to transmit the tag data for the image tile (including the portion of the tag) is significantly reduced compared to transmitting the entire image.

[0035] In some examples, the central server may generate an ideal tag using the decoded data and a bar code symbol identifier associated with the tag (received by the server as tag data). In an exemplary embodiment, the ideal tag generated by the central server is an ideal form of the tag scanned by the tag reader. One of ordinary skill in the art will appreciate that the ideal tag corresponds to an image of the scanned tag. Thereafter, the central server may modify the image of the ideal tag so that the ideal tag (in the modified image) has the same orientation and size as the orientation and size of the tag in the scanned image. In some examples, the central server may modify the image of the ideal tag using a homography matrix. In an exemplary embodiment, the homography matrix may correspond to a transformation matrix that represents the relationship between the one or more tag positions in the scanned image and the one or more ideal tag positions.

[0036] In some examples, the central server may be further configured to retrieve a clean image tile from the modified image of the ideal tag using the one or more image tile positions (received in the tag data). In an exemplary embodiment, the clean image tile includes a portion of the ideal tag. In an exemplary embodiment, since the ideal tag has the same orientation and size as the orientation and size of the tag in the scanned image in the modified image, the clean image tile may include the same content as the content included in the scanned image tile. For example, both the scanned image tile and the clean image tile include the portion of the tag. More specifically, the portion of the ideal tag (included in the clean image tile) is the same as the portion of the scanned tag (included in the image tile). In some examples, the portion of the ideal tag (generated by the central server) may be noise-free and may be referred to as a clean portion of the tag. On the other hand, the portion of the tag (retrieved from the tag data) is considered to be a damaged portion of the tag. Thereafter, the central server may be configured to train a machine learning (ML) model using the clean portion of the tag and the damaged portion of the tag as training data.

[0037] Figure 1 A system environment 100 is shown in which various embodiments of the present disclosure are implemented. In an exemplary embodiment, the system environment 100 includes a badge scanner 102, a network 104, and a central server 106.

[0038] In an exemplary embodiment, the indicia scanner 102 may correspond to a mobile device, such as a handheld indicia scanner, a portable data terminal, a mobile phone, a tablet computer, a portable computer, etc., or may be a fixed terminal such as a fixed terminal fixed to a single location along an assembly line capable of capturing the one or more images. In an exemplary embodiment, the indicia scanner 102 may be capable of identifying and decoding indicia in the captured images, such as in combination with Figure 3Further described. In addition, the tag scanner 102 can be configured to generate tag data based on decoding the tag, such as Figure 3 Further described in. Figure 2 The structure of the indicia scanner 102 is further described.

[0039] The network 104 corresponds to a medium through which content and messages flow between various devices (e.g., the central server 106 and the tag scanner 102) in the system environment 100. Examples of the network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a wireless area network (WAN), a local area network (LAN), or a metropolitan area network (MAN). The various devices in the system environment 100 may connect to the network 104 according to various wired and wireless communication protocols, such as, but not limited to, the transmission control protocol and the Internet protocol (TCP / IP), the user datagram protocol (UDP), and 2G, 3G, 4G, or 5G communication protocols.

[0040] In an exemplary embodiment, the central server 106 may refer to a computing device that can be configured to communicate with the tag scanner 102. The central server 106 may include one or more processors and one or more memories. The one or more memories may include computer readable code that can be executed by the one or more processors to perform predetermined operations. In addition, the central server 106 may include one or more interfaces that can facilitate communication with the tag scanner 102 via the network 104. In an exemplary embodiment, the central server 106 may be configured to receive tag data from the tag scanner 102, such as Figure 8 In addition, the central server 106 can be configured to generate training data from the labeled data, such as Figure 8 Examples of central server 106 may include, but are not limited to, a personal computer, a laptop, a personal digital assistant (PDA), a mobile device, a tablet computer, or other such computing devices. Fig.10 The structure of the central server 106 is further described.

[0041] Figure 2 A block diagram of a landmark scanner 102 is shown according to one or more embodiments described herein. In some examples, the landmark scanner 102 includes a first processor 202 , a first memory device 204 , a first communication interface 206 , an image capture unit 208 , and a decoder unit 210 .

[0042] The first processor 202 may be implemented as a device including one or more microprocessors with one or more accompanying digital signal processors, one or more processors without accompanying digital signal processors, one or more coprocessors, one or more multi-core processors, one or more controllers, processing circuits, one or more computers, various other processing elements (including integrated circuits, such as, for example, application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)), or some combination thereof. Therefore, although in Figure 2 102. Although shown as a single processor in the figure, in an embodiment, the first processor 202 may include multiple processors and signal processing modules. The multiple processors may be implemented on a single electronic device, or may be distributed across multiple electronic devices that are collectively configured to function as the circuitry of the indicia scanner 102. The multiple processors may be in operative communication with each other and may be collectively configured to perform one or more functions of the circuitry of the indicia scanner 102, as described herein. In an exemplary embodiment, the first processor 202 may be configured to execute instructions stored in the first memory device 204 or otherwise accessible to the first processor 202. When these instructions are executed by the first processor 202, the circuitry of the indicia scanner 102 may be caused to perform one or more functions, as described herein.

[0043] Whether the first processor 202 is configured by a hardware method, a firmware / software method, or a combination thereof, the first processor may include an entity capable of performing operations according to the embodiments of the present disclosure while being configured accordingly. Thus, for example, when the first processor 202 is implemented as an ASIC, FPGA, etc., the first processor 202 may include specially configured hardware for performing one or more operations described herein. Alternatively, as another example, when the first processor 202 is implemented as an executor of instructions (such as those that may be stored in the first memory device 204), the instructions may specifically configure the first processor 202 to perform one or more algorithms and operations described herein.

[0044] Therefore, the first processor 202 used herein can refer to a programmable microprocessor, a microcomputer, or one or more multiprocessor chips, which can be configured by software instructions (applications) to perform various functions including the functions of the various embodiments described above. In some devices, multiple processors dedicated to wireless communication functions and a processor dedicated to running other applications can be provided. Software applications can be stored in internal memory before being accessed and loaded into the processor. The processor may include an internal memory sufficient to store application software instructions. In many devices, the internal memory can be a volatile or non-volatile memory such as a flash memory or a mixture of the two. The memory can also be located inside another computing resource (for example, to enable computer-readable instructions to be downloaded via the Internet or another wired or wireless connection).

[0045] The first memory device 204 may include suitable logic, circuitry, and / or interfaces adapted to store a set of instructions executable by the first processor 202 to perform predetermined operations. Some of the commonly known memory implementations include, but are not limited to, a hard disk, a random access memory, a cache memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and an electrically erasable programmable read-only memory (EEPROM), a flash memory, a cassette, a magnetic tape, a magnetic disk storage device or other magnetic storage device, a compact disk read-only memory (CD-ROM), a digital versatile disk read-only memory (DVD-ROM), an optical disk, a circuit configured to store information, or some combination thereof. In one embodiment, without departing from the scope of the present disclosure, the first memory device 204 may be integrated with the first processor 202 on a single chip.

[0046] The first communication interface 206 may correspond to a communication interface that may facilitate transmitting messages and data to and receiving messages and data from various devices operating in the system environment 100 via the network 104. For example, the first communication interface 206 is communicatively coupled to the central server 106 via the network 104. In some examples, the tag scanner 102 may be configured to transmit tag data to the central server 106 via the first communication interface 206. Examples of the first communication interface 206 may include, but are not limited to, an antenna, an Ethernet port, a USB port, a serial port, or any other port that may be suitable for receiving and sending data. The first communication interface 206 sends and receives data and / or messages according to various communication protocols, such as, but not limited to, I2C, TCP / IP, UDP, and 2G, 3G, 4G, or 5G communication protocols.

[0047] In an exemplary embodiment, the image capture unit 208 may include suitable logic and / or circuitry that enables the image capture unit 208 to capture an image of the field of view of the indicia scanner 102. In some examples, the image may include an image of the indicia. In an exemplary embodiment, the image capture unit 208 may include an image sensor. In some examples, an image sensor is referred to as a solid-state device that is capable of generating an electrical signal corresponding to an optical signal projected on the image sensor. Some examples of image sensors may include a color or monochrome 1D or 2D charge coupled device (CCD), a complementary MOSFET (CMOS), a contact image sensor (CIS), or any other device capable of generating an electrical signal based on a received optical signal. In addition to the image sensor, the image capture unit 208 may include one or more optical components, such as one or more lenses, one or more gratings, and / or one or more mirrors that may facilitate directing light from the field of view onto the image sensor. Additionally or alternatively, the image capture unit 208 may also include one or more drive units that are capable of causing the one or more optical components to follow a predetermined path. Such penetration of the one or more optical components may enable the light to be focused onto the image sensor.

[0048] The decoder unit 210 may comprise suitable logic and / or circuitry that may enable the decoder unit 210 to identify markers in an image, such as Figure 3 In addition, the decoder unit 210 may be configured to determine a bar code symbol identifier associated with the indicia, such as in combination with Figure 3 Further described. In an exemplary embodiment, the bar code symbol identifier of the mark may indicate the type of mark. Some examples of the type of mark may include, but are not limited to, Code 39, Code 128, Code 11, PDF417, Data Matrix Code, QR Code, Aztec Code. In addition, in some examples, the decoder unit 210 may be configured to determine one or more marker locations in the image, such as Figure 3 Determining the one or more marker positions may include determining one or more coordinates of the one or more marker positions in the image, such as Figure 3 In some examples, the one or more marker positions may correspond to the corners of the marker. In an exemplary embodiment, the one or more marker positions may surround the marker in the image, such as Figure 7 shown.

[0049] Additionally or alternatively, the decoder unit 210 may be configured to retrieve an image tile from the image such that the image tile includes a portion of the marker, such as Figure 3 Furthermore, the decoder unit 210 may be configured to determine the one or more image tile positions, such as Figure 3In an exemplary embodiment, the decoder unit 210 may be configured to decode the markers in the image to generate decoded data, such as Figure 3 In some examples, the decoder unit 210 may be configured to generate the tag data. In some examples, the tag data includes the decoded data (obtained from the tag in the image), the one or more tag positions (surrounding the tag in the image), the one or more image tile positions (surrounding the portion of the tag), the bar code symbol identifier of the tag, the portion of the tag (i.e., the image tile), and the size of the image (e.g., the resolution of the image).

[0050] In an exemplary embodiment, the decoder unit 210 may be configured to manipulate and / or modify the image using one or more known image processing techniques. Some examples of the one or more image processing techniques may include, but are not limited to, edge detection and object barcode symbol identifier techniques, such as scale-invariant feature transform (SIFT). The decoder unit 210 may be implemented using one or more of an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA).

[0051] Combination Figure 3 The operation of the indicia scanner 102 is further described.

[0052] Figure 3 and Figure 5 According to an exemplary embodiment of the present disclosure, a Figure 1An exemplary flowchart and exemplary method of the operations performed by the device of the tag scanner 102. It should be understood that each box in the flowchart, and the combination of boxes in the flowchart, can be implemented by various devices (such as hardware, firmware, one or more processors, circuits, and / or other devices associated with the execution of software including one or more computer program instructions). For example, one or more of the above processes can be embodied by computer program instructions. In this regard, the computer program instructions embodying the above processes can be stored by the memory of the device using the embodiment of the present invention and executed by the processor in the device. It is understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine so that the resulting computer or other programmable device provides an implementation of the functions specified in one or more flowchart boxes. These computer program instructions can also be stored in a non-transitory computer-readable storage memory, which can instruct a computer or other programmable device to work in a specific manner so that the instructions stored in the computer-readable storage memory produce a product, the execution of which can implement the functions specified in one or more flowchart boxes. The computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented method, so that the instructions executed on the computer or other programmable device provide operations for implementing the functions specified in one or more flowchart blocks. Figure 3 and Figure 6 The operations, when executed, convert a computer or processing circuit into a specific machine configured to perform the exemplary embodiments of the present invention. Figure 3 and Figure 6 The operations of define an algorithm for configuring a computer or processor to perform an exemplary embodiment. In some cases, an instance of a processor may be provided for a general-purpose computer, and the instance performs Figure 3 and Figure 5 The invention provides an algorithm to transform a general-purpose computer into a specific machine configured to perform the exemplary embodiments.

[0053] Therefore, the blocks in the flowchart support a combination of means for performing the specified functions and a combination of operations for performing the specified functions. It will also be understood that one or more blocks in the flowchart and a combination of blocks in the flowchart can be implemented by a hardware-based special-purpose computer system that performs the specified functions or a combination of special-purpose hardware and computer instructions.

[0054] Capture tag data at the tag scanner

[0055] Figure 3 A flow chart 300 is shown of a method for acquiring indicia data at an indicia scanner 102 according to one or more embodiments described herein.

[0056] At step 302, the tag scanner 102 may include a device for receiving input that captures an image of the field of view of the tag scanner 102, such as the first processor 202. In an exemplary embodiment, an operator of the tag scanner 102 may provide input to the tag scanner via a trigger button on the tag scanner 102. For example, the operator may press the trigger button. In an exemplary embodiment, in response to pressing the trigger button, the trigger button of the tag scanner 102 may generate a trigger signal indicative of the input received from the operator. In some examples, the scope of the present disclosure is not limited to the operator pressing the trigger button to provide input to the tag scanner 102. In an exemplary embodiment, once the tag scanner 102 detects an object present in the corresponding field of view, the tag scanner 102 may be automatically triggered. In another example, the tag scanner 102 may receive input from a remote computer, such as a user computing device operated by an operator.

[0057] At step 304, the indicia scanner 102 may include means for capturing an image of the field of view of the indicia scanner 102, such as the first processor 202, the image capture unit 208, etc. In some examples, the first processor 202 may cause the image capture unit 208 to capture an image in response to receiving a trigger signal from a trigger button. As discussed, the image capture unit 208 may include an image sensor and one or more optical components that may direct light from the field of view of the indicia scanner 102 onto the image sensor. Thus, the image sensor may generate an electrical signal representing the image. Additionally or alternatively, the first processor 202 may be configured to render an image based on the electrical signal received from the image sensor.

[0058] At step 306, the marker scanner 102 may include a device for identifying markers in an image, such as a first processor 202, a decoder unit 210, etc. In an exemplary embodiment, the decoder unit 210 may be configured to utilize known techniques to identify markers in an image. Some examples of known techniques may include, but are not limited to, neural networks, ML models, scale-invariant feature transforms (SIFT), etc. In some examples, the scope of the present disclosure is not limited to identifying markers in an image. In an exemplary embodiment, the decoder unit 210 may be configured to identify other content in the image. For example, other content in the image may include address fields, text content, graphic content such as trademarks, etc. For the purposes of the ongoing description herein, the decoder unit 210 may be configured to identify markers in an image. Markers

[0059] In some examples, during bar code symbol recognition of a mark in an image, the decoder unit 210 may be configured to determine the one or more mark positions. The one or more mark positions in the image correspond to the position of the mark in the image. In an exemplary embodiment, the one or more mark positions (in the image) may define the perimeter of the mark. Thus, the one or more mark positions may surround the mark in the image. In an exemplary embodiment, identifying the one or more mark positions may include determining the coordinates of the mark in the image.

[0060] At step 308, the indicia scanner 102 may include a device for decoding the indicia in the image to generate decoded data, such as the first processor 202, the decoder unit 210, etc. In an exemplary embodiment, the decoder unit 210 may be configured to decode the indicia using one or more known decoding algorithms. Before decoding the indicia, the decoder unit 210 may be configured to determine a bar code symbol identifier associated with the indicia. As discussed, the bar code symbol identifier may depict the type of indicia. For example, the decoder unit 210 may be configured to identify a known identifier on the indicia to determine the type of indicia in the image. In an exemplary embodiment, the decoder unit 210 may be configured to identify a known identifier on the indicia using one or more known image processing techniques such as edge detection, object recognition, image binarization, etc.

[0061] For example, if the decoder unit 210 recognizes one or more square patterns on the corners of the mark, the decoder unit 210 may identify the type of mark as a QR code. In another example, if the decoder unit 210 recognizes continuous lines on two orthogonal edges of the mark, the decoder unit 210 may identify the type of mark as a data matrix code. In yet another example, if the decoder unit 210 recognizes multiple parallel lines in the mark, the decoder unit 210 may be configured to identify the type of mark as a 39 code. Thereafter, based on the known identification, the decoder unit 210 may be configured to determine a bar code symbol identifier. Thereafter, the first decoder unit may be configured to decode the mark based on the bar code symbol identifier using one or more known decoding algorithms.

[0062] At step 310, the indicia scanner 102 may include means for determining whether decoding of the indicia is successful, such as the first processor 202, the decoder unit 210, the decoder unit 210, etc. If the decoder unit 210 determines that decoding of the indicia is unsuccessful, the decoder unit 210 may be configured to repeat step 302. However, if the decoder unit 210 determines that decoding of the indicia is successful, the decoder unit 210 may be configured to perform step 312.

[0063] At step 312, the marker scanner 102 may include a device for segmenting the image into a plurality of image tiles of a predetermined size, such as the first processor 202, the decoder unit 210, etc. In an exemplary embodiment, the predetermined size may define a count of pixels included in each of the plurality of image tiles. Thereafter, the first processor 202 may be configured to select one or more image tiles from the plurality of image tiles that include the marker. In an exemplary embodiment, the first processor 202 may be configured to identify one or more image tiles from the plurality of image tiles based on the one or more marker positions in the image. For example, the first processor 202 may be configured to select the one or more image tiles that include the marker. Additionally or alternatively, the first processor 202 may be configured to select the one or more image tiles (from the plurality of image tiles) that are surrounded within the one or more marker positions. Since the marker is surrounded within the one or more marker positions, the one or more image tiles may include the marker. Figure 4 An exemplary segmentation of an image is shown.

[0064] At step 314, the tag scanner 102 may include means for selecting an image tile from the one or more image tiles, such as the first processor 202, the decoder unit 210, etc. In an exemplary embodiment, the image tile includes a portion of the tag. Figure 6 Selection of an image tile is further described. In some examples, selecting an image tile (of the one or more image tiles) includes determining one or more image tile positions in the image that correspond to the position of the image tile in the image. In an exemplary embodiment, the one or more image tile positions (in the image) may define a perimeter of the image tile in the image. Thus, the one or more image tile positions may surround the image tile in the image. In an exemplary embodiment, identifying the one or more image tile positions may include determining coordinates of the one or more image tile positions in the image. In combination Figure 6 Determination of the position of the one or more image tiles in the image is further described.

[0065] At step 316, the indicia scanner 102 may include means for generating indicia data, such as the first processor 202 or the like. In an exemplary embodiment, the indicia data includes decoded data (obtained from the indicia in the image), the one or more indicia positions (surrounding the indicia in the image), the one or more image tile positions (surrounding portions of the indicia), a bar code symbol identifier for the indicia, portions of the indicia (i.e., image tiles), and dimensions of the image (e.g., resolution of the image). Figure 7 Exemplary labeling data is shown in .

[0066] At step 318 , the indicia scanner 102 may include means for transmitting the indicia data to the central server 106 , such as the first processor 202 , a first communication interface, or the like.

[0067] Figure 4 An exemplary image 400 is shown according to one or more embodiments described herein.

[0068] The decoder unit 210 identifies a marker 406 in the image 400, as described in step 306. In addition, the decoder unit 210 identifies one or more marker positions 402a, 402b, 402c, and 402d surrounding the marker 406. In addition, the decoder unit 210 may be configured to segment the exemplary image 400 into a plurality of image tiles 404a, 404b, 404c, ..., and 404n. In an exemplary embodiment, the decoder unit 210 may be configured to segment the image using a sliding window of a predetermined size. For example, the size of the sliding window is 48 pixels by 48 pixels. Therefore, in some examples, each image tile 404a, 404b, 404c, ..., and 404n in the image has a size of 48 pixels by 48 pixels. In addition, the plurality of image tiles 404a, 404b, 404c, ..., and 404n include one or more image tiles 404a, 404b, 404c, ..., 404l, which include a portion of the mark 406. As discussed, the first processor 202 may select an image tile from among the one or more image tiles 404a, 404b, 404c, ..., 404l. For example, the first processor 202 selects the image tile 404b as the image tile.

[0069] In some examples, the scope of the present disclosure is not limited to using a sliding window to segment an image. In an exemplary embodiment, the decoder unit 210 may be configured to segment an image using other techniques known to those skilled in the art.

[0070] Figure 5 A flowchart 314 of a method for retrieving an image tile from an image is shown according to one or more embodiments described herein.

[0071] At step 502, the tag scanner 102 includes means for retrieving the one or more image tiles from the image (based on the segmentation of the image in step 312), such as the first processor 202, the decoder unit 210, etc. As discussed in step 312, each of the one or more image tiles includes the portion of the tag.

[0072] At step 504, the mark scanner 102 includes a device for determining a quality score of each of the one or more image tiles in the image, such as the first processor 202, the decoder unit 210, etc. In an exemplary embodiment, in order to determine the quality score of the one or more image tiles, the decoder unit 210 may be configured to determine a measure of one or more image features associated with each of the one or more image tiles. Some examples of the one or more image features may include, but are not limited to, sharpness, noise, dynamic range, etc. In some examples, the sharpness of the image may correspond to the amount of detail that the image can convey. The sharpness of the image may be affected by aberrations of the one or more optical components (design and manufacturing quality, focal length, aperture, and distance from the center of the image) and the sensor (pixel count and anti-aliasing filter), focusing accuracy, and atmospheric interference (thermal effects and aerosols), etc. In some examples, noise may correspond to random variations in pixel density in the image. For example, visible particles in the image may correspond to noise in the image. In some examples, the dynamic range of the image may correspond to the difference between the minimum pixel value and the maximum pixel value in the image.

[0073] In an exemplary embodiment, the decoder unit 210 may be configured to determine a weighted sum of the measures of the one or more image features associated with each of the one or more image tiles. In some examples, the weighted sum may correspond to a quality score associated with each of the one or more image tiles.

[0074] In some examples, the scope of the present disclosure is not limited to the quality score being a weighted sum of the one or more image features. In an exemplary embodiment, the decoder unit 210 may determine the quality score of each of the one or more image tiles in the image using other mathematical methods. For example, since each of the one or more image tiles includes a marked portion, the decoder unit 210 may determine the quality score of each of the one or more image tiles using ANSI X3.182, ISO 15415, and ISO / IEC 15416 standards.

[0075] At step 506, the marker scanner 102 includes a device for selecting an image tile from the one or more image tiles in the image, such as the first processor 202, the decoder unit 210, etc. In an exemplary embodiment, the decoder unit 210 may be configured to select the image tile with the minimum quality score among the one or more image tiles. In an exemplary embodiment, the decoder unit 210 may be configured to treat the selected image tile in the image as the image tile. In addition, the decoder unit 210 may treat the marked portion in the image tile as the marked portion. Since each of the one or more image tiles has a size of 48×48 (i.e., the size of the first sliding window used to scan the image), the size of the image tile is also 48×48.

[0076] In some examples, the scope of the present disclosure is not limited to selecting the image tile with the smallest quality score among the one or more image tiles. In an exemplary embodiment, the decoder unit 210 may be configured to randomly select an image tile from the one or more image tiles. In yet another embodiment, the scope of the present disclosure is not limited to selecting only one image tile. In an exemplary embodiment, the decoder unit 210 may be configured to select more than one image tile based on the quality score associated with each image tile in the plurality of image tiles. For example, the decoder unit 210 may be configured to select two image tiles with the lowest quality scores among the one or more image tiles.

[0077] At step 508, the mark scanner 102 includes means for determining the one or more image tile locations in the image surrounding the image tile in the image, such as the first processor 202, the decoder unit 210, etc. For example, the decoder unit 210 may be configured to define a bounding box surrounding the selected image tile in the image. Thereafter, the decoder unit 210 may be configured to determine the coordinates of the one or more corners of the bounding box surrounding the selected image tile. In an exemplary embodiment, the decoder unit 210 may be configured to regard the coordinates of the one or more corners of the bounding box as the one or more image tile locations in the image.

[0078] Figure 6An exemplary image 400 is shown according to one or more embodiments described herein. In some examples, the exemplary image 400 includes a marker 406 surrounded by one or more marker locations 402a, 402b, 402c, and 402d. In addition, the decoder unit 210 retrieves an image tile 404b that includes a portion of the marker 406 (depicted by 608) from one or more image tile locations 606a, 606b, 606c, and 606d. In addition, the decoder unit 210 may be configured to decode the marker 402 to generate decoded data (depicted by 610). Additionally or alternatively, the decoder unit 210 may be configured to determine a bar code symbol identifier (depicted by 612) associated with the marker 402. For example, the bar code symbol identifier (depicted by 612) is code 39. In an exemplary embodiment, a bar code symbol identifier associated with the tag 402 (depicted by 612), one or more tag locations 502a, 502b, 502c, and 502d, one or more image tile locations 606a, 606b, 606c, and 606d, a portion of the tag (depicted by 608), and decoded data (depicted by 610) constitute tag data 614. Additionally or alternatively, the tag data 614 may include the dimensions of the image (e.g., the resolution of the image).

[0079] As discussed, the tag scanner 102 can be configured to transmit tag data (eg, tag data 614) to the central server 106. The structure and operation of the central server 106 are described herein.

[0080] Structure of the Central Server 106

[0081] Figure 7 A block diagram of the central server 106 is shown according to one or more embodiments described herein. In an exemplary embodiment, the central server 106 includes a second processor 702, a second memory device 704, a second communication interface 706, an image processing unit 708, and a training data generation unit 710.

[0082] The second processor 702 may be implemented as a device including one or more microprocessors with one or more accompanying digital signal processors, one or more processors without accompanying digital signal processors, one or more coprocessors, one or more multi-core processors, one or more controllers, processing circuits, one or more computers, various other processing elements (including integrated circuits, such as, for example, application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)), or some combination thereof. Therefore, although in Figure 7106. Although shown as a single processor in the figure, in an embodiment, the second processor 702 may include multiple processors and signal processing modules. The multiple processors may be implemented on a single electronic device, or may be distributed on multiple electronic devices that are collectively configured to function as the circuit of the central server 106. The multiple processors may be in operative communication with each other and may be collectively configured to perform one or more functions of the circuit of the central server 106, as described herein. In an exemplary embodiment, the second processor 702 may be configured to execute instructions stored in the second memory device 704 or otherwise accessible to the second processor 702. When these instructions are executed by the second processor 702, the circuit of the central server 106 may be caused to perform one or more functions, as described herein.

[0083] Whether the second processor 702 is configured by a hardware method, a firmware / software method, or a combination thereof, the second processor may include an entity capable of performing operations according to the embodiments of the present disclosure while being configured accordingly. Thus, for example, when the second processor 702 is implemented as an ASIC, FPGA, etc., the second processor 702 may include specially configured hardware for performing one or more operations described herein. Alternatively, as another example, when the second processor 702 is implemented as an executor of instructions (such as those that may be stored in the second memory device 704), the instructions may specifically configure the second processor 702 to perform one or more algorithms and operations described herein.

[0084] Therefore, the second processor 702 used herein can refer to a programmable microprocessor, a microcomputer, or one or more multiprocessor chips, which can be configured by software instructions (applications) to perform various functions including the functions of the various embodiments described above. In some devices, multiple processors dedicated to wireless communication functions and a processor dedicated to running other applications can be provided. Software applications can be stored in internal memory before being accessed and loaded into the processor. The processor may include an internal memory sufficient to store application software instructions. In many devices, the internal memory can be a volatile or non-volatile memory such as a flash memory or a mixture of the two. The memory can also be located inside another computing resource (for example, to enable computer-readable instructions to be downloaded via the Internet or another wired or wireless connection).

[0085] The second memory device 704 may include suitable logic, circuitry, and / or interfaces adapted to store a set of instructions executable by the second processor 702 to perform predetermined operations. Some of the commonly known memory implementations include, but are not limited to, hard disks, random access memories, cache memories, read-only memories (ROMs), erasable programmable read-only memories (EPROMs), and electrically erasable programmable read-only memories (EEPROMs), flash memory, cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, compact disk read-only memories (CD-ROMs), digital versatile disk read-only memories (DVD-ROMs), optical disks, circuits configured to store information, or some combination thereof. In one embodiment, without departing from the scope of the present disclosure, the second memory device 704 may be integrated with the second processor 702 on a single chip.

[0086] The second communication interface 706 may correspond to a communication interface that may facilitate transmitting messages and data to and receiving messages and data from various devices operating in the system environment 70 via the network 74. For example, the second communication interface 706 is communicatively coupled to the tag scanner 102 via the network 74. In some examples, the central server 106 may be configured to receive tag data from the tag scanner 102 via the second communication interface 706. Examples of the second communication interface 706 may include, but are not limited to, an antenna, an Ethernet port, a USB port, a serial port, or any other port that may be suitable for receiving and sending data. The second communication interface 706 sends and receives data and / or messages according to various communication protocols, such as, but not limited to, I2C, TCP / IP, UDP, and 2G, 3G, 4G, or 5G communication protocols.

[0087] The image processing unit 708 may comprise suitable logic and / or circuitry that may enable the image processing unit 708 to process one or more images, such as Figure 8 , Fig. 9 and Fig.10 . In an exemplary embodiment, the image processing unit 708 may be configured to process one or more images using one or more known image processing techniques. Some examples of the one or more image processing techniques may include, but are not limited to, edge detection, scale-invariant feature transform (SIFT), etc. The image processing unit 708 may be implemented using one or more of an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA).

[0088] The training data generation unit 710 may comprise suitable logic and / or circuitry that may enable the training data generation unit 710 to generate training data. In an exemplary embodiment, the training data generation unit 710 may generate paired data including an ideal labeled portion and a labeled portion, such as Figure 8In an exemplary embodiment, the training data generation unit 710 may treat the paired data as training data. The image processing unit 708 may be implemented using one or more of an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA).

[0089] Figure 8 , Fig. 9 and Fig.10 According to an exemplary embodiment of the present disclosure, a Figure 1 The exemplary flowchart and exemplary method of the operation performed by the device of the central server 106 of the present invention. It should be understood that each box in the flowchart, and the combination of boxes in the flowchart can be implemented by various devices (such as hardware, firmware, one or more processors, circuits, and / or other devices associated with the execution of software including one or more computer program instructions). For example, one or more of the above processes can be embodied by computer program instructions. In this regard, the computer program instructions embodying the above processes can be stored by the memory of the device using the embodiment of the present invention and executed by the processor in the device. It is understood that any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to produce a machine so that the resulting computer or other programmable device provides an implementation of the functions specified in one or more flowchart boxes. These computer program instructions can also be stored in a non-transitory computer-readable storage memory, which can instruct a computer or other programmable device to work in a specific manner so that the instructions stored in the computer-readable storage memory produce a product, and its execution can realize the functions specified in one or more flowchart boxes. The computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device, thereby producing a computer-implemented method, so that the instructions executed on the computer or other programmable device provide operations for implementing the functions specified in one or more flowchart blocks. Figure 8 , Fig. 9 and Fig.10 The operations, when executed, convert a computer or processing circuit into a specific machine configured to perform the exemplary embodiments of the present invention. Figure 8 , Fig. 9 and Fig.10 The operations of define an algorithm for configuring a computer or processor to perform an exemplary embodiment. In some cases, an instance of a processor may be provided for a general purpose computer that performs Figure 8 , Fig. 9 and Fig.10 The invention provides an algorithm to transform a general-purpose computer into a specific machine configured to perform the exemplary embodiments.

[0090] Therefore, the blocks in the flowchart support a combination of means for performing the specified functions and a combination of operations for performing the specified functions. It will also be understood that one or more blocks in the flowchart and a combination of blocks in the flowchart can be implemented by a hardware-based special-purpose computer system that performs the specified functions or a combination of special-purpose hardware and computer instructions.

[0091] Clean / damaged tile pair generation

[0092] Figure 8 A flowchart 800 is shown of a method for operating a central server 106 according to one or more embodiments described herein.

[0093] At step 802, the central server 106 may include means for receiving tag data from the tag scanner 102, such as the second processor 702, the second communication interface 706, etc. As discussed above, the tag data includes decoded data, the one or more tag locations surrounding the tag (scanned by the tag scanner 102) in the image (captured by the tag scanner 102), the one or more image tile locations surrounding the portion of the tag in the image, the image tile (including the portion of the tag), the size of the image (e.g., the resolution of the image), and the bar code symbol identifier of the tag.

[0094] At step 804, the central server 106 may include means for generating an ideal tag based on the decoded data and a bar code symbol identifier associated with the tag, such as the second processor 702, the image processing unit 708, etc. As discussed, the bar code symbol identifier associated with the tag may indicate the type of the tag (e.g., the type of the tag may include a bar code). In some examples, the image processing unit 708 may be configured to encode the decoded data (obtained in the tag data) to generate an ideal tag, wherein the bar code symbol identifier of the ideal tag is the same as the bar code symbol identifier of the scanned tag (scanned by the tag scanner 102). For example, if the bar code symbol identifier of the tag indicates that the tag is a code 39, then the ideal tag (generated by the image processing unit 708) will also be a code 39. In an exemplary embodiment, the image processing unit 708 may generate the ideal tag using one or more known methods, such as, but not limited to, a ZINT bar code generator and / or any other application that outputs a graphical image of an ideal tag. Since the image processing unit 708 uses the decoded data to generate the ideal tag, the ideal tag is the same as the scanned tag. The ideal tag is the ideal form of the tag. Since the image processing unit 708 generates an ideal mark, the ideal mark should be noise-free. On the other hand, the mark in the image (captured by the mark scanner 102) may be noisy due to environmental factors and / or tolerances associated with the image capture unit 208. Some examples of environmental factors and / or tolerances of the image capture unit 208 may include, but are not limited to, lens aberrations, dynamic range of the image sensor, sensitivity of the image sensor, lighting conditions of the environment in which the mark scanner 102 operates, etc. Additionally or alternatively, the mark in the image may be noisy due to printing errors and / or wear and tear on the label during shipping. One of ordinary skill in the art will appreciate that generating an ideal mark includes generating an image of the ideal mark.

[0095] At step 806, the central server 106 may include means for determining the one or more ideal marker positions in the image of the ideal marker such that the one or more ideal marker positions may surround the ideal marker in the image of the ideal marker, such as the second processor 702, the image processing unit 708, etc. For example, the one or more ideal marker positions may define the perimeter of the ideal marker in the image of the ideal marker. In an exemplary embodiment, the image processing unit 708 may determine the one or more ideal marker positions using the method described in step 306.

[0096] At step 808, the central server 106 may include means, such as the second processor 702, the image processing unit 708, etc., for modifying the image of the ideal marker based on the one or more scanned marker locations, the one or more ideal marker locations, and the size of the image. Fig. 9Modification of the image of the ideal marker is further described. Modifying the image of the ideal marker results in generating a modified image of the ideal marker. In an exemplary embodiment, the size of the modified image of the ideal marker is the same as the size of the image. In addition, the positioning and orientation of the ideal marker in the modified image of the ideal marker is the same as the positioning and orientation of the marker in the image.

[0097] At step 810, the central server 106 may include a device for retrieving clean image tiles from the modified image of the ideal tag, such as the second processor 702, the image processing unit 708, etc. In some examples, the image processing unit 708 may be configured to retrieve clean image tiles from the modified image of the ideal tag using the one or more image tile positions received in the tag data. As discussed above in flowchart 300, the one or more image tile positions may surround the portion of the tag in the scanned image. In addition, as discussed in step 808, the positioning and orientation of the ideal tag in the modified image of the ideal tag are the same as the positioning and orientation of the tag in the image. In addition, as discussed in step 808, the image and the modified image of the ideal tag have the same size. Therefore, in the modified image of the ideal tag, the one or more image tile positions may surround the portion of the ideal tag. More specifically, the portion of the ideal tag surrounded by the one or more image tile positions in the modified image of the ideal tag is the same as the portion of the tag (surrounded by the one or more image tile positions in the image). In some embodiments, this allows the second processor 702 to avoid creating tiles for the entire ideal image and only create tiles for corresponding locations in the ideal image. Since the portion of the ideal tag is retrieved from the ideal tag, the portion of the ideal tag can be cleaner and less noisy than the portion of the scanned tag. Since the tag (scanned by the tag scanner 102) and the ideal tag (generated by the image processing unit 708) are the same, the image processing unit 708 can treat the identified tiles of the ideal tag as clean portions of the tag.

[0098] At step 812, the central server 106 may include means for removing the non-marked image portion from the image tile, such as the second processor 702, the image processing unit 708, etc. In an exemplary embodiment, the non-marked image portion in the image tile may include auxiliary content. In some examples, the auxiliary content may correspond to content other than the marked portion. For example, the auxiliary content may include, but is not limited to, numbers, text characters, and / or any other symbols that do not correspond to the mark. The removal of the non-marked image portion from the scanned image tile is Figure 8 The image patch obtained after removing the non-marked image part is called the damaged image patch.

[0099] At step 814, the central server 106 may include a device for generating training data, such as the second processor 702, the training data generation unit 710, etc. In an exemplary embodiment, the training data generation unit 710 may generate paired data, the paired data including the marked damaged part and the marked clean part. In addition, the training data generation unit 710 may regard the paired data as training data.

[0100] Fig. 9 A flowchart 808 of a method for rendering a modified image of an ideal marker is shown according to one or more embodiments described herein.

[0101] At step 902, the central server 106 may include means for determining a homography matrix between the scanned image and the image of the ideal tag based on the one or more tag positions in the scanned image, the one or more tag positions in the ideal tag, and the size of the scanned image, such as the second processor 702, the image processing unit 708, etc. In an exemplary embodiment, the homography matrix may correspond to a transformation matrix that maps the one or more tag positions in the scanned image to the one or more tag positions in the image of the ideal tag. In an exemplary embodiment, the image processing unit 708 may be configured to determine the homography matrix using the following equation:

[0102]

[0103] in

[0104] Homography matrix;

[0105] one or more marker locations on the ideally marked image; and

[0106] Scan one or more marked locations on the image.

[0107] At step 904, the central server 106 may include a device for modifying the image of the ideal marker, such as the second processor 702, the image processing unit 708, etc. In an exemplary embodiment, the image processing unit 708 may be configured to modify the image of the ideal marker using a homography matrix to generate a modified image of the ideal marker. For example, the image processing unit 708 may be configured to transform one or more ideal marker positions (of the ideal marker in the image surrounding the ideal marker) into one or more clean image tile positions using a homography matrix. In some examples, the image processing unit 708 may be configured to determine the one or more clean image tile positions using the following formula:

[0108] X 2 =HX 1(2)

[0109] in,

[0110] X 2 : one or more clean image tile locations;

[0111] H: homography matrix; and

[0112] X 1 : One or more ideal marker locations in an ideally labeled image.

[0113] In an exemplary embodiment, the ideal marker needs to be positioned within the one or more clean image tile locations. In some embodiments, the process of positioning the ideal marker within the one or more clean image tile locations is called a perspective transform. Additionally, in some examples, the image processing unit 708 may be configured to fill pixels so that the size of the image of the ideal marker is the same as the size of the image.

[0114] Fig.10 A flowchart 812 is shown of a method for removing non-marked image portions (containing auxiliary information) from marked portions according to one or more embodiments described herein.

[0115] At step 1002, the central server 106 may include a device for generating a mask image, such as a second processor 702, an image processing unit 708, etc. In some examples, the image processing unit 708 may be configured to retrieve a clean image tile from the modified image, as described in step 810. Thereafter, in an exemplary embodiment, the image processing unit 708 may be configured to determine the intersection between the clean image tile and the modified image of the ideal mark to generate the mask image. In an exemplary embodiment, the intersection between the two images identifies the common parts in the two images. For example, the intersection between the clean image tile and the modified image of the ideal mark may include the part of the ideal mark included in the clean image tile. Therefore, the mask image includes the common part of the ideal mark in the clean image tile and the modified image of the ideal mark. In addition, in the mask image, the part of the ideal mark is blackened. For example, the color associated with the pixel representing the part of the ideal mark in the mask image is converted to black.

[0116] At step 1004, the central server 106 may include means for generating an inverted image of the mask, such as the second processor 702, the image processing unit 708, etc. In an exemplary embodiment, the image processing unit 708 may be configured to invert the color scheme in the mask image to generate an inverted mask image. For example, portions of the mask image including black pixels are converted to white pixels, and vice versa.

[0117] At step 1006, the central server 106 may include means for removing non-marked image portions (including auxiliary content) from the image tiles, such as the second processor 702, the image processing unit 708, etc. In an exemplary embodiment, the image processing unit 708 may be configured to perform an AND operation between the inverted mask image and the image tiles. The inverted mask image highlights the portion of the image that does not contain the ideal mark by performing an AND operation between the inverted mask image and the image tiles to remove the auxiliary content included in the image tiles. Therefore, the modified image tiles do not contain auxiliary content. In addition, the modified image tiles are referred to as marked damaged segments.

[0118] In some examples, the scope of the present disclosure is not limited to using a mask image to remove non-marked image portions from an image tile. In an exemplary embodiment, the image processing unit 708 may be configured to use one or more edge detection techniques to distinguish the portion of the scanned mark from the auxiliary content. Therefore, the image processing unit 708 may remove the auxiliary content from the portion of the scanned mark. In another example, the image processing unit 708 may use one or more known object recognition techniques (such as SIFT) to distinguish the auxiliary content from the portion of the mark.

[0119] Fig.11 is an exemplary flow chart 1100 illustrating operations performed at the central server 106 according to one or more embodiments described herein.

[0120] Exemplary flow chart 1100 shows labeling data 614. Figure 6 As discussed, the tag data includes a barcode symbol identifier (depicted by 612) associated with the tag 402, one or more tag positions 402a, 402b, 402c, and 402d, one or more image tile positions 606a, 606b, 606c, and 606d, an image tile 608, and decoded data (depicted by 610). In some examples, the image processing unit 708 generates an image of the ideal tag 1102 including the ideal tag 1104 by using the decoded data and the barcode symbol identifier (depicted by 612) associated with the scanned tag. Thereafter, the image processing unit 708 determines one or more ideal tag positions 1106a, 1106b, 1106c, and 1106d of the ideal tag in the image of the ideal tag. The one or more ideal tag positions 1106a, 1106b, 1106c, and 1106d may correspond to the positions of one or more corners of the ideal tag 1104 in the image of the ideal tag 1102.

[0121] In an exemplary embodiment, the image processing unit 708 may be configured to modify the image of the ideal marker 1102 based on the one or more marker positions 402a, 402b, 402c, and 402d (received in the marker data 614) and the one or more ideal marker positions 1106a, 1106b, 1106c, and 1106d, as described in flowchart 812. In some examples, the ideal marker 1104 in the modified image of the ideal marker 1108 has the same perspective as the marker in the image (captured by the marker scanner 102). Thereafter, the image processing unit 708 may be configured to retrieve a clean image tile including a portion 1110 of the ideal marker 1104 from the modified image of the ideal marker 1108 using the one or more image tile positions 606a, 606b, 606c, and 606d (received in the marker data 614), as described in step 810. In addition, the image processing unit 708 may be configured to generate a mask image 1112 and an inverted mask image 1114 by determining the intersection between the clean image tile and the modified image of the ideal mark 1108, as described in step 812. As depicted in the mask image 1112, a portion 1116 is blacked out. The portion 1116 corresponds to a common portion in the ideal mark 1104 and a portion 1110 of the ideal mark 1104. In some examples, the common portion may correspond to a portion of the ideal mark 1104. The inverted mask image 1114 includes a color scheme opposite to that of the mask image 1112. Thereafter, the inverted mask image is ANDed with the image tile 608 to remove the auxiliary content (i.e., the non-marked portion) in the image tile 608. The resulting image 1118 obtained after the AND operation is referred to as the damaged portion of the mark 406. In addition, the portion 1110 of the ideal mark 1104 is referred to as the clean portion of the mark. The paired data of the clean part of the label 1110 and the damaged part of the label 1118 constitute the training data 1120.

[0122] In this specification and the accompanying drawings, typical embodiments of the present disclosure have been disclosed. The present disclosure is not limited to such exemplary embodiments. The use of the term "and / or" includes any and all combinations of one or more associated listed items. The accompanying drawings are schematic representations and are therefore not necessarily drawn to scale. Unless otherwise indicated, specific terms are used in a general and descriptive sense and not for the purpose of limitation.

[0123] The above detailed description has been used to explain various embodiments of devices and / or processes by using block diagrams, flow charts, schematic diagrams, examples and examples. As long as such block diagrams, flow charts, schematic diagrams and examples contain one or more functions and / or operations, each function and / or operation in such block diagrams, flow charts, schematic diagrams or examples can be implemented separately and / or together by various hardware.

[0124] In one embodiment, examples of the present disclosure may be implemented via an application specific integrated circuit (ASIC). However, the embodiments disclosed herein may be equivalently implemented in whole or in part in a standard integrated circuit as one or more computer programs running on one or more computers (e.g., one or more programs running on one or more computer systems), one or more programs running on one or more processing circuits (e.g., microprocessing circuits), one or more programs running on one or more processors (e.g., microprocessors), firmware, or nearly any combination thereof.

[0125] In addition, those skilled in the art will recognize that the exemplary mechanisms disclosed herein may be capable of being distributed as program products in a variety of tangible forms, and that the exemplary embodiments apply equally regardless of the particular type of tangible instruction bearing medium used to actually perform the distribution. Examples of tangible instruction bearing media include, but are not limited to, the following: recordable types of media such as floppy disks, hard drives, CD ROMs, digital tapes, flash drives, and computer memory.

[0126] The various embodiments described above can be combined with each other to provide additional embodiments. For example, two or more of the exemplary embodiments described above can be combined to, for example, improve the safety of laser printing and reduce the risks associated with laser-related accidents and injuries. These and other changes can be made to the inventive system and method in accordance with the above detailed description. Therefore, the present disclosure is not limited by the disclosure, but its scope will be determined by the following claims.

Claims

1. A method for generating data for training a machine learning (ML) model, comprising: receiving, by a processor, marker data from a marker scanner, wherein the marker data includes at least decoded data obtained based on decoding a marker in an image, an image tile including a portion of the marker, and / or a location in the image of one or more corners of the portion of the marker; generating, by the processor, an image of an ideal marker based at least on the decoded data; modifying, by the processor, the image of the ideal marker to generate a modified image of the ideal marker, wherein an orientation of the ideal marker in the modified image is the same as an orientation of the marker in the image; retrieving, by the processor, a clean image patch from the modified image of the ideal marker based on a location of the one or more corners of the portion of the marker in the image, wherein the clean image patch includes a portion of the ideal marker; as well as Training data is generated by the processor, wherein the training data includes the labeled portion and the ideal labeled portion.

2. The method according to claim 1, further comprising: receiving a location of one or more corners of the indicia in the image and a bar code symbol identifier associated with the indicia, and An image of the ideal indicia is generated based on the bar code symbol identifier.

3. The method according to claim 2, further comprising: The processor determines the position of one or more corners of the ideal marker in the image of the ideal marker.

4. The method according to claim 3 also includes determining, by the processor, a homography matrix based on the position of the one or more corners of the ideal marker in the image of the ideal marker and the position of the one or more corners of the marker in the image.

5. The method of claim 4 further comprises modifying, by the processor, the image of the ideal marker based on the location of the one or more corners of the ideal marker in the image of the ideal marker and the homography matrix. 6 . The method of claim 1 , further comprising generating, by the processor, a mask image and an inverted mask image based on an intersection between the ideal labeled modified image and the clean image patch. 7 . The method of claim 6 , further comprising removing, by the processor, auxiliary content from the marked portion based on the mask image and the inverted mask image to generate a modified image tile, wherein the modified image tile constitutes the training data.

8. The method of claim 6, wherein at least a portion of the mask image is blacked out, the portion of the mask image corresponding to a common portion of the ideal marking.

9. The method of claim 1, wherein the tag data further includes a size of the image.

10. The method of claim 1, further comprising automatically triggering the mark scanner to capture an image.

11. The method according to claim 1, further comprising: In response to user input via a trigger button, the indicia scanner is triggered to capture an image.

12. The method according to claim 1, further comprising: In response to user input via the remote computer, the indicia scanner is triggered to capture an image.

13. A server, comprising a memory device and a processor, the memory device comprising a set of instructions, the processor executing the set of instructions to cause the server to perform any one of the methods according to claims 1 to 12.

14. A computer program product, comprising at least one non-transitory computer-readable medium having computer-readable program instructions stored therein, the computer-readable program instructions comprising instructions, which are configured to cause the device to perform at least any one of the methods according to claims 1 to 12 when executed by the device.

Images