Edge-based electronic signature determination for genuine product marks using parallel processing

Abstract

A system including an image sensor, a multi-core processing device, and a non-transitory computer-readable medium tangibly encoding instructions of a computer program configured to cause the multi-core processing device to perform operations including parallel processing, by the multi-core processing device, of the image data received from the image sensor, the image data representing a mark printed on a product, where the mark provides product information. The parallel processing includes measuring, by the multi-core processing device, differential characteristics of the mark at multiple locations to generate a set of metrics, and generating, by the multi-core processing device, an electronic signature for the mark based on the set of metrics. The operations include sending, by the multi-core processing device, the electronic signature directly to a remote data store that is communicatively coupled with the multi-core processing device via an inter-network.

Description

[0001] EDGE-BASED ELECTRONIC SIGNATURE DETERMINATION FOR GENUINE PRODUCT MARKS USING PARALLEL PROCESSING

[0002] BACKGROUND

[0003] Counterfeit products are, unfortunately, widely available and often hard to spot. When counterfeiters produce fake goods, they typically copy the labeling and barcodes in addition to the actual products. At a superficial level, the labels and barcodes may appear genuine and even yield valid data when scanned (e.g., decode to the appropriate code), for example, Universal Product Code (UPC). While there are many technologies currently available to counter such copying, most of these solutions involve the insertion of various types of codes, patterns, microfibers, microdots, and other indicia to help thwart counterfeiting. Such techniques require manufacturers to use additional equipment and material and add a layer of complexity to the production process.

[0004] Optical systems sometimes are utilized to obtain images of moving material, for example for quality inspection and defect detection during production of the material and / or to extract useful information from images of the printed materials. However, consumer goods packaging operations can be high volume and can run at rates well above 2000 parts per minute (ppm), which can overwhelm current serialization and brand-protection solutions which run at around 1000 ppm and which can require extensive hardware including multiple cameras.

[0005] SUMMARY

[0006] This specification describes technologies for an integrated serialization, brand protection, and vision system. These technologies generally involve an imaging system using a multi-core, multi-thread parallel processing of image data collected by an image sensor of a printed mark on a product item in a production environment. The imaging system can perform serialization, vision, and brand protection at high speeds (e.g., >2000 ppm, up to 3000 ppm) using an edge device having sufficient compute power and smaller footprint. The imaging system can capture high-resolution images of printed marks, processing the microvariations present in the printed marks, and encoding these variations into a unique electronic signature. The imaging system then provides the electronic signature for storage in a cloudbased data store for future reference in authentication processes. The system achieves these operations at high speeds, making it an efficient solution for applications where rapid processing of printed marks is required, such as in manufacturing or logistics.

[0007] In general, one innovative aspect of the subject matter described in this specification can be embodied in a system including an image sensor, a multi -core processing device, and a non-transitory computer-readable medium tangibly encoding instructions of a computer program and configured to cause the multi-core processing device to perform operations. The operations include parallel processing of image data received from the image sensor. The image data represents a mark printed on a product, where the mark provides product information. The parallel processing also includes measuring differential characteristics of the mark at multiple locations to generate a set of metrics. The operations include determining an electronic signature for the mark based on the set of metrics, and sending the electronic signature directly to a remote data store that is communicatively coupled with the multi-core processing device via an inter-network. For example, multiple GPU processors can simultaneously calculate metrics in different regions of the same mark and multiple CPU processors can simultaneously calculate metrics from multiple marks in different images. The performance of each approach is measured, and the system can either be statically or dynamically configured to maximize throughput.

[0008] Other embodiments of this aspect include corresponding methods, apparatus, and computer programs recorded on one or more computer storage devices.

[0009] The subject matter described in this specification can be implemented in particular embodiments so as to realize one or more of the following advantages. The imaging system can capture and analyze vital production data for insights that enhance decision-making and process optimization, provide real-time monitoring to inform decisions, and champions sustainability goals through small footprint, and minimizing waste through accurate inspections supported by AI / ML and data insights. As described here, the system is a fully- integrated technology to capture high-resolution images of printed marks, process the microvariations within the printed marks, and encode these variations into an electronic signature at high speeds (e.g., >2000 ppm) in a self-contained solution. This technology provides a significant improvement over existing solutions by offering enhanced speed, accuracy, and efficiency in symbology processing and encoding through the use of multi -core or multithreading techniques in a GPU processor, allowing for rapid parallel processing of the image data, electronic encoding, and direct transmission for storing of the electronic signature in a cloud-based storage system.

[0010] The system's use of multi-core processing allows for scalability. As the production line speed or complexity increases, additional cores and / or cameras can be utilized to maintain the system's high-speed performance. This makes the system adaptable to a wide range of production environments, from small-scale operations to large factories. However, adding cores and / or cameras can require different hardware which can increase an amount of physical space needed to accommodate the additional components.

[0011] The system can integrate with existing packaging workflows and removes the need for multiple layers of technology by allowing direct connection from the packaging line to the cloud. A compact hardware footprint can reduce emissions, aligning with sustainability goals and resulting in a low total cost of ownership. For example, by using a standalone system including on-board image capture and image processing can improve an image transfer speed by about 5X, e.g., a transfer speed up about 5 Gbit / second.

[0012] The system can provide flexible configuration of packaging workflows and use cases through a fully mobile-ready web-based graphical user interface. The relatively small footprint and hardware architecture of the edge-based solution can be compatible with other embedded vision systems and can be integrated directly into many packaging equipment solutions while requiring minimal or no external line control system. For example, the edgebased solution can incorporate an electronic signature vision processing with rules-based vision processing, high-speed imaging solutions, and edge learning vision processing, which can enable the edge-based solution to support other use cases in packaging lines. Further details of electronic signature vision processing and high-speed imaging solutions can be found in U.S. Patent US9519942B2, fded on February 17, 2015 and entitled “Methods And A Computing Device For Determining Whether A Mark Is Genuine,” U.S. Patent US8190279B2, filed on November 12, 2009 and entitled “Production Line Management System,” and U.S. Patent US11811983B2, filed on June 16, 2021 and entitled “Methods And System For Imaging Of Moving Printed Materials By An Optical Device Having Plurality Of Cameras Arranged In An Array,” which are each incorporated herein by reference in their entireties. The edge-based solution can enable vision-based inspections which require machine learning capabilities with on board GPU edge learning, e.g., optical character recognition (OCR) or object detection, by reducing complexities with training models.

[0013] By capturing high-resolution images and analyzing micro-variances on product marks, the system can aid in quality control by detecting printing defects or inconsistencies that may indicate issues in the manufacturing process. This can lead to early detection of potential problems, reducing wastage and improving overall product quality. This can use AI / ML capabilities of the system by capturing images of a production run, allowing the user to annotate features in the images indicating manufacturing issues, and training an Al model on the images. A convoluted neural network (CNN) model can be used for defect detection; other and additional AI / ML implementations are possible. When running, the model will detect those issues and alert the user or line controller of the issue.

[0014] The user interface can provide a simple and intuitive platform for users to manage and monitor the system's operation. It can display real-time data from the smart camera and processor and that allows users to control the system's settings and operations.

[0015] By facilitating data minimization, the system can contribute significantly to data security and privacy. In today's digital age where data breaches are a constant threat, the system's ability to send only electronic signatures to the cloud, rather than actual high- resolution images of barcodes, can provide a significant security enhancement. This approach reduces the volume of sensitive data in transit, thereby minimizing the potential impact of a data breach. Unlike systems that might transmit actual images of barcodes or product marks to the cloud, this system can send electronic signatures, significantly enhancing the security for authentication. This approach reduces the risk of a data breach because these electronic signatures, while crucial for authentication and product verification, do not contain any explicit or easily decipherable product information. For example, even if counterfeiters managed to breach the electronic signature transfer, the intercepted data would be practically useless to them. The electronic signatures can only be decoded and understood by using the system described herein, creating an additional layer of security. In some implementations, this additional layer of security is achieved through an algorithm that ensures the electronic signatures are uniquely tied to the specific micro-variations of each printed mark. The system’s ability to capture and encode these minute differences into a secure, non-reversible electronic signature makes it virtually impossible for counterfeiters to replicate. Even if the electronic signatures are intercepted, they cannot be reverse-engineered to reveal the original printed mark or the product information.

[0016] The approach described herein essentially creates a digital enigma, a code that is practically only readable with the tools that are part of this system. The combination of simultaneously achieving high-speed processing and robust security has traditionally been mutually exclusive in the realm of image-based product authentication systems. There is often a trade-off where systems can either process images rapidly or secure the data effectively, but not both. The disclosed system can uniquely address this challenge by employing multi-core, multi-thread parallel processing techniques to generate electronic signatures from high-resolution images of printed marks at speeds exceeding 2000 parts per minute. The generated electronic signatures, which encapsulate micro-variations and unique artifacts of the printed marks, are inherently secure even if intercepted. This dual capability of maintaining high throughput on production lines while ensuring data security represents a significant advancement.

[0017] Further, this approach reduces the volume of data being transferred, significantly lowering the bandwidth requirements and associated costs of operating the solution. Moreover, it eases the burden on cloud storage systems as storing electronic signatures requires significantly less space and resources than storing high-resolution images. Additionally, this minimized transmission and storage of potentially sensitive data can help manufacturers comply with increasingly stringent data protection regulations around the world.

[0018] Data minimization and unique encoding approach not only protects the manufacturers from potential losses due to counterfeiting but also safeguards the end consumers from being duped into purchasing counterfeit products. Moreover, it provides an additional layer of trust and security in the supply chain, contributing to a safer and more reliable marketplace.

[0019] Finally, the system not only keeps pace with high-volume and high-speed production environments, but it also adds a non-intrusive layer of security and verification. It effectively merges speed with security, an essential combination in the fast-paced, interconnected, and security-conscious world we live in today. Thus, the system not only enhances the efficiency and accuracy of product authentication but also significantly contributes to a safer and more secure digital sphere in the realm of product manufacturing and logistics. The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0020] BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG. l is a block diagram of an example of an operating environment in which an imaging system can perform edge-based electronic signature determination for genuine product marks using parallel processing.

[0022] FIG. 2 is a block diagram of an example of a production line.

[0023] FIG. 3 is a block diagram of an example of a workflow for edge-based electronic signature determination for genuine product marks using parallel processing.

[0024] FIG. 4 is a block diagram of an example of an edge-based system architecture for implementing the edge-based electronic signature determination.

[0025] FIG. 5 is an example of an embedded application architecture for real-time visualization and communication services for external data integration.

[0026] FIG. 6 is a flow diagram of an example of a process of the imaging system.

[0027] FIG. 7 is a flow diagram of an example of a process of the imaging system.

[0028] FIG. 8 is a block diagram of an example of an embedded item process stream.

[0029] Like reference numbers and designations in the various drawings indicate like elements.

[0030] DETAILED DESCRIPTION

[0031] In general, the imaging system is operable to capture high-resolution images of symbologies, process the captured images to identify micro-variations, determine an electronic signature based on these micro-variations, and store the determined electronic signature in a cloud-based storage system. The imaging system achieves these operations at high speeds (e.g., >2000 parts per minute, up to 3000 parts per minute or greater), making it an efficient solution for applications where rapid processing of symbologies is required. The system's operation can be synchronized with the movement of the product transport (e.g., conveyor belt) on which the symbologies (e.g., printed marks) are printed on product items moving on the product transport. The image sensor captures each symbology as it comes into a field of view of the image sensor, and the processor processes the captured symbology before the next one appears before the camera.

[0032] FIG. 1 is a block diagram of an example of an operating environment 100 in which an imaging system 102 can perform edge-based electronic signature determination for genuine product marks using parallel processing. Imaging system 102 is operable to acquire and process image data collected of printed marks on product items 104 in a production environment 106. A production environment can include, for example, a product manufacturing, packaging, and / or distribution facility. The production environment can include a production apparatus 108, (e.g., a printer), and a product transport 110, (e.g., a conveyer belt or another type of movement apparatus), for transporting the product items in the production environment 106.

[0033] Printed marks can be found, for example, on the product item itself, product packaging, or a product label. Although referred to herein as a “printed” mark, the term should not be read as limiting to traditionally printed (e.g., inkjet or LaserJet) marks. Printed can refer to, for example, a scribed mark (e.g., laser scribed), an indentation-type mark (e.g., stamping, pressing), or another form of establishing a mark on a product item.

[0034] The printed mark can include, for example, linear barcodes, matrix barcodes such as QR (Quick Response) codes, and / or other data-encoding visual markers such as company logos. Printed marks are also referred to herein as symbologies. In other words, a printed mark can include something that identifies a brand (e.g., a logo) or something that bears information. For example a barcode (e.g., a one-dimensional (" 1 D") barcode, such as such as European Article Number (EAN), Universal Product Code (UPC), Interleaved Two of Five (ITF) code, etc., a two-dimensional ("2D") data matrix barcode as specified in the International Organization for Standardization ("ISO") and the International Electrotechnical Commission ("IEC") standard ISO / IEC 16022, a Quick Response (QR) code, etc.), an expiration date, or tracking information such as a serial number), or a decoration. In some implementations, the printed mark can include another suitable region of interest on the product item.

[0035] The system 102 includes an image sensor 112 and an imaging controller device 114. Although depicted in FIG. 1 as separate from the image sensor 112, the imaging controller device 114 can be integrated with the image sensor 112. Additionally, imaging system 102 can be a fully integrated, self-contained apparatus, where the multiple components described here with reference to the imaging system 102 can be included in a standalone unit. For example, the imaging system 102 can be a self-contained apparatus deploy able within the production environment.

[0036] The image sensor can be, for example, a high-speed image sensor configured to capture image data, for example, to capture a two-dimensional (2D) pixel matrix in a single exposure cycle. The exposure cycle includes a duration during which the image sensor is actively engaged in the collection of light to construct an image representation. For example, the period during which the pixels of the image sensor are exposed to light, thereby capturing the essential visual information. The exposure cycle can include the entire sequence or a portion thereof, from the initiation of light exposure to the final readout of the captured image data. The exposure cycle can be optimized to ensure that high-resolution images are captured with minimal motion blur and maximum detail, even at rapid production speeds, e.g., >2000 parts per minute. The image sensor may have relatively high resolution, for example, 20 micrometers / pixel or higher resolution. In some implementations, an image sensor can be a camera, a machine-vision device, or a scanner (e.g., line scanner) configured to capture image data of at least a portion of the printed mark 103 on product item 104.

[0037] The imaging system 102 can include and / or be in data communication with one or more light source(s) 116. Light source(s) 116 can be, for example, strobe light(s) operable to illuminate the printed mark of a product item during capture of the image data by the image sensor 112. For example, the light source(s) 116 can be an integrated, high-brightness illumination device with the image sensor of the imaging system 102.

[0038] In some implementations, imaging system 102 can include multiple image sensors 112 arranged in an array with respect to each other image sensor and within the production environment 106 to have overlapping fields of view of the product item 104 moving through an imaging region of the product transport 110. For example, eight image sensors can be arranged in a line over a wide web containing multiple printed labels and triggered simultaneously. Each image sensor captures a part of the total web and can be processed in parallel using multiple processors. The image sensors 112 can be arranged to capture different views of the product item 104 such that at least one image sensor 112 of the multiple image sensors captures the printed mark 103 in one or more images as the product item 104 is transported by the product transport 110.

[0039] In some implementations, image sensor 112 is coupled to one or more optical lenses where the optical lenses can be configured to focus an imaging region of the printed mark on the product item onto the imaging plane of the image sensor 112 when the printed mark is within the field of view of the image sensor. For example, an optical lens can be a fish eye lens to increase a field of view of the image sensor 112 of the printed mark on the product item 104. In another example, an optical lens can be a magnifying lens to adjust a magnification of the printed mark 103 on the product item 104 captured by the image sensor 112.

[0040] Imaging controller device 114 includes a processor 118 in data communication with a computer readable memory 120. Computer readable memory 120 can be configured to store computer readable instructions executable by processor 118. The computer readable memory 120 may include volatile memory to store computer instructions, such as Random Access Memory (RAM), and may also include persistent memory such as, for example, a hard disk, hard-drive or any other stable storage space, e.g., a secure digital (“SD”) card, a flash drive, etc., in various embodiments.

[0041] Processor(s) 118 includes a graphics processing unit (GPU) 122 and a central processing unit (CPU) 124. For example, processor 118 can be a Jetson Xavier™, ARM process-based system-on-module produced by Nvidia Corporation based in Santa Clara, California. Each of the GPU 122 and CPU 124 can be multi-core processing devices and / or can use multi -threading to perform parallel processing operations described herein. For example, the processor 118 can include 12 cores, 16 cores, 64 cores, or any other suitable number of cores. Processor 118 can include a different number of CPU cores and GPU cores. A GPU can include one or more tensor cores. For example, processor 118 can include a 512 core GPU (including 64 tensor cores) and an 8 core CPU. Further, in some implementations, the operations described herein can be implemented using hardware components, firmware components, software components, or any combination thereof.

[0042] In some implementations, multi-core processor(s) 118 can be configured to dynamically allocate processing resources (e.g., including multiple cores and divided between CPU and GPU processing) in response to one or more of various demands of the imaging system. For example, the processor can be configured to dynamically allocate cores for determining the electronic signature for the mark using the set of metrics and based on the computational requirements and priorities for the cores. In another example, the processor can be configured to dynamically allocate one or more cores for each of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation. In another example, the processor can be configured to dynamically allocate one or more cores and priority levels for determining the electronic signature for the mark based on the set of metrics synchronized with a speed of the product item on a production line. In another example, the processor can be configured to allocate one or more cores to a particular process for determining the electronic signature based on computational resources required for the particular process.

[0043] In some implementations, multi-core processor(s) 118 can be configured to assign processing resources (e.g., including multiple cores and divided between CPU and GPU processing) in response to one or more various demands of the image processing system as indicated by performance metrics of the different sections of processing. For example, the processor can be configured to assign higher priorities to operations that are critical to determining the electronic signature.

[0044] In some implementations, the processor 118 is configured to assign a fixed number of cores to one or more set of feature characteristics to measure and analyze in parallel. For instance, one set of cores can be dedicated to analyzing geometric features such as angles, lengths, and curvatures, while another set focuses on colorimetric features like hue, saturation, and brightness. In another example, the processor can be configured to assign one or more cores to process in parallel one or more of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation. In another example, the processor can be configured to assign priorities to the image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation based on real-time requirements, and dynamically allocates the cores based on the assigned priorities.

[0045] In some implementations, the processor 118 is configured to assign processing resources to operations including, but which are not limited to, image processing, feature extraction, metric calculation, artifact analysis, electronic signature generation and data transmission or their respective sub-operations. For example, image processing may involve, but is not limited to, noise reduction, contrast enhancement, and edge detection to ensure that the image data is of high quality before further analysis. Some of these preprocessing steps may be needed for some or all parts of the captured image to accurately capture the microvariations in the printed mark. In some implementations, the processor can be configured to measure various characteristics of the mark at multiple locations, with each core in the multicore processor assigned to a specific set of characteristics to measure. The processor can be configured to generate a unique electronic signature for its assigned characteristics based on the analyzed metrics, with each core's signature combined to form a comprehensive signature for the genuine mark. For example, one core may measure edge linearity and contrast variations, while another measures feature shape and aspect ratios, and a third measures color and pigmentation.

[0046] Processor(s) 118 can implement high-speed processing algorithms to process the captured image data and determine electronic signatures for the printed marks of product items at high-speeds (e.g., >2000 ppm). For example, the system's high-speed processing capabilities can be achieved through the use of multi-core or multi-threading techniques found in commercial packages, e.g., OpenCV (Open Source Computer Vision Library), Compute Unified Device Architecture (CUD A®) by Nvidia® located in Santa Clara, CA, Integrated Performance Primitives by Intel® located in Santa Clara, CA. Then the electronic signatures are directly transmitted from the imaging system for storage in the cloud-based database 132. Database can be a remote data store, e.g., a database in a warehouse management systems (WMS) and enterprise resource planning (ERP) system, and / or manufacturing execution system (MES) system.

[0047] In some implementations, the processing algorithms implemented by the processor 118 begin with the smart camera capturing a high-resolution image of the symbology. The image is then sent to the processor, which identifies the micro-variations present in the symbology. This identification process involves analyzing the image data to detect small differences in the symbology's patterns. The processor then generates an electronic signature based on these identified micro-variations. This generation process involves encoding the micro-variations into a unique digital representation that serves as the symbology's electronic signature. The electronic signature is then sent to the cloud-based storage system for storage. The data flow within the system begins with the smart camera capturing high-resolution images of symbologies. These images are then sent to the processor, where they are processed to identify micro-variations. The processor generates an electronic signature based on these micro-variations and sends this signature to the cloud-based storage system for storage.

[0048] In various implementations, the processes (e.g., parallel processes) described herein can be divided between the CPU and GPU, where the processes can be performed by multicores and / or multi-thread techniques by the CPU and / or the GPU.

[0049] In some implementations, parallel processing the image data includes dividing, by the multi-core processor (e.g., by the CPU), the image data received from the image sensor, and providing, by the multi-core processor (e.g., by the CPU) and to at least two core processors of the multi-core GPU, different portions of the spatially divided image data to generate the set of metrics. At times, determining the electronic signature for the mark based on the set of metrics includes determining, by the multi-core CPU and from the output set of metrics by the multi-core GPU, the electronic signature, where sending the electronic signature directly to the remote data store includes sending, by the multi-core CPU, the electronic signature directly to the remote data store.

[0050] In some implementations, the CPU determines the electronic signature during the parallel processing of the image data received from the image sensor by the GPU. Determining the electronic signature for the mark based on the set of metrics can include determining, by the multi-core GPU, the electronic signature. Sending the electronic signature directly to the remote data store can include sending, by the multi-core CPU, the electronic signature directly to the remote data store.

[0051] In some implementations, the electronic signature is transferred to a cloud storage via a network 134 using HTTP / 2 protocol, e g., using gRPC protocol, a REST API, or another high-performance framework, which facilitates fast and reliable communication between the imaging system and an application 138 operating on a cloud-based server 139. For example, gRPC protocol implements HTTP / 2 protocol for connections, enabling dual asynchronous messaging between the imaging system and the cloud as well as security with TSL / SSL. Asynchronous messaging allows building messages with multiple electronic signatures, transmission of messages, the storage of messages into a database, and the acknowledge of messages all to be done concurrently. Using normal messaging, the above tasks would be done sequentially, and the next message would have to wait until the prior one completed all the same steps. Examples of communication networks 134 include a local area network (LAN) and a wide area network (WAN), e.g., the Internet. Further details related to determining the electronic signature are described with reference to FIG. 6.

[0052] Imaging controller device 114 includes an image data acquisition controller 126 operable to trigger capture of image data by the image sensor 112 (and optionally of the light source 116) of the printed mark 103 on product item 104. At times, the capture of image data is performed during transport of the product item 104 on the product transport 110, e.g., as the product item is being moved on a conveyer belt in a production environment. Controller 126 can include timing circuitry operable to time a strobe of light source 116 and capture of image data by image sensor 112 with a movement of the product item 104 on product transport 110. For example, controller 126 can receive encoder data from encoder 128 and / or proximity sensor data from sensor(s) 129 to determine a speed at which the product item 104 is moving on product transport 110 and / or a location of the product item with respect to a field of view of the image sensor 112, and provide control signals that trigger a strobe by light sources 116, capture image data by image sensor, and process the image data by the processor 118 to determine an electronic signature based on a set of metrics of the mark 103.

[0053] As will described in further detail with reference to FIGS. 3 and 6, the processor 118 receives the captured image data and measures, using multiple parallel processes, characteristics of micro-variations and / or anomalies in the printed mark to determine an electronic signature for the printed mark 103 and directly transmits, using interface 130, the electronic signature (e.g., “hash” or “product hash”) to a cloud-based repository of electronic signatures (e.g., database 132) via network 134.

[0054] Further details of the imaging system 102 including, for example, calibration and operation of the edge-based imaging system to capture and analyze image data to determine electronic signatures, can be found in U.S. Patent US11811983B2, granted on November 7, 2023, and entitled “Methods and system for imaging of moving printed materials by an optical device having plurality of cameras arranged in an array,” and U.S. Patent US9519942B2, granted on December 13, 2016, and entitled “Methods and a computing device for determining whether a mark is genuine,” which are each hereby incorporated by reference in their entirety.

[0055] A user of a user device 136 can interact with an application 138 hosted by a cloudbased server 139 over the network 134 and through a user interface 140 of the user device 136. Application 138 can be a web-based, browser-accessible application including a graphical user interface 140 to provide a method of interacting, configuring, and visualizing all of the system level operations of the imaging system 102. In other words, the application 138 can be used to configure the system and generate runtime visualization data using this edge-hosted, web-based graphical user interface (GUI). For example, the application can be designed to be modular in order to support a multitude of varied use cases, e.g., pharma packaging lines, food / beverage packaging lines, and the like. The user interface 140 can follow an industrial standard layout with user group centric data visualization 142 including widgets and access control. In some implementations, the application 138 can be a locally- hosted application, e.g., a locally-hosted copy on the imaging system 102, including a graphical user interface 140 to provide a method of interactive, configuring, and visualizing all of the system level operations of the imaging system 102. Further details of the graphical user interface 140 are described with reference to FIG. 5.

[0056] FIG. 2 is a block diagram of an example of a production line 200. As described with reference to FIG. 1, the imaging system 202 includes a sensor 204 (e.g., high-speed camera). The sensor 204, e g., a high-speed camera, can include an optical lens 206 to align the product item, e.g., product item 208, with an imaging plane of the camera. The imaging system 202 can be equipped with strobe lights 209 programmed to trigger during image acquisition. A controller 210 is operable to capture, using the camera and the strobe lights, high-resolution image data of symbologies located on product items (e.g., objects to inspect) in real-time and includes and / or is in data communication with a multi-core processor 218 to perform image processing of the captured images in parallel processes using multiple cores and / or multi-threading processes.

[0057] As depicted in FIG. 2, the camera, mounted on the support frame 212, captures high- resolution images of symbologies as they pass on the product transport 214 (e.g., conveyer belt) that is operated by a motor 216 coupled to the product transport 214. Though depicted in FIG. 2 as located adjacent to (e.g., above) the product transport, the image sensor can be positioned adjacent to the product transport to have a side-view of the product items as they are moved along the product transport or at a different angle with respect to the product transport. The image data is provided to the multi-core processor of the imaging system 202, which uses the multiple cores or multi-threading techniques to process the images. The processor 218 generates an electronic signature 224 based on the micro-variations present in the symbologies and directly transmits, from the imaging system the electronic signature 224 to a cloud-based data storage system 220, over the network 222. A user interface presented in a user device (e.g., a computer, smart phone, tablet, etc.) allows users to manage and monitor the system operations.

[0058] FIG. 3 is a block diagram of an example of a workflow 300 for an edge-based electronic signature determination system, e.g., system 100 of FIG. 1, for genuine product marks using parallel processing. At (1) the product transport transports the product item, e.g., product item 104, into a field of view of the image sensor. A product item can include, for example, a bottle, a vial, a box, a blister, etc. A signal (e.g., discrete signal) is provided by the image data acquisition controller to an image sensor to capture an image. For example, the signal can be provided by a sensor, encoder, or another component of the machine / product handling system to trigger image acquisition by the image sensor. At (2) the image sensor, e.g., image sensor 112, captures high-resolution image data of the symbology on the product item. At (3) the image data is provided to the processor, e.g., a processor 118 of the imaging system 102, which processes, using multiple parallel processes including running parallel processes on a multi-core GPU, the image data and identifies the micro-variations present in the symbology. The processor uses one or more high-speed processing algorithms to extract the micro-variations and / or artifacts from the image data of the symbology to generate the electronic signature based on a set of metrics using the identified micro-variations. At (4) the processor transmits the electronic signature to the cloud-based storage system, e.g., database 132, for storage, e.g., using the gRPC protocol. At (5) the electronic signature is stored in the cloud-based database for future use, e.g., for tracking the product item. At (6) a user of a user device can access a web-based API including graphical user interface to remotely view a runtime processing and status of the imaging system, e.g., user device 136 including a user interface 140. This can include, for example, real-time images, counters, alarms, as well as user set up and configuration screens for the imaging system. In other words, a user can interact with the web-based GUI to cause the multi-core processor (e.g., processor 118) to receive configuration data from a remote computing device (e.g., a mobile device and / or desktop computer communicatively coupled with the ERP, WMS and / or MES system) and send runtime visualization data to the remote computing device via the network 134.

[0059] Additionally, the user can provide, through the GUI, an image of the symbology of a product item (e.g., by capturing an image of the symbology using a camera of the user device) and interact with the GUI to validate that the symbology is genuine and / or discover product information for the product item associated with the symbology. Further details of the process of the imaging system are described with reference to FIGS. 6 and 7.

[0060] FIG. 4 is a block diagram of an example of an edge-based system architecture 400 for imaging system, e.g., imaging system 102, to perform the edge-based electronic signature determination. FIG. 4 depicts a visualization of the edge-based solution as a unified stack. In other words, the integrated layers of the unified stack can be implemented without using separate physical, datalink, and network communication layers of a network communications protocol stack. The edge-based solution includes a Jetson Xavier™, ARM processor-based system-on-module paired with a high-resolution image sensor, optical lens, and integrated high brightness illumination device, with interfaces to a vision interface kit (VIK) having an input / output and rejection tracking module. System configuration and runtime visuals are performed using an edge-hosted, web-based graphical user interface (GUI). The edge-based solution 402 includes internal components 404, e.g., the image sensor 406 and user data storage 408 (e.g., local solid-state drive for storing image data). The edge-based solution 402 further includes a system on module 410 (e.g., the multi-core processor) including a USB 3.0 interface 412 configured to facilitate data communication between the image sensor 406 and the components of the system on module 410, a universal asynchronous receiver / transmitter (UART) 414, the SD card 416, ethernet connectivity 418, e.g., GigE Ethernet, for transmitting electronic signature to the cloud-based database, a general purpose input / output (GPIO) pulse width modulator (PWM) driver 420, a GPIO Image sensor 422, and a UART RS-232424 to communicate with a vision interface kit (VIK) Module 426. The edge solution 402 further includes add-on components 434, for example, a high brightness illumination device 428, and the VIK module 426. The edge solution 402 is configured to communicate with external systems 436, for example, a programmable logic controller (PLC) 430 and a cloud-based client (user) device 432. As depicted in FIG. 4, the edge solution 402 includes the integrated components including the internal components 404, system on module 410, and add-on components 434 in a self-contained unit which can be deployed into the production environment as detailed above.

[0061] In some implementations, as described with reference to FIG. 1, a web-based, browser-accessible graphical user interface, e.g., visualization 142, provides the primary method of interacting, configuring, and visualizing all the system-level operations. The highly responsive interface follows an industrial standard layout with user group centric data visual widgets and access control. The GUI can present configurable widgets that are reusable for multitude of data visual needs and can be grouped to create screens. The GUI can provide data trending over a period of time with options to plot failed inspection images, alarms or critical system events. The GUI can provide a product and recipe management console with state-of-the-art automatic product setup function that takes a hands-off approach in configuring, an otherwise complex product setup process.

[0062] In some implementations, the user interface is a web-based GUI allowing users to interact with the application remotely through a network, e.g., network 134. Client-server interaction can provide seamless user experiences across various devices and browsers. Benefits can include, for example, remote monitoring from various devices to provide realtime visibility into processes without the need to be physically located next to the unit. It can allow users to configure the device, receive real-time alerts, remotely troubleshoot, and view real-time images, counters, and production progress.

[0063] FIG. 5 is an example of an embedded application architecture 500 of program code implemented as firmware on an edge device, e.g., the standalone system, for real-time visualization and communication services for external data integration. The embedded application architecture in, e.g., application 138, includes five application layers which are supported by a GUI for visualization and communication services for external data integration. A workflow layer 502 is an embedded version of a workflow engine, e.g., an Items Process Stream engine (IPS) that provides user-configurable functional blocks. As depicted in FIG. 5, the workflow layer can include blocks for logic operations (Logic Ops) 504, methods to communicate with external open platform communications (OPC) clients (OPC Ops) 506, machine vision operations (Vision Ops) 508, database operations (DB Ops) 510 for serialized product tracking (SPT), means to communicate with external warehouse management systems (WMS) and enterprise resource planning (ERP) systems for master data, e.g., Ext. Comm 512, printer drivers 514 and vision interface tracking using a vision interface kit (VIK), e.g., I / O Ops 516. The workflow layer 502 also provides means to create recipes / classes or templates, that a group of products can use to run on the packaging line.

[0064] A vision processing layer 520 provides access to TIPS vision tools 522, AI / ML tools 524, and / or other 3rdparty vision tools 526, e.g., TensorFlow, PyTorch, OpenCV, etc., which are user configurable and are linked to the workflow engine IPS of the workflow layer 502. A device management layer 530 provides means to configure internal or external line devices, e.g., camera 532, printers 534, eS-VIK 536, that are linked to the workflow engine. A gateway layer 540 provides messaging access for client-side software stack, such as User Interface 590 to the back-end services, e.g., via a gRPC Server 542, OPC Server 544, or a Proxy Server 546. A database layer 550 provides the means for in-memory and persistent data accessibility for runtime visuals and short to mid-term storage. The database layer 550 includes, for example, datasets for alarms (Alarm DB) 552, lot history (History DB) 554, audit trail 556, serialized product tracking data (SPT DB) 558 and run time events (Log DB) 560.

[0065] A communication services layer 570 (e.g., “Com Services”) provides ability to send or receive master data from factory ERP, WMS, or MES systems using any of the supported data connectors including, for example, RESTful API 572, Open Device Protocol (ODP) 574, SAP iDOCs 576, open database connectivity (ODBC) 578, OPC 580, simple mail transfer protocol (SMTP) or flat fde format, e.g., FTP 582 and text fde format 584. In addition, gRPC service 586 is used to stream all production data from edge device to the cloud for long term storage and to perform additional cross lot analytics.

[0066] A user interface layer 590 provides graphical user interface for a user to interact with the IPS, e.g., the workflow layer 502 via the gateway layer 540. The user interface layer 590 includes various management tools through which the user can interact with the IPS and, optionally, make changes to the operations of the IPS. The user interface layer 590 can include, for example, a layout manager 592, widget manager 594, user manager 596, alarm / event manager 598, trend viewer 591, quick help widget 593, recipe manager 595, and a localization widget 597.

[0067] An item process stream (IPS) can provide a control mechanism for virtual items which are, in effect, digital twins of physical items (e.g., product items), using one or more of serialized product tracking (SPT) processes organized into one or more streams to track items (e.g., product items) in the respective one or more streams. Further details related to the item process stream (IPS) can be found in U.S. Patent US8190279B2, granted on May 29, 2012, and entitled “Production Line Management System,” which is hereby incorporated by reference in its entirety.

[0068] As described above, processor(s) 118 of the edge device, e.g., imaging system 102, can allocate / assign processing resources to cores to run parallel processes in support of serialization, brand protection, and vision goals of items in a process stream. Incorporating the parallel processing techniques to the IPS can yield an embedded item process stream that is operable on an edge-device and having enhanced serialization, brand protection, and vision capabilities. FIG. 8 is a block diagram of an example of an embedded item process stream 800 including items (e.g., items 1-6) in the process stream (e.g., process steam 1) operating on an edge device.

[0069] Embedded Item Process Stream (elPS) 800 can enhance the operations of IPS control 802 as a fully-integrated processes that can be run on an edge-device. The IPS control 802 is configured to operate as an intermediary between devices management 804, in-memory data storage 806 and a gateway 808 between the edge device and cloud-based instantiation 810 of the IPS and the edge device and a graphical user interface, e.g., GUI 812, via a UI engine 814. The IPS control 802 allocates / assigns processing resources to run multiple parallel processes as items 816 in one or more processing streams 818. Additionally, IPS control 802 can include a processor affinity for multi-core processor(s), e.g., processor(s) 118, configured to allocate processing resources for processor intensive tasks, e.g., image processing, by associating dedicated threads within elPS to a specific processor set via their affinity settings. The elPS 800 includes in-memory data storage 806 which can be utilized for real-time, persistent data storage for short- to mid-term storage needs, for example, to recover from external communication disruptions. This can also serve as data-streaming gateway 808 to queue data before transferring to a cloud-based storage on a cloud-based instantiation 810 to increase parallel processing throughput.

[0070] As depicted in FIG. 8, Item 1, Item 2, Item 3, Item 4, Item 5, and Item 6 are examples of tasks in the stream 1, which are divided from the various processes managed by the IPS control 802 using the parallel processing techniques. The different processes managed by the IPS control 802 can be separated into items, e.g., items 1-6, in one or more processing streams 818, e.g., stream 1, to prioritize and allocate parallel processing of the tasks. For example, the elPS 800 can receive and allocate processing resources for Vision Processes 820, e.g., image acquisition and processing, used to configure vision verification tools within a stream of one or more processing streams 818. In another example, elPS 800 can receive and allocate processing resources for Communication (Com) processes 822 to perform data or image transfer to an external cloud-based data store as part of a workflow stream.

[0071] In some implementations, the elPS can leverage core processes 824, e.g., encoder counts, to trigger inputs from devices 826, e.g., vision interface kit (VIK), camera, in data communication with the IPS control 802 via the devices management 804 and from Vision processes 820. For example, the elPS can leverage core processes 824 to trigger generation of visualizations (e.g., histogram, Delta Machine Vision™ (DMV) System tools etc., in response to core processes 824. In another example, the elPS can leverage core processes 824 to automatically process a physical mark on a product and configure the product in the production environment with all required parameters, e.g., as trigger delay, field of view of the camera, brightness of the image etc. In another example, elPS can be used to configure / calibrate the imaging system without direct operator intervention, which can reduce line downtime to pre-configure a product.

[0072] In some implementations, the elPS 800 can integrate serialized product tracking (SPT) processes 828 organized into one or more processing streams 818 to track items (e.g., product items) and to store the exchange of futures for physical (eFP) transactions in a database.

[0073] In some implementations, the elPS 800 can integrate driver processes 830 into one or more processing streams 818. For example, to allocate processing of driver operations for a pulse width modulator (PWM) driver, printer drivers, and the like. FIG. 6 is a flow diagram of an example process 600 for an integrated serialization, brand protection, and vision system, e.g., an example process of the operating environment 100 of FIG. 1. For convenience, the process 600 will be described as being performed by a system of one or more computers and programmed appropriately in accordance with this specification. For example, a system including a multi-core processor, e.g., processor 118 of the imaging system 102 of FIG. 1, appropriately programmed, can perform the process 60. As described in further detail with reference to FIG. 8, the processor can include an elPS 800 including an IPS control 802, where the IPS control 802 is configured to allocate resources for multiple processes into one or more processing streams 818 including multiple items, e.g., tasks, which can be processed in parallel, e.g., using multi-core processor including a GPU and a CPU.

[0074] A mark applying device applies (602) a genuine mark (e.g., a printed mark) to a product item. The mark-applying device can be, for example, a printer, a print and apply machine, or another suitable device. The mark-applying device can be located in the production environment, e.g., production environment 106, where the genuine mark is applied to a product item as the product item moves along a product transport, e.g., product item 104 on product transform 110. The genuine mark can be, for example, a brand logo, barcode, or a decorative element and includes unintentionally-produced artifacts that are unique and not reproducible.

[0075] An image sensor captures (604) image data of the genuine mark on the product item. For example, the image sensor captures the image data of the genuine mark as the product item is moved by a product transport, e.g., product transport 110. The image sensor can operate under controlled circumstances, e.g., under known conditions including the speed of the product transporter, a controlled light exposure (e.g., strobe light), etc. The image sensor can be, for example, a camera, a machine-vision device, scanner, or an image sensor component of a self-contained high-speed capturing technology, e.g., the imaging system as described herein. As described with reference to FIG. 2, a controller can synchronize a timing of the image capture with a high-brightness light source, e.g., strobe light, in response to a trigger from the product transport, e.g., a motor operating a conveyer belt.

[0076] The image sensor provides (606) the captured image data to a processor, e.g., processor 118. The processor can be, for example, a desktop or laptop computer, a rack- mounted server, a tablet, a mobile phone, or a multi-core processor component within a self- contained high-speed capturing technology, e.g., the imaging system 102 as described herein.

[0077] The processor receives the captured image data and measures, using parallel processing techniques, identifies various characteristics of the mark at multiple locations of the mark and determines (608) a set of metrics. For example, the IPS control 802 may allocate parallel processing resources to perform to identify characteristics of the printed mark and determine the set of metrics. The characteristics can include, for example, feature shape, feature aspect ratios, feature locations, feature size, feature contrast, edge linearity, region discontinuities, extraneous marks, printing defects, color, pigmentation, and contrast variations.

[0078] The processor analyzes the set of metrics and determines (610) unique artifacts of the mark from the set of metrics and generates (612) a unique electronic signature for the genuine mark based on the set of metrics. The electronic signature can be based on both the artifacts of the mark and, if applicable, the encoded data carried by the mark (e.g., barcode, QR code, etc.). In some implementations, analyzing the set of metrics includes performing the analyzing in parallel using the multi-core graphics processing unit as part of the parallel processing of the image data.

[0079] In some implementations, determining unique artifacts of the mark for the electronic signature includes ranking, by the processor, the set of metrics associated with each characteristic by magnitude, such that only those metrics that reach a predetermined threshold are used as part of the electronic signature. In some implementations, determining unique artifacts of the mark for the electronic signature includes weighing, by the processor, the measured characteristics based on factors, for example, image resolution, idiosyncrasies of the mark-applying device, or error-correcting information included in the mark.

[0080] In some implementations, the processor provides the generated electronic signature for the genuine mark to a cloud-based storage over the network, e.g., database 132 via network 134. The stored electronic signatures can be accessible by one or more other devices, e.g., user devices 136 during an electronic signature validation process for a printed mark on a product item, e.g., as described with reference to FIG. 7.

[0081] FIG. 7 is a flow diagram of an example process 700 for an integrated serialization, brand protection, and vision system. For convenience, the process 700 will be described as being performed by a system of one or more computers, located in one or more locations, and programmed appropriately in accordance with this specification. For example, a cloud-based server running an application, e.g., application 138, appropriately programmed, can perform the process 700. Alternatively, or additionally, some or all of the processes 700 can be performed by processor 118 of imaging system 102, and / or using a user interface 140 operating on a user device 136. In some implementations, process 700 can be performed by an edge device running a local version of the elPS 800 described with reference to FIG. 8.

[0082] A system receives (702) a captured image of a candidate mark and a request to determine whether the candidate mark is genuine. For example, a user can capture a candidate mark of a product item using a camera of a user device, e.g., user device 136, where a local version of the application 138 generates an electronic signature of the candidate mark and submits the candidate mark through the GUI 140 of the application 138 to the system, e.g., cloud-based system. The user device, which can include a multi-core processor, e.g., as described with reference to processor 118, can determine (704) an electronic signature for the candidate mark in a same manner as described above with reference to FIG. 6 to determining the electronic signature of the genuine mark.

[0083] The system compares (706) the electronic signature of the candidate mark with the electronic signature of the genuine mark to determine whether the candidate mark is genuine. In some implementations, the system compares the electronic signature of the candidate mark with a repository of electronic signatures stored in a database, e.g., database 132.

[0084] If the system determines (708) that the candidate mark is genuine, the system transmits an indication that the candidate mark is genuine. If the system determines (710) that the candidate mark is not genuine, the system can transmit an indication (e.g., an alert) that the candidate mark is not genuine. For example, the system can provide, through the GUI presented on the user device, an alert stating whether or not the candidate mark is genuine.

[0085] In some implementations, the processes described with respect to FIG. 7 can be performed by processor 118 of the imaging system 102 running a local version of the elPS 800, where a same system is configured to determine the electronic signatures from the genuine marks as well as validate candidate marks captured by a user device using the GUI 140 of application 138. The subject matter and the actions and operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter and the actions and operations described in this specification can be implemented as or in one or more computer programs, e.g., one or more modules of computer program instructions, encoded on a computer program carrier, for execution by, or to control the operation of, data processing apparatus. The carrier can be a tangible non-transitory computer storage medium. Alternatively, or in addition, the carrier can be an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be or be part of a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. A computer storage medium is not a propagated signal.

[0086] The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. Data processing apparatus can include special-purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application-specific integrated circuit), or a GPU (graphics processing unit). The apparatus can also include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0087] A computer program can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program, e.g., as an app, or as a module, component, engine, subroutine, or other unit suitable for executing in a computing environment, which environment may include one or more computers interconnected by a data communication network in one or more locations.

[0088] A computer program may, but need not, correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single fde dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code.

[0089] The processes and logic flows described in this specification can be performed by one or more computers executing one or more computer programs to perform operations by operating on input data and generating output. The processes and logic flows can also be performed by special-purpose logic circuitry, e.g., an FPGA, an ASIC, or a GPU, or by a combination of special-purpose logic circuitry and one or more programmed computers.

[0090] Computers suitable for the execution of a computer program can be based on general or special-purpose microprocessors or both, or any other kind of central processing unit and / or graphics processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random-access memory or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry.

[0091] Generally, a computer will also include, or be operatively coupled to, one or more mass storage devices, and be configured to receive data from or transfer data to the mass storage devices. The mass storage devices can be, for example, magnetic, magneto-optical, or optical disks, or solid-state drives. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0092] To provide for interaction with a user, the subject matter described in this specification can be implemented on one or more computers having, or configured to communicate with, a display device, e.g., a LCD (liquid crystal display) monitor, or a virtual- reality (VR) or augmented-reality (AR) display, for displaying information to the user, and an input device by which the user can provide input to the computer, e.g., a keyboard and a pointing device, e.g., a mouse, a trackball or touchpad. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback and responses provided to the user can be any form of sensory feedback, e.g., visual, auditory, speech or tactile; and input from the user can be received in any form, including acoustic, speech, or tactile input, including touch motion or gestures, or kinetic motion or gestures or orientation motion or gestures. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser, or by interacting with an app running on a user device, e.g., a smartphone or electronic tablet. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0093] This specification uses the term “configured to” in connection with systems, apparatus, and computer program components. That a system of one or more computers is configured to perform particular operations or actions means that the system has installed on its software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. That one or more computer programs is configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions. That special-purpose logic circuitry is configured to perform particular operations or actions means that the circuitry has electronic logic that performs the operations or actions.

[0094] In addition to the embodiments of the attached claims and the embodiments described above, the following numbered embodiments are also innovative.

[0095] Embodiment 1 is a system, the system comprising an image sensor, a multi-core processing device, and a non-transitoiy computer-readable medium tangibly encoding instructions of a computer program configured to cause the multi-core processing device to perform operations comprising parallel processing, by the multi-core processing device, of image data received from the image sensor, the image data representing a mark printed on a product, wherein the mark provides product information. The parallel processing comprises measuring, by the multi-core processing device, differential characteristics of the mark at multiple locations to generate a set of metrics, and determining, by the multi-core processing device, an electronic signature for the mark based on the set of metrics, and sending, by the multi -core processing device, the electronic signature directly to a remote data store that is communicatively coupled with the multi-core processing device via an inter-network.

[0096] Embodiment 2 is a system of embodiment 1, wherein the multi-core processing device comprises a computing board comprising a multi-core central processing unit and a multi-core graphics processing unit.

[0097] Embodiment 3 is a system of any one of embodiments 1 or 2, wherein the parallel processing of the image data received from the image sensor further comprises dividing, by the multi-core central processing unit, the image data received from the image sensor, and providing, by the multi-core central processing unit and to at least two core processors of the multi-core graphics processing unit, different portions of the divided image data to generate the set of metrics.

[0098] Embodiment 4 is a system of any one of embodiments 1 to 3, wherein determining the electronic signature for the mark based on the set of metrics comprises determining, by the multi-core central processing unit and from the set of metrics generated by the multi-core graphics processing unit, the electronic signature, and wherein sending the electronic signature directly to the remote data store comprises sending, by the multi-core central processing unit, the electronic signature directly to the remote data store.

[0099] Embodiment 5 is a system of any one of embodiments 1 to 5, wherein the multi-core central processing unit determines the electronic signature during the parallel processing of the image data received from the image sensor by the multi-core graphic processing unit.

[0100] Embodiment 6 is a system of any one of embodiments 1 to 3, wherein determining the electronic signature for the mark based on the set of metrics comprises determining, by the multi-core graphics processing unit, the electronic signature, and wherein sending the electronic signature directly to the remote data store comprises sending, by the multi-core central processing unit, the electronic signature directly to the remote data store.

[0101] Embodiment 7 is a system of any one of embodiments 1 to 6, comprising a light source positioned to illuminate a field of view of the image sensor, and a product transport configured to move the product thru the field of view of the image sensor, wherein the image sensor is positioned adjacent to the product transport.

[0102] Embodiment 8 is a system of any one of embodiments 1 to 7, wherein the instructions of the computer program comprise web-based graphical user interface instructions configured to cause the multi-core processing device to receive configuration data from a remote computing device and send runtime visualization data to the remote computing device via the inter-network.

[0103] Embodiment 9 is a system of any one of embodiments 4 or 6, wherein the electronic signature is sent to the remote data store using a REST API or a gRPC service.

[0104] Embodiment 10 is a system of any one of embodiments 1 to 9, wherein the instructions of the computer program implement communications via the inter-network without using separate physical, datalink, and network communication layers of a network communications protocol stack.

[0105] Embodiment 11 is a system of any one of embodiments 1 to 10, wherein the set of metrics comprise variations in feature shape, feature aspect ratios, feature locations, feature size, feature contrast, edge linearity, region discontinuities, extraneous marks, printing defects, color, pigmentation, and contrast.

[0106] Embodiment 12 is a system of embodiment 11, wherein the determining comprises ranking the set of metrics based on respective magnitudes of the variations corresponding to the metrics and determining the electronic signature from only metrics of the set of metrics meeting a threshold magnitude of the variation for the metric.

[0107] Embodiment 13 is a system of any one of embodiments 1 to 12, wherein the determining comprises analyzing the set of metrics to identify artifacts usable to distinguish the mark from counterfeit versions of the mark and generating the electronic signature for the mark using the identified artifacts.

[0108] Embodiment 14 is a system of embodiment 11, wherein the determining comprises ranking the set of metrics based on one or more of (A) image resolution, (B) characteristics of a mark-applying device that printed the mark on the product, or (C) error correction information included in the mark, and determining, from the ranked set of metrics, the electronic signature.

[0109] Embodiment 15 is a system of embodiment 13, wherein the generating comprises using data encoded in the mark to generate the electronic signature for the mark.

[0110] Embodiment 16 is a system of embodiment 13, wherein the analyzing comprises performing the analyzing in parallel using a multi-core graphics processing unit as part of the parallel processing of the image data. Embodiment 17 is a system of any one of embodiments 1 to 16, wherein the image sensor, the multi-core processing device, and the non-transitory computer-readable medium are integrated into a housing of a single camera unit.

[0111] Embodiment 18 is a system of any one of embodiments 1 to 17, wherein the non- transitory computer-readable medium further comprises tangibly encoding instructions of the computer program configured to cause the multi-core processing device to perform operations comprising capturing, by the image sensor, the image data representing the mark printed on the product item.

[0112] Embodiment 19 is a system of any one of embodiments 1 to 18, further comprising providing, for presentation in a graphical user interface on a user device, an electronic signature repository including electronic signatures for product items, receiving, through the graphical user interface on the user device, a request for product information including an electronic signature corresponding to a product item, and providing, through the graphical user interface on the user device, the product information for the product item corresponding to the electronic signature.

[0113] Embodiment 20 is a system of any one of embodiments 1 to 19, further comprising providing, for presentation in a graphical user interface on a user device, a repository of electronic signatures for product items, receiving, through the graphical user interface on the user device, a request for validation of a candidate mark corresponding to a product item, determining, for the candidate mark, a candidate electronic signature, determining, from the candidate electronic signature and the repository of electronic signatures, an electronic signature from the repository of electronic signatures having at least a threshold match to the candidate electronic signature, and in response to determining the at least threshold match, providing, through the graphical user interface on the user device, product information for the product item corresponding to the electronic signature, and in response to determining that a threshold match does not exist, providing, through the graphical user interface on the user device, an alert for the candidate mark of product item.

[0114] Embodiment 21 is a system of any one of embodiments 1 to 20, wherein the multicore processing device is further configured to dynamically allocate cores for determining the electronic signature for the mark based on the set of metrics based on computational requirements and priorities for the cores. Embodiment 22 is a system of any one of embodiments 1 to 21, wherein the multicore processing device is further configured to dynamically allocate one or more cores for each of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation.

[0115] Embodiment 23 is a system of any one of embodiments 1 to 22, wherein the multicore processing device is further configured to dynamically allocate one or more cores and priority levels for determining the electronic signature for the mark based on the set of metrics synchronized with a speed of the product item on a production line.

[0116] Embodiment 24 is a system of embodiment 23, wherein the multi-core processing device is further configured to assign higher priorities to operations that are critical to determining the electronic signature.

[0117] Embodiment 25 is a system of any one of embodiments 1 to 24, wherein the multicore processing device is further configured to allocate one or more cores to a particular process for determining the electronic signature based on computational resources required for the particular process.

[0118] Embodiment 26 is a system of any one of embodiments 1 to 25, wherein the multicore processing device is further configured to assign one or more cores to one or more set of feature characteristics to measure and analyze in parallel.

[0119] Embodiment 27 is a system of any one of embodiments 1 to 26, wherein the multicore processing device is further configured to assign one or more cores to process in parallel one or more of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation.

[0120] Embodiment 28 is a system one of embodiment 27, wherein the multi-core processing device is further configured to assign priorities to the image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation based on real-time requirements, and dynamically allocates the cores based on the assigned priorities.

[0121] Embodiment 29 is a system of embodiments 27 or 28, wherein the multi-core processing device is further configured to measure various characteristics of the mark at multiple locations, with each core in the multi-core processor assigned to a specific set of characteristics to measure. Embodiment 30 is a system of any one of embodiments 1 to 29, wherein the multicore processing device is further configured to generate a unique electronic signature for its assigned characteristics based on the analyzed metrics, with each core's signature combined to form a comprehensive signature for the printed mark.

[0122] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of what is being claimed, which is defined by the claims themselves, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claim may be directed to a subcombination or variation of a subcombination.

[0123] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this by itself should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0124] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.

[0125] What is claimed is:

Claims

CLAIMS1. A system comprising: an image sensor; a multi-core processing device; and a non-transitory computer-readable medium tangibly encoding instructions of a computer program configured to cause the multi-core processing device to perform operations comprising parallel processing, by the multi-core processing device, of image data received from the image sensor, the image data representing a mark printed on a product, wherein the mark provides product information, and the parallel processing comprises measuring, by the multi-core processing device, differential characteristics of the mark at multiple locations to generate a set of metrics; and determining, by the multi-core processing device; an electronic signature for the mark based on the set of metrics; and sending, by the multi-core processing device, the electronic signature directly to a remote data store that is communicatively coupled with the multi-core processing device via an inter-network.

2. The system of claim 1, wherein the multi-core processing device comprises a computing board comprising a multi-core central processing unit and a multi -core graphics processing unit.

3. The system of claim 2, wherein the parallel processing of the image data received from the image sensor further comprises: dividing, by the multi -core central processing unit, the image data received from the image sensor; and providing, by the multi-core central processing unit and to at least two core processors of the multi-core graphics processing unit, different portions of the divided image data to generate the set of metrics.

4. The system of claim 3, wherein determining the electronic signature for the mark based on the set of metrics comprises determining, by the multi-core central processing unit and from the set of metrics generated by the multi-core graphics processing unit, the electronic signature, and wherein sending the electronic signature directly to the remote data store comprises sending, by the multi-core central processing unit, the electronic signature directly to the remote data store.

5. The system of claim 4, wherein the multi-core central processing unit determines the electronic signature during the parallel processing of the image data received from the image sensor by the multi-core graphic processing unit.

6. The system of claim 3, wherein determining the electronic signature for the mark based on the set of metrics comprises determining, by the multi-core graphics processing unit, the electronic signature, and wherein sending the electronic signature directly to the remote data store comprises sending, by the multi-core central processing unit, the electronic signature directly to the remote data store.

7. The system of any one of claims 1 to 6, comprising: a light source positioned to illuminate a field of view of the image sensor; and a product transport configured to move the product thru the field of view of the image sensor, wherein the image sensor is positioned adjacent to the product transport.

8. The system of claim 1, wherein the instructions of the computer program comprise web-based graphical user interface instructions configured to cause the multi-core processing device to receive configuration data from a remote computing device and send runtime visualization data to the remote computing device via the inter-network.

9. The system of claim 6, wherein the electronic signature is sent to the remote data store using a REST API or a gRPC service.

10. The system of claim 8, wherein the instructions of the computer program implement communications via the inter-network without using separate physical, datalink, and network communication layers of a network communications protocol stack.

11. The system of claim 1, wherein the set of metrics comprise variations in feature shape, feature aspect ratios, feature locations, feature size, feature contrast, edge linearity, region discontinuities, extraneous marks, printing defects, color, pigmentation, and contrast.

12. The system of claim 11, wherein the determining comprises: ranking the set of metrics based on respective magnitudes of the variations corresponding to the metrics; and determining the electronic signature from only metrics of the set of metrics meeting a threshold magnitude of the variation for the metric.

13. The system of claims 11 or 12, wherein the determining comprises: analyzing the set of metrics to identify artifacts usable to distinguish the mark from counterfeit versions of the mark; and generating the electronic signature for the mark using the identified artifacts.

14. The system of claim 11, wherein the determining comprises: ranking the set of metrics based on one or more of (A) image resolution, (B) characteristics of a mark-applying device that printed the mark on the product, or (C) error correction information included in the mark; and determining, from the ranked set of metrics, the electronic signature.

15. The system of claim 13, wherein the generating comprises using data encoded in the mark to generate the electronic signature for the mark.

16. The system of claim 13, wherein the analyzing comprises performing the analyzing in parallel using a multi-core graphics processing unit as part of the parallel processing of the image data.

17. The system of claim 1, wherein the image sensor, the multi-core processing device, and the non-transitory computer-readable medium are integrated into a housing of a single camera unit.

18. The system of claim 17, wherein the non-transitory computer-readable medium further comprises tangibly encoding instructions of the computer program configured to cause the multi-core processing device to perform operations comprising: capturing, by the image sensor, the image data representing the mark printed on the product.

19. The system of any one of claims 1 to 6, 8 to 12, and 14 to 18, further comprising: providing, for presentation in a graphical user interface on a user device, an electronic signature repository including electronic signatures for product items; receiving, through the graphical user interface on the user device, a request for product information including an electronic signature corresponding to a product item; and providing, through the graphical user interface on the user device, the product information for the product item corresponding to the electronic signature.

20. The system of claim 1, further comprising: providing, for presentation in a graphical user interface on a user device, a repository of electronic signatures for product items; receiving, through the graphical user interface on the user device, a request for validation of a candidate mark corresponding to a product item; determining, for the candidate mark, a candidate electronic signature; determining, from the candidate electronic signature and the repository of electronic signatures, an electronic signature from the repository of electronic signatures having at least a threshold match to the candidate electronic signature; and in response to determining the at least threshold match, providing, through the graphical user interface on the user device, product information for the product item corresponding to the electronic signature; and in response to determining that a threshold match does not exist, providing, through the graphical user interface on the user device, an alert for the candidate mark of product item.

21. The system of claim 1, wherein the multi-core processing device is further configured to dynamically allocate cores for determining the electronic signature for the mark based on the set of metrics based on computational requirements and priorities for the cores.

22. The system of claim 21, wherein the multi-core processing device is further configured to dynamically allocate one or more cores for each of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation.

23. The system of claims 21 or 22, wherein the multi-core processing device is further configured to dynamically allocate one or more cores and priority levels for determining the electronic signature for the mark based on the set of metrics synchronized with a speed of the product on a production line.

24. The system of claim 23, wherein the multi-core processing device is further configured to assign higher priorities to operations that are critical to determining the electronic signature.

25. The system of any one of claims 21, 22 and 24, wherein the multi-core processing device is further configured to allocate one or more cores to a particular process for determining the electronic signature based on computational resources required for the particular process.

26. The system of claim 21, wherein the multi-core processing device is further configured to assign one or more cores to one or more set of feature characteristics to measure and analyze in parallel.

27. The system of any one of claims 21 and 26, wherein the multi-core processing device is further configured to assign one or more cores to process in parallel one or more of image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation.

28. The system of claim 27, wherein the multi-core processing device is further configured to assign priorities to the image capture, characteristic measurement, metric analysis, signature ranking and weighting, and signature formation based on real-time requirements, and dynamically allocates the cores based on the assigned priorities.

29. The system of claim 28, wherein the multi-core processing device is further configured to measure various characteristics of the mark at multiple locations, with each core in the multi-core processing device assigned to a specific set of characteristics to measure.

30. The system of any one of claims 21, 22, 24, 26, 28 to 29, wherein the multi-core processing device is further configured to generate a unique electronic signature for its assigned characteristics based on analyzed metrics, with each core's signature combined to form a comprehensive signature for the printed mark.

Images